Creating a Foliage Shader in Unity URP Shader Graph

NedMakesGames
23 min readOct 7, 2021

--

Video version of this tutorial.

Hi, I’m Ned, and I make games! In this tutorial, I’ll show how to create a foliage shader for Unity’s Universal Render Pipeline using the shader graph! It’s great for low poly trees, complex trees, fields of grass, and more! These types of things don’t look quite right using a default lit shader since it lacks important features, like two-sided normals, translucency and wind deformation.

Tested in Unity 2020.3 and Unity 2021.1

This tutorial covers creating a foliage shader with the URP shader graph. It is complete with diffuse, specular and translucency lighting. It supports all light types, their shadows, global illumination and fog. We’ll also look at wind deformation and two-sided rendering with normal mapping (for grass billboards).

This tutorial will not cover how to build any foliage models or textures. It will not show how to place grass blades around the game world or support character interactions through physics or otherwise. I have tutorials planned for all these topics in the near future!

I’ll assume you know the basics of Unity, URP, and the shader graph. You’ll also need to understand simple HLSL, the shader programming language, and what control keywords are (we’ll use both in custom functions). And, although I’ll give a general overview of lighting techniques, normal mapping, and tangent space, I’ll leave other tutorials to dive deep into them. Check out the bibliography at the end of this article for further reading.

Project set up

Get started by setting up your Unity project. You’ll want to go through my custom lighting in the shader graph tutorial, which provides the base for our shader. It covers adding light, shadow and global illumination support, as well as diffuse and specular lighting. Once you’ve finished, either duplicate or rename the completed shader files to “FoliageLighting.hlsl,” “FoliageLighting.subgraph,” and “TestFoliage.shadergraph.”

In FoliageLighting.hlsl, rename mentions of “custom” to “foliage.”

Then in the FoliageLighting subgraph, rename the custom function node name field to match.

Alright, set up a simple test scene to test the shader out on various types of common foliage meshes. Create a flat plane for the ground, to test shadows. Then, create a sphere game object to simulate simple, low poly trees. For a grass or bush billboard, create a quad game object and position it attached to the ground.

Now, to test for complex trees with many leaf cards — or double sided quads — craft something in Blender, or another 3D modeling program. Create three quads and rotate them so there’s one in each plane. Create edges to split each into quarters and merge vertices to weld the planes together.

Import this funny cube-like shape back into Unity and add it to your scene. Create three materials using the foliage shader, one for each model, and assign it.

To better prepare for grass and leaf cards, add alpha clipping support to the shader. In the TestFoliage graph, click “alpha clip” in the graph inspector. Add a float alpha clip threshold property and route it into the alpha threshold field of the master stack. Then, route the alpha output from the albedo texture sample into the alpha field of the master stack.

In GIMP, or another photo editor, create a simple texture with an alpha channel for the cut out texture. In Unity, set “alpha is transparency” to true.

Set up all your materials, making sure to add the alpha channel texture to the grass material.

To make sure everything’s working, test out a few lights. To test global illumination, set all the objects to static and make sure double sided global illumination is enabled on the grass and tree materials. Then, bake a light map. All good!

Double sided materials

You might notice the grass and tree meshes are invisible on one side! This is due to something called backface culling. For meshes like the sphere, it allows Unity to ignore triangles on the inside of the model. But for double-sided geometry, like foliage cards, we need to make some adjustments.

In TestFoliage, turn on the “two-sided” option in the graph inspector.

Now both sides are visible; however, the lighting is incorrect. The mesh’s normals point outwards from the front face, causing both sides to receive the same light. To fix this, flip the mesh’s normal vector when drawing the back face. To support normal mapping, make sure to use this flipped normal when transforming from tangent to world space.

Both sides have roughly the same shading, even though the sun is behind the mesh on the right side. There are slight differences due to specular lighting.

Create a subgraph called “CalcDoubleSidedNormal.” This will take in a tangent space normal and output a world space normal. This “is front face” node returns true if rendering on the front face. Use a branch node to switch between 1 and -1 depending on this value.

The Transform node cannot be modified to use a different normal vector, so we must recreate it. Construct a “tangent to world” matrix using the modified normal and multiply it with the tangent space normal, and normalize. Again, for more information on tangent space and normal mapping, check out the links in the video description.

In the main graph, route your normal map sample through this subgraph instead of the transform node.

In the scene, lighting should look correct on both sides, but you might have to turn off shadows to tell.

Unfortunately, this shadow problem is difficult to fix within the shader graph, since we cannot modify the shadow caster pass. Try adjusting shadow bias settings on the light or URP asset. If this issue turns out to be a deal breaker, I have a solution in the text shader version of this tutorial.

Shadow “acne” artifacts are more visible on the back face since the shadow caster normal still points the wrong way. This is not adjustable in the shader graph. Try changing depth bias to help.

“Shape” normals

One of the tricky things about foliage is that it should be translucent! Light filters through leaves, changing the way lighting and shadows affect them. Short of raytracing, we can’t have real translucency. But, we can add several techniques to approximate it — the first is a method using what I call “shape” normals.

Shape normals are separate normal vectors used exclusively for diffuse lighting. As opposed to the regular mesh normals we know and love, (which describe a face’s orientation) shape normals follow the overall shape of a plant. For instance, grass’s shape normals should follow the terrain, while a tree’s should point outward from a central location, following a sphere or cone. Using shape normals for diffuse lighting hides the fact that many foliage meshes are made of a bunch of flat planes.

Using mesh normals (left) vs shape normals (right) for diffuse lighting.
Using mesh normals (left) vs shape normals (right) for diffuse lighting.

You might be tempted to use shape normals for all lighting calculations. Regular normals, which I’ll refer to as “mesh normals,” are still useful for specular lighting and another technique we’ll get to later. The best thing to do is keep track of both. I’ll cover strategies to do this, but first let’s prepare the lighting algorithm.

Open FoliageLighting.hlsl. In the data structure, rename the current normalWS to meshNormalWS, which will hold the regular mesh normals. Add a new float3 shapeNormalWS field.

Go through the file renaming normalWS to the correct variable. Use the mesh normal in the reflection and fresnel calculations, shape normal in the diffuse formula, mesh normal in the specular dot product, shape and mesh for the diffuse and specular node previews, and shape normal to mix GI.

Look for “//←” for changed lines.

In the shader graph wrapper function, CalculateFoliageLighting_float, split the Normal argument into MeshNormal and ShapeNormal. Set them in the data structure. Then, pass ShapeNormal in the OUTPUT_SH and SAMPLE_GI macros.

Look for “//←” for changed lines.

In the FoliageLighting subgraph, update the custom function to match the wrapper function. Add a new shape normal property and route it into the custom function.

In TestFoliage, temporarily route the double sided normal into the shape normal.

Return to the scene. Things should be clear of errors, but everything will look the same. Now, to calculate the shape normals for each of our example meshes. For the sphere, which models a low poly tree or hedge, the shape normal is equal to the mesh normal. That’s how things are at the moment.

For the grass, we want the shape normal to point upward from the terrain. We need a vector which points upwards along the grass card. This corresponds to the mesh’s bitangent vector!

Test shader routing a bitangent node into base color. It’s green! Or (0, 1, 0), pointing up.

To support both shape normal options in the same shader, we can use an enumeration keyword. These allow you to change code depending on an option in the material. In the blackboard, add a new enum keyword called “ShapeNormal” (the option is under the new property menu). Set the definition to “shader feature,” the scope to “local,” and the exposed setting to checked. Add an entry for “MeshNormal” and “Bitangent.” You can now select an active entry in the material inspector.

If you drag this keyword property into the graph, Unity will create a branch node for you. It passes through the value connected to the currently selected keyword entry. Route a normal vector and bitangent vector into the correct fields, and then into the shape normal field of the lighting node.

In the scene editor, select the bitangent option in your grass material. Much better!

Create a C# script called “FoliageShaderSupport.cs.” Inside, add a list of transforms called normal foci. This script finds the closest focus to each vertex and calculates a vector pointing from the focus to the vertex. This will be the shape normal.

Back in the scene editor, add this script to your mesh game object. Create another GameObject for the normal focus and position it near the center of the tree; then, set it in the script. To test things out, create a vertex color node in your test foliage graph and route it directly to the base color output field.

Press play, and you should see colors on the tree mesh! Looking at it through the scene view, the colors should roughly correspond to the little XYZ compass in the corner.

The normal focus game object is selected.

Back in TestFoliage, remove the vertex color node and route the lighting back into the base color. Using another keyword entry, add the option to use vertex colors as shape normals.

Back in the scene editor, change the tree material to use vertex colors. Then, press play and check it out! If things look strange, try adjusting the position of the normal focus, adding more foci, or even adding more vertices to the mesh.

Translucency

Shape normals create smooth, more realistic lighting, but the meshes still look opaque. If you view a light through a foliage card, it should glow a little! We can simulate this translucency effect using a simple subsurface scattering lighting algorithm.

In my lighting formula explorer program, which you can check out here, I created a diagram detailing the algorithm. The effect is strongest when the view direction and light direction are opposite, which we can calculate mathematically by taking the dot product of the view direction and negative light direction.

This works well, but can make materials look flat. In reality, when light exits a transparent material, it changes direction slightly towards the surface’s normal vector. By adding the normal, scaled by a scattering coefficient, to the negative light direction, we can simulate this.

Let’s ignore scattering for now and add simple translucency to the lighting algorithm. In FoliageLighting.hlsl, add a float3 subsurface color variable — a tint applied to any light filtering through the mesh — and a float thinness variable — the translucent effect’s strength.

In the wrapper function, add arguments for these new fields and set them in the data structure.

In FoliageLightHandling, set the translucency light radiance, which is just the “regular” radiance tinted by the subsurface color. Calculate the translucency dot from the negative light direction and view direction. Similarly to specular highlights, tighten the dot product with a smoothness power and multiply by thinness. Finally, multiply the albedo, translucency radiance, and translucency strength together and add it to the final color.

In the FoliageLighting subgraph, add a new subsurface color and thinness custom function argument, and graph properties for each.

In TestFoliage, add subsurface color and thinness property as well.

In the scene editor, set your main light to point straight in the Z-direction and take a look at it through any of your meshes. You may or may not see a glow, depending on your shadow settings. The problem is the object’s cast shadows will attenuate translucency lighting as well.

The best solution is to remove shadow attenuation from the translucency radiance formula. This can cause translucency to shine in some places it shouldn’t, but In my experience, it’s not very noticeable. The final call is yours!

Let’s return to scattering and add it to our algorithm. In FoliageLighting.hlsl, add a new float field to the data structure.

And a corresponding argument in the wrapper function.

Then, in FoliageLightHandling, calculate the scattered light direction by adding the scaled mesh normal to the negative light direction. Normalize it and use it in the dot product formula.

Update the custom function and add properties in both graphs.

Then, in the scene editor, check out your materials.

It really adds a lot of depth!

So that takes care of most translucent effects, but it would be nice to approximate translucency from indirect lights too. Since we can’t change Unity’s lightmapper, the best way to do this is to add an ambient subsurface lighting strength, which gives foliage a constant glow.

In FoliageLighting.hlsl, add a new float field to the data structure.

And a corresponding argument in the wrapper function.

In FoliageGlobalIllumination, add a subsurface term to the indirect diffuse formula, multiplying the albedo, subsurface color, thinness and ambient strength.

Update the FoliageLighting custom function and add a property in both graphs.

Take a look in the scene editor. A little goes a long way!

Data Maps and Lighting Refinements

So far, all of these various lighting properties are constant over the mesh. It would be better to be able to read them from a texture, similarly to smoothness or occlusion maps. However, there are so many properties that putting each in individual textures would slow down the shader. We can combine four into one data texture, storing each in a color channel.

You can choose to vary any four properties and organize them in any channel order; however, for this tutorial, I will use the following layout: red contains smoothness, blue contains specular highlight strength, green contains translucency thinness, and alpha contains ambient occlusion.

My test data texture, created using Substance Designer.

The lighting algorithm is ready to handle this, except there’s no specular highlight strength. I feel that this is useful to prevent highlights in foliage creases, so let’s add it really quick. While we’re in there, let’s also add a multiplier to fade the ambient reflection rim effect, which might not be very prominent on foliage.

In FoliageLighting.hlsl, in the data structure, add a specular strength field and an environment reflection strength field.

Add variables for these in the custom function wrapper.

In FoliageGlobalIllumination, multiply the indirect specular result by the environment reflection strength and the specular strength. Then, in FoliageLightHandling, multiply the specular value by the specular strength.

Update the FoliageLighting subgraph with the new custom function arguments and add corresponding properties. Add properties to TestFoliage as well.

Then see how it affects your materials. OK, now it’s ready for a data map.

No specular highlight or rim lighting!

In the TestFoliage graph, add a new data map texture2D field and sample it. I like to keep around all the individual properties to use as multipliers for each channel, but you can also delete them for more minimal materials. Route and rearrange everything. Double check that everything is hooked up correctly.

Head into your texture creation program of choice and construct a simple test data texture. Draw a different pattern in each channel, preferably with soft gradients to test a variety of values.

Back in Unity, turn off sRGB and “alpha is transparency” in the texture importer and place it in your materials. This makes it really easy to see how each value affects lighting. You’re all set to create intricate plant-life!

Wind deformation

With that, we’re done with the lighting in our foliage shader; however, it’s too stiff to feel like real plant matter. Besides translucency, plants’ tendency to move with the wind is another challenge. We can animate meshes in the shader very efficiently by passing new vertex positions to the master stack, and it’s perfect for wind!

Create a subgraph called WindDeformation. This will take in a vertex position and output a new, deformed vertex position. Add a Vector3 wind direction property and a float strength property. Multiply the strength by the direction and add it to the position, returning the result.

We can vary the wind strength over time and space using a noise texture. Add a Texture2D noise texture property and sample it using a sample LOD node (it must be an LOD node, since we’ll use this during the vertex shader stage). Next, calculate a UV. World position should influence it, so use a Swizzle node to select the XZ world space position and multiply it with a new noise scale property. Similarly, multiply time by a noise frequency property.

The output of the texture sample ranges from zero to one, but that would weight the deformation on one side of the model. Use a remap node to change it to range from -1 to 1. Use a split node to grab the red channel and multiply it with the wind strength.

Plants won’t sway in a straight line, due to wind turbulence. To simulate it, add a float turbulence strength property and a cross wind direction. This “cross” direction will be perpendicular to the main direction. Multiply the turbulence strength by the wind strength (so it is proportional), the noise green channel, and finally the cross wind direction. Add the offsets together.

In test foliage, add the new WindDeformation node. Create properties for wind direction, strength, turbulence, noise texture, noise scale, and noise frequency. Route all these into the wind deformation node, as well as vertex position, in world space.

For cross direction, we need a perpendicular to the wind direction, but still looks natural (it shouldn’t point into the ground, for example). The cross product of the wind direction and the “shape normal” will do the trick. Finally, route the deformed position through a “world to object” transformation node and into the position field of the vertex stage master stack.

Open your image processor of choice and create a noise texture. Make sure it has independent noise in the red and green channels. You can use any shape you want, but I found simple cloud or Perlin noise does the trick. In Unity, make sure the texture importer has sRGB turned off. You also safely turn off mipmaps, as well as compression if the wind motion ever feels jittery.

Set up your materials! So, we have wind, but there are several problems. First off, wind causes the grass to detach from the ground! We need a way to scale wind strength over the mesh so it’s weaker near the bottom of the grass quad.

Luckily, the UV’s y-coordinate follows exactly the pattern we need.

UV y-coordinate displayed as mesh color

First, in the wind deformation subgraph, add a float dampening property. Multiply it with the final offset before adding to vertex position.

In the main graph, we need to route the UV Y coordinate into this new field, but only for grass. Otherwise, dampening should be one. Another keyword will do the trick: add an enum keyword called WindStrength, with all the same settings as before. Create two entries: “Constant” and “UV Y.”

Drag it onto the graph, connect a Float node with a value of 1 into Constant, and a UV node with a Split node, to grab the Y coordinate, into UV Y. Route that into the dampening field.

In the scene editor, change the wind strength enum on your grass material to UV Y. Much better.

Let’s take a look at the tree. I see two problems. First, imagine if a leaf texture was assigned to this material. The wind would cause it to stretch and distort in unnatural ways. To fix this, we should use the same position for all vertices when sampling the wind noise. Second, parts of the mesh attached to the trunk should not move. Imagine the center vertex is anchored in this way.

We can fix both of these problems by storing custom wind noise anchors and dampening values in the mesh vertex data. Meshes can store a bunch of independent UV coordinates, and the shader graph can access four. The last set, referred to as “UV3” or “TexCoord3,” is usually unused, so we can use our C# script to store this wind data inside it.

Open FoliageShaderSupport.cs. I’ll go over this code quickly, since it’s tangential to the foliage shader. Our goal here is to set the wind noise positions to a constant position (the center of the mesh’s bounds will work), and the wind dampening values to a value proportional to distance from the closest focus. Anchors are stored in the XYZ channels of the TexCoord3 vector, while dampening in the W channel. Set the array in the mesh’s UV stream, and it’s ready to be read by the shader.

In the WindDeformation subgraph, add a new Vector3 noise position property. Use that instead of vertex position when calculating the noise UV.

In TestFoliage, add another enum keyword called WindAnchor. Add two entries: Position and TexCoord3. Drag the keyword onto the graph and route a position node, in world space, and a UV node, in UV3 channel mode, into the fields. Connect that into the noise position field of the wind deformation node.

To support precalculated wind dampening values, add a third entry to the wind strength enum keyword called TexCoord3. Use a split node to grab the W-component from another UV node in UV3 channel mode into the new option.

In the scene editor, verify your tree model has the script attached and that the TexCoord index is 3. Switch the tree material keyword modes to TexCoord3 and enter play mode, so the script runs. That’s more like it! Feel free to modify the C# script as you need — this data is highly dependent on each individual model. But for now, time to address the last problem.

If you look closely on any model — but it’s especially apparent on the sphere — you’ll notice that wind deformations don’t affect specular lighting. They should, since wind deformations change the apparent normal of the mesh. The normal vector doesn’t automatically update to match new vector positions, so we need to recalculate them — and tangents as well. It’s difficult to do perfectly, since a shader only knows the position of one vertex at a time, but we can estimate it.

The shader is outputting normals as the base color. Note, they don’t change as vertices move.

The strategy is to run three points through the wind deformation algorithm — the original position, and the original position slightly offset in the tangent and bitangent directions — and see how the deformation algorithm affects them. We can construct new tangents and bitangents and thus a normal from their cross product.

Implement it in a new subgraph called “WindDeformationWithOrientation.” Inside, output a position, normal, and tangent vector, and add properties for wind direction, cross direction, strength, turbulence, noise texture, noise position, noise scale, noise frequency, and dampening. Create a WindDeformation node and route all the properties into it. Duplicate the deformation node twice, leaving everything attached.

Route world space position into the top wind deformation node. The middle node takes the position nudged a small distance in the tangent direction. The bottom node needs the position nudged along the bitangent, but due to some quirks with the way the shader graph bitangent node works, it’s safer to calculate the bitangent yourself. Route the normal and tangent through a cross product node and normalize the result.

Subtract the top node’s output from both of the other outputs and take their cross product. The order is important. Connect up the deformed position and normal outputs. For the deformed tangent, normalize the first cross vector — it’s guaranteed to be perpendicular to the normal, thanks to the cross product.

In TestFoliage, replace the WindDeformation node with a WindDeformationWithOrientation node. Route the normal and tangent vectors through a world to object transform node before connecting them to the master stack.

In the scene editor, you won’t really notice any changes. This is because we separated vertex position from the noise UV, which partially determines wind offsets. In the WindDeformationWithOrientation subgraph, add the tangent and bitangent offsets to the noise positions.

This looks great on the sphere!

But, not so much on the tree and grass. The shine seems almost random, which is an artifact of the distance between vertices skipping over and not reflecting changes in the wind noise. Adding more vertices would certainly help, but perhaps the right solution is to turn off normal recalculation when using low poly meshes.

In the main graph, add a boolean keyword called “WindDeformsNormals_On.” The “On” suffix is important to make sure it is editable in the material. Drag this on to the graph twice, connecting the deformed normal and tangent into the “on” fields. Route a normal and tangent node, both in object space, into the “off” fields. Connect these to the master stack.

In the scene editor, turn off normal deformations for the grass and tree materials. As I mentioned, it’s difficult to get normal deformation perfect without employing some costly techniques. If it is important to your game, you can continue refining the algorithm to match your model’s needs. Maybe calculate dampening based on position, or use a function instead of a noise texture (so you can calculate exact normals). Regardless, this option is available.

Wrap up

I think that’s about it for foliage in the shader graph. You can endlessly tweak things to fit your specific model. Speaking of that, I’d love to see your trees, grassy plans, or video game gardens! Feel free to send me screenshots!

For inspiration, here are the material settings I used to create the scenes in this video. The tree has a custom support script to set wind data, which finds connected pieces of the mesh, making sure each chunk has the same wind noise position.

Some effects currently not possible in the shader graph, like screen space ambient occlusion, do really improve the final look. You can add support pretty easily when 2021.2 releases, or take a look at the upcoming text shader version of this tutorial, which already supports it.

The left side of the tree has SSAO turned on. Note the soft shadows between the leaf cards.

Thank you so much for reading! Foliage is really fun to create, I think, and I’ve only really scratched the surface of foliage rendering here. I have several more videos planned, including one specifically on covering terrain with fields of grass. Please keep an eye out for it!

Here is the final version of the scripts in the project, for cross referencing.

If you have any questions, feel free to contact me at any of the links at the bottom of this article.

If you want to see this tutorial from another angle, I created a video version you can watch here.

Try out my Interactive Lighting Math Explorer to visualize how diffuse, specular, translucency and other lighting calculations work: https://nedmakesgames.itch.io/lighting-explorer

I want to take a moment to thank David Cru for his support, all my patrons from the last few months for making this tutorial possible: Adam R. Vierra, Aleksandr Molchanov, Alina Matson, Alvaro LOGOTOMIA, Andre Schuch, Antonio Ripa, anzhony manrique, Ash Free, Ben Luker, Ben Wander, BM, Bohemian Grape, Brooke Waddington, Cameron Horst, Candemir, ChainsawFilms, Chris, Christopher Ellis, Connor Wendt, Danny Hayes, darkkittenfire, David Cru, Evan Malmud, Eren Aydin, Erica, FABG Team, Georg Schmidbauer, hyunsookim, Isobel Shasha, Jack Phelps, Jerzy Gab, JP Lee, jpzz kim, Justin Criswell, Karthick Gunasekaran, Kyle Harrison, Leafenzo, Lhong Lhi, Luke Hopkins, Mad Science, Mark Davies, masahito nagasaka, Matt Anderson, maxo, Mike Young, NG, nobuhiko yamakoshi, Omar Sadek, Oskar, Oskar Kogut, Pat, Patrik Bergsten, peter janosik, phanurak rubpol, Qi Zhang, Quentin Arragon, rafael ludescher, Sam Slater, Samuel Ang, Sebastian Cai, SlapChop, starbi, Steph, Stephen Sandlin, Tomasz Beneś, Tvoyager, Voids Adrift, Will Tallent, Winberry, 성진 김

If you would like to download a completed version of this tutorial and many of the files featured here, consider joining my Patreon. You will also get early access to tutorials, voting power in topic polls, and more. Thank you!

Thanks so much for reading, and make games!

🔗 Tutorial list website ▶️ YouTube 🔴 Twitch 🐦 Twitter 🎮 Discord 📸 Instagram 👽 Reddit 🎶 TikTok 👑 Patreon 📧 E-mail: nedmakesgames gmail

Credits, references, and special thanks

All code appearing in GitHub Gist embeds is Copyright 2021 NedMakesGames, licensed under the MIT License.

--

--

NedMakesGames
NedMakesGames

Written by NedMakesGames

I'm a game developer and tutorial creator! If you prefer video tutorials, check out my YouTube channel NedMakesGames!

No responses yet