Dynamic Foliage

Hi! I’m Will, a graphics engineer working on Project-S.

Foliage is an important aspect of any game, especially a game like this with such a focus on immersion and weather. Recently I have been working on systems to allow us to have realistic, dynamic foliage in our game and I have written this blog post detailing part of that process.

The Beaten Path

The most common approach for dynamically deforming foliage in games is World Position Offset. This is an effect run in the vertex shader, and is used to change the position of vertices in a mesh at runtime. In older games, the vertices were usually offset by a simple noise texture and scaled by distance to the ground (to avoid the look of foliage sliding on the ground).

A more modern approach has been to bake data such as branch pivots into vertex colors to allow for more complex calculations and per-branch deformation. Often, additional calculations are needed to transform the vertex normals based on the deformation which added extra overhead.

Modern advancements such as Virtualised Geometry (Nanite in UE5) have meant that a change is sorely needed. The biggest performance cost of World Position Offset (WPO) is that it sends high-detail Nanite meshes down a separate rasterisation path, instead of just batching it together with regular geometry. Non-WPO geometry all gets batched into a single raster bin (batch of meshes to be processed together), whereas every WPO material & mesh needs its own individual raster bin which greatly increases the performance cost.

Our approach to deformation

To solve the issue of processing per-vertex deformation calculations, we first looked at Unreal Engine’s new, experimental Nanite Foliage and Dynamic Wind plugins. Since these are very recent additions to the engine, they provided a great starting point but needed modification to fit our needs.

Nanite Foliage helps reduce the amount of overdraw caused by micro-geometry such as leaves by voxelising it at a distance, with voxels usually being 1 pixel in size so that there is little-to-no visual quality lost.

Voxelised foliage shown with ”r.Nanite.ViewMeshLODBias.Offset=6”

The Dynamic Wind Plugin provided a great base framework for using Skeletal Meshes that we could apply Per Bone animation to instead of Per Vertex. It already had the skinning paths integrated and a compute shader for animation generation that we could modify.

This plugin did have a number of limitations, such as no support for wind variation based on the world-space location of the foliage instance, which is much needed for a dynamic weather system like ours, since our weather-map and wind-map are rendered across the entire level.

Tree skeletons view

In our use case, Skeletal Animation is much cheaper than WPO, although it does have a greater memory overhead. It also allows us to have a much better quality of animation, since we can apply more physically-based techniques to a skeleton using a compute shader, rather than deterministic, per-vertex animation. Having a rigged skeleton also helps us to avoid warping artefacts since the animation is driven by Rotational Translation rather than Positional Translation (used with WPO).

How we render wind

In Project S we have 2 main sources of wind: Wind Emitters and the Weather Map. The Weather Map provides a large-scale ambient wind map, scaled based on how intense the weather is. Tornado wind vectors are also rendered into the Weather Map. 

Wind Emitters are used for small scale wind such as from player or vehicle movement and are rendered locally into a map 300m around the player. This allows us to have players and vehicles interact with foliage.

Our approach to animation generation

Attempt 1

My first idea for allowing spatialised variation for each foliage object was to sample our Wind Map at its location, and then generate its animation based on the Wind Strength and Wind Direction. Unfortunately, with potentially hundreds of thousands of foliage instances in a level, it was far too performance intensive to generate a unique animation per-instance.

Attempt 2

This attempt was promising but limited - if I couldn’t sample the Wind Map at each individual foliage instance’s location, what if I sorted them into a grid, and sampled at that grid’s centre? This meant I could generate one animation per-foliage-type per-grid-cell which was a huge performance saving. However, this meant that all identical foliage meshes in a cell had an identical animation, which not only meant that every tree not only looked identical but also bent at exactly the same time and in exactly the same way.

Attempt 3

This attempt solved all of our existing issues. There are still some limitations that we are working out but overall it has been much better than any other technique we tried.

The first step was to do a pre-pass every frame over all foliage in the level (or at least, all foliage that was close enough to the player to animate) by sorting each instance into a table. 

This is cheap because we piggyback off an existing function - “GetSkinningDataOffset()” which is already called per-instance at runtime.

On the X axis of the table was Wind Strength and the Y axis was Wind Direction (more accurately Relative Wind Direction, the direction of wind relative to the direction that the foliage is rotated). We used an 8x8 table for this, so each foliage type was sorted into one of 64 groups. Then since each item in the table would have an identical (or near enough) wind strength and wind direction, we generate one animation per table item, which then gets propagated to all the instances registered with that table cell. This is significantly cheaper than any other technique, only having to generate up to (since not all table slots have any entries) 64 unique animations per foliage type.

The main limitation of this setup is that we don’t get much “resolution” in terms of wind strength - on a normalised 0-1 scale, the increments are fairly large with just an 8x8 table (0, 0.125, 0.25, etc) which can result in jumpiness with fast-changing wind strength and direction. I am currently looking into solutions and workarounds for this, such as dynamically changing the strength increments based on what would be most useful in order to smooth the transitions more.

Foliage in a test level, deformed by wind