Update (2023-09-06): Some of the content in this article is outdated. I’m inactively working on a new version and I’ll probably update this later _(:з)∠)_

Note: This is just a proof of concept and won’t be production ready any time soon.

Overview

This is an early attempt at recreating Genshin Impact’s interactive sand deformation. Genshin Impact’s implementation appears to use a depth capture + dynamic tessellation approach that creates 3D trails from any mesh that intersects with the desert terrain. My implementation builds on top of Batman: Arkham Origins‘s depth capture approach, adding support for complex terrains, indent rims, etc.

Requirements

Genshin Impact’s recent 3.1 update features a desert region with some interesting visual effects. The desert terrain deforms in response to contact with meshes, leaving indented trails that closely fit the contour of the meshes. These indents bury into the ground and appear to push some sand outwards, forming rings of raised sand. Contact from most types of meshes can all trigger this effect, including characters, mobs, and even weapon slashes, so it seems to be a very flexible implementation.

We can identify the following key requirements for this feature:

1. Dynamic tessellation of ground mesh.
2. Ground deforms by vertex displacement in response to contact with objects.
3. Trail shapes closely follow the contour of any mesh that intersects with the ground.
4. Ring of raised sand around the indent, simulating how sand is pushed away from the point of contact.
5. Trails gradually regenerate over time.
6. Unrestricted area of effect (i.e. compatible with open world scenes. Trails can be created anywhere in the scene where the ground material uses the corresponding shader).
7. Easy integration. Does not intervene with the existing entity development workflow. i.e. For Genshin Impact, it has to be compatible with all existing characters as well as any new characters in the future, ideally developers don’t have to set up special configurations for characters and objects in order for this effect to work on them.
8. Integrated with the terrain system.

Existing Solutions

Particle Emitter Approaches

Some interactive snow shaders found online require special per-object setups. For example, adding particle emitters to a character’s left and right foot, then using a camera to capture particles into a render texture, which the ground shader uses as a height map. These methods can work really well, but when placed in the context of Genshin Impact, there can be several drawbacks:

• Aesthetics – The particles emitted usually have fixed shapes which result in fixed-shape indents. This won’t meet requirement 3, and this doesn’t look like what Genshin Impact uses.
• Flexibility – A lot of these implementations only work on predifined areas in a scene. While it is possible to apply this effect to multiple predefined areas, the performance cost would make it technically unfeasible for an ope world game (one 1 x 1 area requires something like a 512 x 512 render texture). To address this we might either need some kind of render texture pooling system, or some mechanism to accurately offset a single large render texture based on main camera movement.
• Scalability – Using a particle emitter approach, it would take a lengthy configuration process just for one type of object to be able to create indents in a very specific way. This would take a lot of effort to do and seems to be unfit for a live-service game that’s constantly pushing out updates at a fast pace.

In particular, scalability is a crucial aspect to consider, especially for games like Genshin Impact, where there’a a large expanding open world and new characters are being constantly added every 42 days. Adding another lengthy step onto the character configuration process just to support one effect that’s only available in certain regions of a scene does not sound very elegant. It would be better to have a system that is decoupled from the character configuration process. Genshin Impact’s implementation works not just on characters, but also all sorts of meshes. It would be unrealistic for the developers to set up configurations for all these objects in a game that pushes out lots of new content at a fast pace. We need an approach that has the potential to hit the mark aesthetically and doesn’t require special per-object setups.

Depth Capture

Batman: Arkham Origins uses a depth capture method, described in this GDC video. This is a scalable approach. Camera depth textures can capture object shapes with higher precision, and we don’t have to setup particle systems for every character or object we want to capture. There are some limitations though. This method only supports flat terrain, and doesn’t work well on uneven surfaces. Objects on a higher elevation might be ignored because they’re outside the depth threshold. One way to address this is to take into consideration the worldspace vertex height in the post-tessellation vertex shader and calculate normalized indentation depth values relative to terrain height (there might me more efficient ways to do this).

Another problem is open world compatibility. Batman: Arkham Origins uses a pooling approach to manage multiple depth textures for different pre-defined areas. There may be some ways to make this easier for open world games.

• Have a automated pooling approach that allocates render textures at the right positions based on where the player character is.
• OR constantly offset render texture contents based on the world offset of our character. There might be precision problems, so if this doesn’t work we’ll have to fallback to the pooling approach.

My implementation mainly addresses the flat ground limitation + adds indent rims. Open world compatibility is something I want to look into in the future.

Microsplat Trax

Microsplat’s Trax module is a really good production-ready solution with its own buffer mechanism for open world compatibility. I don’t know whether it can have a ring of raised sand around trails as per requirement 4.

Sandprints

The current WIP Sandprints effect uses a modified depth capture approach to achieve similar results as Batman: Arkham Origins.

Update (2023-09-06): The following sections showcase a legacy version of the project.

Features

√ Trail shapes closely follow the contour of any object that intersects with the ground.

√ Ring of raised sand around an indent, simulating how sand is pushed away from the point of contact.

√ Indents slowly regenerate over time.

? Unrestricted area of effect using the RT offset method. There is likely a better way to do this.

√ Easy integration. No special configuration needed. Any mesh can leave trails on the terrain as long as they’re in the target layers.

How it works

• Use a 2-camera setup: Bottom camera to capture object depths, top camera to capture terrain depth. The depth textures are combined to generate a single normalized depth texture of object depths.
• Apply Kawase blur to the depth texture to smooth the result. Apply an extra pass to generate outlines on a different color channel.
• Areas where sand should be lowered are now marked by R pixels; Areas where sand should be raised are marked by G pixels.
• Add the new pixels to the current heightmap texture. A compute shader updates the texture to fade pixels over time, which regenerates the indentations.
• An external script passes required information to the sand shader and controls the worldspace position of the depth cameras. This script can be configured to follow the current active character gameobject and provide its wordspace position change per frame. The heigthmap is shifted based on target position change. This way, the effect will always be available within a range around the player.
• The sand shader uses the heightmap texture to perform distance-based tessellation. The shader also maps worldspace position change to texture uv, so that deformation is applied at the correct position.

Issues

• I’m using the RT offset method for open world compatibility. Due to precision problems, the heigtmap under bilinear/trilinear filtering will be come a bit more blurry every time it’s shifted. A potential solution is to accumulate character position change and only offset the texture when we reach whole values.
• Genshin Impact has special smoothed self-shadows in the indented area. This is not implemented here.
• No rim light and sand sparkles, etc. at the moment.
• No Unity terrain system integration.