GSWT: Gaussian Splatting Wang Tiles

The in-browser demo is very cool! It's not clear from the linked page, but the GitHub repo[0] includes links to sample tile datasets that can be used for the demo.

[0] https://github.com/zengyf131/gswt_renderer

Crazy to see 2 of my niche interests interact. Great idea, you could extent the idea to use example based texture synthesis, such as Image Quilting https://www.merl.com/publications/docs/TR2001-17.pdf

There's no link to the paper, so I can only infer, but, if I understand correctly, this is a very simple idea: take a single Gaussian splat "tile" and find a cut when two copies are placed near each other and overlapping, using dynamic programming to vary the size of overlap and where the cut should be. Have a variety of cuts to break a uniform tiling (the Wang tiles part) and now you have different tiles with different nearest neighbor constraints that you can use to tile the plane.

Probably a lot of details to be worked out in how to stitch Gaussian splats together but I imagine it's pretty do-able.

I think one of the problems with Gaussian splatting is generating content. You can take a static picture of something but it's hard to know how to use it for interaction. This is a way to generate 3d textured sheets, like sunflower fields, walls, caves, etc.

In my opinion, great idea.

The gaussian splatting never cease to amaze me. I wonder if it would be OK to proceduraly (not by LLM) generate natural worlds for video games with that...

Everybody Wang Tiles tonight.

But seriously, I didn't realize I wanted this. I was hoping to experiment with just repeating the same tile. This gives me hope that other people will make these techniques approachable.