alt Hacker NewsIn case it's not obvious, this isn't a model, this is python scripts that call out to models (code not available).
It's mostly "did you know you can attach an LLM to a PCG system?", but there is one idea here you don't see much of: an image model performs the composition (which image models are really good at), and then you extract the objects into 3d via things like SAM3D before placing them in the world, which is pretty interesting.
The rest is standard stuff you'll find in your favorite PCG system/game engine. Still, think most people don't realize how good LLMs are at 3D these days, especially Fable 5 (which this project predates).
Replies
> Still, think most people don't realize how good LLMs are at 3D these days, especially Fable 5
Can you explain more? Can I use Fable 5 to generate a 3d model from a couple of photographs?
That’s exactly how I’ve used tools like Tripo and Meshy.
Use any image generation tool (or any image) and have a multimodal LLM like Gemini extract individual parts of the image “isolate with a transparent background” - then you can use those images for image-to-3d in those tools.
Don’t forget to use low or “smart” poly features otherwise you get too many vertices to the point you can’t performantly raycast etc.
But yeah - it’s there.