alt Hacker NewsI co-authored this paper. It's a new technique to generate 3D graphics as source code instead of a point cloud.
Under the hood, it generates 3D objects with separate, sophisticated internal assembly, producing an editable "kit of parts" (instead of monolithic blobs).
E.g. imagine you generated a 3D washing machine via this approach. It's not merely going to be just "geometry" that looks like a washing machine. We actually know that there is a `Door`, `Drum`, `Control_panel` etc. Which things belong to which assemblies. What moves and where its pivot is. And eventually what those components are supposed to do.
Most current 3D GenAI cannot do this since it generates "monolithic blobs" that look good, but are unusable in downstream workflows (e.g. game engines). I.e. if you generate a 3D bicycle using traditional approaches, it's basically a blob. When you need the wheels to turn, a human (or another AI) must spend time cutting the blob into parts, naming them, placing pivots and rigging joints. I.e. you need post-generation segmentation workflows of some sort.
The paper breaks down the whole technique, and there's a showcase (+ github repo) you can play around with: https://nova3d.xyz/
Replies
Have you explored optimizing the assets to be game-ready?
This kind of decomposition works if you have a single object on screen, and it's super artist + programmer friendly. But the generated assets have ~50 mesh parts, which means importing just a couple of these into a scene and you've blown your entire draw call budget for a shippable game; once you start adding things like shadowing it's game over. It's the brick wall every gamedev realizes after trying to make a scene out of easy-to-work-with primitives. You just can't hit a playable frame rate like this unless your entire game consists of just a few objects.
Have you experimented with atlasing, mesh fusion, baking animations, standardizing PSO's to a scene budget, etc? Because if this can't be automated, I've found it really limits the utility of such freeform generation techniques, since the approach is fundamentally incompatible with performance on today's graphics stack.
This looks super interesting. I'm trying out the hosted app using "bring your own key", I've added an OpenAI key but it doesn't seem to let me generate a 3d model. It's still saying I need credits. Is this expected?
How does this compare to parametric modeling tools like Fusion/Solidworks/ProE ? Is it more about the integration with game specific tools?
Cool. Do the individual parts still use point clouds? Or are they meshes or CSG?
How far are we from speaking a GameCube-era game into existence as a pastime?
Did I read this right? The objects are still just surfaces, not solids?
> there's a showcase (+ github repo) you can play around with: https://nova3d.xyz/
Before anyone else bothers giving them your Google account, there's zero free generations, something they conveniently don't disclose until after funneling you to sign up.