alt Hacker NewsIf you're at (-2, 3, 5), pointing in the (2, 1, -1) direction (as a direction vector), then which is in front of the other from your point of view: the object at (4, 5, -8) or the one at (6, 4, 3)? (or neither)
You can't just defer this decision to the three.js program because you need to understand this sort of relationship yourself if you're going to suitably place objects (and your own view) in a three.js scene in the first place. Building a complex scene involves making hundreds, or more, of decisions about where to place coordinates in 3d space.
This example is underspecified (e.g. what are the shapes and sizes of the objects and what is the angular width of your view) but it illustrates the problem. Even with a good intuitive understanding of 3d space, we would struggle with this. LLMs are not calculators, so it's surprising if they manage it.
I think of all these responses this articulates my point the most. If humans would struggle with it, why is it still supposed to be "spatial reasoning"? And underspecified or not, you frame this exactly in the way an LLM would tackle this: if I am making a game, I don't place objects in arbitrary positions in a space and try to keep it all in my head, but in relations to one another. Then, the point in front of another is the one where A-B>0.