alt Hacker NewsOne conference proceeding paper and one preprint, about LLMs encoding either relative geometric information of objects or simple 2D paths.
One of the papers call this "programming language semantics", but it is using a 2D grid navigation DSL. The semantics of that language are nothing like actual programming language semantics.
These are not the same as the concept being discussed here, a human "world model" of a computer system, through which to interpret the semantics of a program.
Well I didn't find any papers off the bat for code world models but if they can create a world model for the task given, such as geometric manipulation, I don't see why they wouldn't in terms of code.