In Factorio's paper [1] page 3, the agent receives a semantic representation with coordinates. Have you tried this data format?

[1]: https://arxiv.org/pdf/2503.09617