>So when do robots gain their own agency? Will AGI have it's own goals and agenda? If so, will it be merely for self-replication? Like a paperclip maximizer, but for robots? Is that all we are?

They already have the agency they need in the form of the Business Model. That is all the framework that is needed. AI companies competing against AI companies because they are developed to work within the scope of the business model. Chasing efficiency, margin, lowering costs, increasing profits. These rules are as simple as what drives evolution via natural selection. Unfortunately this business model creates a cancer: the company striving for more growth and lower costs will eat the companies that aren't or aren't as efficient.

Whether the host (planet, I guess) lives is another story. In nature, parasitic relationships are far more likely than symbiotic relationships to evolve. The overshooters need to die and be outcompeted by the symbiotes who better protect their host. However, we only have 1 host here. Once that is overshot, that is it. No diverse competitive environment for which a new host preserving victor to emerge from. Maybe one might emerge, but not after unspeakable pain for all life on earth, if life on earth even is allowed to exist.