Not being a 'party trick' and being 'AGI' are wildly different assumptions. Apart from AGI being a poorly defined term, I don't see any reason why the LLM methodology has the potential to scale to AGI.

There's other avenues which are probably needed to make the cohesive AGI picture come true, such as World Models. Simply learning from text and being able to generate language is not, in my mind, the end-all-be-all for intelligence.