you can cache layouts with immediate mode libraries. in terms of scaling, it really depends - if you have many components updating at once, the benefits of retained mode libraries are diminished if not reversed. when people create demos of 10,000 list elements and updating a single one, it's basically designed as a best case scenario for retained mode. try updating the entire list...
but i can make strong claims about the benefits of not needing to duplicate your state inside of your ui framework. that alone removes an entire category of work, and i would argue scales better from a conceptual/dev standpoint.