The Bremermann limit is about computational density. It makes sense to talk about computers that are ever more powerful but not necessarily larger.

So we talk about the supercomputers we call cell phones that are orders of magnitude more powerful than the desktop computers I used years ago.

It doesn’t make sense to talk about making super large computers before reaching the density limit, that’s a confusion of concepts.

Making computers faster has involved making them smaller because the speed of signal propagation.