High quality camera lenses take space. That alone starts to dictate the size of the phone. Add in the desire for all day battery life, and we start talking more volume. And the fastest chips take more power, so again more battery. The best way to justify the increased size is with a bigger screen and now you’re stuck with a larger phone.
Or a smaller phone that is more expensive to produce. In which case, it won’t sell well regardless.
It’s easy to see how we got to this point of large phones, but it is not very clear how someone like Apple can make a significantly smaller phone and keep customers happy.
As you mentioned, other vendors probably will have better luck.
Faster chips absolutely don't have to take more power. Apple's fastest chips tend to be some of their most power-efficient. It's sustained workload that takes power, and the faster chips just have a higher performance budget.