Ah OK, that's basically exactly the setup I'd use as well. Surprising that the server alone couldn't handle the traffic, as the sibling says, 4 rps isn't that much when you cache (cache hits are basically free).

I imagine 90% of the traffic (or more) is anonymous users, which can be cached, doesn't Varnish handle that without breaking a sweat?