I probably wouldn't start with buying a big server if I was just starting out. I would consider renting one from Hetzner, OVH, or many others.

There are also many other alternatives. Example: VPS providers like Vultr which have expanded to offer more traditional "cloud" features like object storage, load balancers, managed DBs, etc. Their pricing is way more competitive than AWS, especially when you consider bandwidth.