Go ahead, self-host Postgres

Self-hosting Postgres is so incredibly easy. People are under this strange spell that they need to use an ORM or always reach for SQLite when it’s trivially easy to write raw SQL. The syntax was designed so lithium’d out secretaries were able to write queries on a punchcard. Postgres has so many nice lil features.

Just don't try to build it from source haha. Compiling Postgres 18 with the PostGIS extension has been such a PITA because the topology component won't configure to not use the system /usr/bin/postgres and has given me a lot of grief. Finally got it fixed I think though.

I've had to set up postgres manually ( before docker TBF) and it's best described as suffering.

Things will go wrong. And it's all your fault. You can't just blame AWS.

Also are we changing the definition of self hosting. Self hosting on Digital Ocean ?!

Disk read write performance is also orders of magnitude better/cheaper/faster.

Without disagreeing:

Sometimes it is nice to simplify the conversation with non-tech management. Oh, you want HA / DR / etc? We click a button and you get it (multi-AZ). Clicking the button doubles your DB costs from x to y. Please choose.

Then you have one less repeating conversation and someone to blame.

Cooking the RDS equivalent is reasonable amount of work, and pretty big amount of knowledge (easy to make failover solution have lower uptime than "just a single VM" if you don't get everything right)

... but you can do a lot with just "a single VM and robust backup". PostgreSQL restore is pretty fast, and if you automated deployment you can start with it in minutes, so if your service can survive 30 minutes of downtime once every 3 years while the DB reloads, "downgrading" to "a single cloud VM" or "a single VM on your own hardware" might not be a big deal.

People really love jumping through hoops to avoid spending five dollars.

I moved from AWS RDS to ScaleWay RDS, had the same effect on cost

Huh? Maybe I missed something, but...why should self-hosting a database server be hard or scary? Sure, you are then responsible for security backups, etc...but that's not really different in the cloud - if anything, the cloud makes it more complicated.

Pros self-host their DB's

Does anyone offer a managed database service where the database and your application server live on the same box? Until, I can get such latency advantages of such a set-up, we've found latency just too high to go with a managed solution. We are already spending too much batching or vectorizing database reads.

Better yet, self host Postgres on your own open source PaaS with Coolify, Dokploy, or Canine, and then you can also self host all your apps on your VPS too. I use Dokploy but I'm looking into Canine, and I know many have used Coolify with great success.

Everyone and their mother wants to host Postgres for you!

Some fun math for you guys.

I had a single API endpoint performing ~178 Postgres SQL queries.

  Setup              Latency/query   Total time
  -------------------------------------------------
  Same geo area      35ms            6.2s
  Same local network 4ms             712ms
  Same server        ~0ms            170ms

Now for the next step... just use SQLite (it's possible it will be enough for your case).

Disclaimer: there's no silver bullet, yadda yadda. But SQLite in WAL mode and backups using Litestream have worked perfectly for me.

And then there is the urge to Postgres everything.

I was disappointed alloy doesn't support timescaledb as a metrics endpoint. Considering switching to telegraf just because I can store the metrics on Postgres.

> When self-hosting makes sense: 1. If you're just starting out in software & want to get something working quickly [...]

This is when you use SQLite, not Postgres. Easy enough to turn into Postgres later, nothing to set up. It already works. And backups are literally just "it's a file, incremental backup by your daily backups already covers this".

I've operated both self-hosted and managed database clusters with complex topologies and mission-critical data at well-known tech companies.

Managed database services mostly automate a subset of routine operational work, things like backups, some configuration management, and software upgrades. But they don't remove the need for real database operations. You still have to validate restores, build and rehearse a disaster recovery plan, design and review schemas, review and optimize queries, tune indexes, and fine-tune configuration, among other essentials.

In one incident, AWS support couldn't determine what was wrong with an RDS cluster and advised us to "try restarting it".

Bottom line: even with managed databases, you still need people on the team who are strong in DBOps. You need standard operating procedures and automation, built by your team. Without that expertise, you're taking on serious risk, including potentially catastrophic failure modes.

The author's experience is trivial, so it indicates nothing. Anybody can set up a rack of postgresql servers and say it's great in year 2. All the hardware is under warranty and it still works anyway. There haven't been any major releases. The platform software is still "LTS". Nobody has needed to renegotiate the datacenter lease yet. So experience in year 2 tells you nothing.

I was on a severely restricted budget and self hosted everything for 15+ years, while the heavily used part of the database was on a RAM card. The RAM drive was soft raided to a hard drive pair which were 3Ware raid1 hdds, just in case, and also did a daily backup on the database and during that time never had any data loss and never had to restore anything from backup. And my options were severely restricted due to a capped income.

The real downside wasn't technical. The constant background anxiety you had to learn to live with, since the hosted news sites were hammered by the users. The dreaded SMS alerts saying the server was inaccessible (often due to ISP issues) or going abroad meant persuading one of your mates to keep an eye on things just in case, created a lot of unnecessary stress.

AWS is quite good. It has everything you need and removes most of that operational burden, so the angst is much lower, but the pricing is problematic.

(This is very reductionist)

A lot of this comes down to devs not understanding infrastructure and infrastructure components and the insane interplay and complexity. And they don't care! Apps, apps apps, developers, developers, developers!

On the managerial side, it's often about deflection of responsibility for the Big Boss.

It's not part of the app itself it can be HARD, and if you're not familiar with things, then it's also scary! What if you mess up?

(Most apps don't need the elasticity, or the bells and whistles, but you're paying for them even if you don't use them, indirectly.)

I wish this post went into the actual how! He glossed over the details. There is a link to his repo, which is a start I suppose: https://github.com/piercefreeman/autopg

A blog post that went into the details would be awesome. I know Postgres has some docs for this (https://www.postgresql.org/docs/current/backup.html), but it's too theoretical. I want to see a one-stop-shop with everything you'd reasonably need to know to self host: like monitoring uptime, backups, stuff like that.

Another thread where I can't determine whether the "it's easy" suggestions are from people who are clueless or expert.

[dead]

honestly at this point I'm actually surprised that there aren't specialized linux distributions for hosting postgres. There's so many kernel-level and file-system level optimizations that can be done that significantly impact performance, and the ability to pare down all of the unneeded stuff in most distributions would make for a pretty compact and highly optimized image.

Recommends hosting postgres yourself. Doesn't recommend a distribution stack. If you try this at a startup to save $50 a month, you will never recoup the time you wasted setting it up. We pay dedicated managed services for these things so we can make products on top of them.