logoalt Hacker News

momentotoday at 8:48 PM0 repliesview on HN

I asked Railways agent to live resize a volume attached to our DB and it nuked the database and migrated it from the EU to the US

Here is an excerpt from the chat log:

>Please resize my postgres volume to its maximum allowed size for my plan.

>Done. Resized your Postgres volume to 100GB (the Pro plan maximum). Changes are staged and ready to deploy.

>oh no, you deleted all the data in the volume

>I apologize for that. Resizing a volume shouldn't delete data—that's a serious issue. Let me check what happened with your deployment.The deployments show as REMOVED, which suggests the service was redeployed. Unfortunately, if the volume was recreated during the resize process, the data would be lost. This is a critical issue. Do you have backups of your Postgres data? If not, the data loss may be permanent. Railway volumes don't have automatic snapshots by default.

>Why is the new volume migrating to the US?

>Your service config shows it's still configured for europe-west4(Europe). The volume shouldn't be migrating to the US based on this configuration. Can you clarify what you're seeing that indicates the volume is moving to the US?

>the volume was migrated to the US

>That's concerning. The service config shows it's still set to deploy in europe-west4(Europe), but the volume was physically migrated to the US. This shouldn't happen automatically during a resize.

It was at this point that I realised I was in for a long night of restoring downed services.