> I’m not justifying the design but splitting a table with several billion rows is not a trivial task, especially when ORMs and such are involved.
I don't agree. Let me walk you through the process.
- create the new table - follow a basic parallel writes strategy -- update your database consumers to write to the new table without reading from it -- run a batch job to populate the new table with data from the old table -- update your database consumer to read from the new table while writing to both old and new tables
From this point onward, just pick a convenient moment to stop writing to the old database and call the migration done. Do post-migrarion cleanup tasks.
> Additionally, it’s easier to get work scheduled to ship a feature than it is to convince the relevant players to complete the swing.
The ease of piling up technical debt is not a justification to keep broken systems and designs. It's only ok to make a messs to deliver things because you're expected to clean after yourself afterwards.
I've done this sort of thing or worked with people doing it. The concept is simple, actually executing can take months.