I even found a way to make ad-hoc union types of element types from other packages that does exhaustive switch/case checking. I quickly wrote down a PoC so I wouldn't forget[0]. It needs wrapper types and sealed interfaces in the consuming app/package but is manageable and turned out better than other attempts I'd made.

[0] https://github.com/karmakaze/AdHocUnion