In my experience, as someone who has gone through this as maintainer of two decent sized projects, that simply doesn't work.
The author of the 'port' probably doesn't know your whole codebase like you, so they are going to need help to get their code polished and merged.
For endian issues, the bugs are often subtle and can occur in strange places (it's hard to grep for 'someone somewhere made an endian assumption'), so you often get dragged into debugging.
Now let's imagine we get everything working, CI set up, I make a PR which breaks the big-endian build. My options are:
1) Start fixing endian bugs myself -- I have other stuff to do!
2) Wait for my 'endian maintainer' to find and fix the bug -- might take weeks, they have other stuff to do!
3) Just disable the endian tests in CI, eventually someone will come complain, maybe a debian packager.
At the end of the day I have finite hours on this earth, and there are just so few big endian users -- I often think there are more packagers who want to make software work on their machine in a kind of 'pokemon-style gotta catch em all', than actual users.