It will never make sense to me why KVMs are such a hard problem to solve. It seems like something we should have a good answer for by now but we still really just don't without dropping hundreds of dollars, and even then it still feels like a crap shoot.
They aren't a hard problem to solve. In the server market it's completely solved with a BMC. The problem being solved here is someone wants to make a product using some commodity product like a raspberry pi, perform video capture on a VGA/HMDI/DP port. This is not actually a problem end users have.
If you want to plug into a system that isn't server class, then they should be producing a video card that hooks into the USB bus, the always on rail of the power supply, the power switch pin of the power supply, and has an RJ45 jack. The contents of the card should be an off the shelf BMC chip.
But realistically if you want this kind of functionality, just buy server class systems that come with it.
It’s bizarre that one needs to do all this via a keyboard/monitor interface.
Nearly thirty years ago when I used to work on Silicon Graphics kit everything from powering on and off a server to installing the OS from scratch over the network could be done out of band via serial - and automatically, using expect(1)
That’s progress, I guess.