For open implementations, look at ffmpeg.wasm - it's FFmpeg compiled to WebAssembly and supports a wide range of codecs. It's open source and actively maintained.
Some truly open/royalty-free codecs you could use - video: VP8, VP9, AV1. audio: Opus, Vorbis, FLAC.
That said, building a VLC in the browser gets complicated quickly because of licensing - even if the decoder implementation is open source, some codecs have patent licensing requirements depending on jurisdiction and use case. For example, H.264's basic patents have mostly expired, but I'd verify the specific profiles you need.