You are correct. However, AVIF does not require this conversion. For lossless it supports an "identity" matrix coefficient, which means no conversion.
It isn't particularly efficient in terms of file size, but as others have said, lossless images within a web page is extremely niche.
It has YCgCo too, which should have the efficiency of YCbCr while being lossless.