That’s not how it works. You are probably adding two locales instead of two keyboards under a single locale. Yes, it’s confusing, I know.
The shortcuts next to the menu items in LibreOffice apps show me French translations, e.g. "Ctrl+Maj+Espace" for "Ctrl+Shift+Space" even though my language is set to Dutch everywhere (and other texts in LibreOffice are properly translated to Dutch). Apparently it has to do with me using Azerty keyboard layout. I use Belgian Azerty, not French Azerty. Yes, it's confusing.
It is on Windows 10 which I run at home. On Windows 11 which I use at work they changed it to the way you say.
But even then it didn't work to simply have English locale and Norwegian and US keyboard layouts under it. I can't recall what they messed up right now, but I fought Windows 11 for quite some time.
Finally settled on English locale with US keyboard layout, and Norwegian locale with Norwegian keyboard layout, which mostly works in terms of keyboard layout, but now my Weather app is showing Fahrenheit instead of Celsius, despite my regional settings being Norwegian.
Like, how hard can it be to just not fuck it up? They had it working fine for decades!