While I was writing the reply, I considered giving the absolutely exact same explanation, which I agree with, down to the example you gave.
Two examples in portuguese that always trip me up, and absolutely never used to before are (i) "voz" (voice) and "vós" (you); and (ii) "trás" (back) and "traz" (bring).
I also do a lot more code-switching.