As far as I know, recovery mode doesn't let you do anything without either successfully authenticating or wiping the entire disk. An evil maid could theoretically compromise recovery mode at the login screen (with a USB device or something) but if they were able to do that, they could probably disable SSV themselves even if you hadn't done so. Therefore, disabling SSV shouldn't create additional risk involving recovery mode.

If the evil maid could boot macOS from an external disk, on the other hand, that would definitely be a problem. I think you need to authenticate in order to boot from an external disk for the first time (cf. [1]) but I'm not sure how this works.

[1] https://eclecticlight.co/2023/03/15/ownership-of-apple-silic...

Edit: Actually I guess an attacker trying to disable SSV themselves (via exploit of recovery mode) wouldn’t have the machine owner key needed to sign the new LocalPolicy. But could they reset it and still keep the data somehow? I don’t know.