This is in fact the main argument to me why swaps would never work at all, economically: the "state" of the battery is a significant part of the value of the car. Being swapped to a worse one makes you several thousand dollars worse off.
It only works in a leasing scenario, and everyone hates those.
I used to own a truck whose fair market value would double, temporarily, whenever I filled the tank.
How's that any different than putting new tires on a shitbox?
I think you'd need a contract where you buy (or lease) the car without a battery and lease/rent the battery separately. With some guaranteed offer for a battery with minimum spec X at maximum price Y when you exit the battery program. Preferably, not a mandatory offer, because one hopes specs go up and prices go down over time.
Then have you pay something per month for having a battery (maybe depends on the specific battery installed), something per kWh for charge used, plus a rebate per kWh for charge added. Or roll it into usage tiers, whatever.
There's lots of people that love leasing cars. I don't understand it, but it makes a lot of people happy?