You're right, but it's slightly more complicated than that.
My understanding is that payment processors are obligated to follow the policies of Visa/MasterCard, AmEx, and Discover, but that those parties' policies don't explicitly ban these specific things for sale. Instead, they "strongly encourage" processors to ban them in their user agreements under the implicit threat of their risk level being increased, which in turn impacts the fees they pay to the credit card companies.
I've not been deep in this world since ~2014, but at that time the only processor I could find that wasn't specific to the porn industry, offered physical terminals, had reasonable (if high) fees, and didn't ban legal transactions in their user agreement was PAI ("Payment Alliance International"). A quick look at their site today shows that they seem to have been acquired by Brinks, so that may no longer be the case.
MasterCard have a specific restricted list that bans an awful lot of things in any adult context.
Some of how to interpret that is left up to the processor, but it is broadly under MCs and to a lesser extent Visa's control.