Average human threat perceptions simply aren't useful here. People will also make wild assumptions about what kind of catastrophic thing could happen in aviation and then happily enter their car to drive somewhere without a thought in the world. In fact noone thought about designing gasoline fuel tanks in a safe way before we had cars. Not even really until people started burning. If we're already thinking about transporting antimatter safely today, this kind of technology will probably have an even better track record than planes.

Antimatter reactions are about a million times more powerful than conventional combustion. They surpass even nuclear explosions in energy release. That means even a small mishap becomes a large mishap.

Except rocket fuel lines are often leaking, and the most common cause of launch delays.

With antimatter the tiniest leak will annihilate your ship.