> but the rest of society (including non-drivers) pay through income taxes, sales taxes, and property taxes.

The rest of society also pays for the transportation costs of consumer goods, which include diesel and gas taxes which end up funding roads. Every time you buy something that rode on a truck, a fraction of the price you pay is road taxes.