The requirement to actually pay will keep much of the riff-raff out. In my local bus system, you theoretically have to pay but the drivers are not going to throw you off the bus if you don't and so the buses all have a few homeless guys who just ride all day.
Are you assuming people share your prejudice? Who is 'riff-raff'? Maybe I think you are (and vice versa). I've never had a problem with someone who seems to be unhoused (I wouldn't know). I have had problems with people on their phones in big SUVs, or who just feel like being a*holes.
Don't know what town you live in, but here in Seattle, very few bus routes have homeless people who ride them all day.
The vast majority don't.
The reason transit in this city sucks (still head and shoulders above the vast majority of the US) isn't because there's 12,000+ homeless people living in it[1], it's because the buses don't run frequently enough and because all the fucking single-occupant car traffic turns what would be a 20 minute bus ride into a 40 minute slog, and because you'd be insane to bike for your last-mile.
---
[1] Increasing every year, and under the current mayor's tenure, we lost a net of 200 shelter beds.