> A lot of sites have terms and conditions which explicitly disallow the use of site content as a part of another service.

Having terms and conditions in itself is irrelevant. Because in order for them to have any legal meaning, it is necessary for the other party to agree to them.

An agreement can be implicitly enforced by law. Or explicitly enforced by the website itself before giving access to the data. If neither of those are present, there is no enforced agreement. And agreeing to it becomes optional. Such sites should be considered, in my opinion, publicly accessible.

> If I have a bike and you start renting it out without my permission, surely you are committing theft of some sort.

Of course. But that is a bad analogy. No one is "renting" or "taking" anything from those websites. The bots are just reading it.

Therefore, a better analogy would be that you have a bike, parked out in the public, and people are looking at it. By looking at it they steal nothing from you. And the bike and all of its parts remain yours at all times. That is a suitable analogy, in my opinion, to what those bots are doing.