alt Hacker NewsI do. It's wrong to ignore a robots.txt.
The fact someone else may have licenced content for commercial use doesn't entitle someone else to do the same without paying for it. That's the whole point of copyright law.
I do. It's wrong to ignore a robots.txt.
The fact someone else may have licenced content for commercial use doesn't entitle someone else to do the same without paying for it. That's the whole point of copyright law.
> It's wrong to ignore a robots.txt.
It's not nice. It is, however, not illegal.
And before you agitate against that, think deeply about the consequences. A world where website owners can legally dictate exactly what software you use to access their content.