lol. AI firms stop reading robots.txt
I think the idea is that all parties would find it beneficial so they would be incentivized to read it:
Leeds told Ars that the RSL standard doesn’t just benefit publishers, though. It also solves a problem for AI companies, which have complained in litigation over AI scraping that there is no effective way to license content across the web.
I imagine that when it comes to scraping data, the benefits of having the data far out away the risk of getting sued for scraping it.
Not to be That Guy, but with kindness I offer a small correction: “out away” -> “outweigh”.
Thank you, I was talking to the phone and it did not know what out we were weighing.
If nothing else it gets rid of one of the arguments they’re currently using in their defense at trial.
I hear you, and totally agree. I just remain a bit skeptical that something like this will have any effect as long as AI is making money hand over fist
If there is a way to combine this with something like anubis this could be very interesting, then you are also not dependent on the honesty of reading the robots.txt
Have a RSL license? here you go scrape the content.
Trying to freeload? anubis time.
