Not that I’m an Amazon apologist, but Perplexity has been accused of ignoring robots.txt quite a bit. It’s the “AI” app I’ve used most as I like the citations and find it does the most accurate data analysis (especially for my bored baseball stat searches), but I’m starting to question their integrity more than the usual tech company.
There isn’t a single AI company that abides by robots.txt. They don’t give a fuck and will happily ddos your site because hey, watcha gonna do, sue them?
Not that I’m an Amazon apologist, but Perplexity has been accused of ignoring robots.txt quite a bit. It’s the “AI” app I’ve used most as I like the citations and find it does the most accurate data analysis (especially for my bored baseball stat searches), but I’m starting to question their integrity more than the usual tech company.
There isn’t a single AI company that abides by robots.txt. They don’t give a fuck and will happily ddos your site because hey, watcha gonna do, sue them?
Robots.txt has always seemed more like a treasure map than a Keep Out sign.