David Gerard@awful.systemsM to

TechTakes@awful.systemsEnglish · 4 days ago

Fighting the AI scraper bots at Pivot to AI and RationalWiki

pivot-to-ai.com

6

34

Fighting the AI scraper bots at Pivot to AI and RationalWiki

pivot-to-ai.com

David Gerard@awful.systemsM to

TechTakes@awful.systemsEnglish · 4 days ago

6

We’ve covered the AI scraper bots before. These just hit web pages over and over, at high speed, to scrape new training data for LLMs. They’re an absolute plague across the whole World Wide Web and…

that s3kr1t method: https://www.jwz.org/blog/2025/05/user-agent-blocking/#comment-259266

Chat

jlow (he / him)@discuss.tchncs.de
link
fedilink
English
arrow-up
4·
3 days ago
Ohhhh, thanks for the mention of Wordfence. I’d love for Anubis to be available for Wordpress but I’ll take this in the meantime!

TechTakes@awful.systems

techtakes@awful.systems

You are not logged in. However you can subscribe from another Fediverse account, for example Lemmy or Mastodon. To do this, paste the following into the search field of your instance: [email protected]

Big brain tech dude got yet another clueless take over at HackerNews etc? Here’s the place to vent. Orange site, VC foolishness, all welcome.

This is not debate club. Unless it’s amusing debate.

For actually-good tech, you want our NotAwfulTech community

Visibility: Public

This community can be federated to other instances and be posted/commented in by their users.

122 users / day
450 users / week
2.38K users / month
5.39K users / 6 months
1 local subscriber
1.9K subscribers
741 Posts
20.4K Comments
Modlog

mods:
David Gerard@awful.systems