285
Aggressive AI scrapers are making it kinda suck to run wikis (weirdgloop.org)
submitted 3 months ago by to c/technology@lemmy.world
22 comments fedilink hide all child comments
you are viewing a single comment's thread
view the rest of the comments
[–] 1 point 3 months ago (1 child)

Probably can configure anubis to require challenge that is proportional to the CPU time needed to render each page?

  • permalink
  • fedilink
  • source
  • parent
  • hideshow 1 child comment
    •
  • [–] 1 point 3 months ago

    That seems to be a lot of peoples approach, but if they cared about time or bandwidth they wouldn't be spidering Dow into your commit history multiple times a day. They have more patience and resources than your human readers.

  • permalink
  • fedilink
  • source
  • parent