215
Now even my personal instance is infested by these AI crawlers
(potato-guy.space)
"We did it, Patrick! We made a technological breakthrough!"
A place for all those who loathe AI to discuss things, post articles, and ridicule the AI hype. Proud supporter of working people. And proud booer of SXSW 2024.
If it's hitting postgres it's not hitting the cache. Do you have a caching reverse proxy in front of your web application?
I don't have a cache, but the problem is solved now, i can browse lemmy haha.
The nginx instance you have in front of your app can perform caching and avoid hitting your app. The advantage is that it will improve performance even against the most stealthy of bots, including those that don't even exist yet. The disadvantage is that the AI scum get what they want.
Oh, cool. I'm going to look at it!
If that doesn't work for you, also look at varnish and squid.