Egan already told Hanson to fuck off in a previous story, Zendegi:
"My IQ is one hundred and sixty ... You can always reach me through my blog, Overpowering Falsehood dot com, the number one site for rational thinking about the future —"
jwz gave the game away, so i'll reveal:
the One Weird Trick for this week is that the bots pretend to be an old version of Chrome. So you can block on useragent
so I blocked old Chrome from hitting the expensive mediawiki call on rationalwiki and took our load average from 35 (unusable) to 0.8 (schweeet)
caution! this also blocks the archive sites, which pretend to be old chrome. I refined it to only block the expensive query on mediawiki, vary as appropriate.
nginx code:
# block some bot UAs for complex requests
# nginx doesn't do nested if, so we set a test variable
# if $BOT is both Complex and Old, block as bot
set $BOT "";
if ($uri ~* (/w/index.php)) {
set $BOT "C"; }
if ($http_user_agent ~* (Chrome/[2-9])) {
set $BOT "${BOT}O";}
if ($http_user_agent ~* (Chrome/1[012])) {
set $BOT "${BOT}O";}
if ($http_user_agent ~* (Firefox/3)) {
set $BOT "${BOT}O";}
if ($http_user_agent ~* (MSIE)) {
set $BOT "${BOT}O";}
if ($BOT = "CO") {
return 503;}
you always return "503" not "403", because 403 says "fuck off" but the scrapers are used to seeing 503 from servers they've flattened.
I give this trick at least another week.
it will explain its position with interpretive twerking