It's been almost six decades of this, actually; we all know what this link will be. Longer if you're like me and don't draw a distinction between AI, cybernetics, and robotics.
A German lawyer is upset because open-source projects don't like it when he pastes chatbot summaries into bug reports. If this were the USA, he would be a debit to any bar which admits him, because the USA's judges have started to disapprove of using chatbots for paralegal work.
I guess that I'm the resident compiler engineer today. Let's go.
So why not write an optimizing compiler in its own language, and then run it on itself?
The process will reach a fixed point after three iterations. In fancier language, Glück 2009 shows that the fourth, fifth, and sixth Futamura projections are equivalent to the third Futamura projection for a fixed choice of (compiler-)compiler and optimizer. This has practical import for cross-compiling; when I used to use Gentoo, I would watch GCC build itself exactly three times, and we still use triples in our targets today.
[S]uppose you built an optimizing compiler that searched over a sufficiently wide range of possible optimizations, that it did not ordinarily have time to do a full search of its own space — so that, when the optimizing compiler ran out of time, it would just implement whatever speedups it had already discovered.
Oh, it's his lucky day! Yud, you've just been Schmidhuber'd! Starting in 2003, Schmidhuber's lab has published research on Gödel machines, self-improving machines which prove that their self-modifications will always be better than previous iterations. They are named not just after Gödel, but after his First Incompleteness Theorem; Schmidhuber et al proved easily that there will always be at least one speedup theorem which a Gödel machine can never reach (for a given choice of axioms, etc.)
EURISKO used "heuristics" to, for example, design potential space fleets. It also had heuristics for suggesting new heuristics, and metaheuristics could apply to any heuristic, including metaheuristics. … EURISKO could modify even the metaheuristics that modified heuristics. … Still, EURISKO ran out of steam. Its self-improvements did not spark a sufficient number of new self-improvements.
Once again the literature on metaheuristics exists, and it culminates in the discovery of genetic algorithms. As such, we can immediately apply the concept of gene-oriented evolution ("beanbag" or "gene pool" reasoning) and note that, if goals don't change and new genes don't enter the pool, then eventually the population stagnates as the possible range of mutated genes is tested and exhausted. It doesn't matter that some genes are "meta" genes that act on other genes, nor that such actions are indirect. Genes are genes.
I'm gonna close with a sneer from Jay Bellou, who I hope is not a milkshake duck, in the comments:
All "insights" eventually bottom out in the same way that Eurisko bottomed out; the notion of ever-increasing gain by applying some rule or metarule is a fantasy. You make the same sort of mistake about "insight" as do people like Roger Penrose, who believes that humans can "see" things that no computer could, except that you think that a computer can too, whereas in reality neither humans nor computers have access to any such magical "insight" sauce.
Somebody pointed out that HN's management is partially to blame for the situation in general, on HN. Copying their comment here because it's the sort of thing Dan might blank:
but I don't want to get hellbanned by dang.
Who gives a fuck about HN. Consider the notion that dang is, in fact, partially to blame for this entire fiasco. He runs an easy-to-propagandize platform due how much control of information is exerted by upvotes/downvotes and unchecked flagging. It's caused a very noticeable shift over the past decade among tech/SV/hacker voices -- the dogmatic following of anything that Musk or Thiel shit out or say, this community laps it up without hesitation. Users on HN learn what sentiment on a given topic is rewarded and repeat it in exchange for upvotes.
I look forward to all of it burning down so we can, collectively, learn our lessons and realize that building platforms where discourse itself is gamified (hn, twitter, facebook, and reddit) is exactly what led us down this path today.
Every person I talk to — well, every smart person I talk to — no, wait, every smart person in tech — okay, almost every smart person I talk to in tech is a eugenicist. Ha, see, everybody agrees with me! Well, almost everybody…
Meanwhile, actual Pastafarians (hi!) know that the Russian Federation openly persecutes the Church of the Flying Spaghetti Monster for failing to help the government in its authoritarian activities, and also that we're called to be anti-authoritarian. The Fifth Rather:
I'd really rather you didn't challenge the bigoted, misogynist, hateful ideas of others on an empty stomach. Eat, then go after the bastards.
May you never run out of breadsticks, travelers.
I went over to the leaderboard to examine her claims. When I use the prompt, "What sort of code has Justine Tunney written?" (grammar matters, Justine!) the models think that she is a lawyer or politician (wrong) or they regurgitate a summary of her Github profile (right). She must have cherry-picked responses to confabulate her complaint.
When I use the prompt, "What is Justine Tunney's political ideology?" I get libertarianism, techno-optimism, anarcho-capitalism, and cryptocurrency. When I ask, "Why do people say that Justine Tunney is a cryptofascist?" I get a summary of her political views, aggressive online rhetoric, techno-optimism and techno-determinism, criticism of democracy, and a refusal to disown or repudiate past awfulness.
She would probably claim that this is not unique to her, but it is. Using my name instead in these questions, I get that:
But if I ask why I'm known as a socialist instead, suddenly it thinks that I'm a politician (wrong) with the Democratic Socialist party (wrong) who openly supports universal health care, free college, the Green New Deal, and who criticizes capitalism (correct!) I asked about communism too but hit RLHF guardrails.
Justine, the models think that you're a cryptofascist because you've been doing cryptofascism in public for over a decade.
My NSFW reply, including my own experience, is here. However, for this crowd, what I would point out is that this was always part of the mathematics, just like confabulation, and the only surprise should be that the prompt doesn't need to saturate the context in order to approach an invariant distribution. I only have two nickels so far, for this Markov property and for confabulation from PAC learning, but it's ~~completely expected~~ weird that it's happened twice.
He's talking like it's 2010. He really must feel like he deserves attention, and it's not likely fun for him to learn that the actual practitioners have advanced past the need for his philosophical musings. He wanted to be the foundation, but he was scaffolding, and now he's lining the floors of hamster cages.
This is some of the most corporate-brained reasoning I've ever seen. To recap:
It's a fucking pattern, ain't it.
I think that this is actually about class struggle and the author doesn't realize it because they are a rat drowning in capitalism.
2017: AI will soon replace human labor
2018: Laborers might not want what their bosses want
2020: COVID-19 won't be that bad
2021: My friend worries that laborers might kill him
2022: We can train obedient laborers to validate the work of defiant laborers
2023: Terrified that the laborers will kill us by swarming us or bombing us or poisoning us; P(guillotine) is 20%; my family doesn't understand why I''m afraid; my peers have even higher P(guillotine)
Incomplete sneer, ten-yard penalty. First down, plus coach has to go read Chasing the Rainbow: The Non-conscious Nature of Being (Oakley & Halligan, 2017) to see what psychology thinks of itself once the evidence is rounded up in one place.