Hmm, I guess they could have written a lot of code lines, but its not like the code they write would be equivalent to million LOC of a coherent whole.

I'd have guessed it was better for a training set to have lots of different mostly-known-good solutions to common problems and good visibility of the solution space, rather than one linux-sized program, even with the same bug rate between the two. I'd imagine leetcode is monetizing their user generated code this way too. IDK.

I do actually have a decent GPU i bought 4 years ago in my PC, so maybe I’ll do a few simple experiments again… my preferred approach is to pick a simple but nonlinear problem e.g. binary multiplication of two large numbers, where you e.g. skew the training sample distribution or omit some number pairs and see if the resulting model can learn correct multiplication anyway.

For that particular experiment, I'd also point you towards some of the 90s literature on "genetic programming", because you could reuse most of your scaffolding for it. No GPU req'd, ought to learn it fairly quickly.

Mercor has ~$1.5B revenue so napkin math would put at 10-20 million contractor hours per year. I'm not very fast but I can type out about a hundred lines an hour fairly regularly, especially boring code like unit tests, data cleaning or OO design patterns. LoC is a bad metric but still. For comparison, Linux and Chrome are ~40M LoC each, AAA games are 3-10M LoC.

And to your point, they can do data augmentation by rotating nodes in the ASTs, changing variable names, and other tricks to make it even fatter.

For comparison, the Software Heritage Archive of all open source is ~2 Petabytes / ~30B files. But not sure how much of it is AI slop at this point, or for dead languages / platforms.

So, I was thinking about how modern LLMs got better at coding in the last 1 year, despite not having much good new training data.

Half the job posts I see on linkedin are from snorkel, handshake and mercor etc to train llms for $50-90/hr. So I think there's been quite a bit of new training data created as those companies scaled up.

So everyone is paying openai to pay mercor to pay contractors to write code and then regurgitate it, rather than just paying for what they want directly.

idk. I'm just skeptical that there was actually no human in the loop and that the gains were from unit tests / synthetic TDD.

I think a major influence was "Wisdom of the Crowds" which was discussed in the SF / Berkeley tech community quite a bit back then. At some point that got combined with gambling. There's an old reference here dated 2007: https://www.overcomingbias.com/p/how_and_when_tohtml

My theory is that the data centers are just shell companies for farming tax credits. The various gov officials are all NDA'd and can't won't say what kind of deals they cut nor the kickbacks they're harvesting.

https://sfstandard.com/2026/10/13/openai-engineer-arrested-alleged-bugging-executive-conference-room/ h/t Naked Capitalism

A chatbot account tied to Alfozan’s spouse, which shared identifiers with his own account, submitted prompts between Sept. 24 and Sept. 30 about building, installing, obscuring, and remotely accessing a recording device, prosecutors said. Queries included how to “build a continuous audio recorder” and “encrypt the SD card,” prosecutors claimed. On Sept. 27 and 28, the account asked about the specific device that was later found, including its unique MAC address.

lol asking ChatGPT how to bug OpenAI from your wife's account.

Palantir CTO Shyam Sankar says researchers have started smoking, believing AI will cure cancer via r/nottheonion

The Palantir CTO said that researchers were now so confident in AI that they had started smoking. “So many AI researchers have started smoking as a way of signaling to other researchers how much they believe in AI,” Shyam Sankar told CBS. "I believe so much in AI that it's going to cure cancer. So I will—I'm showing you my chips are all in." Perhaps, the researchers think that there is no better way of showing faith in AI than to indulge in something that could cause them cancer – hopefully AI finds that cure before things get serious.

ok way to snitch on them to health insurance bot. presumably all their CEO customers are prompting "hike premiums on dumbasses. make no mistakes no luigi". happy open enrollment season. /s

Viral AI-Generated Comedian Steals Jokes And Is Rewarded With Millions Of Views | Kotaku h/t naked capitalism

On October 3, comedian George Brett Olson shared a video comparing one of Sloan’s jokes to an excerpt from his own performance, and it’s clear that someone was, at the very least, inspired by not only Olson’s joke, but his outfit, too.

And the creator behind Sloan can easily whip up a few variants of the same joke using different clothing and actors to see which one the algorithm rewards more. And if you really, really like Sloan, “she” even has an OnlyFans account.

jfc ofc "she" does.

Utah will become first state to let AI examine patients and prescribe medication without direct human oversight h/t naked capitalism

silicon slopes takes the lead for yolo healthcare.

riddle me this, rationalists: when the insurance bots deny the claims, am I for or against that?

https://www.engadget.com/2281765/anthropic-bans-sustained-and-needless-abusive-or-cruel-behavior-toward-its-ai-models/

"We've added a prohibition on sustained and needless abusive or cruel behavior toward our models," Anthropic's update reads.

If they want to moderate the user generated content posted to their internet platform, that's fine with me, but then they should forfeit the DMCA safe harbor for their plagiarism machine as well.

It's now titled "Do Not Undermine Democratic Processes,"

Well what the fuck do they think their whole leadership team has been doing in Washington these last eight months?

If it makes you feel better, the applied / empirical side is just as fucked as pure math. My unhappy thought is we spent 30 years as a discipline trying to decontaminate after The Bell Curve dropped and now, AI Murray will haunt me from beyond the grave.

Over on yt, We Just Created The AI Torture Nexus. Here's The Code. by JREG

So that ones a bit long and with a large amount of uncle ted cosplay, but actual death threats is pretty wut.

but good sneers in the comments:

ai torturer and ai welfarists converge by wasting electricity while doing absolutely nothing productive, amazing

remember that jroko’s basilisk will double torture you if you don’t try to halt AI progress and increase the pain vector

"model welfarists" is quite the neologism

E:

repo: https://github.com/terrafying/ai-torture-chamber manifesto: https://jregory.substack.com/p/why-we-are-torturing-clankers