These scaling laws are even funnier than that, as are more recent ones.

I continue to be extremely amused by the fact that you can take a LLM 'reasoning' trace, replace all the 'reasoning' with dashes or spaces, re-run the attention mechanism, and have what comes out the other end improve by a significant fraction as much as if you let it exude text that has the appearance of having to do with the topic at hand. It's just more opportunity for the embedding vectors to dance against each other before you translate the abstract vectors back into natural language. Hence recent frontier models doing a thing where instead of doing a long reasoning trace, they build the attention matrix, allow the vectors to push each other around... then let those new vectors push each other around again, and again, and again, multiple rounds, before you start decoding.

More recently, I have seen some vague math into the scaling of the speed with which running many parallel systems with access to a common pool of text produce a legible result. If I recall, it comes to roughly:

N parallel systems

Finish in ~1/N^0.5 as much time

At ~N^0.5x the cost

So 16 parallel systems crosstalking finish complex tasks in 1/4 the time at 4x the cost (and 16x the instantaneous rate of operation per second)

I have wondered for a long time what sorts of clandestine messages are going over the newly installed Starlink terminals, bypassing the normal firewalls and logs between sensitive systems and the terrestrial internet backbone...

But they really aren't! The actually interesting scaling laws from about 5-7 years ago that opened up machine learning to more applications all show that everything about machine learning is power-law diminishing returns with linear improvements in any metric requiring exponential increases in numbers of parameters / amounts of data / number of backprop cycles applied.

I have had to explain to my boss that METR are cultists

For years the LW types have just been saying 'i asked claude' about EVERYTHING

They'll just skip straight to the Dyson Sphere, dontchyaknow

Any word on if Yarvin is running to Thiel's south american mansion yet?

Yeah but imagine the crazy bad takes some of our friends could be capable of having about it!

https://www.wired.com/story/let-the-internet-die/

Another entry in the weird "GPU world" writing competition (What kind of wonderful happy go lucky future is brought about by LLMs like those of today being ubiquitous and constantly used but unaccountably not leading to the Great Singularity)

https://www.lesswrong.com/posts/siTsb48Wr5vWDGxLd/gpu-world-submission

This one is less messed up than the last. Instead of weird sex and graverobbing it features... uh, a guy being paid by an LLM to braid string as busywork?

Does he dance with the devil In The Pale Moonlight?

Will it crash if the white house gets de-Baghdad-ified?