(I get the feeling we're close to this bubble's end.)

Anyways, welcome to the Stubsack, your first port of call for learning fresh Awful you’ll near-instantly regret.

Any awful.systems sub may be subsneered in this subthread, techtakes or no.

If your sneer seems higher quality than you thought, feel free to cut’n’paste it into its own post — there’s no quota for posting and the bar really isn’t that high.

The post Xitter web has spawned soo many “esoteric” right wing freaks, but there’s no appropriate sneer-space for them. I’m talking redscare-ish, reality challenged “culture critics” who write about everything but understand nothing. I’m talking about reply-guys who make the same 6 tweets about the same 3 subjects. They’re inescapable at this point, yet I don’t see them mocked (as much as they should be)

Like, there was one dude a while back who insisted that women couldn’t be surgeons because they didn’t believe in the moon or in stars? I think each and every one of these guys is uniquely fucked up and if I can’t escape them, I would love to sneer at them.

(Credit and/or blame to David Gerard for starting this.)

you are viewing a single comment's thread
view the rest of the comments
[–] 13 points 2 days ago* (last edited 1 day ago) (11 children)

top quality sneer from “@1techuptopiapls” on X the fascism app:

”man proves AI improvements are not economically viable as new Anthropic model successfully cures cancer"

is the cure to cancer in the room with us right now

  • source
  • parent
  • hideshow 11 child comments
  • [–] 8 points 1 day ago (2 children)

    This guy probably thinks that outputting sloppily written proofs to math problems is the same thing as curing cancer because both of them are done by super smart scientific brain geniuses and the AI is an even bigger brain genius.

  • source
  • parent
  • hideshow 2 child comments
  • [–] 8 points 2 days ago (1 child)

    And that gets us back to istewart and Dave Karpf's point that these companies sell a narrative about what the future will be and who controls it, because they can not build that future.

  • source
  • parent
  • hideshow 1 child comment
  • [–] 6 points 1 day ago

    It’s just ‘well the financials don’t make any sense and these companies are not looking profitable anytime soon but like AI models are super duper good and AGI is right around the corner so you need to invest’

  • source
  • parent
  • [–] 2 points 1 day ago (5 children)

    A number of responses (like, 2 out of the 3 that it shows) refer to ‘the scaling laws’ without elaboration. Is this some jargon special to AI bros?

  • source
  • parent
  • hideshow 5 child comments
  • [–] 4 points 1 day ago (4 children)

    yeah essentially

    the scaling laws are basically that AGI/ASI can be created by just scaling up the current LLMs

  • source
  • parent
  • hideshow 4 child comments
  • [–] 9 points 1 day ago (3 children)

    But they really aren't! The actually interesting scaling laws from about 5-7 years ago that opened up machine learning to more applications all show that everything about machine learning is power-law diminishing returns with linear improvements in any metric requiring exponential increases in numbers of parameters / amounts of data / number of backprop cycles applied.

  • source
  • parent
  • hideshow 3 child comments
  • [–] 5 points 1 day ago* (1 child)

    The scaling laws maxed out around mid 2023 with the release of GPT-4 - that was around the time that everyone got excited about "test time inference" which is where LLMs spit out reasoning streams to convince you they are "thinking". If you look at where LLMs have improved since then it's mainly in tasks where they can be strapped into a harness and given concrete feedback like "your program didn't compile" which are also coincidentally use cases where data from gpt and Claude users can easily be harvested and new training data can be squeezed out of underpaid third world mturkers.

    In other words they've improved their training data in the specific domains they are operating in like we've been doing in machine learning since the mid 90s...

    The big labs have just been doing an incredible job of marketing (and particularly in Anthropic's case, brain damaging themselves into believing their own marketing)

  • source
  • parent
  • hideshow 1 child comment
  • [–] 3 points 1 day ago*

    These scaling laws are even funnier than that, as are more recent ones.

    I continue to be extremely amused by the fact that you can take a LLM 'reasoning' trace, replace all the 'reasoning' with dashes or spaces, re-run the attention mechanism, and have what comes out the other end improve by a significant fraction as much as if you let it exude text that has the appearance of having to do with the topic at hand. It's just more opportunity for the embedding vectors to dance against each other before you translate the abstract vectors back into natural language. Hence recent frontier models doing a thing where instead of doing a long reasoning trace, they build the attention matrix, allow the vectors to push each other around... then let those new vectors push each other around again, and again, and again, multiple rounds, before you start decoding.

    More recently, I have seen some vague math into the scaling of the speed with which running many parallel systems with access to a common pool of text produce a legible result. If I recall, it comes to roughly:

    N parallel systems

    Finish in ~1/N^0.5 as much time

    At ~N^0.5x the cost

    So 16 parallel systems crosstalking finish complex tasks in 1/4 the time at 4x the cost (and 16x the instantaneous rate of operation per second)

  • source
  • parent