134
AI’s recursive self-improvement might not come so quickly after all (www.technologyreview.com)
submitted 1 day ago* by to c/technology@lemmy.world
49 comments fedilink hide all child comments
you are viewing a single comment's thread
view the rest of the comments
[–] 13 points 1 day ago* (last edited 1 day ago) (9 children)

Lol what? Of course it won't, If the AI slop ends with recursive edits it's just going to cause degradation and collapse. I swear techbros have reality confused with their favorite fantasy fiction books.

EDIT: To demonstrate, 90% accuracy of 90% is 81%. Even the best most specific models on earth are not capable of self improvement because they will never reach much less exceed their training data's capability even if the largest most perfect dataset existed. They might think that by simply adding more layers of machines running in parallel and killing off models which underperform creating a system similar to evolutionary adaptation that it might eventually reach that 91%, but our current approach and level of technology have never demonstrated that capability not even theoretically.

  • permalink
  • fedilink
  • source
  • hideshow 9 child comments
    •
  • [+] -7 points 1 day ago (8 children)

    What happened in the computer programming space (with testable outputs) is that the first pass 80% accuracy nailed down an 80% success rate - wrote code that successfully met requirements 4/5 trials. Then, the agents were able to repeat the 1/5 failing trials with "sufficient heat" to both find their problems and create workable solutions, again 4/5 trials - so 80% success rate becomes 96% success rate, and so on... Back in early 2025, programming LLM agents would get themselves caught in iterative loops - trying, failing, trying again, failing again, then trying the first approach again - failing indefinitely. By mid 2026, I don't see that behavior anymore - if the first "light pass - quick attempt" solution doesn't succeed, they dig in deeper - do more research specifically focused on the problem areas identified in the first failure and try again, generally successful by the 2nd try, almost always by the 3rd - I haven't had to break a "trying the first unworkable solution again because I can't think of anything else to do" loop in over 6 months.

    Not all problem spaces are as clear-cut as software creation, but many have similar rules that just take a bit more training to learn.

  • permalink
  • fedilink
  • source
  • parent
  • hideshow 8 child comments
    •
  • [–] -1 points 1 day ago (5 children)

    You're entirely right. This won't replicate to other fields like writing and creative arts in general, but software engineering is just not that hard and can basically be brute forced with a good harness.

    It's a done deal and there is no world where people will write professional code by hand. I like it cause it really separates coding (the job) from coding (the art form). People will code by hand for aesthetic reasons just like people learn the violin instead of using a synth and we'll have a generation of lovingly crafted stuff. But boring software will be entirely automated, if not generated on the fly based on immediate needs.

  • permalink
  • fedilink
  • source
  • parent
  • hideshow 5 child comments
    •
  • [–] 2 points 1 day ago (4 children)

    software engineering is just not that hard

    I'll disagree on semantics here, it's precisely because software engineering is hard (not difficult, but rigid - objective) that makes it a good fit for LLM agent execution. Soft, squishy, ill-defined fields are going to be a worse fit for LLMs because the practitioners themselves can't create clear cut (hard) definitions of what it is they expect out of their practitioners, they just "know it when they see it." As for relative difficulty, the "soft" fields have a very sliding scale for that with a lot of allowance given to newbies that isn't accepted "at the highest levels" whereas, software engineering just is what it is, it doesn't get more difficult as you progress in the field. Your job as a software architect / engineer is actually to find the easiest workable solution(s).

    People will code by hand for aesthetic reasons just like people learn the violin instead of using a synth

    I think it's more like: people will code C or Rust or Python by hand just like people still code assembly by hand - exceptionally rare stubbornness with an exceptionally small audience who could even understand what they have done to begin to care about it. Violin vs synth - most of the world can listen and appreciate and have an opinion even if a vanishingly small fraction could ever hope to have the patience, let alone skill, to compose or perform at the highest levels of either form. "Synth" is a very broad target these days, varying from direct composition to performance digital transformation, through interfaces of every description and complexity: simple contact closure keyboards through multi-dimensional velocity, attack angle, strike momentum, and many dimensions of aftertouch bends which allow more expressivity than even bow and fingers on strings do, if the performer cares to train in that popularly scorned field. Having done a little amateur composition to performance vs performance capture synth work, I'll say: once you have trained to work with the complex input devices, capture of live performance is hundreds of times more efficient than specifying all the nuance of a real performance as notation in a composition. The main reason people hate synth performances is that most synth performances are hack level, because hack level is easier (read: possible) on synth than a minimally passable live performance on violin with strings and bow.

    Similarly, most people are hating on AI slop because it's so easy to produce and so many untalented hacks are using it to produce sub-par whatever it is they are making: code, prose, art, music... used as a tool, with a high bar of standards required before publication and release, LLMs are a powerful tool that can accelerate many creative processes, not just produce a lot of slop quickly.

  • permalink
  • fedilink
  • source
  • parent
  • hideshow 4 child comments
    •
  • [–] 1 point 23 hours ago (3 children)

    it’s precisely because software engineering is hard (not difficult, but rigid - objective) that makes it a good fit for LLM agent execution

    Yes i think we're actually in agreement here. I said "hard" (not difficult) as a reference to "hard problems", a term that comes from complexity theory but is now commonly used to describe problems which can't be reduced to an algorithm or evaluated objectively, and thus can't readily be "solved".

    Squishy subjects like music and sociology are full of hard problems, while solid subjects like math and coding are full of easy problems. Now the change introduced by LLMs is that as long as a problem is "easy", it can no longer be so laborious as to be impossible. Every software problem is solvable, modulo the effort/computing power you can spend on it.

    Your job as a software architect / engineer is actually to find the easiest workable solution(s).

    You also get bonus point if your solution is average (standard, unsurprising etc...), which makes it particularly soluble in LLMs which, by definition, can only produce output that is within the distribution of their training set.

    As for relative difficulty, the “soft” fields have a very sliding scale for that with a lot of allowance given to newbies that isn’t accepted “at the highest levels”

    That's not where i would put the difference. If you take a field like music, the problem is that it can't "just work". A nostalgic song may move the masses today but you can't say "okay we've solved nostalgia let's get to serenity next". Soon enough you'll need a new nostalgic song and by definition it will be out of distribution. You can't find it in a high dimensional representation of past music, and, well, you can't train on future data, so there is no way an LLM finds it and recognizes it for what it is.

    with a high bar of standards required before publication and release, LLMs are a powerful tool that can accelerate many creative processes

    I still believe they'll never amount to much regarding artistic processes, and not just for the reasons i already mentioned. To make something good you need to sit with it and walk with it and spend some time in it doing all the tedious little tasks until it really feels like home and you can express yourself in it. You can't achieve that if a machine speedruns all the little tasks for you.

  • permalink
  • fedilink
  • source
  • parent
  • hideshow 3 child comments
    •
  • [–] 1 point 22 hours ago (2 children)

    solid subjects like math and coding are full of easy problems.

    You are straying far afield from colloquial usage of the term "easy" - yes, solid subjects like math have solid problems: objectively verified if they have been solved or not. Yet, with 8 billion people on this planet and mathematical prizes ranging up to $1M and more, many of these problems you are calling "easy" have gone unsolved for decades - and a few of those are starting to be shown objectively solved via use of the new LLM tools...

    Squishy subjects like music and sociology are full of hard problems

    Again with the colloquial usage - squishy is a better term than "hard" - you can present very heavily referenced and logical and self consistent positions in subjects like sociology, psychology, literature etc. and "the powers that be" may simply refute your position as incorrect or irrelevant without presenting any concrete evidence or argument as to why other than "it does not conform to our (undefined, unexplained) standards of practice."

    On the other hand, a "darling of the field" may present a position and be instantly loved and accepted by the field, again for undefined and unexplained reasons - though rather transparently the true reasons often appear to be pedigree, likeability, stroking of the establishment's egos and pride... LLMs can be trained to do all those things, but the fact that they present in an LLM body will just as assuredly damn their viewpoint as if a black woman walked into a Southern University with a revolutionary new idea in the 1950s.

    LLMs which, by definition, can only produce output that is within the distribution of their training set.

    I'm fairly certain that's not a strict definition - LLMs also seem very heavily influencable via their context window inputs, though those are more limited and transient than the training set.

    there is no way an LLM finds it and recognizes it for what it is.

    Again, I believe if you study large populations of music listeners, tease out what drives their opinions, distill that, and feed it back into a LLM-like composition engine, the LLM approach may be able to produce both unique and paletable productions. Now, you may need a fleshy front-man/woman to appear to perform the composition, because that is part of the formula - relatability, the audience often likes to fantasize about being the performer(s) and that's not going to happen for a data-center.

    For comparison, a LOT of popular songs/music has been written by old Jewish men - but they don't perform their compositions because they're not personally appealing to the target audiences...

    You can’t achieve that if a machine speedruns all the little tasks for you.

    I agree, for a speedrun of a bunch of very common material. I knew a young singer, attractive, good voice, had a good backing band - she went fairly far in "America's got talent" but was knocked out in the last round before TV appearance by an apples/orange comparison - only one was going forward and the two weren't really directly comparable, her band lost. Never made sense to me until a few years later I went to Disney's Pleasure Island - they had an array of talented bands performing and as I walked from one to the next through the night, I saw one after another after another young female singers who were all just as attractive, talented, backed by just as good of a band... she was a commodity - as talented as you could ask, more talented than many national acts, but not unique.

  • permalink
  • fedilink
  • source
  • parent
  • hideshow 2 child comments
    •
  • [–] 1 point 22 hours ago (1 child)

    squishy is a better term than “hard”

    Eh, agree to disagree. My use of the phrase might be easier to parse if you think about the "hard problem of consciousness" as opposed to the "easy problem of consciousness". Also, more remotely, the notion of NP-hard vs NP-easy in computational complexity. I guess we are arguing about semantics aren't we ?

    I’m fairly certain that’s not a strict definition - LLMs also seem very heavily influencable via their context window inputs, though those are more limited and transient than the training set.

    While LLMs don't simply "statistically predict the next token" (common over-simplification which bears no relation to what is actually going on), the output they produce does exhibit the same statistical distribution as their training data. That's how you get intelligible language, code that compiles, chains of thought that makes sense etc... The context input will steer the output towards a certain subset of the corpus (highly optimized C functions have slightly different distribution than throw-away Python scripts), but it can never direct the model fully out of distribution as those out of distribution vectors cannot be expressed within the model's embedding space.

    she was a commodity - as talented as you could ask, more talented than many national acts, but not unique

    Yes exactly, if music was solvable then the most skilled people would systematically outclass the least skilled ones which is obviously not the case.

  • permalink
  • fedilink
  • source
  • parent
  • hideshow 1 child comment
    •
  • [–] 1 point 16 hours ago

    if music was solvable then the most skilled people would systematically...

    Music is like the other "squishy" problems, defined by gatekeepers who don't have fixed or knowable criteria for success - they "know it when they see it" and the target moves depending on their moods - inputs that range from the gatekeepers' own personal history through their recent experiences. Nobody can really "win" these fields without the gatekeepers permitting it to happen, and no matter how well man, woman or machine may think they know the gatekeepers' criteria, they can pivot on a whim.

    Then there's the age-old criteria of the casting couch - which LLMs won't be achieving any time soon, at least not without appealing sex-bot embodiments.

  • permalink
  • fedilink
  • source
  • parent
  • [–] 1 point 1 day ago* (1 child)

    They Don't pass 4/5.

    They pass 0/5 because they are 80% (that number is way too optimistic btw) accurate to human output on every one of the five attempts.

    They also can't be forced to learn and retake the trial because they don't have any contextual awareness, they just guess the next word in a sequence.

    Even if a machine made 4 self edits sucessfully, it would be permanently disfigured by the one failure and no longer be capable of making good edits.

  • permalink
  • fedilink
  • source
  • parent
  • hideshow 1 child comment
    •