22
you are viewing a single comment's thread
view the rest of the comments
view the rest of the comments
this post was submitted on 05 Jul 2026
22 points (100.0% liked)
TechTakes
2622 readers
71 users here now
Big brain tech dude got yet another clueless take over at HackerNews etc? Here's the place to vent. Orange site, VC foolishness, all welcome.
This is not debate club. Unless it’s amusing debate.
For actually-good tech, you want our NotAwfulTech community
founded 3 years ago
MODERATORS
I studied transformer architecture models and have played around with them (unfortunately) enough to understand how they work. Under the surface the model produces what look like XML tags
<thinking> </thinking>to designate which tokens are thinking tokens and which are "normal" output. That is literally the only hard difference between the two output modes. The reinforcement learning might tune the thinking to be more like "what a human would expect to see in a thinking block" but it's still the same RNG madlib process generating everything underneath and any attempt to ascribe intelligence to this process should be met with ~~lethal force~~ incredulous cynicism.Just like any claim that "we don't know how they work" - actually yes we know exactly how they work. What we can't comprehend is the exact numbers and weights inside the massive pile of probabilistic algebra being processed to generate your slop. If I flip 5 coins in a row and the observer's belief is anything other than "you just got very lucky" most people would call them crazy rather than join the cult and worship the coin god...
God I remember having to explain to dozens of people that 'reasoning' models just exude a lot of text 'talking to themselves' and then summarize it. They were all just "It CANT be that silly" and many outright would not believe me, because that was not 'reasoning'
Sometimes they also glue a bunch of "tools" to help the model reason. The model can call by extruding tokens with the right syntax and then get back information from the "tool" shoved in its context! That way, the model can at least handle stuff like basic arithmetic correctly! Except only sometimes, because the models frequently screw up calling the tools correctly or skip using the tool or any number of other completely dumb mistakes. Oh, and if the tool connects the model to the internet (or any insecure source of text) in any way, shape, or form, congratulations, you've now got a massive security vulnerability!