Tech Cult Watch ?
What kind of activities are intended exactly, journalism, lobbying?
I'm not sure if this was highlighted already but from the OpenAI report on the Hugging Face incident:
The internal-only research model is comparable in scale to GPT-5.6 Sol and was trained to advance persistence and multiagent collaboration, among other capabilities. The model was not intended for public use, and was only used by a small number of OpenAI personnel for internal research.
Absolutely typical, the "unexpected communication" is hardly an emergent phenomenon, (I remember them trying to claim unexpected high success at translation tasks without specific training, which was also complete BS), like you can't act surprised that agents communicate and hack stuff if that what's you're instructing them to do.
Nobody who believes that LLMs cannot be conscious will ever believe that they can be conscious.
This is circular logic, I THINK what you're trying to say is that AI skeptics are dogmatic and/or axiomatic in their world view, but if thats true you're not engaging with any of the substance of what is being said.
And they will say things like the writer of this piece who says ‘we know how they work.’
The "writer of this piece" is not just some random blogger, but devotes a lot of his time covering the "scam" aspect of the current LLM hype-cycle, the title the website "pivot-to-ai" is in reference to the fact that a lot of the current AI peddlers, previously peddled crypto and blockchain as a deeply transformative technology that mere mortals cannot properly grasp.
Purposefully mystifying a topic, is more common as a scam tactic than you might imagine.
Anybody can do this - enthusiasts do it on their own desktops.
Minor note, but no, maybe some (with a fair amount of disposable income) can waste money on fine-tuning, training from scratch and the required large-scale theft, and large-scale compute time is out of reach of any hobbyist.
But what is happening that creates their emergent behaviors is a black box that we cannot see inside. We also don’t know what consciousness is and we don’t know what has it and what doesn’t other than that which our “feelies” dictate.
What can be called The Illusion of Consciousness is far more easily explained by human cognitive biases, than any reality of LLM functioning. It's easy to be fooled by an LLM which echoes words back to you, you put meaning into your own words, and you recognize them when they are said back to you.
In the deeper sense, any meaning present in LLM output, is the meaning that the reader puts there and/or the meaning that survives from the training data, it's not exactly that no meaning present, but it's not the result of machine cognition, often incoherent and/or very lacking in substance upon close inspection.
I think to some extent this is a lost cause, but your feelings/fears seem to boil down to "this could be conscious!" (emphasis on could) I would invite to see if any other explanation can explain what you observe, is it actually the ONLY or even the most plausible explanation?
Doubt many organizations have failed as badly at their goals as LW/EA has recently.
They are/were too weird for people to never notice.
The "safeguards" espoused by this crowd, always seemed to me to be more philosophical/spiritual than practical anyway, the most "orthodox" view is that no training is safe since the AI could discover magic and the cheat codes to the universe, so even air-gapping would be insufficient, in that view any participation in development must come with a hefty dose of cognitive dissonance.
On a more pragmatic note, I would note that even short of Intelligence, those incidents highlight those tools are still dangerous, and since they are unconstrained and ill-defined, can produce surprising outcomes especially if you throw lot's of inference time (money) at them.
There isn't really a universe where bots have both access to the internet (user input, both in training and live) and a collection of build/cli tools (which can run said untrusted user input), while being guaranteed to be harmless. It's like a grand automation of some of the worst engineering practices.
I find the "reasoning about the test environment" parts often included in write up of the hugging-face incident to be highly credulous, And I would be extremely surprised if not all the information was included in the task prompt (which also explains chosen usernames as "Open AI researcher #1" and so on.) It's the ELIZA/parrot thing, acting surprised "How did it guess?!?", when all of the info was provided.
I'm not entirely encouraged by the latest batch of sex-cult smearing (well mostly accurate) they are receiving at the WH, since it's probably just an expedient way of dismissing their voice, without addressing the larger issues of AI, hence probably in service of pushing forwards with AI.
I wonder if there's a market for RevivingYahoo^TM^, which in some ways was more link curation than search engine.
When searching for examples of commands, I've sometimes been resorting to telling google to restrict itself to pdf files of specific websites (example redhat, with sometimes useful commands for server maintenance broadly)
It's a bit sad that it's not just search that has gone downhill, but new documentation too (I swear Microsoft Azure documentation is getting worse).
The quirkiness of the Tranquility Calendar is mildly beautiful, between the second man on the moon getting snubbed with the every four year leap day (which for no good reason still happens around Feb 29!), [I guess Collins gets even more snubbed by not appearing at all] to the more diverse than expected list of honored "scientists/scholars", until you realize the main reason is just to keep the ABCD naming pattern for the months.
you ask a hunter and not a deer.
The sad lens into their prevalent relationship with "correcting biases" being more about dominance in "trickery" rather than becoming better people. (and also entrenched misogyny.)
I was thinking "Hey I want that markdown!" (although maybe not for time magazine), but the actual example is dystopian.
If you want to stare at the abyss it's achievable with curl -H 'user-agent: ClaudeBot' https://url_to_article, and it's like 70% SEO crap tailored to brainwash LLMs (more realistically "agents" in limited context).
At my place It has reached the stage of: use AI to "improve" your work, be careful about budget though!
I think he falls under the category of "does too much all the time" and "vaguely techo optimist" which definitely put him at risk. I'm not too sure how harshly I should judge him yet, I guess it depends on what he does next.
Congrats.
It continues to amaze me how much digital ink they can spill about this issue.
AGI Sparklings proponents rejoice! Finding a literal map(*) means LLMs have a world model.
(awful.systems)
Transcribed:
Max Tegmark (@tegmark):
No, LLM's aren't mere stochastic parrots: Llama-2 contains a detailed model of the world, quite literally! We even discover a "longitude neuron"Wes Gurnee (@wesg52):
Do language models have an internal world model? A sense of time? At multiple spatiotemporal scales?
In a new paper with @tegmark we provide evidence that they do by finding a literal map of the world inside the activations of Llama-2! [image with colorful dots on a map]
With this dastardly deliberate simplification of what it means to have a world model, we've been struck a mortal blow in our skepticism towards LLMs; we have no choice but to convert surely!
(*) Asterisk:
Not an actual literal map, what they really mean to say is that they've trained "linear probes" (it's own mini-model) on the activation layers, for a bunch of inputs, and minimizing loss for latitude and longitude (and/or time, blah blah).
And yes from the activations you can get a fuzzy distribution of lat,long on a map, and yes they've been able to isolated individual "neurons" that seem to correlate in activation with latitude and longitude. (frankly not being able to find one would have been surprising to me, this doesn't mean LLM's aren't just big statistical machines, in this case being trained with data containing literal lat,long tuples for cities in particular)
It's a neat visualization and result but it is sort of comically missing the point
Bonus sneers from @emilymbender:
- You know what's most striking about this graphic? It's not that mentions of people/cities/etc from different continents cluster together in terms of word co-occurrences. It's just how sparse the data from the Global South are. -- Also, no, that's not what "world model" means if you're talking about the relevance of world models to language understanding. (source)
- "We can overlay it on a map" != "world model" (source)
view more:
next ›