151

Another OpenAI researcher quits—claims AI labs are taking a ‘very risky gamble’ with humanity amid the race toward AGI (fortune.com)

submitted 5 months ago by [email protected] to c/[email protected]

46 comments fedilink hide all child comments

The comments come amid increased attention on a global AI race between the U.S. and China.

all 48 comments

sorted by: hot top new old

[-] [email protected] 46 points 5 months ago* (last edited 5 months ago)

the race toward AGI

lol

[+] [email protected] -8 points 5 months ago

Are we not heading towards AGI then?

[-] [email protected] 38 points 5 months ago

In the same way that if you start digging a hole in northwestern Spain you are heading towards New Zealand.

[-] [email protected] 1 points 5 months ago

That doesn’t sound right at all, comparing AGI to digging a hole from Spain to New Zealand is hyperbolic. Sounds like more like “electricity will never cover the whole world, maybe one day it’ll have an impact, but powering cars and homes? No way”. AGI and SGI is almost our only way to communism, with Deepseek and other open source models capitalists won’t be able to keep up, especially if AGI becomes available to your average person. In a few years, hell just one year alone, LLMs have made substantial progress that we can only assume it will continue to grow. Acting as if though AGI is like fusion generators is naive, unlike containing the sun, AGI is far more possible because it is. There’s no stopping it at this point, my professor told me that they have stopped trying to catch AI as a university because it’s impossible to do so now, unless you’re a child and just copy everything and it’s obvious. It’s time to stop assuming AGI will never come because it will, and it is.

[+] [email protected] -9 points 5 months ago* (last edited 5 months ago)

The difference here is that you're never going to reach New Zealand that way but incremental improvements in AI will eventually get you to AGI*

*Unless intelligence is substrate dependent and cannot be replicated in silica or that we destroy ourselves before we get there

[-] [email protected] 24 points 5 months ago

It's very easy with an incremental improvement tactic to get stuck in a local maximum. You've then hit a dead end, every available option leads to a degredation and thus isn't viable. It isn't a sure thing incremental improvements lead to the desired outcome.

[+] [email protected] -8 points 5 months ago* (last edited 5 months ago)

I simply cannot imagine a situation where we reach a local maximum and get stuck in it for the rest of human history. There's always someone else trying a new approach. We will not stop trying to improve our technology. Even just simply knowing what doesn't work is a step in the right direction.

We already know that General Intelligence is possible. The question that remains is wether it can be replicated artificially.

[-] [email protected] 8 points 5 months ago* (last edited 5 months ago)

I can imagine it really easily for the foreseeable future, all that would need to happen is for the big corporations and well funded researchers to stick to optimizing LLMs and for that to be a dead end.

Yeah that's not the rest of human history (unless the rest of it isn't very much) but enough to make concerns about AGI into someone else's problem.

(Edit, clarified)

[-] [email protected] -5 points 5 months ago

Like I said; I've made no claims about the timeline. All I've said is that incremental improvements will lead to us getting there eventually.

[-] [email protected] 7 points 5 months ago

In this scenario reaching the goal would require an entirely different base technology, and incremental improvements to what we have now do not eventually lead to AGI.

Kinda like incremental improvements to cars or even trains won't eventually get us to Mars.

[-] [email protected] -4 points 5 months ago

Firstly, I’ve been talking about improvements in AI technology broadly, not any specific subfield. Secondly, you can’t know that. While I doubt LLMs will directly lead to AGI, I wouldn’t claim this with absolute certainty - there’s always a chance they do, or at the very least, that they help us discover what the next step should be.

[-] [email protected] 3 points 5 months ago

It's true that I can't know for sure that they won't lead to AGI (or like you say give clues) - however it's definitely a scenario I can imagine, and that's what I was responding to: The idea that incremental improvements Must lead to a given goal. I don't think that's the case. Here in particular I think it's not only possible that it won't, it's even somewhat likely.

[-] [email protected] -3 points 5 months ago

This doesn't just apply to AGI, same could be said about any technology. If it can be created and there's value in creating it, then it'll just be a matter of time untill someone invents it unless we go extinct before that.

[-] [email protected] 2 points 5 months ago

It also applies to technologies that don't in fact exist but could. Those are much harder to name (besides sci-fi) since almost by definition we don't know about most of them. Nor how many, compared to existing tech.

I'm not actually saying it's impossible, just saying that local maximums (as described by the other users here) are a thing and it's possible to be trapped for a very long time by them. Potentially forever, but you're right that odds of breaking out do increase over time.

[-] [email protected] -3 points 5 months ago* (last edited 5 months ago)

Yeah, I agree with all of this. What I’m pushing back against is the absolute, dismissive tone some people take whenever the potential dangers of AGI are brought up. Once someone is at least willing to accept the likely reality that we’ll have AGI at some point, then we can move on to debating the timescale.

If an asteroid impact were predicted 100 years from now, at what point should we start taking steps to prevent it? Framing it this way makes it feel more urgent - at least to me.

[-] [email protected] 1 points 5 months ago

Just like incremental improvements in the bicycle will eventually allow for hypersonic peddling.

[-] [email protected] 2 points 5 months ago

By saying this aren’t you assuming that human civilization will last long enough to get there?

Look at the timeline of other species on this planet. Vast numbers of them are long extinct. They never evolved intelligence to our level. Only we did. Yet we know our intelligence is quite limited.

What took biology billions of years we’re attempting to do in a few generations (the project for AI began in the 1950s). Meanwhile the amount of non-renewable energy resources we’re consuming has hit exponential takeoff. Our political systems are straining and stretching to the breaking point.

And of course progress towards AI has not been steady with the project. There was an initial burst of success in the ‘50s followed by a long AI winter when researchers got stuck in a local maximum. It’s not at all clear to me that we haven’t entered a new local maximum with LLMs.

Do we even have a few more generations left to work on this?

[-] [email protected] -3 points 5 months ago

I'm talking about AI development broadly, not just LLMs.

I also listed human extinction as one of the two possible scenarios in which we never reach AGI, the other being that there's something unique about biological brains that cannot be replicated artificially.

[-] [email protected] 0 points 5 months ago* (last edited 5 months ago)

We could witness a collapse of our high tech civilization that effectively ends AI research without necessarily leading to extinction. Think of a global warming supercharged Mad Max post-apocalyptic future. People still survive but the population has crashed and there’s a lot of fighting for survival and scavenging among the ruins of civilization.

There’s gotta be countless other variations on this theme. Global dystopian techno-feudalism perhaps?

[-] [email protected] -2 points 5 months ago

Sure, but that's still just a speedbump. In a few hundred or thousand years the civilization would rebound and we'd continue from where we left.

[-] [email protected] 1 points 5 months ago

I don’t think there’s any guarantee that civilization would rebound. Fossil fuels were a one-shot deal in the geological history of the planet. For all of our efforts to build a sustainable future with renewable energy, fossil fuels remain critical for a lot of non-energy uses: food production (fertilizers), plastics, steel, and even cements for construction.

Another major issue is critical minerals for building renewable energy infrastructure. These minerals are being mined at an incredible rate, processed and turned into technology (think circuit boards full of components), aging out, and ending up as e-waste. Unfortunately our e-waste recycling infrastructure is a total nightmare involving the shipping of this stuff across the ocean to 3rd world countries where it gets picked over, scavenged for valuables, and the rest turned into toxic landfill.

All of that technology lifecycle creates huge amounts of toxic pollution and consumes huge amounts of fossil fuels (in particular for the mining, processing, and shipping). So in fact without fossil fuels we don’t even know how to build any technology, let alone renewable energy.

[-] [email protected] 16 points 5 months ago

AI in general yes. LLMs in particular, I very much doubt it.

[-] [email protected] 7 points 5 months ago

That assumes that whatever we have now is a precursor to AGI. There's no evidence of that.

[-] [email protected] 1 points 5 months ago

What do you mean there’s no evidence? This seems like a difference of personal explanation of what AGI is where you can move the goal post as much as you want “it’s not really AGI until it can ___, ok just because it can do that doesn’t mean it’s AGI, AGI needs to be able to do _____”.

[-] [email protected] -5 points 5 months ago

No, it doesn't assume that at all. This statement would've been true even before electricity was invented and AI was just an idea.

[-] [email protected] 7 points 5 months ago

It is a common misconception that incremental improvements must equate to eventually achieving the goal, but it is perfectly possible that progress could be asymptotic and we never reach AGI even with constant “advancements”

[-] [email protected] 6 points 5 months ago

Yeah not with LLMs though.

[-] [email protected] 20 points 5 months ago

Would we know it if we saw it? Draw two eye spots on a wooden spoon amd people will anthromorphise it. I suspect we'll have dozens of false starts and breathless announcements of AGI, but we may never get there.

More interestingly, would we want it if we got it? How long will its creators rally to its side if we throw yottabytes of data at our civilization-scale problems and the mavhine comes back with "build trains and eat the rich instead of cows?"

[-] [email protected] -3 points 5 months ago

Would we know it if we saw it?

That seems besides the point when the question is about wether we're getting closer to it or not.

[-] [email protected] 1 points 5 months ago

How can we ask "are we closer" if we don't know what the destination is?

LLMs might still end up being an interesting special-purpose system, perhaps with fairly broad applications, but in a direction that's different from where true AGI ends up coming from.

[-] [email protected] 40 points 5 months ago

The delusions of grandeur required to think your glorified auto complete is going to turn into a robot god is unreal. Just wish they'd quit boiling the planet.

[-] [email protected] 4 points 5 months ago

Oh man 100% this.

A little while ago there was a thread about what people are actually using LLMs for. The best answer was that it can be used to soften language in emails. FFS.

[-] [email protected] 2 points 5 months ago

I also wish they'd put up or shut up, goddamn lol. Hopefully DeepSeek has lit some fires under some asses 🍑🔥

[-] [email protected] 1 points 5 months ago

I love how Jon Stewart put it: AI is losing its job to AI.

[-] [email protected] 2 points 5 months ago

I was super disappointed with his take this week in general though (which I see is reflected in the YouTube comments).

Context: https://www.youtube.com/watch?v=Byg8VZdKK88

[-] [email protected] 2 points 5 months ago* (last edited 5 months ago)

alternatively, the delusions of grandeur required to think your opinion is more reliable than that of many of the leaders in the field

they’re not saying that LLM will be that thing; they’re saying that in the next 30 years, we could have a different kind of model - we already have the mixture of experts models, that that mirrors a lot of how our own brain processes information

once we get a model that is reliably able to improve itself (and that’s, again, not so different from adversarial training which we already do, and MLP to create and “join” the experts together) then things could take off very quickly

nobody is saying that LLMs will become AGI, but they’re saying that the core building blocks are theoretically there already, and it may only take a couple of break-throughs in how things are wired for a really fast explosion

[-] [email protected] -2 points 5 months ago

I know the infinite conversation has only gotten better and better! 🤪

[-] [email protected] 3 points 5 months ago

good decision, imo. sometimes i also get annoyed if the one i am watching becomes too sweaty completionists.

[-] [email protected] 3 points 5 months ago

I happily welcome our merciful and benevolent machine overlords.

[-] [email protected] 2 points 5 months ago

I’d take Skynet over what’s currently going on.

[-] [email protected] 1 points 5 months ago

I think about this. Boy to bad we don't have a general ai to run things given what we have gotten or maybe a nice interstellar race that got past the great filter can upload us and leave the planet to recover.

this post was submitted on 29 Jan 2025

151 points (94.7% liked)

Technology

3382 readers

903 users here now

Which posts fit here?

Anything that is at least tangentially connected to the technology, social media platforms, informational technologies and tech policy.

Post guidelines

[Opinion] prefix

Opinion (op-ed) articles must use [Opinion] prefix before the title.

Rules

1. English only

Title and associated content has to be in English.

2. Use original link

Post URL should be the original link to the article (even if paywalled) and archived copies left in the body. It allows avoiding duplicate posts when cross-posting.

3. Respectful communication

All communication has to be respectful of differing opinions, viewpoints, and experiences.

4. Inclusivity

Everyone is welcome here regardless of age, body size, visible or invisible disability, ethnicity, sex characteristics, gender identity and expression, education, socio-economic status, nationality, personal appearance, race, caste, color, religion, or sexual identity and orientation.

5. Ad hominem attacks

Any kind of personal attacks are expressly forbidden. If you can't argue your position without attacking a person's character, you already lost the argument.

6. Off-topic tangents

Stay on topic. Keep it relevant.

7. Instance rules may apply

If something is not covered by community rules, but are against lemmy.zip instance rules, they will be enforced.

Companion communities

[email protected]
[email protected]

Icon attribution | Banner attribution

If someone is interested in moderating this community, message @[email protected].

founded 2 years ago

MODERATORS

[email protected]