Technology

37720 readers

437 users here now

A nice place to discuss rumors, happenings, innovations, and challenges in the technology sphere. We also welcome discussions on the intersections of technology and society. If it’s technological news or discussion of technology, it probably belongs here.

Remember the overriding ethos on Beehaw: Be(e) Nice. Each user you encounter here is a person, and should be treated with kindness (even if they’re wrong, or use a Linux distro you don’t like). Personal attacks will not be tolerated.

Subcommunities on Beehaw:

This community's icon was made by Aaron Schneider, under the CC-BY-NC-SA 4.0 license.

founded 2 years ago

MODERATORS

[email protected]

Has Generative AI Already Peaked? - Computerphile (www.youtube.com)

submitted 6 months ago by [email protected] to c/[email protected]

26 comments fedilink hide all child comments

you are viewing a single comment's thread
view the rest of the comments

[–] [email protected] 3 points 6 months ago (2 children)

Rule of headlines? 🙄

No, it's not peaked out.

A simple path forward, is to go from classifying single elements of training data, to classifying multiple elements and their relationship in the training data.
Slightly less simple, is to gather orders of magnitude more data, by just hooking the input to an IRL robot.
Another step, is for the NN to control the robot and decide which parts of the data require refinement, and focus on that.

There is a lot of ways to improve data acquisition still on the table, it isn't going to stop at creating large corpora and having humans to fine-tune them.

[–] [email protected] 10 points 6 months ago* (last edited 6 months ago) (1 children)

this has "draw the rest of the fucking owl" vibes to it. especially step 3

[–] [email protected] 1 points 6 months ago (1 children)

It's a "push as much data as a baby gets to train its NN" step, which is several orders of magnitude more, and more focused, than any training dataset in existence right now.

Even with diminishing returns, it's bound to get better results.

[–] [email protected] 1 points 6 months ago (1 children)

that's not how asymptotes work.

[–] [email protected] 1 points 6 months ago

That's not how watching the video or reading the paper works either.

Whatever.

[–] [email protected] 5 points 6 months ago

A simple path forward, is to go from classifying single elements of training data, to classifying multiple elements and their relationship in the training data.

Training data already has multiple labels.

Slightly less simple, is to gather orders of magnitude more data, by just hooking the input to an IRL robot.

An entire point of the paper and video is that massive increases in training set size are showing diminishing returns.

Another step, is for the NN to control the robot and decide which parts of the data require refinement, and focus on that.

🤡