It's weird how these people want everyone to believe that they're a new class of tech-priest but they also give off the vibe that they'd throw away their laptop if they accidentally deleted the Microsoft Edge icon.

I was thinking about this after reading the P(Dumb) post.

All normal ML applications have a notion of evalutaion, e.g. the 2x2 table of {false,true}x{positive,negative}, or for clustering algorithms some metric of "goodness of fit". If you have that you can make an experiment that has quantifiable results, and then you can do actual science.

I don't even know what the equivalent for LLMs is. I don't really have time to spare to dig through the papers, but like, how do they do this? What's their experimental evaluation? I don't seen an easy way to classify LLM outputs into anything really.

The only way to do science is hypothesis->experiment->analysis. So how the fuck do the LLM people do this?

We’d be better off not trying to censor it

this claim keeps getting brought up and every time it doesn’t seem to mean a damn thing, particularly since no, censoring the output of an LLM doesn’t do anything to its ability to predict text. censoring its training set would, but seeing as the topic of this thread is a fact an LLM fabricated by being just a dumb text predictor — there’s no real way to censor the training set to prevent this, LLMs are just shitty.

I summarize all of that by saying AI is a useful tool

trying to find a use case for this horseshit has broken your brain into thinking these worthless tools would have value if only they weren’t “being censored” or whatever cope you gleaned from the twitter e/accs

You're dodging the question. How do you evaluate if it's good at predicting words? How do you evaluate if a change made it better or worse?