41
Trusting your own judgement on AI is a huge risk (www.baldurbjarnason.com)
submitted 1 week ago by [email protected] to c/[email protected]
5 comments fedilink hide all child comments

The article explains the general problem on the example of software development. But given that AI models are heavily promoted by billion-dollar US companies and important actors in that space are not at all friendly to the European Union, I think the relevance can be far larger.

Generally, the article explains that judging usefulness of AI models, specifically LLMs, by trying them out is very prone to the same psychlological traps like astrology, tarot cards or psychics - the so-called Barnum effect. This is specifically because these models are carefully engineered to produce plausible-sounding andwers! And even very intelligent but unaware people can easily fall prey to it.

top 5 comments
sorted by: hot top new old
[-] [email protected] 21 points 1 week ago* (last edited 1 week ago)

What called my attention is that assessments of AI are becoming polarized and somewhat a matter of belief.

Some people firmly believe LLMs are helpful. But tasks lile programming are logical tasks and LLMs absolutely can't think - only generate statistically plausible patterns.

The author of the article explains that this creates the same psychological hazards like astrology or tarot cards, psychological traps that have been exploited by psychics for centuries - and even very intelligent people can fall prey to these.

Finally what should cause alarm is that on top that LLMs can't think, but people behave as if they do, there is no objective scientifically sound examination whether AI models can create any working software faster. Given that there are multi-billion dollar investments, and there was more than enough time to carry through controlled experiments, this should raise loud alarm bells.

[-] [email protected] 11 points 1 week ago* (last edited 1 week ago)

Here's a big important test you can use to see if something is actually useful and effective.

Pick a random tool and ask all your friends and neighbors what they last used it for. "Hey bob, I was wondering, what was the last thing you used your belt sander/hammer/paintbrush for?". You'll probably get a very accurate answer about something that needed doing. "Oh, I had to sand down the windowsill because the paint was cracked" or "I tightened the screws on my coffeetable"

Now do the same for AI.

The big problem with asking if AI is useful is that people suck at figuring out how to do someone else's work, but they've got a pretty good idea what their own work is like. As a result, it's very easy to think that AI can do someone else's job, but for YOUR job, that you actually understand, you can easily see what bullshit AI spouts and how it misses all the important bits.

Sure, if your idea is that "Programmers write code", then yeah, AI can do that. Similarly, "authors write stories" is true, and AI can write stories. But if you know very slightly more, you realize that programmers only write code like 10% of the time, and authors probably write words less than 10% of the time. The job is about structuring and planning and laying out, the typing is just the final details.

But if you understand fuckall about a job, then yeah AI can definitely generate stuff that looks like other stuff, because it's a machine specifically designed to make stuff that looks like other stuff.

[-] [email protected] 3 points 1 week ago

Is actually have a good answer but in my case it'd be "I wanted to know what that one plant I saw was". AI-based pattern matching to identify plant or animal species is pretty handy.

It's also way more sensible than trying to use text generation for anything useful.

[-] [email protected] -1 points 1 week ago

Fair, I was mostly talking about LLMs and other generative AI.

[-] [email protected] -1 points 1 week ago

This is a mischaracterization of how AI is used for coding and how it can lead to job loss. The use case is not "have the AI develop apps entirely on its own" it's "allow one programmer to do the work of 3 programmers by using AI to write or review portions of code" and "allow people with technical knowledge who are not skilled programmers to write code that's good enough without the need for dedicated programmers." Some companies are trying to do the first one, but almost everyone is doing the second one, and it actually works. That's how AI leads to job loss. A team of 3 programmers can do what used to take a team of 10 or so on.

this post was submitted on 10 Jun 2025
41 points (100.0% liked)

Europe

6360 readers
817 users here now

News and information from Europe 🇪🇺

(Current banner: La Mancha, Spain. Feel free to post submissions for banner images.)

Rules (2024-08-30)

  1. This is an English-language community. Comments should be in English. Posts can link to non-English news sources when providing a full-text translation in the post description. Automated translations are fine, as long as they don't overly distort the content.
  2. No links to misinformation or commercial advertising. When you post outdated/historic articles, add the year of publication to the post title. Infographics must include a source and a year of creation; if possible, also provide a link to the source.
  3. Be kind to each other, and argue in good faith. Don't post direct insults nor disrespectful and condescending comments. Don't troll nor incite hatred. Don't look for novel argumentation strategies at Wikipedia's List of fallacies.
  4. No bigotry, sexism, racism, antisemitism, islamophobia, dehumanization of minorities, or glorification of National Socialism. We follow German law; don't question the statehood of Israel.
  5. Be the signal, not the noise: Strive to post insightful comments. Add "/s" when you're being sarcastic (and don't use it to break rule no. 3).
  6. If you link to paywalled information, please provide also a link to a freely available archived version. Alternatively, try to find a different source.
  7. Light-hearted content, memes, and posts about your European everyday belong in [email protected]. (They're cool, you should subscribe there too!)
  8. Don't evade bans. If we notice ban evasion, that will result in a permanent ban for all the accounts we can associate with you.
  9. No posts linking to speculative reporting about ongoing events with unclear backgrounds. Please wait at least 12 hours. (E.g., do not post breathless reporting on an ongoing terror attack.)
  10. Always provide context with posts: Don't post uncontextualized images or videos, and don't start discussions without giving some context first.

(This list may get expanded as necessary.)

Posts that link to the following sources will be removed

Unless they're the only sources, please also avoid The Sun, Daily Mail, any "thinktank" type organization, and non-Lemmy social media. Don't link to Twitter directly, instead use xcancel.com. For Reddit, use old:reddit:com

(Lists may get expanded as necessary.)

Ban lengths, etc.

We will use some leeway to decide whether to remove a comment.

If need be, there are also bans: 3 days for lighter offenses, 7 or 14 days for bigger offenses, and permanent bans for people who don't show any willingness to participate productively. If we think the ban reason is obvious, we may not specifically write to you.

If you want to protest a removal or ban, feel free to write privately to the primary mod account @[email protected]

founded 11 months ago
MODERATORS
[email protected]
[email protected]
[email protected]
[email protected]