resipsaloquitur comments on Linus Torvalds shares his thoughts on Anti-AI coding rules

[–] resipsaloquitur@lemmy.cafe 1 point 10 hours ago (5 children)

Absolutely not. In a world where hallucinating machines are writing code, humans should be writing the tests at a bare minimum.

AI writing code to pass tests written by AI will get people killed.

permalink

fedilink

source

parent

hideshow 5 child comments

[–] MangoCats@feddit.it 1 point 9 hours ago (4 children)

Boeing managed to kill plenty of people without a single bit of help from AI / LLMs in their MAX 10 software.

Used properly, LLMs could have articulated why their approach would lead to multiple crashes.

Used improperly, any tool is dangerous, from a chainsaw to a plastic spoon, the potential for injuries and deaths are unlimited.

permalink

fedilink

source

parent

hideshow 4 child comments

[–] resipsaloquitur@lemmy.cafe 2 points 5 hours ago (1 child)

Boeing's executives had motivated reasoning to skimp on actual engineering and give the results the exec wanted over the objections of actual engineers.

Sound familiar?

Frankly, no. LLMs are so sychophantic, there's little evidence they would have flagged a warning that what he executives wanted was dangerous.

And no, LLMs are not just some tool. Never in human history has a hammer given it's users psychosis.

This is a completely nonsense argument.

permalink

fedilink

source

parent

hideshow 1 child comment

[–] MangoCats@feddit.it 0 points 5 hours ago

LLMs are so sychophantic

that they would have articulated the reasoning of the systems engineers who would have been lobbying for better human factors, training, and visibility of the potential problems. If the systems engineers hadn't been downsized that cycle, it's a here today gone tomorrow discipline at many companies.

Frankly, I think the Star Wars trope of humans disrespecting droids was prophetic. We're coming to a world where an LLM will spout off far more about a topic than you want to hear, and your attention just wanders off of them - rudely if they were an organic humanoid - without a further thought, until you want them to talk to the navicomputer or something else that they're suited for.

Never tell me the odds.

permalink

fedilink

source

parent

[–] unknownuserunknownlocation@kbin.earth 2 points 8 hours ago (1 child)

The max issue on the Boeing planes wasn't a software issue in that sense. The software actually worked as intended. Boeing just hid the fact that the software existed and made it purposefully hard to override, giving pilots quite the surprise when the plane didn't operate the way they expected, especially when the angle of attack sensor wasn't working.

permalink

fedilink

source

parent

hideshow 1 child comment

[–] MangoCats@feddit.it 1 point 8 hours ago* (last edited 5 hours ago)

There are all kinds of aspects to software development. Under-training pilots and then having software lead them to crash a plane full of passengers counts as a software development fail in my book. Also a human factors fail, marketing fail, a management fail, a quality systems fail, probably a regulatory oversight fail... it's up there with ignoring the guy telling the Space Shuttle mission control not to launch when it has been under 32F the night before, which wasn't a software fail at all, but was a broad cultural failure. Boeing's MAX-10 situation just happened to have software in the crux of the matter.

I do software development. When I see something, I say something. I'm frequently ignored. Several times over the years, I've had the people that ignored me come back asking for help fixing the issue, after they realized that it was more important than they thought it was when they initially ignored me. Luckily, none of the people who died in those situations died (or were harmed in any significant way) because of the problem, but people dying "near the problem" brought the seriousness of it to a level where it finally got addressed.

permalink

fedilink

source

parent