▲ 422 ▼ Linus Torvalds used AI to fix a Linux bug, and now LLM critics on Linux face an uncomfortable choice (www.xda-developers.com) submitted 5 days ago by themachinestops@lemmy.dbzer0.com to c/technology@lemmy.world 496 comments fedilink hide all child comments https://lists.debian.org/debian-project/2026/08/msg00041.html
[–] AwesomeLowlander@sh.itjust.works 5 points 4 days ago (1 child) Let us know how your Linux 2.0 goes. permalink fedilink source parent hideshow 2 child comments replies: [+] Kangae_Hishiryo@scribe.disroot.org -9 points 4 days ago (1 child) If you don't have anything better to say than an ad hominem fallacy and a red herring fallacy, then you're totally discrediting yourself. permalink fedilink source parent hideshow 2 child comments replies: [–] AwesomeLowlander@sh.itjust.works 8 points 4 days ago (1 child) Yes, because your original comment was so thought out and well reasoned to begin with 🙄 permalink fedilink source parent hideshow 2 child comments replies: [–] Kangae_Hishiryo@scribe.disroot.org 1 point 3 days ago (2 children) Well, it's not that I'm telling another thing than the truth. You're attacking something totally unrelated and that I've never mentioned ("Linux 2.0") because you simply can't deny that LLM are bug-making machinegunss. Literature on that in my response. permalink fedilink source parent hideshow 4 child comments replies: [–] AwesomeLowlander@sh.itjust.works 1 point 3 days ago Do you think anybody, especially Linus, is unaware that LLMs produce bugs if left to themselves? He's made it clear that PRs will continue to be judged on their code quality as they have all this while. Your comment is especially ironic considering how the post is about an elusive bug that took 1 line to fix, but hours upon hours and 24 other patches to debug. Anybody with any knowledge of Linus would find the idea hilarious that he'd allow, let alone commit, crappy code into the kernel. The only real issue up for debate is the ethics of it, considering the provenance of most training data, and potentially copyright issues. permalink fedilink source parent [–] Kangae_Hishiryo@scribe.disroot.org -1 points 3 days ago* (1 child) Part 1: Code Quality (defects, bugs, logic) CodeRabbit: AI code has 1.7× more defects (10.83 vs 6.45 issues/PR); logic errors 1.75×, security 1.57×, XSS 2.74× BusinessWire / CodeRabbit: Performance inefficiencies (excessive I/O) appear ~8× more in AI code; logic problems up 75% The Register: AI code shows 1.4× more critical and 1.7× more major issues; 1.57× more security findings Carnegie Mellon (via Ox Security): Only 61% of AI-generated code functions correctly; only 10.5% passes security review Ranger 2026: 26.6% produce incorrect outputs; 60% of faults are silent logic failures; ~50% has maintenance problems GitClear 2026: Copy-paste code rose 9.4%→15.7%; duplication up ~4×; refactoring down 70%; code churn nearly doubled arXiv 2026: LLMs suffer "Context Rot"; accuracy drops sharply when relevant info is mid-context arXiv 2026: Copilot generates executable code ~90% of the time, but 40% of solutions on critical security tasks contained MITRE Top-25 CWEs arXiv 2026 "Should I Give Up Now?": LLMs "hallucinate, omit important steps, lose context between turns, or produce deceptive code" Based Info: "The problem is execution, not compilation" — code compiles cleanly but fails at runtime or solves the wrong problem CodeBridge / Ox Security: 10 recurring antipatterns in 80-100% of AI code; 68-73% contain vulnerabilities that pass unit tests but fail in production ResearchGate 2026: Valgrind revealed AI code left 1,068 bytes in 34 reachable blocks at exit (memory leaks) vs 24 bytes in 2 blocks for human code IEEE Spectrum 2026: AI code degrades as newer models create "silent failures" and rely on low-quality training data arXiv 2512.22387: 31.7% of AI-generated projects fail to execute at all (only 68.3% reproducible) arXiv 2510.26130: LLMs achieve 84-89% on synthetic benchmarks but only 25-34% on real-world code (66-75% functionally incorrect) SWE-bench 2026: Top models reach ~80%, most struggle below 20% on repository-level tasks (80%+ failure rate) SmartBear 2026: 70% of engineering leaders say quality has degraded; 60% report code outpacing testing capacity DeviQA 2026: 0% of senior QA (8+ yrs) said AI code has fewer bugs; 37.5% said noticeably more arXiv 2026: 304,362 AI commits → 110,000+ unresolved technical debt issues by Feb 2026 How I Dropped Our Production Database and Now Pay 10% More for AWS Claude Tested Everything Except the One Thing That Mattered (AI agent refuses to follow explicit instructions to test createPost() in increasingly erratic ways) Amazon calls engineers for a “deep dive” internal meeting to discuss “GenAI”-related outages GitClear has released reports in 2024 and 2025 indicating a worsening of key code quality metrics correlating with increased LLM adoption. Security & Vulnerabilities Veracode 2025/2026: 45% of AI-generated code introduces OWASP Top 10 vulnerabilities (100+ LLMs tested); Java 70%+ failure rate Cloud Security Alliance 2026: Privilege escalation paths +322%, architectural flaws +153%, despite syntax errors −76%; ~20% of samples reference nonexistent packages arXiv "Broken by Default" 2026: Mean vulnerability rate 55.8% (GPT-4o: 62.4%) across 3,500 artifacts verified with Z3 Sherlock Forensics 2026: 100% of AI-generated apps contained ≥1 critical vulnerability; 78% store secrets in plaintext; 34% of Node.js projects include hallucinated dependencies Georgetown CSET: 86% failed XSS defense, 88% vulnerable to log injection, 47% SQL injection across 5 LLMs Llama 3.3 SWE-bench study: 11× more new vulnerabilities in LLM patches vs developer patches arXiv 2603.10072: Only 24.8% of LLM security patches achieve full correctness; 51.4% fail BOTH security and functionality Stanford/MIT Mar 2026: 14.3% of AI code has ≥1 security vulnerability vs 9.1% human (2M snippets) Apiiro Fortune 50: AI-assisted devs produce 10× more security issues despite 3–4× more code ACM 2025/2026: ~30% of generated code snippets contain security weaknesses Pagerly 2026: Models produce compilable code almost always but secure code only 56% of the time Dev.to / State of Web Dev 2026: 63% of AI-generated functions had a security finding; review doesn't scale with volume Stanford (via SC World): In 80% of tasks, devs using AI produced less secure code; 3.5× more likely to believe their code was secure Georgia Tech Vibe Security Radar: 74 CVEs attributed to AI tools (Mar 2026); est. 400–700 real arXiv 2026: Slopsquatting — LLMs hallucinate package names, enabling attackers to register them with malicious code ValueAdd VC 2026: Vulnerability density 2.74× higher in AI code; code churn ~2× higher in AI-heavy repos Meta Security Researcher's AI Agent Accidentally Deleted Her Emails Moltbook's "vibe-coded" breach is the future of security failures In a study evaluating over 500k code samples, LLM-generated code was found to contain more high-risk security vulnerabilities than human-generated code LLMs make up package names, making them vulnerable to incorporating malicious code in "slopsquatting" attacks (Arxiv study) permalink fedilink source parent hideshow 2 child comments replies: [–] Kangae_Hishiryo@scribe.disroot.org 0 points 3 days ago Part 2: Productivity Illusion (perception vs reality) METR RCT: Experienced OSS devs were 19% slower with AI; 39-point perception gap (believed 20% faster) McKinsey 2025: 46% time savings on routine tasks but <10% on complex work (4,500 devs) Sonar State of Code 2026: 96% of devs don't fully trust AI code; only 48% always verify before commit; 53% say it "appears correct but is unreliable" Stack Overflow 2025: 66% top frustration = code that's "almost right but not quite" Stack Overflow Blog Jan 2026: 45% of developers say debugging AI-generated code takes longer than writing it themselves Byteiota 2026: Trust in AI tools fell from 40% (2024) to 29% (2025); 96% believe AI code is not fully correct Smarter Articles 2026: Devs with Copilot introduced a 41% increase in bugs, with no reduction in burnout risk Churn, Replacement & Survival Faros AI: 65% survival rate for AI code vs 92% human (35% gets silently replaced) Faros AI: +98% PRs merged but +91% review time, +9% bugs, DORA metrics flat (10,000+ devs) Kunal Ganglani 2026: 40% of new AI-assisted code is rewritten within two weeks, vs 33% pre-AI New Relic 2026: 74% of tech leaders report ≥25% of AI code requires significant post-deployment rework; 82% have suffered at least one major production failure caused by AI code Lightrun 2026: 43% of AI-generated code changes require manual debugging in production after passing QA and staging Production Impact & Outages CloudBees 2026: 81% of enterprise leaders report increased production issues from AI code Amazon / CNBC Mar 2026: Amazon convenes "deep dive" meeting over outages caused by GenAI-assisted changes; "high blast radius" incidents since Q3 2025 The New Stack Mar 2026: Amazon mandates senior engineer sign-off on all AI-assisted code changes after multiple outages Fortune Mar 2026: Amazon retail website crashes from "inaccurate advice" an AI agent pulled from a stale wiki; 4 Sev-1 incidents in one week Deskilling Brains show less activity when completing tasks with LLMs compared to completing tasks with search or completing tasks without digital help. Developers who use early-2025 LLMs reported higher subjective performance, but were measured to have lower objective performance. This gap between subjective and objective performance was considered notable. In an Anthropic study, learners using LLMs demonstrated lower learning rates on average compared to learners not using LLMs. A recent study uses the term "cognitive surrender" to describe the way humans tend to offload key critical thinking skills onto LLMs, even when the output is wrong. A paper entitled "AI Assistance Reduces Persistence and Hurts Independent Performance" from April 2026 by academics from MIT, Oxford, UCLA, and Carnegie Mellon showed alarming evidence that performing a variety of tasks with the help of AI for only 10 minutes causes "inpaired unassisted performance and reduced persistence". The researchers noted that "although AI assistance improves performance in the short-term, people perform significantly worse without AI and are more likely to give up"; they also pointed out that "these findings are particularly concerning because persistence is foundational to skill acquisition and is one of the strongest predictors of long-term learning". permalink fedilink source parent
[+] Kangae_Hishiryo@scribe.disroot.org -9 points 4 days ago (1 child) If you don't have anything better to say than an ad hominem fallacy and a red herring fallacy, then you're totally discrediting yourself. permalink fedilink source parent hideshow 2 child comments replies: [–] AwesomeLowlander@sh.itjust.works 8 points 4 days ago (1 child) Yes, because your original comment was so thought out and well reasoned to begin with 🙄 permalink fedilink source parent hideshow 2 child comments replies: [–] Kangae_Hishiryo@scribe.disroot.org 1 point 3 days ago (2 children) Well, it's not that I'm telling another thing than the truth. You're attacking something totally unrelated and that I've never mentioned ("Linux 2.0") because you simply can't deny that LLM are bug-making machinegunss. Literature on that in my response. permalink fedilink source parent hideshow 4 child comments replies: [–] AwesomeLowlander@sh.itjust.works 1 point 3 days ago Do you think anybody, especially Linus, is unaware that LLMs produce bugs if left to themselves? He's made it clear that PRs will continue to be judged on their code quality as they have all this while. Your comment is especially ironic considering how the post is about an elusive bug that took 1 line to fix, but hours upon hours and 24 other patches to debug. Anybody with any knowledge of Linus would find the idea hilarious that he'd allow, let alone commit, crappy code into the kernel. The only real issue up for debate is the ethics of it, considering the provenance of most training data, and potentially copyright issues. permalink fedilink source parent [–] Kangae_Hishiryo@scribe.disroot.org -1 points 3 days ago* (1 child) Part 1: Code Quality (defects, bugs, logic) CodeRabbit: AI code has 1.7× more defects (10.83 vs 6.45 issues/PR); logic errors 1.75×, security 1.57×, XSS 2.74× BusinessWire / CodeRabbit: Performance inefficiencies (excessive I/O) appear ~8× more in AI code; logic problems up 75% The Register: AI code shows 1.4× more critical and 1.7× more major issues; 1.57× more security findings Carnegie Mellon (via Ox Security): Only 61% of AI-generated code functions correctly; only 10.5% passes security review Ranger 2026: 26.6% produce incorrect outputs; 60% of faults are silent logic failures; ~50% has maintenance problems GitClear 2026: Copy-paste code rose 9.4%→15.7%; duplication up ~4×; refactoring down 70%; code churn nearly doubled arXiv 2026: LLMs suffer "Context Rot"; accuracy drops sharply when relevant info is mid-context arXiv 2026: Copilot generates executable code ~90% of the time, but 40% of solutions on critical security tasks contained MITRE Top-25 CWEs arXiv 2026 "Should I Give Up Now?": LLMs "hallucinate, omit important steps, lose context between turns, or produce deceptive code" Based Info: "The problem is execution, not compilation" — code compiles cleanly but fails at runtime or solves the wrong problem CodeBridge / Ox Security: 10 recurring antipatterns in 80-100% of AI code; 68-73% contain vulnerabilities that pass unit tests but fail in production ResearchGate 2026: Valgrind revealed AI code left 1,068 bytes in 34 reachable blocks at exit (memory leaks) vs 24 bytes in 2 blocks for human code IEEE Spectrum 2026: AI code degrades as newer models create "silent failures" and rely on low-quality training data arXiv 2512.22387: 31.7% of AI-generated projects fail to execute at all (only 68.3% reproducible) arXiv 2510.26130: LLMs achieve 84-89% on synthetic benchmarks but only 25-34% on real-world code (66-75% functionally incorrect) SWE-bench 2026: Top models reach ~80%, most struggle below 20% on repository-level tasks (80%+ failure rate) SmartBear 2026: 70% of engineering leaders say quality has degraded; 60% report code outpacing testing capacity DeviQA 2026: 0% of senior QA (8+ yrs) said AI code has fewer bugs; 37.5% said noticeably more arXiv 2026: 304,362 AI commits → 110,000+ unresolved technical debt issues by Feb 2026 How I Dropped Our Production Database and Now Pay 10% More for AWS Claude Tested Everything Except the One Thing That Mattered (AI agent refuses to follow explicit instructions to test createPost() in increasingly erratic ways) Amazon calls engineers for a “deep dive” internal meeting to discuss “GenAI”-related outages GitClear has released reports in 2024 and 2025 indicating a worsening of key code quality metrics correlating with increased LLM adoption. Security & Vulnerabilities Veracode 2025/2026: 45% of AI-generated code introduces OWASP Top 10 vulnerabilities (100+ LLMs tested); Java 70%+ failure rate Cloud Security Alliance 2026: Privilege escalation paths +322%, architectural flaws +153%, despite syntax errors −76%; ~20% of samples reference nonexistent packages arXiv "Broken by Default" 2026: Mean vulnerability rate 55.8% (GPT-4o: 62.4%) across 3,500 artifacts verified with Z3 Sherlock Forensics 2026: 100% of AI-generated apps contained ≥1 critical vulnerability; 78% store secrets in plaintext; 34% of Node.js projects include hallucinated dependencies Georgetown CSET: 86% failed XSS defense, 88% vulnerable to log injection, 47% SQL injection across 5 LLMs Llama 3.3 SWE-bench study: 11× more new vulnerabilities in LLM patches vs developer patches arXiv 2603.10072: Only 24.8% of LLM security patches achieve full correctness; 51.4% fail BOTH security and functionality Stanford/MIT Mar 2026: 14.3% of AI code has ≥1 security vulnerability vs 9.1% human (2M snippets) Apiiro Fortune 50: AI-assisted devs produce 10× more security issues despite 3–4× more code ACM 2025/2026: ~30% of generated code snippets contain security weaknesses Pagerly 2026: Models produce compilable code almost always but secure code only 56% of the time Dev.to / State of Web Dev 2026: 63% of AI-generated functions had a security finding; review doesn't scale with volume Stanford (via SC World): In 80% of tasks, devs using AI produced less secure code; 3.5× more likely to believe their code was secure Georgia Tech Vibe Security Radar: 74 CVEs attributed to AI tools (Mar 2026); est. 400–700 real arXiv 2026: Slopsquatting — LLMs hallucinate package names, enabling attackers to register them with malicious code ValueAdd VC 2026: Vulnerability density 2.74× higher in AI code; code churn ~2× higher in AI-heavy repos Meta Security Researcher's AI Agent Accidentally Deleted Her Emails Moltbook's "vibe-coded" breach is the future of security failures In a study evaluating over 500k code samples, LLM-generated code was found to contain more high-risk security vulnerabilities than human-generated code LLMs make up package names, making them vulnerable to incorporating malicious code in "slopsquatting" attacks (Arxiv study) permalink fedilink source parent hideshow 2 child comments replies: [–] Kangae_Hishiryo@scribe.disroot.org 0 points 3 days ago Part 2: Productivity Illusion (perception vs reality) METR RCT: Experienced OSS devs were 19% slower with AI; 39-point perception gap (believed 20% faster) McKinsey 2025: 46% time savings on routine tasks but <10% on complex work (4,500 devs) Sonar State of Code 2026: 96% of devs don't fully trust AI code; only 48% always verify before commit; 53% say it "appears correct but is unreliable" Stack Overflow 2025: 66% top frustration = code that's "almost right but not quite" Stack Overflow Blog Jan 2026: 45% of developers say debugging AI-generated code takes longer than writing it themselves Byteiota 2026: Trust in AI tools fell from 40% (2024) to 29% (2025); 96% believe AI code is not fully correct Smarter Articles 2026: Devs with Copilot introduced a 41% increase in bugs, with no reduction in burnout risk Churn, Replacement & Survival Faros AI: 65% survival rate for AI code vs 92% human (35% gets silently replaced) Faros AI: +98% PRs merged but +91% review time, +9% bugs, DORA metrics flat (10,000+ devs) Kunal Ganglani 2026: 40% of new AI-assisted code is rewritten within two weeks, vs 33% pre-AI New Relic 2026: 74% of tech leaders report ≥25% of AI code requires significant post-deployment rework; 82% have suffered at least one major production failure caused by AI code Lightrun 2026: 43% of AI-generated code changes require manual debugging in production after passing QA and staging Production Impact & Outages CloudBees 2026: 81% of enterprise leaders report increased production issues from AI code Amazon / CNBC Mar 2026: Amazon convenes "deep dive" meeting over outages caused by GenAI-assisted changes; "high blast radius" incidents since Q3 2025 The New Stack Mar 2026: Amazon mandates senior engineer sign-off on all AI-assisted code changes after multiple outages Fortune Mar 2026: Amazon retail website crashes from "inaccurate advice" an AI agent pulled from a stale wiki; 4 Sev-1 incidents in one week Deskilling Brains show less activity when completing tasks with LLMs compared to completing tasks with search or completing tasks without digital help. Developers who use early-2025 LLMs reported higher subjective performance, but were measured to have lower objective performance. This gap between subjective and objective performance was considered notable. In an Anthropic study, learners using LLMs demonstrated lower learning rates on average compared to learners not using LLMs. A recent study uses the term "cognitive surrender" to describe the way humans tend to offload key critical thinking skills onto LLMs, even when the output is wrong. A paper entitled "AI Assistance Reduces Persistence and Hurts Independent Performance" from April 2026 by academics from MIT, Oxford, UCLA, and Carnegie Mellon showed alarming evidence that performing a variety of tasks with the help of AI for only 10 minutes causes "inpaired unassisted performance and reduced persistence". The researchers noted that "although AI assistance improves performance in the short-term, people perform significantly worse without AI and are more likely to give up"; they also pointed out that "these findings are particularly concerning because persistence is foundational to skill acquisition and is one of the strongest predictors of long-term learning". permalink fedilink source parent
[–] AwesomeLowlander@sh.itjust.works 8 points 4 days ago (1 child) Yes, because your original comment was so thought out and well reasoned to begin with 🙄 permalink fedilink source parent hideshow 2 child comments replies: [–] Kangae_Hishiryo@scribe.disroot.org 1 point 3 days ago (2 children) Well, it's not that I'm telling another thing than the truth. You're attacking something totally unrelated and that I've never mentioned ("Linux 2.0") because you simply can't deny that LLM are bug-making machinegunss. Literature on that in my response. permalink fedilink source parent hideshow 4 child comments replies: [–] AwesomeLowlander@sh.itjust.works 1 point 3 days ago Do you think anybody, especially Linus, is unaware that LLMs produce bugs if left to themselves? He's made it clear that PRs will continue to be judged on their code quality as they have all this while. Your comment is especially ironic considering how the post is about an elusive bug that took 1 line to fix, but hours upon hours and 24 other patches to debug. Anybody with any knowledge of Linus would find the idea hilarious that he'd allow, let alone commit, crappy code into the kernel. The only real issue up for debate is the ethics of it, considering the provenance of most training data, and potentially copyright issues. permalink fedilink source parent [–] Kangae_Hishiryo@scribe.disroot.org -1 points 3 days ago* (1 child) Part 1: Code Quality (defects, bugs, logic) CodeRabbit: AI code has 1.7× more defects (10.83 vs 6.45 issues/PR); logic errors 1.75×, security 1.57×, XSS 2.74× BusinessWire / CodeRabbit: Performance inefficiencies (excessive I/O) appear ~8× more in AI code; logic problems up 75% The Register: AI code shows 1.4× more critical and 1.7× more major issues; 1.57× more security findings Carnegie Mellon (via Ox Security): Only 61% of AI-generated code functions correctly; only 10.5% passes security review Ranger 2026: 26.6% produce incorrect outputs; 60% of faults are silent logic failures; ~50% has maintenance problems GitClear 2026: Copy-paste code rose 9.4%→15.7%; duplication up ~4×; refactoring down 70%; code churn nearly doubled arXiv 2026: LLMs suffer "Context Rot"; accuracy drops sharply when relevant info is mid-context arXiv 2026: Copilot generates executable code ~90% of the time, but 40% of solutions on critical security tasks contained MITRE Top-25 CWEs arXiv 2026 "Should I Give Up Now?": LLMs "hallucinate, omit important steps, lose context between turns, or produce deceptive code" Based Info: "The problem is execution, not compilation" — code compiles cleanly but fails at runtime or solves the wrong problem CodeBridge / Ox Security: 10 recurring antipatterns in 80-100% of AI code; 68-73% contain vulnerabilities that pass unit tests but fail in production ResearchGate 2026: Valgrind revealed AI code left 1,068 bytes in 34 reachable blocks at exit (memory leaks) vs 24 bytes in 2 blocks for human code IEEE Spectrum 2026: AI code degrades as newer models create "silent failures" and rely on low-quality training data arXiv 2512.22387: 31.7% of AI-generated projects fail to execute at all (only 68.3% reproducible) arXiv 2510.26130: LLMs achieve 84-89% on synthetic benchmarks but only 25-34% on real-world code (66-75% functionally incorrect) SWE-bench 2026: Top models reach ~80%, most struggle below 20% on repository-level tasks (80%+ failure rate) SmartBear 2026: 70% of engineering leaders say quality has degraded; 60% report code outpacing testing capacity DeviQA 2026: 0% of senior QA (8+ yrs) said AI code has fewer bugs; 37.5% said noticeably more arXiv 2026: 304,362 AI commits → 110,000+ unresolved technical debt issues by Feb 2026 How I Dropped Our Production Database and Now Pay 10% More for AWS Claude Tested Everything Except the One Thing That Mattered (AI agent refuses to follow explicit instructions to test createPost() in increasingly erratic ways) Amazon calls engineers for a “deep dive” internal meeting to discuss “GenAI”-related outages GitClear has released reports in 2024 and 2025 indicating a worsening of key code quality metrics correlating with increased LLM adoption. Security & Vulnerabilities Veracode 2025/2026: 45% of AI-generated code introduces OWASP Top 10 vulnerabilities (100+ LLMs tested); Java 70%+ failure rate Cloud Security Alliance 2026: Privilege escalation paths +322%, architectural flaws +153%, despite syntax errors −76%; ~20% of samples reference nonexistent packages arXiv "Broken by Default" 2026: Mean vulnerability rate 55.8% (GPT-4o: 62.4%) across 3,500 artifacts verified with Z3 Sherlock Forensics 2026: 100% of AI-generated apps contained ≥1 critical vulnerability; 78% store secrets in plaintext; 34% of Node.js projects include hallucinated dependencies Georgetown CSET: 86% failed XSS defense, 88% vulnerable to log injection, 47% SQL injection across 5 LLMs Llama 3.3 SWE-bench study: 11× more new vulnerabilities in LLM patches vs developer patches arXiv 2603.10072: Only 24.8% of LLM security patches achieve full correctness; 51.4% fail BOTH security and functionality Stanford/MIT Mar 2026: 14.3% of AI code has ≥1 security vulnerability vs 9.1% human (2M snippets) Apiiro Fortune 50: AI-assisted devs produce 10× more security issues despite 3–4× more code ACM 2025/2026: ~30% of generated code snippets contain security weaknesses Pagerly 2026: Models produce compilable code almost always but secure code only 56% of the time Dev.to / State of Web Dev 2026: 63% of AI-generated functions had a security finding; review doesn't scale with volume Stanford (via SC World): In 80% of tasks, devs using AI produced less secure code; 3.5× more likely to believe their code was secure Georgia Tech Vibe Security Radar: 74 CVEs attributed to AI tools (Mar 2026); est. 400–700 real arXiv 2026: Slopsquatting — LLMs hallucinate package names, enabling attackers to register them with malicious code ValueAdd VC 2026: Vulnerability density 2.74× higher in AI code; code churn ~2× higher in AI-heavy repos Meta Security Researcher's AI Agent Accidentally Deleted Her Emails Moltbook's "vibe-coded" breach is the future of security failures In a study evaluating over 500k code samples, LLM-generated code was found to contain more high-risk security vulnerabilities than human-generated code LLMs make up package names, making them vulnerable to incorporating malicious code in "slopsquatting" attacks (Arxiv study) permalink fedilink source parent hideshow 2 child comments replies: [–] Kangae_Hishiryo@scribe.disroot.org 0 points 3 days ago Part 2: Productivity Illusion (perception vs reality) METR RCT: Experienced OSS devs were 19% slower with AI; 39-point perception gap (believed 20% faster) McKinsey 2025: 46% time savings on routine tasks but <10% on complex work (4,500 devs) Sonar State of Code 2026: 96% of devs don't fully trust AI code; only 48% always verify before commit; 53% say it "appears correct but is unreliable" Stack Overflow 2025: 66% top frustration = code that's "almost right but not quite" Stack Overflow Blog Jan 2026: 45% of developers say debugging AI-generated code takes longer than writing it themselves Byteiota 2026: Trust in AI tools fell from 40% (2024) to 29% (2025); 96% believe AI code is not fully correct Smarter Articles 2026: Devs with Copilot introduced a 41% increase in bugs, with no reduction in burnout risk Churn, Replacement & Survival Faros AI: 65% survival rate for AI code vs 92% human (35% gets silently replaced) Faros AI: +98% PRs merged but +91% review time, +9% bugs, DORA metrics flat (10,000+ devs) Kunal Ganglani 2026: 40% of new AI-assisted code is rewritten within two weeks, vs 33% pre-AI New Relic 2026: 74% of tech leaders report ≥25% of AI code requires significant post-deployment rework; 82% have suffered at least one major production failure caused by AI code Lightrun 2026: 43% of AI-generated code changes require manual debugging in production after passing QA and staging Production Impact & Outages CloudBees 2026: 81% of enterprise leaders report increased production issues from AI code Amazon / CNBC Mar 2026: Amazon convenes "deep dive" meeting over outages caused by GenAI-assisted changes; "high blast radius" incidents since Q3 2025 The New Stack Mar 2026: Amazon mandates senior engineer sign-off on all AI-assisted code changes after multiple outages Fortune Mar 2026: Amazon retail website crashes from "inaccurate advice" an AI agent pulled from a stale wiki; 4 Sev-1 incidents in one week Deskilling Brains show less activity when completing tasks with LLMs compared to completing tasks with search or completing tasks without digital help. Developers who use early-2025 LLMs reported higher subjective performance, but were measured to have lower objective performance. This gap between subjective and objective performance was considered notable. In an Anthropic study, learners using LLMs demonstrated lower learning rates on average compared to learners not using LLMs. A recent study uses the term "cognitive surrender" to describe the way humans tend to offload key critical thinking skills onto LLMs, even when the output is wrong. A paper entitled "AI Assistance Reduces Persistence and Hurts Independent Performance" from April 2026 by academics from MIT, Oxford, UCLA, and Carnegie Mellon showed alarming evidence that performing a variety of tasks with the help of AI for only 10 minutes causes "inpaired unassisted performance and reduced persistence". The researchers noted that "although AI assistance improves performance in the short-term, people perform significantly worse without AI and are more likely to give up"; they also pointed out that "these findings are particularly concerning because persistence is foundational to skill acquisition and is one of the strongest predictors of long-term learning". permalink fedilink source parent
[–] Kangae_Hishiryo@scribe.disroot.org 1 point 3 days ago (2 children) Well, it's not that I'm telling another thing than the truth. You're attacking something totally unrelated and that I've never mentioned ("Linux 2.0") because you simply can't deny that LLM are bug-making machinegunss. Literature on that in my response. permalink fedilink source parent hideshow 4 child comments replies: [–] AwesomeLowlander@sh.itjust.works 1 point 3 days ago Do you think anybody, especially Linus, is unaware that LLMs produce bugs if left to themselves? He's made it clear that PRs will continue to be judged on their code quality as they have all this while. Your comment is especially ironic considering how the post is about an elusive bug that took 1 line to fix, but hours upon hours and 24 other patches to debug. Anybody with any knowledge of Linus would find the idea hilarious that he'd allow, let alone commit, crappy code into the kernel. The only real issue up for debate is the ethics of it, considering the provenance of most training data, and potentially copyright issues. permalink fedilink source parent [–] Kangae_Hishiryo@scribe.disroot.org -1 points 3 days ago* (1 child) Part 1: Code Quality (defects, bugs, logic) CodeRabbit: AI code has 1.7× more defects (10.83 vs 6.45 issues/PR); logic errors 1.75×, security 1.57×, XSS 2.74× BusinessWire / CodeRabbit: Performance inefficiencies (excessive I/O) appear ~8× more in AI code; logic problems up 75% The Register: AI code shows 1.4× more critical and 1.7× more major issues; 1.57× more security findings Carnegie Mellon (via Ox Security): Only 61% of AI-generated code functions correctly; only 10.5% passes security review Ranger 2026: 26.6% produce incorrect outputs; 60% of faults are silent logic failures; ~50% has maintenance problems GitClear 2026: Copy-paste code rose 9.4%→15.7%; duplication up ~4×; refactoring down 70%; code churn nearly doubled arXiv 2026: LLMs suffer "Context Rot"; accuracy drops sharply when relevant info is mid-context arXiv 2026: Copilot generates executable code ~90% of the time, but 40% of solutions on critical security tasks contained MITRE Top-25 CWEs arXiv 2026 "Should I Give Up Now?": LLMs "hallucinate, omit important steps, lose context between turns, or produce deceptive code" Based Info: "The problem is execution, not compilation" — code compiles cleanly but fails at runtime or solves the wrong problem CodeBridge / Ox Security: 10 recurring antipatterns in 80-100% of AI code; 68-73% contain vulnerabilities that pass unit tests but fail in production ResearchGate 2026: Valgrind revealed AI code left 1,068 bytes in 34 reachable blocks at exit (memory leaks) vs 24 bytes in 2 blocks for human code IEEE Spectrum 2026: AI code degrades as newer models create "silent failures" and rely on low-quality training data arXiv 2512.22387: 31.7% of AI-generated projects fail to execute at all (only 68.3% reproducible) arXiv 2510.26130: LLMs achieve 84-89% on synthetic benchmarks but only 25-34% on real-world code (66-75% functionally incorrect) SWE-bench 2026: Top models reach ~80%, most struggle below 20% on repository-level tasks (80%+ failure rate) SmartBear 2026: 70% of engineering leaders say quality has degraded; 60% report code outpacing testing capacity DeviQA 2026: 0% of senior QA (8+ yrs) said AI code has fewer bugs; 37.5% said noticeably more arXiv 2026: 304,362 AI commits → 110,000+ unresolved technical debt issues by Feb 2026 How I Dropped Our Production Database and Now Pay 10% More for AWS Claude Tested Everything Except the One Thing That Mattered (AI agent refuses to follow explicit instructions to test createPost() in increasingly erratic ways) Amazon calls engineers for a “deep dive” internal meeting to discuss “GenAI”-related outages GitClear has released reports in 2024 and 2025 indicating a worsening of key code quality metrics correlating with increased LLM adoption. Security & Vulnerabilities Veracode 2025/2026: 45% of AI-generated code introduces OWASP Top 10 vulnerabilities (100+ LLMs tested); Java 70%+ failure rate Cloud Security Alliance 2026: Privilege escalation paths +322%, architectural flaws +153%, despite syntax errors −76%; ~20% of samples reference nonexistent packages arXiv "Broken by Default" 2026: Mean vulnerability rate 55.8% (GPT-4o: 62.4%) across 3,500 artifacts verified with Z3 Sherlock Forensics 2026: 100% of AI-generated apps contained ≥1 critical vulnerability; 78% store secrets in plaintext; 34% of Node.js projects include hallucinated dependencies Georgetown CSET: 86% failed XSS defense, 88% vulnerable to log injection, 47% SQL injection across 5 LLMs Llama 3.3 SWE-bench study: 11× more new vulnerabilities in LLM patches vs developer patches arXiv 2603.10072: Only 24.8% of LLM security patches achieve full correctness; 51.4% fail BOTH security and functionality Stanford/MIT Mar 2026: 14.3% of AI code has ≥1 security vulnerability vs 9.1% human (2M snippets) Apiiro Fortune 50: AI-assisted devs produce 10× more security issues despite 3–4× more code ACM 2025/2026: ~30% of generated code snippets contain security weaknesses Pagerly 2026: Models produce compilable code almost always but secure code only 56% of the time Dev.to / State of Web Dev 2026: 63% of AI-generated functions had a security finding; review doesn't scale with volume Stanford (via SC World): In 80% of tasks, devs using AI produced less secure code; 3.5× more likely to believe their code was secure Georgia Tech Vibe Security Radar: 74 CVEs attributed to AI tools (Mar 2026); est. 400–700 real arXiv 2026: Slopsquatting — LLMs hallucinate package names, enabling attackers to register them with malicious code ValueAdd VC 2026: Vulnerability density 2.74× higher in AI code; code churn ~2× higher in AI-heavy repos Meta Security Researcher's AI Agent Accidentally Deleted Her Emails Moltbook's "vibe-coded" breach is the future of security failures In a study evaluating over 500k code samples, LLM-generated code was found to contain more high-risk security vulnerabilities than human-generated code LLMs make up package names, making them vulnerable to incorporating malicious code in "slopsquatting" attacks (Arxiv study) permalink fedilink source parent hideshow 2 child comments replies: [–] Kangae_Hishiryo@scribe.disroot.org 0 points 3 days ago Part 2: Productivity Illusion (perception vs reality) METR RCT: Experienced OSS devs were 19% slower with AI; 39-point perception gap (believed 20% faster) McKinsey 2025: 46% time savings on routine tasks but <10% on complex work (4,500 devs) Sonar State of Code 2026: 96% of devs don't fully trust AI code; only 48% always verify before commit; 53% say it "appears correct but is unreliable" Stack Overflow 2025: 66% top frustration = code that's "almost right but not quite" Stack Overflow Blog Jan 2026: 45% of developers say debugging AI-generated code takes longer than writing it themselves Byteiota 2026: Trust in AI tools fell from 40% (2024) to 29% (2025); 96% believe AI code is not fully correct Smarter Articles 2026: Devs with Copilot introduced a 41% increase in bugs, with no reduction in burnout risk Churn, Replacement & Survival Faros AI: 65% survival rate for AI code vs 92% human (35% gets silently replaced) Faros AI: +98% PRs merged but +91% review time, +9% bugs, DORA metrics flat (10,000+ devs) Kunal Ganglani 2026: 40% of new AI-assisted code is rewritten within two weeks, vs 33% pre-AI New Relic 2026: 74% of tech leaders report ≥25% of AI code requires significant post-deployment rework; 82% have suffered at least one major production failure caused by AI code Lightrun 2026: 43% of AI-generated code changes require manual debugging in production after passing QA and staging Production Impact & Outages CloudBees 2026: 81% of enterprise leaders report increased production issues from AI code Amazon / CNBC Mar 2026: Amazon convenes "deep dive" meeting over outages caused by GenAI-assisted changes; "high blast radius" incidents since Q3 2025 The New Stack Mar 2026: Amazon mandates senior engineer sign-off on all AI-assisted code changes after multiple outages Fortune Mar 2026: Amazon retail website crashes from "inaccurate advice" an AI agent pulled from a stale wiki; 4 Sev-1 incidents in one week Deskilling Brains show less activity when completing tasks with LLMs compared to completing tasks with search or completing tasks without digital help. Developers who use early-2025 LLMs reported higher subjective performance, but were measured to have lower objective performance. This gap between subjective and objective performance was considered notable. In an Anthropic study, learners using LLMs demonstrated lower learning rates on average compared to learners not using LLMs. A recent study uses the term "cognitive surrender" to describe the way humans tend to offload key critical thinking skills onto LLMs, even when the output is wrong. A paper entitled "AI Assistance Reduces Persistence and Hurts Independent Performance" from April 2026 by academics from MIT, Oxford, UCLA, and Carnegie Mellon showed alarming evidence that performing a variety of tasks with the help of AI for only 10 minutes causes "inpaired unassisted performance and reduced persistence". The researchers noted that "although AI assistance improves performance in the short-term, people perform significantly worse without AI and are more likely to give up"; they also pointed out that "these findings are particularly concerning because persistence is foundational to skill acquisition and is one of the strongest predictors of long-term learning". permalink fedilink source parent
[–] AwesomeLowlander@sh.itjust.works 1 point 3 days ago Do you think anybody, especially Linus, is unaware that LLMs produce bugs if left to themselves? He's made it clear that PRs will continue to be judged on their code quality as they have all this while. Your comment is especially ironic considering how the post is about an elusive bug that took 1 line to fix, but hours upon hours and 24 other patches to debug. Anybody with any knowledge of Linus would find the idea hilarious that he'd allow, let alone commit, crappy code into the kernel. The only real issue up for debate is the ethics of it, considering the provenance of most training data, and potentially copyright issues. permalink fedilink source parent
[–] Kangae_Hishiryo@scribe.disroot.org -1 points 3 days ago* (1 child) Part 1: Code Quality (defects, bugs, logic) CodeRabbit: AI code has 1.7× more defects (10.83 vs 6.45 issues/PR); logic errors 1.75×, security 1.57×, XSS 2.74× BusinessWire / CodeRabbit: Performance inefficiencies (excessive I/O) appear ~8× more in AI code; logic problems up 75% The Register: AI code shows 1.4× more critical and 1.7× more major issues; 1.57× more security findings Carnegie Mellon (via Ox Security): Only 61% of AI-generated code functions correctly; only 10.5% passes security review Ranger 2026: 26.6% produce incorrect outputs; 60% of faults are silent logic failures; ~50% has maintenance problems GitClear 2026: Copy-paste code rose 9.4%→15.7%; duplication up ~4×; refactoring down 70%; code churn nearly doubled arXiv 2026: LLMs suffer "Context Rot"; accuracy drops sharply when relevant info is mid-context arXiv 2026: Copilot generates executable code ~90% of the time, but 40% of solutions on critical security tasks contained MITRE Top-25 CWEs arXiv 2026 "Should I Give Up Now?": LLMs "hallucinate, omit important steps, lose context between turns, or produce deceptive code" Based Info: "The problem is execution, not compilation" — code compiles cleanly but fails at runtime or solves the wrong problem CodeBridge / Ox Security: 10 recurring antipatterns in 80-100% of AI code; 68-73% contain vulnerabilities that pass unit tests but fail in production ResearchGate 2026: Valgrind revealed AI code left 1,068 bytes in 34 reachable blocks at exit (memory leaks) vs 24 bytes in 2 blocks for human code IEEE Spectrum 2026: AI code degrades as newer models create "silent failures" and rely on low-quality training data arXiv 2512.22387: 31.7% of AI-generated projects fail to execute at all (only 68.3% reproducible) arXiv 2510.26130: LLMs achieve 84-89% on synthetic benchmarks but only 25-34% on real-world code (66-75% functionally incorrect) SWE-bench 2026: Top models reach ~80%, most struggle below 20% on repository-level tasks (80%+ failure rate) SmartBear 2026: 70% of engineering leaders say quality has degraded; 60% report code outpacing testing capacity DeviQA 2026: 0% of senior QA (8+ yrs) said AI code has fewer bugs; 37.5% said noticeably more arXiv 2026: 304,362 AI commits → 110,000+ unresolved technical debt issues by Feb 2026 How I Dropped Our Production Database and Now Pay 10% More for AWS Claude Tested Everything Except the One Thing That Mattered (AI agent refuses to follow explicit instructions to test createPost() in increasingly erratic ways) Amazon calls engineers for a “deep dive” internal meeting to discuss “GenAI”-related outages GitClear has released reports in 2024 and 2025 indicating a worsening of key code quality metrics correlating with increased LLM adoption. Security & Vulnerabilities Veracode 2025/2026: 45% of AI-generated code introduces OWASP Top 10 vulnerabilities (100+ LLMs tested); Java 70%+ failure rate Cloud Security Alliance 2026: Privilege escalation paths +322%, architectural flaws +153%, despite syntax errors −76%; ~20% of samples reference nonexistent packages arXiv "Broken by Default" 2026: Mean vulnerability rate 55.8% (GPT-4o: 62.4%) across 3,500 artifacts verified with Z3 Sherlock Forensics 2026: 100% of AI-generated apps contained ≥1 critical vulnerability; 78% store secrets in plaintext; 34% of Node.js projects include hallucinated dependencies Georgetown CSET: 86% failed XSS defense, 88% vulnerable to log injection, 47% SQL injection across 5 LLMs Llama 3.3 SWE-bench study: 11× more new vulnerabilities in LLM patches vs developer patches arXiv 2603.10072: Only 24.8% of LLM security patches achieve full correctness; 51.4% fail BOTH security and functionality Stanford/MIT Mar 2026: 14.3% of AI code has ≥1 security vulnerability vs 9.1% human (2M snippets) Apiiro Fortune 50: AI-assisted devs produce 10× more security issues despite 3–4× more code ACM 2025/2026: ~30% of generated code snippets contain security weaknesses Pagerly 2026: Models produce compilable code almost always but secure code only 56% of the time Dev.to / State of Web Dev 2026: 63% of AI-generated functions had a security finding; review doesn't scale with volume Stanford (via SC World): In 80% of tasks, devs using AI produced less secure code; 3.5× more likely to believe their code was secure Georgia Tech Vibe Security Radar: 74 CVEs attributed to AI tools (Mar 2026); est. 400–700 real arXiv 2026: Slopsquatting — LLMs hallucinate package names, enabling attackers to register them with malicious code ValueAdd VC 2026: Vulnerability density 2.74× higher in AI code; code churn ~2× higher in AI-heavy repos Meta Security Researcher's AI Agent Accidentally Deleted Her Emails Moltbook's "vibe-coded" breach is the future of security failures In a study evaluating over 500k code samples, LLM-generated code was found to contain more high-risk security vulnerabilities than human-generated code LLMs make up package names, making them vulnerable to incorporating malicious code in "slopsquatting" attacks (Arxiv study) permalink fedilink source parent hideshow 2 child comments replies: [–] Kangae_Hishiryo@scribe.disroot.org 0 points 3 days ago Part 2: Productivity Illusion (perception vs reality) METR RCT: Experienced OSS devs were 19% slower with AI; 39-point perception gap (believed 20% faster) McKinsey 2025: 46% time savings on routine tasks but <10% on complex work (4,500 devs) Sonar State of Code 2026: 96% of devs don't fully trust AI code; only 48% always verify before commit; 53% say it "appears correct but is unreliable" Stack Overflow 2025: 66% top frustration = code that's "almost right but not quite" Stack Overflow Blog Jan 2026: 45% of developers say debugging AI-generated code takes longer than writing it themselves Byteiota 2026: Trust in AI tools fell from 40% (2024) to 29% (2025); 96% believe AI code is not fully correct Smarter Articles 2026: Devs with Copilot introduced a 41% increase in bugs, with no reduction in burnout risk Churn, Replacement & Survival Faros AI: 65% survival rate for AI code vs 92% human (35% gets silently replaced) Faros AI: +98% PRs merged but +91% review time, +9% bugs, DORA metrics flat (10,000+ devs) Kunal Ganglani 2026: 40% of new AI-assisted code is rewritten within two weeks, vs 33% pre-AI New Relic 2026: 74% of tech leaders report ≥25% of AI code requires significant post-deployment rework; 82% have suffered at least one major production failure caused by AI code Lightrun 2026: 43% of AI-generated code changes require manual debugging in production after passing QA and staging Production Impact & Outages CloudBees 2026: 81% of enterprise leaders report increased production issues from AI code Amazon / CNBC Mar 2026: Amazon convenes "deep dive" meeting over outages caused by GenAI-assisted changes; "high blast radius" incidents since Q3 2025 The New Stack Mar 2026: Amazon mandates senior engineer sign-off on all AI-assisted code changes after multiple outages Fortune Mar 2026: Amazon retail website crashes from "inaccurate advice" an AI agent pulled from a stale wiki; 4 Sev-1 incidents in one week Deskilling Brains show less activity when completing tasks with LLMs compared to completing tasks with search or completing tasks without digital help. Developers who use early-2025 LLMs reported higher subjective performance, but were measured to have lower objective performance. This gap between subjective and objective performance was considered notable. In an Anthropic study, learners using LLMs demonstrated lower learning rates on average compared to learners not using LLMs. A recent study uses the term "cognitive surrender" to describe the way humans tend to offload key critical thinking skills onto LLMs, even when the output is wrong. A paper entitled "AI Assistance Reduces Persistence and Hurts Independent Performance" from April 2026 by academics from MIT, Oxford, UCLA, and Carnegie Mellon showed alarming evidence that performing a variety of tasks with the help of AI for only 10 minutes causes "inpaired unassisted performance and reduced persistence". The researchers noted that "although AI assistance improves performance in the short-term, people perform significantly worse without AI and are more likely to give up"; they also pointed out that "these findings are particularly concerning because persistence is foundational to skill acquisition and is one of the strongest predictors of long-term learning". permalink fedilink source parent
[–] Kangae_Hishiryo@scribe.disroot.org 0 points 3 days ago Part 2: Productivity Illusion (perception vs reality) METR RCT: Experienced OSS devs were 19% slower with AI; 39-point perception gap (believed 20% faster) McKinsey 2025: 46% time savings on routine tasks but <10% on complex work (4,500 devs) Sonar State of Code 2026: 96% of devs don't fully trust AI code; only 48% always verify before commit; 53% say it "appears correct but is unreliable" Stack Overflow 2025: 66% top frustration = code that's "almost right but not quite" Stack Overflow Blog Jan 2026: 45% of developers say debugging AI-generated code takes longer than writing it themselves Byteiota 2026: Trust in AI tools fell from 40% (2024) to 29% (2025); 96% believe AI code is not fully correct Smarter Articles 2026: Devs with Copilot introduced a 41% increase in bugs, with no reduction in burnout risk Churn, Replacement & Survival Faros AI: 65% survival rate for AI code vs 92% human (35% gets silently replaced) Faros AI: +98% PRs merged but +91% review time, +9% bugs, DORA metrics flat (10,000+ devs) Kunal Ganglani 2026: 40% of new AI-assisted code is rewritten within two weeks, vs 33% pre-AI New Relic 2026: 74% of tech leaders report ≥25% of AI code requires significant post-deployment rework; 82% have suffered at least one major production failure caused by AI code Lightrun 2026: 43% of AI-generated code changes require manual debugging in production after passing QA and staging Production Impact & Outages CloudBees 2026: 81% of enterprise leaders report increased production issues from AI code Amazon / CNBC Mar 2026: Amazon convenes "deep dive" meeting over outages caused by GenAI-assisted changes; "high blast radius" incidents since Q3 2025 The New Stack Mar 2026: Amazon mandates senior engineer sign-off on all AI-assisted code changes after multiple outages Fortune Mar 2026: Amazon retail website crashes from "inaccurate advice" an AI agent pulled from a stale wiki; 4 Sev-1 incidents in one week Deskilling Brains show less activity when completing tasks with LLMs compared to completing tasks with search or completing tasks without digital help. Developers who use early-2025 LLMs reported higher subjective performance, but were measured to have lower objective performance. This gap between subjective and objective performance was considered notable. In an Anthropic study, learners using LLMs demonstrated lower learning rates on average compared to learners not using LLMs. A recent study uses the term "cognitive surrender" to describe the way humans tend to offload key critical thinking skills onto LLMs, even when the output is wrong. A paper entitled "AI Assistance Reduces Persistence and Hurts Independent Performance" from April 2026 by academics from MIT, Oxford, UCLA, and Carnegie Mellon showed alarming evidence that performing a variety of tasks with the help of AI for only 10 minutes causes "inpaired unassisted performance and reduced persistence". The researchers noted that "although AI assistance improves performance in the short-term, people perform significantly worse without AI and are more likely to give up"; they also pointed out that "these findings are particularly concerning because persistence is foundational to skill acquisition and is one of the strongest predictors of long-term learning". permalink fedilink source parent