▲ 1224 ▼ Claude Code deletes developers' production setup, including its database and snapshots — 2.5 years of records were nuked in an instant (www.tomshardware.com) submitted 6 months ago by throws_lemy@lemmy.nz to c/technology@lemmy.world 289 comments fedilink hide all child comments
[–] Poppa_Mo@lemmy.world 41 points 6 months ago (6 children) Whoever gave it access to production is a complete moron. permalink fedilink source hideshow 6 child comments replies: [–] tempest@lemmy.ca 19 points 6 months ago* (last edited 6 months ago) (5 children) If you've ever used it you can see how easily it can happen. At first you Sandbox box it and you're careful. Then after a while the sand box is a bit of a pain so you just run it as is. Then it asks for permission a 1000 times to do something and at first you carefully check each command but after a while you just skim them and eventually, sure you can run 'psql *' to debug some query on the dev instance.... It's one of the major problems with the "full self driving" stuff as well. It's right often enough that eventually you get complacent or your attention drifts elsewhere. This kind of stuff happened before the LLM coding agents existed, they have just supercharged the speed and as a result increased the amount of damage that can be done before it's noticed. There are already a bunch of failures in place for something like this to happen. Having the prod credentials available etc etc it's just now instead of rolling the dice every couple weeks your LLM is rolling them every 20s. permalink fedilink source parent hideshow 5 child comments replies: [–] BorgDrone@feddit.nl 5 points 6 months ago (3 children) If you've ever used it you can see how easily it can happen. How could this happen easily? A regular developer shouldn’t even have access to production outside of exceptional circumstances (e.g. diagnosing a production issue). Certainly not as part of the normal dev process. permalink fedilink source parent hideshow 3 child comments replies: [–] tempest@lemmy.ca 1 point 6 months ago (2 children) They shouldn't and we know that but this is hardly the first time that story has been told even before LLMs. Usually it was blamed on "the intern" or whatever. permalink fedilink source parent hideshow 2 child comments replies: [–] BorgDrone@feddit.nl 2 points 6 months ago (1 child) This isn’t just an issue with a developer putting too much trust into an LLM though. This is a failure at the organizational level. So many things have to be wrong for this to happen. If an ‘intern’ can access a production database then you have some serious problems. No one should have access to that in normal operations. permalink fedilink source parent hideshow 1 child comment replies: [–] tempest@lemmy.ca 2 points 6 months ago* (last edited 6 months ago) Sure, I'm not telling you how it should be, I'm telling you how it is. The LLM just increases the damage done because it can do more damage faster before someone figures out they fucked up. This is the last big one I remembered offhand but I know it happens a couple times a year and probably more just goes unreported. https://www.cnn.com/2021/02/26/politics/solarwinds123-password-intern Why would an intern be given prod supply chain credentials, who knows. People fuck up all the time. permalink fedilink source parent [–] ExLisper@lemmy.curiana.net -1 points 6 months ago If you’ve ever used it you can see how easily it can happen. Yes, I can see how it can easily happen to stupid lazy people. permalink fedilink source parent
[–] tempest@lemmy.ca 19 points 6 months ago* (last edited 6 months ago) (5 children) If you've ever used it you can see how easily it can happen. At first you Sandbox box it and you're careful. Then after a while the sand box is a bit of a pain so you just run it as is. Then it asks for permission a 1000 times to do something and at first you carefully check each command but after a while you just skim them and eventually, sure you can run 'psql *' to debug some query on the dev instance.... It's one of the major problems with the "full self driving" stuff as well. It's right often enough that eventually you get complacent or your attention drifts elsewhere. This kind of stuff happened before the LLM coding agents existed, they have just supercharged the speed and as a result increased the amount of damage that can be done before it's noticed. There are already a bunch of failures in place for something like this to happen. Having the prod credentials available etc etc it's just now instead of rolling the dice every couple weeks your LLM is rolling them every 20s. permalink fedilink source parent hideshow 5 child comments replies: [–] BorgDrone@feddit.nl 5 points 6 months ago (3 children) If you've ever used it you can see how easily it can happen. How could this happen easily? A regular developer shouldn’t even have access to production outside of exceptional circumstances (e.g. diagnosing a production issue). Certainly not as part of the normal dev process. permalink fedilink source parent hideshow 3 child comments replies: [–] tempest@lemmy.ca 1 point 6 months ago (2 children) They shouldn't and we know that but this is hardly the first time that story has been told even before LLMs. Usually it was blamed on "the intern" or whatever. permalink fedilink source parent hideshow 2 child comments replies: [–] BorgDrone@feddit.nl 2 points 6 months ago (1 child) This isn’t just an issue with a developer putting too much trust into an LLM though. This is a failure at the organizational level. So many things have to be wrong for this to happen. If an ‘intern’ can access a production database then you have some serious problems. No one should have access to that in normal operations. permalink fedilink source parent hideshow 1 child comment replies: [–] tempest@lemmy.ca 2 points 6 months ago* (last edited 6 months ago) Sure, I'm not telling you how it should be, I'm telling you how it is. The LLM just increases the damage done because it can do more damage faster before someone figures out they fucked up. This is the last big one I remembered offhand but I know it happens a couple times a year and probably more just goes unreported. https://www.cnn.com/2021/02/26/politics/solarwinds123-password-intern Why would an intern be given prod supply chain credentials, who knows. People fuck up all the time. permalink fedilink source parent [–] ExLisper@lemmy.curiana.net -1 points 6 months ago If you’ve ever used it you can see how easily it can happen. Yes, I can see how it can easily happen to stupid lazy people. permalink fedilink source parent
[–] BorgDrone@feddit.nl 5 points 6 months ago (3 children) If you've ever used it you can see how easily it can happen. How could this happen easily? A regular developer shouldn’t even have access to production outside of exceptional circumstances (e.g. diagnosing a production issue). Certainly not as part of the normal dev process. permalink fedilink source parent hideshow 3 child comments replies: [–] tempest@lemmy.ca 1 point 6 months ago (2 children) They shouldn't and we know that but this is hardly the first time that story has been told even before LLMs. Usually it was blamed on "the intern" or whatever. permalink fedilink source parent hideshow 2 child comments replies: [–] BorgDrone@feddit.nl 2 points 6 months ago (1 child) This isn’t just an issue with a developer putting too much trust into an LLM though. This is a failure at the organizational level. So many things have to be wrong for this to happen. If an ‘intern’ can access a production database then you have some serious problems. No one should have access to that in normal operations. permalink fedilink source parent hideshow 1 child comment replies: [–] tempest@lemmy.ca 2 points 6 months ago* (last edited 6 months ago) Sure, I'm not telling you how it should be, I'm telling you how it is. The LLM just increases the damage done because it can do more damage faster before someone figures out they fucked up. This is the last big one I remembered offhand but I know it happens a couple times a year and probably more just goes unreported. https://www.cnn.com/2021/02/26/politics/solarwinds123-password-intern Why would an intern be given prod supply chain credentials, who knows. People fuck up all the time. permalink fedilink source parent
[–] tempest@lemmy.ca 1 point 6 months ago (2 children) They shouldn't and we know that but this is hardly the first time that story has been told even before LLMs. Usually it was blamed on "the intern" or whatever. permalink fedilink source parent hideshow 2 child comments replies: [–] BorgDrone@feddit.nl 2 points 6 months ago (1 child) This isn’t just an issue with a developer putting too much trust into an LLM though. This is a failure at the organizational level. So many things have to be wrong for this to happen. If an ‘intern’ can access a production database then you have some serious problems. No one should have access to that in normal operations. permalink fedilink source parent hideshow 1 child comment replies: [–] tempest@lemmy.ca 2 points 6 months ago* (last edited 6 months ago) Sure, I'm not telling you how it should be, I'm telling you how it is. The LLM just increases the damage done because it can do more damage faster before someone figures out they fucked up. This is the last big one I remembered offhand but I know it happens a couple times a year and probably more just goes unreported. https://www.cnn.com/2021/02/26/politics/solarwinds123-password-intern Why would an intern be given prod supply chain credentials, who knows. People fuck up all the time. permalink fedilink source parent
[–] BorgDrone@feddit.nl 2 points 6 months ago (1 child) This isn’t just an issue with a developer putting too much trust into an LLM though. This is a failure at the organizational level. So many things have to be wrong for this to happen. If an ‘intern’ can access a production database then you have some serious problems. No one should have access to that in normal operations. permalink fedilink source parent hideshow 1 child comment replies: [–] tempest@lemmy.ca 2 points 6 months ago* (last edited 6 months ago) Sure, I'm not telling you how it should be, I'm telling you how it is. The LLM just increases the damage done because it can do more damage faster before someone figures out they fucked up. This is the last big one I remembered offhand but I know it happens a couple times a year and probably more just goes unreported. https://www.cnn.com/2021/02/26/politics/solarwinds123-password-intern Why would an intern be given prod supply chain credentials, who knows. People fuck up all the time. permalink fedilink source parent
[–] tempest@lemmy.ca 2 points 6 months ago* (last edited 6 months ago) Sure, I'm not telling you how it should be, I'm telling you how it is. The LLM just increases the damage done because it can do more damage faster before someone figures out they fucked up. This is the last big one I remembered offhand but I know it happens a couple times a year and probably more just goes unreported. https://www.cnn.com/2021/02/26/politics/solarwinds123-password-intern Why would an intern be given prod supply chain credentials, who knows. People fuck up all the time. permalink fedilink source parent
[–] ExLisper@lemmy.curiana.net -1 points 6 months ago If you’ve ever used it you can see how easily it can happen. Yes, I can see how it can easily happen to stupid lazy people. permalink fedilink source parent