Technology

No, we had Sentinelone take down our company a few months ago. Granted, not a global outage, but it's something similar. I'm sure that if you went back in news archives, you'd find articles about major Sentinelone outages. I think Crowdstrike is just the biggest one in recent history. It's certainly not unprecedented.

It feels like a pattern though. I’ve not seen too much from them but they seem to be saying factually correct stuff. But neither worded correctly nor at the right time.

Even if that's the case, how is it Crowdstrike's place to call these other companies out for claiming something similar will never happen to them?

I agree completely, which is why I added that last sentence in an edit. This is a bad look for CrowdStrike, even if I agree with the sentiment.

Thus far, it had only ever happened to CS.

Everybody fucks up now and then. That's my point. It's why you shouldn't trust one company to automatically push security updates to critical production servers without either a testing environment or disaster recovery procedures in place.

I doubt you'll find any software company, or any company in any industry, that has not fucked up something really important. That's the nature of commerce. It's why many security protocols exist in the first place. If everyone could be trusted to do their jobs right 100% of the time, you would only need to worry about malicious attacks which make up only a small fraction of security incidents.

The difference here is that CrowdStrike sold a bunch of clients on the idea that they could be trusted to push security updates to production servers without trsting environments. I doubt they told Delta that they didn't need DRP or any redundancy, but either way, the failure was amplified by a collective technical debt that corporations have been building into their budget sheets to pad their stock prices.

By all means, switch from CrowdStrike to a competitor. Or sue them for the loss of value resulting in their fuckup. Sort that out in the contracts and courts, because that's not my area. But we should all recognize that the lesson learned is not to switch to another threat prevention software company that won't fuck up. Such a company does not exist.

If you stub your toe, you don't start walking on your hands. You move the damn coffee table out of the pathway and watch where you're walking. The lesson is to invest in your infrastructure, build in redundancy, and protect your critical systems from shit like this.

House of cards is exactly right. At every IT job I've worked, the bosses want to check the DRP box as long as it costs as close to zero dollars as possible, and a day or two of 1-2 people writing it up. I do my best to cover my own ass, and regularly do actual restores, limit potential blast radii, and so on. But at a high level, bosses don't give AF about defense, they are always on offense (i.e. make more money faster).

This is the first time I've heard someone call it a house of cards and I think that fits it perfectly!

Number fifteen...

Microsoft gave CrowdStrike unfettered access to push an update that can BSOD every Windows machine without a bypass or failsafe in place. That turned out to be a bad idea.

CrowdStrike pushed an errant update. Microsoft allowed a single errant update to cause an unrecoverable boot loop. CrowdStrike is the market leader in their sector and brings in hundreds of millions of dollars every year, but Microsoft is older than the internet and creates hundreds of billions of dollars. CrowdStrike was the primary cause, but Microsoft enabled the meltdown.