kinda surprising it didn't happen earlier
post
"We don't need our own backups. The cloud takes care of everything."
Do they have a Go Fund Me or something I can donate to or something to keep them rolling?
Neither Amazon or Islamic Republic of Iran need your money, friend. Buy yourself a nice hat in stead.
So like... the opposite of a data breach, the headlines I am used to reading about "customer data"? Hmmm... Ok.
The only way to get them to delete your data.
At this point, I wouldn't be surprised if a bombing on US territory was cheered if it was an AWS data center (or Meta, xAI, etc). Hell, Iran could probably even find US citizens for sabotage at this point if they just ask, lol.
I expect Iran already hires Americans for unorganized sabotage, I know both Iran and Russia utilises Telegram to hire teenagers "torpedoes" as they're called among mobsters.
It's a fully gangster technique, but it's effective. One example would be when the Israeli embassy in Copenhagen had two kids attempt to throw a grenade their way in 2024 - It's proven to be related to Foxtrot (the largest Swedish organised crime syndicate), which Iran has influence on through one of the leaders living in Iran
the cloud is much more secure because your data is decentralized
- Every major cloud provider when advertising their services over on-prem soloutions.
* But only if you pay for the most redundant storage, which cost all the money.
Never thought I'd be a cheerleader for Iran against my own country. But please, Iran, keep making America better!
Anyone in IT who has ever been near a backup or had to do a DR exercise knows full well not to have a backup in the same geographic area as your primary workload. Usually it’s for natural disasters, but it very much applies to war too.
This is just as much, if not more, on the companies that didn’t have geographicly dispersed backups.
I think pretty much every software dev has used AWS at some time, and when you set up new services, it specifically warns you to use multiple geographic areas for your services. It doesn't say "war" or something, only that it's good for redundancy, IIRC.
So anyway, yes, I follow instructions, but I'm always surprised that you have software people and systems admins who can't even follow basic instructions. I don't know what percentage of people it is, but it feels like over 50% who can't follow even the simplest instructions.
I'm not surprised at all. It costs more to use more regions, both due to literal cost and in time to set it up.
The extra time to set up in more regions is pretty trivial, at least in my experience. It's either an option when setting up the service, or it's something that you need to cover in any case while documenting how to set up the service. But I suspect the devs I'm complaining don't read basic instructions also don't write basic instructions.
As for the monetary cost, that's a business decision. The devs I am complaining about are not making business decisions. They're just ignoring basic instructions.
A not insignificant part of my first azure cert covered different redundancy options and pricing. If the data was that critical, the tenants would/should have mirrored it to another region.
Speaking as a member of a company who has just recently realized that we can have products in regions other than us-east-1, this is not surprising. If UE1 went down the economy would collapse overnight
I've been working on Disaster Recovery for my company and the amount of sleep I'm losing on this is insane in general.
Our devs just write their own IaC stuff so like, they can provision their own databases and shit without us knowing with no backups or customer keys and stuff to back up properly.
It's incredibly hard to reign that behavior in once it's entrenched as well.
You basically gotta take away privileges from everyone and it becomes a huge fight if you don't have organizational support. Usually you have to wait for something to explode spectacularly for that to happen.
Fellow SRE here.
The number of times I've heard "we have replication" followed by "we've tested it...on paper" make me cry.
Trying to convince a CEO that the only way to know for sure that their backups work is to do an actual disaster test is harder than trying to convince my parents that I don't need to eat more.
Our devs just write their own IaC stuff so like, they can provision their own databases and shit without us knowing with no backups or customer keys and stuff to back up properly.
That'd give me nightmares too on top of the DR, hopefully at the minimum they can't spin up resources in anything but a dev/test environment.
Best of luck to you, governance is super important, if it makes you feel any better you're definitely not alone.
War damage to data centers exceeded what AWS services are designed to withstand.
Well good thing it isn't a war then
2+1 isn't this building and the neighboring building. No back up no compassion. (Kein Backup kein Mitleid)
Someone needs to shine a laser at bezos' noggin for a better permanent loss.
top 50 comments