It's the late 90s and I was working in IT for a small (200 employees) company.
One of my tasks was performing daily data backups, which ran for about four hours and started after everyone else had left for the day.
We had an intercom system in the building, where if you picked up your phone and dialed a specific extension, you could page someone, or make announcements so everyone could hear.

One night I was in the process of running Ethernet cable from switches on the second floor down to individual terminals on the first floor, so there was a lot walking around the building.

I like to have music when I'm working, but I didn't have an MP3 player at the time, so I thought... "I have music on my PC and no one is around. Why don't I just...", and I went around and cranked the volume knob on every ceiling intercom speaker in the whole building. Then I dialed into the intercom from the phone in my office, put the handset up as close as I could to my PC speakers, and boom, music throughout the building.

Cut to a few hours later, it's late, I'm tired, the backup is finished, so I close down the system, hit the lights, lock the doors, and go home.

The next day, I come in, not thinking about anything from the night before, and get to work. About 11:00am, the owner's secretary got on the intercom to page someone.... and blew everyone out of their seats. There was an almost collective, "What the fuck??!!" from everyone, and an "Ohhh, shiiit" from me, because of course I immediately knew what happened.

I ran around like a madman, turning the volume down on all of the speakers that I had forgotten the night before, while saying, "No idea... Yeah it's crazy... No no, it's fixed now..."

No one ever found out exactly why the volume was so loud that morning.

all 26 comments

sorted by: hot top controversial new old
[–] 1 point 16 minutes ago

I once took down a production server with a script that was an unintentional resource thief. The "oh sh-t" was watching the script grind to a halt... and then the phone started ringing.

Borderline on the non-destructive there because the server recovered fine after a reboot and nothing was lost except uptime. Sometimes a server would choke anyway, and the company assumed it was that, but that time it was my fault.

I had to answer more than the one call regarding that, feeling sick to my stomach, so I didn't escape unpunished.

  • source
  • [–] 7 points 2 hours ago*

    Worked in an ESA lab with cleanroom. My work had to do with plants, their composition etc.

    Naturally I had to dry some plants before throwing them into the analyser, but my designated oven was occupied.

    Fear not, I thought, there is another oven over there. I put my samples in and call it a day.

    Come over the next day, kinda late, lab chief looks at me and is only like "ummm, you should head over to the offices and see big boss"...go over, boss guy very angry. Turns out, the other oven in the lab was reserved for ExoMars parts. Like, they disinfect the shit out of it for weeks, like, take it apart, haul every screw to the cleanroom, clean it, autoclave it, clean it again and repeat that a billion times over, certify it and clean it again. So I kinda destroyed the work of months, maybe I alone was responsible for one of the many launch delays (probably not, but I've never been told).

    Arrogant young me in my first job basically just was like "maybe put a sign on it next time, lol", boss guy pauses and says "maybe you're right". So we reclean the shit out of the oven and put a sign on it. My "punishment" was to spend a lot of time in the clean room to clean some ExoMars parts. I thought "cool, stuff I had in my hands is going to land on Mars".

    Fast forward some years, Russia invades Ukraine, the pipes I cleaned will never go to Mars 😢.

  • source
  • [–] 11 points 3 hours ago

    It wasn't at work, but I think someone will find it funny regardless.

    During my time at university I had a web development exam for which we had to do a group project, you guessed it, making a website. This included having a bunch of different input fields, and the requirement was to properly validate each one and provide clear feedback to the user.

    While adding some inputs I was focused on having all the checks in place correctly, and so I decided to put some placeholder text for the errors and do a pass later changing them with more informative ones. Which I did, but I forgot about one.

    So, presentation day comes along, the professor is looking at our website, asking us to show him that we had all the checks in place and correct. He asks me to input an invalid date in a calendar widget, which I happily do, knowing I 100% had all the required checks in place. The verification runs and fails, showing the error popup I set up, which read something along the lines of:

    "date's fucked, mate"

    Shit. My placeholder message. Which I obviously forgot to change. I was mortified. Luckily the professor chuckled and said that technically the check worked so that was fine.

  • source
  • [–] 5 points 3 hours ago*

    In March of 2017 we were running several unpatched server 2012 systems with just about every port open to the internet. Uptimes of 500ish days kind of servers.

    Took a couple weeks to patch everything and lock it down. Mega glad I did because May in 2017 was an interesting month for IT.

    Also, community plug !talesfromtechsupport@lemmy.world exists please share stories.

  • source
  • [–] 7 points 4 hours ago

    "Shutdown -P now" in the wrong terminal window and shut down the production server on the other side of the country instead of the test server on my desk. No one noticed but I had to call the IT guy on site to turn it on.

    Yanked a cable in an electrical cabinet we were installing and blacked out the entire workshop at the 500 level in the mine. Had to go to the electrician and ask for the key to the transformer station and turn it back on.

  • source
  • [–] 20 points 6 hours ago*

    Decade or so ago, logging into a work computer and suddenly the screen is just tiled with the word "virus", but a bit janky looking like it was a screenshot of the word that was enlarged and tiled with no thought to whether the right side of the image lined up with the left and such - it was a very 90s cheesy movie like depiction of what happens to your comp when you get a virus, like you'd expect the next scene to be some shady hacker on the other end exclaiming "I'm in!"

    Then again, I don't know jack fuck about viruses, and the machine is behaving super weird, so I unplugged the internet cable from the back, and called my manager over. He had a similar "...is this for fuckin real?" reaction that I did. Mind you, this is a hospital, so if one of our computers is fucked, this is a big deal for potentially leaking patient info, so even though it had that kind of slapstick appearance, no one was fucking around with it: so, manager pushed it up the chain, and after a few mins a fucking SWAT team of IT guys comes in and starts doing their own assessment.

    After some deliberation, they start troubleshooting - keystrokes and whatnot don't seem to be responsive. One of them eventually reaches back behind the monitor and 'poof' the screen pops back to the normal login screen.

    So, that facility had a fancy custom login screen, which was some generic desktop background type scenery and a wall of text that none of us ever put any thought into. One of the snipits on that wall of text was instructions on what to do if you suspect your machine has virus (which was basically what I did - leave it running, unplug it from the internet, and call IT... hooray, I'm not completely stupid!). What ended up being the cause of the "virus" tiling was the fucking VGA cable slowing worming its way out of the back of the monitor - when I got to it, that thing was hanging on by a thread, and touching the desk it was on was just enough of a bump to make it cross that line between having enough of a connection to do its job vs not. That micrometer of pull of on that cable wasn't enough to completely cut its signal, but it wasn't enough to produce a normal display either: it just picked a random spot on the screen, and filled the monitor with that repeating image. The spot it just happened to do this to was the word "virus" in the custom login text explaining what to do if you think you have a virus, lol.

    TLDR, got my management chain from my dept to the top dogs running the hospital, and the entire fucking IT department to swarm to my unit over a potential data breach that ended up being a wiggly VGA cable.

    Whoops.

  • source
  • [–] 19 points 6 hours ago (1 child)

    Years ago I was working as a pharmacist in a very busy pharmacy in a city center. This was during the opioid epidemic in America and being a high volume store, had a lot of regulatory oversight on our controlled substance dispensing.

    One Friday evening a young kid came in with a prescription that he said was for his grandmother. It was for a lot of oxycodone from a city two hours away. He had two girls with him when he dropped off the prescription and had never filled with us before, and wanted to pay cash. We had a policy that we didn't fill prescriptions for maintenance controlled substances for new patients >100mi from our address. We only got so much of each controlled substance so we adopted the policy so we'd have enough for our regular patients. I told him we couldn't fill it. He started to get angry and mouth off but left.

    Cut to Monday which was my next shift and there is a folder with the DEA logo on the front of it in front of my terminal. I ask what it is to someone who worked the weekend and they said an agent dropped it off. I open it and the only thing inside is a business card slotted into those slits you sometimes see on the face of one of the pockets. I call the number and after some introductions the agent asks me if I was the one who denied the prescription on Friday. I confirm and the guys says that it was part of a multi-jurisdiction sting operation to catch a mid-level dealer who was buying scripts from people. He told me I wasted a bunch of their time and next time to just fill it. I told him it's emphatically my job to not do that and that I didn't appreciate him using my store since we had a bunch of other issues that local police weren't dealing without him adding to it but he just hung up. Whatever.

    My heart dropped when I saw that folder though and I'm sure that is what he wanted when he did it.

  • source
  • hideshow 1 child comment
  • [–] 8 points 3 hours ago

    Wait, so the DEA agent essentially scolded you for NOT contributing to the problem and ordered you (well, tried to) to be part of the problem going forward? What a fucking joke. If he wanted you to participate in this super important multi-jurisdiction sting he could've contacted you (as he's clearly capable to do) beforehand. The only way the agency comes out of it not looking like a clown car is if there was no sting and he was an impersonator, with falsified documents and a fake folder, looking to establish a steady drug source. It might have been worth it to report it to his boss, but maybe I'm too European to properly understand all the risks involved in doing so.

  • source
  • parent
  • [–] 26 points 7 hours ago (3 children)

    I was working on prescribed fire as in controlled wildfire to support ecosystem function. The wind changed in expectedly. The fire jumped containment. We switch into actual firefighting mode for a solid 1/2 hour. Then the wind switches back to predictions. Problem solved. Mop up of the slopover still takes hours and it gets dark.

    As we get back to the truck, keys are dropped and kicked in an attempt to catch them. There is tall grass everywhere. We search for an hour in the dark after an exhausting day for our only way to go home. Keys are found. We work at dawn the next day. Not my worst day in the office, just the least predictable.

  • source
  • hideshow 3 child comments
  • [–] 9 points 5 hours ago (2 children)

    Always carry a magnet with you.

    I work in installations where vegetation is allowed to grow nearly wild, except for the mandatory twice a year cut and I've had a situation like the one you describe, with a very important key falling out of a pocket.

    We also usually carry a large magnet around, to pull open iron covers. It's about 15cm wide and 2cm thick. It can pull.

    The key just vanishes into the plants covering the ground. We fetch the magnet, point it into the general direction of where we think we heard the key fall and cross our fingers. We were rewarded with a very satisfying metalic ding after a few passes.

    Then it was the hassle of prying the key from the magnet.

  • source
  • parent
  • hideshow 2 child comments
  • [–] 33 points 7 hours ago

    Mid '90s, was doing something on the company's NT 4.0 PDC. The thing about NT 4.0, when the screen locked it would show the "Press Ctrl-Alt-Del to log in" screen, but if you hit a key or moved the mouse quick enough it wouldn't actually lock and wake up again. So, as I'm sitting there waiting for the server to do something the login screen shows up. I'm in the zone, so I lightning quick give it the three-finger salute, type in the password (usa1, security wasn't super-high) and hit enter. Another thing about NT 4.0, if you hit Ctrl-Alt-Del when the machine was not locked you'd get a box similar to XP's, with options to lock the computer, log off, shut down, etc. Now, you'll notice one of the options, shut down, starts with the letter "S". And the key to press to select that option is also "S". And you'll further notice that the password I typed in also has an "S". But all is good, it asks you to confirm before it actually shuts down, right? Except the shortcut to accept that option is Enter, which I hit after entering the password. So yeah, I shut down the company's PDC. Fortunately it was late in the day and most people had already left so the impact was minor. But I was doing some pants crapping for about five minutes while the machine came back up.

  • source
  • [–] 16 points 6 hours ago (2 children)

    So for a game I published, there are male/female characters with what I called undermeshes, which is basically underwear that's painted onto the skin texture. By design, there is no "nudity", as the underwear is the most exposed you can get.

    At the time, there were two torso 'top' textures, both created from a base design. One was a sports bra looking thing for women, and just a bare (notably not hairy) chest for men.

    Things were getting crunchy and I had a few NPCs that I was trying to dress differently for a town to make them distinctive. There was an unused semi-low-cut dress asset so I put that on a single NPC, but it looked funky with the sports bra.

    So I assigned the male torso top texture to the lady so the bra wasn't showing, made sure nothing scandalous was visible, and moved on.

    Game gets released.

    Weeks later, I'm troubleshooting a quest dialogue bug, and recalled this particular NPC has different lines (and animations!) if you're a specific sex/race.

    I swap races and do the whole quest again, only to (re)discover that the reason the dress was unused was because it wasn't 100% properly set up. So when the NPC moved about with the different animations, the dress didn't move with her, making her temporarily very topless and with the bare male texture, showing a lot more than she was supposed to be able to.

    In short, this is quite bad and could have resulted in a not-too-happy e-mail from Valve if someone else found it.

    Fortunately (AFAIK) I'm the only one who caught it but it was a cold chill followed by an absolute panic hotfix.

  • source
  • hideshow 2 child comments
  • [–] 18 points 7 hours ago (3 children)

    I don't have any non-destructive ones. I have plenty of destructive ones if you are fascinated, however.

  • source
  • hideshow 3 child comments
  • [–] 13 points 6 hours ago (2 children)
  • [–] 16 points 6 hours ago* (1 child)

    So I work IT and I am damn good at it, relatively speaking.

    Recently, I had a caller contact me, she was a coworker but in a different department, about an issue they were having with their laptop of it not allowing them to use some tool or something? I forget the specifics. But I checked the laptop and saw it had an uptime of like a month and a half. So I thought, oh just give it a reboot.

    I sent a command "RESET COMPUTER" and as it turns out that was a fucking factory reset! So she then lost absolutely everything off her laptop. I got off with no real issue as all my boss did was give me a stern talking to, but no write up or pay dock or demotion or whatever, but holy shit that coworker? She was pissed at me.

    I made it up to her by taking her out for drinks and letting her spend all my money, sobbing.

  • source
  • parent
  • hideshow 1 child comment
  • [–] 1 point 58 minutes ago

    Not your problem. She could have lost everything to an unfortunate drop of her laptop on her way to a meeting room five minutes later anyways. There should be backups of the device. Or all work needs to be saved/synced to a network drive regularly and it should be made clear to everyone that one doesn't save important data locally.

    It was a surprise test of her business continuation planning.

  • source
  • parent
  • [–] 10 points 7 hours ago* (last edited 7 hours ago)

    First thing that comes to mind is driving semi in Saskatchewan once I came across an older slower moving car (a Chrysler K car) full of people on the highway. I pulled out to pass them, and as I was passing the car I saw in the mirror that they started to wander into my lane. I ended up putting half the truck into the ditch to avoid them getting under my trailer. I had 9 tires on the grass before they realized what they were doing and swerved back into their lane. They had wandered far enough over that they were completely in the oncoming lane at that point, leaving me nothing but shoulder and ditch.

    I was fully loaded (around 80,000 lbs) If I hadn't watched my mirrors as closely as I did and they got under my trailer they would have been nothing but a smear on the road.


    Edit: Another time When I was 18 I was working at the Wpg international airport. A baby had died in a house fire up north and it was sent down to Winnipeg for an autopsy. I guess the people working up there had run out of cargo tags so they had put a regular baggage tag on the bag the body was in, and neglected to tell anyone. My job was to unload aircraft, so seeing a regular baggage tag I just tossed it onto the cart and it went to where you pick up your luggage. It was there for about 40 minutes before an ambulance showed up asking for it, and none of us had any clue about it. Thankfully nobody snagged it because they would have been in for one hell of a surprise.

  • source
  • [–] 19 points 8 hours ago

    Pretty awesome you put your music on the entire building ngl. Glad you didn't get caught too.

    I can't really remember anything comparable of my own, but enjoyed the read. ty OP.

  • source
  • [–] 2 points 5 hours ago (2 children)

    I did the very classic "sw t al v 320" which effectively isolated an entire floor.

  • source
  • hideshow 2 child comments
  • [–] 2 points 2 hours ago (1 child)

    what the hell are you talking about

  • source
  • parent
  • hideshow 1 child comment
  • [–] 1 point 1 hour ago

    "switchport trunk allowed vlan 320"

    The classic mistake: On a trunk port uplinking a floor's switch (Cisco IOS), running that command replaces the list of allowed VLANs with only VLAN 320, instead of adding 320 to the existing list.

    A trunk carries traffic for multiple VLANs to other floors/upstream, suddenly permitting just one VLAN effectively cuts off every other network on that floor

  • source
  • parent
  • [–] 8 points 8 hours ago

    Haha that's great! I cant think of one off the top of my head but I'm sure I have plenty.

  • source