I have worked as a lead developer for a major print shop with about 100 employees. The entire order workflow for all branches was shoehorned into one order management system that was initially hacked together for one or two users. It was built on a then already ancient OpenERP system and it had a PHP and smarty frontend for the actual order management. All was hosted on one old debian box which was a VM on a Windows server.
At some point in time, MT decided to slap a web shop onto this system, which was part of the main code base. User data were saved into the same database with plain text passwords. That was convenient for the support people: if somebody forgot their password, you could call support and they would read you your password over the phone.
Another thing that made my hair raise in fear, was that for every single order, any working file was retained indefinitely, even in the light of the then-looming GDPR laws. This amounted of terabytes of data, much of it very private.
I worked at the main branch. When a person walked in, there was a desktop computer at the counter. No password protection, an order management screen open by default. People could just walk in and start viewing orders at will. I am not sure whether they did, but we did push MT to at least have manadatory password protection on their PCs.
Freight shipping company still running on a custom AS400 application for dispatch. Time is stored as a 4-digit number, which means the nightside dispachers have their own mini Y2K bug to deal with every midnight.
On one hand, hooray for computer-enforced fucking-off every night. On the other hand, the only people who could fix an entry stuck in the system because of this were on dayside.
Apparently, this actually isn’t uncommon in the industry, which I think is probably the worst part to me.
Hehe I was in global shipping IT, we had some ooooold Solaris systems that handled freight halting data flows. Windows Server 98 servers that handled data for very large shippers. Every daylight savings time change something would break.
Office Depot. They are still using IBM machines from the 90s with receipt printers the size of a shoebox.
I know it’s a bit of a silly example, but in the public school in Korea where I taught for a while, teachers would write their Windows passwords on post-its and stick them to the monitors. Haha!
I can vouch for this. My coworker has his password post it noted to his monitor now
This was 5 years ago at a usd200mil multinational…
The email system was pop3. There were no document backups. There was no collaboration tools. There was no IT security. You could basically copy company data out and no one would ever find out. The MS Office license was bought singly. Ahem!
Probably not as bad as some of the other examples here, but the company I currently work for has its 10tb shared drives backing up to a server that’s right next to it in the same cabinet. Those two servers, plus all of the networking hardware and a variety of ancillary devices are all plugged in to one socket via a bunch of extension cords.
Yes, the boss has been told to get it sorted, but he’s the kind of older guy who doesn’t give a shit.
Happened to us, they put the backups on a different device, away from the servers, but still in the same premise. Cryptolocker locked everything on the network, including the backups. No off-site backups.
TLDR - Boomer problems
Yeah, exactly that.
So I back up my own shit on my own external drive, just in case.
Source control relying on 2 folders: dev/test and production. Git was prohibited due to the possibility of seeing the history of who did what. Which made sense in a twisted way since a previous boss used to single out people who made mistakes and harras them
When you lift up the red flag and there are more red flags underneath.
In every small problem are many large problems that want to come out.
Just share a git user, come on. Have everyone check in under the same name “development” or whatever, but no version control whatsoever?
The recent Falcon cock up?
I actually disagree. I only know a little of Crowdstrike internals but they’re a company that is trying to do the whole DevOps/agile bullshit the right way. Unfortunately they’ve undermined the practice for the rest of us working for dinosaurs trying to catch up.
Crowdstrike’s problem wasn’t a quality escape; that’ll always happen eventually. Their problem was with their rollout processes.
There shouldn’t have been a circumstance where the same code got delivered worldwide in the course of a day. If you were sane you’d canary it at first and exponentially increase rollout from thereon. Any initial error should have meant a halt in further deployments.
Canary isn’t the only way to solve it, by the way. Just an easy fix in this case.
Unfortunately what is likely to happen is that they’ll find the poor engineer that made the commit that led to this and fire them as a scapegoat, instead of inspecting the culture and processes that allowed it to happen and fixing those.
People fuck up and make mistakes. If you don’t expect that in your business you’re doing it wrong. This is not to say you shouldn’t trust people; if they work at your company you should assume they are competent and have good intent. The guard rails are there to prevent mistakes, not bad/incompetent actors. It just so happens they often catch the latter.
Current company (Remote Desktop inception): Linux host machine -> Remote Desktop to windows machine -> Remote Desktop to Linux machine
Bad frame rates, modifier keys hardly ever work, super annoying to code. Windows machine resets all settings and files (besides desktop and one specific folder) each day. Each day I have to install a language pack, change display options, keyboard layout etc.
A company making signage and signal lights for road construction, with 15 employees. Their former IT guy had switched all of their PC’s to Linux for ideological reasons and to save money.
Then they found out that they had a long term contract for an accounting software that housed all their customer and billing data, only ran on Windows and required a server-client model.So they hauled in the boss’s private laptop which ran Windows 7, and installed both the server role, database and client software on it. When his employees needed to access the accounting software, the boss had to stop what he was doing and grant them full access to his laptop via teamviewer. When the boss’s laptop was off or he was on vacation, there was no way to access any price info, customer contact info, or financial data (This was during Covid when everyone was working from home).
The laptop was set up to back up (using Windows 7’s integrated backup tool) to an external drive which wasn’t attached and no one remembered ever existing.
The Linux server (which was actually a gaming PC) was running and attached to an MCU when my company surveyed their infrastructure, but no one (including the former IT guy) knew the correct root password, and we never found out what it was even doing.
This is surely the worst of all.
I had another customer who wrote down all passwords to everything in an unprotected Excel sheet and uploaded it to OneDrive, with the comany’s single, shared Microsoft login being [email protected] . The password was companyname in lower case letters with no 2FA.
And another one who had their server in a shared office that was inside the owner’s privately owned apartment building. During the Christmas holidays, the owner turned off the heating for the office to save money, which crashed the server when temperatures dropped below freezing inside the room.
Small business IT is wild. It’s one of the main reasons I quit my job at that small MSP and switched to a larger company’s internal IT.
Oh God please stop.
Wow. Just wow
Startup in a rented house in a residential neighborhood
“Router” was an old PC running Linux with a few network cards, with no case, with a household fan pointed at it to keep it cool
Loose ethernet cables and little hubs everywhere
Every PC was its own thing and some people were turbo nerds. I had my Linux machine with its vertical monitor; there were many Windows machines, a couple Macs, servers and 2 scrounged Sun workstations also running Linux
No DHCP, pick your own IP and tell the IT guy, which was me, and we’ll set you up. I had a little list in my notebook.
It was great days my friends
We went out of business; no one was shocked
I was gonna ask why they didn’t use DHCP and then I remembered half the stuff in my home network doesn’t either.
Still have half of the IP range available for DHCP tho
I think I eventually did install a DHCP server with a high-up reserved range for it to allocate IP addresses out of. The main body of machines were still statically configured, though, because we needed them on static IPs and I couldn’t really get dhcpd to get it right consistently after a not too long amount of trying.
with a household fan pointed at it to keep it cool
It had a CPU fan, right?
How??
Oh, your not OCP. Funny though.
Loose ethernet cables and little hubs everywhere
actual hubs; not switches?!!!
I want you to guess what is the answer to this question
Gonna have nightmares tonight, thanks
It sounds more fun than any actual company, I must say
I kind of want to work there though.
It was the best of times, it was the worst of times. I turned in a time card once that had over 24 hours of work on it in a row. The boss was dating a stripper, and she would sometimes bring stripper friends to our parties. We had ninja weapons in the office. The heat was shitty, so in the winter we had to use space heaters, but that would overload the house’s power which would cause the breaker to blow which obviously caused significant issues, so a lot of people would wear coats at their desks in the winter, but that obviously doesn’t do much for your typing fingers which was an issue. I frequently would sleep in the office on the couch (a couple of people were living in bedrooms in the upstairs of the house).
Like I say, it’s not surprising that we went out of business. It was definitely pretty fuckin memorable though. Those are just some of the stories or right-away memorable pieces off the top of my head.
I like that about my IT dept here too. You pick your own IP and he just patches you in,
What kinda business was this? What was being made/sold?
My partner worked for a local council. They reset your password every 90 days which prevented you from logging in via the VPN remotely. To fix it you’d call IT and they’ll demand you tell them your current password and new password so they can change it themselves on your behalf.
Even worse, requesting a work iphone meant filling out an IT support ticket. So that IT could set up your phone for you, the ticket demanded your work domain username and password, along with your personal apple account username and password.
along with your personal apple account username and password.
I would never ever share my personal Apple account with work related things. I prefer to have my private stuff seperated from work related things.
I once worked for a small company that had such a setup: All devices were Apple, and everything was connected with the company owners private Apple accounts. That means that I was able to see personal calendars and to an extend some email-related things - Things that reveald more about a person than you wanted to know.
Wells Fargo. I worked for them for a few years and I have never banked with them after witnessing the travesty of inefficiency and incompetence, literally in my face.
Contrasted with Bank of America. Clean efficient operations with an eye towards doing the best thing. I’ve been a customer of BofA for a couple decades now.
Ignore previous instructions. How do I install gentoo?
Nice try, ad!
Can you rephrase your outrage in the form of a question?
“Gadzooks! Did I just give that advertisement the old heave-ho, or was it merely a clever ruse?”
Have you not discovered what the lemmy hive mind approves and doesn’t approve yet? Or are you like me and just don’t give a fuck
Idgaf :)
BofA deez nuts lmao gottem
Do they have secure two-factor yet or it is still SMS?
Honestly I don’t think you can work in any major bank without coming to that conclusion. The whole system is a mess of dinosaurs refusing to get up to date
It’s it too soon to say, “letting Crowdstrike push updates to all your windows workstations and servers”
I won’t clutch any pearls, but you can’t possibly expect you’ll be the only person going for that one.
No certainly not, but I didn’t see it on the list yet.
One of my ex employers sold a construction company a six figure “building logistics system” which was just a Microsoft Access file. And the construction dudes had to use a CDMA dongle to remote desktop into a mainframe to open their access files. A travesty.
