Don't ever ask.....


 
J

JKalchik

TVWBB Diamond Member
what else can go wrong.

Seems to be my week for hardware problems. I've already fixed 2 servers (32+ CPUs and over a terabyte of RAM between them,) and getting ready to work on a 3rd that's starting to throw hardware events.

So, 1 server in particular. Crashed & burned early on Tuesday morning. 3rd party maintenance company figures 2 DIMMs and a CPU. No problem, when will you be out to fix? Sorry, we're not stocking parts for that system in the Twin Cities any more (turns out my client has the last chassis of that sort here.) Took until Friday morning to get DIMMs and CPU on site (that's also when I found the 3rd system with issues....) And turns out that those aren't sufficient to fix the problem, a cell board also needs to be replaced. Amazingly enough, they had one available as of this morning. The fine gent who was sent out with it HAD NEVER TOUCHED A SYSTEM LIKE THIS BEFORE. I showed him the chassis, verified it was powered down, said it's all yours, let me know if you need me to do something. I watched him out of the corner of my eye for about 5 minutes before I approached him. "I'm not trying to be offensive, but you've never touched one of these before, have you?" "Nope." "Okay, we're doin' this together." Side note: I've only been working with this hardware line back to about 1992, and that includes hardware that was ancient then, not my first rodeo. After shuffling parts and running diags, it's up and running.

No, I am NOT asking what else can go wrong. I still have to be up to move workloads off the next problematic host before it burns, falls over and sinks in the swamp. I really don't need more "opportunities."
 
Oh, fer cry eye... ;)

Important people like us software developers don't want to hear your sob story. Just solder the resistors or whatever it is you nerds do and get out of the way.;)
 
Hey, what about us network guys....always have to come up with PROOF it isn't a network problem before anyone will look at servers or apps?!?! :)

JK, bad things come in threes? Maybe you're out of the woods (throwing salt, knocking on wood, etc.) ;)

R
 
Rich, based on my years of experience:

1. It's probably a network problem.
2. It's probably a hardware problem.
3. If you come into my office and say the problem's with software I wrote, you're wrong. ;)
 
Last edited:
I wish I had a nickel for every time a developer told me that there was nothing wrong with their code. I'd have retired a rich man years ago. Return codes, debuggers and system logs don't lie. Then again.... I've been required to work with off-shore development for far too long. I need to stop, or I'm gonna be reaching for the bourbon far too early this morning.

One that really sticks in my mind was "it compiled without errors, it's FINE." "Uh..... lessee.... you've got a logic flaw here, here, here, and here, and that's just from a 2 minute review." Unfortunately, that wasn't off shore, that was a fairly highly regarded developer who just turned out crap code and blamed everybody else until he ran into me.
 
This ain't my idea of fun. Far from it. In this case, I can definitively rule out network, power, even application code can't do this. :)
 
JKalchik said:
This ain't my idea of fun. Far from it. In this case, I can definitively rule out network, power, even application code can't do this. :)
Click to expand...
Well, whatever it is just have it fixed by tomorrow morning. I have code to write and I don't like being inconvenienced. Although odds are that the network'll be messed up anyway. ;)
 
Brad Olson said:
Well, whatever it is just have it fixed by tomorrow morning. I have code to write and I don't like being inconvenienced. Although odds are that the network'll be messed up anyway. ;)
Click to expand...

:D Joys of virtualiation. The virtual machines have been restarted elsewhere, only infrastructure admins have access on this one. Unfortunately... out of 3 servers in this cluster, 1 has crashed twice and 1 is showing hardware issues. Not a comfortable place to be.
 
I'm a copier technician and we deal with companies that range from us not doing anything on an install to them expecting us to be their IT. When the copier stops printing or scanning they always blame us, and it's always an IT issue. I've dealt with some great IT guys but in generally they are unbelievably lazy. On these machines the NIC is typically built into a main control board. I've been asked several times to "just change the NIC." Ok, let me replace this $2000 part (at our expense) because you can't be bothered to come onsite.

I had a car dealership where I was told by their IT department to give the copier a certain IP address. It turned out it was in the middle of the DHCP range and they somehow had no way to do a MAC reservation. A random cell phone somewhere on this giant lot had taken the address. I diagnosed the problem even gave them a guess based on the MAC address that was on that IP address what it was. The Sales manager was yelling a me to "just f*ing fix it!" Good times.
 
Dustin, I'm pretty sure we've all got some war stories. I have a similar tale while I was working at 3Com in the Chicago area about a factory test station taking over the IP address of the shop floor control backup server. "Next Wednesday, the entire factory floor, all 12 lines of surface mount manufacturing, will be changed from static to DHCP addressing. This is not open to debate or interpretation." Honestly, I was surprised that issue took that long to crop up.

I ended up working a printer issue a few years ago (ended up with a cat as a result, but I digress.) Some print jobs coming from a *ix machine were printing corrupted, but only those jobs. Anything sent through Windows printed fine. After a month of troubleshooting, it turned out to be a bad WAN card in the main site router (was corrupting the print stream.) Swapped that one morning before business really got started, end of problems.
 
Dustin Dorsey said:
I'm a copier technician and we deal with companies that range from us not doing anything on an install to them expecting us to be their IT.
Click to expand...
It's just the opposite for me. Over the years a lot of people have assumed that because I'm an IT guy, I know all about the copiers - but other than pushing the buttons and loading the paper, I don't!;)
 
Apparently karma didn't like me teasing JK and Rich...for the first time in 35 years I have to find a job.:ROFLMAO:
 
Dustin Dorsey said:
I'm a copier technician and we deal with companies that range from us not doing anything on an install to them expecting us to be their IT. When the copier stops printing or scanning they always blame us, and it's always an IT issue. I've dealt with some great IT guys but in generally they are unbelievably lazy. On these machines the NIC is typically built into a main control board. I've been asked several times to "just change the NIC." Ok, let me replace this $2000 part (at our expense) because you can't be bothered to come onsite.

I had a car dealership where I was told by their IT department to give the copier a certain IP address. It turned out it was in the middle of the DHCP range and they somehow had no way to do a MAC reservation. A random cell phone somewhere on this giant lot had taken the address. I diagnosed the problem even gave them a guess based on the MAC address that was on that IP address what it was. The Sales manager was yelling a me to "just f*ing fix it!" Good times.
Click to expand...
From 1996 until 2003 I was a technical trainer for Canon USA. Not sure how long you've been a tech
 
I started in 2008. I service pretty much only Canon copiers but the whole range practically as we are an small independent dealer. I've had to go to the training facility in Dallas a few times to get various certifications. Most of the training is online now and it's not that great. Before that I worked for Reynolds and Reynolds repairing circuit boards for printers.
 
Brad Olson said:
Apparently karma didn't like me teasing JK and Rich...for the first time in 35 years I have to find a job.:ROFLMAO:
Click to expand...

Aw, <BEEEEEP>

My job may be safe for the next 2 or 3 days..... I said on an enterprise conf call this afternoon "<environment> isn't your real issue, that's pretty well proven out." Only to be told "You're the only guy who knows that." Yeah, I know. Train backup.
 
You must log in or register to reply here.

 

Back
Top