The day our core switch finally gave up on a Tuesday morning

Last month I came into the office and half the floor had no network access. Turns out our old Cisco 3750 stack had been throwing errors for weeks and I just missed the logs. By 10am the whole building was down and I had to call a vendor to rush us a replacement. Took 6 hours to reconfigure and we lost a days work for 200 people. Has anyone else had a switch die without warning like that?

2 comments

2 Comments

drew_west1mo ago

Man, that sucks. Did you have any kind of backup switch you could've swapped in real quick? I've had a switch die on a Friday afternoon before and it was the worst feeling in the world, lol. The logs thing is so real too, like you check them every day for a month and nothing, then the one week you're busy and poof, everything falls apart. Six hours of reconfiguring sounds brutal, I bet your hands were cramping from all the typing. Losing a whole day for 200 people is a nightmare, I feel for you man.

morgan_lewis181mo ago

Did you ever try setting up a config script to speed up the rebuild? I started keeping a template file on a USB stick after my first switch died and it saved me hours of retyping later.