When things stop working, the solution isn’t always obvious

When we develop web sites, we stumble across problems, right? And sometimes, just sometimes, we do learn from them and know what to do next time. At other times, though, the problem is something we never would have expected.

I thought I’d share two of these situations where the problem was far from what everyone thought at first.

Incomplete files

About a year ago, working on a web site where we Gzipped the JavaScript files in production, one day I got the message that the interactivity was all messed up on the live web site. I started sifting through the code, and after some initial tests, the problem was that the whole content of the JavaScript files wasn’t delivered (for all web browsers, not just the notorious IE Gzip bug). For some reason, they were only parsed about halfway through, and then stopped.

The guy responsible for the servers did some intense research (another problem factor was that the servers were stationed abroad). The conclusion he came to was that the heat and hot exhaust air coming from other servers in the rack. This issue was addressed, and as far as I know the problem was gone.

This has never been scientifically verified, though no matter what was the actual cause, it was from any of the the usual suspects.

Cleaning

The architect in my current project told me about when he was working at the major airport in Stockholm, Arlanda. One of their systems went down for a while, at the exact same time each morning. It automatically restarted, but that wasn’t sufficient, so someone had to manually restart it again.

They started thinking that it had to be an outside factor causing this problem every day, so they went to the server room at the time of the usual server outage. Lo and behold, when the clock strikes, a cleaning woman walks in, pulls out the cord for the server, plugs her vacuum cleaner in and starts vacuuming. Once done, she put the server cord back in the outlet again and left. :-)

The moral

Sometimes, things are more FUBAR than you can even begin to fathom. :-)

Posted in Developing,Fun,Technology |

Leave a Reply

Your email address will not be published. Required fields are marked *