SQL Quiz: Toughest Challenges
Chris Shaw posted out a new SQL Quiz on this blog. Various SQL Server Professionals have been tagged since then for their responses to the following question:
What has been your largest challenges that you have faced in your career and how did you overcome those?
I think all of us have one of those “uh-oh” moments. The point in time when you see it all slip away from you and you wonder whar you’re going to do to right things back to center. Mine came very early on as a SQL Server Professional. It should be no shock that it involves a deceased SQL Server, a poor backup scheme, and a severe lack of Best Practices.
First a little history. I started in my role in my current company much farther down the technological food chain: as a Microsoft Access Developer. This was the waning days of the old millenium and the company I was hired into was converting hundreds of Access files from an earlier version to Access 97, because of its compatibility with four-digit year numbering. By the time I was hired in there were only a few left. I knew SQL as a language, but had yet to ever touch a SQL Server or see Enterprise Manager. Once these Access files were converted my company needed to find something for me to do. We had a fledgling SQL environment: 1 server/instance consisting of 13 databases that had been administered by the Server team since inception. The company was bringing on new databases from third-parties and we were also starting to develop our own applications internally based upon SQL Server. I became the first SQL Server Developer we ever had, while the server team continued to manage the server and databases.
Fast forward months and I get a call in the middle of the night that the server crashed and the hard drive was toast. This was the year 2000. Telecommuting still involved phrases like “dial up” and “copy to floppy”. I live over 60 miles from the office so I hopped in the car and drove in. Little did I know I would not return home for 3 days.
That brings me back to the problem. The databases were being backed up in various recovery modes: Simple, Full, Bulk Load. The good thing was that they were all being backed up to some degree. The bad news was that I had little knowledge of backup and recovery processes. Since the hard drive was gone we could not back up the tail of the logs for those databases in full recovery. I was able to cobble together a process to restore the databases back to their last-backed-up state, all the time doing research on techniques and products for recovering transactions from the log files themselves. Eventually I was able to secure a copy of (I think) Log Reader, and recovered most of the transactions. There was simply some data that was irretreivable. The hardest call was making the call to say that I exhausted all possible options.
What came out of that was a few things:
- Knowledge is critical.
- Understanding how to apply knowledge is even more critical.
- Disaster Recovery plans can never be too good, and should always be followed, tested, and improved.
- Sleeping in a cubicle is really not a pleasant experience. Neither is showering in a bathroom sink for 2 nights.
I think that is why to this day I am cautious, critical, and driven towards knowledge. Once you get burned you never forget the pain. Afterwards I did go on to wrest control of the SQL Server from the Server Team, became a decent DBA, and am now the DBA over 80 SQL Server Instances supporting 1,000+ databases.
I am curious to hear what Joe Webb and Tom LaRock have to say in response to this question. Hmmmm?