Rants/ SQL/ Syndicated/ Tech

A Line in the Sand

Timothy Best Practices, Rants, SQL, Syndicated 6 Comments Mar 6, 2009

I am sure I’m not alone in my support of a mixed environment of internally-developed and vendor-designed database solutions.  In my organization we have a Buy Before Build mandate; so long as a third-party product exists that meets most of our customer needs buy it.  We will only internally develop applications and databases that don’t fit that bill or need to interface between different commercial products.

This always makes the Database Administrator’s job an interesting one.  We’re assailed when performance is in the dumper, but when we perform due diligence and find out poor design is to blame we can only make recommendations to the vendor for perhaps an indexing recommendation, schema change, or some other standard alteration that we, the DBA feel, will help in performance.  We will even go so far as proving the change works to this end goal by testing in a comparable non-prod environment and provide them with the data they need to make an educated decision on whether to allow us to make the change or, more appropriately (if access is granted) to have them make the change to their product in our domain.  We give them trace files, execution plans, whatever we can compile for them to reach the same conclusion as we have.

We then sit on our hands and wait while the Application Analysts, Project Managers, Application Managers, Product Users, Manager of the Product Users… pester (yes I said pester, because when you all do it every 3.5 minutes it is not constructive; I’d daresay it’s harassment at that point) us.

 

Them #1: “So, you fix that performance issue yet?”

Me: “I’ve identified the issue, it’s with the product and I’ve provided the vendor with the information to allow them to decide whether to make the change that will take care of this.”

Them #2, 2 minutes later: “So you make that change?”

Me: “Have not heard from the vendor”

Them #3, while standing right next to Them #2: “Why haven’t you heard back yet?”

Me: “Perhaps the phone lines are down in Jakarta.  Perhaps it’s 3 in the morning there and they don’t have tech support 24X7. Do you now need me to do your job and work with the vendor’s helpdesk to ask them why they did not immediately respond to our slowness issue?  We’re not *down*, we’re just slow-ish.  Where do you think this falls on their SLA?  Have you read the SLA?  Do you know what SLA means?”

Them #1, #2, #3 (5 minutes later):  “Just go ahead and make the change.  Why are you waiting for the vendor to make that decision for you?  Chickenshit?”

Me:  “Ah, figured me out.  I’m afraid to do my job, and thereby render the entire product unsupported because I am informed enough, and prepared to do the right thing.”

 

Eventually, after all the analysis, testing, and dare I say AWESOME team dynamics we finally get the response back from the vendor:

 

85%:  “We see that would work, but if you do it, we’ll not support your installation”

10%:  “Um, we can’t open this trace file you sent us.”

5%:  “Our programmer doesn’t understand what you’re talking about.  We don’t have a DBA to run this past, but our SQL Guy (programmer) says that what he built is the right way to do it.  Indexes just get in the way.”

3%:  “We do all our database queries ad-hoc in the C# code so we can be platform independent on the database side.  If you change the schema we’d need to change our application code and that’s not going to happen.”

3%:  “Perhaps you need to upgrade from SQL 2005 to SQL 7  SQL 2005 came out, like 2 years ago dude.  7 is the bomb!” <Editor’s Note> OK, first of all, this is real.  Second, it was from about 2 days ago (do the math.) Thirdly, apparently he is confusing Windows 7 with SQL.

.05%:  “Oh, wow, you’re right.  Yeah either go ahead and make the change and send us the script so we can implement it if we see fit on other installs or we can do it for you.  Good catch!”

 

Before I hear any complaints about my math, yes I know that is 100-something%.  That’s just how ****ed up our vendors are though.  Something% decent, solid application and database solutions providers and the other 100% royal pains-in-the-bajoombas <Editor’s note: run on sentence prevoked by ADD follows> (taking a guess on the anatomy courtesy of Rachel Hawley aka @RachelHawley, formerly of Red Gate and still an all around fantastic programmer who you should hire immediately, if you’re in the UK, give her a call on the telly or the BBC or whatever they use over there for communication.  Crumpets? Dunno, I’m an ingorant product of the public school system in the States.)

So my question is, where do you draw the line in the sand when it comes to doing the *right* thing?  Do you make the change, knowing that it will speed up performance or remedy whatever the situation is at hand?  Do you place the onus 100% of the time on the vendor?  What if the vendor is great at application development but when it comes to RDMBS understanding can’t tell you what “referential integrity” is and refers to SQL Server as Sequel?  Likewise you may know what the right thing to do in the database would be, but you have no understanding of what may be going on in the application.  Perhaps your doing of things right, may be catastrophic 1% of the time in the application.  Or simply-put, the SLA is the law.  

What are your thoughts, horror stories, and suggestions?  I’m tagging Brent Ozar and Tom LaRock and ask that they tag two additional database professionals from there.  We’ll call it a Ponzi Scheme of Vendor Love and Hate.

 

(Also, I hope that I used the right reference for Bajoombas.  Rachel would not clarify, nor would she show them to us and would not sit still long enough to pull them in on Google Earth yesterday.)