Monday, July 13, 2009

Bad cache

We had some code break down today. Here's the story.

One of the great tricks in the object oriented world is late bound variables. When it's expensive to initialize a variable, or you're not sure you're going to always use it, you don't initialize it until first use.

Public Readonly Property Something() As SomeObject
If _someObject Is Nothing Then
_someObject = DoExpensiveInitialization()
End If
Return _someObject
End Get
End Property
Private _someObject As SomeObject

Another great trick is caching data. If you have some data that you're going to use frequently and that data doesn't change, then load it into shared memory and let all instances of your objects use it without being chatty with the database.

Private Shared s_someLookup As Dictionary(Of String, String)
Shared Sub New()
s_someLookup = GetLookup()
End Sub

So today, we had a whole site break down due to a slight oversight and a misapplication of these concepts. Here's a variation of the code:

Public Shared Readonly Property DataLookup() As DataSet
If s_dsLookup Is Nothing Then SetLookup()
Return s_dsLookup
End Get
End Property
Private Shared s_dsLookup As DataSet

Private Shared Sub SetLookup()
s_dsLookup = New DataSet() ' -- OUR LATE BOUND PROPERTY IS NO LONGER NULL!!!
Dim adp As New SqlDataAdapter(CNN_STR, GetConnection()) ' -- EXCEPTION THROWN

Catch ex As Exception
' s_dsLookup = Nothing ' -- THE SINGLE LINE FIX

' Handle the exception, but we forget that
' we initialized s_dsLookup so now we're dead
' on every call to the DataLookup property
End Try
End Sub

So, code that never failed before, failed today. And I had to track it down. And tracking it down was a PAIN because:
  1. The breakage presented itself far away from the actual problem. Rule #1 - fail fast!
  2. The breakage was not reproducible in any other environment, because the failure was a fluke occurrence.
  3. It was a Monday, we were short staffed, overloaded with hot projects that needed to be done yesterday, with a bug that affects the whole company, and I am neither the author of the code nor the primary developer for the broken site. Murphy's law.
  4. The fix involved the inevitable discussion with management about why these sorts of things happen, and why we use these newfangled development techniques.
Doing tricks for performance purposes always has a tradeoff. Usually it's in the area of maintainability. Not that I'm complaining. Finding these sorts of things and fixing them makes me a better developer, and also gives me great fodder for the blog. I just wish that this stuff wouldn't inevitably happen on Mondays. Or late Friday afternoons.