Reinventing the wheel and associated pains

Tuesday, October 17th, 2006

Have you ever visited Starke County, Indiana? I haven’t. I wouldn’t even have heard of it if it wasn’t for the little misadventure I’m about to relate. What lead me to Starke County is a rather bad habit that afflicts most of us, developers. No, not the lousy t-shirts… not the heavy coffee drinking… not the sleepless nights spent hacking… do we have that many bad habits? The one I want to single out today is our tendency to reinvent the wheel. I don’t really know why but we all believe we can do better than whatever framework, library or system currently exists. The pursuit of perfection is a laudable ambition but all too often, it is the way to pain, because, as this post will highlight, we underestimate the effort required to get a decent wheel.

Spinnin’ wheel

I’ll start by contradicting what I just wrote: reinventing the wheel is a good thing. Take web-frameworks. They come in all shapes, sizes and colors. The variety is so abundant there has to be one that fits your needs perfectly — finding it is the topic of another post. Why would you want to come up with your own implementation? As James Brown famously said please, please, please, please to which, far lesser known fact, he added not another flamin’ web-framework!. He was being polite. Yet, when Rails came out, it shook the web-framework arena with its simplicity and elegance. You might disagree with me regarding Rails’ qualities, but you cannot dispute the seismic impact it has had on web development and on the popularity of dynamic languages. If this does not convince you, take a look at Seaside, KPAX or Django. Obviously reinventing the wheel has some advantages, at the very least it fosters ideas and progress. Heck, Linux started as a reinvention of existing UNIX systems.

Wheel of misfortune

But there are contexts when reinventing the wheel is a very, very bad idea. The worst context I can think of is in a corporate environment when the wheel is not your core business. Why? Well, there are several reasons:

  1. you are going to spend time and resources building this new wheel,
  2. you are going to spend time and resources testing this new wheel,
  3. you are going to spend time and resources documenting this new wheel,
  4. you are going to spend time and resources maintaining this new wheel.

See a recurring pattern? To be fair, the first point is rarely underestimated. What tends to happen though is that the promised benefits seem to justify the building cost because the others are underestimated.

The testing cost is always neglected. Think about it: how can you ensure your code receives at least the same exhaustive testing as the one it is supposed to better. Consider the constant QA bashing endured worldwide and around the clock by Struts, Tapestry or Spring. Sure, you will still find the occasional bug. You will lose time and patience working around the issue, or even better, you will fix it and submit a patch. Will someone do this for your code?

The documentation cost… well… what documentation? Up-to-date documentation is the unicorn of software development: a beast of legends. Mind you, if you have a good unit-test coverage, it can serve as documentation and it is guaranteed to be in sync with the code. This still falls short of the documentation of the wheel you are replacing: books, articles or blogs. What’s your plan?

The maintenance cost is directly related to the previous two. The more tested your wheel is, the least it is likely to break down and require maintenance. If it does, the more documentation there is, the easiest it will be for a third party to resolve the issue. Let’s face it, chances are the people implementing your new wheel today will be gone tomorrow, there are so many wheels to reinvent elsewhere!

Back to Starke County

You are probably wondering what is the link with Starke County, so here is my little anecdote. It all started by a bug report sent by the sales team. They were demonstrating the product to some potential customers in New-York. Naturally, they had set the system’s time-zone to EDT/EST in accordance with the local time. The time on the command-line interface to the system was accurate but, somehow, the time on the web front-end was off by one hour. The issue was logged and we got cracking straight away.

To give you a bird’s eye view of the concerned parts of the system, the command line interface is implemented in C/C++, whereas the web front-end is Java based. It turned out the system uses its own home-made time-zone database for reasons I have yet to fathom. This database maps time-zones set by the administrator to UNIX or Java time-zones depending on the interface. We quickly spotted that EDT_EST was mapped to US/Indiana-Starke.

We ran simple tests on various platforms and where surprised to discover that the latest version of the JVM was giving the erroneous off-by-one answer. Hmm… weird. At that point we started to suspect something fishy. We even checked out Google earth to find out where the elusive Starke, unsuccessfully. Eventually, I stumbled upon this:

[...] other entries represent smaller regions like Starke County, Indiana, which switched from central to eastern time in 1991 and switched back in 2006.

It turned out the time-zone database on the Linux system was not up to date but the JVM one was. Java was right all along. Whoever populated the time-zone mapping database made a very poor choice when they picked US/Indiana-Starke as a representative of EDT_EST, especially when other candidates would be America/New_York, US/Eastern, EST5EDT… but what do I know.

Better still, the time-zone mapping table should have gotten the boot. There are wheels you should not even think about replacing. Without our home-made “improved” wheel, the choice of US/Indiana-Starke or America/New_York or anything else for that matter would have been left to the end-user, untouched by the system. Granted, the 25,000 potential users living in Starke County would still have suffered the bug, but the 20,000,000 potential users in New-York State would not have been affected — and I’m not even counting the other millions of people leaving on eastern time.

Where the wheel comes full circle

I hope you will remember this little anecdote next time you are thinking about reinventing the wheel. If you do, slow down and weight your options carefully. Is there an existing framework, library or system that already provides the functionality? To an acceptable degree? If not, is your employer fully aware of the risk and cost associated with your proposal? Make it clear! If you are self-employed, are you willing to take the risk and pay the cost? Are the benefits worth it? You get the picture: think it through. If you reach the conclusion that it is all worth it, you’re either mad or incredibly lucky, because you will get to fulfill your inner-geek deepest desire: inventing your very own wheel!

technorati tags:,

About these ads

5 Responses to “Reinventing the wheel and associated pains”


  1. [...] Olivier Ansaldi has a nice post on reinventing the wheel. I think reinventing is required for software development to evolve and flow. However, the effort for reinventing the wheel should be justified in terms of benefit to the project/customer. This can explicitly sideline the inner-geek desire. [...]

  2. Brian Connell Says:

    This is your Carlberg. Probably the best post you’ve written so far!

    I must say that I prefer reading your more life-observational posts ….


  3. Thanks Brian. Non-Irish readers might want to checkout YouTube for the Carlsberg reference.

  4. Tony Byrne Says:

    Olivier,

    I’ve come to this post somewhat late. I have to echo Brian’s comments – this is excellent writing. The costs of “not invented here” syndrome are significant and are always overlooked. Developers (and I count myself in there) have a tendency to prefer their own solutions to common problems and often there isn’t enough understanding or involvement by the customer of a piece of software to curb this behavior.

    There’s also a ‘law’ (but maybe a truism) about wheel re-invention. The chances that a developer will decide to re-invent a particular wheel is indirectly proportional to their ability to do a proper job of it. The WTF site (http://worsethanfailure.com/) is littered with real world examples of this in action.


  5. Hi Olivier

    read your blog…very interesting…as far i see, reinventing things rather than stabilizing the ones that already exist come with the man’s nature of being insatiable for growth and popularity.Everybody wants to deliver something new to have its identity known by its new creation.


Comments are closed.

Follow

Get every new post delivered to your Inbox.

%d bloggers like this: