A few days ago Hudson began emailing alerts that a freight calculation unit test was failing in production. I emailed the developers responsible for the code and asked them to investigate this right away. The funny thing about unit tests failing in production is that developers can fall into a mindset of "the code is working on production, so there must not be a problem". The freight calculations did appear to be working correctly on production. Because we are so busy, no further investigation was done until yesterday.

Yesterday afternoon two developers realized that this failing unit test was the result of a serious regression in our "premium" freight calculation. This freight calculation type is most commonly used for furniture items and can easily calculate a shipping cost well over $100. This particular bug was calculating the "premium" freight costs as free. Now HauteLook's business model is flash sales. This means new sales events start at 8:00 am PT each morning and most of the business is done within the first few hours of the event. If there had been a furniture sale event within the past couple of days, HauteLook would have given away thousands of dollars worth of free shipping (or even tens of thousands if the event was large enough) before we would have noticed and fixed the issue. Shipping costs were not the only thing we saved either. The amount of time, energy and stress that it would have taken for the various departments to address this issue would have been high.

I saw an immediate change in the way my colleagues view unit tests. This was a clear indication that our investment has paid off. In this case we had well written unit tests that quickly alerted us to a problem before it costs us anything. I know next time a unit tests fails on production, HauteLook developers will not assume everything is working.