I’ve been programming computers for a long time. Getting paid to write computer programs, even, which I thought was pretty darned funny when I first started. People were paying me to do something that I loved. But I digress.
After 30 years, you’d think that I would have learned that there’s no such thing as a small change that you can push into production code without having to test. You might get away with it from time to time, but eventually that arrogance is going to cost you.
But, hey, it’s a simple change! What could go wrong?
When you hear yourself say that, think about what you’re saying. And then spend the few minutes it will take to test your assumption. If nothing else, you’ll save yourself the embarrassment of explaining to your business partner that you made the kind of mistake that you’d reprimand an employee for.
Fortunately, all it cost me was a little embassassment, a few hours’ lost sleep, and an additional hour of down time for the crawler. I got off easy.
My research assistants wonder why I have them spend nearly all their time regression testing (i.e., comparing the results from today’s software with yesterday’s, and if there is a difference, figuring out if it’s intentional and desirable — which is sometimes a tricky question with AI software). The answer: In real life, 70% isn’t the passing grade.
There are two big problems with small changes. 1) The idea that you can completely understand the effects of a small enough change is intuitively appealing. 2) More often that not you actually CAN make small changes successfully, which lulls one into thinking it can be done always.
Problem #1 can be trivially solved by keeping a simple record of how well small changes have worked out.
Problem #2 is trickier, and the missing piece in the risk equation is the cost of having something go wrong.
Compounding these problems is that when there’s no pressure a lot of us do things the right way, but when the pressure’s on we take the easy way out. That pressure is often something like a newly discovered, horrible bug already in production or a looming deadline or a hyper urgent customer request. And those are exactly the times when it’s most important to take the extra few minutes to do it right.