Is software development a craft, or an engineering discipline?  As with so many such questions, the answer is the ever annoying “It depends.”  It depends on two things: the complexity of the project, and the skill of the implementers.  Unfortunately, defining complexity and skill turns out to be quite difficult.  Not to worry, as I’ll come back to those two issues at a later time.

I postulate that the amount of engineering (up-front design work) required for a particular software project is determined by an inverse relationship between the project’s complexity and the skill of the implementers.  That is, a simple project being constructed by a very skilled implementer will require minimal up-front design.  The theory is that the implementer is a skilled craftsman who has constructed similar projects in the past, and is capable of building this project with a minimum of outside help.  Conversely, a complex project being built by a team of relatively inexperienced developers will require much more up-front design work.  The relationship, however, is not linear.  A very complex project will require a large amount of up-front engineering, regardless of the implementers’ skills.  And regardless of how much up-front engineering goes into a project, a certain minimum skill level is required on the part of the implementers. It’s simply not true that sufficient design will enable any group of monkeys to bolt a system together.  That was the theory behind offshore development which, last I checked, hasn’t reduced the demand in this country for skilled developers.

Now, I’m not claiming to be breaking any new ground here.  Practitioners of the various agile development methods have hinted at this relationship, as have various authors of “best practices” books over the years.  I’m just laying this stuff out in simple terms in an attempt to wrap my head around the problem.  After over 20 years of developing software for a living, I’ve come to the conclusion that there is no single “best” way to develop software.  The “best” way is highly dependent on the project’s size, complexity, and development team.  But it seems that there is no efficient and effective method for dealing with the types of systems that are being contemplated today.  The problem is that the software systems we are being asked to create, the systems on which they are implemented, and the user base they are expected to serve are becoming increasingly complex.  Our tool set—the development tools and methods—have improved, but have not kept pace with the demands.

As I said, I don’t expect to come up with “the answer,” but I do hope to identify and understand the factors that determine the most appropriate development method for a particular class and size of development project.  As always, audience participation is encouraged.