Every morning at 3:00, one of our servers grabs the latest code from our source repository and runs the build script. As you would expect, the build usually completes without error and everything’s fine. From time to time, though, one of us will forget to check in a file or dependent project, and the build will fail. At that point, it’s nice to have a way to tell everybody that the build failed, and why.
The build is an MSBUILD script that compiles all of our projects and dependencies, and then copies the results to a staging directory from which we can run unit tests or build distribution packages for our internal customers (see below). To this point, everything can be done with a minimal batch file script, the MSBUILD program supplied with the .NET development tools, and of course the subversion command line client. We have one other tool called sendEmail that notifies me of the build status.
I’d like to notify everybody when the build fails, but doing so requires that I tell them why it failed. And the generated build log is very large: about 120 kilobytes, most of which is irrelevant. The important information is typically the last 10 or so lines of the file, and that’s what I’d like to send to people when the build fails. Those lines say, in effect, “The build failed for these reasons.” A programmer who receives that message can quickly determine if it’s his responsibility, and take steps to fix the problem.
The only trouble I have is that there is no simple way with Windows-supplied tools to extract those pertinent lines from the file. At least, I can’t think of a way. But the GNU awk (gawk) can do it trivially.
When the build fails, the last thing that MSBUILD outputs is a line that says, “Build FAILED”, followed by some lines that describe the error or errors. So all I need is a program that will go through the file, locate the “Build FAILED” line, and then output that line and all following lines to the end of the file. It’s been 20 years since I did any awk programming, but this script was simple:
gawk "{ if (/^Build FAILED/) { doit=1 } if (doit) print $0 }" < buildlog.txt
Done and done.
The only problem I have now is deciding whether I want to install the full GNU Tools for Windows package on my server, or if I should just grab Gawk for Windows. The full package is probably the right way to go because I suspect I’ll be needing some other tools in the future.
Either way, I’m annoyed that Windows doesn’t include these simple text processing tools. I can perhaps understand why they don’t exist in desktop versions, but we do these types of things on servers all the time, and the standard server install should include a more robust toolset.
Above I mentioned “internal customers.” In reality, we are our own customers. There are only five of us here, and one of us doesn’t use the tools that the build creates. In light of that, it’d be easy to take a more cavalier attitude towards our build process. I’ve found, though, that things run smoother if, as a programmer, I think of the users of my software (in this case, the crawler subsystem and the tools that process the collected data) in much the same way as I would an external customer. Even though the primary user of the crawler is me. I wear a number of different hats around here (as does everybody else–we’re a startup, after all), and it’s useful to think of Jim the SysAdmin as a separate person from Jim the Programmer. That way, when we can afford to hire a system administrator to take those duties from me, the systems will already be in place for him to step right in.
Like source code version control, a formal build process is one of those things that you don’t need to implement until the size of your project team exceeds zero people.