Build Automation / Build Systems
The first goal of a new project is to make the process of building and deploying the project as easy as possible. Build automation is one of my favorite parts of computer programming (in the beginning), but it can also be the most frustrating (any time after). In fact, this is so true that many developers punish bad development (breaking the build) by putting a person at fault in charge of the build. After a project is off the ground, no one wants to be in charge of the build. It’s all too tedious, and you end up spending a lot of time fixing headers, rewriting old scripts, and generally managing the team’s build process – or lack thereof.
Build automation is the act of automating as much of this as possible. On a good team, in a good company, this is set up at the start of the project and is used and maintained throughout. I am lenient in my description in saying “automating as much as possible” because realistically there’s no reason why it can’t be “automating everything”. Builds should be so easy to make from scratch that – given dependencies are met – the user may run a single command in order to run through the entire build chain and have a packaged application ready for distribution.
A build system is the process that the command goes through in order to build, package, and sometimes distribute the software – relating specifically, but not limited to; everyday development builds, automated builds, and distribution builds. A build system is all-encompassing of this entire process.
Note: Visual Studio is NOT a build system, it’s a tool used to build applications, which is a part of the build chain.
The Build Chain
Above is a very general overview of a build chain. It is the idea of how we go about building software without getting into specific alterations to the chain. Any step that you add the the build chain should be automatically tacked on to one of the above nodes. Let’s discuss each node really quickly, and their purpose.
Find dependencies (on the developer’s machine or otherwise), generate code, and gather information to prepare the build. For large projects, information should be cached so that reconfiguration doesn’t waste too much time. If the configuration fails, a good build system will report back and tell the developer what is wrong, but will NOT continue the build chain.
Create the files that will be used for building the binaries. (makefile, solution, etc.)
The entire process of building a binary target from the project. (In Visual Studio, the Build button) This is where most developers will spend their time in the build chain.
Taking the final binaries and packaging them up for distribution. (Generate an installer, documentation, distribution materials, etc.) Possible to automate, but likely to be forgotten.
The act taking the packaged binaries and distributing them so that people have access to the latest build. Have you ever had an artist complaining (rightfully so) that they didn’t get a build until way too late? It’s likely this is the problem.
Now you’ll notice that I took the time to split Build into its separate subsections. The reason for this is because there are sometimes things you want to do within those subsections that you can automate through your build system. For example, updating out-of-date intermediate files (which are not necessarily source files; models, textures, animations, etc.) can be a Postbuild step. Another very important rule about all of these steps: ANY of the above steps should be able to be carried out with a SINGLE command!
Advantages of Build Automation
The most obvious advantage of the above information is a much smoother workflow. A good build system will be able to run through the entire chain without hiccups or complaints, and spit out any intermediates that the developer wishes the chain to produce. (Should be able to go from freshly checked-out source to an installer with a single command, or from a generated project to a deployment without reconfiguring or regenerating.) What information can we extract through this process? Below is just a small list of things we can automate for a build system to take into account.
- Compile/Link Times (Preferably separate)
- Repository checkout time
- Files updated (svn/hg/git checkout, who to blame if this build fails)
- Automation of unit tests and test statistics (times, pass/fail, etc.)
- Code coverage statistics (gcov, Bullseye)
- Configuration/Compilation Warnings/Errors
- Build time/date
This only covers gathering information about the build. There is actually more information we can grab in the form of the binaries the build system spits out. Why would we just throw those away when we have non-development team members complaining about lack of build updates? There’s more information we can grab…
- Documentation (doxygen)
- Binary files (lib, exe, app, etc.)
- Packaged files (msi, bsh, dmg, etc.)
- Commit messages (Update logs)
This is valuable information, and it’s all available at this time. Why not grab it, package it, and upload it to a server somewhere so that other teammates have access to it? Even a directory they can access as an “Index of” page with zip files listed by date is better than nothing. (Though, if you have the time, you can even do better than that.)
Great! How Do I Get Started?
In the following series of posts, I will cover the methods I use with a build system I’ve grown quite fond of (CMake). Stay tuned!