Image may be NSFW.
Clik here to view.
As a Kick-Ass Developer you should automate as much as you can, so you’re able to do more coding. Looking at the development cycle the most obvious automation are your builds. Nowadays I don’t know many teams that are not using a Continuous Integration server that kicks off a build after every check in (in fact I just know one). But how much effort did you put into to making your builds Kick-Ass?
If your builds can execute fast, your developers can react directly when the build fails. They maybe haven’t even started with their next task, so there is almost no context switching involved. The changes are fresh in their mind and they can fix the problems fast. So we want fast builds to shorten our feedback loop. We want to fail faster. Do you know if your build scripts and concepts are effective?
Here are 5 lessons we learned at Atlassian to make our build pipeline kick-ass again:
Kill hung builds automatically
First we start with an easy Kick-Ass advice, that you probably can implement in a few minutes. How do you currently find out if your build is blocking? Well, I guess you have to look at your CI-Server UI if you haven’t set up a notification on build success. If a build hangs for hours you will end up with a lot of build requests in your queue. All modern CI-Server are capable on killing hung builds. Just define a time and automatically rerun the built. If it hangs for a second time you can setup a notification and have a look at it.
Image may be NSFW.
Clik here to view.
Pass the artifact
This is easy and short: Pass your artifacts like JARs, EARs and WARs. You can use something like Artifactory to check in the artifacts you are building or just put them on a file server and grab them from there. Reuse your artifacts if you’re running UI tests or Integration tests instead of building it all the time from scratch. Sounds obvious. Yes, but look how much the JIRA team cut down the build time since we passed around the artifact.
Run tests in parallel
Another way to make your builds fast is to split your test suite into batches. You normally don’t have to run them in sequential order. Take some tests that belongs together and run them on a different server in parallel to other tests. Most of the modern CI servers allows you to install build agents that can be triggered and run the tests on their own. Some tools (like Atlassian Bamboo and Teamcity) allows you to aggregate the test results in a single build result. Atlassian had some awesome success when splitting up our tests and distributed them on different servers. If something goes wrong, we find it out fast.
Have a build strategy
Image may be NSFW.
Clik here to view.
Let’s face it: Even we’re trying hard to make our test fast, there will still be some tests that takes a long time. At Atlassian we test against different databases that works with JIRA. We also have tests for different operation systems and for different browsers. So you can imagine if we automatically test all the combinations it will take some time. But as a developer you want to have fast feedback. You don’t want to work for hours on the next task and get interrupted by a build you broke with your merge 2 hours ago. Also if you have 20 developers each checking in code every 2 hours, you have to start 10 builds per hour. You need to have a lot of hardware to support this or you have a long queue with builds waiting for execution.
We operate in different loops:
- inner loop: This is starting a build for every commit. We’re just checking out the code, compiling it and run the unit tests
- middle loop: This is timed (every hour). We take the last successful build artifacts and run some UI tests for the most common browsers
- outer loop: We ran this every night. A full integration UI test and smoke tests.
That way we tightened our feedback loop. When the middle loop fails things are still fresh in mind. Of course this is the pay off you have with long running tests but we think it is a good compromise.
Look at your stats
Find your very own bottleneck to kick-ass with your Continuous Integration / Delivery / Deployment. Which test takes so long? How often do we checkout the code for a build? These and more statistics can tell you where your bottlenecks are. If all of sudden your build time goes up, you might want to have a look at it. Normally your build server just sits in the corner and works silently. But it is a good time investment to have a look once in a while at the build time statistics.
Image may be NSFW.
Clik here to view.
Use more robot power!
As a Kick-Ass Software Developer you:
- have short build feedback loops to avoid context switching
- know how long parts of your builds takes
- constantly work on improving your builds
- make sure all builds stay green, otherwise you stop the current work
Image may be NSFW.
Clik here to view.
Image may be NSFW.Clik here to view.
