Optimising CI Runs
I spent some time optimising the CI runs for a recent project.
In this case there were two dimensions to optimise against:
- total CI minutes used (each month)
- duration of each CI run
Usually improvements to one dimension improve the other. But sometimes they’d conflict.
In this case, slightly increasing each run duration led to a big reduction in total CI minutes used.
The setup was a number of jobs running in parallel:
- shortJobA - shortJobB - shortJobC - longJob
longJob would always run, even if all of the
shortJobs failed. Changing the run ordering to:
- shortJobA --| - shortJobB --|--> longJob - shortJobC --|
longJob would only run after all the
shortJobs passed. This made individual CI runs take slightly longer, but massively reduced the overall number of CI minutes used.
Other optimisations fell into the following few categories:
- Remove unneccessary work
- Optimise slow work
- Cache results of expensive work
Remove, reduce or re-use. I guess that’s a general lesson for any code.