Automated Checks

When using Code Reviews and you want to reduce the overhead for developers so that they can give better feedback. You want to automate parts of your Codeline Policy. This pattern describes how to simplify developer workflows related to reviews and codeline management while making it easier to maintain quality.

Varying Strengths and Levels of Attention

Developers have limited time and energy. Focusing on small details can obscure attention to more significant issues. Some mundane things are worth checking thoroughly.

Even the best developers make mistakes. There are multiple classes of errors that a code review could identify:

Humans can catch many of the mechanical issues, but identifying them often depends on a mix of highly detailed syntactic analysis (syntax, style) and encyclopedic knowledge (vulnerabilities). Even with the best intentions, this kind of analysis is error-prone.

Tools can identify higher-level issues like functions or blocks with excess complexity. In some cases, the complexity might be justifiable, so you don’t always want a tool to reject code automatically. A tool can detect if a change has associated added tests, but it can’t detect if the test logic is correct.

Automated tools can provide feedback but can’t offer the same nuanced feedback as team members who have context.

Code that doesn’t meet minimal standards isn’t ready for a final review, and while feedback on work in progress can be helpful, you want to help make effective use of developer time while still fostering an environment of collaboration.

Automate Rote Work; Allow for Evaluation

** Add Automated Checks to your build pipeline to address issues such as style, formatting, complexity, and vulnerability issues. These checks should be able to run both in a Developer Workspace and in a CI Environment. Evaluate the results of these checks in the context of Code Review. Include Automated Checks that run with each commit to a Task Branch, and make that feedback visible as part of a Pull Request**

Automated Checks can include everything from compilation to a style checker and even automatically reformatting code. Unit Tests in the context of an Integration Build can also be considered part of the Automated Checks.

Other checks can:

Run these checks and unit tests during development in a developers workspace and when code is merged to a Task Branch to identify obvious issues. It is ideal if the tool can flag potential problems so that the reviewer can see them in context.

Ideally, automated checks that identify critical issues (compilation, test execution, formatting etc) should pass before sharing the code for review. Other checks should run before the review so that the developer and reviewer can discuss how to address them (or if).

Based on the kinds of Automated Checks you could evaluate the risk of a change to determine the amount of review a change should get. A small, low-risk change might not need any formal review step, though feedback is always helpful for improving understanding and quality,

Automated checks are a way to reduce the cost of pushing out changes {Forsgren et al., 2018 #92987}:

One important strategy to reduce the cost of pushing out changes is to take repetitive work that takes a long time, such as regression testing and software deployments, and invest in simplifying and automating this work.

Cautions

While automated checks offload work from the human reviewers, they may not capture every issue, so incorporate feedback about their effectiveness into your Retrospective process.

Getting to Automated Checks

The easiest way to add automated checks is to integrate these tools into your build pipeline. Tools can be added individually, or you can use platforms that combine tools and integrate the results with your review platform, such as Pull Requests.

Adding the ability to run tests from your build tool will enable developers to run these tests while coding. Add them to the CI pipeline to ensure that they are run, especially if the checks take a non-trivial amount of time.

Next Steps