Evaluation of Software Quality for Development of the ATLAS Software
Collaborative partners:  University of Edinburg

The purpose of this working package is to promote software quality evaluation as an integral part of the ATLAS software development process. This will require the development of tools and practices in two key areas.

- Firstly, a robust defect checking service of all the main ATLAS offline development branches will provided. Software quality tools such as Coverity and cppcheck are currently run infrequently over just the single master development branch. They therefore attend to deploy a framework to ensure these tools are run at regular intervals and to respond to any issues with maintaining the ongoing service. Where appropriate expertise to assist software quality coordinators with defect triage and resolution will be provided.
- Secondly, recent wholesale changes to the ATLAS software build infrastructure have provided an ideal opportunity to apply software quality evaluation as an integral part of the development workflow. The framework can be leveraged to include the primitive checking of code defects (i.e. before code is introduced into the development branches) by the addition of software quality tests within the new Continuous Integration (CI) test chain.
- Early feasibility testing in this area by the Georgian team together with the ATLAS software coordinators motivated the construction of a test development environment hosted at multiple locations and isolated from the operational build infrastructure. The use of Docker containers enabled the rapid emulation of key services used in the build infrastructure thereby creating a sandbox to develop new CI tests and to explore how to interpret and report software quality test results. Since further development of the workflow is expected, exploration of new approaches to accommodate results gathering and comparison from other software quality tools will be required.
- Furthermore, it is intend to continue developing software quality CI tests for more comprehensive defect checking coverage. A deployment pipeline has recently been established with the ATLAS Software Infrastructure Team to allow their development effort to be validated and then included onto the production software build infrastructure. This will allow to gather experience from code review shifters and developers to help optimise testing coverage and results presentation.

It is also intended to build upon preliminary studies into methods to promote defect resolution action through code reviewer-led triage and in harnessing results from trend analysis on generated code quality indicators as part of an overall software quality evaluation strategy.