Quality assurance
Quality assurance (QA) is the formal, systematic process of ensuring our analysis meets appropriate standards of quality and is suitable for its intended use. It means planning how we will check the work, carrying out those checks, and keeping clear evidence of what we did. This plan was created based on Quality assurance - DES RAP Book.
Quality assurance plan
Quality assurance when scoping the project and designing the analysis
We are documenting what we want to do publicly within this Quarto site - e.g., Conceptual modelling page. This ensures are aims are clear and transparent, and means we can check against plans later in V&V.
We will plan and document how the analysis will work, and how it will be checked. This includes:
- Our design decisions and analytical approach - methods, data, software, assumptions - see Code structure/infrastructure plans page and Conceptual modelling page.
- Decide on our verification and validation strategy before writing code - see Verification page and Validation page.
Quality assurance when performing the analysis
We will carry out verification and validation - which will include sensitivity analysis.
We will adhere to best practice for our code and workflows - see Criteria page.
We will create clear and comprehensive documentation including docstrings and comments, as well as several things that will be documented within this Quarto site:
- User documentation explaining how to run and interpret the model
- Technical documentation explaining the model structure and implementation
- An ongoing record of data sources, assumptions, inputs, and decisions - including any changes to the analytical plan or decisions made during analysis.
This documentation will be reviewed by ?
To discuss:
- Related to question about Code review on earlier page - review of documentation too.
Roles
QA roles, as per the GOV.UK AQuA book:
| Role | Responsibilities | Who |
|---|---|---|
| Commissioner | Requests the analysis, sets requirements, confirms the approach will meet their needs, accepts the final work as fit for purpose. | Ambulance service partner |
| Analyst | Designs and carries out the analysis, performs self-assurance (including verification and validation), acts on assurer feedback, documents the work. | Amy Heather |
| Assurer | Reviews the analyst’s assurance work, performs additional verification and validation checks, reports issues, confirms the work is appropriately scoped, executed, validated, verified, and documented. Must be independent from the analyst. | Supervisor and ambulance service partners |
| Approver | Scrutinises the work of analyst and assurer, confirms appropriate assurance has occurred, provides formal sign-off. | Supervisor and ambulance service partners |
Given time-scales, scope and risk, I have assumed that not including an independent assurer for this work is appropriate.
To discuss:
- Thoughts on this plan? Inter-related with the code review question, and what sort of review / assurance there is of code / decisions / etc.
Quality assurance log
We are using GitHub projects as a QA log - so it shows what the plans and issues were during development, who flagged them, who addressed them, and details on the issues and outcomes. This is inspired by The Strategy Unit’s New Hospital Programme model QA project board.
You can view our QA log here: https://github.com/orgs/ambmodels/projects/1/views/1
