A study report QC checklist for GLP and non-GLP studies, and where an agent can check every number

QC and QA are different jobs

Two checks stand between a draft report and the sponsor, and they belong to different people. Quality control is the study team's own check. OECD GLP Advisory Document No. 23 describes QC as routine independent checking, measuring and testing to ensure the accuracy of data, carried out under the responsibility of the study director. It also says plainly that "QC is not the responsibility of QA personnel."

Quality assurance is the independent audit. Under 21 CFR 58.35(a) the QA unit must be "entirely separate from and independent of the personnel engaged in the direction and conduct of that study", and under 58.35(b)(6) it reviews the final report to assure that "the reported results accurately reflect the raw data". No. 23 recommends the final inspection of raw data "at the final report draft stage, when all raw data have been gathered, all quality control checks on raw data have been completed and no more major changes are intended".

QC should be finished before QA starts its review. When QC is thin, QA ends up doing the study team's checking and the report goes round again. Any tool that helps here belongs on the QC side, under the study director. It should never stand in for the QA audit. In a non-GLP discovery report there may be no QA audit at all, which puts the whole weight on QC.

The checklist

Use this for GLP and non-GLP reports alike. In a non-GLP discovery report some items will not apply, but the number checks always do. The references in parentheses point to the 21 CFR 58.185(a) items behind each check.

Run it on the draft that goes to QA, after the study director has finished editing. A check run on an earlier draft has to be repeated for every section that changed since, and that repeat is where errors slip back in.

• Every value in every table matches the source data or the approved export, including individual animal or well-level appendices.
• Group sizes match the data for every group and time point, and any excluded animals or wells are explained.
• Units and decimal places follow the template and agree between the tables and the text.
• Group and dose labels match the protocol and every amendment to it.
• Every number quoted in the text matches the table it comes from.
• Cross-references to tables, figures, appendices and contributing reports point to the right place.
• Every deviation and unforeseen circumstance in the log is described, with its impact on the data (item 9).
• Calculations and statistical methods are described and match what was actually done (items 3 and 11).
• Every contributing scientist report is present, signed and dated (item 12).
• Storage locations for specimens, raw data and the final report are stated (item 13).

Where an agent fits

Most of that checklist is reconciliation: take a number in the report, find its source and compare the two. The work is mechanical and repetitive, and checking every value by hand can be time-consuming. An agent can check every value on every report and show its work.

The setup is easy to describe. The agent reads the draft report and the approved exports from your study management system, LIMS or analysis software. For each reported value it finds the source and compares them, then checks group sizes, units, rounding, labels and cross-references. The output is a flag list: each mismatch, where it sits in the report, what the source says and why it was flagged. Values it cannot trace to any source are flagged too.

What counts as the source depends on the report. A PK report traces back to the concentration data and the noncompartmental analysis output. An in vitro panel traces back to plate data and curve fits from your analysis software, including the reference compound, and an efficacy study to its body weight and tumor volume records. Agree that list per report type before you start, because it is also the list of exports the agent needs.

Two things stay off limits for the agent. It never edits the report or the data, and it never clears its own flags. A QC reviewer works through the flag list and records how each one was resolved. QA receives the cleared draft and that record, and audits as it always has. Nothing about the agent changes what 58.35 asks of QA.

Controls that keep the check honest

Before using the tool on live studies, test it on old reports with known errors planted in them, and record what it catches and what it misses. Then keep these controls in place:

• The agent reads only approved exports or read-only copies, never the live system of record.
• Every flag shows the reported value next to the source value, with its place in the report.
• A named QC reviewer clears each flag and records the resolution, and the agent cannot clear its own flags.
• A random sample of unflagged values is still checked by hand, so you keep measuring what the agent misses.
• The agent's version and its flag list are kept with the study file under your SOPs.
• Any change to the agent goes through your change control before it touches another live study.

Data integrity and validation

Data integrity guidance gives you the vocabulary for those controls. FDA's 2018 data integrity guidance, written for CGMP manufacturing, says data should be "attributable, legible, contemporaneously recorded, original or a true copy, and accurate (ALCOA)". OECD Advisory Document No. 22 on GLP data integrity refers to ALCOA and to its later extension, ALCOA+, which adds complete, consistent, enduring and available.

For an agent the practical reading is short. The workflow does not replace source raw data, so it works on copies. Each flag names the source it was compared against, and its outputs and review records are retained under your SOPs.

For GLP use, document the workflow's intended use and validate its relevant functions in proportion to risk. No. 17 says the validation effort "should be driven by the intended use of the system and potential risks to data quality and data integrity". Assess effects on connected systems through change control. A person clearing every flag and a hand-checked sample remain controls within that process. Our page on GxP validation for AI workflows explains how we document it.

Some flags will turn out to be errors in the source data rather than in the report. The correction then follows your raw data rules. Under 21 CFR 58.130(e), a change to an automated data entry must not obscure the original entry, must give the reason and must be dated, with the responsible person identified. The agent's flag can explain why a correction looks necessary. A person makes the correction in the system that holds the record.

Worked example: what the hours look like

This is an illustrative example with assumed numbers, not a client result. Assume a typical draft report holds 1,200 reported values across tables and text, and a careful manual check takes about 20 seconds a value. A full manual QC check is then 400 minutes per report, close to 7 hours. If the QC team checks 4 reports a week, that is 1,600 minutes, or about 26.7 hours.

With the agent, assume 3 of the 4 reports draw on approved electronic exports it can read, which is what routine means in the calculator below. On those, the agent flags 3% of values, 36 per report, and each takes about 3 minutes to clear, so 108 minutes. A reviewer also hand-checks a 10% random sample of 120 values at 20 seconds each, another 40 minutes. That makes 148 minutes per report. The fourth report, with paper raw data, is checked by hand as today at 400 minutes. The week comes to 844 minutes, about 14.1 hours, which gives back about 12.6 hours a week.

Keep a second pair of numbers in view as well: QA findings per report and the number of drafts QA sends back. We would not predict either one. Measure both for a few months before and after the change.

Starting small with report QC

Take one finished report whose QC and QA findings you already know. Run the agent against the draft that went to QC and its source exports, and compare its flag list with what your reviewers found. Then plant a handful of errors in a copy, such as a transposed digit or a wrong group size, and see whether it catches them. Start with a small feasibility sample like this, and define validation coverage separately.

Those tests become the acceptance criteria we sign before we build, for example a catch rate on planted errors and zero edits to source data. We build it in your tenant in two weeks, and you pay $10,000 only after every criterion passes. Running it afterwards is optional, from $1,200 a month.

Common questions