Validating AI Workflows in GxP: What to Validate, What to Keep Outside, and How GAMP 5 and CSA Apply

Start with the intended use

Every framework that matters here starts in the same place: a written description of what the AI does and which record or decision it feeds. The draft EU GMP Annex 22 asks a process expert to document and approve the intended use before acceptance testing begins. FDA's computer software assurance guidance makes identifying the intended use its first step, and FDA's draft guidance on AI for regulatory decision-making builds its seven-step credibility framework around the context of use. This is our reading of these documents as of October 2026, not legal advice, and your QA unit owns the validation decision.

What goes inside the validated boundary

For a typical AI workflow in a GLP or GMP setting, the validated part is smaller than the whole system. It covers the steps whose failure would change a regulated record: pulling released results from the source system, placing values in the report, the checks that compare output against source, the audit trail and the approval with its electronic signature. Where those records are required by a predicate rule, Part 11 applies, as FDA's scope and application guidance explains, and for GLP work the OECD advisory documents on computerised systems and data integrity apply too.

My opinion is that a language model should never write a GxP number. Values should come from the released system and be copied by ordinary deterministic code, with the model limited to the prose around them. That design shrinks the validation effort to the copy, the checks and the approval, all of which can be tested against fixed expected results. I described the same split for preclinical reports in an earlier post.

What stays outside, with a human in the loop

Drafting text that a qualified person reviews and owns, searching SOPs with citations and summarising meetings can sit outside the validated boundary, provided nothing they produce becomes a GxP record without review. The draft Annex 22 says generative AI and large language models should not be used in critical GMP applications, and that in non-critical uses qualified personnel should always be responsible for the outputs, which it calls human-in-the-loop. FDA's draft AI guidance likewise excludes AI used for operational efficiencies that do not affect patient safety, drug quality or the reliability of study results.

Human review has to be real to count. The draft Annex 22 asks for the operator's training and consistent performance to be monitored like any manual process, and for records of the review to be kept. If a reviewer approves 200 drafts a day without opening the sources, the record shows a signature without a real check, and an auditor will notice.

Where GAMP 5 and the ISPE AI guide fit

ISPE published GAMP 5 Second Edition in July 2022, a 404-page update that keeps the risk-based framework of the 2008 edition, stresses critical thinking by experienced subject matter experts and adds material on AI and machine learning. The ECA Academy's review describes it as aligned with FDA's computer software assurance approach, which was then a September 2022 draft. In July 2025 ISPE followed with the 290-page ISPE GAMP Guide: Artificial Intelligence, which covers AI-enabled computerised systems in GxP areas across their life cycle and is meant to be used alongside GAMP 5.

Neither is regulation. Both are industry good practice that inspectors and sponsor auditors know well, so a validation package that follows their structure is easier to defend. I would use GAMP 5 for the overall approach and the AI guide for the model-specific parts such as data, testing and monitoring.

What computer software assurance changes

FDA finalised its computer software assurance guidance on 24 September 2025 and replaced it on 3 February 2026 with Computer Software Assurance for Production and Quality Management System Software. FDA's device and biologics centres prepared it in consultation with the drug centre, and its scope is software used in medical device production and quality systems under 21 CFR Part 820. It names artificial intelligence and machine learning tools among the automation its risk framework can cover.

For a CRO, CSA applies by analogy rather than by law, but the thinking travels well. It sorts software features into high process risk and not high process risk, and suggests scripted testing where the risk is high and unscripted or exploratory testing where it is not. GAMP 5's second edition points the same way, so a QA team that scales effort to risk has industry guidance on both sides of the Atlantic.

Annex 22: status, and what to borrow now

Annex 22 went to consultation with a revised Annex 11 and Chapter 4 from 7 July to 7 October 2025, according to PIC/S. It is not final: EudraLex Volume 4 still lists Annex 11 in its January 2011 revision and has no Annex 22, and EMA's June 2026 data steering group minutes target publication in early 2027. EMA's workshop report (EMA/156789/2026), first published on 2 October 2026, concludes that there may be viable pathways for adaptive, probabilistic and generative AI in GMP, subject to appropriate controls and a robust risk-based framework.

The draft is still worth borrowing from today. It asks for acceptance criteria at least as high as the performance of the process the model replaces, test data kept independent of development, and change control before the model goes live. For an LLM workflow, I would treat a change of model version, prompt, retrieval sources or report template as a trigger to rerun the acceptance tests on a held-back sample. The caveat is that the final Annex 22 may move the line on generative AI, so build the evidence in a way that survives either outcome.

Where QueryNow fits

QueryNow builds AI workflows with the validation evidence designed in: the intended use, signed acceptance criteria, test evidence, the audit trail and the change-control triggers, handed to the QA unit that makes the validation decision. We build one AI workflow in two weeks for $10,000, paid only after signed acceptance tests pass, and our GxP AI validation page describes the method. The closest example is the compliance scanner we built for a European pharmaceutical regulator, where every finding is cited and review that took 2 to 3 hours per asset now takes about 2 minutes.