When Does an AI Tool Become a GxP Computerized System?

The answer depends on what you use it for, not what it is.

Quality teams are hearing the same question more and more: "Is this AI tool GxP?" Someone wants to use an LLM to draft deviation reports, summarize batch records or answer questions about SOPs, and Quality has to decide whether it needs to be validated.

The question, as asked, has no good answer. The same AI tool can be completely outside GxP scope in one use and squarely inside it in another. What decides scope is the use, not the product.

Start with intended use

This is the same principle that underpins Computer Software Assurance generally. FDA's CSA guidance begins with the intended use of each feature, function or operation. GAMP 5 (Second Edition), including Appendix D11 on AI and machine learning, takes the same approach. An AI tool is a computerized system like any other. What makes it GxP-relevant is what it does in your process.

So the useful question isn't "is this tool GxP?" It's "in this use, does this tool's output affect GxP work?"

Three questions that decide it

1. Does the output become, or feed, a GxP record or decision?

If what the AI produces ends up in a batch record, an investigation, a complaint file, a regulatory submission, a release decision or any other regulated record, or directly shapes one of those decisions, the use is GxP-relevant.

2. Could an error affect patient safety, product quality or data integrity?

Ask what happens if the AI is wrong: it misses something, invents something or misclassifies something. If that error could plausibly reach a patient, a product or a record's reliability, the risk is real and has to be assessed.

3. What is the human actually doing?

Most AI uses in regulated work involve a person reviewing the output. That's important, but it doesn't automatically take the tool out of scope. The real question is whether the person independently verifies the output against a source, or effectively relies on it. A reviewer who skims a fluent, confident summary is relying on it, not verifying it. Human review lowers risk only when it's designed to: defined, attributable, and able to catch the errors the AI is likely to make.

A spectrum, not a switch

Applied to real uses, those three questions give a range of outcomes:

• Drafting an internal, non-GxP email or meeting summary: outside GxP scope. Normal IT and information-security controls apply.
• Helping staff find relevant sections of approved SOPs, with the SOP itself as the reference: usually low risk, provided users act on the controlled document and not the AI's paraphrase.
• Drafting deviation or investigation narratives that QA reviews and approves: in scope. The output becomes part of a GxP record, and the review has to be designed to catch omissions and fabrications.
• Classifying or triaging deviations, complaints or adverse events: in scope and higher risk. A misclassification can send a quality or safety signal down the wrong path.
• Contributing to batch disposition, inspection results or other release-relevant decisions: in scope and high risk, with the most rigorous assurance and governance.

The same LLM could sit behind every one of these uses. Each use needs its own determination.

What changes once a use is in scope

Once an AI use is GxP-relevant, it needs the same elements as any computerized system, plus a few that are specific to AI:

• Defined intended use and risk classification, recorded and approved.
• Supplier assessment, including how the vendor manages model changes, data handling and its own quality practices.
• Records and audit trail: capturing inputs, outputs, model or version identifiers and human review decisions so they are attributable and reviewable.
• Assurance testing suited to AI: accuracy against a reference set, behavior at the boundaries, consistency across repeated runs, and structured testing for hallucination. Acceptance criteria are set before testing begins.
• Governance after release: performance monitoring, change control for model and prompt changes, and defined triggers for re-assurance when the vendor updates the model.

Where the regulations stand

No single regulation answers this question yet, but the direction is consistent. GAMP 5 (Second Edition) Appendix D11 and ISPE's GAMP guidance on AI give industry a working framework. 21 CFR Part 11 and EU GMP Annex 11 already apply to the electronic records and signatures involved.

In Europe, a new EU GMP Annex 22 on artificial intelligence was released for consultation in 2025, alongside a revised Annex 11 and Chapter 4. At the time of writing it hasn't been adopted. The draft limited critical GMP uses to static, deterministic models, and published reports from 2026 regulator–industry discussions suggest that position may change for large language models. Check the current status before relying on any specific provision. The principles it emphasizes, though, are already sound practice: defined intended use, clear accountability, risk-based validation and lifecycle oversight.

Where to start

Begin with an inventory of AI use, not just AI tools. Ask teams how they're actually using AI in their work, including tools nobody formally approved. Then apply the three questions to each use. Most organizations find that most of their AI use is low-risk, a few uses clearly need assurance, and a handful need a decision before they continue.

That inventory is the starting point for everything else: classification, supplier assessment, testing and governance.