Can a GMP Factory Use Generative AI in Production?
Europe’s draft GMP annex on artificial intelligence draws a line every regulated factory should read: generative AI stays out of decisions that touch product quality. Here is what the draft allows, what it rules out, and why it matters in Malaysia.
Quick answer
Yes, but not where it matters most. The European Commission’s draft Annex 22 to the EU Good Manufacturing Practice (GMP) guide, published for consultation in July 2025 and developed with the Pharmaceutical Inspection Co-operation Scheme (PIC/S), states that generative AI and large language models should not be used in critical GMP applications, meaning any use with a direct impact on patient safety, product quality or data integrity. Only static, deterministic machine-learning models qualify for those jobs, and each must pass a documented test against independent data before it is switched on. Generative tools may be used for non-critical tasks, provided a qualified person stays responsible for checking every output. The draft is not yet final and formally covers medicines and active substances, but Malaysia’s regulator is a PIC/S participating authority, and cosmetics, supplements and food follow the same GMP logic in their own codes. This article is general education, not regulatory advice.
Key takeaways
The draft sorts AI by consequence, not by cleverness. A model that decides whether a batch passes is critical. A model that drafts a training slide is not. The rules apply in full only to the first group.
Two technical tests decide whether a model can be used in a critical application: it must be static, meaning it stops learning once deployed, and deterministic, meaning the same input always gives the same output. Generative AI fails both.
A model may not lower the bar. The draft requires its acceptance criteria to be at least as high as the performance of the process it replaces, which means the factory must first know how good the human or legacy process actually is.
Test data must be independent of training data, protected by access control, and never seen by the people who trained the model. The draft also says generating test data with generative AI is not recommended.
The United States Food and Drug Administration’s January 2025 draft guidance takes a similar risk-based line for drug and biological products: credibility is judged for a defined context of use, with scrutiny scaled to how much the model influences a decision and what happens if it is wrong.
What does “critical GMP application” mean, and why does it decide everything?
A critical GMP application is any use of a computerised system that directly affects patient safety, product quality or data integrity. That is the hinge the whole draft turns on. Fact: the draft Annex 22 text published by the European Commission scopes itself to AI models “used in critical applications with direct impact on patient safety, product quality or data integrity, e.g. to predict or classify data”, and positions itself as additional guidance to Annex 11, the existing annex on computerised systems. The consultation ran from 7 July to 7 October 2025 alongside revised drafts of Chapter 4 on documentation and Annex 11 itself.
Analysis: the practical effect is a sorting exercise before any technology decision. A vision model that rejects mislabelled bottles, a model that classifies tablets as accept or reject, or one that predicts whether a batch meets its release specification are all critical, because a wrong answer changes what reaches the customer. A chatbot that answers staff questions about a standard operating procedure, a tool that summarises deviation reports for a manager, or a drafting assistant for supplier questionnaires are non-critical, because a human still decides and the record of record is still produced the normal way. The draft does not ban the second group. It asks that “personnel with adequate qualification and training should always be responsible for ensuring that the outputs from such models are suitable for the intended use”, which it calls a human-in-the-loop.
Opinion: most of the generative AI pilots I see in factories sit in the non-critical group, and that is fine. The problems start when a non-critical tool quietly becomes critical, for example when a summary written by a language model is pasted into a batch record without anyone re-checking the source data. The classification has to be revisited each time the use changes, not once at purchase.
Why does the draft rule out generative AI for critical uses?
Because generative models fail two properties the draft treats as non-negotiable. Fact: the annex applies only to static models, “models that do not adapt their performance during use by incorporating new data”, and states that dynamic models which keep learning in use “should not be used in critical GMP applications”. It also applies only to models with a deterministic output, where identical inputs give identical outputs; models with a probabilistic output “are not covered by this document and should not be used in critical GMP applications”. It then draws the conclusion explicitly: “the document does not apply to Generative AI and Large Language Models (LLM), and such models should not be used in critical GMP applications.”
Analysis: this is not hostility to the technology. It is a consequence of how GMP validation works. A validated system is one whose behaviour has been tested and then frozen, so that the test result still describes the system in use. A model that updates itself after deployment has, by definition, left the state that was tested. A model that can give two different answers to the same input cannot be validated in the ordinary sense at all, because there is no single behaviour to validate. Generative models are built to be both adaptive and varied, which is what makes them useful for writing and useless as a release decision. The discipline, as with any inspection model, is in the testing, not the algorithm.
Fact: the draft reinforces the point in its test-data section, where it says that generation of test data or labels “by means of generative AI, is not recommended and any use hereof should be fully justified”. Synthetic examples cannot stand in for the real variation a line produces.
What must a factory prove before an AI model touches quality decisions?
More than most vendors’ validation packs currently contain. The draft lists the evidence in order. The intended use must be described in detail, including the full range of inputs the model will see and the rare variations, and a process subject matter expert must sign off before testing starts. Test metrics must be defined for that use, with the annex offering a confusion matrix, sensitivity, specificity, accuracy, precision and F1 score as examples for an accept-or-reject classifier. Acceptance criteria must be set in advance, and the draft adds a rule it calls “No decrease”: the criteria “should be at least as high as the performance of the process it replaces”, which implies the factory must have measured the performance of the manual process it is automating.
Fact: test data must be independent. The draft requires technical or procedural controls so that data used to test the model “is not used during development, training or validation of the model”, protected by access control and audit trail, with no copies outside the repository, and with staff who have seen the test data kept out of training the same model. Explainability is also required during testing: systems “should capture and record the features in the test data that have contributed to a particular classification or decision”, using tools such as SHAP values, LIME or heat maps, and a review of those features becomes part of approving the test. Finally, the model should log a confidence score for each prediction, with a threshold so that a very low score makes the model “flag the outcome as ‘undecided’, rather than making potentially unreliable predictions”. After deployment, the model, its system and the whole process go under change control, and the factory must monitor both model performance and whether live inputs are drifting outside the sample space it was tested on.
Analysis: read together, these requirements describe a project that is mostly data management and process measurement, with the model as a small part in the middle. That matches what I have seen when lines are automated more generally, which I covered in What Breaks First When a Line Is Automated: the machine is rarely the bottleneck, the records around it are. It also matches the FDA’s approach. Fact: the agency’s January 2025 draft guidance on AI to support regulatory decision-making proposes a risk-based credibility assessment framework in which a model is judged for a specific context of use, and the depth of evidence scales with model risk, a combination of how much the model influences the decision and how serious the consequence of a wrong output would be.
Does this matter for a cosmetics or supplement factory in Malaysia?
Directly for some products, and by example for the rest. Fact: Malaysia’s National Pharmaceutical Regulatory Agency is a participating authority of PIC/S, the inspection co-operation scheme that co-developed the draft annex, and PIC/S documents are the reference for GMP inspection of registered medicines, traditional products and health supplements in Malaysia. When PIC/S adopts an AI annex, the thinking arrives in Malaysian inspections with it. Cosmetics are notified rather than registered and follow the ASEAN cosmetic GMP guideline, which does not yet mention AI, but the underlying expectation that any system affecting quality must be validated and under change control is the same.
Get Creaton Poh’s stories in your inbox
Join Medium for free to get updates from this writer.
Fact: data integrity rules already apply regardless of how a record was produced. The PIC/S guidance on good practices for data management and integrity, in force since July 2021, expects records to be attributable, legible, contemporaneous, original and accurate, among other properties. An AI output that feeds a GMP record is a record, and the person who accepted it is the person it is attributed to. For factories exporting to the United States, electronic records and signatures must also meet the requirements of 21 CFR Part 11, which cares about audit trails and controlled access rather than about the software’s cleverness.
Analysis: for a contract manufacturer, this becomes an audit question before it becomes a technology question. A brand owner or inspector who sees an AI tool on the line is entitled to ask which category it sits in, what the acceptance criteria were, and where the test data came from. Factories that already run certified GMP and ISO quality systems have the change-control and validation machinery to answer those questions; the AI model simply becomes another item inside it. The same applies to the production equipment and process controls a model might be bolted onto: the model’s input space is defined by that equipment, so a change to a camera, a light or a conveyor is a change to the model’s intended use. Audit readiness in general was the subject of What a Factory Audit Actually Checks, and the AI annex adds one more file to the same cabinet.
Opinion: the draft is a sensible first step, and the industry comments it attracted suggest the final text will loosen some wording. I would not expect the core line to move. Regulators have spent two decades getting factories to prove that their computerised systems do what they claim; they are not going to accept a class of system that cannot, by design, give the same answer twice.
Frequently asked questions
Is draft Annex 22 already law?
No. It was published for consultation on 7 July 2025 with comments closing on 7 October 2025, and had not been finalised at the time of writing. It signals the direction of EU and PIC/S inspection expectations rather than imposing a current obligation. Factories supplying the EU should assume the principles will be applied by inspectors in substance well before the text is formally adopted.
Can a factory use ChatGPT or a similar tool at all under GMP?
Yes, for non-critical tasks such as drafting, summarising or answering staff questions, provided a qualified person remains responsible for checking the output before anything is acted on, and provided no GMP record is created or altered by the tool without that review. The draft explicitly allows this human-in-the-loop use. What it rules out is using such tools to make or inform decisions with a direct impact on product quality, patient safety or data integrity.
What is the difference between a static and a dynamic model?
A static model has all its parameters fixed once training is complete, so its behaviour in use is the behaviour that was tested. A dynamic model continues to learn from new data during use and therefore changes over time. The draft annex covers only static models and says dynamic models should not be used in critical GMP applications, because a tested state that keeps changing cannot be relied on.
Does an AI vision system for defect detection count as critical?
Usually yes, if its output decides whether a unit is accepted or rejected, because that is a direct impact on product quality. It would then need a defined intended use, pre-set acceptance criteria at least as good as the human inspection it replaces, independent test data, explainability records and a confidence threshold. If the system only flags units for a human to re-inspect and the human’s decision is recorded, the testing burden may be reduced, but the draft then expects the operator’s performance to be monitored like any other manual process.
Sources
European Commission, EudraLex Volume 4, draft Annex 22 Artificial Intelligence (consultation text, July 2025) — https://health.ec.europa.eu/document/download/5f38a92d-bb8e-4264-8898-ea076e926db6_en?filename=mp_vol4_chap4_annex22_consultation_guideline_en.pdf
European Commission, Stakeholders consultation on EudraLex Volume 4 Chapter 4, Annex 11 and Annex 22 (7 July to 7 October 2025) — https://health.ec.europa.eu/consultations/stakeholders-consultation-eudralex-volume-4-good-manufacturing-practice-guidelines-chapter-4-annex_en
United States Food and Drug Administration, Considerations for the Use of Artificial Intelligence to Support Regulatory Decision-Making for Drug and Biological Products, draft guidance, January 2025 — https://www.fda.gov/regulatory-information/search-fda-guidance-documents/considerations-use-artificial-intelligence-support-regulatory-decision-making-drug-and-biological
Pharmaceutical Inspection Co-operation Scheme, PI 041–1 Good Practices for Data Management and Integrity in Regulated GMP/GDP Environments, 1 July 2021 — https://picscheme.org/docview/4234
Pharmaceutical Inspection Co-operation Scheme, participating authorities — https://picscheme.org/en/members
United States Code of Federal Regulations, Title 21 Part 11, Electronic Records; Electronic Signatures — https://www.ecfr.gov/current/title-21/chapter-I/subchapter-A/part-11
This article is general education, not regulatory or legal advice for a specific product, facility or jurisdiction.
Written by Creaton Poh — Industry Researcher · Author · Vlogger · Manufacturing Strategist. Turning ideas into products. Turning experience into knowledge.
Creaton Poh is the pen name of Poh Tze Kheng, founder of ORIZI Group — one of Malaysia’s most comprehensive one-stop OEM, ODM, private label and contract manufacturing groups, spanning skincare, cosmetics, personal care, perfume and fragrance, supplements, functional food and nutrition, pet care, home care and medical devices. Across six specialised companies and three manufacturing divisions, the Group brings together 20+ years of industry experience, 5,000+ formulations, and internationally recognised quality, manufacturing and regulatory standards. These articles are written to educate, not to promote.
Connect on LinkedIn.
