CAPA effectiveness check: how to prove a fix worked before closing
How to define a CAPA effectiveness check before the action starts: measurable criteria, sensible timing, the hierarchy of controls, and what to do when the check fails.
The CAPA has been open for ten weeks. The SOP was revised, the operators signed their training records, and the owner wants it closed before the management review. The effectiveness section of the form says "No recurrence observed". That sentence is the reason so many CAPA records reopen at the next audit, and the reason the same deviation turns up again six months later under a new number. This article sets out how to define a CAPA effectiveness check before the action is implemented, how to write criteria you can measure, how long to wait, why the type of action you choose decides whether the check can succeed, and what to do when it fails.
What the regulations ask for
Every framework a European manufacturer works under expects the effectiveness of corrective and preventive action to be reviewed, and none of them accept closure on implementation alone.
ISO 13485:2016 includes "reviewing the effectiveness of corrective action taken" in clause 8.5.2 and the equivalent wording for preventive action in clause 8.5.3. For medical device manufacturers, Regulation (EU) 2017/745 goes further in Article 10(9)(l), which names "management of corrective and preventive actions and verification of their effectiveness" as part of the quality management system.
For medicinal products, EudraLex Volume 4, Chapter 1, section 1.4(xiv), says appropriate CAPAs should be identified and taken in response to investigations, and that "the effectiveness of such actions should be monitored and assessed, in line with Quality Risk Management principles". ICH Q10 makes the same point in its table on CAPA across the lifecycle: during commercial manufacturing, CAPA should be used and the effectiveness of the actions should be evaluated.
None of these texts tells you how to check effectiveness. That is left to your procedure, which is why the quality of effectiveness checks varies so much between sites.
Verification of implementation and verification of effectiveness
These are two different questions, and a CAPA record needs answers to both.
Verification of implementation asks whether you did what you said you would. The SOP is at revision 4 and effective, the fixture was installed and qualified, the supplier's specification was updated and acknowledged. This can usually be confirmed on the day the action completes, from documents.
Verification of effectiveness asks whether the problem has stopped. It needs data from after the change, collected over enough time or enough units for the original failure to have had a fair chance to recur. A CAPA that closes the day the training records are signed has verified implementation and nothing more.
Define the effectiveness check before the action starts
The best time to write the effectiveness criteria is when the action plan is approved, before anyone knows the outcome. Written at closure, the criteria tend to describe whatever happened. Written up front, they force the team to state what success means while the investigation is still fresh.
A useful effectiveness check answers four questions:
- What will be measured, and from which data source?
- What result counts as effective, stated as a number or a clear pass/fail condition?
- When, or after how many units, batches or events, will it be measured?
- Who will review the result, and what happens if it fails?
The baseline matters as much as the target. If you do not know how often the problem occurred before the action, you cannot say the action reduced it. Pull the baseline from the deviation log, complaint data or inspection records as part of the investigation, and record it in the CAPA.
Measurable criteria: examples that work
Vague criteria are the most common weakness. "No recurrence" without a time window or population cannot fail, so it cannot prove anything. Compare these pairs.
A labelling deviation where the wrong lot number was printed on cartons. Weak: "No further labelling errors." Stronger: "Zero lot number discrepancies found in 100% reconciliation of label usage for the next 20 packaging runs on line 2, compared with 3 discrepancies in the previous 20 runs."
A complaint trend about a loose cap on a dosing device. Weak: "Complaint rate reduced." Stronger: "Complaints coded as 'cap detachment' at or below one per 50,000 units shipped for the two quarters after the moulding tool change, measured from the complaint system, against the rate in the two quarters before."
A repeat internal audit finding that controlled copies of SOPs were out of date at workstations. Weak: "Staff reminded of procedure." Stronger: "Spot checks of 15 workstations in each of the next three months find every printed document matching the current effective revision."
A temperature excursion in a stability chamber caused by an alarm that did not escalate out of hours. Stronger criteria here are about the control itself: "Monthly alarm challenge tests for the next six months show the out-of-hours call reaches the on-call engineer within the defined time on every test."
Notice that each strong criterion names a data source, a population, a threshold and a time window. Where the original event was rare, a pure recurrence check may take years to mean anything, and a direct test of the control, as in the alarm example, is the better measure.
Timing: how long to wait before checking
There is no regulatory number of days. The right interval is the one that gives the original failure mode a realistic chance to recur. A daily process with a known defect rate can produce meaningful data in weeks. A failure that showed up twice a year needs a longer window, or a direct challenge of the new control.
Base the interval on how often the activity happens. "Next 30 batches" is better than "three months" when production volume varies. For seasonal or campaign-based manufacturing, make sure the window covers at least one period in which the conditions that caused the problem are present again.
Keep the CAPA open during the effectiveness period, or close the action phase and track the effectiveness check as a dated, owned task that cannot be forgotten. Either is defensible if the procedure says which you do. What an inspector will question is an effectiveness check that was planned for March and never happened.
The CAPA hierarchy of effectiveness
Some actions are far more likely to work than others, and the effectiveness check often fails because the action was weak to begin with. The idea of a hierarchy of effectiveness, borrowed from occupational safety's hierarchy of controls, ranks actions from most to least reliable:
| Level | Type of action | Example |
|---|---|---|
| 1 | Eliminate the cause by design | Change the connector so it can only fit one way |
| 2 | Engineer it out | Add a vision system that rejects mislabelled cartons automatically |
| 3 | Mistake-proof the step | Barcode scan that blocks a run if the label template does not match the work order |
| 4 | Detect it reliably | Add an in-process check with a defined sampling plan |
| 5 | Administrative controls | Revise the SOP, add a checklist, add a second signature |
| 6 | Training or retraining | Brief operators on the revised procedure |
Retraining sits at the bottom for a reason. It depends on memory and attention, both of which degrade over time and with staff turnover. If the investigation concluded that an operator made a mistake, the obvious question is why the process let one person's slip become a nonconforming product. EU GMP Chapter 1, section 1.4(xiv), addresses this directly: where human error is suspected or identified as the cause, this should be justified having taken care that process, procedural or system-based errors have not been overlooked.
Training still has a place. When you change a procedure or install a new control, people need to know how it works. Where it falls short is as the only action. A useful rule for your procedure: if the only action is at levels 5 or 6, the CAPA approver must record why a higher-level action is not feasible, and the effectiveness check should be shorter and more frequent to catch the expected decay.
What to do when an effectiveness check fails
A failed check is information. It means either the root cause was wrong, the action did not address it, or the action was not implemented as intended. Treat it in that order.
Start by confirming the failure is genuinely the same problem. A labelling error caused by a different mechanism is a new nonconformity in its own right, though it may share a systemic cause worth looking at.
If it is the same problem, reopen the investigation as well as the action. Review the root cause evidence with fresh eyes, and check whether the action was implemented across every line, shift and site within scope. Then move the next action up the hierarchy. If retraining failed, the next step is a procedural or physical control; a second round of retraining will fail the same way.
Record the failed check and the reasoning in the same CAPA, or in a linked CAPA that references the original. Do not close the original as effective and open a fresh one quietly. An auditor who finds the same failure mode under two CAPA numbers, the first closed as effective, will conclude that your effectiveness checks are a formality, and that conclusion spreads to every other CAPA in the log.
CAPA closure: what the record should show
At closure, the record should show the baseline, the pre-defined criteria, the data collected, the result against the criteria, who reviewed it, and the decision. If the result was borderline, record the judgement and why it was accepted. The person approving closure should be able to say yes from the record alone.
Take one CAPA closed last quarter and check whether its effectiveness criteria were written before the action, whether they had a number and a time window, and where the action sat in the hierarchy. If any of those is missing, change the CAPA form so the next one cannot be approved without them. For a system that keeps the effectiveness review as a scheduled, owned step in each record, see how InnoQualis handles CAPA.
