Quality KPIs for a small regulated manufacturer: what to track
Which quality KPIs are worth tracking in a small medical device or pharmaceutical company, how to define each one so it cannot drift, and how to present them to management.
Most small manufacturers already report some quality numbers. The trouble is that the numbers were chosen once, by whoever built the first spreadsheet, and nobody wrote down exactly how each one is calculated. Six months later "CAPA on time" means something different in each monthly report, the training figure sits close to 100% while an operator signs a batch record against an SOP revision they never read, and the management review spends its time arguing about definitions. This article lists the quality KPIs worth tracking in a small medical device or pharmaceutical company, gives a precise definition for each, separates leading from lagging indicators, names the vanity metrics to drop, and shows how to present the set to management.
What the standards ask you to measure
No regulation hands you a list of KPIs. What they require is that you measure and act on the results. ISO 13485:2016 clause 5.4.1 requires quality objectives that are measurable and consistent with the quality policy. Clause 8.2.5 asks for suitable methods to monitor and, as appropriate, measure the processes of the quality management system. Clause 8.4 requires a documented procedure to determine, collect and analyse data, using appropriate methods including statistical techniques, covering feedback, conformity to product requirements, characteristics and trends of processes and product, suppliers, audits and service reports. The results feed management review under clause 5.6.2. The standard itself is on ISO's page for ISO 13485:2016.
For medicines, ICH Q10 section 4.1 expects management review to assess "performance indicators that can be used to monitor the effectiveness of processes" within the quality system, naming the complaint, deviation, CAPA and change management processes, feedback on outsourced activities, self-assessment and external assessments. EU GMP Chapter 1 adopts the same model.
So the choice of metrics is yours. An inspector or auditor will judge whether they tell you anything, whether they are calculated consistently, and whether anyone acted when they moved.
How to define a quality metric so it stays honest
Every KPI needs a short definition record, kept as a controlled document or an appendix to your data analysis procedure. Without one, the number changes whenever the person producing it changes. A useful definition has these fields:
| Field | What to write |
|---|---|
| Question | What decision this metric supports |
| Numerator and denominator | Exactly what is counted, with inclusions and exclusions |
| Clock | The events that start and stop any time measurement |
| Source | The system or log the data comes from |
| Frequency | How often it is calculated and over what window |
| Owner | Who produces it and who acts on it |
| Action rule | The threshold or trend that triggers a response |
The clock is where most metrics go wrong. "CAPA age" can start at the triggering event, at CAPA initiation or at QA approval of the plan, and the difference can be weeks. Pick one, write it down and never change it quietly.
Leading and lagging quality indicators
A lagging indicator reports something that has already happened: complaints, recalls, batch rejections, repeat deviations, major audit findings. It tells you how the system performed, too late to prevent the outcome. A leading indicator measures the condition of the system before it fails: overdue CAPA actions, training not completed before an SOP goes effective, documents past their review date, audit findings ageing without closure.
A small company needs both, roughly balanced. A set built only on lagging metrics looks fine until the quarter it doesn't. A set built only on leading ones measures activity without checking it produced results. Most of the KPIs below come in a pair where a leading measure predicts a lagging one.
The quality KPIs worth tracking
CAPA cycle time and CAPA overdue rate
CAPA cycle time is the number of calendar days from CAPA initiation to closure, reported as the median for CAPAs closed in the period, with the oldest open CAPA shown alongside. Split it into phases (investigation, implementation, effectiveness check) so you can see where the time goes. The median resists distortion from one long-running CAPA better than the mean.
CAPA overdue rate is the number of open CAPAs, or open CAPA actions, past their due date at month end, divided by all open CAPAs at month end. Measure against the original due date and count extensions separately. A process that meets every deadline by extending it is still late.
Pair both with an effectiveness measure, such as the proportion of CAPAs whose effectiveness check found the problem had not recurred. Speed without effectiveness encourages early closure. Our article on checking CAPA effectiveness covers how to design that check.
Deviation recurrence rate
Deviation recurrence is the number of deviations opened in the period that share a root cause category and the same process, product or equipment with a deviation closed in the previous twelve months, divided by all deviations opened in the period. It is one of the most useful lagging indicators a small site has, because recurrence is direct evidence that an investigation or a CAPA did not work.
It depends on a consistent root cause taxonomy. If investigators can choose "human error", "procedure" or "other" freely, the metric measures their habits. A short controlled list of categories and a rule for when to use each are prerequisites. The deviation management guide discusses classification in more detail.
Right-first-time
Right-first-time is the number of batches, lots or device history records completed in the period with no deviation, no rework and no correction requiring QA review, divided by the number completed. Decide up front whether minor good documentation practice corrections count; either answer is defensible, but it must be the same answer every month. Report it by product line or process so a problem on one line is not averaged away.
Training compliance
The usual figure, percentage of the training matrix complete on a given day, is a snapshot that hides the risk. A better measure is training assignments completed by their due date, divided by assignments due in the period, with due dates set so training finishes before an SOP revision becomes effective. Add a lagging check: the number of deviations where the person performing the task was not trained on the current revision. The target for that one is zero. Our article on an inspection-ready GMP training matrix explains how to set the matrix up so this is measurable.
Documents overdue for periodic review
This is the number of controlled documents past their periodic review date, divided by the number of controlled documents with a review date. Report the oldest few by name. A second, optional metric is the median time from change request approval to the revised document going effective, which shows whether document control is a bottleneck for CAPA and change control.
Audit finding closure
Audit finding closure is the number of internal, supplier and external audit findings closed by their agreed date, divided by findings due in the period. Show the age of open major findings separately, since one major finding open for six months matters more than ten minor ones. Track repeat findings, where the same clause or process is cited again within a year; repeats point to a weak root cause analysis on the first occasion.
Complaint rate and trends
Report complaints as a rate against units sold or shipped, because a raw count rises with sales. Trend by failure mode or complaint category over at least twelve months, so a slow rise in one mode is visible even when the total is flat. For device manufacturers, add the time from receipt to the decision on whether the complaint is a reportable serious incident, since the vigilance timelines under MDR Article 87 start running when you become aware.
Avoiding vanity metrics
A vanity metric goes up when people are busy and tells you nothing about control. Common ones include the number of CAPAs closed (which rewards closure whether or not anything was fixed), the number of SOPs written or revised, the number of training sessions delivered, and audits completed against plan presented without findings or closure data. None of them says whether product quality or compliance improved.
The other trap is a metric that can be met by gaming the process. If CAPA cycle time is the only CAPA measure, CAPAs will close before effectiveness is known. If deviation count is the only deviation measure, people will stop raising minor deviations, and an inspector reading a site with very few recorded deviations will ask why. Pair each speed or volume metric with a quality metric, and treat a sudden improvement with the same curiosity as a sudden decline.
Building a quality dashboard for management
Keep it to one page and around eight metrics. For each, show a twelve-month run chart, the target or action threshold, and one or two lines of commentary saying what changed and what is proposed. Colour coding is fine if the rule behind it is written down; a metric turns amber when it crosses a defined threshold.
Small numbers need care. A company that opens four CAPAs a quarter will see its overdue rate jump from zero to 25% because of one late action. Show the count next to every percentage, and look at trends over several periods before drawing conclusions. Where volumes are low, a list of the actual overdue items with owners is often more useful than a ratio.
Finally, review the set itself once a year. Drop metrics nobody acted on, add ones that would have warned you about this year's problems, and record why. That review is a natural item for the clause 5.6 management review.
The takeaway
Choose fewer metrics and define each one precisely: numerator, denominator, clock, source, owner and the threshold that triggers action. Balance leading and lagging indicators, pair every speed metric with a measure of effectiveness, and show counts alongside percentages. A small set that is calculated the same way every month will tell management more than a large dashboard nobody trusts.
These definitions are easier to hold steady when deviations, CAPA, training, documents and audit findings are recorded in one system with consistent dates and statuses, which is what the InnoQualis eQMS platform provides.
