Audit Trail Review That Actually Finds Things

Most audit trail reviews get signed off. Far fewer would catch a problem.

Ask most sites whether they review audit trails and the answer is yes. Ask what the last review found and the answer is often a pause. In many organizations, audit trail review has become a signature on a checklist: someone opens the trail, scrolls, and records "reviewed, no issues."

That satisfies the procedure. It doesn't do what the review is for, which is detecting changes to critical data that weren't appropriate, authorized or explained. Here's how to design a review that can actually find things.

What regulators expect

The expectation is well established. FDA's data integrity guidance for CGMP says audit trails that capture changes to critical data should be reviewed with each record, before the record is approved. MHRA's GxP data integrity guidance and PIC/S guidance PI 041 set out similar expectations. EU GMP Annex 11 requires audit trails for GMP-relevant changes and deletions, and the draft revision of Annex 11 sets out audit trail expectations in much more detail.

None of these asks you to read every line of every audit trail. They ask for a review that is risk-based, effective and documented.

1. Start with critical data, not the whole trail

A chromatography data system can log thousands of events a day. Most don't matter for product quality. Decide in advance which data is critical, such as reportable results, integration parameters, sample identities, specifications and batch-related entries, and which events affecting that data the reviewer must look at.

Events that usually deserve attention:

• Modifications or deletions of results, and anything marked as reprocessed or reintegrated
• Runs or injections that were aborted, invalidated or excluded
• Changes to methods, processing parameters or specifications
• System clock or time zone changes
• User privilege changes and administrator activity
• Repeated failed log-ins or unusual access times

2. Separate record-level and system-level review

Two different reviews often get blurred into one:

• Record-level review happens with each batch, result or record before approval. The reviewer asks whether anything happened to this record's data that shouldn't have, or that isn't explained.
• System-level review happens periodically. It looks across the system at configuration changes, user account management, administrator actions, audit trail settings and security events.

Each needs its own owner, its own frequency and its own place in your procedures. Record-level review usually sits with the person reviewing the data. System-level review usually sits with QA or the system owner, and should be independent of the system administrators whose actions it covers.

3. Make sure the trail can show what you need

An audit trail review can only be as good as the audit trail itself. During assurance testing, and again at periodic review, confirm that:

• The audit trail is enabled and can't be switched off by ordinary users or the people whose actions it records
• It captures who, what, when and why: the user, the old and new values, a reliable timestamp and a reason for change where needed
• Entries can't be edited or deleted
• Any report or filter used for review has been verified to show the right events completely

That last point is often missed. If reviewers rely on a filtered report and the filter drops an event type, the review is blind to it, every time.

4. Tell reviewers what to look for

"Review the audit trail" isn't an instruction anyone can follow consistently. Give reviewers a short, system-specific guide: which report to run, which events to examine, and what normal and abnormal look like. Red flags include:

• Results reprocessed several times until they pass
• Data deleted or excluded without a documented reason
• Activity at unusual times or by unexpected users
• Gaps in sequences, or test or trial injections that aren't part of the reported run
• Changes made after a result was reported or approved
• Shared or generic accounts appearing in the trail

5. Set frequency by risk

Record-level review of high-risk data belongs with every record. System-level review can be monthly, quarterly or tied to periodic review, depending on the system's risk and how much administrative activity it sees. Write down the rationale. An inspector will ask why you chose the frequency, not just what it is.

6. Document the review, not just the signature

"Reviewed, no issues" tells an inspector almost nothing. A useful record shows what was reviewed (which records, which report, which period), what was found, and what happened to anything unusual: whether it was explained, escalated or turned into a deviation.

Reviews that never find anything deserve a second look. Either the system is unusually clean, or the review isn't looking in the right places.

7. Use technology, and assure it

Exception reports, review-by-exception dashboards and automated flags can make review faster and more effective, especially in high-volume systems. Treat them as part of the computerized system. They need risk-based assurance like any other function, because a review tool that misses events is worse than none: it creates false confidence.

Common traps

• Reviewing everything, so nothing gets real attention
• Reviewing after release, when a finding can no longer stop a bad result
• Administrators reviewing their own activity
• Assuming the vendor's default audit trail settings are adequate without checking
• Training reviewers once and never again, even after system upgrades change what the trail shows