Continuous control monitoring: how to test controls automatically between audits

A surveillance audit can surface failures that nobody noticed all year: a leaver whose account stayed active for months, a storage bucket created without encryption, a backup job that had been failing silently. Each of those controls can be well designed and still fail, because nobody checks it between audits.

Continuous control monitoring closes that gap: automated tests check each control's own data on a schedule and alert its owner when it fails. This guide is for heads of engineering, security leads and compliance managers at UK B2B tech firms facing SOC 2 Type 2 or ISO 27001 surveillance audits. It shows how to specify each monitored control as a test card, set frequencies and thresholds by risk, handle failures, and turn the results into evidence that your controls operated all year.

What is continuous control monitoring?

Continuous control monitoring is a set of automated tests that check a control's own data on a schedule, from hourly to monthly, compare the result with a pass threshold and alert the control owner when it fails. It tests whether controls work, not whether attacks happen, and it turns year-round operation into audit evidence.

The idea comes from a wider discipline. NIST SP 800-137 defines information security continuous monitoring as maintaining ongoing awareness of information security, vulnerabilities and threats to support risk management decisions. Our NIST 800-137 page covers the publication itself.

One note in that definition matters more than any other. NIST says "continuous" and "ongoing" mean that controls and risks are assessed at a frequency sufficient to support risk-based decisions. Continuous does not mean every second. It means often enough that a failure is found while it is still small.

How is continuous control monitoring different from security monitoring and periodic testing?

Continuous control monitoring differs from security event monitoring because it asks whether a control is working, not whether an attack is under way. It differs from periodic manual testing because it runs on a schedule against whole populations rather than samples taken once a year. The three answer different questions and complement each other.

Security event monitoring sits under controls such as ISO 27001 Annex A 8.16 monitoring activities, which covers watching networks, systems and applications for anomalous behaviour. A SIEM will tell you someone logged in from an unusual country. It will not tell you that a leaver's account is still enabled.

Periodic testing still has a place wherever no machine-readable data exists. Continuous control monitoring takes the repetitive, data-driven checks off the tester's list.

Which controls suit continuous control monitoring?

Controls suit continuous control monitoring when their evidence is machine-readable and their pass condition is objective: an account is enabled or it is not, a disk is encrypted or it is not. Controls that need judgement, such as whether a risk assessment is sound or a policy fits the business, stay with people.

NIST makes the same point plainly. SP 800-137 says organisation-wide monitoring cannot be achieved efficiently through manual processes alone or automated processes alone. It adds that process-related controls, such as procedures and templates, do not tend to be volatile and typically do not lend themselves well to automation.

A quick test for each candidate control:

1. Is the data in a system you can query? Identity providers, cloud configuration, endpoint management, backup consoles and ticketing tools usually qualify.
2. Can you write the pass condition as a rule? "Every active user has MFA enrolled" qualifies. "Access is appropriate" does not, until you define appropriate.
3. Does the control change often? Volatile controls gain most from frequent testing.
4. Would a failure matter before the next audit? If not, a quarterly manual check may be enough.

Design effectiveness stays manual. A test can prove a control ran; only a person can judge whether it was the right control. That judgement belongs to the control-testing work your auditors and internal reviewers already do.

What goes on a continuous control monitoring test card?

A continuous control monitoring test card is a short specification for one automated test. It records the assertion being tested, the data source, the population, the frequency, the pass threshold, the owner and the escalation on failure. Writing the card before building anything keeps the test honest and gives an auditor something to read.

We recommend keeping each card to one assertion. A test that checks three things at once fails for unclear reasons and is hard to defend.

What do test cards look like for six common controls?

Six controls make good first test cards because their data is machine-readable and their failures are costly: MFA coverage, leaver deprovisioning, critical patch age, backups with restore tests, encryption at rest and privileged access reviews. The examples below are illustrative. Set each threshold from your own policy and risk appetite.

MFA coverage. The identity team owns failures. Count accounts that can sign in without MFA, not only accounts without a device enrolled: a policy exception can bypass enrolment. Escalate to the security lead if any account stays non-compliant for more than a day.

Leaver deprovisioning. The HR source matters as much as the identity data. If leavers reach the identity team by email, the test cannot see them. Owner: IT. Escalate to the head of engineering when the SLA is missed.

Critical patch age. If you hold or want Cyber Essentials, the requirements for IT infrastructure (v3.3) set the window: updates fixing critical or high-risk vulnerabilities must be installed within 14 days of release. Owner: the platform or IT lead.

Backup success and restore. A successful backup job proves very little on its own. The second assertion, that a restore was tested, is the one that protects you.

Encryption at rest. This control changes every time someone creates a resource, so test it on change where your cloud platform allows. Owner: the platform team.

Privileged access review. The test checks that the review happened and its removals took effect. Whether the reviewer judged well is a manual question.

How often should each control be tested, and what counts as a pass?

Test each control as often as its risk demands: frequently when the control changes often or protects critical systems, less often when it is stable. Set the pass threshold from your policy, with zero tolerance where one failure is serious. The frequency decides how long a failure can sit unnoticed, so tie it to impact.

NIST SP 800-137 lists the factors to weigh. The first is security control volatility, which NIST describes as how often a control is likely to change after implementation. Configuration controls are its example of volatile controls; personnel screening is its example of a stable one. Controls on high-impact systems, controls that provide critical security functions and controls with known weaknesses also earn more frequent monitoring.

For a lean firm, three tiers are usually enough:

1. Hourly or on change for configuration that drifts, such as encryption and public exposure of storage.
2. Daily for identity and patching, where a day's delay already matters.
3. Weekly or monthly for slower controls, such as restore tests and review completion.

Thresholds need the same care. Some controls allow no tolerance: one unencrypted production database is a failure. Others need a defined grace period, such as a new device that has not yet received its first patch cycle. Write the tolerance on the card, so a pass means the same thing every time.

What happens when a continuous control monitoring test fails?

When a continuous control monitoring test fails, a ticket opens for the named owner with a time to fix set by severity. The owner fixes the cause, the test re-runs to confirm, and the record closes. If the business decides to tolerate the failure, that becomes a documented risk acceptance with an approver and an expiry date.

A workable exception process has five parts:

1. A ticket, not a message. Each failure creates a ticket with the test name, the failing items and the time of detection.
2. A time to fix by severity. We recommend agreeing these in advance, so nobody negotiates during an incident.
3. A retest. The ticket closes only when the next run passes.
4. Escalation. A missed time to fix goes to the escalation contact on the card.
5. Risk acceptance. When a failure is tolerated, record who approved it, why, what compensates for it and when it expires.

Repeated failures of the same control are a different problem. They suggest the control is badly designed, not badly run, and belong in your corrective action process, such as ISO 27001 clause 10.2 on nonconformity and corrective action.

How do you stop continuous control monitoring creating alert fatigue?

Stop alert fatigue by sending each failure to one named owner, grouping repeats into one open ticket, and tuning any test that fails without leading to action. Low-severity failures can go into a daily digest. Review noisy tests every month, because a test people ignore is worse than no test: it looks like control.

Practical habits that keep the signal clean:

• One alert per failing control, not per failing item. Fifty devices past the patch window is one ticket with a list.
• No shared channels for failures. Everyone sees the alert, so nobody owns it.
• Tune or retire. If a test fails often and nobody acts, the threshold, the population or the test is wrong.

How do continuous control monitoring results become audit evidence?

Continuous control monitoring results become audit evidence when each test run is kept as a dated record showing the population, the result and the follow-up of every failure. For a SOC 2 Type 2 report, they show operation across the period. For ISO 27001, they feed clause 9.1 monitoring and the evaluation it requires.

SOC 2 Type 2. A Type 2 report covers whether controls operated effectively across a period, not on a single date. Our Trust Services Criteria guide explains the common criteria. CC4.1 asks the organisation to select, develop and perform ongoing and/or separate evaluations of whether the components of internal control are present and functioning. A continuous control monitoring programme is a direct answer to the ongoing half.

ISO 27001 clause 9.1. The clause asks you to decide what to monitor and measure, how, when, and who analyses and evaluates the results. Our clause 9.1 guide covers what auditors sample. Test cards answer most of those decisions on paper. The evaluation, a person judging what the results mean, still has to be recorded.

Three conditions make auditors more willing to rely on the results:

1. The test itself is controlled. Changes to test logic go through change management, and the card records when each version applied.
2. The population is complete. You can show the test read every in-scope item, not only those the tool could reach.
3. Failures are visible. A history with no failures at all invites more questions than one with failures found and fixed.

Your auditor decides how much to rely on your monitoring, and will still test some controls directly. Strong monitoring records can narrow that testing; they never remove it.

How do you roll out continuous control monitoring, starting with five controls?

Roll out continuous control monitoring by choosing five controls with machine-readable data and costly failures, writing a test card for each, running the tests silently for a few weeks to clear the backlog, then switching alerts on. Review results monthly, and add controls only when the first five run cleanly.

A short sequence that works for a lean team:

1. Pick five. MFA coverage, leaver deprovisioning, critical patch age, backup success and encryption at rest are a strong start.
2. Write the cards. Agree owners and thresholds with the people who will receive the alerts.
3. Build the tests. Point each one at the data source and population named on its card.
4. Run in shadow mode. Fix the existing failures before anyone is paged.
5. Switch on alerts and tickets. Start the evidence record from this date.
6. Review monthly. Check noisy tests, missed times to fix and coverage gaps, then add the next controls.

Five controls tested every day give you more assurance than fifty checked once a year.

What are the common mistakes with continuous control monitoring?

The common mistakes with continuous control monitoring are monitoring what is easy rather than what matters, testing an incomplete population, sending alerts to nobody in particular, and treating automated passes as proof of good design. Each one makes the dashboard look healthier than the controls behind it.

• Easy over important. Building first whatever the tooling already exports, rather than the controls that failed at the last audit.
• Silent scope gaps. A test that reads only one cloud account passes while the others drift.
• Ownerless alerts. Alerts without a named owner age into background noise.
• Passes as design proof. A passing test shows the rule held, not that it was the right rule. Keep design reviews on the calendar.
• No history. Overwriting results each run destroys the evidence you built the tests to create.

Frequently asked questions