The Measurement Problem

Most security awareness programmes measure inputs rather than outcomes. Training completion rates, phishing simulation click rates, and quiz scores are the standard currency of programme reporting. They are easy to collect, easy to present, and almost entirely useless as indicators of actual security posture.

The fundamental problem is displacement. An organisation with a 98 per cent training completion rate but a poorly designed approval process for financial transactions is not more secure than one with a 60 per cent completion rate and a robust dual-authorisation protocol. Training completion measures whether people sat through a course. It says nothing about whether any subsequent behaviour changed, and even less about whether the organisation is less likely to experience an incident.

This is not a minor technical problem with how metrics are collected. It is a conceptual problem with what the field has decided to measure, and it has a direct consequence: security leaders cannot make a credible evidence-based case to their boards for investment in human-centred programmes because the data they have does not connect to the risk language boards understand.

The COM-B Model as a Measurement Framework

The COM-B model introduced in Part 1 is not only a design tool. It is a diagnostic framework that can structure the measurement of a human security programme with considerably more precision than completion rates allow. By separately assessing capability, opportunity, and motivation for each target behaviour, security teams can identify exactly where a programme is failing and what kind of intervention is needed to address it.

Source: Adapted from Michie et al. (2011). COM-B measurement framework applied to security behaviour contexts.

Leading and Lagging Indicators

The distinction between leading and lagging indicators is borrowed from safety management, where it has a long and successful track record, but it applies with equal force to security (Hopkins, 2009). Lagging indicators measure what has already happened: incident counts, breach costs, and mean time to detection. They are important for accountability and trend analysis, but they are backwards-looking and provide no actionable signal until something has gone wrong.

Leading indicators measure the conditions that predict future outcomes: reporting rates, MFA adoption, time to complete critical patches, and phishing simulation results interpreted correctly as a diagnostic rather than a performance score. A programme that tracks only lagging indicators is managing the past. One that also tracks leading indicators has the information to manage the future.

A Five-Level Maturity Model

Maturity models provide a structured way to assess programme development and communicate progress to stakeholders. The following model describes five levels of maturity in human cyber risk management, from the reactive approaches that characterise most organisations to the predictive capability of the most advanced programmes.

Most organisations sit at Level 2. The transition to Level 3 is the most practically impactful step available, and it does not require sophisticated technology. It requires a shift in the questions being asked: from 'did people complete the training?' to 'did any behaviour change, and what drove the change?'

Communicating Human Risk to Boards

One of the most frequently cited frustrations among security professionals is the difficulty of communicating human risk to boards and executive teams in terms that generate meaningful engagement. The measurement frameworks described above are part of the solution, but the framing matters as much as the data.

Boards understand risk in financial terms. They understand probability and consequence, even if not in those words. A presentation that states '23 per cent of employees clicked on last quarter's phishing simulation' communicates almost nothing to a board, because the board has no reference point for whether 23 per cent is good, bad, or irrelevant to business risk. A presentation that states 'our current phishing susceptibility rate, applied to our transaction volumes, implies an expected annual exposure from business email compromise of approximately £400,000' is a different conversation entirely.

The methodology for converting behavioural metrics into financial risk estimates need not be actuarially precise. It needs to be defensible, transparent in its assumptions, and directionally useful. Start with published industry loss data, apply your organisation's known exposure profile, and adjust for your measured behavioural risk indicators. The conversation that follows will be more productive than any simulation click rate could generate.

A 90-Day Implementation Plan

For practitioners who want to move from the ideas in this series to practical action, the following phased approach provides a starting structure. It is not prescriptive, but it reflects the logical dependencies between diagnosis, design, and measurement.

The Ethical Dimension of Measurement

Programmes that track individual employee behaviour raise legitimate concerns about privacy and the potential for surveillance. Three principles should govern any measurement programme.

Transparency requires that employees know what is being measured and why. Covert individual monitoring, particularly when linked to punitive consequences, erodes the trust that effective security culture depends upon. Research by Lain and colleagues (2022) found that organisations with punitive phishing programmes had lower reporting rates than those with supportive approaches. The surveillance dynamic actively undermines the security outcome it purports to serve.

Aggregation means reporting behavioural metrics at the team or departmental level wherever possible. This provides the diagnostic granularity needed for intervention design without creating an individual performance assessment. Individual-level data should be accessed only when there is a specific, documented justification.

Purpose limitation requires that data collected for security improvement not be repurposed for performance management or disciplinary action. Under the UK GDPR and equivalent frameworks, this is both a legal and an ethical obligation. Compliance with it is also a practical requirement: employees who believe their security behaviour data could be used against them will behave accordingly.

Conclusion: From Awareness to Human Risk Management

Across this four-part series, the argument has been consistent. Understanding why people behave as they do requires theory, specifically dual process theory and the COM-B model. Understanding what makes them exploitable requires knowledge of cognitive biases and how attackers operationalise them. Changing behaviour requires an intervention design that addresses the actual barriers rather than assuming information provision will suffice. And knowing whether any of it has worked requires measurement frameworks that connect behaviour to risk, not to training completion.

The shift from security awareness to human risk management is not a rebrand. It is a substantive change in how the profession understands and addresses its most persistent challenge. The evidence base is mature, the frameworks are practical, and the tools exist. What has been lacking in many organisations is the willingness to measure honestly and to let the evidence determine the response. That willingness is, in the end, the only prerequisite that cannot be designed in by a choice architect.

References

Hopkins, A. (2009). Failure to Learn: The BP Texas City Refinery Disaster. CCH Australia.

Lain, D., Kostiainen, K. and Capkun, S. (2022). Phishing in organizations: Findings from a large-scale and long-term study. In 2022 IEEE Symposium on Security and Privacy (SP), pp.842-859.

Michie, S., van Stralen, M.M. and West, R. (2011). The behaviour change wheel: A new method for characterising and designing behaviour change interventions. Implementation Science, 6(1), p.42.

Service, O. et al. (2014). EAST: Four Simple Ways to Apply Behavioural Insights. Behavioural Insights Team.