Security Awareness Training That Changes Behavior

TL;DR: Security awareness training, as most organizations deliver it, changes little: an eight-month randomized trial at UC San Diego Health found annual courses had no significant effect on phishing clicks and embedded lessons cut clicking by 2%. Interactive lessons did somewhat better. This guide explains why vendor data differs, where training fits among other controls, how to run simulations without losing staff trust, and what to measure instead of click rates.

What security awareness training is and what the evidence says

Security awareness training is the program an organization runs to teach staff to recognize threats and respond to them. A vendor definition puts it plainly: the training "teaches employees how to spot and handle cyber threats" (Proofpoint).

In most large organizations it comes in two forms. The first is an annual course, usually mandatory. The second is embedded phishing training, which researchers at UC San Diego describe as "the practice of sharing anti-phishing information after a user engages with a phishing email sent by their organization as a test" (UC San Diego).

The short answer on the evidence: in a large, independent field test, neither form did much to stop people clicking.

What the UC San Diego Health study found

The researchers ran an eight-month, randomized controlled experiment. They sent 10 phishing campaigns to more than 19,500 employees at UC San Diego Health. Some staff received no training at all, which gave the team a control group.

Clicking also rose over the study period. In the first month, 10% of employees clicked on a phishing link. By the eighth month, more than half had clicked at least once.

The authors' summary is blunt: "Taken together, our results suggest that anti-phishing training programs, in their current and commonly deployed forms, are unlikely to offer significant practical value in reducing phishing risks" (UC San Diego).

How to read this as a security leader

In our reading, the study judges training as it is usually delivered. It does not show that staff behavior cannot change. The qualifier in the authors' own sentence matters: current and commonly deployed forms.

Government guidance points the same way. The UK National Cyber Security Centre (NCSC) warns that phishing mitigations often place too much emphasis on users spotting phishing emails.

One practical step follows now. Before the next renewal, look at what your program reports. Completion rates and click rates show that activity happened. On their own, we think they say little about whether risk went down. The rest of this guide covers what to measure and build instead.

What the large enterprise studies found

The two studies differ in setting, scale and design. Read together, they show what large programs measure, and where the measurement breaks.

The UC San Diego Health trial

The design is what makes this trial useful. The dataset was built with control groups and random assignment: some employees received no training, and others were randomly assigned to different training conditions and lures. Annual training came from KnowBe4. Embedded training ran on the Proofpoint platform.

Three findings stand out beyond the headline results covered earlier.

• Most people never see the training. In any given simulation, a median of 10% of users fail, so the vast majority of users receive no training in that exercise.
• Those who see it leave fast. Over half of all training sessions end within 10 seconds, and less than 24% of users formally complete the materials, according to the paper.
• Format matters, a little. Among users who completed it, static training was negatively correlated with future outcomes, while interactive training cut the likelihood of clicking a later lure by 19%.

In our reading, the lure itself moved results more than training did. Only 1.82% clicked a fake Outlook password update, but 30.8% clicked a fake vacation policy update. In our reading, that is why the authors argue results must always be couched in terms of an underlying control group that receives no training.

Their recommendation points away from content: two-factor authentication and password managers that only work on correct domains.

The Computers & Security study

It tracked click-through rate (CTR).

Its findings were narrower. Employees engaged more with emails that used personalized phrasing. CTR varied between business units. The timing of training before the simulated email did not significantly affect phishing CTR. The authors still conclude that employee awareness will keep mattering alongside technical controls.

In our reading, the studies agree on one practical point: the lure and the audience shape click rates enough that a program without a control group cannot tell whether its training did anything. A security leader can check this directly by asking whether the phishing vendor's dashboard compares trained users against an untrained baseline.

Why vendor data and independent research reach different conclusions

Vendor reports and academic papers often seem to disagree about whether training works. In our reading, much of the gap comes from design: who gets counted, who they are compared with, and what counts as a result. The table sets the four sources side by side. It does not grade any of them.

What the design choices change

The KnowBe4 note matters most. Its starting figure counts all users, but later periods reflect only the subset who received training. We think a falling curve built that way can mix a training effect with changes in who is being measured. The data also spans March 3, 2011 to January 1, 2023, a long window for lure styles and email filters to change.

The SoSafe post reports that susceptibility drops by an average of 40 percent when training follows a click. That is a secondary summary, not new fieldwork.

The academic papers flag their own field's gaps. Ho and colleagues write that earlier research on awareness training largely involved lab settings with university students, and that untrained control groups are rarely included in research or practice.

Questions to put to any results deck

• Is there a control group that received no training over the same period?
• Does every data point count the same people, or only those still enrolled?
• Is the change reported in absolute points or as a relative percentage?
• Were the lures held at similar difficulty from start to finish?
• Is the measure a click, a report, or a real incident?

Training is one layer of phishing defense, not the whole defense

The UK National Cyber Security Centre (NCSC) starts its phishing guidance from a blunt premise. It says phishing mitigations often place too much emphasis on users being able to spot phishing emails, and it treats user education as one part of the defense. It also warns that some phishing attacks will always get through, so plans should assume a click will happen.

The four layers

The NCSC splits its mitigations into four layers:

1. Make it difficult for attackers to reach your users
2. Help users identify and report suspected phishing messages
3. Protect the organization from the effects of undetected phishing emails
4. Respond quickly to incidents

Training sits in the second layer. The NCSC also notes that asking staff to examine every message in depth will not leave enough hours in the day for work tasks.

It offers a metric shift that awareness teams can adopt quickly: as well as measuring how many people clicked on a phishing email, focus on how many people reported it. In our reading, a report rate rewards the behavior that feeds layer four.

Star Blizzard: a campaign built to beat the trained user

The NCSC and partner agencies assess that the actor Star Blizzard is almost certainly subordinate to the Russian Federal Security Service (FSB) Centre 18. Its targets since 2019 have included academia, defence, governmental organisations, think tanks and politicians.

The method is patient. The actor researches each target's real-world social or professional contacts, then writes from accounts that impersonate those contacts. Correspondence can run sometimes over an extended period, as the attacker builds rapport, before any link appears. In our reading, a lure that arrives from a familiar name, in a thread the target has been replying to for weeks, shows none of the warning signs a training module teaches.

The credential step deserves attention. Two-factor authentication, which earlier research in this guide pointed to, is not enough on its own against session-cookie theft, so we think the sign-in method matters as much as its presence.

For a security leader, the practical test is simple. List one control you own in each layer, and check whether your incident runbook covers a compromised personal mailbox belonging to an executive.

How to run phishing simulations without damaging staff trust

Simulations are the part of an awareness program staff feel most directly, and the part most likely to go wrong. The design choices sit with the security leader, and each one either builds trust or wears it down.

What the NCSC warns about

The UK's National Cyber Security Centre (NCSC) names three problems.

• Legal exposure. Punishing people who click on emails the organization itself sent "starts to resemble entrapment" (NCSC). The guidance says to check with your human resources (HR) department before running any simulation (NCSC).
• Blame does not work. People click because of personality traits or circumstances, such as being busy and stressed, and threatening punishment changes neither (NCSC).
• Lost trust. Simulations can erode trust between employees and security, and staff who fear for their jobs will not report mistakes (NCSC).

The NCSC also warns that metrics carry values. A program that seems to prize the absence of problem reports teaches people to stay quiet (NCSC). This is the reason, covered earlier, for counting reports alongside clicks.

Why trained people still click

SoSafe, a training vendor, tells of a trained employee who clicked an email about her salary while a new mortgage was on her mind (SoSafe). Her last training session had taken place twelve months before that email arrived (SoSafe). In our reading, the story makes the NCSC's point in plain terms. In our reading, knowing the warning signs does not hold up against personal worry and time pressure.

The choices a leader controls

A practical check before launch: ask HR to review the lure list and the follow-up wording together. Confirm that simulation results stay out of performance reviews. In our reading, if staff learn of that rule from a written policy before the first test, they have a reason to report their own mistakes.

A first-quarter plan for a security leader who inherits a training program

A new security leader usually inherits three things: a training vendor contract, a dashboard of click rates, and a slide the board has seen many times. The first quarter is a chance to change what that program is for. The order below is our view of a sensible sequence, built on the National Cyber Security Centre (NCSC) guidance covered earlier. For the wider agenda, see our first 90 days plan.

Weeks one to four: audit what is measured

List every metric the program reports and who receives it. The NCSC notes that simulations are popular partly because they allegedly provide clear, quantitative metrics. In our reading, the word allegedly matters. A click rate shows how one campaign landed. It does not show whether the organization can absorb a real attack.

Ask three questions of each metric:

• What decision does it drive?
• What behavior does it reward?
• Has anyone been disciplined because of it?

Bring human resources (HR) into the third question early. The simulation section already covered why.

Weeks five to eight: add controls that do not depend on people

The NCSC says phishing defenses often place too much emphasis on users being able to spot phishing emails. Its first layer is to make it difficult for attackers to reach your users.

• Digital footprint. The guidance suggests turning staff awareness into a clear digital footprint policy for all users, with extra care for high-profile members who attract whaling attacks. Board members belong on that list.

Weeks nine to twelve: make reporting simple and reset the board view

The second NCSC layer is to help users identify and report suspected phishing messages. In practice, we think that means one button in the mail client, a fast human acknowledgment, and a note back when a report caught something real. Staff who feel comfortable reporting can be a valuable early warning system.

Then rewrite the board slide. The NCSC advises building metrics that also include successes. Because some phishing attacks will always get through, the board should also see how fast the team responds.

For framing the conversation, see board cybersecurity oversight and our board committee oversight report.

What this guide does not cover

This guide does not compare, rate or recommend training vendors, and it does not cover course design for threats beyond phishing. It describes what NCSC guidance and the cited studies say about simulations and discipline. It is not legal advice, so take employment questions to your HR and legal teams. For the rules and the record behind it, see CISO First 90 Days: A Week-by-Week Plan for a New Security Leader.