How do you measure the success of your security awareness phishing exercises?

The best way to ensure that you accurately measure the success of your security awareness simulated phishing exercises is to not only count the click rate as raw data but to accurately analyse the susceptibility likelihood and report the response rate incorporating the persuasiveness index.

Have you ever looked at the email phishing programs on the market and thought that they just seem to identify the click rate? Knowing that the click rate alone is not enough, is the first step to getting that clarity.

When it comes to helping your organisation avoid phishing attacks, you need science rather than intuition to guide your efforts. Now is the time to abandon your click rate fixation and begin to understand if your phishing awareness efforts are really working or not.

What metrics should you be using in combination to create one complete and accurate picture of whether your training program is meeting its goals or not? How can you determine if your employees are really becoming ever more successful in avoiding phishing attempts with each iteration?

The changes in click-through rates are different depending upon the inherent persuasiveness of the phishing emails. Spear phishing emails have a higher click rate than generic phishing due to the focused nature of the email.

Therefore, measuring a general/average click rate will not provide you with the correct data to evaluate the success of your phishing program.

Cypro is now providing a newly developed social engineering framework called the PC-ratio.

This involves the persuasive index of the attack, correlated to the capture rate and the reporting index.

What we really need to do is look at:

• How many people click the link?
• Who reports the phishing email?
• How persuasive was the email?
• Was the person’s role more susceptible to this type of email?
• What was the Mean Time between clicks?

This data is then matched to other user behaviour information and other data. We based the persuasiveness scoring on Cialddini’s six key principles, reciprocity, scarcity, authority, commitment, consistency, liking, and consensus (or social proof).

Persuasiveness

The principle behind it is the persuasiveness of the phishing attack email is how easily persuaded would someone be by the different types of emails.

1. Generic. These are the common basic Nigerian scams and lost inheritance or massive prize giveaway scams. These should not be terribly hard to identify, and most people should not be susceptible to falling for them.
2. General. These are the common scams using Banks, ATO, retail stores, and other common scams. Most people these days are aware of these and under normal circumstances, they do not fall for them.
3. Internal. These are the most common Spear Phishing emails that we see today. They are coming, supposedly, from within the organisation or trusted companies. Many people fall for these are they believe it is a trusted source.
4. BEC. (Business Email Compromise) These are very targeted emails coming from an executive within the organisation to request specific actions. These are often very well crafted and hard to identify. Due to their nature, these are quite hard to detect and the person receiving them often feels pressured to respond to them.

Each of these categories has differing weights to reflect their level of persuasiveness and the expected difficulty in detecting them.

We also need to consider the position of the person being targeted. Is the email type and persuasiveness relevant to that person?

Click Rate

Another measure that is taken into account to calculate the susceptibility likelihood is the Click rate. This is an obvious metric, but it must be taken into account with the other factors via the algorithm.

Response Index

To understand staff behaviour, we also need to take into account the Response Index. This is the measurement of the staff reporting the suspected phishing exercise, did they just open it, delete it, click the link etc. These numbers allow us to better understand how staff responded to the phishing email and how they may respond in the future.

Repeat Offence

Finally, we need to consider repeat offenders. Are some staff members continuously falling for these simulated attacks, subsequently not reporting them, and placing the organisation at risk?

Calculation

All these factors are then combined using the calculation below to provide us with the Susceptibility Likelihood or more commonly called the Human Risk.

All these factors are then combined using the calculation below to provide us with the Susceptibility Likelihood or more commonly called the Human Phishing Risk.

All factors have algorithms attached to them so when they are placed within this simple formula, they provide a risk score for each staff member within the organisation, allowing you to continue to measure their success and response to training.