The Ultimate Phishing KPI

#infosec

Key Performance Indicators (KPI) are what people in business generally use to measure the performance of a business function or team. In the Cybersecurity world we are blessed (or cursed) with plenty of KPIs. And even in the cyber sub-section of phishing we still have a lot of metrics that email security vendors force on us – Here are some examples:

Now, let's take a step back! Does any of the above metrics make any sense to a normal (non cyber) person? And who is giving you your cyber budget again..?

So what would be a good KPI to measure the performance of your phishing defenses?

To answer that question, we need to look at what it is that we are defending against. The adversaries goal with ordinary run of the mill phishing is usually to harvest valid credentials from your users. What the adversary is doing with those credentials afterwards, is a completely different story. With phishing defenses you are trying to prevent successful credential harvesting FullSTOP.

This goal now easily translates into the Ultimate Phishing KPI:

Report out on that number on a weekly/monthly/quarterly basis and every report recipient knows exactly what you are talking about.

How to measure that?

Now comes the hard part – How the heck do I know how many credentials have been stolen last week? Well, here you have to make the following assumption:

Adversary will test the credentials soon after they have been harvested!

And that credential testing you can monitor for. Use your authentication logs (incl. MFA logs) to look for logins that are being blocked by the MFA challenge and come from unusual locations (unusual location differs from user to user, therefore a ML based tool that can profile your users is helpful here).

Whatelse?

Besides watching the adversary using the harvested credentials you can also observe when your users interact with phishing websites. A pre-requisite for that is that you know what phishing websites to look for. So you need some kind of reporting mechanism that alerts you about a Phishing URL that has been successfully delivered to your users' inboxes. One of these reporting mechanisms we all have to our disposal: Our users. If you train them, they will report phishing emails to you. Then all you have to do is examine the reported email and look through your firewall and EDR logs (a SIEM is helpful here) for users, who have interacted with that phishing website.

What comes out of that exercise is another good KPI:

Sometimes this particular KPI gets you questions about why you use the word “potentially” but those questions can be answered easily and those answers demonstrate that the cyber game is not trivial as it sometimes sounds ;–)