Best practices for phishing simulation

While phishing is an “old” problem that dates back to 1980s, it remains one of the biggest cyber security risks companies face today, especially with the latest trends that mix phishing with ransomware. 

Number of unique phishing sites detected worldwide 2013-2022. (Source)

Because phishing exploits human psychology and behavior, a perfect security tool that can 100% prevent phishing and avoid false positives does not exist. Despite the continuous advancement of security tools, attackers also evolve their methods and develop sophisticated and innovative phishing schemes to circumvent these new security tools.  

It is only a matter of time before a phishing email makes it through the security tools and lands in the victim’s mailbox. That is why companies are investing in security awareness training (SAT) to make employees less prone to falling victim to phishing. However, it is equally important to conduct phishing simulations to measure the quality and success of SAT. By performing a phishing simulation, it becomes easier to identify employees who are still susceptible to phishing and need more training, and find areas of improvement for the next SAT.

This article will cover seven phishing simulation best practices to improve simulation quality and benefits. Additionally, we will review common phishing simulation mistakes that can cause disruptions and failures.

Summary of best practices for phishing simulation

The table below lists seven best practices to remember when preparing a phishing simulation in your organization. We’ll cover each best practice in detail later in this article.

Seven phishing simulation best practices

This section will describe seven best practices that can enable a successful phishing simulation.

#1 Define clear objectives and scope

Before starting a phishing simulation, defining its objectives and scope is important. This is especially important for large companies with multiple subsidiaries and departments. Sending phishing emails to thousands or hundreds of thousands of employees simultaneously will only create chaos within the company as employees do not always know where to report such emails and forward the email to other colleagues. 

Moreover, it is often the case that different subsidiaries or branches run different SATs (e.g., due to the language or regulatory requirements) and a one-fits-all phishing simulation might be difficult to organize. That is why organizations must carefully define the scope of the phishing simulation and the recipients of the “phishing” emails beforehand. 

Finally, it is essential to consider what the phishing simulation should achieve as this might affect its structure (email template, content, etc.) and target. For example, assume a security report from the security operations center (SOC) team indicates that most phishing incidents originate from the recruitment department receiving external applications via email. The company might decide to test the effectiveness of SAT for this specific department and conduct a small phishing simulation designed to mimic a typical email that external entities send to this department. 

#2 Perform a security awareness training (SAT) beforehand

While a company could decide only to a phishing simulation without a SAT to measure how employees react to phishing emails and then decide if a SAT is needed, this is not recommended. Organizations should consider that there will always be employees with little to no experience with phishing or security. Conducting a phishing simulation without giving them a chance to learn about phishing and its dangers beforehand would be unfair. This might lead to a feeling of blame, guilt, or frustration, which is not the goal of the phishing simulation. 

Employees should be assured that there is no negative evaluation against them for falling for a simulated phish. Additionally, SAT should always be conducted hand in hand with the phishing simulation to give a “fair chance” to all employees, new and old alike.

#3 Use realistic scenarios

One of the reasons phishing can deceive victims is that attackers always adjust their phishing templates based on real-life events around the world to make them more believable. For example, the start of the Ukraine - Russia war (a 250% increase) or the Covid-19 pandemic (a 220% increase) showed how the global phishing trend changed to incorporate these significant global events. Additionally, using the same phishing examples during the SAT or the phishing simulation might make employees more prone to identifying those exact examples and not adapting to other phishing email flavors. 

That is why the phishing simulation should also adapt and remain up-to-date with the latest phishing trends and use real-life examples.

Real-life phishing email example from IronScale. (Source)

#4 Implement a phishing reporting mechanism

It is important to measure the number of employees who clicked the phishing link or provided their credentials. However, it is equally important to understand how employees handle phishing emails. Do they ignore the phishing email? Do they report it? Do they read most of their emails after a few days? Reporting phishing emails is crucial for the incident response (IR) team, which might start the IR process for a phishing email that was unknown to them or didn't trigger an alert. That is why having a “report phishing” button in your company's email client is important.

 IronScales phishing button for the Outlook email client. (Source)

For example, the “Report Phishing” button from IronScales goes one step further and includes a list of common reasons why the email is suspicious. This might provide interesting insights into how different employees would (maybe differently) categorize the same phishing email. 

#5 Run simulations frequently

Like SAT, phishing simulations need to be conducted frequently, preferably for every SAT your company organizes. Since the content of the SAT can change from training to training, it is still important to measure its quality and results with a corresponding phishing simulation. While this might be too demanding or require a lot of resources, your company can consider outsourcing the phishing simulation preparation to third parties specializing in phishing simulations.

#6 Measure the results

At the end of the phishing simulation test(s), the phishing simulation tool should be able to provide different kinds of reports that would help measure the results of the phishing campaigns. The type of report will greatly depend on the objectives and scope that were specified at the beginning of the phishing simulation. Additionally, since the phishing simulations results might be interesting for different departments, reports that measure different aspects of the phishing simulation might be required. 

For example, the SOC team might be interested in the number of blocked emails or whether they could detect all phishing link clicks. At the same time, the CISO might only be interested in the number of employees falling victim to the phishing simulation. 

#7 Gather feedback and improve

At the end of a phishing simulation, it is important to gather feedback from employees regarding the phishing simulation. The employees can be asked about several aspects of the phishing simulation, such as: 

• How realistic the simulation was
• Did they notice any mistake that made it “obvious”
• How did they recognize that it was a phishing attempt
• If they have any suggestions for further improvements

With every iteration, the phishing simulation can adapt, improve, and become even more realistic.

Five common phishing simulation mistakes

The phishing simulation best practices above provide great examples of how to make your next simulation successful. It’s also important to avoid common phishing simulation mistakes. Below are five mistakes to avoid as you prepare for a phishing simulation.

#1 Not defining a scope

Whenever you simulate a phishing attack, you must specify the scope correctly so that emails are not sent to unnecessary people or outside your company. This might have legal consequences, especially if your phishing emails are sent externally. Ensure the phishing campaign is restricted to only groups or departments of interest.

#2 Not updating allowlists properly

It is crucial to update allowlists with the domain, URLs, or other indicators of compromise (IoCs) you use in the phishing simulation. During the phishing simulation, you will need a domain to send emails and at least one URL that leads to the phishing website. 

Unless you allowlist these IoCs beforehand, your security tools (secure email gateway, Integrated Cloud Email Security (ICES) email server, AV, proxy, etc.) might block the emails or the URLs, thus disrupting the phishing campaign.

#3 Communicating poorly

Not informing the IT, help desk, and other affected parties before a phishing simulation can create confusion within the organization. For example, if a phishing campaign is not announced to the right stakeholders, the help desk or the IT (security) department may incorrectly respond to phishing reports and waste time. Make sure to report the phishing simulation to all affected parties in advance and request approval from the CISO or CIO of the company.

#4 Storing user credentials in logs

While we all hope that no user falls victim to the phishing simulation, this is bound to happen: victims will enter their credentials, which will end up in the logs. It is important not to save or log user passwords because this increases the risk of crediental compromise. 

It is possible to identify users who clicked the phishing link and those who submitted credentials without knowing exactly what password was entered. Adjust your logging format or the web application behind the phishing URL to not store information such as passwords, just the username of the individuals who fell victim to the phishing email.

#5 Not tracking progress

Check whether the phishing simulation was completed, passed/failed, etc. This information could be required for risk insurance, compliance mandates, or internal governance.

Conclusion

With a well-structured phishing simulation, you can adequately gauge SAT effectiveness and business risk. The seven phishing simulation best practices in this article can help organizations conduct effective simulations, as long as they avoid the mistakes that can negatively impact a simulation, such as poor communication and ambiguous scope. 

Although organizing and managing phishing simulations might be a lot of work, there are already solutions in the market that handle most of the process and can take the burden off your shoulders.