Executing effective simulation exercises and rigorously testing the organization’s incident response capability has been proven to positively impact the organization’s ability to recover. There is sufficient evidence to show that organizations can reduce the cost of a breach by more than 30%1 simply by having an Incident Response (IR) team following a thoroughly tested IR plan. In this document, we will explore the fundamentals of testing an organization’s IR plan using cyber simulation exercise.
The planning of cyber simulation exercises is fundamentally the same across all organizations, with the organizational objectives driving the details of scenario development and execution. Before getting into the details of the exercise, we’ll do some necessary preparation and define some important parameters for the exercise in order to maximize benefits.
To prepare for the undertaking of an effective cyber simulation exercise, there are several considerations necessary. The most important is the objective of the exercise. At a high level, the objective should convey the definition of a successful simulation. What does the organization want to accomplish through the exercise?
In this case, we are planning to test our IR plan, but our ultimate objective is to improve our organization’s ability to respond to a cyber incident by improving our IR plan. This improvement to our IR plan encompasses several lower-level objectives including identifying gaps in current policies and procedures, ensuring that team members are familiar with their particular role in the case of an incident, optimizing response processes for efficiencies, and others. But for clarity, the objective should be set at a level that enables crisp communication across the organization, so words like “improve” need to be quantified. We’ll attach our success to specific improvement in our customer communication of an incident. So, the final objective could look like this – Improve security incident response, as measured by the cycle time needed for the first external communication of security-related incidents, by 15%.
As part of exercise preparation, it’s also advisable to assign an exercise lead or planner role that will be responsible to implement and facilitate the exercise planning, execution and follow-up/reporting. This is usually an individual but can also be assigned to a team with various leadership responsibilities split among the team members. This depends on the size and complexity of the organization and the intended size and complexity of the cyber simulation exercise. In any case, the exercise planner will have the largest time commitment to the exercise, so should be prepared to devote the necessary time in order to make the exercise successful.
Depending on the size and scope of the cyber simulation exercise and the objective set by the organization, it may be necessary to form an exercise team consisting of those in the organization in roles directly responsible for action in the case of an incident. The team size will vary depending on the size and complexity of the organization, but usually ranges from about 4 to 10 people. Some functional groups with members that might be considered for inclusion on the team could be:
The exercise scenario describes the scope and details of the exercise in a manner sufficient to communicate objectives and expectations. In other words, the scenario actually sets up the exercise and drives the associated exercise activity and planned events. For example, an exercise scenario could be that a former, disgruntled employee secretly modified commercial product source code, prior to leaving the organization. The modified source code that was injected with malware was being shipped to customers as new installations or upgrades, for several months prior to the malware being discovered.
The Master Scenario Events List (MSEL) is the list of planned exercise events, sometimes referred to as injects. The injects should be easily understood by exercise participants, so that they can respond appropriately. They should be defined sequentially with the timing of the injects simulating a real-world scenario as much as possible, providing the IR team with ample opportunity for realistic testing.
The MSEL should list the exercise injects in chronological order, for ease of tracking as the exercise progresses.
The MSEL could include the following details for each of the injects:
NOTE: We have developed a Master Scenario Events List (MSEL) template available for you to download. Just click to download your MSEL template.
After developing the MSEL, reviewing the plan with the exercise team, clarifying roles and responsibilities, and making sure internal systems are ready to support, it’s time to execute the exercise! Be sure your team planner(s) monitor the exercise properly and record activity detail will be used to develop the Exercise Simulation Report (ESR). The ESR will summarize the exercise and provide details of identified gaps in processes/policies/procedures.
NOTE: We have developed a sample Exercise Simulation Report (ESR) available for you to download. Just click to download your sample ESR.
After developing the ESR, schedule a series of review meetings to discuss the results of the exercise. Schedule the initial meeting with the exercise team, review the ESR, develop exercise communication to be shared with the rest of the organization. Depending on the size and complexity of the organization, it could take several meetings to communicate to all stakeholders. Be consistent by sharing the same basic communication with all stakeholders, slightly modified for specific functions and responsibilities. For example, it would be beneficial to include some treatment of compliance effectiveness when meeting with your Regulatory team.
To summarize what we have done in our testing exercise, here are the ordered steps that we followed:
CEO, Vigilant Ops
Former Head of Medical Device Cybersecurity, Bayer
Founded in 2019, Vigilant Ops is an innovator in the medical device cybersecurity industry. Led by seasoned medical device cybersecurity experts with more than forty years of combined experience, Vigilant Ops provides medical device manufacturers and hospitals with unprecedented insight into device risk profiles, enabling proactive management of threats before they impact the quality of patient care.