Executing effective simulation exercises and rigorously testing the organization’s incident response capability has been proven to positively impact the organization’s ability to recover. There is sufficient evidence to show that organizations can reduce the cost of a breach by more than 30%1 simply by having an Incident Response (IR) team following a thoroughly tested IR plan. In this document, we will explore the fundamentals of testing an organization’s IR plan using cyber simulation exercise.
The planning of cyber simulation exercises is fundamentally the same across all organizations, with the organizational objectives driving the details of scenario development and execution. Before getting into the details of the exercise, we’ll do some necessary preparation and define some important parameters for the exercise in order to maximize benefits.
Exercise Objective
To prepare for the undertaking of an effective cyber simulation exercise, there are several considerations necessary. The most important is the objective of the exercise. At a high level, the objective should convey the definition of a successful simulation. What does the organization want to accomplish through the exercise?
In this case, we are planning to test our IR plan, but our ultimate objective is to improve our organization’s ability to respond to a cyber incident by improving our IR plan. This improvement to our IR plan encompasses several lower-level objectives including identifying gaps in current policies and procedures, ensuring that team members are familiar with their particular role in the case of an incident, optimizing response processes for efficiencies, and others. But for clarity, the objective should be set at a level that enables crisp communication across the organization, so words like “improve” need to be quantified. We’ll attach our success to specific improvement in our customer communication of an incident. So, the final objective could look like this – Improve security incident response, as measured by the cycle time needed for the first external communication of security-related incidents, by 15%.
Exercise Lead/Planner
As part of exercise preparation, it’s also advisable to assign an exercise lead or planner role that will be responsible to implement and facilitate the exercise planning, execution and follow-up/reporting. This is usually an individual but can also be assigned to a team with various leadership responsibilities split among the team members. This depends on the size and complexity of the organization and the intended size and complexity of the cyber simulation exercise. In any case, the exercise planner will have the largest time commitment to the exercise, so should be prepared to devote the necessary time in order to make the exercise successful.
Exercise Team
Depending on the size and scope of the cyber simulation exercise and the objective set by the organization, it may be necessary to form an exercise team consisting of those in the organization in roles directly responsible for action in the case of an incident. The team size will vary depending on the size and complexity of the organization, but usually ranges from about 4 to 10 people. Some functional groups with members that might be considered for inclusion on the team could be:
- Product Development
- Product Security
- Information Technology (IT)
- Customer Support/Complaint Handling
- Legal
- Compliance/Regulatory
- Human Resources
- Senior Management/Executives
Exercise Scenario
The exercise scenario describes the scope and details of the exercise in a manner sufficient to communicate objectives and expectations. In other words, the scenario actually sets up the exercise and drives the associated exercise activity and planned events. For example, an exercise scenario could be that a former, disgruntled employee secretly modified commercial product source code, prior to leaving the organization. The modified source code that was injected with malware was being shipped to customers as new installations or upgrades, for several months prior to the malware being discovered.
Master Scenario Events List (MSEL)
The Master Scenario Events List (MSEL) is the list of planned exercise events, sometimes referred to as injects. The injects should be easily understood by exercise participants, so that they can respond appropriately. They should be defined sequentially with the timing of the injects simulating a real-world scenario as much as possible, providing the IR team with ample opportunity for realistic testing.
General Format of the MSEL
The MSEL should list the exercise injects in chronological order, for ease of tracking as the exercise progresses.
The MSEL could include the following details for each of the injects:
- Date and time of occurrence (within the exercise)
- Synopsis or description of the event
- Responsible personnel
- Expected action from responsible personnel
- Attachments (for example, an email received from a customer)
- And other details of the injects
NOTE: We have developed a Master Scenario Events List (MSEL) template available for you to download. Just click to download your MSEL template.
Execute
After developing the MSEL, reviewing the plan with the exercise team, clarifying roles and responsibilities, and making sure internal systems are ready to support, it’s time to execute the exercise! Be sure your team planner(s) monitor the exercise properly and record activity detail will be used to develop the Exercise Simulation Report (ESR). The ESR will summarize the exercise and provide details of identified gaps in processes/policies/procedures.
NOTE: We have developed a sample Exercise Simulation Report (ESR) available for you to download. Just click to download your sample ESR.
Report the Results
After developing the ESR, schedule a series of review meetings to discuss the results of the exercise. Schedule the initial meeting with the exercise team, review the ESR, develop exercise communication to be shared with the rest of the organization. Depending on the size and complexity of the organization, it could take several meetings to communicate to all stakeholders. Be consistent by sharing the same basic communication with all stakeholders, slightly modified for specific functions and responsibilities. For example, it would be beneficial to include some treatment of compliance effectiveness when meeting with your Regulatory team.
Cyber Simulation Exercise Summary
To summarize what we have done in our testing exercise, here are the ordered steps that we followed:
- Develop a clear, concise objective for the exercise that can be easily communicated
- Identify exercise lead(s) with the responsibility to manage the exercise
- Assemble the exercise team – usually cross-functional representatives from the organization
- Develop the exercise scenario – have fun with it, but make it realistic for believability
- Based on the exercise scenario, develop the exercise activities, known as events or injects
- Compile developed events into the MSEL (Master Simulation Events List)
- Distribute and review the MSEL with the team, make sure everyone understands their roles!
- EXECUTE THE EXERCISE
- Develop the Exercise Simulation Report
- Share the results of the exercise with the organization
Ken Zalevsky
CEO, Vigilant Ops
Former Head of Medical Device Cybersecurity, Bayer
- 2019 Cost of a Data Breach Report. Ponemon Institute. IBM Security
