Incident Response (IR) Exercise Phases
Establish Exercise Interval and Planning timelines: First, establish the IR exercise execution interval (i.e., Annally, bi-annually, quarterly, monthly, etc). From the set interval back track time for planning and approval of exercises.
Exercise Design & Development: Guided by leadership intent and program priorities, this phase involves using risk assessments, plans, policies, and past After-Action Reports (AARs) to design exercises that assess and validate organizational capabilities.
Exercise Execution: Covers preparation, management, and wrap-up activities. Discussion-based exercises focus on facilitation and dialogue, while operations-based exercises span activities between StartEx and EndEx.
Exercise Evaluation: Measures performance against objectives, identifying strengths and areas for improvement, and is integrated throughout the planning and improvement cycle.
Improvement Planning: Focuses on addressing identified gaps to refine planning, training, and operational capabilities, driving continuous improvement in preparedness.
Exercise Design and Development
Main subphases of Exercise Design & Development
Concept Development Meeting
Design Scenario for Exercises
Key scenario elements
Exercise style
Defining objectives and outcome
Identifying additional organizations or partners for the exercise planning meeting
Exercise Approval
Exercise Planning Meeting
Exercise Planning documentation and Scheduling to Approvers
Concept Development Meeting
Establishes the exercise framework by drafting a high-level scenario
Selecting the exercise style (e.g., tabletop, hybrid, or live)
Defining objectives and outcomes
Identifying additional organizations or partners for the initial planning meeting
Design Scenario for Exercises:
The scenario serves as the foundation of the exercise, integrating realistic threats into a cohesive and plausible storyline. Each scenario element must align with and support the exercise objectives.
Key Scenario Elements:
Scenario Objectives: Break down exercise objectives into actionable activities, structured as event threads.
Road to War: Provide an overview of the situation leading up to the exercise.
Threat: Define threat actors, their motivations, tactics, techniques, and procedures (TTPs), and the potential use of live OPFOR (Opposing Force).
Target: Identify the systems, information/data, people, and processes at risk.
Operational Effect: Focus on the intended impacts on the target, including discovery, timeframes, and operational disruptions (beyond just business impact).
Common Exercise Objectives:
Evaluate the effectiveness of pre-exercise cyber education.
Assess incident reporting and analysis guides for addressing deficiencies.
Test the training audience’s ability to detect and respond to hostile activity.
Measure the organization’s capacity to assess operational impacts and implement recovery from cyberattacks.
Validate scenario planning and execution among key stakeholders.
Explore the implications of losing IT system trust and identify workarounds.
Identify and address weaknesses in cybersecurity systems.
Highlight and improve gaps in cyber operations policies and procedures.
Determine enhancements needed to protect systems and operate in hostile environments.
Ensure training injects align with objectives.
Strengthen cyber awareness, readiness, and coordination.
Selecting the exercise style (e.g., tabletop, hybrid, or live)
There are two primary categories of exercises: Discussion-Based and Operations-Based.
Discussion-Based Exercises focus on familiarizing participants with plans, policies, agreements, and procedures. Examples include:
Tabletop Exercises (TTX)
Seminars
Workshops
Games
Operations-Based Exercises validate plans, policies, agreements, and procedures while defining roles and responsibilities. Examples include:
Drills
Functional Exercises
Full-Scale Exercises
Defining objectives and outcomes
The Defining Objectives and Outcomes phase is a critical step in incident response exercise planning. It sets the foundation for the exercise by establishing clear, measurable goals and desired results. Key aspects include:
Aligning with Organizational Priorities:
Objectives are tailored to address specific organizational needs, such as improving detection, response, and recovery capabilities.
Outcomes reflect the organization's goals, including validating policies, enhancing team coordination, or uncovering process weaknesses.
Ensuring Clarity and Focus:
Well-defined objectives provide a clear framework for all exercise activities.
Each objective is actionable, measurable, and tied to specific capabilities or processes to be tested.
Promoting Realism and Relevance:
Objectives are aligned with real-world threats and scenarios relevant to the organization’s environment.
Outcomes are designed to provide practical insights into preparedness and areas for improvement.
Facilitating Stakeholder Engagement:
Engaging stakeholders early ensures objectives reflect diverse perspectives, including IT, security, legal, and leadership priorities.
Collaboration helps align outcomes with strategic goals and operational requirements.
Identifying additional organizations or partners for the exercise planning meeting
Effective planning for an incident response exercise relies on collaboration with key stakeholders, including subject matter experts (SMEs) and trusted agents. Their input ensures the exercise is comprehensive, realistic, and aligned with organizational needs.
Key partner input SMEs (A single person could be a SME in more than one of the categories):
Threat Identification and Relevance SME:
Leverage SME expertise to define credible and relevant threats based on the organization's threat landscape.
Cyber Defense Capabilities SME:
Evaluate existing cyber defense tools, technologies, and processes to incorporate them effectively into the exercise.
Highlight key detection, response, and recovery capabilities to be tested during the exercise.
Align exercise scenarios to validate the efficacy of these systems and identify gaps.
Policies and Procedures SME:
Review and incorporate organizational policies and incident response procedures to ensure the exercise aligns with established protocols.
Use the exercise to validate and refine these policies, ensuring they are actionable during real-world incidents.
Understands coordination with other plans like Disaster recovery.
IT Operations SME:
Focuses on response procedures for end-user systems and devices, such as re-imaging infected endpoints.
Provides insights into practical recovery steps for user-facing technology during an incident.
IT Engineering SME:
Specializes in response procedures for network-based defenses like firewalls, web proxies, and intrusion detection/prevention systems.
Ensures realistic simulation of network-level incident response capabilities.
Exercise Approval
Internal approval to proceed with exercise planning meeting. The results of the exercise planning meeting should be a fully planned and scheduled exercise.
Exercise Planning meeting
The IPM establishes the framework and key details for the exercise. Key outcomes include:
Exercise Objectives: Finalized and clearly defined.
Scenario Development: Agreement on the exercise scenario.
Rules of Engagement (ROE): Established in collaboration with the Response Team (RT).
Timeline: Scheduled follow-up planning sessions and exercise execution dates.
Point of Contact (POC) List: Compiled and shared comprehensively.
Assigned Oversight Responsibilities: Designated planners to manage:
Notifications and coordination with internal and external organizations/partners.
Logistics, including exercise location and other operational needs.
Required resources, such as network ranges, types, and diagrams.
Action Items: Assigned with clear deadlines and responsible POCs.
Scenario Documentation (MSword and Powerpoint):
Scenario and Mapping to Threat
MSEL & Objective
Type of exercise
Key scenario elements
Scenario Participants:
Primary
Secondary (On call)
Facilitation guide with injects:
Incident Setup
Initial Incident alert/notification (simulated event data)
Triage/identification (using simulated data and tools)
Incident declaration
Communications/Escalation
Coordination with other departments and plans
Incident Containment (using simulated tools)
Incident Eradication
Incident Recovery
Incident lessons Learned
MSEL Updates: Progressed planning for the Master Scenario Events List (MSEL).
The MSEL "Master Scenario Event List" is an excel file that has:
Scenario, Mapping to threat, Scenario Objective, Scenario Planning Date, Scenario Planning approval date, Scenario document link (Powerpoint, word), Scenario execution date/time, link to incident response report & AAR (added to the MSEL at the conclusion of the exercise)
Exercise Planning documentation and Scheduling to Approvers
This lets the approvers see final plans and make any last minute recommendations as well as lets approvers know the schedule date/time for the exercise.
Exercise Conduct
Take roll
For: Tabletop Exercises (TTX), Seminars, Workshops
(Note: The Tabletop Exercises (TTX), Seminars, Workshops can also include drills and functional exercises. For example, do research with a query XYZ and interpret results)
Facilitator walk through Scenario Powerpoint:
Scenario and Mapping to Threat
MSEL & Objective
Type of exercise
Key scenario elements
Scenario Participants:
Primary
Secondary (On call)
Facilitation guide with injects:
Incident Setup
Initial Incident alert/notification (simulated event)
Triage/identification (using simulated data and tools)
Incident declaration
Communications/Escalation
Coordination with other departments and plans
Incident Containment (using simulated tools)
Incident Eradication
Incident Recovery
Incident lessons Learned
Participant hot wash - Opportunity for exercise participants to discuss exercise strengths and areas for improvement immediately following the conduct of an exercise.
For: Drills, Functional Exercises, Full-Scale Exercises
Facilitator walk through Scenario Powerpoint (At key IR phases implement the drill or functional exercise instead of having a discussion about response steps and actions at that phase):
Scenario and Mapping to Threat
MSEL
Type of exercise
Key scenario elements
Scenario Participants:
Primary
Secondary (On call)
Facilitation guide with injects:
Facilitator requests participants conduct drills and functional exercises (using simulated data and tools)
Participant hot wash - Opportunity for exercise participants to discuss exercise strengths and areas for improvement immediately following the conduct of an exercise.
Simulated data and tools
To enhance realism, exercises should prioritize the use of automated simulated data and tools, often referred to as "cyber ranges." These environments mimic real-world conditions, allowing participants to engage with realistic scenarios. However, creating a fully functional and realistic cyber range can be time-intensive and cost-prohibitive.
Key Considerations:
Cyber Ranges:
Commercially available cyber ranges exist but are often generic and may not reflect an organization's specific data, tools, or environment.
Customizing cyber ranges to align with organizational systems provides greater authenticity but may require significant resources.
Use of existing Organizational Data:
In the absence of a cyber range, real organizational data and systems can be used for exercises, as incident response activities generally do not alter or delete data.
Strict precautions must be taken to ensure no sensitive data is exposed during the exercise.
Leveraging Penetration Testing Data:
Insights from successful penetration tests can be valuable for simulating threat actor activities and crafting realistic scenarios.
Limitations of Using Real Data and Systems:
Some activities, such as malware deployment or command-and-control simulations, may be restricted to avoid disrupting production systems.
Response processes like isolating or re-imaging systems may also be limited due to operational constraints.
Balancing realism with practicality ensures exercises are effective while maintaining operational security and minimizing disruption. Using a mix of cyber ranges, penetration testing data, and real organizational systems can help create a realistic and impactful incident response training experience.
Exercise Evaluation
Exercise Debrief: Debrief is a meeting for controllers, facilitators, and evaluators to
assemble after the exercise to discuss observations and provide information to create a shared understanding of the exercise.
Exercise Incident Response Report with AAR: Drafting of an incident response report for the exercise. This function also serves as practice for drafting real incident response reports.
Exercise Incident Response Report AAR review Meeting: Treated like an actual incident response AAR meeting.
Exercise review meeting: This meeting could have a lot of overlap with the 'Exercise Incident Response Report AAR review Meeting'; but is focused with what went right and wrong with the exercise. Some guiding questions for this meeting:
What happened? What was supposed to happen based on current plans, policies, and
procedures?
Was there a difference? What was the impact?
Were the consequences of the action (or inaction/decision) positive, negative, or neutral?
Do plans, policies, and procedures support activities and associated tasks?
Are participants familiar with these documents?
What are the strengths and areas of improvement to remedy deficiencies?
Improvement Planning
Add incident response report with AAR to MSEL
Update all IR policies, plans, and SOPs with relevant information learned from exercise
Create any new IR SOPs related to IR exercise
Ensure personnel responsible for IR functions are trained on updates to IR SOPs and how to execute added capabilities and procedures to organizational standards.
Ensure IR personnel who missed the exercise receive have a schedule time for walk through of the exercise.
Include IR exercise improvement ideas into planning for the next exercise.
Comments