2022_01_06_Instruction_107-1-1_After-ActionReviewImplementation_F.pdf

Instruction 107-1-1

Instruction 107-1-1: After-Action Review

Implementation

January 2022 Version 1.0

David Maurstad

Associate Administrator (A), Resilience

January 6, 2022

Table of Contents

OVERVIEW

OVERARCHING APPROACH

TIER CONSIDERATIONS

DEVIATION FROM FACTORS

REGIONAL AND HQ COORDINATION

AFTER-ACTION REVIEW DEVELOPMENT

AFTER-ACTION REVIEW TIER ACTIVITIES

ISSUE ELEVATION PROCESS

PUBLICATION

OTHER RESOURCES

ADDITIONAL INFORMATION ABOUT PUBLIC ASSISTANCE COST ESTIMATES

REQUESTING ADDITIONAL SUPPORT

RECOMMENDED MILESTONES FOR CONDUCTING AFTER-ACTION REVIEWS

CIIB PEER REVIEW QUESTIONS

RESPONSIBLE PROGRAM OFFICE DECISION DEFINITIONS

ADDITIONAL INFORMATION

REVIEW CYCLE

AUTHORITIES

REFERENCES

DEFINITIONS

ACRONYM LIST

MONITORING AND EVALUATION

QUESTIONS

Overview

Instruction 107-1-1 provides Continuous Improvement Program (CIP)1 personnel guidance, recommendations and baseline requirements for how to conduct after-action reviews2 for a Stafford Act Major Disaster Declaration (MDD) incident. This

Instruction expands on concepts and governance outlined in FEMA Directive

107-1: Continuous Improvement Program, which requires an after-action review following all MDDs.3 This Instruction provides a standardized approach across

FEMA CIP—including those at headquarters (HQ), in the regions, and in the field at all disaster locations—for determining the type of after-action review to conduct following a MDD, the maximum length of time each review and corresponding product should take to conduct and develop, and considerations for distribution of the final product. CIPs implementation of this Instruction ensures a consistent and common approach to future after-action reviews, greater assurance that corresponding products are produced in a timely manner for all types of MDD incidents, and that information is readily available from all regions. By accomplishing this, CIP can further assist the agency with learning from past events and enable responsible program offices (RPO) at the HQ and regional levels to take improvement actions.

Overarching Approach

Per FEMA Directive 107-1, Regional Continuous Improvement Coordinators

(RCIC), in support of their regional and field leadership, shall ensure after-action reviews are conducted for all MDDs. The FEMA CIP HQ component, the

Continuous Improvement Integration Branch (CIIB), is responsible for managing and updating doctrine, guidance, manuals, and related operating protocols for conducting after-action reviews and developing report or briefing products.4 This

Instruction documents additional guidance for conducting after-action reviews to

1 CIP, which is in the National Preparedness Directorate, includes staff in the Continuous Improvement Integration Branch (CIIB) based at FEMA Headquarters (HQ), Regional Continuous Improvement Coordinators (RCIC) based throughout the FEMA regional officers, and field staff, including Continuous Improvement Coordinators at Recovery Offices.

2 Per FEMA Directive 107-1, after-action reviews are defined as “targeted collections – led or supported by CIP – with defined time periods to identify strengths, potential best practices (i.e., innovations), and areas for improvement across the disaster lifecycle. After-action review products will vary in format and length. Interim and final regional products, produced periodically across the disaster lifecycle of major disaster declarations and major exercises, contain validated findings and recommendations from each review to be approved by field and regional leadership for dissemination within the Agency.”

3 The scope of Instruction 107-1-1: After-Action Review Implementation is limited to real-world MDD incidents and does not include exercises or other non-MDD events.

4 FEMA Directive 107-1: Continuous Improvement Program, Responsibilities, Page 7.

ensure a consistent approach across the program. This document, in laying out these responsibilities, also outlines how RPOs and the broader FEMA workforce support the process of developing after-action review products.

There are three tiers of action-action reviews outlined in this Instruction, which correspond with the severity and complexity of the incident assessed. Each tier includes considerations for determining which type of after-action review and associated product to complete and high-level requirements for completing the review, such as maximum timelines for finalizing products. In addition, this

Instruction outlines considerations for HQ CIIB involvement in after-action review efforts and product publication.

Tier Considerations

CIP has developed three tiers of after-action reviews, corresponding to the relative scale, cost, scope, and complexity of the MDD. The After-Action Review

Development section supplies an overview and requirements for each tier.

CIP identified the following factors as indicators of the MDD’s scale, scope, and complexity that should be considered in identifying the appropriate tier and corresponding product. The Regional Administrator (RA) responsible for the affected area determines the tier and product to develop, sometimes with advice from the HQ CIIB Chief and other HQ leadership with equities, if the incident meets the factors outlined in the Regional and HQ Coordination section. Following a MDD, the factors outlined below are intended as planning considerations to drive a scoping discussion and determine which tier and product is most appropriate for development.

1. A Tier III product will generally be developed for disasters when:

a. The MDD is for Public Assistance (PA) only

b. The disaster is managed by an entirely virtual Joint Field Office (JFO) or primarily virtual, with fewer than 20% of staff deployed to the field

c. The overall PA estimated costs should also be considered when determining the after-action review level of effort and resulting product5

2. A Tier II product will generally be developed for disasters when:

a. The MDD includes Individual Assistance (IA), or only covers IA

b. The MDD is issued for a Federally Recognized Tribal Government6

c. The incident affects multiple states

5 More details about PA Cost Estimates and how to determine the base amounts can be found in the Additional Information about PA Cost Estimates section.

6 Due to the infrequency of IA only or tribal declarations, it is especially significant to capture the lessons learned from these incidents.

d. The disaster type is not commonly experienced (e.g., pandemic, volcano, tsunami) in the affected jurisdiction or region

e. Deployment of a National Incident Management Assistance Team

(IMAT)

f. Deployment of a Regional IMAT for more than 30 days

g. The activation of the Regional Response Coordination Center (RRCC) in response to the incident

h. Mission Assignments for federal agencies are issued for the incident

i. The overall PA estimated costs should also be considered when determining the after-action review level of effort and resulting product

3. A Tier I product will generally be developed for disasters when:

a. The incident affects multiple FEMA regions

b. The disaster receives an expedited declaration

c. The disaster is considered catastrophic7

d. The disaster includes complex operational requirements, which may vary by region. Those complex requirements may include: a direct housing mission, state-managed operations, operations outside the continental United States (OCONUS), a debris management mission which may include private property debris removal or increased federal cost share for debris removal, new / pilot program implementation, or infrequently activated resources

e. The overall PA estimated costs should also be considered when determining the after-action review level of effort and resulting product

Figure 1, below, displays each tier and its factors for consideration.

7 The Post-Katrina Emergency Management Reform Act of 2006 defines the term “catastrophic incident” as “any natural disaster, act of terrorism, or other man-made disaster that results in extraordinary levels of casualties or damage or disruption severely affecting the population (including mass evacuations), infrastructure, environment, economy, national morale, or government functions in an area.”

Figure 1: After-Action Review Tier Considerations

These considerations should inform decision making. When factors from multiple tiers are met, it is recommended that the region pursue the tier and product that will enable the most learning possible for the field, region, and Agency based on the programs activated and/or complexity of the disaster. CIP should try to produce the higher-level tier and product when possible and where appropriate.

A region experiencing multiple similar disasters in nearby locations in a short period of time may decide to combine MDDs into a single product.

The Federal Preparedness Coordinator (FPC) will host a scoping meeting when a MDD has occurred to discuss the after-action review effort with relevant regional and field stakeholders, which may include the Office of the Regional

Administrator (ORA), Federal Coordinating Officer (FCO), Response and

Recovery Division Directors and associated leadership, Regional Integration

Branch Chief (RIBC), or equivalent, and RCIC, to discuss:

• The tier of after-action review

• The product to develop

• The targeted timeline for completion

• Specific disaster priorities

• Disaster specific data methodology (e.g., custom disaster specific survey and/or interview questions, response collection, deployment feasibility, etc.)

• Any other questions and/or concerns

A scoping form will be completed that captures the relevant and applicable MDD data and incorporates fields for the considerations above to inform the discussion. Within 10 business days of the MDD date, the region will determine the tier of review and corresponding product to include expedited declarations.

The region must notify HQ CIIB within 15 business days of the MDD date of the results of this decision by submitting the scoping form. At the regions request, HQ CIIB can participate in the scoping meeting or provide technical assistance on the implementation of this Instruction during a separate regional coordination call. Figure 2 depicts the overall process for determining the after-action review tier.

Figure 2: Tier Determination Process

HQ CIIB may assist with after-action reviews as requested by the region. For regions that wish to request support, as detailed in the CIP Deployment Standard

Operating Guide, RCICs should coordinate with the HQ CIIB Regional & Field

Integration (RFI) Section Chief (SC) if they require support beyond the capacity of the affected region.8 For details on this process, refer to the Requesting

Additional Support in the Other Resources section.

Given that FEMA Directive 107-1 requires after-action reviews after MDDs, and this Instruction is an extension of the directive, this Instruction does not include guidance on conducting after-action reviews for emergency declarations, exercises, and/or other non-MDD incidents, which is left to the discretion of the regions and HQ CIIB, as applicable. However, information presented within this

Instruction may be beneficial for use in those instances.

Deviation from Factors

The FPC will coordinate with the HQ CIIB Chief and National Preparedness

Assessments Division (NPAD) Director when there is significant deviation from the tier-based factors outlined above, including in instances specified below:

• A region plans to not conduct an after-action review or will not produce a product for an MDD

• If an after-action review product will not be finalized within the maximum allotted time limit as identified in this Instruction

The RA and Director of the Office of National Assessments & Integration (ONA&I) will decide the appropriate resolution if the parties outlined above cannot reach an agreement. Any other significant deviations that occur should be coordinated between the FPC and the HQ CIIB Chief.

Regional and HQ Coordination

Certain conditions may warrant more active HQ CIIB participation in regional after-action review efforts and/or tier determination. Examples of instances include but are not limited to:

• HQ CIIB may lead or participate significantly throughout the after-action review process in regular and close coordination with the affected region(s):

o For a multi-regional or national incident where a Tier I product is developed o For a unique, infrequent, or unprecedented incident or event for which there is limited historical data (e.g., pandemic, volcano, tsunami, non-Stafford Act incident with mid-to-large-scale FEMA engagement)

8 Continuous Improvement Program: Deployment Standard Operation Guide, Section 2.2:

Requesting Additional Support, Page 6.

o When directed by senior agency leadership to develop an agency-wide product

• HQ CIIB will consult with the affected region(s) to determine the after-action review product, provide data for product development, and/or review the product before finalization for:

o An incident or event where the National Response Coordination

Center (NRCC) is activated for longer than 30 days o An incident or event where more than one National Incident

Management Assistance Team (N-IMAT) is deployed due to either significant disaster severity or size

In the event that the above conditions arise, the HQ CIIB Chief will confer with the FPC and, where appropriate, the RIBC (or equivalent), the RCIC of the affected region, and the NPAD Director to identify the responsible party and appropriate course of action. If regional and HQ staff are collaborating on an after-action review, only one product per declared disaster is required, but more may be developed at leadership’s discretion.

After-Action Review Development

After-Action Review Tier Activities

The three tiers of after-action review activities are outlined below and include high-level requirements for the collection, analysis/writing, validation, and product review/finalization processes for each tier. These activities do not encompass all the products and services that CIP offers, but act as the baseline for determining what minimum activity to conduct following an MDD. The timelines specified represent the maximum amount of time allotted and adjustments may occur based on senior leadership direction.

Within each of the tiers, there is flexibility as to the type of product produced.

Catastrophic events or unique longer-term recovery missions may require flexible, dynamic reviews, and products. The most common products include:

• After-Action Report: a report developed after exercises and real-world incidents that summarizes key evaluation information and continuous improvement-related analytical findings, including strengths, potential best practices (e.g., innovations), and areas for improvement, as well as recommendations for potential courses of actions.

• Quick-Look Report: a time-sensitive document that may precede a full

After-Action Report and highlights major findings or high-priority issues from an after-action review that require attention. Generally, a QLR does not contain as detailed analysis as what would be found in an After-Action

Report. It often includes a limited set of priorities with supporting narrative, a brief overview of the methodology, and may include recommended courses of action.

• After-Action Review Briefing: a presentation to leadership on the disaster(s) that includes major takeaways for leadership through observations with supporting data. The briefing may include methodology, recommended courses of action, and other elements of the After-Action

Report and Quick-Look Report.

• National Collection Analysis Priorities (NCAP) Survey Summary: a report summarizing the data collected through the NCAP survey from a disaster, which includes an incident overview.

1. Tier III: After-Action Review

a. Collection Process:

i. CIP will distribute one survey9 70 to 85 days post-MDD to deployed personnel which includes NCAP10 questions and any questions the region desires to include based on regional and field priorities and identified in the scoping meeting. The survey will remain open for two weeks. No deployment or additional collection is required for this tier, but the Region as part of the scoping process may decide to do a review beyond the NCAP survey and include a deployment or other data collection as part of the process.

b. Analysis and Writing Process:

i. Survey results will be analyzed using qualitative and quantitative approaches. HQ CIIB will analyze NCAP survey data; the region, led by the RCIC, will analyze the data for questions they submitted and conduct appropriate qualitative and quantitative analysis for the additional data collection determined. While all survey results must be analyzed, only substantive trends that represent significant, and/or actionable results, and/or align with recurring trends observed across multiple disasters will be included in the summary, as determined by the lead developing the product.11

c. Product Review and Finalization Process:

9 The NCAP survey is sent to all staff who have been deployed to a MDD for five days or more.

HQ CIIB uses DTS to identify those staff members; if field leadership identifies additional personnel that they would like to receive the survey, their emails are added to the distribution list.

10 CIIB will lead an annual review of the HQ-level collection priorities to develop a set of NCAP in coordination with Agency leadership and program offices.

11 For anonymity purposes, a NCAP survey summary is developed only when a minimum of five responses have been received.

i. HQ CIIB will produce the NCAP Survey Summary within 45 days of the survey closing and provide it to the Region and the disaster FCO.

ii. The region, led by the RCIC, is responsible for providing a summarized analysis of data from region and field survey questions, if included in the survey, or any other data collected.12 The Region can determine what product type is desired for regional data collection during the scoping meeting, which may include a Quick-Look Report, observations within the NCAP Survey Summary, or a similar product. If a separate regional product is developed in addition to the NCAP

Survey Summary, the RCIC will facilitate the review and finalization process, which must be finalized no later than 180 days post-MDD.

iii. HQ CIIB will analyze and summarize aggregated NCAP survey data on a quarterly basis and brief the Associate Administrators and Regional

Administrators. These results will be shared with the regions, including

FPCs, Response and Recovery Division Directors, RIBCs (or equivalent), and RCICs.

2. Tier II: After-Action Review

a. Collection Process:

i. The lead CIP personnel will work with pertinent FEMA field, regional, and HQ program offices, as well as relevant Emergency and Recovery

Support function agencies and state, local, tribal, and territorial (SLTT) stakeholders (as appropriate), to collect information for analysis and elicit recommendations for improvement. If the MDD is declared within

30 days of the incident start date and is operationally feasible, a minimum one-week response focused field collection and deployment is required. This collection should happen as close as possible to the

MDD date, but no later than 45 days post-MDD. In addition to a potential response collection, all Tier II after-action reviews require a minimum one-week recovery deployment (recommended between 90 to 120 days post-MDD) if operationally feasible.13 Additional field visits may occur as needed to complete the product.

ii. Qualitative data collection: Conduct hotwashes, interviews, and document reviews as deemed relevant by the lead CIP personnel.

NCAP interview questions must be asked as applicable based on operational requirements to deployed personnel. Disaster documents

12 If the region conducts a separate regional and/or field-specific survey, they may choose to add a supplementary section to the Tier III product, at their discretion.

13 Deployments or contract support are coordinated with the region and disaster leadership and are paid through disaster funds.

like Situation Reports, Incident Action Plans, and Incident Support

Plans contain important information about operations and should also be reviewed for relevant data.

iii. Quantitative data collection: Review existing available and relevant quantitative data (e.g., data from incident reports, surveys, Deployment

Tracking System [DTS], WebEOC, PA Grants Manager, Recovery

Report and Analytics Division [RRAD] products, Common Operating

Picture [COP] Dashboard, and/or Analytics Hub).

iv. Survey data collection: CIP will distribute survey(s) to deployed personnel for all Tier II incidents which includes NCAP questions and any questions the region desires to include based on regional and field priorities and identified in the scoping meeting. HQ CIIB will distribute a recovery focused survey 70 to 85 days post-MDD; however, if the MDD is declared within 30 days of the incident start date, an initial response focused survey will be distributed 20 to 40 days post-MDD, in addition to the recovery survey distributed 70 to 85 days post-MDD survey.14

b. Analysis and Writing Process:

i. In developing observations, CIP uses multiple sources of data and/or collection methods to ensure that observations extend beyond opinion.15 All Tier II products include qualitative analysis and limited quantitative analysis. The lead CIP personnel should use existing quantitative data (such as the examples listed above in the quantitative data collection section) as available and relevant to support observations.16

ii. Survey analysis: HQ CIIB will analyze aggregated NCAP survey data on a quarterly basis; the region, led by the RCIC, is responsible for analyzing the data for questions they submitted. While all survey results must be analyzed, the lead CIP personnel can incorporate survey data into the tier product(s) as they deem applicable. It is encouraged to incorporate substantive survey trends that represent impactful, and/or actionable results, and/or align with recurring trends observed across multiple disasters.

c. Validation Process:

14 In instances where an incident receives two surveys, HQ CIIB will endeavor to distribute the surveys as far apart as possible within the stated timeframes and review the deployed personnel to minimize staff receiving both surveys to decrease duplicative collection efforts and ensure focus areas (i.e., response and recovery) are captured.

15 HQ CIIB is crafting additional but separate guidance for data collection methods which will be released at a future date.

16 CIP personnel should follow the guidance provided in the CIP Observation Development Guidelines for additional information on how to write an observation.

i. The lead CIP personnel will work with pertinent FEMA field, regional, and HQ program offices, Emergency and Recovery Support function agencies, and SLTT stakeholders (as appropriate) involved in response and recovery operations to validate and review findings for accuracy and develop and/or corroborate recommendations from participants.17 All observations should be reviewed with the relevant entities and individuals for validation.

ii. As stated in FEMA Directive 107-1, CIP will work, when applicable, with RPOs to ensure that observations reflect both field-level, regional, and HQ programmatic understanding, and that conclusions are drawn with as comprehensive perspective as possible.18 For observations whose recommendations are expected to exceed the region’s scope and require HQ action, HQ CIIB will support coordination of subject-matter expert (SME) consultation with HQ program areas. RCICs will request HQ program validation and provide the latest draft of the observation to HQ CIIB for program review. HQ CIIB Program Liaisons may coordinate a meeting between the RCIC, HQ program area, and when applicable, deployed SMEs and regional representatives to discuss and validate the draft observation(s). If a program point of contact cannot be identified, a validation request will be sent to the HQ

Program Action Office for adjudication.

iii. The RA, or their designee, has final approval authority over the validation of observations within their region, consistent with FEMA

Directive 107-1.

d. Product Review and Finalization Process:

i. CIP will develop and finalize an after-action review product, no later than 240 days post-MDD, that highlights observations from the disaster. Typically, a product in this tier will include fewer priorities assessed and will include fewer observations than a Tier I product. The

Region can determine what product type is desired during the scoping meeting, which may include a Quick-Look Report or similar product.

ii. RCICs facilitate the review and finalization process with regional leadership. Regional Continuous Improvement Working Groups

(CIWG) review observations and identify if they are the responsibility of the field, region, or HQ to address. Refer to the Issue Elevation

Process section for additional information about the elevation process.

3. Tier I: After-Action Review

17 FEMA Directive 107-1: Continuous Improvement Program, Requirements: Validation, Page 4.

18 FEMA Directive 107-1: Continuous Improvement Program, Requirements: Validation, Page 4.

a. Collection Process:

i. The lead CIP personnel will work with pertinent FEMA field, regional, and HQ program offices, as well as relevant Emergency and Recovery

Support function agencies and SLTT stakeholders (as appropriate), to collect information for analysis and elicit recommendations for improvement. If the MDD is declared within 30 days of the incident start date, a response focused field collection and deployment of at least one week is required if operationally feasible. This collection should happen as close as possible to the MDD date, but no later than 45 days post-MDD. In addition to a response collection, all Tier I after-action reviews require a one-week minimum recovery deployment

(recommended between 121 to 165 days post-MDD) if operationally feasible. Additional field visits may occur as needed to complete the product.

ii. Qualitative Data collection: Conduct interviews with relevant staff (at minimum Command and General staff and Branch Directors), hotwashes, document review, and other research methods as deemed necessary by the lead CIP personnel. NCAP interview questions must be asked as applicable based on operational requirements to deployed personnel. Disaster documents like Situation Reports, Incident Action

Plans, and Incident Support Plans contain important information about operations and should also be reviewed for relevant data.

iii. Quantitative Data collection: Perform incident/collection specific data acquisition, cleaning, and analysis. Review existing available and relevant quantitative data that can validate observations (e.g., data from incident reports, surveys, DTS, WebEOC, PA Grants Manager, RRAD products, COP Dashboard, and/or Analytics Hub).

iv. Tier I and Tier II have the same survey process; refer to Tier II

Collection, Analysis and Writing, and Review and Finalization sections for details.

b. Analysis and Writing Process:

i. In developing observations, CIP uses more than one data collection method to ensure that observations extend beyond opinion. For all Tier

I observations, thorough analysis is applied using qualitative and quantitative approaches. All observations should be supported by quantitative data.

c. Validation Process:

i. Tier I and Tier II have the same validation process; refer to Tier II

Validation section for details.

d. Product Review and Finalization Process:

i. CIP will develop and finalize an after-action review product, usually an

After-Action Report for Tier I, no later than 300 days post-MDD, that details key evaluation information and continuous improvement-related analytical findings including observations and recommendations.

ii. Regions may choose to do a Quick-Look Report on specific operational priorities requiring faster feedback or to determine if a broader review and full After-Action Report may be required.

iii. RCICs facilitate the review and finalization process with regional leadership. Regional CIWGs review observations and identify if they are the responsibility of the field, region, or HQ to address. Refer to the

Issue Elevation Process section for additional information about the elevation process.

Table 1, below, is a matrix that crosswalks activities related to each tier. Refer to this table for a side-by-side tier comparison, along with the Recommended

Milestones in the Other Resources section for a recommended timeframe approach for conducting each process within all tiers, along with a supporting graphic.

Table 1: After-Action Review Product Requirements

Categories Tier III Tier II Tier I

Survey Scope

NCAP, with additional regional and field questions as desired.

NCAP, with additional regional and field questions as desired.

NCAP, with additional regional and field questions as desired.

Survey Timeline

One survey, distributed 70 to 85 days post-MDD.

Recovery survey distributed 70 to 85 days post-MDD.

If the MDD is declared within 30 days of the incident start date, a response survey will be distributed 20 to 40 days post-MDD, in addition to the recovery survey.

Recovery survey distributed 70 to 85 days post-MDD.

If the MDD is declared within 30 days of the incident start date, a response survey will be distributed 20 to 40 days post-MDD, in addition to the recovery survey.

Disaster Collection Analysis Plan (DCAP)

Not required unless additional collection takes place.

NCAP, regional, and field leadership priorities.

NCAP, regional, and field leadership priorities.

Deployment Not required. Requires a minimum one-week recovery deployment, as operationally feasible.

Additionally, if the MDD is declared within 30 days of the incident start date, a one-week response focused deployment is

Requires a minimum one-week recovery deployment, as operationally feasible.

Additionally, if the MDD is declared within 30 days of the incident start date, a minimum one-week response focused required within the first 45 days of the MDD in addition to the recovery deployment.

deployment is required within the first 45 days of the MDD in addition to the recovery deployment.

Qualitative Data Collection

Not required. Hotwashes, interviews and document review as deemed relevant by lead.

NCAP interview questions must be asked as applicable based on operational requirements to deployed personnel.

Review disaster documents relevant to collection priorities and/or observations being developed.

Interviews with relevant staff (at minimum Command and General staff and Branch Directors), hotwashes, document review, and other research methods as deemed necessary by the lead.

NCAP interview questions must be asked as applicable based on operational requirements to deployed personnel.

Review disaster documents relevant to collection priorities and/or observations being developed.

Quantitative Data Collection

Not required. Review existing available and relevant quantitative data (e.g., data from incident reports, surveys, DTS, WebEOC, PA

Grants Manager, RRAD products, COP

Dashboard, and/or

Analytics Hub).

Perform incident/ collection-specific data acquisition, cleaning, and analysis. Review existing available and relevant quantitative data (e.g., data from incident reports, surveys, DTS, WebEOC, PA Grants Manager, RRAD products, COP Dashboard, and/or Analytics Hub).

Analysis Limited analysis based on survey data.

Qualitative analysis and limited quantitative analysis; lead CIP personnel should use existing quantitative data as available and relevant to support the observations and ensure they extend beyond one’s opinion.

Thorough analysis applied using qualitative and quantitative approaches. All observations should be supported by quantitative data and extend beyond one’s opinion.

Responsible Party

HQ CIIB will provide a summarized analysis of NCAP survey data. The region is responsible for providing a summarized

Most commonly a regionally led effort, but HQ CIIB may participate in the review when certain factors are

Most commonly a regionally led effort, but

HQ CIIB may lead or participate in review efforts for a multi-regional analysis of data from region and field survey questions, if included in the survey, or any other data collected.

triggered, such as when the NRCC is activated for longer than 30 days and/or more than one N- IMAT is deployed.

or national incident, and/or a unique, infrequent, or unprecedented incident or event for which there is limited historical data, or when directed by senior agency leadership to develop an agency-wide product.

Product Details

An incident-specific summary of the survey results which includes an incident overview and a brief analysis of the survey data.

An after-action review product that highlights observations from the disaster. Typically, this product assesses fewer priorities and includes fewer observations than a Tier I product.

An after-action review product, usually an After- Action Report, that details key evaluation information and continuous improvement-related analytical findings including observations and recommendations.

Product Timeline

(Not to Exceed)

Completed within 180 days (6 months) post-

MDD.

Completed within 240 days (8 months) post-

MDD.

Completed within 300 days (10 months) post-

MDD.

Per FEMA Directive 107-1, HQ CIIB tracks the status of all after-action reviews and resulting products for MDDs.19 RCICs inform regional leadership and HQ CIIB of the status of each MDD after-action review.20 Status updates should be provided at minimum on a monthly basis via the CIP After-Action Review Product Tracker and should include the major phase in which each MDD after-action review currently stands: New MDD (default), Collection Scheduled, Collection in Progress, Writing/Analysis/Validation, Review/Concurrence, Approved/Final, or On-hold.

Issue Elevation Process

For findings that exceed the region’s scope and that require HQ action, RCICs will elevate CIWG designated observations to HQ CIIB.21 RCICs are encouraged to submit validated observations to HQ CIIB at the earliest possible opportunity to increase relevancy; regions may elevate observations as soon as the CIWG approves it, even if that is before the entire after-action review product (and other observations within the product) are finalized.

19 FEMA Directive 107-1: Continuous Improvement Program, Responsibilities, Page 8.

20 FEMA Directive 107-1: Continuous Improvement Program, Responsibilities, Page 9.

21 FEMA Directive 107-1: Continuous Improvement Program, Requirements: Validation, Page 5.

For observations elevated to HQ CIIB, and/or at the request of a region, HQ CIIB conducts a peer review which considers whether an observation has an appropriate depth of validation or analytical rigor to warrant programmatic action;

the peer review question list can be found in the Other Resources section. If the peer review indicates a need for revision, HQ CIIB will provide suggested edits to the region for consideration. This peer review does not mandate revisions but serves as a quality control step in the process to ensure observations include root cause analysis and information about operational consequences that enables RPOs to take appropriate action.

Observations that do not meet HQ CIIB quality control requirements after regional revisions, but are likely to have RPO interest (e.g., pre-identified RPO priorities or observations that may rarely occur but have anecdotal merit), will be elevated for situational awareness, rather than for action, and maybe considered in future trend analyses. Ultimately, the region (or originator) of an observation is responsible and owns this content; others (HQ CIIB or RPOs) may request revisions of the region but cannot make revisions independently. Similarly, RPOs are responsible for project content and may only delegate this to Supporting

Program Offices.

Through the Issue Resolution System,22 HQ CIIB elevates observations to RPOs and documents all major decisions made by the RPO. RPOs can select from:

accept for action, accept for situational awareness, request revision, hold, previously completed, decline, or incorrect program office identified.23 RPOs are not mandated to act on observations elevated by CIP, but CIP does request that

RPOs supply rationale if they choose not to act. HQ CIIB will provide regions with quarterly reports and transparent data access on elevated observation status, RPO project progress, and RPO communications.

Publication

Per FEMA Directive 107-1, regions share resulting products and observations with NPAD to support cross-cutting trend analyses. HQ CIIB tracks the status of all after-action reviews and resulting products for MDDs. Within 15 days of finalization, after-action review products are to be shared with HQ CIIB via the

CIP After-Action Review Product Tracker. Validated observations and after-action review products are posted by HQ CIIB to a searchable observation

22 HQ CIIB is crafting additional but separate guidance for the Issue Resolution System which will be released at a future date.

23 A definition for each RPO decision status can be found in the Other Resources section.

dataset, such as the Comprehensive After-Action Report Analysis Tool

(CAARAT). At the regional level, after-action review products should be shared with relevant field and regional program offices at regional leadership’s discretion, but at minimum shared with the relevant disaster FCO, regional

Division Directors, and ORA.

Generally, due to the scope and nature of after-action review products, they are intended for internal FEMA use and are often not applicable to external audiences. However, to contribute to a larger body of knowledge and foster lessons learned sharing within the emergency management community, after-action review products should be made available to jurisdictions and other federal agencies that request them or as directed by the RA, where appropriate.

This decision falls with regional leadership. If a jurisdiction involved in the disaster and as part of the after-action review process requests the after-action review product and there are relevant observations for the jurisdiction, effort should be made to ensure that at least the relevant observations are shared. This may include sharing the entire product, summarized findings of relevancy and trends done for each product or annually, presentations for relevant stakeholders, or aggregate documents. General public release of after-action review products will only happen for reports of national significance as deemed by agency leadership (e.g., COVID-19 or 2017 Hurricane Season After-Action

Reports).

If CIP personnel intend to share an after-action review product with external partners, it is recommended to use good judgement on submitting the document through Triad, at either the HQ or regional level, if the product contains significant legal or policy issues. Triad includes the Office of Chief Counsel, the Office of Policy and Program Analysis, and the Office of External Affairs.

Other Resources

Additional Information about Public Assistance Cost Estimates

To determine the after-action review tier, CIP has identified multiple factors as indicators of the MDD’s scale, cost, scope, and complexity. PA costs estimates is one element to consider, which are available as a data point in a timely manner for most incidents and account for different state and territory capacities by establishing a per capita standard which attempts to equalize across rural and urban states and territories. To receive a Presidential disaster declaration for PA, states and territories must have damage estimates that reach a statewide per capita indicator; FEMA evaluates the estimated cost of Federal and non-Federal public assistance against the population to give some measure of the per capita impact. FEMA uses a per capita amount as an indicator that the disaster is of such severity and magnitude that it might warrant Federal assistance, and adjusts this figure annually, based on the Consumer Price Index. Federally https://analytics.fema.net/#/site/AnalyticsHub/views/CAARATDashboard_15762670116090/Welcome?:iid=1 recognized tribes that are requesting their own Presidential disaster declaration do not need to meet a per capita indicator. However, absent extraordinary circumstances, they must reach a minimum damage amount of $250,000.24

Given that each state/territory has varying thresholds at which MDDs are made based on PA cost estimates using this per capita indicator, CIP includes this as one indicator to be considered in determining the appropriate tier of review and product. CIP will measure the percentage above the per capita baseline to determine a “base amount” for each incident. Base amounts for determining the tier for product development are calculated by multiplying the state’s decennial population by the annual public assistance per capita indicator.25 CIP personnel can refer to FEMA state-wide PA per capita impact indicators26 and decennial census data27 to obtain this information. However, at the beginning of every year, HQ CIIB will provide the calculated base amounts to all CIP personnel.

Requesting Additional Support

As stated in the CIP Deployment Standard Operating Guide, pages 6 – 11, RCICs and HQ CIIB staff frequently deploy to support each other and integrate into disaster collection teams. Staff requirements are determined by the affected region and designated FCO. RCICs should coordinate with the RFI SC if they require support beyond the capacity of the affected region. RCICs should begin assessing staffing needs during the onset of an incident and communicate any projected gaps to the RFI SC early, allowing adequate time for identifying available resources. For new RCICs, or those who may already be committed to other responsibilities, support from another region or the RFI SC may be requested to assist with mission scoping.

In alignment with FEMA deployment processes, requests for staff resources are initially sourced within the affected region. In the event of staffing deficiencies within the affected region, a standard deployment request for a specific FEMA

Qualification System (FQS) position or a non-standard name request is submitted through the appropriate regional Staffing Point of Contact (SPOC). The

RFI SC serves the role of a traditional FQS Cadre Coordinator in supporting the

24 FEMA Preliminary Damage Assessment Guide, Estimated Cost of Assistance, Page 57

25 Disasters where estimated damage is less than $1 million do not qualify for an MDD. For states and territories where the base amount is below $1 million, their base amount has been updated to reflect the minimum of $1 million. For these states and territories, the 150% and 350% above base amount consideration will reflect either the decennial population multiplied by the PA per capita indicator or $1 million, whichever is higher.

26 Source: FEMA Disaster & Assistance, Per Capita Impact Indicator and Project Thresholds | FEMA.gov

27 Source: United States Census Bureau, Decennial Census by Decades https://usfema.sharepoint.com/teams/RESNPD/NPAD/open/CIP/Documents/Forms/AllItems.aspx?id=%2Fteams%2FRESNPD%2FNPAD%2Fopen%2FCIP%2FDocuments%2FDisaster%20Deployment%20Resources%2FDeployment%20Standard%20Operating%20Guide%20V%2E11%2020211018%2Epdf&parent=%2Fteams%2FRESNPD%2FNPAD%2Fopen%2FCIP%2FDocuments%2FDisaster%20Deployment%20Resources https://www.fema.gov/sites/default/files/2020-07/fema_preliminary-disaster-assessment_guide.pdf https://www.fema.gov/assistance/public/applicants/per-capita-impact-indicator https://www.fema.gov/assistance/public/applicants/per-capita-impact-indicator https://www.census.gov/programs-surveys/decennial-census/decade.html adjudication of resource requests and coordinating staff for deployment. If any deployment actions are initiated at the HQ level, the RFI SC coordinates approval with either the region or disaster operation. Figure 3 depicts the process for regionally initiated requests.

Figure 3: Process for Regions to Initiate Staff Requests

Regionally initiated staffing requests should include the following steps:

1. The RCIC consults with regional Incident Management (IM) leadership to assess regional CIP resource availability and notifies the RFI SC whether regional assets are available.

2. The RCIC consults with their FPC/regional leadership, FCO, and relevant disaster leadership to determine the scope of the request. The following essential elements should be included:

a. Locations of deployment, including JFO and other field sites

b. Deployment duration and specific dates, if available

c. Collection priorities

d. Estimated number of interviews

e. Estimated staffing requirement

f. Anticipated need for additional deployments

3. The RCIC submits a request, including all essential elements of information listed above, to the RFI SC.

4. The RFI SC reviews the request, and in collaboration with the affected region, the RCIC determines the ideal team composition. The ideal team composition is compared to the list of available staff.

a. If the available staff cannot fill the ideal team composition, the RFI SC will contact staff previously listed as unavailable to see if there is any flexibility

b. The RFI SC will reach out to FPCs of non-affected regions, HQ components, and cadres to solicit staff to fill the requirement as needed

c. When there are concurrent incidents or large disasters spanning several regions, the RFI SC will coordinate a conference call with affected regions to adjudicate the staffing requirements and deployments

5. The RFI SC will finalize the staffing plan with the RCIC and request the deployment of the team via supervisors as needed.

a. The RCIC must obtain the approval of the disaster point of contact and the

FPC before staff are deployed

6. Once the team composition and deployment dates are set, the RFI SC will email the information, along with a copy of the collection analysis plan (CAP) with any known follow-up deployments, to RCICs and HQ CIIB.

Recommended Milestones for Conducting After-Action Reviews

A Tier I product is expected to be finalized no later than 300 days post-MDD, 240 days post-MDD for Tier II, and 180 days post-MDD for Tier III. Below are recommended milestones for completing each process listed within all tiers;

Figure 4 is a graphic that depicts these recommended dates. The region has discretion to adjust the timelines to fit their needs, so long as the product is completed within the finalization dates listed above.

Tier III - After-Action Review:

• Collection Milestones:

o The survey will be distributed 70 to 85 days post-MDD and remain open for two weeks, once issued.

• Analysis and Writing Milestones:

o Once the survey closes, HQ CIIB will clean the NCAP data and send it to CIP personnel responsible for crafting the product within 105 days post-MDD. The analysis and writing process should happen immediately following this and take approximately two to three weeks to complete. This process should occur between 105 to 135 days post-

MDD.

• Product Review and Finalization Milestones:

o The review process for the Tier III product should begin immediately following the analysis and writing phase and take approximately three weeks to complete. This process should occur between 136 to 180 days post-MDD.

o The product must be finalized within 180 days post-MDD.

Tier II - After-Action Review:

• Collection Milestones:

o If the MDD is declared within 30 days of the incident start date, a one-week response focused field collection and deployment is required, if operationally feasible, and should happen as close as possible to the

MDD date, but no later than 45 days post-MDD.

o In addition to a potential response collection, all Tier II after-action reviews require at least one one-week recovery deployment, if operationally feasible, which is recommended to occur between 90 to

120 days post-MDD.

• Analysis and Writing Milestones:

o The analysis and writing should happen following the termination of each deployment (response and/or recovery) and preferably completed within one month after the end of the deployment.

• Validation Milestones:

o The validation phase should begin immediately following the analysis and writing phase. This process should take about three weeks to complete and should occur between 151 to 195 days post-MDD.

• Product Review and Finalization Milestones:

o The review and concurrence process should begin immediately following the analysis phase. This process should take approximately one month to complete should occur between 196 to 240 days post-

MDD.

o The product must be finalized within 240 days (8 months) post-MDD.

Tier I - After-Action Review:

• Collection Milestones:

o If the MDD is declared within 30 days of the incident start date, a one-to-two-week response focused field collection and deployment is required, if operationally feasible, and should happen as close as possible to the MDD date, but no later than 45 days post-MDD.

o In addition to a response collection, all Tier I after-action reviews require a minimum one-week recovery deployment, if operationally feasible, which is recommended to occur between 120 to 165 days post-MDD.

• Analysis and Writing Milestones:

o The analysis and writing should happen following the termination of each deployment (response and/or recovery) and preferably completed within one month after the end of the deployment.

• Validation Milestones:

o The validation phase should begin immediately following the analysis and writing phase. This process should take about three weeks to complete and should occur between 211 to 255 days post-MDD.

• Product Review and Finalization Milestones:

o The review and concurrence process should begin immediately following the analysis phase. This process should take approximately one month to complete should occur between 256 to 300 days post-

MDD.

o The product must be finalized within 300 days post-MDD.

Figure 4: After-Action Review Recommended Milestones

CIIB Peer Review Questions

Table 2 depicts…