Nassau County TTX -Decision-Making Analysis (November 2015).pdf

Nassau County TTX - Decision-Making Analysis

AlphaTRAC Point of Contact Ms. Heather M. Hodgin •720-263-4464 • hhodgin@alphatrac.com

Nassau County TTX - Decision-Making Analysis

Project 755901 Page ii

10385 Westmoor Drive, Suite 310

Westminster, CO 80021-2597 www.alphatrac.com

November 6, 2015

Nassau County TTX - Decision-Making Analysis

Project 755901 Page iii

Contents

A SYSTEM

7.2 RECOMMENDATION 2: GAIN RAPID SITUATIONAL UNDERSTANDING AND SHARE IT

THROUGHOUT THE RESPONSE STRUCTURE

7.3 RECOMMENDATION 3: KNOW WHAT FORMAL PROTOCOLS EXIST AND HOW THEY WILL

BE TRIGGERED

7.4 RECOMMENDATION 4: CONDUCT DECISION SPACE ANALYSES TO IMPROVE CRITICAL

DECISION INFORMATION FLOW

7.5 RECOMMENDATION 5: DEVELOP WORK AIDS FOR KEY DECISIONS AND ACTIONS

8 RECOMMENDATIONS FROM PRIOR ANALYSES

Nassau County TTX - Decision-Making Analysis

Project 755901 Page iv

9 RECOMMENDATIONS FOR THE CHEMICAL DEFENSE DEMONSTRATION

PROJECT

MAKING TO OTHER VENUES

CONSEQUENCE EVENTS

Nassau County TTX - Decision-Making Analysis

Project 755901 Page 5

1 INTRODUCTION

On October 21, 2015, AlphaTRAC, Inc. (AlphaTRAC) observed a modified tabletop exercise (TTX) conducted in Nassau County, NY. The exercise, focused on multi-agency response to a mass casualty incident resulting from a chemical release in a public venue, was co-sponsored by the Department of Homeland Security (DHS) Chemical Defense Program (CDP) and the Nassau County Fire Marshal (NCFM). The exercise was conducted as a component of the Chemical Defense Demonstration Project (CDDP). AlphaTRAC’s role was to observe and evaluate decision-making processes employed during the exercise.

This report summarizes key observations, analyses and recommendations as a result of the exercise.

This document is organized into the following major sections:

• Introduction

• The Nassau County Tabletop Exercise

• Prior Examination of Decision-Making in Nassau County

• Approach

• Teams Observed and Example Decisions

• Themes

• Recommendations

• References

• Abbreviations and Acronyms

2 THE NASSAU COUNTY TABLETOP EXERCISE

The Nassau County TTX was conducted at the Morrelly Homeland Security Center in Bethpage, NY. Approximately 100 representatives from agencies, offices, and associations, within local, regional, state and federal levels of government, academia and the private sector participated in the exercise.

Three primary goals were identified for the TTX:

• Enhance community-oriented response for large-scale chemical incidents

• Validate the critical decisions and tasks required during the immediate response to a chemical incident

Nassau County TTX - Decision-Making Analysis

Project 755901 Page 6

• Inform the development of a chemical incident CONOPS to include agency authorities, roles and responsibilities, agency coordination and information sharing

Four objectives were established in order to meet these goals:

• Examine multi-agency coordination and information flow between unified command and agency leadership, between responders and medical centers and between medical centers

• Examine how agencies make decisions with uncertain and/or conflicting information and the processes they use to overcome this

• Identify the requirements and existing capacity and capability to provide lifesaving medical treatment at the incident site and at area hospitals

• Test the ability of stakeholders to quickly develop and disseminate protective action decisions and other information and warnings to the public

The scenario for the TTX involved a hypothetical organophosphate (OP) pesticide intentionally released at an entry point at the Nassau Veterans Memorial Coliseum. The simulated attack occurred as 500 people gathered at the entry point prior to security screening. The hypothetical release was accomplished with a pyrotechnic dissemination device.

3 PRIOR EXAMINATION OF DECISION-MAKING IN NASSAU COUNTY

AlphaTRAC examined decision-making in Nassau County Emergency Management during July 2015. The examination was conducted through both in-person and remote interviews with Nassau County first responders and medical first receivers. The interviews employed a Critical Decision-making approach in which interviewees discussed their decision processes in context of actual responses they had experienced or observed.

AlphaTRAC’s analysis of the interviews resulted in six findings:

• Naturalistic decision-making is the primary method of emergency decision-making in Nassau County

• The Recognition-primed decision model is a good descriptor of the decision-making process

• Judgment-based decision-making is highly collaborative

• The first few decisions are critical – and the KISS principle applies

• Experience teaches decision makers to act with consideration

• Judgment- and experience-based decision-making will become more difficult as the community experience base diminishes through retirement

Nassau County TTX - Decision-Making Analysis

Project 755901 Page 7

4 APPROACH

AlphaTRAC’s activities in the Nassau County TTX supported the following exercise goal and objective:

• Exercise Goal: Validate the critical decisions and tasks required during the immediate response to a chemical incident

• Exercise Objective: Examine how agencies make decisions with uncertain and/or conflicting information and the processes they use to overcome this

An additional objective for our effort was to further evaluate and confirm or reject the findings from the interview-based analysis.

Two decision psychologists from AlphaTRAC conducted observations during the exercise. To maximize the breadth of observations we use a “roving observer” approach, in which we moved frequently among teams and participants. We observed multiple simultaneous decision processes in progress and usually did not follow any single process from beginning to end.

The exercise was conducted in two facilities within the Morrelly Center:

• A large open meeting room

• The Emergency Operations Center

We conducted our observations in the meeting room.

Tables were distributed around the meeting room to simulate team locations during the multi-agency response. A waist-high divider separated the space used for onsite teams (e.g., fire, law enforcement, onsite medical, incident command) and that dedicated to offsite teams (e.g., hospitals, Medical Control). We focused our observations and analyses on the onsite teams and their decision-making.

5 TEAMS OBSERVED AND EXAMPLE DECISIONS

AlphaTRAC observed decision processes conducted at four onsite response locations (tables) during the exercise:

• Unified Command (Incident Command)

• Police

• Fire

• Fire Marshal HAZMAT

Examples of decisions observed at each location are summarized in Table 1. Note that the decision processes observed were not complete and potentially not final. In some cases, later (unobserved) discussions may have led to modified decisions.

Nassau County TTX - Decision-Making Analysis

Project 755901 Page 8

Table 1. Example decisions observed during the Nassau County TTX

Table/Team Decision Question Decision Observed

Unified Command

What should be the extent of public evacuation?

Use the Emergency Response Guide to establish evacuation zone.

What approach should be used for decontamination?

Use water spray from fire trucks to conduct gross decontamination. HAZMAT Team should be in charge. Set up a bleach solution operation in a trailer.

How should victims be managed for decontamination?

Remove victims from hot zone, decontaminate them, load on busses for transport to hospital

What factors should be considered in setting up decontamination?

Determine wind direction for plume flow;

locate fire hydrants for water supply.

Police What resources are required immediately (initial response)?

Ambulances are required. Request immediately.

How should the threat be contained?

Shut down the east side of the building and parking lot.

Where should ambulances be staged?

Stage ambulances to the southeast side of the parking lot.

Fire How severe an event is this?

Event is a Mass Casualty Incident. Send the Mass Casualty Incident unit to the scene.

What level of Personal Protective Equipment (PPE) should be required for operation in the hot zone?

Full PPE should be required.

How should victims be transported to the hospital?

Transport the victims using busses.

Nassau County TTX - Decision-Making Analysis

Project 755901 Page 9

How should effective communication and collaboration among teams be ensured?

Establish a Unified Command Post at the south entrance to the Coliseum.

Fire Marshal

HAZMAT

What is the best way to gain entry to the building?

Find a building manager or other personnel who is familiar with the building and use their advice.

What type of decontamination should be implemented?

Fast (mass) decontamination should be conducted – using water only.

How far out should public evacuations be conducted?

Use recommendation from DOT Emergency Response Guide for Organophosphate.

How should victims be treated after decontamination at scene?

Inform hospital that multiple decontaminated casualties will be arriving from the scene.

6 THEMES

In its analysis of observations made during the Nassau County TTX, AlphaTRAC identified five main decision-making themes:

• Judgment-based naturalistic decision-making was dominant

• Situational understanding was an early focus

• Situational understanding may not have triggered protocols

• Recognition-Primed Decision Model applied

• Flow of critical decision information was delayed

6.1 Theme 1: Judgment-based Naturalistic Decision-making Was Dominant

6.1.1 OBSERVATIONS

We observed a consistent behavior among decision makers across all teams during the exercise. The important decisions were made using expert judgment, based on experience and prior knowledge. Checklists, protocols and logic-based methods were used to implement the decisions, but almost never to make the decisions. There was

Nassau County TTX - Decision-Making Analysis

Project 755901 Page 10 usually discussion among teams and decision makers, but the greater experience usually trumped.

We observed responders thinking of a similar situation (or multiple related situations) and remembering how events unfolded and what actions were taken in that situation.

Even when the past situation was only somewhat similar, pieces of information were used, and then modified to better mold to the exercise scenario. Event recognition led to situational understanding and then to a workable action plan.

Illustrative comments heard during the exercise and hot wash included:

“Again, based on my experience, I just don’t have the personnel to do it.”

“How did you know what to do?” “Lived it.” “Common sense, experience. We’ve seen it.”

“Sure, I could… but in reality I would talk to onscene HAZMAT and get their guidance.”

“If you have any experience at all, you will know… It’s simple… In reality, it wouldn’t transpire that way.”

“Just so you all know what will happen… We had an incident at Garden City Hotel. People running in all directions.”

“Look, in my experience, there is no way to contain the people. The mother and the father are gonna take their kids and run. I do not have the resources to stop them.”

“The ERG is the universal unknown guide.” “Are we gonna really know if we are exactly 300 yards out? No. Common sense after doing this for so many years tells me where we are gonna go.”

“I have 18 years of experience. Now we know people are gonna be self-referring. Tell the hospitals to expect certain signs and symptoms. You’ve got to EXPAND your scope based on the seriousness. It may not be protocol, you have to make judgment calls. We have something bigger than just the Coliseum.

I know from experience, and I think the Chief will agree with me. You’re not gonna be able to stop people leaving. You do the best you can and stop first responders rushing in. That’s all we can control.”

6.1.2 ANALYS IS

We found that the judgment-based decision-making observed during the exercise was consistent with a decision approach known as Naturalistic Decision-Making (NDM) (Lipshitz, et. Al, 2001, Todd and Gigerenzer, 2001).

NDM focuses on people who use their experience to make decisions in real-world contexts, and aims to examine how successful decision makers size up the decision problem and make reasonable decisions that are compatible with the situation.

According to this view, experienced decision makers can skillfully use their experience

Nassau County TTX - Decision-Making Analysis

Project 755901 Page 11 and prior knowledge to assess a situation and appraise decision options — a process sometimes colloquially referred to as “intuition” — and quickly settle on a course of action. NDM has been found to apply particularly to complex decision-making under conditions of uncertainty, time pressure, and stress (Klein, 1998).

6.2 Theme 2: Situational Understanding Was an Early Focus

6.2.1 OBSERVATIONS

During the exercise we observed an early struggle to gain situational understanding.

Individual teams worked to gather and make sense out of the available information.

They worked to sift key information out of the data flow and use it to make sense of the situation.

Illustrative comments heard during the exercise and hot wash included:

“This is [Medical Control] (Medcon). What do you have on symptoms?”

“Here’s all I know. Nerve agent, mass fatalities, someone asked for a Chempack on the radio.”

“But I’d want to know: Is there a terrorist/or perp in custody? Could there be a secondary device?”

“Throw up? People sick all over the parking lot? Then gear on and it’s chemical.”

6.2.2 ANALYS IS

Gaining situational understanding is a common – and appropriate – early focus during emergencies. Penetrating the “fog of war” allows decision makers to understand the event, choose effective courses of action and acquire/apply appropriate resources.

Situational understanding resolves to determining an incident’s type, scale, and scope.

• Type refers to a description of the threat/emergency (e.g., chemical, active shooter, structure fire)

• Scale refers to the severity and extent of the threat/emergency (e.g., amount, number of shooters, # of rooms/floors involved), and

• Scope refers to the setting and conditions (e.g., weather, location, exposures).

In the Nassau County exercise, each organization had its own priorities when assessing the event’s type/scale/scope, based on their missions during the emergency. For example, in the first few minutes the police were focused on whether the threat type was chemical and whether it was accidental or malevolent intent; HAZMAT teams focused on identifying the specific chemical; and medical teams were more interested in whether the event would require medical intervention or primarily decontamination.

We observed that sifting through the data flow to find and understand relevant information was a difficult and often frustrating experience for the exercise participants.

Nassau County TTX - Decision-Making Analysis

Project 755901 Page 12

We also observed that the process of situational understanding was not particularly coordinated or centralized. Each team was pursuing this critical understanding independently and from its own perspective. Sharing of conclusions reached with other teams was limited. In addition, the Unified Command table did not seem to focus on gaining a centralized understanding of “the big picture” and pushing that understanding out to all teams. Their focus was mostly on “medium picture” issues in support of other teams, such as decontamination planning. Some of the observed behavior may have been an exercise artifact, as much of the big picture understanding was provided to the participants prior to the exercise, and many of the participants also took part in exercise planning.

6.3 Theme 3: Situational Understanding May Not Have Triggered Protocols

6.3.1 OBSERVATIONS

During earlier interviews with Nassau County emergency response agencies and responders we identified that many organizations had tactical level protocols, procedures and Standard Operating Guides (SOGs). We found that often an NDM-like process would used to gain situational understanding (“I know what we’re dealing with here”). This understanding immediately would lead to selection of an appropriate SOG or protocol if one existed. This seemed to be a good blend for the Nassau County culture – NDM decision-making leading to procedure-based actions, with experience used to adapt procedures.

During the exercise we watched for statements like “Implement the MCI protocol.”

Instead we observed statements such as “Send in the MCI unit.”

We did not observe this expected behavior during the exercise. In every case we observed, situational understanding led to development of an action plan from scratch based on knowledge and experience.

6.3.2 ANALYS IS

Emergency response organizations usually have well defined SOGs and protocols for tactical-level response actions. Typical examples include

• Operating in PPE

• Hazardous materials (HAZMAT) reconnaissance and sampling

• Decontamination

• Medical triage

• Medical treatment

• Mass Casualty protocols

• Hospital protocols for treatment of contaminated victims

• Public Information

There are several possible reasons that we did not observe the behavior we expected from the interviews:

Nassau County TTX - Decision-Making Analysis

Project 755901 Page 13

• Protocols were selected and implemented but we didn’t observe the actions

• Protocols would have been selected and implemented at a level below the teams that were participating in the exercise

• The TTX play stayed at a high enough level that the selection and implementation of protocols did not occur

• Naturalistic Decision-Making dominates to a tactical level with unique courses of action being developed for each situation and existing protocols not employed

If the last option applies, Nassau County would benefit from ensuring that tactical protocols and SOGS are developed, trained and used in practice where appropriate.

6.4 Theme 4: Recognition-Primed Decision Model applied

6.4.1 OBSERVATIONS

During the exercise we observed a consistent approach to naturalistic decision-making among the participants. Decision makers would:

• Gather and sift information

• Gain situational awareness by comparing the information to their experiences or knowledge

• Use the situational awareness to select a protocol or build an action plan based on relevant experience

• Implement the protocol or action plan

Illustrative comments heard during the exercise and hot wash included:

“Here’s all I know. Nerve agent, mass fatalities, someone asked for a Chempack on the radio. This tells me I need PPE, we need to decon, we need more Chempacks, and we’re going to have walking wounded and self-referrals coming in soon. So the question now is: what are our capabilities?”

“Throw up? People sick all over the parking lot? Then gear on and it’s chemical.

We all know. Then get eyes on (protected) and just make an educated decision.”

"It's like math, you have to continually assess. There might not be a 'best solution,' but you just do the best you can."

"Start getting information, what'd you see, where is it, did you smell anything, is it up, is it down?"

6.4.2 ANALYS IS :

Sometimes referred to as “gut instinct,” “intuition,” or “common sense,” it appeared that responders relied substantially on their experience, thinking of a similar situation (or multiple related situations) and remembering how events unfolded and what actions

Nassau County TTX - Decision-Making Analysis

Project 755901 Page 14 were taken in that situation. Even when the past situation is only somewhat similar, pieces of information can be used and then modified to better mold to the current one.

This helped them to anticipate how events would play out during the exercise and determine what actions were feasible.

The decision-making observed during the exercise was consistent with a specific NDM approach known as the Recognition-primed Decision (RPD) model (Klein, 1993; 1998).

It is considered by some to be the most appropriate framework for describing how proficient decision makers operate, especially under conditions of time pressure and uncertainty (Lipshitz, et al. 2001). According to the RPD model, the goal of a decision maker is often to rapidly find and select the first reasonable and workable solution to resolve or control a situation.

The RPD model is highly intuitive, to the point that most decision makers do not realize that they are using the process. When asked how they made a key decision in an emergency, the most common answers were “It was experience” and “I just knew what to do.” When examined in more detail, they usually discover that they were implementing an RPD-like process.

Presented below is a simplified form of the RPD model that AlphaTRAC has observed in multiple crisis decision-making environments. We believe this is representative of the process most often observed in the Nassau County exercise. An example decision, victim decontamination, is tracked through the process.

6.4.2.1 STEP 1: GATHER KEY INFO – RAPID SIZE UP - CHARACTERIZE

The initial focus is on a rapid collection of key information about the event (called “cues” in the RPD model). Often termed “Initial Size up” the decision maker or team seeks to identify the type, scale, and scope of the event (See discussion of Situational Understanding above).

Example size up cues:

• There are multiple victims

• Symptoms are consistent with chemical exposure rather than injury

• Witnesses describe a device being detonated

6.4.2.2 STEP 2: IDENTIFY KEY CUES AND RELATE TO PAST EXPERIENCE - RECOGNIZE

In the RPD model the key cues gathered about the event lead to recognition of similar experiences (e.g., past events responded to, training scenarios, shared experiences).

The initial size up information leads to recognition of candidate experiences, which reveals additional key cues that will converge the decision maker toward situational understanding. The process iterates until the decision maker has recognized one or more cases that are representative of the current situation.

Example cases:

Nassau County TTX - Decision-Making Analysis

Project 755901 Page 15

• Highway accident involving release of toxic chemical

• Industrial accident impacting multiple workers

• Emergency response training exercise during Weapons of Mass Destruction

(WMD) class

6.4.2.3 STEP 3: EVALUATE KEY INDICATORS FROM PAST EXPERIENCE(S) – ANALYZE

Next the decision maker critically examines the case or cases to determine if they really apply to the current situation. A case or blend of cases is chosen to use as a basis for decision-making and development of a course of action. The key information carried forward from the cases includes situational understanding, candidate courses of action and lessons-learned (what worked well, what to avoid).

Example considerations:

• Highway accident produced contamination for the surrounding area and a toxic atmospheric plume

• Industrial accident resulted in a mass casualty incident with several contaminated victims

• WMD class stressed the importance of rapid decontamination to reduce continued exposure

6.4.2.4 STEP 4: MODIF ICATION OF PAST PLAN OR COURSE OF ACTION – CUSTOMIZE

While past experiences form the basis for decision-making, it is also true that every event is unique. In this step the decision maker identifies the key differences between the current situation and the cases recalled from memory. The previous courses of action are adapted and the lessons-learned applied. A top-level (go/no-go) decision is made. A course of action (strategic and/or tactical) is developed based on adaptations of the previous cases.

Example considerations:

• Contamination area is limited because release was contained indoors

• Limiting exposure is an urgent need because of nature of chemical

• There are many more victims than previously experienced

Example decision:

• Implement Gross Decontamination as a top priority

Example course of action:

Nassau County TTX - Decision-Making Analysis

Project 755901 Page 16

• Set up water-based gross decontamination corridor using fire truck water sprays and operate as a “production line.”

6.4.2.5 STEP 5: REHEARSE COURSE OF ACTION BEFORE IMPLEMENTING - DRAMATIZE

An important step that is easy to overlook under urgent emergency conditions is rehearsal of the course of action. A “quick and dirty” rehearsal of the action plan can identify and address flaws before they are encountered on the ground, greatly increasing the chance of success. This can be done as a mental process by the decision maker alone, discussed with a deputy or team or drawn out using a makeshift sandbox. Discipline must be exercised to conduct this step when the situation is driving the decision maker to action.

Example rehearsal considerations:

• Route from fire truck staging area to decontamination location is blocked – clear the route

• Clean and warm clothing not available for victims after decontamination – get hospital gowns to site from hospital immediately, follow up with clothing from Red Cross

• There is no shelter for hundreds of wet and cold victims after decontamination – get busses with heaters to decontamination site

6.4.2.6 STEP 6: IMPLEMENT COURSE OF ACTION - UTIL IZE

Once a “good enough” solution has been identified, rehearsed and tested, the final step is to implement the course of action - immediately. Courage and self-discipline must be exercised to commit to a course of action and see it through. A 75% solution implemented early can be modified and will move the response toward control and resolution of the event. Waiting for a best solution may allow the event to get irretrievably ahead of the responders.

Example implementation:

• Implement the decontamination plan and reassess continually to adapt

• Further reassess and adapt as additional resources arrive at scene

• Look forward 30 minutes to 1 hour to identify next steps

6.5 Theme 5: Flow of Critical Decision Information Was Delayed

6.5.1 OBSERVATIONS:

As is the case with almost all exercises and actual emergencies, information flow is a major challenge and typically is a key factor in the speed and success of event

Nassau County TTX - Decision-Making Analysis

Project 755901 Page 17 resolution. This was the case for critical decision information in the Nassau County exercise.

Illustrative comments heard during the exercise and hot wash included:

• “We are at the point…. Seriously… we need information NOW.”

• “Do we here at the hospitals know what the substance is?”

• “This is Medcon. What do you have on symptoms?”

• “We needed to know more the event sooner, to enhance the timeliness of hospital notification.”

• “I understand that good accurate information is what you want to put out, but it just didn’t feed the need. Information needed to come out sooner.”

• “We needed to get victim information to med sooner. We need constant updates so we can get more resources.”

• “On scene responders are trying to send out accurate info. Understood. But we need something reliable sooner for us get resources. Time is important. Less important is preciseness or accuracy.”

6.5.2 ANALYS IS :

Information flow is being examined as an emphasis area for the exercise. Many of the related issues are elated to information and communications management and will not be addressed here. We have identified two issues that are particularly important to decision-making that will be examined further in this report:

• Identification of decision-critical information

• Speed/accuracy tradeoff

6.5.2.1 IDENTIF ICATION OF DECIS ION-CRIT ICAL INFORMATION

We observed in the exercise that each team focused on obtaining the critical information that it needed to make its decisions. However, there was general difficulty in identifying where that information could be obtained and effectively getting it. Needed information was often available but not communicated.

A root cause for this problem was a singular focus within all teams on “what information do we need and how do we get it.” An additional focus is needed: “What critical information do we have that other teams need and how do we get it to them.”

Adding this perspective to the Nassau County culture and process will require a deliberate analysis and action on the part of all agencies (perhaps facilitated by the Office of Emergency Management). This analysis would produce SOGs or checklists of critical information that each team would provide to other teams. For instance, HAZMAT might provide substance ID to Incident Command and Medcon (who would pass it on to hospitals); Hot Zone size to Incident Command, Fire and Police; Protective Action Distance to Incident Command. Obtaining and communicating this information

Nassau County TTX - Decision-Making Analysis

Project 755901 Page 18 outward would be automatically triggered by the type, severity and scope of the event (situational understanding).

One formal process for determining these critical information needs is a Decision Space Analysis. A Decision Space Analysis identifies:

• The environment in which decisions will be made: who, what, when, where, why

• The key decisions to be made

• The options for each decisions

• The critical information needed for each decision

• How the information will be obtained

• How the decision will be made

6.5.2.2 SPEED / ACCURACY TRADEOFF

One significant barrier to the sharing of critical decision information during the exercise was a mismatch in the levels of confidence and accuracy needed in the information.

Often, teams or individuals generating or obtaining information would hold communication until the level of confidence or accuracy was high. At the same time, teams that needed the information required much less certainty and accuracy to use the information in decision-making. This resulted in a significantly slower pace for critical decision-making during the exercise.

In experimental psychology this effect is well known and is termed the Speed/Accuracy Tradeoff. In time-dependent tasks there is usually a tension between the speed with which an answer is produced and the accuracy/certainty of the answer.

Ideally, in a time-urgent situation (such as an emergency response), each piece of information should be determined to the lowest acceptable level of accuracy and certainty needed for the decision it will support, then immediately released. The information can then be further refined and revised, pending other demands for information and action (a kind of “information triage”).

In practice, speed/accuracy tradeoff is usually a cultural characteristic, often related to the professional background of the generating organization (e.g., technical organizations are usually more focused on accuracy) or the risk tolerance of the organization (e.g., on-the-ground response teams are usually more risk tolerant). This produces a different speed/accuracy culture for each organization and team (and often individuals). The result: impeded flow of critical decision information, leading to delays in decisions.

This issue can be addressed by changing the paradigm – tying information accuracy to the decision it will support, rather than the culture of the producing organization.

Information accuracy can be addressed as part of the Decision Space Analysis process described above.

Nassau County TTX - Decision-Making Analysis

Project 755901 Page 19

7 RECOMMENDATIONS FROM THE NASSAU COUNTY TTX

Our analyses of the Nassau County TTX have led to five recommendations:

The judgment-based, naturalistic decision-making approach used by responders and agencies in Nassau County is almost universal across agencies, but it is basically an oral tradition. We believe that the methods and processes associated with this approach are fundamentally consistent across agencies. We recommend that these methods and processes be captured in simple, high level descriptions (perhaps based on the RPD model presented above) that can serve as the basis for a common understanding and application.

In all crisis situations, situational understanding (lifting “the fog of war”) is critical to rapid, effective decision-making. We recommend that practices and procedures include a focus on obtaining the critical few pieces of information that will lead to situational understanding and then stepping back for a moment to ask “what are we dealing with here?” This should be a primary function of each team lead, and especially of the Incident Commander. We further recommend that a formal process of sharing situational understanding be implemented, possibly through scheduled brief team leader conferences (perhaps over a dedicated radio channel).

It is easy in the stress and urgency of an emergency to forget what formal protocols, procedures and SOGs are in place and “shoot from the hip” instead. When protocols are available it is almost always best to start with them and adapt as needed for the situation – more careful thought can be applied during protocol development than is usually available in the heat of an emergency. We recommend that each organization create a simple work aid that links key decisions to available protocols. In this way the process of identifying and triggering protocols can be almost automatic.

We recommend that each organization conduct an analysis of their critical decisions and the information required to make them. A process such as the Decision Space Analysis above can be used. Then the analyses should be combined and structured so

Nassau County TTX - Decision-Making Analysis

Project 755901 Page 20 that each organization knows the information and accuracy that it “owes” to each other organization.

7.5 Recommendation 5: Develop Work Aids for Key Decisions and Actions

It is difficult to remember and properly execute all of the steps in a complex process.

This is especially true under the high stress of a life-and-death emergency. Work aids can significantly aid an emergency decision maker in making sure that all important information are considered and necessary actions are taken. Work aids can include simple process checklists, decision-making tools and analytical tools. The KISS principle should applied in developing all work aids. Work aids already exist for most technical emergency response disciplines. We recommend that Nassau County, working with DHS, identify, develop, implement, train to and apply such works aids for decision-making.

8 RECOMMENDATIONS FROM PRIOR ANALYSES

We previously developed recommendations resulting from interview-based analyses of emergency management in Nassau County. Those recommendations are repeated here for completeness.

8.1 Recommendation 6: Focus First on the First Ten Minutes of an Event

We saw repeatedly that the decisions made in the first few minutes of an event set the stage for response success or failure. The decision makers during this period are typically “first arriving officers” or the first arriving officer with incident command experience. We recommend that a focus be placed on preparing these decision makers for the decisions they must make. In particular, a transfer of knowledge and experience from more senior responders, through training and exercises, is recommended. This will not only prepare younger decision makers for effective response, but will also help to carry forward the experience and knowledge base.

8.2 Recommendation 7: Reduce the Time for Key Decisions

The Chemical Defense Demonstration Project has determined that streamlining the process for reaching and implementing the most impactful life and safety decisions in a chemical mass casualty event can radically improve performance in saving health and lives. We recommend that Nassau County, working with DHS, conduct a similar analysis and implement the identified processes.

8.3 Recommendation 8: Prepare for Unusual or Low Probability Events

The major limitation of judgment-based, naturalistic decision-making is in application to unusual or low-probability events – events where there is little organizational knowledge or experience to draw on. These events, which may happen every few decades, test and define the emergency response culture for a community. The Avianca Flight 52

Nassau County TTX - Decision-Making Analysis

Project 755901 Page 21 crash was an example of such an event. A large-scale chemical incident involving mass casualties would be another example. Organizations that employ judgment-based, naturalistic decision-making can prepare for such events by deliberately creating the needed knowledge and experience base through targeted planning, training and exercises. We recommend that Nassau County identify a specific list of low probability events that the County wishes to prepare for and implement such preparedness programs. The Chemical Defense Demonstration Project can serve as a template for these initiatives. We further recommend that the NIMS ICS structure for responding to large-scale events be examined for more formal implementation.

8.4 Recommendation 9: Train the Next Generation

The foundations of a judgment-based, naturalistic decision-making culture are the personal and corporate knowledge and experience that decision makers draw on. Like most communities, Nassau County will be faced with the loss of much of its corporate knowledge and experience as its emergency decision-making corps retires. The success of safety and preventive engineering has, thankfully, reduced the number, severity and complexity of emergency responses. A side effect is a smaller experience base for younger responders. The result for Nassau County will be a reduced ability to apply judgment and experience in emergency decision-making. Nassau County can mitigate this impact by deliberately creating the needed knowledge and experience base in the upcoming generation of decision makers through increased planning, training and exercises. We recommend that Nassau County implement such a long-term development program for new officers and upcoming incident commanders, with a focus on leadership skills such as decision-making.

9 RECOMMENDATIONS FOR THE CHEMICAL DEFENSE DEMONSTRATION

PROJECT

AlphaTRAC has also identified five recommendations for the Chemical Defense Demonstration Project, based on our observations and analyses in Nassau County:

9.1 Recommendation 10. Support Further Implementation for Nassau County

It will be necessary for Nassau County to develop SOGs, work aids, training and other processes and tools in order to prepare for decision-making and response in large-scale chemical events involving mass casualties. We recommend that DHS support this next phase of preparedness in Nassau County. Nassau County could then act as a test bed for developing and testing such processes and tools.

We found that Nassau County relies heavily on judgment-based, naturalistic decision-making for management of emergencies. Anecdotal discussions suggest that other

Nassau County TTX - Decision-Making Analysis

Project 755901 Page 22 communities may employ different mixes of decision-making styles (e.g., programmed/procedural vs. judgment-based/naturalistic) and that the choices may be cultural in many cases. The Chemical Defense Demonstration Project provides an ideal opportunity to examine this question, with definitive results a realistic expectation. We recommend that DHS extend the examination of decision-making styles and methods to include the other venues in the Chemical Defense Demonstration Project.

9.3 Recommendation 12: Understand and Share the Findings

The analysis and findings regarding emergency decision-making in Nassau County are potentially useful to the larger emergency management community. We recommend that these findings be examined more thoroughly and then be published and disseminated in ways that will reach the broad community.

9.4 Recommendation 13: Generalize the Results for Broad Application

It is anticipated that the Chemical Defense Demonstration Project will result in decision-making SOGs, work aids and processes specific to Nassau County. Many of these could have applicability beyond Nassau County. We recommend that selected processes and tools be generalized for broad application and made available to the emergency management community.

The Chemical Defense Demonstration Project is developing and testing methods for preparing a community to respond to large-scale chemical events with mass casualties.

We feel that these approaches could be more broadly applicable. We recommend that DHS formalize and extend these approaches for addressing other low-probability / high consequence emergency events, both natural and man-made.

10 REFERENCES

Klein, G.A. (1993). A recognition-primed decision model of rapid decision making. In

G.A. Klein, J. Orasanu, R. Calderwood, & C.E. Zsambok (Eds). Decision Making in Action: Models & Methods. Ablex Publishing, Norwood, Connecticut

Klein, G.A. (1998). Sources of Power: How People Make Decisions. MIT Press, Cambridge, Massachusetts.

Lipshitz, R., Klein, G.A., Orasanu, J., & Salas, E. (2001). Taking stock of naturalistic decision making, Journal of Behavioral Decision Making, 14, 331-352.

Nassau County TTX - Decision-Making Analysis

Project 755901 Page 23

11 ABBREVIATIONS AND ACRONYMS

CDDP Chemical Defense Demonstration Project

CDP Chemical Defense Program

DHS Department of Homeland Security

HAZMAT Hazardous materials

NCFM Nassau County Fire Marshal

NDM Naturalistic Decision Making

OHA Office of Health Affairs

PPE Personal Protective Equipment

RPD Recognition-Primed Decision

SOG Standard Operating Guides

TTX Tabletop Exercise

WMD Weapons of Mass Destruction