BAA Sources Sought Notice - Cognitive Biases Taxonomy.pdf

Ver. 20DEC2016

W911NF-18-S-0005-FY2023

SOURCES SOUGHT NOTICE

REQUEST FOR WHITE PAPERS

BAA TOPIC A-8.b.1: STRATEGIC LEADERSHIP

Establish Empirically Based Taxonomy of Cognitive Biases

INTRODUCTION

Broad Agency Announcement (BAA) W911NF-18-S-0005 was publicized on SAM.gov and Grants.gov on 1 May 2018. This Sources Sought Notice calls for White Paper submissions in reference to the BAA Topic A-8.b.1: STRATEGIC LEADERSHIP. The United States Army Research Institute for the Behavioral and Social Sciences (ARI) Broad Agency announcement W911NF-18-S-0005, issued under the provisions of paragraph 6.102(d)(2) of the Federal Acquisition Regulation, provides for the competitive selection of basic and applied research and that part of development not related to the development of a specific system or hardware procurement. A Proposal submitted in response to this BAA and selected for award is considered to be the result of full and open competition and in full compliance with the provisions of Public Law 98-369, “The Competition in Contracting Act of 1984,” and subsequent amendments.

Funding of research and development (R&D) within ARI areas of interest will be determined by funding constraints and priorities set during each budget cycle. Any award related to the submission of a White Paper and subsequent Proposal requested by this Notice is subject to funds availability and priorities.

ARI may choose not to select any new award due to unavailability of funds or other factors.

The sequence of steps leading to an award is:

1) Request for White Paper initiated by ARI through this Sources Sought Notice

2) Submission of a timely White Paper no more than five pages in length to the POC for the U.S.

Army Contracting Command, wilveria.a.sanders.civ@army.mil, and copy furnish (CC) the ARI Technical Point of Contact (TPOC), ava.j.loer.civ@army.mil.

3) Written or telephonic feedback provided to the Offeror regarding the White Paper’s scientific merit and potential contributions to the Army mission as per established criteria presented in Part III

4) If the White Paper merits it, a request of a formal proposal initiated by ARI

5) Submission of a timely, formal proposal

6) Evaluation of the formal proposal as per established criteria presented in Part III

7) Award for selected proposal based on availability of funds or other factors

This sequence allows earliest determination of the potential for funding and minimizes the labor and cost associated with submission of a full proposal that has minimal probability of being selected for mailto:wilveria.a.sanders.civ@army.mil mailto:ava.j.loer.civ@army.mil funding. Note that an interested Offeror must submit a White Paper electronically to be eligible to submit a formal proposal under this Notice. This Notice requires that a White Paper be submitted electronically no later than 12 September 2022, 5:00 PM Eastern Daylight Time. See Part V, Deadlines, for additional details. BAA W911NF-18-S-0005 allows several potential instrument types (e.g., contract, grant, cooperative agreement) to result from a successful proposal. For this Notice, the intention of the Government is to award a contract.

THOSE SUBMITTING A WHITE PAPER/PROPOSAL ARE CAUTIONED THAT ONLY A GOVERNMENT

CONTRACTING OR GRANTS OFFICER CAN OBLIGATE THE GOVERNMENT THROUGH AWARD OF A LEGAL

INSTRUMENT INVOLVING EXPENDITURE OF GOVERNMENT FUNDS.

This Sources Sought Notice for a Requested White Paper consists of seven parts as follows:

• Part I: Research and Development Interests of the Requested White Paper

• Part II: Preparation and Submission

• Part III: Evaluation Criteria

• Part IV: Feedback

• Part V: Deadlines

• Part VI: Inquiries

• Part VII: References

ACC (APG) RTP Agency Point of Contact: The POC for the US Army Contracting Command (Aberdeen Proving Ground) Research Triangle Park Division is Ms. Wilveria Sanders, (919) 549-4328, wilveria.a.sanders.civ@army.mil.

ARI Agency Point of Contact: The ARI POC for technical matters for this White Paper topic is Dr. Ava Loer, (913)702-5057, ava.j.loer.civ@army.mil.

I. RESEARCH AND DEVELOPMENT INTERESTS OF THE REQUESTED WHITE PAPER:

The United States Army Research Institute for the Behavioral and Social Sciences is the Army’s lead agency for the conduct of research, development, and analyses for the improvement of Army readiness and performance via research advances and applications of the behavioral and social sciences that address personnel, organization, training, and leader development issues. ARI contracts with educational institutions, non-profit organizations, and private industry for research and development (R&D) in different areas, including the areas specifically identified in Paragraph 8 of W911NF-18-S-0005.

Efforts funded under this White Paper request will only include Applied Research and/or Advanced Technology Development.

Applied Research provides a systematic expansion and application of knowledge to design and develop useful strategies, techniques, methods, tests, or measures that provide the means to meet a recognized and specific Army need. Applied Research precedes system specific technology investigations or development, but it should have a high potential to transition into the Advanced Technology Development (ATD) Program.

The ARI ATD Program includes the development of technologies, components, or prototypes that can be tested in field experiments and/or simulated environments. Projects in this category have a direct relevance to identified military needs. These projects should demonstrate the general military utility or cost reduction potential of technology in the areas of personnel selection, assignment, and retention;

training strategies and techniques; leader education and development; performance measurement; and team and inter-organizational mission effectiveness. These projects should be focused on a more direct operational benefit and if successful, the technology should be available for transition.

WHITE PAPER TOPIC: Establish Empirically Based Taxonomy of Cognitive Biases

Cognitive biases are predispositions towards certain perspectives based on improperly weighed information (Haselton & Nettle, 2006). A dual-process theory of higher cognition has been invoked in explanations of why cognitive biases occur (Kahneman, 2003). The two processes have been labeled by Stanovich and West (2000) as System 1 and System 2. System 1 is a quick and intuitive process that tends to rely on heuristics. System 2 is a slow, deliberate, and analytic process. There are many times when people may privilege System 1 processing over System 2 processing, such as when decisions need to be made quickly, when there is not enough information or there is too much information, or solutions are not guaranteed (Gigerenzer & Gaissmaier, 2011). Most of the time, System 1 processing can be reasonably expected to lead to viable and timely solutions (Evans, 2010). However, it has also been observed that engaging in System 1 processing increases susceptibility to cognitive biases, which in turn may lead to erroneous decisions (Kahneman, 2011; Korteling et al., 2018).

Many types of cognitive biases have been identified. The National Research Council (2015, Chapter 3), in their report recommending that the U.S. Army Research Institute (ARI) support research to understand cognitive biases, noted that the Wikipedia page listing cognitive biases describes 92 decision-making biases, belief bias, and behavioral biases; 27 social biases; and 48 memory errors and biases. The number of identified cognitive biases have grown since then. At the time this Sources Sought Notice was written, the same Wikipedia page listed 124 belief biases, decision-making biases, and behavioral biases organized into 14 categories; 31 social biases organized into five categories; and 66 memory errors and biases organized into two categories. The Wikipedia page of cognitive biases may not be considered a definitive list because some items listed on that page may not be considered cognitive biases (National Research Council, 2015, Chapter 3). However, the page illustrates that there is a wide variety of ways that people can be biased when they make decisions. Manoogian (2016) took a more aesthetically pleasing approach to organizing cognitive biases. In his Cognitive Biases Codex, 188 cognitive biases were graphically presented as a radial dendrogram with four categories—i.e., what should we remember, too much information, need to act fast, and not enough meaning—and 20 subcategories.

ARI is particularly interested in understanding and mitigating cognitive biases that detrimentally affect Army leaders’ decision-making in operational environments. Army leaders make decisions in operational environments that are volatile, uncertain, complex, and ambiguous. This difficult task is exacerbated by technological advancements that automate warfighting functions, which truncate Army leaders’ decision-making cycles. Consequently, Army leaders may rely more on System 1 processing, a factor contributing to the expression of cognitive biases in operational decision-making (Stormer, 1991).

Military leaders are very much cognizant of how cognitive biases might have led to suboptimal decisions in the past. Scholars of military history reviewed past high-profile military operations and proposed cognitive biases that might have shaped the outcome of those operations, based on existing knowledge about individuals who made the most significant decisions in the operations and the conditions under which those decisions were made. The following examples are a small subset of the numerous accounts of past military operations that may have been influenced by cognitive biases. One example is how Robert E. Lee’s loss at the Battle of Gettysburg in 1863 has been attributed to his confirmation bias because he might have refused to believe during the three days of battle that his forces might not win (Janser, 2007). Another example is how the U.S. military’s strategic missions in Bosnia from 1995-2004, and in Haiti in 1994, were detrimentally affected by force-protection issues due to imaginability bias, specifically, the inaccurate perception that those operational environments would be similar to the 1993 operational environment in Somalia that resulted in the deaths of U.S. Army Rangers (Williams, 2010).

Yet another example is how the U.S. did not leave Vietnam until 1973 due to the sunk cost fallacy, or arguments that withdrawing from Vietnam would mean that the American Soldiers who were killed in action there died in vain (Kai, 2016). Still yet another example, albeit with a positive twist, is King’s (2019) retelling of how Abraham Wald, a researcher at Columbia University, prevented the War Department from committing survivorship bias. According to King, Wald discovered that most of the damage to bombers that made it back to the base were to the wings and the fuselage, and that engines and cockpits tended to be less damaged, leading the War Department to conclude that the wings and the fuselage should be reinforced with armor plating. However, Wald argued that armor reinforcement should be placed on engines and cockpits because damage to those areas most likely meant that the bombers would not make it back to the base (King, 2019).

Military leaders are also acutely aware of how cognitive biases might lead to suboptimal decisions in the future. The following sources illustrate, but should not be considered an exhaustive account of, military scholars’ concerns regarding specific cognitive biases in current operational environments.

Stormer (1991) described how military leaders made many significant decisions over the course of the day, and that those decisions often involved multiple variables and best guesses. Stormer cautioned that the cognitive baggage that every individual carried could especially corrupt the military decision-making process and might result in devastating consequences, and that it was important that Army leaders spent time in Service schools studying the decision process and the biases that influenced decisions.

Stormer listed representativeness bias, availability bias, probability bias, confirmatory bias, and anchoring and adjustment as prevalent biases that military decision-makers had a high probability of encountering. Yancy (2012) made a similar argument that individuals needed to be aware of their own tendencies and cognitive biases that might cause inaccurate beliefs and errors in judgments. Yancy stressed that this awareness could be accomplished when organizations facilitated metacognition. Yancy identified anchoring bias and confirmation bias as especially insidious for military decision-making.

Janser (2007) pointed out that research should identify cognitive biases that officers might be susceptible to, as Army manuals about the Military Decision Making Process (MDMP) recognize that general biases may affect decision-making but do not discuss specific biases or how to alleviate the detrimental effects of biases. Janser acknowledged that the availability heuristic, the representativeness heuristic, and the anchoring heuristic typically helped people make quick and effective decisions.

However, Janser also observed that the same heuristics opened the door to 10 different cognitive biases: overconfidence bias, insensitivity-to-sample-size bias, availability bias, illusory-correlation bias, retrievability-of-instances bias, escalation bias, break-even bias, snake bite bias, fear-of-regret bias, and confirmation bias. Williams (2010) echoed Janser’s concerns regarding the shortcomings of MDMP with respect to cognitive biases. Also, similar to Janser, Williams identified the availability heuristic, the representativeness heuristic, and the anchoring heuristic as instrumental for military decision-making.

Williams also described how each heuristic might lead to different biases. The availability heuristic might lead to retrievability bias, search set bias, imaginability bias, and illusory correlation bias. The representativeness heuristic might lead to insensitivity to prior probability of outcomes, base-rate neglect, insensitivity to sample size, misconceptions of chance, and failure to identify regression to the mean. The anchoring heuristic might lead to biases in the evaluation of conjunctive and disjunctive events.

Kai (2016) identified overconfidence bias, confirmation and disconfirmation bias, availability bias, sunk cost fallacy, and anchoring bias to have the potential to affect how operational staff might evaluate situations, consequently affecting how operational and strategic leaders might make decisions. Aside from the possibility of being provided information that had been subject to cognitive biases of the operational staff, higher level leaders themselves are especially susceptible to cognitive biases. At higher echelons, tasks are vaguer and more uncertain, compelling higher level military decision-makers to rely more on heuristic processes (Stormer, 1991).

There is a wide variety of cognitive biases that military scholars have cautioned against. Some of these cognitive biases have appeared in many publications while other cognitive biases have appeared in only a few publications. Changes in the operational environment, such as technological advances and the changing nature of warfare, may have also introduced cognitive biases that have hampered military decision-making in significant ways but are yet to be named in published literature. A gap exists in the literature, where the cognitive biases to which Army leaders may be susceptible during combat operations are yet to be established through systematic empirical data collection and data analyses. The requested research will conduct the necessary scientific investigations to identify these cognitive biases and to develop assessments of Army leaders’ susceptibility to the cognitive biases, as well as to identify interventions that can help Army leaders mitigate the detrimental effects of those cognitive biases on their decision-making.

The specific research objectives for this effort are:

1. Identify common cognitive biases associated with System 1 thinking among Army leaders by producing a front-end analysis. The front-end analysis will involve reviewing military and academic literature on cognitive biases, as well as collecting data from military personnel and academic subject matter experts. An Army-specific taxonomy of cognitive biases will be developed based on this front-end analysis to enable targeting of most critical biases to which Army leaders are susceptible.

2. Develop assessments to enable individual diagnosis of susceptibility to key cognitive biases, which will involve designing, evaluating, and refining a psychometric assessment of Army leader cognitive biases to include data collection, data analysis, design, development, refinement, and final presentation of the research product.

3. Develop interventions leveraging the cognitive bias taxonomy and individual assessment of cognitive biases to mitigate the potential negative impact of those biases on Army leaders’ decision-making process. This objective will involve military subject matter expertise to support development of interventions most viable for the Army leader development context, as well as military-relevant examples and case studies, and adaptation of the interventions/product to the unique cases represented by different stakeholders/anticipated transition partners.

4. Document all findings for publication.

The White Paper shall demonstrate the Offeror’s expertise in cognitive bias research, a clear understanding of how cognitive biases may affect how Army leaders make decisions in combat environments, and knowledgeability in how assessments can be used to guide learning development.

The White Paper shall also demonstrate the Offeror’s experience with writing manuscripts in accordance with current American Psychological Association (APA) Style.

The award will be approximately a 36-month period of performance with a budget not to exceed $600,000, distributed as follows:

- Base period: 6 months not to exceed $100,000

- Option 1: 12 months not to exceed $200,000

- Option 2: 12 months not to exceed $200,000

- Option 3: 6 months not to exceed $100,000

The Army Contracting Command, Aberdeen Proving Ground, RTP Division has the authority to award a variety of instruments, to include contracts, grants, and cooperative agreements. The ACC (APG) RTP Division reserves the right to use the type of instrument most appropriate for the effort proposed.

II. PREPARATION AND SUBMISSION OF A WHITE PAPER:

Preparation of White Paper

A White Paper should focus on describing details of the proposed research for both the base and option approach, including how it is innovative and how it could substantially advance the state of the science.

Army relevance and potential impact should also be described, as well as an estimate of total cost for both the base and option approach. White Papers should present the effort in sufficient detail to allow evaluation of the concept's technical merit and its potential contributions to the Army mission.

A White Paper must be limited to five (5) pages plus the cover page and an addendum in which the Offeror must include a biographical sketch (up to 300 words per individual) of all key personnel (i.e., Principal Investigators and Co-Principal Investigators) who will perform the research, highlighting their qualifications and experience as discussed below. All files and forms must be compiled into a single PDF file or MS Word document before submitting. Reviewers will be advised that they are only to review the cover page and up to five pages plus the addendum. Any pages submitted in excess of the five (5) page limit will not be reviewed or evaluated.

TECHNICAL INFORMATION FOR A WHITE PAPERS:

1. Technical Approach: A detailed discussion of the effort's scientific research objectives, approach, relationship to similar research, level of effort, and estimated total cost; include the nature and extent of the anticipated results, and if known, the manner in which the work will contribute to the accomplishment of the Army's mission related to this request and how this would be demonstrated.

2. Requests for Government Support: The type of support, if any that the Offeror requests of the Government (such as facilities, equipment, demonstration sites, test ranges, software, personnel, or materials) shall be identified as Government Furnished Equipment (GFE), Government Furnished Information (GFI), Government Furnished Property (GFP), or Government Furnished Data (GFD). Offeror shall indicate any Government coordination that may be required for obtaining equipment or facilities necessary to perform any simulations or exercises that would demonstrate the proposed capability.

3. Key Personnel Biographical Information: As an addendum to the White Paper, the Offeror must include a biographical sketch (up to 300 words per individual) of all key personnel (i.e., Principal Investigators and Co-Principal Investigators) who will perform the research, highlighting their qualifications and experience.

RESTRICTIVE MARKINGS ON WHITE PAPERS:

1. The Offeror must identify any proprietary data the Offeror intends to be used only by the Government. The Offeror must also identify any technical data or computer software contained in the White Paper that is to be treated by the Government as limited rights or restricted rights respectively. In the absence of such identification, the Government will assume to have unlimited rights to all technical data or computer software presented in the White Paper.

Records or data bearing a restrictive legend may be included in the White Paper but must be clearly marked. It is the intent of the Army to treat all White Papers as procurement sensitive information before the award and to disclose their contents only to Government employees or designated support contractors for the purpose of procurement related activities only.

Classified, sensitive, or critical information on technologies should not be included in a White Paper.

2. An Offeror is cautioned that portions of White Papers may be subject to release under terms of the Freedom of Information Act, 5 U.S.C. 552, as amended.

Submission of White Paper

White Papers must be submitted by e-mail to the POC for the U.S. Army Contracting Command, wilveria.a.sanders.civ@army.mil, and cc’d to the ARI Point of Contact (POC), ava.j.loer.civ@army.mil, in electronic MS Word document format or PDF file format. Cite “ARI BAA W911NF-18-S-0005Y, Establish Empirical Taxonomy of Cognitive Biases” in the e-mail subject line.

III. EVALUATION CRITERIA:

White Papers and full Proposals received in response to this request will be evaluated by the ARI designated point of contact identified in this request using the following factors/criteria:

1. Overall scientific and/or technical merits of the proposed research: The Offeror must demonstrate a thorough understanding of the research problem and articulate a clear approach to addressing the research objectives. The proposed approach must be based on feasible and realistic scientific methods and provide the Government with a high level of confidence of successful completion within proposed timeline.

2. Potential contributions of the effort to the U.S. Army Research Institute’s mission: In addition to showing how the proposed research aligns with the solicited topic, the Offeror must also specify how the proposed research will contribute to or advance the current U.S. Army Research Institute’s behavioral and social science research mission.

3. The Offeror’s capabilities, related experience, facilities, techniques, or unique combination of these which are integral factors for achieving the proposed research objectives: the Offeror must demonstrate relevant organizational capabilities [e.g., experience, facilities, techniques] beyond individual capability to ensure the proposed research objectives can be achieved by the organization as a whole.

4. The qualifications, capabilities, and experience of the proposed Principal Investigator and other key personnel who are critical to the achievement of the proposed research objectives: The Offeror must demonstrate that the key personnel have the education and experience necessary to successfully complete the research objectives. Personnel capabilities need to be aligned with the expertise needed to achieve the required research objectives.

5. The Offeror’s record of past performance: Past performance information obtained from the Offeror’s White Paper, centralized past performance databases, and other sources will be used to evaluate past performance on projects of similar size, scope, and complexity to determine timely compliance with Government specifications. For an Offeror with no relevant past performance, the Government may take into account information regarding the past performance of predecessor companies, subcontractors, or proposed personnel that will perform key aspects of the proposed research.

6. Realism, affordability, and appropriateness of proposed costs: The Offeror must demonstrate that the proposed costs are appropriate given the suggested research approach. The Government will also evaluate the proposed costs in relation to the scope of the research objectives and the U.S. Army Research Institute’s budget.

7. Current and/or future relationship or interface with ongoing activities in academia and/or industry: The Offeror must demonstrate how the research findings and/or products could inform current or future scientific research, impact industry, and/or meet Army needs and goals.

The request for a proposal will be made based on the overall evaluation of a White Paper using the seven criteria listed above. Criterion (1) is most important; the other criteria are of equal importance to one another. A request for proposal may not necessarily be made to the lowest proposed price. During the evaluation of White Papers, ARI’s POC for technical matters may request a discussion with an Offeror, but discussions are not guaranteed nor required for competition and award purposes. ARI’s POC for technical matters reserves the right to evaluate a White Paper and request a proposal without discussions The Offeror’s initial submission should contain the Offeror’s best terms from a technical and price standpoint. Once a full proposal has been requested, all communications must go through the POC for the U.S. Army Contracting Command.

If the White Paper evaluation results in the request and submission of a full proposal, the proposal will be evaluated by a panel of scientific peers using the same factors/criteria as those listed above under Evaluation Criteria. A request for a full proposal does not guarantee an award. The decision to award will be based on feedback from the panel, considerations presented by ARI’s POC for technical matters identified in this document, and other factors like budgetary constraints. ARI may choose not to select any award due to unavailability of funds or other factors.

IV. FEEDBACK:

Written or telephonic feedback will be provided to the Offeror regarding the White Paper’s scientific merit and potential contributions to the Army mission. If the Government decides to request a full proposal, a written request will be sent to the Offeror. The Written Request will, at a minimum, invite a full proposal. The request may also include feedback intended to improve the proposal’s potential for award.

V. DEADLINES:

Electronic versions of the White Paper must be received by the POC for the U.S. Army Contracting Command and the ARI POC, with e-mail subject line “ARI BAA W911NF-18-S-0005Y, Establish Empirical Taxonomy of Cognitive Biases” by e-mail no later than 5:00 PM Eastern Daylight Time on 12 September 2022. Any extension to the White Paper submission deadline will be posted to SAM.gov as an amendment to this Notice. Note that a timely White Paper received under this Notice will be evaluated and considered for proposal requests throughout the period beginning 12 September 2022 and ending 12 December 2022. An extension of this timeline may be granted based on the number of White Papers submitted or other factors out of the control of the designated point of contact reviewing the White Papers. An Offeror will be notified by email if the White Paper evaluation timeline is extended beyond 12 December 2022.

Instructions for the submission of a full Proposal will be provided by the U.S. Army Contracting Command to an Offeror invited to submit based on the evaluation of their White Paper.

An Offeror is responsible for submitting an electronic White Paper or full proposal so as to be received and accepted at the Government site indicated in this Notice no later than the date and time specified above. When sending electronic files, an Offeror shall account for potential delays in file transfer from the originator’s computer to the Government website/computer server. An Offeror is encouraged to submit their response early to avoid potential file transfer delays due to high demand or problems encountered in the course of submission.

An Offeror should receive confirmation of delivery at the Government site, not just successful relay from the Offeror’s system. Acceptable evidence to establish the time of receipt at the Government site includes documentary and electronic evidence of receipt maintained by the Government site. All submissions shall be submitted before the deadline identified above to be considered – no exceptions.

If an emergency or unanticipated event interrupts normal Government processes so that a White Paper or full proposal cannot be received at the site designated for receipt by the date and time specified above, then the date and time specified for receipt will be deemed to be extended to the same day and time specified in this Notice on the first workday on which normal Government processes resume.

An Offeror agrees to hold the terms of their White Paper and any subsequent proposal valid for 180 calendar days from the date of submission.

VI. INQUIRIES:

ACC (APG) RTP Agency Point of Contact (Contractual Questions): The POC for the US Army Contracting Command (Aberdeen Proving Ground) Research Triangle Park Division is Ms. Wilveria Sanders, (919) 549-4328, wilveria.a.sanders.civ@army.mil.

ARI Agency Point of Contact (Technical Questions): The ARI POC for technical matters for this White Paper topic is Dr. Ava Loer, (913) 702-5057, ava.j.loer.civ@army.mil.

VII. REFERENCES:

Evans, J. St. B.T. (2010). Intuition and reasoning: A dual-process perspective. Psychological Inquiry, 21, 313-326. https://doi.org/10.1080/1047840X.2010.521057

Gigerenzer, G., & Gasissmaier, W. (2011). Heuristic decision making. Annual Review of Psychology, 62, 451-482. https://doi.org/10.1146/annurev-psych-120709-145346

Haselton, M. G., & Nettle, D. (2006). The paranoid optimist: An integrative evolutionary model of cognitive biases. Personality and Social Psychology Review, 10(1), 47-66.

https://doi.org/10.1207%2Fs15327957pspr1001_3

Janser, M. J. (2007). Cognitive biases in military decision making (Accession No. ADA493560). U.S. Army

War College.

Kahneman, D. (2003). A perspective on judgment and choice: Mapping bounded rationality. American

Psychologist, 38(9), 697-720. https://doi.org/10.1037/0003-066X.58.9.697

Kahneman, D. (2011). Thinking, fast and slow. Farrar, Straus and Giroux.

Kai. C. J. (2016). Cognitive biases: The root of irrationality in military decision-making. Pointer, 42(2), 61-

72.

King, I. (2019). What do cognitive biases mean for deterrence? The Strategy Bridge.

https://thestrategybridge.org/the-bridge/2019/2/12/what-do-cognitive-biases-mean-for-deterrence

Korteling, J. E., Brouwer, A., & Toet, A. (2018). A neural network framework for cognitive bias. Frontiers in Psychology, 9, 1-12. https://doi.org/10.3389/fpsyg.2018.01561

List of cognitive biases. (2022, July 1). In Wikipedia.

https://en.wikipedia.org/wiki/List_of_cognitive_biases

Manoogian, J. III. (2016). The cognitive bias codex.

https://commons.wikimedia.org/wiki/File:Cognitive_bias_codex_en.svg

National Research Council of the National Academies (2015). Measuring human capabilities: An agenda for basic research on the assessment of individual and group performance potential for military accession. The National Academics Press.

https://doi.org/10.1080/1047840X.2010.521057 https://doi.org/10.1146/annurev-psych-120709-145346 https://doi.org/10.1207%2Fs15327957pspr1001_3 https://doi.org/10.1037/0003-066X.58.9.697 https://thestrategybridge.org/the-bridge/2019/2/12/what-do-cognitive-biases-mean-for-deterrence https://thestrategybridge.org/the-bridge/2019/2/12/what-do-cognitive-biases-mean-for-deterrence https://doi.org/10.3389/fpsyg.2018.01561 https://en.wikipedia.org/wiki/List_of_cognitive_biases https://commons.wikimedia.org/wiki/File:Cognitive_bias_codex_en.svg

Stanovich, K. E., & West, R. F. (2000). Individual differences in reasoning: Implications for the rationality debate? Behavioral and Brain Sciences, 23, 645-726.

https://doi.org/10.1017/S0140525X00003435

Stormer, W. P. (1991). The decision dilemma—Cognitive bias (Accession No. ADA235660). U.S. Army War

College. https://apps.dtic.mil/sti/pdfs/ADA235660.pdf

Williams, B. S. (2010). Heuristics and biases in military decision making. Military Review.

https://apps.dtic.mil/sti/pdfs/ADA532527.pdf

Yancy, S. D. (2012). From methodical battle to shock and awe: How to guard against the next great idea.

Joint Forces Staff College.

https://doi.org/10.1017/S0140525X00003435 https://apps.dtic.mil/sti/pdfs/ADA235660.pdf https://apps.dtic.mil/sti/pdfs/ADA532527.pdf