Questions and Responses W911NF-18-S-0005-FY20-05- Thinking in Time- Updated 12 September 2020.pdf

Responses to Questions Submitted through 12 September 2020

1. Can you give us a very specific example of courses (real-world applications) that either exist already or need to be introduced to train trainees on Thinking in Time skill development?

Government Response: No, I cannot. This would potentially be an outcome of early stages of the effort, if the approach focused on methods for development.

2. The Army research report (provided in references) lists a broad range of tasks requiring Strategic thinking. Can you list 1 or 2 definite tasks requiring the skill?

Government Response: Doing so at this stage would inappropriately limit the scope of proposals. I suggest you focus on the research approach for developing or measuring the skill.

The approach should facilitate application to many of a mid-grade Army officer’s tasks, such as command and staff work at echelons above Company. The officer’s cognitive skill is the focus, not their task performance.

3. Are we allowed to collect data on Army personnel in their training environment during the BAA execution?

Government Response: Under normal circumstances, this likely could be arranged, assuming unit support and comfort with the degree of intrusion. Feel free to propose such an approach in your white paper, with the understanding that alternative (i.e., virtual) data collection approaches would need to be developed to mitigate the risk presented by the pandemic.

4. Can you provide us access to Army instructors who are currently training personnel on Training in Time skill development or who will use results from our study?

Government Response: After a successful proposal, yes, with the understanding that Army instructors and trainers will largely have a diverse understanding of what the concept means and whether they seek to develop it in their courses and training exercises.

5. We believe the goal is to develop a canonical document during this BAA effort for the Army instructors to facilitate a better understanding of different measures to track Thinking in Time skill acquisition. Kindly confirm. By "canonical document", we mean a report like document detailing the findings/results of the effort addressing the questions listed in the BAA.

Government Response: Partially, yes, in that one of the deliverables expected of the effort will be a scientific technical report summarizing the results of the effort. Depending on the product produced, another report may be appropriate, which would be focused more on communicating guidance to an end-user. Such a report would be less technical in nature. The primary goal (i.e., scope) of the effort is to conduct scientific research in developing and validating either (or both) a measurement tool and/or a development method focused on thinking-in-time as described in the notice. To “facilitate a better understanding of different measures to tracking Thinking in

Responses to Questions Submitted through 12 September 2020

Time skill acquisition” is an appropriate proposal within that scope. To the extent that there are different measures available, the proposed research should focus on how they can be applied to a mid-grade Army officer’s development and how they might be tested within an Army context.

Responses to Questions Submitted through 2 September 2020

1. Was there something specific that drove this topic?

2. How does it fit in with the current research packages at FLRU?

Government Response: As stated on p. 3 of the notice, “Thinking-in-Time is one of the competencies of strategic thinking identified in ARI’s Research Report 1995 (Sackett, Karrasch, Weyhrauch, & Goldman, 2016).” Also, as described on p. 4, mid-grade to emerging senior leaders often have roles entailing forecasting the requirements of future scenarios, which would require skill at thinking-in-time. Research on ways to develop this skill and/or measure it would support ARI’s research package on methods and measures for complex cognitive skills in emerging senior leaders. More generally, historical case studies are a common methodology in professional military education, the study of history is commonly valued by experienced military leaders, and the ability to apply lessons from history is critical to developing strategic thinking.

Exploration of the psychological mechanisms behind this ability will further support the Army’s development of complex cognitive skills in its leaders.

3. How do you view Thinking in Time in relation to adaptability? The two constructs seem similar.

Government Response: Pg. 3 of the notice refers to the book Thinking in Time: The Uses of History for Decision Makers and the three central components described there: the predictive value of history, the role of unique factors in the present compared to history, and the iterative examination of change over time from history to present to possible futures. The key difference between thinking-in-time and adaptability would be in the time scale and the degree of proactive, rational thought involved. Adaptability would refer to a more momentary ability to respond effectively to changes in the environment. Thinking-in-time would be a different manner of skill, applying insights from study and reflection over time to build and maintain the understanding of context necessary to be adaptable when confronted by change. However, thinking-in-time might also reduce the scope of change that is unanticipated, based on the application of historical examples (both recent experience and the broader study of historical events). The generation of novel solutions to a problem in the moment (adaptability) would be supported by a more complex cognitive skill of identifying and applying patterns, trends, and other matters of context. This type of contextual understanding must be developed by examining the past, present, and future as a stream (i.e., thinking-in-time). Thinking-in-time entails the study of history for building situation awareness, improving accuracy in forecasting and anticipating change. Therefore, improved individual and organizational adaptability would likely be a natural benefit of improved thinking-in-time. We encourage further consideration of the relationship between adaptability and thinking-in-time, but do not wish to conflate the two constructs.

4. The solicitation says that the research will develop a method for OTJ development in thinking-in-time and/or a measure of effective engagement in thinking-in-time - is there a preference for a method vs. a measure? If we develop a method, would you also want at least some measure of effectiveness of that method (implying, to some extent, that we will have also measure thinking-in-time following the implementation of that method/intervention)? If a measure, what resources would be available to help gain validity evidence (e.g., access to Soldiers, etc.)?

Government Response: The notice was written explicitly to welcome either a method or a measure. There is no preference for one over the other. All else being equal, proposals that entail measures of effectiveness and/or validity would be stronger than those without. The resources and support made available by the government will be evaluated and considered in relation the approach proposed. ARI would assist with coordinating access to Soldiers as required, although risk mitigation strategies should be considered in light of constraints imposed by the COVID-19 pandemic.

5. Does the budget ($800k) include any anticipated travel (assuming we can travel at some point)?

Government Response: At this stage, the budget limit is a general description of the scope of effort expected and does not include or exclude anything in particular. Specifics regarding how that budget would be allocated would depend on the technical proposal. Risk-mitigation strategies with regard to travel approvals should be considered and will need to be accounted for should the government request a technical and cost proposal.

6. The WP topic description mentions mid-grade to emerging senior leaders on page 4 but then also talks about us identifying the leadership roles that may require thinking-in-time.

Are you thinking that this may be geared to certain positions (e.g., Company Commanders) rather than maybe a certain rank?

Government Response: As one of the research questions of interest, ARI is not pre-specifying which leadership roles require more thinking-in-time than others. The method or measure developed should have as much applicability as is feasible without sacrificing quality (fidelity, validity, impact). The target audience should be driven more by the form of method or measurement, within the stated core group of interest (mid-grade to emerging senior leaders).

We encourage greater focus on the approach at this point, perhaps including a recommendation for a target audience based on that approach. Company Commanders are not in the core group of interest and would require some justification from a developmental perspective if proposed.

However, we are not imposing a focus on specific ranks or positions at this stage, both are viable organizing principles. The choice should be driven mostly by the approach taken to development and/or measurement.

7. How would the behavioral observation rubric developed for strategic thinking align with the development of a behavioral-based assessment for thinking-in-time? Are there specific components of that previously developed rubric that should be incorporated into the more specific measure?

Government Response: The reference to the strategic thinking rubric (and its component behaviors) was made to provide additional clarity on ARI’s concept of thinking-in-time. If the proposed research seeks to develop a behavior-based assessment rubric, the theoretical/conceptual stage of the research should take the components into consideration, but not be constrained by them. An intent of this research is to delve into the thinking-in-time skill more deliberately and formally. This may result in modifications or a redefinition of the construct in the rubric, such is the purpose of further research.

8. What is the make-up your ideal team? Does it involve an academic?

Government Response: The criterion for evaluating the team is described in the ARI Broad Agency Announcement, specifically: “The qualifications, capabilities, and experience of the proposed principal investigator and other key personnel who are critical to achievement of the proposed objectives.” The term academic is not defined in the question. However, taken to mean personnel with extensive knowledge and experience in the area of study, the answer would be yes. The ideal team would depend on the approach taken, but would likely include personnel with a PhD (or equivalent terminal degree) in a related field, such as psychology, history, education, or other behavioral and social sciences. Ideally, the team would include personnel with a demonstrated research expertise in history education (especially at the graduate level), historical thinking, applied history, or a related specialty. Additional team member(s) with expertise in leader development methods and psychometric assessment may also be necessary to achieve the project objectives. There is no particular individual in mind that we would encourage you to seek out.