W911NF-18-S-0005-FY21-03.pdf

W911NF-18-S-0005-FY21-03

SOURCES SOUGHT NOTICE

REQUEST FOR WHITE PAPERS

BAA TOPIC A-8.B.1: STRATEGIC LEADERSHIP

Establish Validity and Reliability of Systems Thinking Assessment

INTRODUCTION

Broad Agency Announcement (BAA) W911NF-18-S-0005 was publicized on FedBizOpps (beta.SAM) and Grants.gov on 30 April 2018. This Sources Sought Notice calls for White Paper submissions in reference to the BAA Topic A-8.b.1: STRATEGIC LEADERSHIP. The United States Army Research Institute for the Behavioral and Social Sciences (ARI) BAA W911NF-18-S-0005, issued under the provisions of paragraph 6.102(d)(2) of the Federal Acquisition Regulation, provides for the competitive selection of basic and applied research and that part of development not related to the development of a specific systems or hardware procurement. A proposal submitted in response to this BAA and selected for award is considered to be the result of full and open competition and in full compliance with the provisions of Public Law 98-369, “The Competition in Contracting Act of 1984,” and subsequent amendments.

Funding of research and development (R&D) within ARI areas of interest will be determined by funding constraints and priorities set during each budget cycle. Any award related to the submission of a White Paper and subsequent proposal requested by this Notice is subject to funds availability and priorities.

ARI may choose not to select any new awards due to unavailability of funds or other factors.

The sequence of steps leading to award is:

(1) Request for White Paper initiated by ARI through this Sources Sought Notice

(2) Submission of a timely White Paper no more than five pages in length to the Point of Contact (POC) for the U.S. Army Contracting Command (ACC) (Aberdeen Proving Ground [APG]) Research Triangle Park (RTP) Division, maria.d.nelson.civ@mail.mil, and copy furnish (CC) the ARI POC for technical matters, ava.j.santos3.civ@mail.mil.

(3) Written or telephonic feedback provided to the Offeror regarding the White Paper’s scientific merit and potential contributions to the Army mission as per established criteria presented in Part III

(4) If the White Paper merits it, a request of a formal proposal initiated by ARI

(5) Submission of a timely, formal proposal

(6) Evaluation of the formal proposal as per established criteria presented in Part III

(7) Award for selected proposal based on availability of funds or other factors

This sequence allows earliest determination of the potential for funding and minimizes the labor and cost associated with submission of full proposals that have minimal probability of being selected for funding. Note that interested Offerors must submit a White Paper electronically in order to be eligible to submit a formal proposal under this Notice. This Notice requires that White Papers be submitted electronically no later than 30 August 2021, 5:00 P.M. Eastern Daylight Time (EDT). See Part V, Deadlines, for additional details. BAA W911NF-18-S-0005 allows several potential instrument types (e.g., contract, grant, cooperative agreement) to result from a successful proposal. For this Notice, the intent of the Government is to award a contract.

THOSE SUBMITTING A WHITE PAPER/PROPOSAL ARE CAUTIONED THAT ONLY A GOVERNMENT

CONTRACTING OR GRANTS OFFICER CAN OBLIGATE THE GOVERNMENT THROUGH AWARD OF A LEGAL

INSTRUMENT INVOLVING EXPENDITURE OF GOVERNMENT FUNDS.

This BAA for a Requested White Paper consists of seven parts as follows:

Part I: Research and Development Interests of the Requested White Paper

Part II: Preparation and Submission

Part III: Evaluation Criteria

Part IV: Feedback

Part V: Deadlines

Part VI: Inquiries

Part VII: References

ACC (APG) RTP Division POC:

The POC for the ACC (APG) RTP Division is Ms. Maria D. Nelson, (919) 541-4992, maria.d.nelson.civ@mail.mil

ARI POC for technical matters:

The ARI POC for technical matters for this White Paper topic is Dr. Ava Santos, (913) 684-9778, ava.j.santos3.civ@mail.mil

I. RESEARCH AND DEVELOPMENT INTERESTS OF THE REQUESTED WHITE PAPER

The United States Army Research Institute for the Behavioral and Social Sciences is the Army’s lead agency for the conduct of research, development, and analyses for the improvement of Army readiness and performance via research advances and applications of the behavioral and social sciences that address personnel, organization, training, and leader development issues. ARI contracts with educational institutions, non-profit organizations, and private industry for research and development (R&D) in different areas, including the areas specifically identified in Paragraph 8 of W911NF-18-S-0005.

Efforts funded under this White Paper request will only include Applied Research and/or Advanced Technology Development.

Applied Research provides a systematic expansion and application of knowledge to design and develop useful strategies, techniques, methods, tests, or measures that provide the means to meet a recognized and specific Army need. Applied Research precedes system-specific technology investigations or development, but it should have a high potential to transition into the Advanced Technology Development (ATD) Program.

The ARI ATD Program includes the development of technologies, components, or prototypes that can be tested in field experiments and/or simulated environments. Projects in this category have a direct relevance to identified military needs. These projects should demonstrate the general military utility or cost reduction potential of technology in the areas of personnel selection, assignment, and retention;

training strategies and techniques; leader education and development; performance measurement; and team inter-organizational mission effectiveness. These projects should be focused on a more direct operational benefit and if successful, the technology should be available for transition.

WHITE PAPER TOPIC: ESTABLISH VALIDITY AND RELIABILITY OF SYSTEMS THINKING ASSESSMENT

Systems thinking is understanding how a variety of elements and domains in a complex and dynamic environment are interrelated and contribute to form a coherent whole (Babin, 2013; Sackett, Karrasch, Weyhrauch, & Goldman, 2016). Systems thinking is a complex cognitive skill that Army leaders need in order to overcome the Army’s challenges. ADP 6-22: Army Leadership and the Profession (2019) stated that systems thinking is important for leaders to understand how different parts of a system work together to get the best performance from the whole system. The Army Modernization Strategy (2019) noted that Army leaders who are proficient systems thinkers can effectively manage the complexities of multi-domain operations (MDO). According to the Army People Strategy (2019), systems thinking will enable Army leaders to function optimally in the Information Age.

In order to prepare Army leaders for the systems thinking rigors of their jobs, systems thinking may be ingrained in Captains’ Career Courses (CCC), in Command and General Staff College (CGSC) courses, and in other required courses and programs. Additional programs, such as the School for Advanced Military Studies (SAMS) program and the University of Foreign Military and Cultural Studies’ (UFMCS) Red Team course, exist for Army leaders to further hone their systems thinking skills. Furthermore, opportunities are available for Army leaders to practice systems thinking as part of their self-development. ARI’s Managing Complex Problems (MCP) website houses various exercises that allow learners to iteratively engage in systems thinking at their own pace.

Feedback is an essential aspect of learning (Fernandez-Toro & Hurd, 2014; Orrell, 2006). Feedback apprises learners of their progress and allows them to identify goals that can further their educational development. Currently, there are no validated and reliable mechanisms in place to provide Army leaders with standardized feedback on their current systems thinking skills and how they can improve as systems thinkers. In line with the Army Modernization Strategy’s (2019) call to update leader development and education processes to improve Army leaders’ systems thinking skills, ARI developed a multi-rater assessment tool of systems thinking. This assessment tool was designed to help Army leaders gauge their own systems thinking proficiencies, as well as to identify behaviors that Army leaders can practice in order to improve their systems thinking skills. The assessment tool identified five different components of systems thinking, examples of behaviors that are indicative of each component, and how the behaviors may vary depending on whether the systems thinker is a novice, an intermediate, or an expert. The multi-rater assessment tool can be used by instructors or trainers to provide feedback to students or trainees. The assessment can also be used by peers to provide feedback to each other.

Lastly, the assessment can also be used by the individual to assess his or her own systems thinking proficiencies.

The requested research will conduct the necessary scientific investigations and revisions in order to establish the multi-rater assessment tool as a scientifically valid and reliable instrument that can gauge Army leaders’ current systems thinking proficiencies, and that can provide guidance on how Army leaders can develop their systems thinking skills.

The specific research objectives for this effort are:

1) Identify the appropriate types of validity and reliability that need to be established in order to demonstrate that the multi-rater assessment tool is a scientifically valid and reliable measure of Army systems thinking.

2) Efficiently collect and analyze data to establish the multi-rater assessment tool’s validity and reliability.

3) Refine the multi-rater assessment tool, based on the outcomes of the data analyses, into a valid and reliable assessment of systems thinking that can be used to help Army leaders gauge their current systems thinking skills and identify how Army leaders can improve as systems thinkers.

4) Document all findings for publication.

The White Paper shall demonstrate the Offeror’s expertise in establishing validity and reliability of tests or scales, a clear understanding of systems thinking and how it is important in the Army, knowledgeability in how assessments can be used to guide learning development, and experience with writing manuscripts using proper American Psychological Association (APA) format. The Offeror is encouraged to provide innovative ideas on how to realistically and efficiently collect data in the Army in ways that reduce the burden on officer participants without compromising scientific integrity.

The award will be approximately an 18-month period of performance with a budget not to exceed $300,000.00.

The U.S. Army Contracting Command (ACC) (Aberdeen Proving Ground [APG]) Research Triangle Park (RTP) Division has the authority to award a variety of instruments, to include contracts, grants, and cooperative agreements. The ACC (APG) RTP Division reserves the right to use the type of instrument most appropriate for the effort proposed.

II. PREPARATION AND SUBMISSION OF A WHITE PAPER

Preparation of White Paper

A White Paper should focus on describing details of the proposed research for both the base and option approach, including how the research is innovative and how it could substantially advance the state of the science. The base period is 8 months, and the option period is 10 months. Army relevance and potential impact should also be described, as well as an estimate of total cost for the base year. White Papers should present the effort in sufficient detail to allow evaluation of the concept’s technical merit and its potential contributions to the Army mission.

A White Paper must be limited to five (5) pages plus the cover page and a personnel addendum. In the personnel addendum, the Offeror must include a biographical sketch (up to 300 words per individual) of all key personnel (i.e., Principal Investigators and Co-Principal Investigators) who will perform the research, highlighting their qualifications and experience as discussed below. All files and forms must be compiled into a single PDF file or MS Word document before submitting. Reviewers will be advised that they are only to review the cover page and up to five pages plus the addendum. Any pages submitted in excess of the five (5) page limit will not be reviewed or evaluated.

TECHNICAL INFORMATION FOR WHITE PAPERS

1. Technical Approach: A detailed discussion of the effort’s scientific research objectives, approach, relationship to similar research, level of effort, and estimated total cost. Include the nature and extent of the anticipated results and, if known, the manner in which the work will contribute to the accomplishment of the Army’s mission related to this request and how this contribution would be demonstrated.

2. Requests for Government Support: The type of support, if any, which the Offeror requests of the Government (such as facilities, equipment, demonstration sites, test ranges, software, personnel, or materials) shall be identified as Government Furnished Equipment (GFE), Government Furnished Information (GFI), Government Furnished Property (GFP), or Government Furnished Data (GFD). Offerors shall indicate any Government coordination that may be required for obtaining equipment or facilities necessary to perform any simulations or exercises that would demonstrate the proposed capability.

3. Key Personnel Biographical Information: As an addendum to the White Paper, the Offeror must include a biographical sketch (up to 300 words per individual) of all key personnel (i.e., Principal Investigators and Co-Principal Investigators) who will perform the research, highlighting their qualifications and experience.

RESTRICTIVE MARKINGS ON WHITE PAPERS

1. The Offeror must identify any proprietary data that the Offeror intends to be used only by the Government. The Offeror must also identify any technical data or computer software contained in the White Paper that is to be treated by the Government as limited rights or restricted rights, respectively. In the absence of such identification, the Government will assume to have unlimited rights to all technical data or computer software presented in the White Paper.

Records or data bearing a restrictive legend may be included in the White Paper, but must be clearly marked. It is the intent of the Army to treat all White Papers as procurement sensitive information before the award and to disclose their contents only to Government employees or designated support contractors for the purpose of procurement related activities only.

Classified, sensitive, or critical information on technologies should not be included in a White Paper.

2. The Offeror is cautioned that portions of White Papers may be subject to release under terms of the Freedom of Information Act, 5 U.S.C. 552, as amended.

Submission of White Paper

White Papers must be submitted by e-mail to the POC for the ACC (APG) RTP Division, maria.d.nelson.civ@mail.mil, and cc’d to the ARI POC for technical matters, ava.j.santos3.civ@mail.mil, in electronic MS Word document format or PDF file format. Cite ‘ARI W911NF-18-S-0005-FY21-03, Establish Validity and Reliability of Systems Thinking Assessment’ in the e-mail subject line.

III. EVALUATION CRITERIA

White Papers and full proposals received in response to this request will be evaluated by the ARI-designated point of contact identified in this request using the following factors/criteria:

1. Overall scientific and/or technical merits of the proposed research: The Offeror must demonstrate a thorough understanding of the research problem and articulate a clear approach to addressing the research objectives. The proposed approach must be based on feasible and realistic scientific methods and provide the Government with a high level of confidence of successful completion within proposed timeline.

2. Potential contributions of the effort to the U.S. Army Research Institute’s mission: In addition to showing how the proposed research aligns with the solicited topic, the Offeror must also specify how the proposed research will contribute to or advance the current U.S. Army Research Institute’s behavioral and social science research mission.

3. The Offeror’s capabilities, related experience, facilities, techniques, or unique combination of these which are integral factors for achieving the proposed research objectives: the Offeror must demonstrate relevant organizational capabilities [e.g., experience, facilities, techniques] beyond individual capability to ensure the proposed research objectives can be achieved by the organization as a whole.

4. The qualifications, capabilities, and experience of the proposed Principal Investigator and other key personnel who are critical to the achievement of the proposed research objectives: The Offeror must demonstrate that the key personnel have the education and experience necessary to successfully complete the research objectives.

5. The Offeror’s record of past performance: Past performance information obtained from the Offeror’s White Paper, centralized past performance databases, and other sources will be used to evaluate past performance on projects of similar size, scope, and complexity to determine timely compliance with Government specifications. For an Offeror with no relevant past performance, the Government may take into account information regarding the past performance of predecessor companies, subcontractors, or proposed personnel that will perform key aspects of the proposed research.

6. Realism, affordability, and appropriateness of proposed costs: The Offeror must demonstrate that the proposed costs are appropriate given the suggested research approach. The

Government will also evaluate the proposed costs in relation to the scope of the research objectives and the U.S. Army Research Institute’s budget.

7. Current and/or future relationship or interface in academia and/or industry: The Offeror must demonstrate how the research findings and/or products could inform current or future scientific research, impact industry, and/or meet Army needs and goals.

The request for proposals will be made based on the overall evaluation of White Papers using the seven criteria listed above. Criterion (1) is most important; the other criteria are of equal importance to one another. During the evaluation of White Papers. ARI’s POC for technical matters may request a discussion with an Offeror, but discussions are not guaranteed nor required for competition and award purposes. ARI’s POC for technical matters reserves the right to evaluate a White Paper and request a proposal without discussions. The Offeror’s initial submissions should contain the Offeror’s best terms from a technical and price standpoint. Once a full proposal has been requested, all communications must go through the POC for the U.S. Army Contracting Command.

If the White Paper evaluation results in the request and submission of a proposal, the proposal will be evaluated by a panel of scientific peers using the same factors/criteria as those listed under Evaluation Criteria. A request for a proposal does not guarantee a contract award. The decision to award a contract will be based on feedback from the panel, considerations presented by ARI’s POC for technical matters identified in this document, and other factors like budgetary constraints. ARI may choose not to select any award due to unavailability of funds or other factors.

IV. FEEDBACK

Written or telephonic feedback will be provided to the Offeror regarding the White Paper’s scientific merit and potential contributions to the Army mission. If the Government decides to request a proposal, a written request will be sent to the Offeror. The written request will, at a minimum, invite a proposal.

The request may also include feedback intended to improve the proposal’s potential for award.

V. DEADLINES

Electronic versions of the White Paper must be received by the POC for the ACC (APC) RTP Division and the ARI POC for technical matters, with the e-mail subject line ‘ARI W911NF-18-S-0005-FY21-03, Establish Validity and Reliability of Systems Thinking Assessment’ by no later than 30 August 2021, 5:00 P.M. Eastern Daylight Time (EDT). Any extension to the White Paper submission deadline will be posted to beta.SAM as an amendment to this Notice. Note that a timely White Paper received under this Notice will be evaluated and considered for proposal requests throughout the period beginning 30 August 2021 and ending 30 September 2021. An extension of this timeline may be granted on the number of White Papers submitted or other factors out of the control of the designated POC reviewing the White Papers. Offerors will be notified by e-mail if the White Paper evaluation timeline is extended beyond 30 September 2021.

Instructions for the submission of a proposal will be provided by the U.S. Army Contracting Command to an Offeror based on the evaluation of the Offeror’s White Paper.

An Offeror is responsible for submitting electronic White Papers or proposals so as to be received and accepted at the Government site indicated in this Notice no later than the date and time specified above. When sending electronic files, an Offeror shall account for potential delays in file transfer from the originator’s computer to the Government website/computer server. An Offeror is encouraged to submit their response early to avoid potential file transfer delays due to high demand or problems encountered in the course of submission.

An Offeror should receive confirmation of delivery at the Government site, not just successful relay from the Offeror’s system. Acceptable evidence to establish the time of receipt at the Government site includes documentary and electronic evidence of receipt maintained by the Government site. All submissions shall be submitted before the deadline identified above in order to be considered—no exceptions.

If an emergency or unanticipated event interrupts normal Government processes so that a White Paper or full proposal cannot be received by the site designated for receipt by the date and time specified above, then the date and time specified for receipt will be deemed to be extended by the same day specified in this Notice to the first work day on which normal Government processes resume.

Offeror agree to hold the terms of their White Paper and any subsequent proposal based on it valid for 180 calendar days from the date of submission.

VI. INQUIRIES

ACC (APG) RTP Division POC (contractual questions)

The POC for the ACC (APG) RTP Division is Ms. Maria Nelson, maria.d.nelson.civ@mail.mil.

ARI POC (technical questions)

The ARI POC for technical matters related to this White Paper topic is Dr. Ava Santos, ava.j.santos3.civ@mail.mil.

VII. REFERENCES

Babin, L.B. (2013). Gender considerations for engagements in Afghanistan and beyond. In B. E. Strong, L.

Brooks Babin, M. Ramsden Zbylut, & L. Roan (Eds.), Sociocultural systems: The next step in Army cultural capability (Research Product 2013-02) (pp.215-227). Fort Belvoir, VA: U.S. Army Research Institute for the Behavioral and Social Sciences.

Department of the Army. (2019). Army leadership and the profession (ADP 6-22). Retrieved from https://armypubs.army.mil/epubs/DR_pubs/DR_a/ARN20039-ADP_6-22-001-WEB-0.pdf

Department of the Army. (2019). Army modernization strategy: Investing in the future. Retrieved from https://www.army.mil/e2/downloads/rv7/2019_army_modernization_strategy_final.pdf

Department of the Army. (2019). The Army people strategy. Retrieved from https://www.army.mil/e2/downloads/rv7/the_army_people_strategy_2019_10_11_signed_fina l.pdf

Fernandez-Toro, M., & Hurd, S. (2014). A model of factors affecting independent learners’ engagement with feedback on language learning tasks. Distance Education, 35(1), 106-125.

http://dx.doi.org/10.1080/01587919.2014.891434

Orrell, J. (2006). Feedback on learning achievement: rhetoric and reality. Teaching in Higher Education, 11(4), 441-456. http://dx.doi.org/10.1080/13562510600874235

Sackett, A.L., Karrasch, A.I., Weyhrauch, W.S., & Goldman, E.F. (2016). Enhancing the strategic capability of the Army: An investigation of strategic thinking tasks, skills, and development (Research Report 1995). Fort Belvoir, VA: U.S. Army Research Institute for the Behavioral and Social Sciences.

Managing Complex Problems. U.S. Army Research Institute for the Behavioral and Social Sciences, 2020, aratsg.com/public/MCP2020. Accessed 28 June 2021.