MilEval PWS.pdf

DRAFT

1 Ver: Oct 2019(1)

Performance Work Statement (PWS) Military Training Evaluation Program

1.0 INTRODUCTION

The Deputy Assistant Secretary of Defense for Force Education and Training (DASD FE&T) implements policies and procedures for Career Investment Programs (CIP) that are critical for war-fighting readiness as well as to ensure that service members are prepared for transition to the civilian workforce following military service. As a CIP, Voluntary Education (VolEd) programs enable service member access to quality postsecondary educational opportunities, empower informed service member decision-making, shape meaningful personal and professional pathways, and drive military student success in higher education. On behalf of the DASD FE&T, the Defense Activity for Non-Traditional Education Support (DANTES), a component of the Defense Human Resources Activity (DHRA), manages a portfolio of Defense voluntary education programs that help service members achieve their personal and professional career goals.

The Military Training Evaluation Program (MilEval) is a Defense education program that assesses the learning gained from military training and occupational experiences MilEval helps to ensure this learning is translated solely into college credit recommendations that are documented on the Joint Services Transcript (JST) and applied to satisfy vocational through graduate level postsecondary degree requirements.

2.0 BACKGROUND

For more than sixty years, the MilEval Program, with the support of a higher education organization, has evaluated and maintained a repository of college credit recommendations gained by service members through military training courses and occupational experiences. The program operates in accordance with DoDD 1322.08E, the Department of Defense (DoD) Voluntary Education (VolEd) Strategic Plan; and specifically, DoDI 1322.25 Encl. 3 para. 6.c.(5)(d). Through the MilEval Program, knowledge gained through military experiences is assessed and translated into recommended postsecondary college credit in terms of semester-hours. These college credit recommendations are currently accepted at over 2,000 academic institutions, as a form of prior learning assessment. Institutional acceptance of these credits is at the discretion of individual colleges and universities and varies greatly based on institutional credit acceptance and award policies. Applying college credit recommendations reduces the number of classes a service member must complete to satisfy degree requirements, thus saving time to degree completion while avoiding Tuition Assistance costs for the DoD. As a result, this program provides fiscal accountability for taxpayer investments on Defense human capital by expanding the benefits of training and occupational experiences beyond warfighting to support educational attainment and transition into the civilian workforce. Credit recommendations are summarized and documented on the JST, and additional information to substantiate each evaluated college credit recommendation is cataloged and archived in an online catalog. The JST Operations Center releases more than 400,000 transcripts to colleges and universities each year on behalf of service members and veterans able to convert military experience into college credit.

Traditionally, the incumbent Contractor has performed onsite academic military training course reviews using multidisciplinary postsecondary faculty teams. After a series of pre-assessment meetings with relevant military training school personnel, the faculty evaluation teams assessed the learning that occurred within military courses having more than 45 contact hours by reviewing course related materials and assessments. Additionally, learning that occurred through occupational experience was evaluated through semi-structured interviews with service member panels that represented each successive career level. In recent years, virtual faculty reviews of course materials uploaded to a secure site began augmenting the on-site review methodology. Using these methods, the DoD currently reviews approximately 440 training courses (an average total of 108,000 academic hours) and 60 occupations each

2 Ver: Oct 2019(1) year. Credit recommendations maintain a ten-year shelf life, with the exception of healthcare and information technology related courses which require more frequent reviews.

Although this evaluation methodology has served the Defense Department well over the years, policy direction for program modernization includes improving the Department’s capacity to:

• Optimize the evaluation process to decrease costs and increase the number of reviews

• Expand the transparency of service members’ equivalent college level learning to support the

Department’s recruitment, retention, readiness, and transition efforts

3.0 SCOPE

The goal of this requirement is to improve the Defense Department’s capacity to evaluate, document, and communicate the college level learning gained by service members as a result of military training courses and on-the-job experiences in a credible manner accepted by a wide range of stakeholders (e.g., the military Services, academic institutions, etc.). The Contractor shall provide an evaluation methodology, personnel, management, materials, information technology infrastructure, and equipment necessary to fulfill these requirements. Depending on the complexity of the course (e.g., length of course, number of programs of instruction, etc.) or occupational assessment, reviews may be conducted either remotely (virtual), face-to-face (on-site), or by some combination thereof. The Contractor shall ensure the output of these evaluations (e.g., postsecondary-level academic learning expressed as college credit equivalents) align with postsecondary academic credit frameworks and are clearly translated into formats accepted by academic institutions. The emphasis for this requirement is on increasing the Department’s capacity to conduct virtual reviews and support training initiatives to improve academic credibility of curriculum, assessments, and feedback on occupational training evaluations.

The Contractor shall leverage the multidisciplinary best and emerging practices from across postsecondary education and prior learning assessment to provide technical and management support in the following areas, further defined under PWS 4.0 REQUIREMENTS, herein:

4.1 Program Management

4.1.1 Program and Project Management

4.1.2 Reporting Requirements

4.1.3 Subject Matter Expert Reviewers

4.1.4 Human Subjects Research Compliance

4.1.5 Contractor Personnel, Disciplines, and Specialties

4.1.6 Confidentiality and Non-Disclosure

4.2 Pre-Evaluation Preparation

4.2.1 Annual Review Schedule Support and Risk Mitigation

4.2.2 Training Site Readiness, Evaluation Methodology

4.2.3 Training Site Readiness, Virtual and Onsite Support

4.3 Evaluation Methodology, Implementation, Assessment, and Refinement

4.3.1 Virtual and On-site Course and Occupational Reviews

4.3.2 Evaluation Types and Number of Reviews

4.3.3 Post Evaluation Workshops

4.3.4. On-Site Inspections

4.3.5 Evaluation Outputs

4.4 Data Management and Security

4.4.1 Data Repository

4.4.2 Data Transmittal for Military Transcript

4.4.3 Interoperability for Stakeholder Data Exchanges

4.4.4 Data Security

4.5 Customer and Stakeholder Support

3 Ver: Oct 2019(1)

4.5.1 Customer Support

4.5.2 Stakeholder Support

All materials accumulated, produced, or developed as a result of this contract are property of the Government and shall be available to the COR via electronic means at the end of the period covered by this contract.

4.0 REQUIREMENTS

The Contractor shall:

4.1 PROGRAM MANAGEMENT

Provide program and project management support to plan and execute all work required under this PWS.

All work conducted under this PWS shall be planned and executed in a manner that will achieve schedule and performance objectives. The Contracting Officer’s Representative (COR) shall serve as the Government’s primary point of contact for all program activities.

4.1.1 Program and Project Management

4.1.1.1 Facilitate a Post-Award Conference with key stakeholders identified by the Government within seven business days after contract award. The date, time, and location for the kick-off meeting shall be coordinated with the COR. Contractor personnel shall discuss technical, management, and security issues during the Post-Award Conference. The Contractor shall be prepared to discuss any items requiring clarification and gather information to support each deliverable and provide a written summary of the conference within five business days after the conference.

Within 14 calendar days of completion of the Post-Award Conference, develop a detailed Plan of Actions and Milestones (POA&M) for each initial requirement that supports the approach and methodology proposed. The plan shall indicate the timeline for deliverables, dependencies and resources needed, roles and responsibilities, and status report meetings. It shall address the major milestones and timing for delivering initial drafts, incorporating the COR’s comments, delivering revised copy/messaging, and delivering final products to the COR for approval. The COR will review the proposed project plan and return it to the Contractor within a reasonable period with comments and necessary adjustments.

4.1.1.2 Develop and execute a detailed Communication Management Plan to govern the process of providing stakeholders with program information (e.g., program/policy staff, military training sites, etc.).

The plan shall outline key messages, supporting information, and status updates communicated to specific audience segments (e.g., military Service training commands, academic institutions, service members, veterans, etc.). Additionally, the plan shall address authorities to release information, when and through what communication channels information will be delivered. The plan shall demonstrate how channels will solicit feedback and identify how communication will be documented and archived. The plan shall be delivered to the COR per the POA&M.

4.1.1.3 Develop and maintain a Risk Management Plan to address risks, their potential impact to the program, and include strategies to reduce these risks. The plan shall focus attention on minimizing threats to achievement of the program objectives. The Risk Management Plan shall include an approach for identifying and assessing risks; determining cost-effective risk reduction actions; and monitoring and reporting progress in reducing risks. The plan shall detail how the Contractor plans on reducing identified risks, to an acceptable level, within a specified timeframe. The plan shall describe how specific approaches will incorporate into program and project management plans to minimize or avoid identified risks; include proactive contingent risk response actions; and enable a means of rapidly implementing risk responses based on timely identification of risk occurrence. To provide visibility of risks and progress in mitigating them, the Risk Management Plan shall include a documented project Risk Watch List. The Risk Watch List shall be reviewed bi-weekly by the Contractor and Government team during routine

4 Ver: Oct 2019(1) meetings. This watch list shall summarize in a table the following items: risk identification/description, risk analysis (on a scale from 1 to 4), risk impact time horizon (e.g., short term or long term/ phase/date), risk control (internal or external), risk priority number, area of impact (e.g., time, cost, quality, other projects/programs, etc.), response strategy(ies), response strategy resources requirements, assigned person to handle risk, and date completed. The Risk Watch List shall function as the mechanism to track and report risks associated with the program. When necessary, risk occurrences shall be elevated to the COR and relevant Government stakeholders for their attention. The plan shall be delivered to the COR per the POA&M for review and approval.

4.1.1.4 Develop and adhere to an internal Quality Control Plan (QCP) to govern all aspects of program/project management, review preparation, the execution of reviews, the management of data, engagement with stakeholders, customer support to protect the integrity of the program and its associated outputs (e.g., college credit recommendations, stakeholder data exchanges, etc.). The Contractor shall include mechanisms to identify and self-assess critical processes under this effort. The plan shall describe measures, inspections, quality checks, monitoring of process parameters, etc. required to assure the evaluation process, the outputs it produces, and the resulting data repository conform to PWS requirements. The plan must detail the information required to control associated processes, produce quality outputs, and maintain improvements over the life cycle of the period of performance. This includes at a minimum, a method for performing inspections; identifying, correcting and preventing problems/defective service; addressing customer complaints, and improving the quality of services over the life of the contract. Consider the QCP a living document that should be updated periodically as the measurement methods and controls improve throughout the life cycle of the period of performance. The plan shall provide the basis for a quarterly risk-based surveillance report that details the on-going self-assessment and monitoring of critical processes. Deliver the plan to the COR per the POA&M for review and approval.

The QCP shall:

a. Demonstrate the reliability and validity of the evaluation methodology (whether virtual, onsite;

etc.) and its outputs. This should include identifying sources of variation or discrepancies in the evaluation process and establishing controls to monitor them. It is the Government’s expectation that a combination of quantitative and qualitative methods will be used to assess and control quality (e.g., statistical process control; process capability measurement; quality scores; measures of reliability; confidence intervals; parameters to determine sampling size and frequency;

comparing and contrasting course and occupational review outputs between virtual and on-site means; attributing data collection; error proofing; visual inspection using panels of experts for random quality checks; applying special characteristics matrices and lessons learned from credit acceptance/application issues; design reviews; process failure mode and effects analysis; team knowledge about the process; etc.).

b. Be transparent and provide assurance to a variety of stakeholders to support academic credit recommendation acceptance by colleges and universities. The QCP shall evolve as identified stakeholder needs evolve.

The Government reserves the right to perform inspections and surveillance to evaluate the Contractor’s compliance to the contract terms and performance of the requirements in the PWS.

4.1.1.5 Develop and execute a Configuration Management Plan to establish and mature the programmatic, technical, and functional attributes of systems that support the program. This includes configuration items such as systems, databases, and related support systems that are specifically used by the Contractor in the execution of this program. It shall govern the ongoing process of identifying and managing changes to deliverables and other program outputs. The plan shall define, document, control, implement, account for, and audit changes to various components of the program’s systems as they

5 Ver: Oct 2019(1) evolve to meet the Services and the Department’s needs. The Contractor shall develop the plan in accordance with (IAW) Federal Information Security Management Act (FISMA) and the National Institute of Science and Technology Special Publication (NIST SP) 800-53, Rev. 4. The Configuration Management Plan shall identify the Contractor’s approach to the systematic evaluation, coordination, approval or rejection, and implementation of all proposed changes in the configuration of a configuration item after formal establishment of its baseline. The plan must include procedures to ensure that changes are accomplished in an organized manner with traceability and accountability so that configuration management requirements are properly implemented. The Contractor shall ensure that requested changes to software, hardware, data, networks, or documentation are formally reviewed and approved or denied in order to allow evaluation of the effect of the change on security, performance, interfaces, acceptability, completeness, and documentation. The plan shall be delivered to the COR per the POA&M for review and approval.

4.1.1.6 The Configuration Management Plan shall include a project task list and corresponding timeline to deliver Configuration Baseline Information to the Government. This collection of information shall describe the technical characteristics of each configuration item (e.g., system, database). The baselines shall serve as technical control points in the life cycle for the evaluation of proposed changes to the technical characteristics. The baseline and approved changes or modifications shall be updated routinely to provide a current description of systems that support the program. The baselines shall include: the functional/requirements baseline; design baseline; development baseline; product baseline; and operational baseline.

The Configuration Management Plan shall also include a change management process that Government reviewers can monitor (e.g., security test and evaluation team) multiple times annually in support of FISMA and other reporting and auditing requirements. This includes the availability of change artifacts (e.g., Change Requests, Test Release Transmittals, Security Impact Analysis, testing evidence, etc.) to verify that the change control process is in place and operating effectively. The change management process must include: change/problem resolution tracking; measurements to determine the status of control implementation and effectiveness for configuration management activities and processes;

configuration status accounting to record, store, and report activities; and release management to govern software, hardware, and license controls.

The Contractor shall cooperate and coordinate with the Government technical team for formal configuration audits. The audits will certify that the design, development, and integration meet the system’s technical requirements, that they are accurately documented, and that they do not include unauthorized changes. To the extent possible, audits shall be performed to minimize impact on the program’s schedules. The Contractor shall address deficiencies through the recommended corrective actions according to the timelines proposed by the Government team.

4.1.1.7 Provide In-Process Reviews (IPRs) either virtually via teleconferencing or at a Government facility (DANTES in Pensacola, FL; the Mark Center Building, Alexandria, VA; or another designated Government site within the Washington, DC metropolitan area). The IPR shall focus on the status, progress, and any issues regarding the requirements. Summarize the IPR in a concise report that shall clearly summarize program data on work accomplished during the reporting period:

a. Semi-annual reporting periods shall be October-March and April-September. The period’s reports shall be published no later than (NLT) 15 days following the end of each reporting period.

Additional IPRs might be scheduled if critical issues arise or significant events or changes occur.

b. Electronic copies of the IPR summary shall be delivered to the COR and Technical Assistant(s) (TA). In addition, members of the InterService Voluntary Education Board (IVEB) may request copies of the reports.

c. Clearly summarize and present work accomplished during the reporting period, including but not

6 Ver: Oct 2019(1) be limited to qualitative and quantitative updates to summarize activity within each of the six PWS 4.0 Requirements (e.g., Program Management; Pre-Evaluation Preparation; Evaluation Methodology, Implementation, and Refinement; Data Management and Security; Customer and Stakeholder Support; and Travel).

In preparation of any IPR, the Contractor shall provide the following deliverable: Briefing Materials (IPR Summary with accompanying PowerPoint slide deck).

4.1.1.8 Records Management

Maintain records and documents used in the evaluation of military educational and occupational experiences including courses, programs of instruction, evaluation outputs, rubrics, etc. The Contractor shall convert all such records that are hard-copy documents into electronic format. In addition to the records and documents submitted by the Military Services, the Contractor shall maintain the documents used in completing the tasks of this statement of work. A central part of the records maintenance by the Contractor shall be a registry of all credit recommendations made as a result of the evaluations.

The purpose of maintaining records is to provide data and information; to develop reports regarding the program; to provide backup information to explain and document evaluations and procedures; to provide long-range management continuity (Government or Contractor); and to provide data and information to prepare products required within this PWS. The documentation should be stored in a manner that is readily accessible to the Government for broader analysis across Voluntary Education programs and systems with the goal of being able to document service member and program outcomes.

4.1.1.9 Corrective Actions

Rectify a task, process, product, or personnel behavior when any of these factors produce errors, problems in producing deliverables, deviations from intended program and project management plans, or impact the credibility of the program and its subsequent credit award. The Contractor shall promptly act to correct nonconforming outputs, processes, procedures, or personnel behavior to preclude the recurrence of the problem and to satisfy contract requirements. In these instances, the Contractor shall develop and submit a Corrective Action Report to the COR within seven business days.

4.1.1.10 Team Conferences

The Contractor’s team shall coordinate and join a conference call involving team members within DANTES and JST. The conference call is a forum to discuss timely aspects of contract performance on a weekly to bi-weekly basis.

4.1.2 Reporting Requirements

4.1.2.1 Deliver Routine Reports to include, but not limited to: program activities, project schedule, budget, issues, risk, planned value, etc.

4.1.2.2 Quarterly Reports

(1) Deliver a Quarterly Status Report (QSR) to the COR for distribution at InterService Voluntary Education Board (IVEB) meetings. The COR shall provide the annual IVEB meeting schedule and may provide a template for use in preparing the report. The QSR shall be delivered at least 15 business days prior to the scheduled IVEB meeting. The intent of this report is to provide the Voluntary Education Service Chiefs relevant information to support data-informed decision making, such as: overall program status, executed and upcoming reviews, issues that require policy level resolution, trends, and a forecast of possible constraints or efficiencies.

(2) Provide a Quarterly Military Programs Report based on the previous three months, to the COR and contract TA(s) and to each Service Program Manager (SPM) by 15 October, 15 January, 15

7 Ver: Oct 2019(1)

April, and 15 July, each year. The Contractor shall collaborate with the COR and contract TA(s) to assure the report’s data representation is clear to the recipients so that they understand the details of work accomplished during the reporting period as well as important trends and issues. The Contractor may report any data considered pertinent including, but not limited to, assessment of the following:

a. Qualitative Summary o Background / general overview of the program period of performance o Review data / synopsis (# of reviews, academic credit by subject area, breakdown by Service, breakdown by course and occupational community) o Outreach, Articles, Publications, and Training o Resource Support / Customer Service o JST / Transcripts o Faculty SMEs o Budget o Executed training site self-studies, preparation events, and reviews o Trends from stakeholder engagements o Programmatic recommendations for consideration

b. Quantitative Summary

At minimum, the report shall cover the following data points by branch of Service and in total:

o Program of Instruction (POI)/Course Planning Document (CPD)

• POIs/CPDs submitted by Service for review

• Credit Extended - No substantial difference in content, Contractor extended the credit evaluation vice conducting full review o Team Course Reviews:

• Onsite - # of team reviews at the training site.

• Virtual - # of team reviews through virtual collaboration (synchronous and asynchronous)

• Hybrid - # of team reviews through aspects of Onsite & Virtual

• Re-Scheduled - A planned Team review is rescheduled, whether onsite or virtual

• Cancelled - A planned Team review is cancelled, whether onsite or virtual o Total # of academic hours reviewed o Total # of Course POIs/CPDs Reviewed: Self-explanatory o Occupational Reviews:

• Occupations Reviewed - The number and type of occupations reviewed according to occupational categories, whether onsite or virtual

• Occupations Pulled from Reviews - Self-explanatory o # of Credits Recommended by vocational, lower undergraduate, upper undergraduate, and graduate level o # of Credits Recommended (in semester hours) by subject area (e.g., general education – math, communication, science, etc.; possible major area—business, information technology, STEM, aeronautics, healthcare management, etc.; graduate level possible major area— business, accounting, management, etc.).

4.1.2.3 Military Programs Data Report

An annual Military Programs Data Report shall report annual totals for each of the quantitative data points included in the Quarterly Military Programs Report for the previous October through September (fiscal year) period. The Military Programs Data Report shall be delivered to the COR NLT 15 Oct, each year, as the basis for the Voluntary Education Enterprise Management Information System (VEMIS), Military Evaluations Program (Sections 25. and 25.B) reporting requirements.

8 Ver: Oct 2019(1)

4.1.2.4 Contract Performance Report

On a monthly basis, provide Contract Performance Reports (CPRs) that briefly summarize contract activities, cost data, progress on plans of actions, key milestones accomplished, challenges/risks regarding deliverables or project progression, and mitigation strategies. Administratively, this may be combined with the other reports as applicable.

4.1.2.5 Information Technology and Data Security Report

On a monthly basis, provide Information Technology and Data Security Reports (ITDSR) to describe products, development schedules, security, etc. See Configuration Management, Data Management and Security, and Security sections of this PWS for details on reporting requirements.

4.1.2.6 Accounting Data Report

Submit a monthly financial report as an attachment to the monthly invoice submitted through Wide Area Work Flow (WAWF). The Accounting Data Report shall provide data and accounts for the outcome of the requirements set forth by this contract, specifically:

a. For IDIQ CLINs, detail events with corresponding levels of effort completed (e.g., preparation, virtual evaluation, onsite evaluations)

b. Travel expenses for the ACE staff and evaluators. This shall include the name of each traveler, travel dates, travel location (from and to), purpose of trip, mode(s) of transportation, transportation costs, per diem, and total costs.

c. The Contractor representative’s signature and date.

The Contractor shall submit one copy of the Accounting Data Report to the COR by the 30th day of the month that follows that in which expenses are incurred. Include the contract number and any other required accounting information on Accounting Data Reports and invoices submitted.

4.1.2.6 Provide monthly Course Planning Document (POI/CPD) Reports listing CDPs received by the Contractor during the reported month. The POI/CPD Reports shall be provided to the COR and, as

9 Ver: Oct 2019(1) applicable:

• Army—Army Credentialing and Continuing Education Services for Soldiers (ACCESS)

• Marine Corps—Marine Training and Education Standards Division (MTESD)

• Navy—Naval Education and Training Command (NETC, N74)

• Air Force—HQAF, Voluntary Education Branch, AF/A1DLV

• Coast Guard—USCG Education and Training Quota Center (ETQC)

4.1.2.7 Ad Hoc Reports

Periodically, furnish performance data relative to requirements of this contract to the COR, the SPMs/TAs, or both. Examples of such data include but are not limited to: POI/CPD reports, recommendation end-date reports, team trip reports, budget reports, etc. Requests for such data on an ad hoc (unscheduled) basis shall not exceed two per quarter and six per 12-month performance period.

4.1.3 Subject Matter Expert Reviewers

4.1.3.1 Secure the services of highly qualified academic subject matter experts (SMEs) qualified to cover each academic discipline and occupational cluster to support the evaluation methodology and quality control process. Evaluators shall be highly qualified professional educators, having at least five years of postsecondary teaching and curriculum development experience specific to the content area of the training course and occupational learning outcomes they will review. Qualifying experience (at least five years) shall come from work at an accredited college or university. In addition, distance learning courses shall be evaluated by SMEs who also have a background in distance learning curriculum development and delivery. The Contractor shall review the qualifications and establish that proposed evaluators are qualified.

4.1.3.2 Create and assemble evaluation teams of at least three highly qualified SMEs, recruited and selected by the Contractor, who are SMEs in the field to be evaluated. The Contractor shall provide the COR with the names of evaluation team members fifteen business days or as required by local or installation access rules, prior to conducting a site evaluation so the COR can furnish letters to identify official teams, manage security concerns, and facilitate installation access.

4.1.3.3 Evaluation teams shall represent a diverse and varied sampling of postsecondary academic institutions (e.g., diversity in terms of accrediting body, region of the country, institution type, Carnegie designations, tier/level, Ivy League, etc.). The diversity of academic institutions shall be reported in routine quarterly reports based on diversity measures that are meaningful and extend beyond accreditation body and institution type (public, private, not-for profit). The Contractor shall provide evaluation team members with an orientation and training on the Program’s evaluation process.

4.1.3.4 Provide all necessary information to the host Service’s designated Government official to satisfy security protocols on all Contractor personnel who require military installation and training resource access at least 15 business days in advance of scheduled reviews and staff assistance visits.

4.1.3.5 Leverage SMEs at various process points. This may include during pre-evaluation planning, out-briefs at the conclusion of reviews, curriculum summits, stakeholder engagement events, developing methodology transparency documentation, conducting virtual reviews, conducting on-site inspections, providing quality control on the evaluation process and its outputs, etc.

4.1.4 Human Subjects Research Compliance

4.1.4.1 Adapt all applicable evaluation methodologies to adhere to:

• Protection of Human Subjects (32 Code of Federal Regulations (CFR) Part 219) https://www.health.mil/Reference-Center/Policies/2005/06/23/Protection-of-Human-Subjects

10 Ver: Oct 2019(1)

• Protection of Human Subjects and Adherence to Ethical Standards in DoD-Supported Research (DoDI 3216.02)

• Protection of Humans Subjects in Research Projects (48 CFR Parts 207, 235 and 252)

4.1.4.2 Comply fully with 32 CFR Part 219 and DoD Directive 3216.02, applicable DoD component policies, and 10 U.S.C. 980. The Contractor shall not commence performance of research involving human subjects that is covered under 32 CFR Part 219 or that meets exemption criteria under 32 CFR 219.101(b), or expend funding on such effort, until and unless the conditions of either the following paragraph (c)(1) or (c)(2) have been met:

(1) The Contractor furnishes to the DoD Human Research Protection Officer (HRPO), with a copy to the COR and the Contracting Officer, an assurance of compliance and Institutional Review Board (IRB) approval and receives notification from the Contracting Officer that the HRPO has approved the assurance as appropriate for the research under the Performance Work Statement and also that the HRPO has reviewed the protocol and accepted the IRB approval for compliance with the DoD component policies. The Contractor may furnish evidence of an existing assurance of compliance for acceptance by the HRPO, if an appropriate assurance has been approved in connection with previous research. The Contractor shall notify the COR and the Contracting Officer immediately of any suspensions or terminations of the assurance.

(2) The Contractor furnishes to the HRPO, with a copy to the COR and the Contracting Officer, a determination that the human research proposed meets exemption criteria in 32 CFR 219.101(b) and receives written notification from the COR and/or the Contracting Officer that the exemption is determined acceptable. The determination shall include citation of the exemption category under 32 CFR 219.101(b) and a rationale statement. In the event of a disagreement regarding the Contractor's furnished exemption determination, the HRPO retains final judgment on what research activities or classes of research are covered or are exempt under the contract.

4.1.4.3 DoD staff, consultants, and advisory groups may independently review and inspect the Contractor's research and research procedures involving human subjects and, based on such findings, DoD may prohibit research that presents unacceptable hazards or otherwise fails to comply with DoD procedures.

4.1.4.4 Failure of the Contractor to comply with the requirements of this clause will result in the issuance of a stop-work order under Federal Acquisition Regulation clause 52.242-15 to immediately suspend, in whole or in part, work and further payment under this contract, or will result in other issuance of suspension of work and further payment for as long as determined necessary at the discretion of the Contracting Officer.

4.1.4.5 The Contractor shall include the substance of this clause, including this paragraph (f), in all subcontracts that may include research involving human subjects in accordance with 32 CFR Part 219, DoD Directive 3216.02, and 10 U.S.C. 980, including research that meets exemption criteria under 32 CFR 219.101(b).

4.1.5 Contractor Personnel, Disciplines, and Specialties

Provide the necessary resources and infrastructure to manage, perform, and administer the contract.

Accomplish the assigned work by employing and utilizing qualified personnel capable of meeting the requirements of the PWS.

The following reflects desired Contractor personnel capabilities and experiences that will mitigate the performance risk for this effort.

DRAFT

https://www.health.mil/Reference-Center/Policies/2011/11/08/Protection-of-Human-Subjects-and-Adherence-to-Ethical-Standards-in-DoD-Supported-Research https://www.health.mil/Reference-Center/Policies/2011/11/08/Protection-of-Human-Subjects-and-Adherence-to-Ethical-Standards-in-DoD-Supported-Research https://www.health.mil/Reference-Center/Policies/2008/10/27/Protection-of-Human-Subjects-in-Research-Projects

11 Ver: Oct 2019(1)

• Demonstrated experience and specialization within higher education, workforce development, competency-based learning, and directly supporting DoD Voluntary Education.

• Experience conceptualizing frameworks and developing solutions to support the evaluation and mapping of knowledge gained by service members from military training and occupational experiences to higher education standards and requirements.

• Demonstrated experience directly supporting the development of technology solutions to enable the modernization of DoD, higher education, and/or workforce programs and strategic initiatives, applying Agile development methodologies.

• Key personnel with experience directly supporting DoD, the higher education community, and/or the workforce development landscape with program evaluation and modernization, strategic planning, concept development, Agile technology solution design, development and implementation, etc.

• Key personnel with experience working with higher education organizations on training, change management, program design, credit evaluations, prior learning assessment, curriculum development and assessment, and institutional or programmatic accreditation. Ph.D. or Doctoral Degree highly desired.

4.1.5.1 Contractor Vacancy Management

Minimize disruptions caused by contract employee resignations and terminations by keeping the COR informed of contract employee departures. Expeditious hiring action by the Contractor is expected within 10 business days in order to mitigate any schedule interruptions. Inability to initiate hiring action within this time period shall be reported to the COR with supporting reason for the delay.

4.1.6 Confidentiality and Non-Disclosure

4.1.6.1 This project and all materials provided to the Contractor by the Government and results, conclusions and recommendations obtained thereof should be considered confidential in nature and treated with the same level of care that the Contractor treats its own confidential business information.

The information shall not be disclosed, copied, modified, used (except in the completion of this project) or otherwise disseminated to any other person or entity at any time to include, but not limited to inclusion in any database external to the Government without the Government's express consent. No data (digital or analog) provided to, or developed by, the Contactor shall be used for any purpose other than this contract.

4.1.6.2 Contractor personnel assigned to this project shall sign a Non-Disclosure Agreement prior to obtaining access to sensitive information. This effort, and all materials provided to the Contractor by the Government and their results, conclusions, and recommendations obtained, should be considered confidential and treated with care in accordance with applicable Government regulations. The information shall not be disclosed, copied, modified, used (except in completion of this project) or otherwise disseminated to any other person or entity at any time to include, but not limited to inclusion in any database external to the Government without the Government's express consent.

4.2 PRE-EVALUATION PREPARATION

4.2.1 Annual Review Schedule Support and Risk Mitigation

4.2.1.1 Support the Military Training Evaluation Program Working Group (PWS 9.1.3) in the development of an annual review schedule. Conduct a Gap Analysis, in coordination with the military Services and the JST Operations Center, no less than on an annual basis, to determine eligible courses and occupations in need of evaluation due to review expiration dates, major course content revisions, newly created courses eligible for review, and/or major revisions to occupational fields. The Gap Analysis Report will include courses and occupations targeted for evaluation priority. Deliver the Gap Analysis Report electronically to the COR, the Working Group, and relevant military Service stakeholders within

12 Ver: Oct 2019(1) three months of contract award and subsequently, no later than (NLT) beginning of month ten of each option year.

4.2.1.2 Include only courses and occupations meeting specified eligibility criteria for evaluation at the discretion of and in accordance with (IAW) prevailing priorities of the military Services. At minimum, to be eligible for academic evaluation courses must:

a. Contain at least 40 hours of engagement in academic activity.

b. Not contain “proprietary” material or the intellectual property of a Non-Federal Entity for purposes of academic a review.

c. Not include accredited curriculum of a nationally or regionally accredited institution (e.g., Community College of the Air Force) as recognized by the U.S. Department of Education.

d. Be developed and taught under the authority of a Military Service Training Command established to train military service personnel as established by appropriate authority of the applicable Military Department.

e. Have undergone major changes in their curriculum, have not previously been evaluated, have moved to a new training location, or have reached their 10-year limit.

1) The Contractor shall provide the training course/service occupations to be end-dated to the designated SPM no later than six months prior to the 10-year end-date.

2) The 10-year limit, does not apply to courses in rapidly changing career fields such as highly technical and medical courses. The Working Group shall establish end-date policies by occupational fields and/or based upon the needs of the Services.

f. Have an end-date-of-course and proctored assessment component if delivered by non-traditional instructional methodology (e.g., distance learning, CBT).

g. Provide for firm identification of the student and rigid control of test conditions.

h. Combine all blended learning components when course content objectives break into smaller units of learning to contain at least 40 hours of instruction, whether covered in a single setting or completed over a specified period using web-based or distance-learning modules. This includes emerging military training initiatives that deploy phased, ready relevant, or other just-in-time types of learning.

i. Be evaluated from Service learning sites and military training centers geographically located within the continental United States for on-site (other than virtual) reviews.

4.2.1.3 Facilitate coordination, prioritization, and final confirmation of a Schedule of Reviews with the Working Group, at least annually, among the COR and Service stakeholders. This includes identifying courses to undergo virtual review readiness assessments and assigning eligibility for virtual or onsite reviews. The Schedule of Reviews will articulate the timeframe assigned for each course and occupation review based on the Gap Analysis Report, Service annual or emerging requirements, and the quantity of reviews ordered by the Government for the period of performance.

a. Evaluations shall be scheduled and conducted in the following order of priority:

o First Priority: Courses for enlisted personnel and enlisted occupations o Second Priority: Courses for limited duty, NEC, and warrant officer personnel and limited duty, NEC, and warrant officer occupations o Third Priority: Courses for officer personnel

b. When scheduling and coordinating evaluations, the Contractor shall give priority to schools attended by more than a single branch of the Armed Forces. The Contractor shall coordinate scheduling joint Services schools with the SPM or point of contact (POC) who represents the lead Service. The Contractor shall resolve scheduling conflicts and schedule with gaining the greatest volume and efficiency in mind.

4.2.1.4 Based on the schedule, course, and occupational review submissions provided by the SPM and the Working Group, submitted by a specified and agreed upon date each year, the Contractor shall coordinate and publish a review schedule, annually. A tentative schedule shall be published with enough lead time

13 Ver: Oct 2019(1) for the Services and the Contractor to ensure review readiness with no break in service once new contract processes are established, but NLT 1 Aug in the base PoP and for each option PoP, thereafter. The Contractor shall ensure each SPM and the Working Group has 45 days to review and approve the tentative schedule and shall work within that time to resolve conflicts that the Government presents. After 45 days, the Contractor shall finalize a schedule that accommodates each Service to the greatest extent possible.

4.2.1.5 Publish the finalized Master Schedule of Reviews at least annually to training, education, and relevant Government stakeholders NLT six months before the earliest scheduled review within the period of performance.

4.2.1.6 Mitigate the risk of cancelled reviews by revising the Schedule of Reviews as situational changes may occur. Post coordinated revisions, agreed upon by the COR, Service Program Managers and Working Group stakeholders, to the Master Schedule of Reviews within two business days of the change(s). If for any reason a scheduled evaluation must be cancelled by the Government, the COR shall issue a 30-day notice to the Contractor in writing (email is acceptable).

a. Postponed reviews shall be rescheduled no earlier than 60 calendar-days later than the original visit date to allow time for team adjustments, scheduling, and logistical arrangements.

b. The Working Group shall establish Service-specific cancellation notification policies that require Service learning site locations to elevate review cancellation requests to increase accountability and transparency to training command and policy stakeholders.

4.2.1.7 The Contractor shall work to re-schedule, rather than cancel any scheduled evaluation the Contractor cannot carry out due to an unforeseeable conflict, outside the Government’s control and inform the COR, SPM, and Working Group as early as possible. Scheduling adjustments or changes due to evaluation team member non-availability or other unforeseeable conflict shall be approved in advance by the COR.

4.2.2 Training Site Readiness Evaluation Methodology

4.2.2.1 Establish, document, communicate, and execute a Pre-Evaluation Preparation Process to prepare for the review, evaluation, assessment, and award of college credit recommendations for military training courses and occupations. This process shall include a method to determine the stage of readiness nominated courses and occupations are in relation to qualifying for a virtual review. The process shall also include a level of effort determination needed for the virtual review, make recommendations for courses and occupations to be reviewed onsite, as well as determine the level of effort needed for the onsite review. The minimum lead-time for review preparation, from scheduling to execution, shall be 180 days*. The Pre-Evaluation Preparation Process shall be documented, coordinated with pertinent Service stakeholders, and delivered to the COR and the Working Group for review, comment, and approval. Per the POA&M, the Contractor shall provide the following deliverable: Pre-Evaluation Process Plan.

*Minimum lead-time requirement may be reduced, with COR approval, when both parties (Government and Contractor) state in writing that preparation can be expedited to fewer than six months without negative impact on evaluation effectiveness and outcomes.

The process must include, but is not limited to:

a. Developing, disseminating, and supporting the completion of self-study/self-assessment to determine training facility/course/occupation readiness for the virtual review process.

b. Conducting a series of trainings and/or meetings with military training facility/schoolhouse personnel to discuss the scope and intent of review as well as self-study requirements.

14 Ver: Oct 2019(1)

c. Establishing a rubric to determine the type and scope of reviews based on statement of readiness, hours of instruction, the number of disciplines/content areas, and the amount of associated labs and/or practical exercises.

d. Drafting a scoring matrix (submitted to the COR and reviewed and approved by the Working Group) based on three clearly distinguishable levels of readiness with corresponding levels of Contractor support needed by the training center to be ready for a virtual evaluation:

1) Level 1 Preparation Event (example criteria may include course materials not in digital formats, curriculum needs substantial revisions, etc.). Self-studies resulting in Level 1 scores are not eligible for virtual reviews and will be considered for remediation and/or onsite reviews by the Working Group. Remediation activities shall be treated as a project with milestones that are reported to the Working Group on an agreed upon schedule based on the score of the project. Each remediation project shall be assigned a Government and Contract project management lead that will develop and ensure execution of a POA&M and corresponding activities that result in training site/course/occupation readiness for virtual review.

2) Level 2 Preparation Event (example criteria may include curriculum is intact, course material in partial digital formats, etc.). Self-studies resulting in Level 2 scores are not eligible for virtual reviews until the identified areas remedied. Remediation activities shall be treated as short term projects with milestones that are reported to the Working Group on an agreed upon schedule based on the score of the project. Each remediation project shall be assigned a Government and Contract project management lead that will develop and ensure execution of a POA&M and corresponding activities that result in training site/course/occupation readiness for virtual review.

3) Level 3 Preparation Event (example criteria may include course materials in digital formats, curriculum needs little to no revisions, etc.). Self-studies resulting in Level 3 scores shall be included on the annual schedule for virtual reviews.

e. Conducting Staff Assistance site visits (on an as needed basis, but can be conducted virtually or via video…