Attachment_1_PWS.pdf

FA7014‐19-R-A015

PERFORMANCE WORK STATEMENT

FOR

DATA SCIENCE AND ANALYTICS COURSE

AT

PENTAGON AFB, WASHINGTON D.C.

8 APRIL 2019

Contents

SECTION I

1.0 DESCRIPTION OF SERVICES

1.1 General

1.2 Background

SECTION II

2.0 Task Descriptions

2.1 Program Management

2.2 Instructional Review and Course Maintenance

SECTION III

3.0 SERVICE SUMMARY

SECTION IV

4.0 DELIVERABLES

SECTION V

5.0 GOVERNMENT FURNISHED PROPERTY OR SERVICES

5.1. Government Furnished Property or Services

SECTION VI

6.0 GENERAL INFORMATION

6.1 Scheduling Concerns

6.2 Kickoff Meeting

6.3 Transition-Out Plan ......................................................... Error! Bookmark not defined.

6.4 Quality Control

6.5 Emergency Operations/Mission Essential Personnel

6.5 Contractor Manpower Reporting Application (CMRA)

6.6 Subcontractor Input in CMRA

6.7 Security Instructions

6.9 Employee Accountability & Turnover

6.10 Miscellaneous Paragraphs

SECTION VII ................................................................................ Error! Bookmark not defined.

APPENDIX 1 ............................................................................. Error! Bookmark not defined.

1.0 DEFINITIONS, ABBREVIATIONS, AND ACRONYMS (If applicable, add in terms that were added in Section II above. You may delete if not needed.) Error! Bookmark not defined.

2.0 LEARNING OBJECTIVES

SECTION I

1.0 DESCRIPTION OF SERVICES

1.1 General. The Director of Force Development, Headquarters Air Force, United States Air

Force (herein referred to as AF/A1D), has a new requirement to maintain and administer training for Air Force senior leaders regarding data science and analytics.

1.1.1 Scope. The Contractor must support the review, maintenance, logistics, and instruction of an executive level data science and analytics course. Successful implementation of an executive level data science and analytics course will inform Air Force leaders about how to make informed evidence based decisions using data science and analytics principles.

1.2 Background.

One of AF/A1D’s missions is to ensure senior leaders are equipped with relevant and strategic education to successfully meet the needs of the National Defense Strategy and the priorities of the Secretary and Chief of Staff of the Air Force. The Assistant Secretary of the Air Force, Manpower and Reserve Affairs (SAF/MR) directed the need for a new course because the field of data science and analytics has grown immensely. In the past decade alone, topics such as artificial intelligence (AI) and machine learning (ML) have emerged and there is a requirement for leaders to better understand these complex topics. This specific contract requirement includes the establishment of curriculum and support of an executive level data science and analytics course. As data science and analytics becomes a core competency for Air Force members, senior leaders must lead from the front and have a firm understanding of how to utilize the competency.

Specifically, they must know what barriers and capabilities exist, how to form questions which leads to more predictive and prescriptive analysis (as opposed to descriptive analysis) and how to lead teams to innovate and solve complex problems using data science and analytics.

SECTION II

2.0 Task Descriptions

The contractor shall provide all services, materials, supplies, equipment, and project supervision, as required in connection with this Performance Work Statement (PWS).

2.1 Program Management

The Contractor shall provide the technical and functional expertise required for effectively managing the technical, schedule, and cost parameters associated with the requirements described in the scope above.

2.2 Instructional Review and Course Maintenance

The Contractor is also responsible for quarterly instructional review, logistics, administration, and maintenance of all courses. The Contractor shall accomplish this maintenance effort through timely updates to course materials including lesson plans, instructional aides, reference books and practical exercises. AF/A1D and AFIT/LS will appoint at least two subject matter experts to coordinate with the Contractor when necessary changes are needed. The Contractor shall be poised to make changes within four weeks of receiving notification of a specific process change.

2.3 Surveys and Assessments

The Contractor must administer a pre- and post-survey to course attendees. The results of the surveys must be shared with AF/A1D and AFIT/LS within 5 business days of the course. If requested, the Contractor must be able to develop pre- and post-assessments for attendees.

2.4 Pilot Details

The Contractor shall provide three pilot courses in Fiscal Year 2019 (FY19) as a test. AF/A1D and AFIT/LS will assess at the end of the three pilot courses whether or not to continue utilizing the Contractor teach and administer the course. The three pilot courses will be administered during the following timeframe:

Class Start Pilot 1: Must have a class offering that starts in the June 2019 timeframe

Class Start Pilot 2: Must have a class offering that starts in the July 2019 timeframe

Class Start Pilot 3: Must have a class offering that starts in August 2019 timeframe

Period of Performance:

- Pilot Period: June-August 2019, with option to extend 4 years

- Target Start Date: 1 June 19

After each pilot the curriculum must be reviewed and modified as required.

2.5. Executive Course

The Contractor must be prepared to conduct at least nine Executive Courses annually. Each course needs to have the flexibility to be tailored, as appropriate to give senior Air Force leaders the background they need to function effectively in their duties. Attendees will normally include

General Officers and Senior Executive Service civilians, as well as identified Colonels and GS-

15 employees (or equivalents). The course will typically be administered in groups of no more than 30 individuals, with each course tailored to the experiences of the specific attendees (the

Contractor shall be responsible for ascertaining the level of experience of the senior ranking attendee and tailoring the course accordingly). Extensive research is required to ensure attendees understand what software platforms the Air Force permits based on security guidelines and authority to operate (ATO) instructions. Ultimately, senior leaders will effectively understand and apply the skills needed to build an evidenced based decision capability.

2.5.1 General Information

Outline should give an executive overview as to the value and benefits of leaders using evidence informed decision making. Must be able to discuss what questions leaders need to ask, what obstacles they currently face in the DoD, and how data can inform decisions, support the

National Defense Strategy and optimize the warfighter.

2.5.2 Purpose of course

DoD Senior Leaders face challenges synthesizing vast amounts of data into patterns, trends and models to solve systemic problems and make evidence informed business decisions. Despite the advances in technology and access to data and tools, the Air Force faces challenges with growing and developing advanced capabilities which integrate models and algorithms to shift from descriptive to predictive and prescriptive decision making.

2.5.3 Key Partnerships for Development of Course

The Contractor shall ensure the availability of a team of broadly experienced experts who can respond to the full range of tasks. The Contractor shall be prepared to work independently or in continual contact with AF/A1D and the Air Force Institute of Technology, School of Logistics and Systems (AFIT/LS). AFIT/LS expertise and knowledge pertaining to data science and analytics. In addition, subject matter experts from the Chief Data Office (SAF/CO), Headquarters Air Staff from both the Secretariat offices and Air Staff offices including, but not limited to, the Directorate of Manpower, Personnel and Services (AF/A1), Directorate of

Intelligence, Surveillance, Reconnaissance, and Cyber Effects Operations (AF/A2/6), Directorate of Operations, Plans and Requirements (AF/A3), Directorate of Logistics, Installations and

Mission Support (AF/A4), Directorate of Strategy, Integration and Requirements (AF/A5), Directorate of Plans and Programs (AF/A8), Directorate of Studies, Analyses and Assessments

(AF/A9), Directorate of Financial Management and Comptroller (SAF/FM), Directorate of

Information Dominance and Chief Information Officer (SAF/CIO), the United States Air Force

Academy, Air University, Air Education and Training Command, and other stakeholders regarding data science and analytics. While AFIT/LS and AF/A1D will provide primary material for course updates, the successful competitor for this work must be prepared to coordinate with other organizations.

2.5.4 Design of Course

The contractor is responsible for the specific design process for the course. The curriculum must align specific learning outcomes, allow for discovery of context, insights, the DoD data environment, current issues, create a learning experience with relevant DoD content (i.e.

concepts, case studies, activities, discussions, etc.) and have a debrief after the courses for future development and courses. Must include case studies from industry, academia and government at a minimum. The course must have the capability to grow and be refined. The framework must addresses the various elements of data science and analytics. Learning objectives must be addressed and can be found in the annex. The following elements must be included:

- Opportunities and Threats

- Skills and Techniques

- Data (Gathering, Engineering, Manipulating, Accessing)

- Change and expectation management

- Emerging Technologies and Tools

- Ethics, Values and Culture of Data Science and Analytics

- Environment

2.5.5 LEARNING OBJECTIVES

Basic Learning Objectives MUST Include:

1. Gain an understanding of applications/opportunities and barriers/threats of using data in evidence informed policy and business decision making by DOD senior leaders to meet the objectives of the National Defense Strategy.

2. Identify how leaders can lead teams to master synthesizing, trend and pattern analysis, and model building so they will covert data into knowledge and provide recommended courses of action

3. Understand how to build a data environment, data governance, data team, and knowledge capture/management capability

4. Identify the factors relating to normalizing data, authoritative sources, integrating desperate data, pattern analysis, model and algorithm building

5. Explore how to incorporate and use analysis in business decision making, the value of testing for reliability/interpretation error reconciling (causation)

6. Understand the need to manage change, resistance and expectations

7. Understanding the art of validating, disclosing, justifying and uses in public and with media

(including testimony)

8. Awareness of emerging technologies/tools and how to pre-position organizations for future evolution

9. Concern of ethics, values, culture and cyber/security of data environments

2.5.6 Scheduling

Course scheduling shall be done either directly between the Contractor and the attendee’s office, or through coordination with AF/A1D delegated representatives. Decisions about how to tailor the course to a particular recipient shall be made between the Contractor, AF/A1D and AFIT/LS representatives, and the senior ranking attendee’s representative. Scheduling will typically occur during the core duty hours of 0800-1600. The course itself will also occur during 0800-1600.

2.5.7 Proctors and Instructors

The Contractor shall identify a Proctor for the course who will set up the training room for the course presentation, ensure all handouts and training materials are current and on-hand, ensure all required audio-visual equipment is available and functioning, and respond to unique requirements dictated by the situation. The Proctor shall be readily available throughout the length of the course to perform facilitator functions, including but not limited to administrative details, schedule adjustments, respond to unforeseen problems and keep the course on schedule and on topic during its execution. The Proctor shall also administer the student critiques/surveys, conduct post-course reviews with the instructors, make all required course changes (lesson plans and presentation materials), and provide updated materials to the instructors in advance. The proctor can be the instructor as well if desired. The instructor must be fully qualified to instruct on data science and analytics and have superior speaking skills to present at an executive level.

Instructor must deliver in seminar style with interactive discussions and case studies. The lead instructor(s) must have an up to date resume and provide if requested.

2.5.8 Audit of Course

As designated by the COR, professors, subject matter experts and Air Force individuals may audit the course at any time. Individuals will not be disruptive or interactive in the course and will only observe.

2.6 Make-Up Courses

At the discretion of AF/A1D representatives, the Contractor shall provide make-up sessions if regularly scheduled courses get cancelled due to inclement weather, government shutdowns, or other circumstances. However, occurrences will be kept at a minimum.

2.7 Handouts and other Instructional Material

The Contractor understands that once they have developed the material for the courses, all materials and content within shall be considered government property and shall not be repurposed for non-Air Force customers. AFIT/LS will provide specific formats for a reference book the Contractor must maintain which will have a summary of all the reference material. The reference book at a minimum will have instructor guides, slides, material briefed, lesson plans, instructional aides and handouts. The reference book will also have student registration and attendance status, historical records and course critiques. Read aheads must be provided two weeks in advance for the attendees to review prior to the course. Each attendee will then receive a bound paper copy of the most current material at the beginning of the course. Additional copies will be made available by the Government electronically on an AF/A1D or AFIT/LS managed Shared Drive, web or SharePoint page.

2.8 Facility Requirements

Contractor must provide location for up to 30 people at the Contractor’s campus/facility/location for each session. Contractor must have the flexibility to teach in the Pentagon or other government facilities if necessary or requested by AF/A1D. Classes must be offered in the D.C.

Metro area. Location will be within a 15 minute walk from public transportation and/or the

Pentagon. Primary location is at the Contractors facility which allows for an interactive course experience. Environment must be clean, professional and quiet. Within first month of contract, and every 6 months, tour of facility is required.

SECTION III

3.0 SERVICE SUMMARY

The Contractor service requirements are summarized into performance objectives that relate directly to mission essential items The performance threshold briefly describes the minimum acceptable levels of service required for each requirement and will be assessed on an

“Acceptable” or “Unacceptable” basis. These thresholds are critical to mission success.

Performance

Objective PWS Paragraph Performance Threshold

Method of

Surveillance

Program

Management

2.1.1 Monthly meetings must be

limited to a maximum of 90 minutes. Must address the topics on the agenda and have solutions 80% of the time.

Periodic

Surveillance

Instructional

Review and

2.2 Contractor must be able to make

extensive changes to content within 4 weeks when new

Periodic

Surveillance

Course

Management material or changes is required for the courses. Also a quarterly instructional review of logistics, administration, and maintenance of the course is required.

Surveys, Assessments and Make Up

Courses

2.3, 2.7 Results from the surveys must be relayed in 5 business days. If changes needed, Contractor must be able to make adjustments. Student responses must meet expectations 85% of the time (based on a like scale questionnaire)

Executive

Course

2.5 Courses must be executed on

time and IAW with scheduling, material, proctor, and instructor.

The material must be tailored which requires extensive interviews and research to understand the DoD and Air

Force.

Periodic

Surveillance

Handouts and

Other

Instructional

Material

2.7 Material must be accessible in

many forms (electronic and hard copy at a minimum) and need to be relevant to the Air Force and strategic in nature (executive level.) Changes to make info more tailored to the Air Force must be given to AF/A1D and

AFIT/LS as Air Force property.

Periodic

Surveillance

Audit of

Course

2.5.7 Must allow for audits of the

course 100% of the time.

Facility

Requirements

2.8 Facility must be clean and

promote and allow for learning to occur.

SECTION IV

4.0 DELIVERABLES

The Contractor shall provide deliverable(s) in a format mutually agreed upon by the Government and the Contractor.

The following enumerated deliverables are not expected to change. Due Date intervals are not expected to change but actual dates may need to be revised depending on actual contract start date.

DELIVERABLE PWS

PARA

DUE DATE DELIVERY

Monthly Progress

Report

2.1.1 Must be conducted NLT

14th day of month

Report must be emailed to

AF/A1D and AFIT/LS and must be available for phone conversation if questions need to be addressed

Instructional Review and Course

Maintenance

2.2 When requested by

AF/A1D or AFIT/LS, contractor has four weeks to make changes and send to AFIT/LS

By email to AFIT/LS in

Microsoft Word format

Surveys and

Assessments

2.3 Pre and Post Surveys and

any requested assesments are due within 5 business days of the course completion

By email to the COR, AFIT/LS and AF/A1D in

Microsoft Word format

Executive Course

Schedule

2.5 Must be established within

first three weeks of contract, updated as needed

By email to the COR, AFIT/LS and AF/A1D in

Microsoft Word format

Executive Course

Material

2.5 Must be provided NLT 2

weeks in advance to attendees

By email to the attendees with and executive summary attached.

Make-Up 2.7 Contractor will have 2 weeks to schedule the requested course.

Confirmed schedule will be sent to COR, AFIT/LS, AF/A1D and attendees.

Handouts/Instructional

Material/Case Studies

2.8 Reference book must be

established 1 month after contract, and updated monthly.

Reference book sent by email to COR, AFIT/LS, AF/A1D in Microsoft

Word format.

Facility Tour 2.8 Within first month of contract, and every 6 months, tour of facility is required.

COR or delegated individual will visit facility with Contractor.

Resumes of Lead

Instructors

2.5.6 Upon Request By email to COR in MS

Word or PDF

Kick Off Meeting

Agenda and Execution

6.2.2

6.2.3

Agenda within first 3 business day and meeting within 10 days of awarded contract

Agenda must emailed.

Meeting must be in person, either at Pentagon or facility of Contractor pending availability

DELIVERABLE PWS

PARA

DUE DATE DELIVERY

Quality Control Plan 6.4.2 Within the first 30 days Emailed to COR and requested individuals.

SECTION V

5.0 GOVERNMENT FURNISHED PROPERTY OR SERVICES

5.1. Government Furnished Property or Services. The Government will not furnishe property or services for this requirement.

SECTION VI

6.0 GENERAL INFORMATION

6.1 Scheduling Concerns.

6.1.1 Core Duty Hours and Base Shutdown/Inclement Weather. Core duty hours are 0800 -

1600, Monday through Friday (excluding federal holidays). The Contractor shall follow guidance of the installation containing their place of performance to determine reporting schedules whether due to a base closure or inclement weather. The website for guidance regarding status of performance for work to be performed in the National Capital Region NCR use http://www.opm.gov/status/.

6.2 Kickoff Meeting

6.2.1 Within ten (10) business days of award, the Contractor shall conduct an orientation briefing for the Government. The intent of the briefing is to initiate the communication process between the Government and Contractor by introducing key task participants and explaining their roles, reviewing communication ground rules, and assuring a common understanding of subtask requirements and objectives.

6.2.2 The Contractor shall schedule and coordinate a Project kick-off Meeting no later than

(NLT) ten (10) calendar days after contract award at the location approved by the Government.

The meeting will provide an introduction between the contractor personnel and Government personnel who will be involved with the contract. The meeting will provide the opportunity to discuss technical, management, and logistic issues; travel authorization; and reporting procedures. At a minimum, the attendees shall include key Contractor personnel, key

Government representatives, and the COR. The Contractor shall provide a Kick-Off Meeting

Agenda that will include, but not be limited to, the following.

• Introduction of personnel

• Overview of project tasks

• Review of organization (complexity)

• Schedule (shows major tasks, milestones, and deliverables; planned and actual start and completion dates for each) http://www.opm.gov/status/

• Communication Plan/lines of communication overview (between both contractor and

Government)

• Travel notification and processes

• Government-furnished information (GFI)

• Security requirements (Building access, badges, Common Access Cards (CAC))

• Analytical Support Status Accreditation (ASSA) Documentation

• Invoice procedures

• Monthly meeting dates

• Reporting Requirements, e.g. Monthly Status Report (MSR)

• POCs

• Roles and Responsibilities

• Overview of incoming Transition Plan to include process, timeframes, and status

• Prioritization of contractor activities

• Any initial deliverables

• Other logistic issues

• Quality Control Plan (QCP)

• Sensitivity and protection of information

• Additional issues of concern (Leave/back-up support)

6.2.3 The Contractor shall provide a draft copy of the agenda within the first three business days for review and approval by the COR prior to finalizing. The Government will provide the

Contractor with the number of participants for the kick-off meeting and the Contractor shall provide sufficient copies of the presentation for all present.

6.4 Quality Control.

6.4.1. Quality Assurance. The Government shall rely on the Contractors’ existing quality assurance system as the method to ensure that the requirements of the contract and performance thresholds are met; however, the Government reserves the right to monitor and evaluate the quality of services provided and compliance with the contract terms and conditions at any time.

6.4.2. Quality Control Plan (QCP). The Contractor will develop and maintain an effective quality control program within the first 30 days to ensure services are performed IAW this PWS, applicable laws and regulations and IAW best commercial practices. The Contractor shall develop and implement procedures to identify, prevent, and ensure non-recurrence of defective services with special emphasis placed on those services listed in this PWS. The Contractor’s quality control program is the means by which it assures itself that the work complies with the requirement of the contract.

6.5 Emergency Operations/Mission Essential Personnel.

6.5.1. Continuation of Essential Contractor Services During Crisis. All services in this PWS

HAVE NOT been defined or designated as essential services for performance during crisis IAW

DFARs 252.237-7023, “Continuation of Essential Contractor Services.”

6.5 Contractor Manpower Reporting Application (CMRA): The Contractor shall report ALL labor hours (including subcontractor labor hours) required for performance of services provided under this task order via the secure Contractor Manpower Reporting Application (CMRA) data collection site. The Contractor is required to completely fill in all required data fields at:

http://www.ecmra.mil. Reporting inputs shall be for the labor executed during the period of performance for each Government fiscal year (FY), which runs 1 October through 30 September.

While inputs may be reported anytime during the FY, all data shall be reported not later than 31

October of each calendar year. The Contractor may direct questions to the CMRA help desk.

6.6 Subcontractor Input in CMRA: Prime Contractors are responsible to ensure all subcontractor data is reported. Subcontractors will not be able to enter any data into CMRA, but will enter their information into a Bulk Loader spread sheet available from the AF CMRA helpdesk. Subcontractor shall fill in columns A-C then return it to the AF CMRA helpdesk after it's completed and a technician team will enter the information into CMRA.

6.7 Security Instructions

6.7.1 Physical Security. The Contractor ensure that government personnel are utilizing a classroom environment in a safe location.

6.7.2 Access Control. The Contractor shall establish and implement methods of ensuring that no building access instruments issued by the Government are lost, misplaced or used by unauthorized persons. Access codes shall not be shared with any person(s) outside the organization. The Contractor shall control access to all Government provided lock combinations to preclude unauthorized entry. The Contractor is not authorized to record lock combinations without written approval by the Government COR. Records with written combinations to authorized secure storage containers, secure storage rooms, or certified vaults, shall be marked and safeguarded at the highest classification level as the classified material maintained inside the approved containers.

6.7.3 Pass and Identification Items. The Contractor shall ensure the pass and identification items required for contract performance are obtained for employees and non-government owned vehicles.

6.7.4 Retrieving Identification Media. The Contractor shall retrieve all identification media, including vehicle passes, from employees who depart for any reason before the contract expires.

6.7.5 Weapons, Firearms, and Ammunition. Contractor employees are prohibited from possessing weapons, firearms, or ammunition, on themselves or within their Contractor-owned vehicle or privately-owned vehicle while at the Pentagon or any other military facility.

6.7.6 Contractor Identification. When conversing with Government personnel during business meetings, over the telephone or via electronic mail, Contractor personnel shall identify themselves as a contractor to avoid situations arising where sensitive topics might be better discussed solely between Government employees. Contractors shall identify themselves on any attendance sheet or any coordination documents they may review. Electronic mail signature blocks shall identify their company affiliation.

6.8 LOCAL TRAVEL: Local travel to attend meetings or events may be required at no cost to the Government. Travel within the National Capital Region commuting vicinity is considered a cost of doing business and shall not be separately reimbursed.

6.9 Employee Accountability & Turnover

6.9.1 Staffing of Contractor Personnel. Contractor shall provide a stable workforce, throughout the duration of the contract, by maintaining a 75% or less employee turnover rate. No positions shall remain vacant for more than 21 working days. The government shall not be billed for positions left vacant over 21 working days unless the vacancy is due to government delay and otherwise approved by the CO.

6.10 Miscellaneous Paragraphs.

6.10.1 Privacy Act. Work on this contract may require that personnel have access to information protected by the Privacy Act. Contractor personnel shall adhere to the Privacy Act, Title 5 of the U.S. Code, Section 552a and applicable agency rules and regulations when handling such information.

6.10.2 Safety Concerns. The Contractor is solely responsible for compliance with OSHA standards for the protection of their employees. The Government is not responsible for ensuring that Contractors comply with “personal” safety requirements that do not present the potential to damage Government resources.

6.10.3 Project Policy. The contractor shall comply with all industry standards. All work shall be done in accordance with all federal, local, and state laws and regulations.

6.10.4 Inherently Governmental Functions. The Contractor shall not perform inherently

Governmental functions as defined in the Federal Acquisition Regulation (FAR) Subpart 7.5 in relation to this PWS.

6.10.5 Ethics. The Contractor shall not employ any person who is an employee of the US

Government if employing that person would create a conflict of interest. Additionally, the

Contractor shall not employ any person who is an employee of the Department of the Air Force, either military or civilian, unless such person seeks and receives approval according to DoDD

5500-7, Joint Ethics Regulation.

6.10.6 Professional Appearance of Work Space. The Contractor shall keep work space areas neat and orderly and avoid conditions leading to safety violations.

6.10.7 Non-Personal Services. The Government shall not supervise or task Contractor employees in any manner that generates actions of the nature of personal services, or that creates the perception of personal services. It is the responsibility of the Contractor to manage its employees directly and to guard against any actions that are of the nature of personal services, or give the perception of personal services to the Government or to Government personnel. If the

Contractor feels that any actions constitute, or are perceived to constitute personal services, it is the Contractor’s responsibility to notify the CO immediately. Non-personal contractor services shall not be used to perform work of a policy/decision making or management nature, i.e.,