Draft PWS - 47QFLA24K0018.pdf

PWS Non-SBIR Contract/Order v1.1

PERFORMANCE WORK STATEMENT

A procurement by the

U.S. General Services Administration, APEX 4 Sector 1 on behalf of

CLIENT AGENCY:

Air Force Research Laboratory

CLIENT PROGRAM:

AFWERX (AFRL/RG)

PROJECT TITLE:

AFVentures SBIR Evaluations Tool

Dated:

06 September, 2024

Post Award Document Revision History

Revision Number

Description of Changes Requested By Date

Contents

1 INTRODUCTION

1.1 Organization

1.2 Objective

1.3 Background and Scope Overview

1.4 Applicable Documents

2 REQUIREMENTS AND TASKS

2.1 General Performance Requirements

2.1.1 Resources

2.1.2 General Communication

2.1.3 Security Requirements

2.1.3 Business Relations

2.1.4 Contractor Response

2.1.5 Professional Appearance

2.1.6 Collaboration

2.2 General Meeting Requirements

2.2.1 Initial Business/Kickoff Meeting

2.2.2 Ad hoc Technical / Work Status

2.2.3 Contract Administration Meetings

2.3 Program/Project Management

2.3.1 General Program/Project Management Requirements

2.3.2 RESERVED

2.3.3 Phase-Out Support

2.4 AF SBIR/STTR Tool Capabilities

2.4.1 Proposal Evaluation Capability

2.4.1.1 Maintain multiple simultaneous workflows and business rules

2.4.2 Automated Data Migration DSIP

2.4.3 Proposal Evaluations Rubric Scoring (Open Topic only)

2.4.4 Proposal Evaluation Reassignments (Open Topic only)

2.4.5 AI Evaluations (Open Topic Only)

2.4.6 Contracting Functions Interface Capability

2.4.7 Award/ Contract Modifications Capability

2.4.8 Due Diligence Reporting and Tracking

2.4.9 Provide an efficient and effective user experience (UX)

2.5 Support Personnel

2.6 Service Level Agreement (SLA)

2.7 Service Ticket Management System

3 QUALITY

3.1 Contractor Quality Management

3.2 Performance Based Requirements - Performance Requirements Summary (PRS)

3.3 Government Quality Assurance

4. DELIVERABLES

4.1 Contractor Submission

4.2 Government Review

4.3 Government Delays in Reviewing Deliverables or Furnishing Items

4.4 Deliverable Table

4.5 Monthly Invoice

4.6 Monthly Status Report

4.7 Trip Report

4.8 Phase-In Plan

4.9 Phase-Out Plan

5 PERFORMANCE

5.1 Period of Performance

5.2 Place of Performance

5.3 Applicability of Telework

5.4 Performance at Contractor Facilities

5.5 Unplanned Government Facility Closures

6. Travel

6.1 Contractor Travel Requests

6.2 Travel Reimbursement

7 Hours of Work

7.1 Standard Duty Hours Support

7.1.2 Unscheduled (work hour category E)

8. Limitations on Contractor Performance

9 PERSONNEL

9.1 General Requirements

9.2 Specific Expertise and Experience

9.3 Training

9.3.1 Contractor Staff Training

9.3.2 Mandatory Government Training

9.4 RESERVED

9.5 Personnel Retention and Recruitment

10. GOVERNMENT FURNISHED PROPERTY/INFORMATION/ACCESS

10.1 General

10.2 RESERVED

10.3 RESERVED

10.4 Use of Government Property

10.4.1 Electronic Mail (E-mail)

10.4.3 Canvassing, Soliciting, or Selling

10.4.4 Security Violations Using Government Equipment

11 SECURITY

11.1 Non-Disclosure Agreements

11.2 Reserved

11.3 Reserved

11.4 Reserved

12. SPECIAL INSTRUCTIONS

12.1 Contractor Performance Assessment Reporting System (CPARS) Assessment

12.2 Personal Service

12.3 Privacy Act

12.4 Section 508

12.5 Close-Out Procedures

12.6 Other Direct Costs (ODCs)

13 ATTACHMENTS

1 INTRODUCTION

1.1 Organization

AFWERX (AFRL/RG) is part of the Air Force Research Laboratory (AFRL) and is responsible for executing the Department of the Air Force (DAF) Small Business Innovation Research and Small Business Technology Transfer (SBIR/STTR) mission. SBIR/STTR encourages domestic small businesses to engage in Federal Research/Research and Development (R/R&D) through a competitive awards-based program. The Program utilizes a cloud-based solution that provides an integrated DAF SBIR/STTR platform for all phases of the awards process. This solution is required to allow small businesses to create, submit, and track proposals, as well as government representatives to evaluate submissions and provide contracting support to those that are awarded funding.

1.2 Objective

The contractor’s Software-as-a-Service (SaaS) solution shall be an automated, online system of integrated capabilities to support the complex DAF SBIR/STTR process. The system shall be capable of reporting and collecting data for Department of Defense (DoD) and Small Business Administration (SBA) data reporting; this includes the incorporation of historical data for longitudinal reporting.

1.3 Background and Scope Overview

The initial Evaluations tool platform was implemented in 2017 and has undergone significant investment over the years to align with DAF contracting workflows to keep pace with evolving requirements of the SBIR/STTR program. The platform has grown from evaluating 6,000 proposals with 1 workflow in 2019 to upwards of 10,000 proposals with eight workflows in 2024. The platform is delivered as a Service- Oriented Architecture (SOA) platform as an automated, online system of integrated capabilities to support the complex DAF SBIR/STTR process delivered via a modern Graphical User Interface (GUI) interface. Additionally, exclusive software and data on legacy SBIR/STTR topics and programs currently reside within its database. The existing system also includes the capability of reporting and collecting data for DoD and SBA data incorporating historical data for longitudinal reporting. The Evaluations tool is delivered in accordance with the applicable Federal, DoD, and DAF directives, instructions, policies, and regulations as detailed in section 1.4, Applicable Documents. Detailed requirements are provided in Section 2.4

1.4 Applicable Documents

The documents (i.e., certifications, specifications, standards, policies, and procedures), versions current at time of contract/order award, identified in the list below are incorporated with the same force and effect as if provided in full text.

Succeeding revisions may be substituted or incorporated as required with full notice and disclosure to the contractor. The Government will provide access to available documents and technical information as required and upon contractor request, if not available via a hyperlink within this PWS.

▪ CJCS M 6510.01B. 10 July 2012. Cyber Incident Handling Program.

▪ CNSSP 15. 20 October 2016. Use of Public Standards for Secure Information Sharing.

▪ DoD 8570.01-M. 19 December 2005. Information Assurance Workforce Improvement Program.

▪ DoDD 8140.01. 05 October 2020. Cyberspace Workforce Management.

▪ DoDD 8523.01. 22 April 2008. Communications Security (COMSEC).

▪ DoDI 5200.39. 28 May 2015. Critical Program Information (CPI) Identification and Protection Within Research, Development, Test, and Evaluation

(RDT&E).

▪ DoDI 8500.01. 14 March 2014. Cybersecurity.

▪ DoDI 8510.01. 19 July 2022. Risk Management Framework for DoD Systems.

▪ DoDI 8551.01. 31 May 2023. Ports, Protocols, and Services Management.

▪ FIPS 140-3. 22 March 2019. Security Requirements for Cryptographic Modules.

▪ FIPS 199. 01 February 2004. Standards for Security Categorization of Federal Information and Information Systems.

▪ FIPS 201-3. 24 January 2022. Personal Identity Verification (PIV) of Federal Employees and Contractors.

▪ NIST. n.d. Cybersecurity Framework. https://www.nist.gov/cyberframework

▪ NIST SP 800-122. 06 April 2010. Guide to Protecting the Confidentiality of Personally Identifiable Information (PII).

▪ NIST SP 800-16. 01 April 1998. Standards for Security Categorization of Federal Information and Information Systems.

▪ NIST SP 800-171. April 2015. Protecting Controlled Unclassified Information in Nonfederal Information Systems and Organizations.

▪ NIST SP 800-47 Rev. 1. 20 July 2021. Managing the Security of Information Exchanges.

▪ NIST SP 800-61 Rev. 2. 06 August 2012. Computer Security Incident Handling Guide.

▪ NIST SP 800-63-3. 02 March 2020. Digital Identity Guidelines.

▪ NIST SP 800-73-4. 12 February 2016. Interfaces for Personal Identity Verification.

▪ SBA Policy Directive. 03 May 2023. Small Business Innovation Research (SBIR) and Small Business Technology Transfer (STTR) Program.

https://www.sbir.gov/sites/default/files/SBA%20SBIR_STTR_POLICY_DIREC TIVE_May2023.pdf

▪ SBA Policy Directive Appendices. 03 May 2023. Small Business Innovation Research (SBIR) and Small Business Technology Transfer (STTR) Program.

https://www.nist.gov/cyberframework https://www.sbir.gov/sites/default/files/SBA%20SBIR_STTR_POLICY_DIRECTIVE_May2023.pdf https://www.sbir.gov/sites/default/files/SBA%20SBIR_STTR_POLICY_DIRECTIVE_May2023.pdf https://www.sbir.gov/sites/default/files/SBA%20SBIR_STTR_FINAL_APPEND ICES_MAY2023.pdf

2 REQUIREMENTS AND TASKS

2.1 General Performance Requirements

2.1.1 Resources

Unless otherwise stipulated, the contractor shall furnish or provide all personnel, personnel management and supervision, all related internal supporting business functions (including background and “overhead” personnel), materials, supplies, equipment, and facilities to perform the full range of services required by this

PWS.

2.1.2 General Communication

Regular and direct contractor interface with the Contracting Officer (CO); the Contracting Officer’s Representative (COR), and other identified Government representatives is mandatory. The contractor shall not contact nor take direction from unauthorized Government representatives, under any circumstances.

2.1.3 Security Requirements

All contractor personnel shall be required to be capable of obtaining a Tier 1 Background Clearance and obtaining a Controlled Access Card (CAC) to access government resources using a .mil email address. Additionally, personnel accessing AFWERX IT network resources shall be issued with government furnished equipment including required auxiliary hardware (peripherals), software and network access to perform the requirements of the PWS. Any contractor violating Government security policies, guidelines, procedures, or requirements while using Government equipment or while accessing the Government network may, without notice, have their computer and network access terminated. The CO/COR will notify the contractor of the security violation and request immediate removal of the contractor employee.

For those times that require contractors attend a government site, the contractor shall comply with all visitor identification requirements when visiting the Government site. When conversing with Government personnel during business meetings, over the telephone or via electronic mail, contractor personnel shall identify themselves as contractors to avoid situations arising where sensitive topics might be better discussed solely between Government employees.

Contractors shall identify themselves on any attendance sheet or any coordination documents they may review. Electronic mail signature blocks shall identify their company affiliation.

https://www.sbir.gov/sites/default/files/SBA%20SBIR_STTR_FINAL_APPENDICES_MAY2023.pdf https://www.sbir.gov/sites/default/files/SBA%20SBIR_STTR_FINAL_APPENDICES_MAY2023.pdf

2.1.3 Business Relations

A primary element of project success is the business relationship between the contractor and Government representatives. Within this context the Government will monitor the contractor’s contribution to business relations and provide feedback when required. The contractor shall make every effort to establish and maintain clear and constant communication channels with the Government primaries (CO, COR, and other identified Government representatives) of the contract/order for the purpose of:

▪ Promptly identifying PWS and/or business relationship issues of controversy and the bilateral development and implementation of corrective action plans.

▪ Ensuring the professional and ethical behavior of contractor personnel.

▪ Maintaining effective and responsive subcontractor management (if applicable).

▪ Ensuring the contractor support team is fully aware and engaged in strengthening the interdependency that exists between the contractors and their government counterparts.

▪ Facilitating contractor–Government collaboration for continuous improvement in performing PWS tasks, reducing risks and costs, and meeting the mission needs.

2.1.4 Contractor Response

The contractor shall ensure prompt response to Government inquiries, requests for information or requests for contractual actions. The response shall be provided within two business days of the inquiry/request.

2.1.5 Professional Appearance

Contractor personnel shall present a neat and professional appearance appropriate to the work being performed at all times when interacting with Government representatives, working in Government facilities, or representing the Government as an authorized government contractor at meetings or before third parties.

2.1.6 Collaboration

The contractor shall utilize the Collaboration tool (to include all applicable collaboration types) within the GSA’s web-based procurement system (i.e., Assisted Services Shared Information System (ASSIST)) for submission of all documentation as required under the awarded contract/order. Data that requires Impact Level 4 (IL4) shall not be submitted via ASSIST and submission shall be coordinated with the client Technical Point of Contact (TPOC), GSA COR and other identified Government representatives as applicable.

2.2 General Meeting Requirements

The contractor shall participate in the following meetings. Nothing discussed in any meetings or discussions between the Government and the contractor shall be construed as adding, deleting, or modifying the contractual agreement without written authorization from the CO.

2.2.1 Initial Business/Kickoff Meeting

Within 15 business days following the contract/order award (or other time mutually agreed between the parties), the contractor shall meet with the GSA CO, GSA COR, and other identified Government representatives to ensure a common understanding of the requirements, goals, expectations, end products, and objectives of the contract/order. The contractor shall discuss the overall understanding of the project and review the background information and materials provided by the Government. Discussions will also include the scope of work, deliverables to be produced, how the efforts will be organized, and project conducted; assumptions made/expected end results. A concerted effort shall be made to gain a thorough understanding of the Government expectations.

However, nothing discussed in this or in any subsequent meetings or discussions between the Government and the contractor shall be construed as adding, deleting, or modifying any contract/order requirements, including deliverable specifications and due dates. The contractor shall provide an initial team roster and shall address the status of any issues that will affect contractor start-up/ramp-up toward achieving full service/support capability. The contractor will be responsible for taking and publishing minutes of this meeting.

2.2.2 Ad hoc Technical / Work Status

The contractor shall, if requested by the Government, participate in monthly status meetings or ad hoc technical meetings at a mutually agreeable time and place to discuss tasking, work progress, technical problems, performance issues, or other technical matters. During these meetings, the contractor shall at least provide accomplishments, problems and issues, and planned actions. The contractor shall take minutes of these meetings and include them in the Monthly Status Report. These meetings will occur at a time and place mutually agreed upon by the parties.

2.2.3 Contract Administration Meetings

The CO may require the authorized contractor representative to meet or participate in a teleconference with authorized Government personnel as often as deemed necessary to discuss performance or administrative issues. The contractor may also request a meeting with the CO when deemed necessary.

The content of meetings shall be documented in writing, as agreed between the parties. Minutes shall be approved by both parties and shall be included in the Government contract file.

2.3 Program/Project Management

2.3.1 General Program/Project Management Requirements

The contractor shall be solely responsible for managing the work performed in the execution of the contract/order. This includes the responsibility to:

▪ Assign appropriate resources to each task.

▪ Maintain clear organizational lines of authority.

▪ Ensure effective task management and administration, following the requirements set forth in the PWS.

▪ Maintain the personnel, organization, and administrative control necessary to ensure that the work delivered meets the specification requirements.

▪ Establish and use proven policies, processes, analyses, and best practices.

▪ The contractor shall be fully responsible for management, control, and performance of any subcontractor used in support of the contract/order.

Use of a subcontractor on the contractor’s team shall not relieve the prime contractor of responsibility nor accountability in the execution of the contract/order.

Additionally, the contractor shall:

▪ Bring problems or potential problems affecting performance to the attention of the CO as soon as possible.

▪ Notify the COR, CO, and other identified Government representatives immediately of any projected, anticipated, or known delays that may impede contactor performance.

▪ When requested, deliver written reports to the CO to memorialize all verbal reports.

▪ Provide, in writing, the results of all meetings in which proposals are put forth that have the potential for affecting and/or changing contract agreements, requirements or conditions, and these shall be brought to the attention of the CO.

2.3.2 RESERVED

2.3.3 Phase-Out Support

The contractor may need to provide phase-out support in accordance with the phase-out plan as required per PWS paragraph 4.8.

2.4 AF SBIR/STTR Tool Capabilities

The contractor’s Software as a Service (SaaS) solution shall be an automated, online system of integrated capabilities to support the complex DAF SBIR/STTR process, as defined in this section and be capable of reporting and collecting data for DoD and SBA data reporting as defined below; this includes the incorporation of historical data for longitudinal reporting.

To align with Department of Defense (DoD) security requirements, only DISA-authorized SaaS solutions that adhere to the DoD Cloud Connection Process

Guide and are FedRAMP authorized will be considered for the AF SBIR/STTR environment. FedRAMP compliance is crucial as it accelerates cloud adoption by establishing transparent standards and processes for security authorizations across government agencies and standardizes security by ensuring cloud services meet the stringent security requirements outlined in FISMA, OMB

Circular A-130, and FedRAMP policy. The FedRAMP Authorization Act, part of the FY23 National Defense Authorization Act (NDAA), further solidifies

FedRAMP as the authoritative standard for cloud security.

The Contractor’s proposed solution shall provide a representative developmental environment hosted in a Government approved cloud environment that segregates development, troubleshooting, and testing from the Production configuration. Functional and non-functional testing shall be executed by quality assurance engineers to identify and resolve any potential issues or bugs in the development environment before deployment to a production environment to show a reduction in manual touchpoint.

The contractor shall utilize, to the maximum extent possible, any DoD approved monitoring tools established in a production environment to ensure stability of the system. If a new tool is developed, the contractor shall demonstrate to the

Government that the stability and reliability of the solution is no worse than the previous application.

The Contractor shall develop testing plans that will outline testing objectives and measure of success for those plans. Such plans will cover both usability testing and end-to-end testing.

The following details the Evaluations Tool capabilities required.

• Proposal Evaluation Capability

• Contracting Functions Interface Capability

• Award/ Contract Modifications Capability

2.4.1 Proposal Evaluation Capability

The contractor shall provide a robust and secure web-based evaluation capability, which houses source selection sensitive information and is available to DAF SBIR/STTR proposal evaluators as soon as the proposal has been processed. The capability shall include automated evaluator assignment (by algorithm) and manual evaluator assignment (by DAF personnel) of proposal evaluations to hundreds of evaluators across the Air Force, customizable evaluation forms, source selection ranking, automated scoring components, automated input for evaluators to identify strengths/weaknesses of proposals for selection scoring, proposal selection/non-selection debriefings. Proposals, proposal data, and evaluation information should be maintained in a manner that allows for transfer to an outside AF contract writing system. This also includes selection lists, select/non select notifications, contractor interface and Phase II RFP generation.

2.4.1.1 Maintain multiple simultaneous workflows and business rules

The contractor shall provide multiple simultaneous workflows depending on the solicitation announcement type to support multiple evaluators for both Open Topic and Specific Topic with the capability to evaluate multiple criteria. The contractor shall then provide auto-generated feedback based on the evaluation rating. Examples of evaluation criteria based on the solicitation announcement are Commercialization, Defense Need, Technical Merit, Technical Approach and Team Qualifications.

Additionally, the priority of the criteria may have descending or equal importance in relationship to each other which their own unique ratings with five (5) levels ranging from Excellent to Poor based on the announcement type. To further illuminate the complexity of this requirement, the following evaluation type information is provided.

A) ABC Schema - Open Topic Only (EQUALLY weighted)

i. Each Evaluation Criteria is assigned to a different person.

ii. A = Person1, B= Person2, C=Person3

iii. Total score is A+B+C

B) Multiple - Specific Topic (default), 2 evaluator types:

i. Lead Evaluator - Evaluate all criteria and cost information, Creates the official Evaluation for record. This evaluation score is used for rank order. This evaluation is the official evaluation of record and available to FOIA and cross agency etc.

ii. Draft Evaluator - Evaluate all criteria and cost information, Evaluation is only used for reference to the lead evaluator, not an official record, not for FOIA. This evaluation type requires a minimum of ONE draft evaluator to complete an evaluation.

iii. Single Evaluator - Specific Topic (Optional) ONLY the Lead evaluator does an evaluation, there are no draft evaluators

There are up to ten (10) simultaneous workflows that must be maintained in their current configuration. A summary of the workflow salient characteristics as referenced in section 2.4.1.1. is provided below.

• US Air Force (USAF) Proposal Evaluation Open Topic

(ABC/Single) defined as multiple criteria, single evaluator

• US Space Force (USSF) Proposal Evaluation Open Topic

(ABC/Single) defined as multiple criteria, single evaluator

• USAF Proposal Evaluation Specific Topic (BAA Single) defined as

Broad Agency Announcement, single evaluator

• USAF Proposal Evaluation Specific Topic (BAA Multiple) defined as

Broad Agency Announcement, multiple evaluators

• USAF Proposal Evaluation Specific Topic (CSO Hybrid) defined as

Commercial Services Offering but also multiple or single evaluation workflow typically associated with BAA

• USSF Proposal Evaluation Specific Topic (BAA Single) defined as

Broad Agency Announcement, single evaluator

• USSF Proposal Evaluation Specific Topic (BAA Multiple) defined as

Broad Agency Announcement, multiple evaluators

• USSF Proposal Evaluation Specific Topic (CSO Hybrid) defined as

Commercial Services Offering but also multiple or single evaluation workflow typically associated with BAA

• USAF Proposal Evaluation Open Topic (ABC/Multiple) defined as multiple criteria, multiple evaluators

• USSF Proposal Evaluation Open Topic (ABC/Multiple) defined as multiple criteria, multiple evaluators

Note: Multiple Evaluators (ABC) (Open Topic Only)

Multiple evaluators can be assigned to each criterion for Open Topics. Multiple evaluators can evaluate each criterion for Open Topics with required scoring, select list, feedback and evaluation forms.

2.4.2 Automated Data Migration DSIP

The contractor shall provide an automated process to migrate proposal data from the DoD mandated proposal submission to DSIP (Defense SBIR/STTR Innovation Portal) into the evaluation tool. The contractor shall create an Application Programming Interface (API) between DSIP and the system. The system will have an API endpoint configured to receive proposal metadata and unstructured data (such as proposal documents).

Metadata shall flow into the system database to display relevant proposal information.

2.4.3 Proposal Evaluations Rubric Scoring (Open Topic only)

The Contractor shall provide the capability to allow Proposal Evaluators to review each proposal and rate them according to a government-provided rubric ranging from Excellent to Unacceptable in accordance with the solicitation types as identified in 2.4.1.1

Additionally, the contractor shall provide reporting capability to include the following Selected Proposal/Funded summary details at the conclusion of each evaluation sprint. At a minimum, the information included must contain:

o Total Proposal Qty o Total SBIR dollar amount o Total SBIR dollar amount + Government matched dollar amount

The interface shall have selectable radio buttons and have the capability to provide the Evaluator with the following displayed information exportable to Microsoft excel as a .csv, Google sheets or similar file format for further analyzing.

o Show/Edit Selected/Funded o Download Selected/Funded o Download Selected/Funded Evaluator Forms

Similarly, the Not Selected Proposals shall display the following information o Total Proposal Qty o Total SBIR dollar amount o Total SBIR dollar amount + Government matched dollar amount

The interface shall have selectable radio buttons and have the capability to provide the Evaluator with the following information that is exportable to Microsoft excel, Google sheets as a .csv (or similar) file for further analyzing.

o Show/Edit Selected/Funded o Download Selected/Funded o Download Selected/Funded Evaluator Forms

2.4.4 Proposal Evaluation Reassignments (Open Topic only)

The contractor shall integrate Natural Language Processing (NLP) and Machine Learning into the assignment workflow to facilitate a continuous improvement on evaluator throughput. The system shall document, track, and assess evaluator performance baselined against their capacity, interest, and expertise. This will generate data that will inform automated proposal reassignments. These Evaluator reassignments will be accomplished without human intervention.

2.4.5 AI Evaluations (Open Topic Only)

The Contractor shall utilize Artificial Intelligence (AI) as it relates to assistance with matching an evaluator’s skillset to proposals using algorithmic assignment to deliver improved evaluation outcomes within the government’s current evaluation process. The tool should allow for flexibility as requirements dictate.

2.4.6 Contracting Functions Interface Capability

The contractor shall provide a capability for the contracting arm of the AF SBIR/STTR Program to provide an import/export function for correlated data such as topic number, proposal number, contract amount, evaluation results, purchase order information contract number, fiscal year of funds, etc. This capability may be asked to interface with CONIT or other AF contracting systems.

2.4.7 Award/ Contract Modifications Capability

The contractor shall provide a web-based award capability to collect, process, and store all reporting information related to AF SBIR/STTR awarded projects. The capability shall allow input from AF personnel on all award records to include incremental funding and any contract modifications, status reports, upload status documents/information, and report outside funding applied to a proposal. In addition, the capability shall house the input of “success stories library” for the AF SBIR/STTR program. Contract and financial information will be input by AF personnel back into the system after award and payments.

2.4.8 Due Diligence Reporting and Tracking

The contractor shall configure the system to facilitate information flow and Due Diligence processing between investigators and SBIR Program Managers. Specifically, the draft Select List will be transmitted to investigators via the portal. The investigative dispositions will be communicated back to Program Managers within the portal. The system shall ensure dispositions are fully approved before notifications to selected and non-selected contractors are submitted. The system will produce auditable reports and historical records of all phases of due diligence processing within the system that shall be available for display and download.

2.4.9 Provide an efficient and effective user experience (UX)

The contractor shall provide a modern, interactive, user-friendly interface

(UI) for the website and web forms utilizing human centered design approach which includes User Experience of the Evaluation Assignment pages, Evaluation Forms, Select List, Program Manager Dashboard, Notification page, Disqualified (DQ) Verification module, and the Evaluator

Dashboard. Contractor proposal packages shall be downloadable within a single Zip file for every instance a proposal package is available for download within the portal. The Contractor’s interface shall be validated by user feedback and testing.

2.4.10 Evaluation Workflows Process Documentation

The Contractor shall document and maintain all evaluation workflows, reporting capabilities, acceptance criteria and testing documents throughout the period of performance.

2.5 Support Personnel

The contractor shall provide support staff to manage the AF SBIR/STTR Tool capabilities and assist the AF personnel users and small businesses users as required. The staff shall provide the following:

2.5.1 The contractor shall review all wording within the AF solicitation requirements documents included in the Government-developed BAA (Broad Agency Announcement) and ensure the SaaS tools are properly configured.

2.5.2 The contractor shall provide helpdesk assistance via Toll free phone number and e-mail address from 8:00 AM to 8:00 PM ET to all AF SBIR/STTR

Tool users. The contractor may be required to assist users in the upload and download processes.

2.5.3 The contractor shall provide the Government with online training materials for government and small business users of the AF SBIR/STTR Tool to guide them in the use of the tool.

2.5.4 The contractor shall provide personnel to support AF SBIR/STTR Tool status meetings as requested by the government.

2.5.5 The contractor shall provide the Government with online training materials for government and small business users of the AF SBIR/STTR Tool to guide them in the use of the tool.

2.5.6 The contractor shall provide personnel to support AF SBIR/STTR Tool status meetings as requested by the government.

2.5.7 The contractor shall respond to ad hoc system data requests not readily available in reports module.

2.5.8 The contractor shall support Government led evaluator and program manager training, ie how to use the software. This may entail demonstration of AF SBIR/STTR Tool functionality and workflows.

2.6 Service Level Agreement (SLA)

Contractor shall, at a minimum, incorporate the following metrics into their technical approach. Contractor’s SLA is valid from the go-live date and is valid until further notice. Any revision to the SLA metrics shall require a task order modification in writing and approved by the Contracting Officer and the

Contractor.

2.6.1 Contractor shall propose means and methods for generating monthly SLA reports for Government review and acceptance. Such reports shall be submitted with the Monthly Status Report.

2.6.2 Contractor shall meet or exceed the following service metric parameters.

SLA

Metric Name Description Criteria Disincentives

1 Availability 24 hours per day x 7 days

> 99% Acceptable

10% monthly payment per week x 365 days per year x 2000 simultaneous-users availability to all application components.

[Total minutes down] / [Total minutes in a month]

(Excepting Planned Service Outages)

97.5 % - 98.9% Fair

< 97.5% Unacceptable reduction for Unacceptable

CPARS

performance ratings will be reflected

2 Planned Service Outages (Maintenance windows)

No less than 5 working days’ notice for all planned service outages and automatic updates. Not to exceed 2 hours during business hours each month.

Pass / Fail 10% monthly payment reduction per incident

CPARS

performance ratings will be reflected

3 Unplanned Service Outages

Not to exceed 1 outage per month.

Pass / Fail 10% monthly reduction per incident

CPARS

performance ratings will be reflected

4 Backups Daily backups:

Monday-full backup;

Tuesday- Friday

Pass / Fail Monthly

Pass / Fail quarterly

Failure shall require contractor to implement incremental backup.

Each week makes one backup cycle;

archive data for each backup cycle off-site at a different contractor facility for the life of the contract.

Verification:

Contractor shall provide alternate URL (on different server); AF

SBIR/STTR

will verify seamless operation of redundant system corrective action plan.

CPARS

performance ratings will be reflected

5 Service Response Times

The amount of time it takes to resolve the issue.

[Total Time of incident] = ([Resolved Time] - [Open Time])

Emergency Response time

<24 Hours Acceptable

24-48 Hours Acceptable with Acceptable Justification for why it was not completed in 24 Hours

>48 Hours Unacceptable

Within 60 minutes of first

10% monthly reduction for Unacceptable

CPARS

performance contact, issue is being actively worked

60-90 minutes Acceptable with Acceptable Justification for delay beyond 60 minutes

>90 minutes Unacceptable ratings will be reflected

6 Response by phone

[Number of calls answered by immediate live support] / [Total Number of calls] each month, during business hours.

> 90% Acceptable

< 89.5% Unacceptable

5% monthly payment reduction for Unacceptable

CPARS

performance ratings will be reflected

7 Response by email

[Number of emails that are responded by live support (not automated response) within 4 hours] / [Total Number of Emails] each month, during business hours.

> 98% Acceptable

< 97.5% Unacceptable

5% monthly payment reduction for Unacceptable

CPARS

performance ratings will be reflected

8 Deliverables Delivered timely and with 100% accurate content

Delivered within 4 hours of agreed upon due date

5% monthly reduction per incident

Deliverable contains no more than two

(2) incorrect lines of data

CPARS

performance ratings will be reflected

2.6.3 Contractor shall provide technical responses to a service request according to the table below. AF SBIR/STTR Program will assign priority.

Priority Definition Correction

A key function (ability to access/upload/update/download) is inoperable and further operation cannot proceed until the problem is fixed.

Within 2 hours

A function of the system is inoperable and further operation of that function cannot proceed but operation of other functions can proceed before this problem is resolved

Within 4 hours

While a function of the system is operable, the operation is inconsistent with the Service Levels and the inconsistency involves a material failure to comply with specifications for the same.

Within 8 hours

An observation which does not affect the functionality of the system is recorded.

Within 5 business days

A request for a new report Within 5 business days

2.7 Service Ticket Management System

The Contractor shall utilize government approved ticket management system to track, manage, and resolve customer issues efficiently through assignment rules, and escalation policies. The ticket management system shall also capture detailed resolution notes with timestamp to track response times IAW with the stated SLAs in section 2.6 and have the capability to pull reports and display health status dashboard. The ticket management system shall have the ability to notify users via email of actions needed and status of their ticket.

3 QUALITY

Both the contractor and the Government have responsibilities for providing and ensuring quality services, respectively.

3.1 Contractor Quality Management

The contractor shall identify and implement its approach for providing and ensuring quality throughout its solution to meet the requirements of the PWS via the contractor’s Quality Management Plan (QMP). The QMP shall describe the application of the appropriate methodology (i.e., quality control and/or quality assurance) for accomplishing performance requirements. The QMP shall describe how the appropriate methodology integrates with the Government’s requirements.

The contractor shall make the QMP available to the Government for review upon request after award and shall obtain acceptance of the QMP by the CO if the CO asserts such requirement. The contractor shall make appropriate modifications to the QMP (at no additional cost to the Government). The Government has the right to require revisions of the QMP (at no cost to the Government) should the QMP fail to deliver the quality of the services required at any time during performance.

3.2 Performance Based Requirements - Performance Requirements Summary

(PRS)

Performance Standards, Acceptable Quality Levels (AQLs), and Incentives/Disincentives are defined in PWS Attachment 1, collectively titled as the PRS. The PRS criteria will be used to determine if performance requirements are met.

3.3 Government Quality Assurance

The Government will perform periodic reviews of the contractor’s performance in accordance with the Government’s Quality Assurance Surveillance Plan (QASP), which will not be incorporated into the resultant contract/order. The Government reserves the right to review services to be provided, including those developed or performed at the contractor's facilities, to determine conformity with performance and technical requirements.

4. DELIVERABLES

4.1 Contractor Submission

Deliverables are to be submitted/transmitted with a cover letter, on the prime contractor’s letterhead, describing the contents, electronically through ASSIST and to any other destination(s) as required per the Government’s request. All deliverables shall be submitted via an ASSIST “Deliverable Collaboration”. As applicable and as requested, the Monthly Status Reports (MSRs) shall also be included as an attachment to the invoice. The contractor shall provide hard copy deliverables as required per the Government’s request. All deliverables shall be produced using software tools/versions as approved by the Government.

4.2 Government Review

Government personnel will have 10 business days to review deliverables (to include resubmissions) and provide written acceptance/rejection. Authorized Government representatives will notify the contractor of deliverable acceptance or provide comments in writing. The contractor shall incorporate Government comments or provide rationale for not doing so within 5 business days of receipt of comments.

Government acceptance of the final deliverable will be based on resolution of Government comments or acceptance of rationale for non-inclusion. Additional changes volunteered by the contractor will be considered a resubmission of the deliverable.

4.3 Government Delays in Reviewing Deliverables or Furnishing Items

If contractor performance or submission of deliverables is contingent upon receipt of Government furnished items (data, equipment, materials, facilities, and support) or input, or upon Government review and approval of interim items or draft documents (collectively referred to as Government Performance), the Government shall specify, if not already specified within this PWS, when it will provide such items or input, or the time it will need to perform reviews or give approvals. If the Government fails to meet item, input, review, or approval deadlines, contractor performance or submission of deliverables shall automatically be extended one business day for each day of government delay. The contractor shall promptly advise the Contracting Officer of any delays in receipt of Government furnished items, input, reviews, or approvals.

4.4 Deliverable Table

The contractor shall deliver the deliverables listed in the following table on the dates specified.

DEL. #

MILESTONE or

DELIVERABLE

PWS

REF.

COMPLETION or DELIVERY

DATE

1 Kick-Off Meeting 2.2.1 NLT 15 business days after contract/order award

Kick-Off Meeting Minutes 2.2.1

NLT 5 business days after the meeting

Monthly Status Report

(MSR) 4.6

15th calendar day of the month following the monthly reporting period

4 Monthly Invoice 4.5

15th calendar day of the month following the monthly reporting period

5 Trip Report(s) 4.7

With the MSR for the monthly reporting period during which travel occurred

6 QMP 3.1

5 business days after Government request (only required if CO asserts such requirement)

7 Phase-In Plan 4.8

If requested by the Government, 120 calendar days prior to contract/order end date

7 Phase-Out Plan 4.9

If requested by the Government, 120 calendar days prior to contract/order end date

Government Furnished Property (GFP) Inventory 10.1

NLT 3 business days after GFP/GFI receipt or changes to inventory

Non-Disclosure Agreement 11.1

Prior to assignment to contract/order

PIV/Security Badge Contract Close-Out Statement 12.5

NLT 15 calendar days after the end of the period of performance

Evaluation Workflow

Process Documentation 2.4.10

NLT 60 business days after contract/order award and updated as required

4.5 Monthly Invoice

The contractor shall provide a monthly invoice, no later than the 15th calendar day of the month following the monthly reporting period, to be submitted simultaneously with the MSR. As applicable, the invoice shall include but is not limited to:

▪ Clear identification of all costs.

▪ Labor hours expended (for labor hours tasks). The labor hours expenditure information shall include the identification of the employee name, labor category, hourly labor rate, and total number of labor hours expended.

▪ Timecards. As required, the contractor shall provide a copy of each employee’s timecard/sheet. The timesheet shall identify the contractor employee name and number of hours claimed per day.

▪ Travel costs.

▪ Supporting documentation for travel costs. Refer to PWS 5.3 for specific requirements.

▪ Other Direct Costs.

▪ Supporting documentation for other direct costs. Refer to PWS 9.6 for specific requirements.

▪ As required, the contractor shall comply with line item (i.e., per individual positions, different programs, program areas, etc.) invoicing requests.

4.6 Monthly Status Report

The contractor shall provide an MSR, no later than the 15th calendar day of the month following the monthly reporting period, to be submitted simultaneously with the monthly invoice. The MSR shall include, but is not limited to, the items identified below.

▪ Status of tasks, schedules, deliverables. Status of tasks shall include a summary description and schedule of all tasks completed during the reporting period, all tasks currently on-going during the reporting period and all known tasks assigned for future reporting periods.

▪ Resource hours and costs against Government defined tasks to support accurate project cost accounting using Government formats and tools.

▪ Current and cumulative task funding status (direct labor, travel, and other direct cost funding status to be reported separately, as required).

▪ Outstanding issues, and proposed resolution approaches and actions to resolve any outstanding issues.

▪ Staffing report identifying current staffing roster, all current vacancies, and a record of all staffing departures as well as identifying which staff members possess a security clearance.

4.7 Trip Report

For travel, the contractor shall submit Trip Reports as attachments to the MSR for the monthly reporting period containing the travel dates. The contractor shall reconcile the Trip Reports with each invoice such that they can be matched month by month. The Trip Report shall include the following information:

▪ Completed Itemized Expense Report (PWS Attachment 2), or Government accepted alternate, for each trip and each employee who traveled. Expense report(s) shall include the contract/task order number, contractor name, traveler’s name, dates of travel, destination, purpose of travel, travel authorization documentation (e.g., identification of approved ASSIST “Contractor Travel Request Collaboration”), and actual cost for each trip.

▪ Summary of trip events and accomplishments.

4.8 Phase-In Plan

The contractor may or may not propose a separately priced transition period, for a duration to be determined and proposed by the contractor, however, it shall not exceed a period of 90 calendar days. The transition period is defined as the period of time (during the phase-in) when the new contractor and the incumbent contractor will both be providing support to the client as required to support the transition to the newly awarded contract/order. If the contractor chooses to propose a transition period, such period shall be included and addressed within the below identified phase-in plan.

The contractor shall develop a phase-in plan. The phase-in plan shall be consistent with the implementation schedule requirements highlighted in PWS para 8.2. Such phase-in plan shall present a clear understanding of the phase-in tasks required, the issues likely to result from non-incumbent contractor performance, and the contractor’s proposal to resolve such issues. The phase-in plan shall include a clear and feasible strategy for delivering services required within the periods specified by the plan and shall include a detailed plan-of-action and milestones to transition the functions identified in this PWS in a well-planned, orderly and efficient manner. The phase-in plan shall include, at a minimum:

▪ Staffing plan.

▪ Development and submission of required deliverables.

▪ Interface with the Government and incumbent contractor (if applicable) during phase-in, to include meetings or status reports, as required.

▪ Approach to maintaining quality and minimizing disruption during phase-in.

▪ Development and dissemination of operating instructions, procedures, and control directives.

▪ Data migration plan

4.9 Phase-Out Plan

During phase-out of the contract/order, which is determined to be a period of 120 calendar days prior to the lifecycle end date of the contract/order, a smooth and orderly transition between the incumbent contractor and the successor contractor may be necessary to ensure a minimum disruption to vital Government business.

The contractor shall cooperate to the extent required to permit an orderly changeover to the successor contractor. The phase-out will be deemed completed by the COR and/or other identified Government representatives when it is determined by the Government that the transition of property, data, and information developed as a part of the contract/order have been successfully changed over from the outgoing contractor to the Government and the successor contractor as required.

If requested by the Government, phase-out activities include, but are not limited to, the tasks below.

▪ Submission of official comprehensive phase-out plan.

▪ Daily communication of staffing status (i.e., projection of when incumbent contractor employees will off-board from the incumbent contract/order and identification of additional incumbent resources, such as a transition team, that may be needed to support the transition efforts) and overall phase out status, in accordance with the accepted phase-out plan.

▪ Maintain the phase out schedule included within the phase-out plan.

▪ Transition of property.

▪ Transition of supporting documentation in a secure IL4 approved solution.

▪ Transition of accounts (e.g., user accounts and user access).

▪ Knowledge transfer on the established installation, operation, and maintenance procedures of the technologies supported. The phase out plan shall clearly describe the proposed methodologies to be utilized for such transfer (e.g., written documentation, manuals, formal classroom type training, one-on-one training sessions, etc.).

▪ Transition of historical data

▪ Execution and submission of phase out checklist, to include Government acceptance.

5 PERFORMANCE

5.1 Period of Performance

The anticipated period of performance(s) will include a base period of 12 months and four (4) six (6)-month option periods.

5.2 Place of Performance

The primary place of performance is identified in the table below.

PWS Task Government or contractor Facilities

Government Location

Entire PWS Contractor N/A

Travel location TBD TBD

5.3 Applicability of Telework

All work performed at locations other than those identified as Government and/or contractor facilities shall be approved prior to performing the work. Federal contractors are not governed by Office of Personnel Management (OPM), GSA, or the individual agency policies; however, this does not prohibit contractor personnel from working at an alternate site, when/as appropriate and specifically authorized by the Government. The contractor shall develop telework policies to comply with the following requirements and address such requirements at a generic level within their QMP. Alternate work arrangements for contractors shall be negotiated with the contractor’s own employer and the appropriate agency official, to ensure policies and procedures are in close alignment and there is a clear and concise arrangement documenting the agreement. It remains the contractor’s responsibility to ensure the services are performed in accordance with the terms and conditions of the contract/order.

▪ The contractor shall address the pertinent facts impacting performance and ensure all affected contractor resumes and other related documentation reflects the applicable work site.