Global Force Management List of Attachments.docx

Global Force Management List of Attachments

1. Base PWS

2. PWS Task Order 1

3. PWS Task Order 2

4. Draft PWS Questions & Answers

5. QASP Base

6. Question Template

7. GFM Non-Disclosure Agreement for Pre-solicitation Conference and Code Review

8. DISA Non-Disclosure Agreement for Contractors

9. DD 254 Form

10. Key Personnel Qualifications

11. Past Performance Cover Letter & Client Questionnaire

12. Past Performance Volume Submission Template

13. Sample Consent / Authorization Letter

14. Small Business Participation Proposal Format

15. Schedule of Services Labor Rates (CLIN Structure) Excel Workbook

16. Task Order 1 Schedule of Services (CLIN Structure) Excel Workbook

17. Task Order 2 Schedule of Services (CLIN Structure) Excel Workbook

18. Subcontracting Plan Checklist image4.emf

4 Draft PWS Questions & Answers.xlsx Sheet1

# Question/Comment/Suggestions File Page Paragraph/Section Answers

1 Can the Government provide the schedule of milestones/desired delivery dates for the final products for Task Order 001 (Develop the JPES Design and establish a DevSecOpns environment) and Task Order 002 (JPES Development)? General Looking for a Contractor provided solution and associated schedule. The Government estimates 3 months for TO1 Design and 9 months for TO2 Development. The Government anticipates an additional 9 month TO to complete the development and 3 month TO for Test and Evaluation to achieve Full Operational Capability.

2 We understand there are existing design documents and code from a previous effort that will be provided to the successful bidder. Can the Government affirm that this is usable data, or will the design and development efforts be a new start effort? General Yes, there is existing design documents and code from a previous effort. The usability of this shall be determined by the Contractor. Future design and development efforts may utilize existing code and/or design.

3 Will the Government consider an Industry Day and One-on-One meetings prior to the release of the final RFP? General The Government plans to hold an Industry Day and one-on-one code preview meetings, but they will occur after the release of final RFP.

4 Will the Government be releasing draft Section L an M prior to the release of the final RFP? No, the Government will not be releasing a draft section L & M.

5 In regards to the upcoming DISA GFM contract, why is it not being released small business? At $45M over 5 years ($9M per year), that can easily be accomplished by a small business. General DISA conducted thorough market research and determined, in accordance with the governing regulation, that it is in the best interest of the Government to release the GFM requirement as full and open competition through Contract Opportunities. Small businesses will have the opportunity to participate as subcontractors for this requirement.

6 When will a Bidder’s Library containing the source code and GFI identified in Base PWS Para 12.2 be available for prospective offerors? General At Industry Day and Code Preview approximately 1-2 weeks after final RFP release.

7 Recommend the Govt consider structuring the task orders to perform the DevSecOps Standup under a single task order (TO1) and to perform the design and development of JPES under a separate task order (TO2). The DevSecOps standup work can be performed independently while the JPES design and development work is closely aligned. General TO1 and TO2 work was split based on severable tasks as well as schedule. Putting design in TO2 will require parallel execution of the TO's, this is not the Governments plan.

8 What’s the overarching schedule and schedule drivers? The JOPES sunset date is listed as Q1FY23 (Oct-Dec 2022) with the PoP for the two TOs as 12-months. When does the Govt anticipate award/start of performance? General Over-arching schedule driver is JOPES sunset date. Government anticipates award Q3FY20 and performance start Q4FY20.

9 Please clarify the Govt’s intent on the use of Waterfall or Agile software development methodologies. The design effort identifies Waterfall events such as PDR and CDR while also using Agile terms such as user story, sprints, and product owners. Is it the Govt’s intent that the JPES modernization be a hybrid of Waterfall and Agile methodologies? General The Government's intent is to utilize Agile software development methodologies. However, the Government needs confidence in the proposed design solution prior to committing to development efforts. TO1 which includes PDR and CDR is intended to provide that confidence and the foundational design, architecture and the agile execution approach.

10 What is the plan for sustainment of JOPES during the modernization (design and development) and transition to JPES? Is sustainment of JOPES within the scope of this effort during both TO1 and TO2? General The sustainment of JOPES is not within scope of this effort. The JPES modernization effort to include design, development, test and sustainment is separate from JOPES sustainment.

11 Regarding access to a SIPRNet test and integration environment, will the work require a SIPRNet connection at the contractor’s facility? Base PWS 2 5 No, T&E work will NOT require a SIPRNet connection at the contractor's facility. All work requiring a SIPRNet connection will be performed at the Government's facility.

12 Recommend “Third-party software defects” clarify that the contractor is responsible for defects in third-party software provided by the contractor—and that the Govt is responsible for defects in any GFI/E provided software. Base PWS 11 6.2.6 No. If there are any defects identified in GFI/E provided software the contractor will be responsible for correcting them.

13 Recommend deleting “immediate” from statement on notifying the Govt project owner of defects. Base PWS 11 6.2.6 "Notification within 24 hours via automated DevSecOps tools." Will update PWS language.

14 Under “Modular Open Systems Approach” there is a requirement for components to “be of a size that supports competitive acquisition as well as reuse.” Is there further technical guidance on how to determine if this requirement is met? Base PWS 20 6.4.1 To further elaborate, this requirement will not be met if a component is overly complex and unique, thus eliminating the possibility of re-use for another purpose.

15 Regarding Test Artifacts, is the Govt’s intent to have a TRR at the end of each Sprint? Base PWS 22 6.5.1 Yes, but since we are doing DevSecOps this should be a very quick and short TRR based on the delta between the last sprint.

16 For unit test execution, there’s a requirement to demonstrate “sufficient” code and test coverage. Does the Govt have specific coverage requirements (e.g., XX% code coverage) to determine sufficiency? Base PWS 22 6.5.2 While there is no hard number, generally industry does about 80% code coverage, this number should go down or up based on the quality of code being developed. For example, poor quality code may require 99% code coverage if there is not a lot of confidence in the actual code.

17 Response time testing across a network includes many variables that are beyond the software developer’s control. Please clarify if the contractor’s responsibilities for response time do not include network performance. Base PWS 23 6.5.4 The developer will be responsible for response time within the network domain which the developer has control over. For example, the packet size and the ability to break up data and retransmit bad data is usually controlled by configurations that the developer has control over. This is the developer's responsibility.

18 Is the Gov’t intent to accept (i.e., verify requirements) software via a demonstration at the end of each Sprint? Base PWS 24 6.5.5 No. The government will accept requirements during the government acceptance testing in the SIPR environment, this is part of the DevSecOps pipeline on the classified side.

19 Is it a requirement that all classified work must be done on-site in the DISA facilities on Ft Meade? Base PWS 44 9 Yes

20 ID #9, the DevOps CONOPS is an important compliance document and will have significant bearing on the technical approach—when will this document be available? Base PWS 50 12.2 DevSecOps CONOPS will be available at contract award. However, the Test & Evaluation Master Plan (TEMP) will be available at time of RFP and contains an overview of the DevSecOps CONOPS. Please reference section 2 of the TEMP for details regarding the DevSecOps CONOPS for use during proposal preparation.

21 ID #60, Is Confluence being used as a collaboration platform or portal (did not see it referenced elsewhere in the PWS)? Base PWS 53 12.2 Currently confluence is being used. The government is open to changes and will require transition of information from the current confluence.

22 Recommend Govt consider changing the frequency of Status Reporting to every two weeks. PWS TO1 3 6.1.3 It is important to have weekly status reporting, but the government is open to recommendations to reduce the scope.

23 Is the IMS developed and maintained in TO1 to include all work to sunset JOPES in Q1FY22? In other words, is the IMS is to include work beyond the scope of TO1? PWS TO1 5 6.2.1 Yes

24 Given the Govt’s intent to use an Agile development methodology, recommend the Govt delete the requirement for EVMS. Specifically, the CPR submittals. If the Govt retains the EVMS requirement, recommend the submittal schedule be aligned with the end of Sprints rather than the calendar or accounting months. PWS TO2 2 6.1.4 The government will retain EVMS requirements and will align to the end of sprints. The PWS will be updated.

25 Recommend Govt consider aligning the Status Report delivery with the end of each Sprint. PWS TO2 3 6.1.5 See answer to question 22.

26 Will the government make the current code base available offline so we can thoroughly review it and analyze its current state? If the above is not possible, will the Government consider breaking the code review session into two events? In the first event, we request having five reviewers for four hours to investigate the code and architecture diagrams. We also request the Government to ensure that there are SMEs available for each component of the architecture to answer technical questions. We request the second event be scheduled a week later to provide adequate time to analyze findings and allow for follow-up questions to be asked about the JPES architecture design. Draft PWS General The current code base will be made available for in-person review. There will be two code review sessions lasting up to 8 hours each. The Government will not answer any questions during code review sessions. Code review sessions will be held after RFP release. Unsupervised access to code base will not be provided to any contractors prior to award.

27 To maximize the effectiveness of the upcoming code review, will it be possible for the government to provide us further architecture diagrams and a detailed listing of software components and their functions before the code review event? Draft PWS General Yes, during the pre-proposal conference after RFP release.

28 Will the Government consider providing a bidders library inclusive of: • architecture diagrams (Design materials, DoDAF views, IDD, etc.)

• detailed listing of software components and their functions

• current requirements backlog

• list of open defects

• list of open help desk tickets

• performance metrics and KPPs

• technical requirements (JC2 architecture, JIE implementation plan, etc.)

• existing incident management plan Draft PWS General Certain items will be provided in the bidder's library. Others will be available during the code review.

29 With GFM being a mission critical program, DISA will benefit from a partner that focuses upon quality processes, provide consistent quality monitoring, apply best practices and lessons-learned, and the development of frameworks for use across government client base. Will the government consider requiring CMMI Dev Level 5 certification from bidders to demonstrate the ability to provide high quality delivery for mission applications and systems. Draft PWS General The government will not require CMMI level 5 certification.

30 Because of the complex nature of the requirement and the criticality in executing the milestones with qualified resources, will the government consider adding key personnel: • Program Manager

• DevSecOps Engineering Lead

• Senior Information Security Engineer

• Senior Software Engineer

• Chief Architect Draft PWS General The government will provide key personnel requirements in the RFP.

31 To ensure that each offeror has demonstrated experience in successfully developing and sustaining a modernized software solution, will the Government consider requiring three past performance examples of similar size, scope, and complexity for the IDIQ, as a critical differentiator in modernized software solution, with one or more being IDIQ level enterprise software delivery management? Draft PWS General Past Performance will be considered as an evaluation factor for making an award determination. The importance and type of past performance that will be required will be detailed in sections L and M of the RFP.

32 The PWS specifies that all hosting environments will be provided by the government via milCloud or other government furnished facility. Is this referring to the milCloud 2 environment or the milCloud 1 environment? Draft PWS General This will be based on what is available and is in the best interest of the government at the time of award. It maybe a mix of milCloud or commercial cloud.

33 The latest DoD CIO memorandum for the fourth estate specifies that as all agencies should be moving to the cloud the first option is the General-Purpose JEDI environment. Is JEDI Microsoft Azure cloud environment one of the possible hosting environments? Draft PWS General Yes. Also, see answer to question number 32.

34 One of the DISA STIGs requires Role Based Access Control (RBAC) for any elevated privilege. milCloud 2 does not provide any RBAC and all users have full elevated privileges. Will this be a consideration in reviewing commercial cloud via JEDI as a possible hosting environment? Draft PWS General RBAC applies to the application not cloud access.

35 Will the government provided hosting environments also include the required security tools (i.e. HBSS, ACAS, Splunk, Firewalls, etc.) required by DISA STIGs? If utilizing a cloud environment, will it also provide the required Virtual Data Center Security Stack (VDSS) and Virtual Datacenter Managed Services (VDMS), required by the DISA Secure Cloud Computing Architecture (SCCA), or will the contractor be required to set that up? Draft PWS General The government will provide the required security tools (HBSS, ACAS, Firewalls, etc.) Details of additional tools will need to be worked between the government, the IA community and the vendor. The contractor will be required to configure and customize these tools to meet the government's needs.

36 Does the government intend to compete and award both TO’s at the same time? Draft PWS General Yes

37 Given the aggressive timelines associated with TO1 (approximately 3 months for approval of PDR, CDR, 60+ deliverables), we recommend that the evaluation criteria for the technical approach consider innovative techniques coupled with domain expertise as part of this solicitation. Draft PWS General Thank you for the recommendation. The government will take it under consideration. Final evaluation criteria will be documented in sections L & M of the final RFP.

38 Given the high complexity of developing a distributed system that meets the functional requirements and key performance parameters of JOPES and JCRM, will the evaluation criteria weight the technical approach as significantly more important than the cost factor? Draft PWS General The government plans to utilize best value trade off analysis. Details of evaluation criteria and their importance will be specified in section M of the RFP.

39 Will the government include Oral Presentations as part of this solicitation to ensure that offerors can explain and answer questions on our approach and solution? Draft PWS General No

40 The PWS specifies that it expects deliveries using Infrastructure as Code (IaC). Azure has Azure Resource Manager (ARM), which is the native IaC capability of JEDI but milCloud 2 does not have a native capability and other government furnished environments may only support certain tools. Is there a preferred IaC technology to integrate into the government provided hosting environment? Draft PWS (Base) 6.1.7 The IaC technology utilized shall be portable across multiple cloud providers.

41 The PWS states that “the Contractor shall maintain a JPES First Look site”. Will the government provide the hardware and/or hosting for this environment or provide a plug ODC line for this? Draft PWS (Base) 6.1.8 The Government will provide the hosting environment, the contractor will be responsible for installation, configuration and possibly licensing of the software. The government will provide a plug ODC number.

42 The PWS states "confirm the software solution can be installed, tested and sustained in the Government's unclassified and classified environment free of defects"…Is "free of defects" referring to complying with the standard of resolving Severity 1 and 2 defects with lower level defects moved to the backlog for potential assignment to a future release based on Government priority and team capacity consistent with standard Agile practices? Draft PWS (Base) 6.2.6 The government understands that no software will ever be 100% defect free. In general, no severity 1 and 2 defects will be allowed, the lower severity defects shall be corrected based on the critically and needs that are negotiated between the government, the cybersecurity community, the test community, the user community and any other required stakeholders.

43 The PWS states "The test cases shall be completely written at the beginning of a sprint and the test scripts shall be delivered alongside source code. Both shall be submitted to the Government for verification."…. What is the timing and scope of the verification activities performed by the Government? Draft PWS (Base) 6.5.5 Based on DevSecOps best practices validation will happen in multiple pipelines at least once after each sprint, in addition there will be third party test events to validate functionality and security.

44 The PWS states “Work hours for Tier 3 are 24x7. Could the government provide more specifics around this requirement, including: • historic volumes of Tier 3 support tickets

• is this 24x7 shift work? 24x7 on-call support?

Draft PWS (Base) Section 9 Tier 3 is generally on call support only. There is no shift work required. Since this system is not yet operational, Tier 3 ticket history is not available. The frequency in general will be dependent on the quality of the software built and the accuracy and completeness of troubleshooting instructions provided to Tier 1 and Tier 2 support.

45 Does the government expect that the only instructor led training under Subtask 8 is the familiarization training described at the end of this section? (Note that there's more familiarization training in Subtask 9/6.1.9)? Draft PWS (Base) 6.18 Familiarization training will be required under subtask 8 and 9. The training may include presentations, online videos, documents, etc. and is not limited to instructor led training only. In addition there might also be train the trainer training.

46 Will the government provide the referenced “DevSecOps CONOPS” so that we can ensure that our proposed pipeline meets the government requirements and expectations? Draft PWS (TO 1) 6.2.6 Yes, at the time of RFP release.

47 Will the government clarify the difference in requirements for section 6.2.6 Operating in the Government DevSecOps Toolchain” between TO1 and TO2? The PWS text is largely duplicated, making it unclear what requirements are different across the task orders. Draft PWS (TO 1 and TO 2) 6.2.6 Both TO's In general for both task order 1 and 2 you will be operating in the same DevSecOps tool chain. The same rules and CONOPS will apply. Task order 1 is more focused on the initial stand-up of the DevSecOps tool chain and the pipeline while task order 2 is focused on further development of the pipeline to support the actual software development.

48 The PWS references “The Contractor shall document the resulting specific human factors design guidance in a UI/UX development guide”. However, the deliverables table at the end of the section does not contain a corresponding deliverable. Is this a contract deliverable, and what is the format/due date/frequency and remarks? Draft PWS (TO 1) 6.3 This document is to ensure that UI/UX development is approved and standardized across the whole project even with different developers. This document will be utilized internally by the developer to standardize and guide UI/UX practices for the whole project.

49 The PWS references “As part of its Open System Management Plan...”. However, the deliverables table at the end of the section does not contain a corresponding deliverable. Is this a contract deliverable, and what is the format/due date/frequency and remarks? Draft PWS (TO 1) 6.4.1 No separate deliverable is required. This shall be part of the software architecture and design documentation.

50 The list of deliverables has several that are due 30 days after award (RMF Security Control Report, System Security Plan, Security Control Required Documentation). Since key design decisions might not have yet been finalized, we recommend that the government modify the due date to be due at the CDR event. Draft PWS (TO 1) 6.8 Draft version of these documents shall be due one week prior to the PDR event. These documents shall be updated accordingly for the CDR event. The due date will be updated in the TO1 PWS.

51 The “Due Date” in the deliverables table references “At CDR as scheduled in IMS”. Does the Government intend for the contractor to conduct additional CDR events as part of TO 2? Draft PWS (TO 2) 6.3.1 Yes. Each Task Order will have PDR and CDR events that are to present the design of the follow-on task order.

52 “Location – The Contractor shall identify a specific methodology to facilitate scrum meetings (daily, planning and retrospectives) at the DISA headquarters complex in Fort Meade, MD, and other similar components of the agile methodology.” Does the Government expect to host the entire agile team at Ft Meade for the daily scrum, planning and retrospectives? Or is the Government expecting the contractor to be remote and provide a telecon/WebEx for participants to join remotely? Draft PWS (TO 1 and TO 2) 6.4.1 Participation can be remote and does not require physical presence at Fort Meade, but the contractor shall ensure full participation in these meetings.

53 The PWS references “As part of its PDR/CDR/Delivery, the Contractor shall...”. Does the government expect the contractor to conduct PDR and CDR events as part of TO 2? Draft PWS (TO 2) 6.4.1 See answer to question 51.

54 Is the Government open to using Commercial-Off-the-Shelf technologies (Components/Software) as part of the JPES system? An example would be adapting and integrating robust scheduling software components versus building scheduling software for JPES. This reduces the development and sustainment costs and increases the licensing cost. If the answer is yes, we suggest that the Government provide language that supports this approach. General Yes, see section 6.4.1 of PWS.

55 The Preliminary Design Review (PDR) and Critical Design Review (CDR) language across TO1 and TO2 may have some inconsistencies and may need clarification. For example, TO1 states, “PDR and CDR: Design JPES and conduct PDR and CDR. Following a successful CDR and with Government concurrence, Development will commence in Task Order Two (TO 0002).” The terms PDR/CDR has formal connotation of not changing the design. The TO2 states that there will be multiple PDR/CDR events.” Can the Government provide clarification on the intent in TO1 & TO2 with respect to system design and design changes? General See answer to question 51. Agile development allows for changes to the design based on issues and constraints found during development sprints, the goal of the PDR/CDR events is to ensure there is an initial design to start off with at the beginning of the development phase. Design changes shall be discussed and approved by the government.

56 For the TO1 PWS, Can the Government confirm that sections 6.4, 6.5, 6.6, and 6.7 and corresponding deliverables belong in TO1 since they relate to post-design activities and expected to be executed in TO2? General Those sections are still valid. For TO1 the focus of those sections is to initially set-up the DevSecOps environment, the pipelines and test and IA tool sets. For TO1 the vendor will need to ensure that the DevSecOps pipelines execute unit tests, functional tests, cybersecurity tests and automated deployment of the software. For TO2, the focus changes to the development of the actual application and DevSecOps activities to support development activities.

57 Suggest that the government offer relief relating the to the clearance requirements for off-site developers. We request that the government adjust the need for an active Secret clearance for all “off-site, non-lead technical personnel” to be an Interim Secret clearance; however, require that all “off-site, lead technical personnel” have active Secret clearance. General No

58 Can DITCO provide additional (detailed) requirements the contractors need to provide as part of the Development Environment. Base PWS 6.2.6 The government does not understand the question. Please elaborate and give examples of what kind of information you are looking for.

59 Can DITCO illuminate what additional tasks (high level) would occupy the remaining 4 years of the PoP? Base PWS 10 The remaining years would consist of follow-on development task orders similar to TO2. Additionally, towards the end operational and cyber test support, sustainment and support task order will come into play.

60 Should contractor teams presume the remaining 4 years of the PoP would be for post-SunDown sustainment of the JOPES environment? Base PWS 10 Not all, but some of the remaining 4 years will be post JOPES sunset.

61 Does the Government expect to have JOPES replaced (Sundowned) by JPES at the conclusion of TO-2? Draft PWS (TO 1) 3 The government expects to sunset JOPES 1QFY23.

62 Regarding the use and maintenance of multiple JPES environments (NIPR & SIPR), are both environments required to be identical or will they have different requirements? Draft PWS (TO 1) 6.5 The goal is to have all environments mirror each other. Classified test and operational data and documentation of system vulnerabilities are restricted to the classified environment.

63 Will the Government consider a Period of Performance beyond 90-days? Draft PWS (TO 1) 10 This is the best government estimate. Vendors are free to propose their own timeline.

64 Can the Government expand and illuminate on the 'end-state' of the 9-month TO-2? Draft PWS (TO 2) 10 The end state of TO2 will be determined at the CDR of TO1.

image5.emf

GFM QASP Base.docx

QUALITY ASSURANCE SURVEILLANCE PLAN (QASP)

1. Contract or Task Order Title:

Joint Planning and Execution Service (JPES)

2. Purpose:

This QASP provides a systematic method to evaluate performance for the stated contract. This QASP explains the following:

· What will be monitored?

· How monitoring will take place.

· Who will conduct the monitoring?

· How monitoring efforts and results will be documented.

This QASP does not detail how the contractor accomplishes the work. Rather, the QASP is created with the premise that the contractor is responsible for management and quality control actions to meet the terms of the contract. It is the Government’s responsibility to be objective, fair, and consistent in evaluating performance. In addition, the QASP should recognize that unforeseen and uncontrollable situations may occur.

This QASP is a “living document” and the Government may review and revise it on a regular basis. Updates shall ensure that the QASP remains a valid, useful, and enforceable document. Copies of the original QASP and revisions shall be provided to the contractor and Government officials implementing surveillance activities.

Work requirements to be monitored are:

· Performance Task Area 1 – Enterprise Management

· Performance Task Area 2 – Planning & Analysis

· Performance Task Area 3 – Design & Architecture

· Performance Task Area 4 – Development

· Performance Task Area 5 – Test and Evaluation (T&E)

· Performance Task Area 6 – Deployment

· Performance Task Area 7 – Sustainment

· Performance Task Area 8 – Risk Management Framework (RMF)

3. Roles and Responsibilities:

The following personnel shall oversee and coordinate surveillance activities.

Program/Project Manager (PM) – The PM provides program oversight and supports the COR’s performance assessment activities.

Contracting Officer (KO) – The KO shall ensure performance of all necessary actions for effective contracting, ensure compliance with the contract terms, and shall safeguard the interests of the United States in the contractual relationship. The KO shall also ensure that the contractor receives impartial, fair, and equitable treatment under this contract. Determine the final assessment of the contractor’s performance.

Contracting Officer’s Representative (COR) – The COR is responsible for providing continuous technical oversight of the contractor’s performance. The COR uses the QASP to conduct the oversight/surveillance process. The COR shall keep a Quality Assurance file that accurately documents the contractor’s actual performance. The purpose is to ensure that the contractor meets the performance standards contained in the contract. The COR is responsible for reporting early identification of performance problems to the KO. The COR is required to provide an annual performance assessment to the KO which will be used in documenting past performance. The QASP is the primary tool for documenting contractor performance. The COR is not empowered to make any contractual commitments or to authorize any contractual change on the Government’s behalf.

Other Key Government Personnel – This may include performance monitors, inspectors, technical experts, workforce service division and center directorate level security personnel, requirements office, or others (i.e. users) who provide information that helps the COR monitor contractor performance.

4. Primary Method of Surveillance:

Periodic Inspection: These services are monitored weekly, monthly, quarterly, semiannually, annually, etc. Periodic types of activities are perfect for periodic inspection because not only are they infrequent, but there is normally a predetermined, specified time frame within which the tasks must be accomplished. Therefore, you know exactly when to conduct the evaluations. Periodic inspections automatically become 100 percent evaluations or "100 percent checks." Inspections should be divided and scheduled by frequency: annual, semiannual, quarterly, monthly, weekly and as required. Sometimes services are required for which the time or frequency cannot be predicted, such as accident investigations, one-time special tasking by higher headquarters, etc. These would be labeled "as required inspections." Others are known and predictable such as the quarterly status report or the monthly travel report currently included in some DISA service contracts.

Surveillance Matrix

The Surveillance Matrix (Appendix 1) is the list of performance objectives and standards that must be performed by the contractor. This matrix details the method of surveillance the COR will use to validate and inspect these performance elements. Inspection of each element will be documented in the COR file.

Performance objectives define the desired outcomes. Performance Standards define the level of service required under the contract to successfully meet the performance objective. The inspection methodology defines how, when, and what will be assessed in measuring performance. If the due date for a deliverable falls on a Federal Holiday, then the deliverable could be delivered on the holiday or the following working day. The Government performance surveillance, using this QASP, to determine the quality of the contractor’s performance as it relates to the performance element standards. The AQL performance Standard Table should be used to form the foundation of the COR’s inspection checklist.

5. Performance Standards:

Task

Performance Standard

Acceptable Quality Level (AQL)

Method of

Surveillance

6.1.2 –

Contract Management

Contract Management Plan

Materials contain required information and are delivered on time at least 95% of the time. Revisions that occur are minor and resolved in timely manner.

Each deliverable will be re-viewed

6.1.5 –

Weekly status Reports

The Final WSR shall be delivered no later than 9:00 AM every Monday.

Materials contain required information and are delivered on time at least 95% of the time. Revisions that occur are minor and are resolved in a satisfactory manner.

Each WSR will be reviewed.

6.1.6 –

Configuration Management

Configuration Management Strategy

Apply documented configuration management processes 95% of the time

Each deliverable will be reviewed

6.1.9 –

User Assistance and Familiarization Training

User Assistance Training

Delivered on time at least 95% of the time. Problems that occur are minor and resolved in a satisfactory manner.

Each deliverable will be reviewed

6.2.1 –

Integrated Master Schedule (IMS)

The schedule shall be baselined and further changes cannot be made without concurrence from the Government. Schedule slips shall be communicated within three (3) working days following the identification of the change. This notification shall include risks, impacts, probability of occurrence and potential schedule recovery mechanisms. The contractor shall identify and propose a mitigation plan for each risk and issue identified.

Materials contain required information and are delivered on time at least 95% of the time. Revisions that occur are minor and are resolved in a satisfactory manner.

Each notification of IMS schedule slip will be reviewed.

6.2.2 –

Software Release Plan (SRP)

Any anticipated changes to the SRP shall be communicated within three (3) working days following the identification of the change and submitted along with the changes of the IMS.

Materials contain required information and are delivered on time at least 95% of the time. Revisions that occur are minor and are resolved in a satisfactory manner.

Each notification of anticipated Software Release Plan change will be reviewed.

6.2.5 –

Software Developmental Life Cycle (SDLC) Plan

SDLC

Apply full SDLC processes 95% of the time

Each deliverable will be reviewed

6.3 –

Design and Architecture

Design Reviews and Design Documentation

Materials contain required information and are delivered on time at least 95% of the time. Performance occurs with no required re-performance or re-work at least 80% of the time. Revisions that occur are minor and resolved at a timely manner.

Each deliverable will be reviewed.

6.5.5 –

Functionality of Software Delivered

Requirements/functionality allocated to a release shall be delivered in accordance with the SRP.

95% AQL.

The percentage of functionality of the release will be calculated by dividing the number of requirements/functions in the release against the number of functions planned in the SRP.

Test – Functionality will be verified during Government Testing

6.5.5 –

New Defects in Software Re-Delivered

Any patches or re-deliveries to resolve issues in software already operationally fielded shall be free of any new defects.

0 Priority 1 defects

0 Priority 2 defects

≤ 50 Priority 3 defects in total backlog

≤ 50 Priority 4 defects in total backlog

Test – To be verified during Government Testing

6.5.6 –

Operational

Testing

The final version of JPES shall be delivered IAW the IMS.

100% AQL

Inspection

6.6 – Deployment Documentation

The contractor is required to provide all deployment documentation deliverables IAW the PWS.

Materials contain required information and are delivered on time at least 95% of the time. Revisions that occur are minor and are resolved in a satisfactory manner. Performance occurs with no required re-performance or re-work at least 80% of the time. Problems that occur are minor and are resolved in a satisfactory manner.

Each deliverable will be reviewed.

6.6 –

Deployment Support

The operational JPES shall meet or exceeds the functionality and performance of the legacy JOPES IT system and the KPPs of the CDP.

100% of the functionality and performance of the legacy JOPES and KPP’s.

To be verified during Government Testing

6.6.1 –

Software Delivery

Each Software Release shall be delivered on time in accordance with the IMS and SRP.

Meets: < Delivery date

Unsat: > 3 days after Delivery date

Inspection

6.7.2 –

Tier 3 helpdesk support

Tier 3 help desk shall provide initial triage and response to issues as follows:

Priority 1: ≤ 2 hours

Priority 2: ≤ 2 hours Priority 3 : ≤ 48 hours Priority 4 : ≤ 48 hours Priority 5 : ≤ 48 hours

Performance occurs with no required re-performance or re-work at least 95% of the time. Problems that occur are minor and are resolved in a satisfactory manner

Weekly inspection of deliverable products and services

6.7.3 –

Software Maintenance

The Contractor shall provide no less than one hundred and eighty (180) days notification to the Government on all forthcoming license renewal and COTS product upgrade requirements.

Materials contain required information and are delivered on time at least 95% of the time. Revisions that occur are minor and are resolved in a satisfactory manner.

Each notification of all forthcoming license renewal will be reviewed.

6.8.2 –

Information Assurance (IA) Vulnerability Notices

The Contractor shall review IA Vulnerability notices and notify the Government whether or not a vulnerability is applicable to the JPES application within 3 days of the notice being released. If the vulnerability is applicable, the Contractor’s response shall indicate to the Government whether or not the vulnerability can be fixed in accordance with the timeframes in the DISA Vulnerability Management Policy memo.

Materials contain required information and are delivered on time at least 95% of the time. Revisions that occur are minor and are resolved in a satisfactory manner.

Each notification of IA Vulnerability Notice will be reviewed.

6. Documenting Performance:

Acceptable Performance.

The Government shall document positive performance. A Performance Assessment Report (PAR) template is attached (Appendix 2). Any report may become part of the supporting documentation for fixed price payments, award/incentive fee payments, Contractor Performance Assessment Reporting System (CPARS) entries, or other actions. Government acceptance of services permits the contractor to invoice for the service fee.

Unacceptable performance.

Unacceptable contractor performance can be identified by customer complaints (Appendix 3) or upon schedule surveillance evaluations. When unacceptable performance occurs, the COR shall inform the contractor. This will normally be in writing unless circumstances necessitate verbal communication. The COR shall document the discussion and place it in the COR file and provide a copy to the KO.

When the COR determines formal written communication is required, the COR shall prepare a Corrective Action Report (CAR). The CAR is forwarded to the KO for dissemination to the contractor. A CAR template is attached to this QASP (Appendix 4).

The contractor shall acknowledge receipt of the CAR in writing. The CAR will specify if the contractor is required to prepare a corrective action plan to document how the contractor shall correct the unacceptable performance and avoid a recurrence. The CAR will also state how long after receipt the contractor has to present this corrective action plan to the KO and COR. The Government may withhold payment for services until corrective action is taken. The Government shall review the contractor's corrective action plan to determine acceptability. Upon Government acceptance of the corrected services the contractor will be permitted to invoice for the service fee.

Any CAR may become a part of the supporting documentation for contract payment deductions, fixed fee deductions, award fee nonpayment, CPARS or other actions deemed necessary by the KO.

7. Mandatory or regulatory compliance items:

The contractor will provide initial proof of compliance with regulatory and compliance items contained in the contract. The Contracting Officer's Representative (COR) and the Contracting Officer (KO) will evaluate the initial submission and will conduct ad hoc reviews of the contractor's compliance throughout the remainder of the contract. The contractor will monitor compliance as required by Federal Acquisition Regulation (FAR) 52.222-50, Combating Trafficking in Persons, and report any issues to the KO.

8. Acceptable Quality Level (AQL):

The AQL’s for this project are listed in Section 5 of this QASP under the third column of the table “Acceptable Quality Levels”. The AQL’s are represented in percentages as well as meets/does not meet timeliness criteria, quality, and levels of reliability.

9. Evaluation Method:

The COR will evaluate deliverables IAW the Performance Standards found in Section 5 of this QASP under the fourth column of the table “Method of Surveillance”. The COR will check that deliverables are turned in on time and will rely on Subject Matter Expert (SME) to verify tests and inspections. All deliverables will be evaluated for depth and breadth of technical discussion and how they meet the requirements of the PWS.

10. Performance Rating:

In evaluating the quality of contractor’s performance, the following performance ratings may be used (same as CPAR ratings).

Performance

Rating

Criteria

Exceptional

Performance meets contractual requirements and exceeds many to the government’s benefits. The contractual performance of the element or sub-element being assessed was accomplished with no problems and contractor actions were highly effective.

Very Good

Performance meets contractual requirements and exceeds some to the government benefits. The contractual performance was accomplished with few minor problems for which corrective actions taken by the contractor were effective.

Satisfactory

Performance meets contractual requirements. The contractual performance contains some minor problems for which corrective actions taken by the contractor were satisfactory.

Marginal

Performance does not meet some contractual requirements. The contractual performance reflects a serious problem for which the contractor has not yet identified corrective actions. The contractor’s proposed actions appear marginally effective or were not fully implemented.

Unsatisfactory

Performance does not meet most contractual requirements and recovery is not likely in a timely manner. The contractual performance contains serious problem(s) for which the contractor’s corrective actions appear or were ineffective.

Appendix 1 - Surveillance Matrix

Performance

Standards

Acceptable Quality

Levels

Methods of

Calculation

Surveillance

Results/Ratings

Contract Management Plan

Target Criteria:

Materials contain required information and are delivered on time 99% of the time.

Minimum Acceptable:

Materials contain required information and are delivered on time at least 95% of the time. Revisions that occur are minor and resolved in timely manner.

Calculation:

Each deliverable will be reviewed.

Measurement Interval:

First deliverable is due within 30 calendar days after award. Subsequent updates required when new task order is awarded.

Measurement Period:

The Measurement Period begins at contract award and ends the last day of the contract.

Source of Measurement Data:

Deliverables will be submitted by the contractor.

Method of Surveillance:

Period inspection.

Timing of Measurement:

Measurements are taken within thirty (30) calendar days of the deliverable due date.

Reporting Frequency:

Reporting to commence upon Contract Award and continue throughout the Period of Performance. Reporting of QASP attainment to occur semi-annual during the first year and annually thereafter.

Performance ratings and associated criteria defined above in section 10 will be used to develop an overall rating for this objective.

The Final Weekly Status Report (WSR) shall be delivered no later than 9:00 AM every Monday.

Target Criteria:

Materials contain required information and are delivered on time at least 95% of the time. Revisions that occur are minor and are resolved in a satisfactory manner.

Calculation:

Each WSR will be reviewed.

Measurement Interval:

Weekly, on 1st workday

Measurements are taken within (7) calendar days of the deliverable due date.

Reporting to commence upon Contract Award and continue throughout the Period of Performance. Reporting of QASP attainment to occur semi-annual during the first year and annually thereafter.

Performance ratings and associated criteria defined above in section 10 will be used to develop an overall rating for this objective.

Configuration Management Strategy

Target Criteria:

Apply documented configuration management processes 99% of the time.

Apply documented configuration management processes at least 95% of the time

Calculation:

Measurement Interval:

First deliverable is due within 60 calendar days after award. Subsequent updates required annually.

Measurements are taken within thirty (60) calendar days of the deliverable due date.

Reporting to commence upon Contract Award and continue throughout the Period of Performance. Reporting of QASP attainment to occur semi-annual during the first year and annually thereafter.

Performance ratings and associated criteria defined above in section 10 will be used to develop an overall rating for this objective.

User Assistance Training

Target Criteria:

Delivered on time 99% of the time.

Delivered on time at least 95% of the time. Problems that occur are minor and resolved in a satisfactory manner.

Calculation:

Measurement Interval:

Deliverables are due as scheduled in the Integrated Master Schedule (IMS), 3 calendar days after delivery. Frequency is estimated to be quarterly.

Measurements are taken within three (3) calendar days of the deliverable due date.

Reporting to commence upon Contract Award and continue throughout the Period of Performance. Reporting of QASP attainment to occur semi-annual during the first year and annually thereafter.

Performance ratings and associated criteria defined above in section 10 will be used to develop an overall rating for this objective.

The schedule shall be baselined and further changes cannot be made without concurrence from the Government. Schedule slips shall be communicated within three (3) working days following the identification of the change. This notification shall include risks, impacts, probability of occurrence and potential schedule recovery mechanisms. The contractor shall identify and propose a mitigation plan for each risk and issue identified.

Target Criteria:

Materials contain required information and are delivered on time at least 95% of the time. Revisions that occur are minor and are resolved in a satisfactory manner.

Calculation:

Each notification of IMS schedule slip will be reviewed.

Measurement Interval:

First deliverable is due within 30 calendar days after award. Subsequent updates required weekly by close of business on Fridays.

Measurements are taken within thirty (30) calendar days of the first deliverable due date and seven (7) calendar days of weekly updates.

Reporting to commence upon Contract Award and continue throughout the Period of Performance. Reporting of QASP attainment to occur semi-annual during the first year and annually thereafter.

Performance ratings and associated criteria defined above in section 10 will be used to develop an overall rating for this objective.

Any anticipated changes to the SRP shall be communicated within three (3) working days following the identification of the change and submitted along with the changes of the IMS.

Target Criteria:

Materials contain required information and are delivered on time at least 95% of the time. Revisions that occur are minor and resolved in timely manner.

Calculation:

Each notification of anticipated Software Release Plan change will be reviewed.

Measurement Interval:

First deliverable is due within 30 calendar days after award. Subsequent updates required ten (10) calendar days after each release.

Measurements are taken within thirty (30) calendar days of the initial deliverable due date and ten (10) calendar days of updates after each release.

Reporting to commence upon Contract Award and continue throughout the Period of Performance. Reporting of QASP attainment to occur semi-annual during the first year and annually thereafter.

Performance ratings and associated criteria defined above in section 10 will be used to develop an overall rating for this objective.

Software Developmental Life Cycle (SDLC) Plan

Target Criteria:

Apply full SDLC processes 99% of the time.

Apply full SDLC processes at least 95% of the time

Calculation:

Measurement Interval:

First deliverable is due within 30 calendar days after award. Updates are required annually.

Reporting to commence upon Contract Award and continue throughout the Period of Performance. Reporting of QASP attainment to occur semi-annual during the first year and annually thereafter.

Performance ratings and associated criteria defined above in section 10 will be used to develop an overall rating for this objective.

Design Reviews and Design Documentation

Target Criteria:

Materials contain required information and are delivered on time 99% of the time. Performance occurs with no required re-performance or re-work 90% of the time.

Materials contain required…