Attachment_10_QASP_Appendix_A_-_Deliverables_SS_Table.docx

Surveillance Documents.

Deliverables. All deliverables will be reviewed for timeliness, accuracy and format. If a deliverable due date falls on a weekend or holiday, the Contractor shall submit the deliverable on the last work day prior to the due date.

TABLE 1 – DELIVERABLES

Updates: The Contractor shall provide any changes or updates to the Government PMO prior to Contractor implementation.

Data Accession List

Reference: PWS paragraph 2.1.9 Details: Contractor provides quarterly listing of documents provided to the government. At a minimum the data accession list shall identify: task order number, PWS task, document title, document description, date provided to the government, and acknowledgement of government purpose rights..

Updates: Quarterly per above.

· What: Reviews Data Accession List

· When: Upon receipt NLT the 15th of the Month after the end of the contractual quarter

· How: SAF/FMF FIAR PM will review the Contractor’s provided information via e-mail and resolve and inconsistencies, etc. as needed.

Details:

· Government PM responsible for setting up meeting

Updates: The Contractor shall disclose new teaming partners/ subcontractors prior to that entity’s participation in response to a task order solicitation. The Contractor shall coordinate with the CO prior to adding or deleting subcontractors from the prime’s subcontractor list. The Contractor shall provide timely notification to the CO of any acquisitions or mergers that impact this contract. DD254 will need to be updated accordingly.

· What: Reviews the Contractor provided Subcontractor List

· When: w/in 10 days of contract award, updates provided when adding/deleting Subs, or if any acquisitions or mergers occur

· How: notification will occur via e-mail. Contractor submissions will receive a receipt acknowledgement from the SAF/FMF FIAR Support PM

Details: Provide quarterly report detailing number of personnel per task order, per task (Task 1, Task 2, etc) by labor category.

The Contractor shall provide the Quarterly Staffing Report every Quarter on 15th of the month after the end of the quarter

· Who: Task Order CORs

· What: Reviews Quarterly Staffing Report

· When: Upon receipt NLT the 15th of the Month after the end of the quarter

· How: Task Order CORs will review the Contractor’s provided information via e-mail.

Individual and/or Summary Subcontracting Report (ISR/SSR).

Due Date: The Contractor shall submit ISR/SSR using the Electronic Subcontracting Reporting System (eSRS) at http://www.esrs.gov) IAW FAR 19.704(a)(10)(iii)(A) and guidance in eSRS.

· Who: AFDW FIAR Support CM and PM What: Reviews Individual and/or Summary Subcontracting Report (ISR/SSR)

· When: Upon receipt

· How: AFDW FIAR Support CM will log in and review the Contractor’s provided information. The PM discusses the information with the CO and CO will take the necessary actions to resolve and questions/inaccuracies

· What: Reviews Contractor Labor Hours Report (eCMRA)

· When: Upon receipt; Contractors are required to input data by 31 October of each year.

· How: AFDW FIAR Support CM will log in and review the Contractor’s provided information. The PM discusses the information with the CO and CO will take the necessary actions to resolve and questions/inaccuracies

· Verbal and electronic notification identifying the nature of the issue to the COR within three (3) business hours of identification.

· Written recommendations and corrective actions required to meet deliverable milestones/timelines within eight (8) business hours of identification, unless otherwise approved in writing by the applicable COR.

· Who: SAF/FMF FIAR Support PM

· What: Contractor-identified concern

· When: Upon receipt

· How: SAF/FMF FIAR Support PM acknowledges receipt of the issue via e-mail. The PM reviews the issue and discusses the impact(s) with the appropriate Task Order CORs, and Contractor as appropriate and takes the necessary actions to resolve/mitigate the impact

1. Scoping and Planning Activities Report

2. Document Testing Report

3. Reporting and Monitoring Results

Reference: PWS Paragraph 2.2.1.8

1. Scoping and Planning Due Date: Annually on August 31st

2. Document and Testing Report Due Date: IAW Scoping and Planning Project Plan

3. Reporting and Monitoring Due Date: Annually on June 30th

· Who: Task Order COR

· What: Contractor-identified concern

· How: Task Order COR acknowledges receipt of the issue via e-mail. The PM reviews the issue and discusses the impact(s) with the appropriate Task Order CORs, and Contractor as appropriate and takes the necessary actions to resolve/mitigate the impact

· What: Accepts completed Workstream Summary Report(s)

· When: Upon receipt (NLT COB Eastern Time)

· How: COR and FIAR Support PMO acknowledge receipt of the report via e-mail. The COR reviews report for accuracy and accepts on behalf of the Government via e-mail

Details: Detail trip scope, actions/findings, resources used, and provide a summary to leadership

· What: Accepts completed Trip Reports

· When: Upon receipt (NLT five business days after the Contractor returns from travel)

· How: FIAR Support PMO and COR acknowledges receipt of the report via e-mail. The COR reviews Trip Report for accuracy and accepts on behalf of the Government via e-mail

2.3.

Details: Report detailing CAPs information for the previous month

· How: COR acknowledge receipt of the report via e-mail. The COR reviews report for accuracy and accepts on behalf of the Government via e-mail

2.3.2.3

· How: COR acknowledge receipt of the report via e-mail. The COR reviews report for accuracy and accepts on behalf of the Government via e-mail

2.1.7

· How: COR acknowledge receipt of the report via e-mail. The COR reviews report for accuracy and accepts on behalf of the Government via e-mail

· How: COR acknowledge receipt of the report via e-mail. The COR reviews report for accuracy and accepts on behalf of the Government via e-mail

· How: COR acknowledge receipt of the report via e-mail. The COR reviews report for accuracy and accepts on behalf of the Government via e-mail

· How: COR acknowledge receipt of the report via e-mail. The COR reviews report for accuracy and accepts on behalf of the Government via e-mail

· How: COR acknowledge receipt of the report via e-mail. The COR reviews report for accuracy and accepts on behalf of the Government via e-mail

Update: Monthly Status Report – PEO Directorate shall address all areas listed in Section 2.7 and sub paragraphs in a format proposed by the Contractor and agreed upon by the Government

· What: Accepts completed monthly status report

· How: Via Email

Monthly Valuation Packages shall address all areas listed in para 2.7.10.7 i

· What: Accepts completed monthly status report

· How: Via Email

· What: Accepts completed monthly status report

· How: Via Email

· What: Accepts completed annual attestation letters

· How: Via Email

· How: Via Email

FA7014-17-R-5002

FIAR

Quality Assurance Surveillance Plan Appendix A

Services Summary. The Services Summary is a list of critical tasks that must be performed by the Contractor, the performance threshold, and the method of surveillance the COR will use to validate/inspect these tasks. Inspection of each task will be documented in the COR folder.

TABLE 2 – SERVICES SUMMARY

a) Qualified key personnel are on the job at contract start, unless previously negotiated by the CO.

b) Vacancies are filled with qualified key personnel within 21 calendar days of vacancy, unless approved in writing or otherwise directed in advance by the CO AND

c) There is no mission impact due to position vacancies or unqualified personnel.

The following acceptable criteria and thresholds are also used for annual CPAR evaluation.

Contract Start:

Minimum acceptable criteria is based on having key personnel in place timely at contract (PoP) start.

Thresholds are as follows:

-- SATISFACTORY: 100% Key personnel in place within first 30 days of PoP start -- VERY GOOD: 100% Key personnel in place within first 15 days of PoP start -- EXCELLENT: 100% Key personnel in place within first 5 days of PoP start

· Who: Primary COR

· What: Reports vacancies, tracks vacancies/fill actions; reports non-conformances

· Where: Reports Contractor personnel vacancies and fill actions to the CO/CM; notifies the CO/CM and the QAPC of non-conformances

· When: Reports contractor personnel vacancies to the CO/CM upon personnel departure; notifies the CO/CM when the position is filled with a qualified person; notifies the CO/CM/QAPC of non-conformance if vacancies exceed 21 calendar days or impact mission

· How: All vacancy, fill action, and non-conformance notifications will be sent by e-mail to usaf.jbanafw.afdw-staff.mbx.afdw-fiar-follow-on-contract@mail.mil. Monthly, COR updates/annotates vacancies and any non-conformance as a result via monthly QA Report—any non- conformance issued is also reported in Wide Area Work Flow WAWF where appropriate

Vacancies (all include with “no impact to mission”):

Minimum acceptable criteria is based on filing vacancies timely for key personnel.

Thresholds are as follows:

-- SATISFACTORY: Key personnel vacancy filled with qualified personnel NLT 21 days, and no gov’t concerns expressed regarding qualification of personnel at the time of rating -- VERY GOOD: All vacancies (not just key personnel) filled with qualified personnel NLT 21 days, and no gov’t concerns expressed regarding qualification of personnel at the time of rating -- EXCELLENT: Key personnel vacancy filled with qualified personnel NLT 10 days, and no gov’t concerns expressed regarding qualification of personnel at the time of rating

Workload balanced effectively, to include minimized use of subcontractors, resulting in minimal negative mission impact

The following acceptable criteria and thresholds are also used for annual CPAR evaluation.

Workforce stability Minimum acceptable criteria is based on the lapse rate for key positions.

Thresholds are as follows:

-- SATISFACTORY: 5-7% quarterly lapse rate on key positions -- VERY GOOD: 3-5% quarterly lapse rate on key positions -- EXCELLENT: <3% quarterly lapse rate on key positions

Who: Primary: AF FIAR Support PM

· Secondary: QAPC, Lead COR

· What: Review Lapse Rate report for accuracy

· Where: Documented in Tab 9 of the COR Folder as defined in Table 5 of the QASP

· When: Quarterly

· How: QAPC and PM compare Contractor provided position vacancy details against published lapse rate threshold of less than 5% during the reporting period for compliance

a) 100% of deliverable requirements are met and received on time

AND

b) Critical information is accurate.

· Who: Primary COR/applicable PM

· What: Inspects, validates, and accepts Table 1 deliverables

· Where: Documented in Tab 9 of the COR Folder as defined in Table 5 of the QASP

· When: On the scheduled times as defined in QASP Para 3.2.1.

· How: Verification and acceptance of all required deliverables will be communicated to the Contractor via e-mail and annotated in the COR’s Monthly Report.

· What: Validate accuracy of Contractor’s Small Business Participation as reported in the electronic Subcontracting Reporting System (ESRS)

· Where: ESRS report found at: www.esrs.gov

· When: Semiannually 31 Mar/30 Sep

· How: Within 30 days of the due dates IAW FAR 52- 219(l)(1)(i), the QAPC validates the SB plan was submitted, and CM validates the accuracy, to include meeting small business goals of 26% or greater IAW Contractor Subcontracting Plan

Provide sufficient strategic development for alternative and future process solutions.

Performance acceptable when no more than 25% deviation from the standard.

Performance acceptable when 75% of recommended CAPs are designed effectively to include evidence of proper analysis and identification of any future/target state impacts and assessments and corrective actions related to business process re-engineering. Evidence of adequate coordination with business process and systems owners, and related stakeholders.

The following acceptable criteria and thresholds are also used for annual CPAR evaluation.

Minimum acceptable criteria include the following quality components:

--Sound root cause analysis with justification --Countermeasures clearly address root cause --Measurable results --Identifies parties responsible for execution; address stakeholder issues; comm plan for buy-in/roll out --Realistic and timely Implementation plan (achievable given govt/AF constraints) Minimum acceptable criteria include the following time component:

- Remediation plan with quality components developed within 6 weeks after NFR issued by the IPA Thresholds for CPARS evaluation are as follows:

SATISFACTORY: 75% of CAPS meet quality standards/accepted by Govt within 6 weeks of signed NFR -VERY GOOD: 85% of CAPS meet quality standards/accepted by Govt within 6 weeks of signed NFR –EXCELLENT: 95% of CAPS meet quality standards/accepted by Govt within 6 weeks of signed NFR

· Who: Government business process area/task lead.

· What: Validate CAP steps and planned implementation timeline, related business processes (legacy and target environments) and re-engineering assessment.

· Where: As documented in monthly CAP report.

· When: Within 6 weeks of NFR. As retesting/validation occurs internally or by external audit entities. Government business process/task lead should perform assessment within 30 days of IPA or external auditor related finding.

How: 100% review of CAP and related strategic analysis documentation. Reported to the COR, in a written documented fashion, on at least a bi-weekly basis when performance thresholds are not met.

Provide sufficient strategic development for alternative and future process solutions.

Performance acceptable when no more than 25% deviation from the standard.

Performance acceptable when 75% of implemented/closed CAPs that are retested by the IPA auditor or other external audit entity do not identify the same control or process failures to the extent the proper root cause and corrective action was not identified (taking into consideration no new processes were developed).

· Who: Government business process area/task lead.

· What: Validate CAP documentation, and corrective action information.

· Where: As documented from follow-on IPA auditor’s external auditor’s testing and documentation.

· When: As retesting/validation occurs. Government business process/task lead should perform assessment within 30 days of IPA or external auditor related finding.

How: 100% review of CAP and related strategic analysis documentation. Reported to the COR, in a written documented fashion, on at least a bi-weekly basis when performance thresholds are not met.

Performance acceptable when 95% of NFRs and CAPs are accurately reflected in recurring internal and external NFR and CAP reporting trackers.

· Who: Government business process area/task lead.

· What: Validate NFR and CAP reporting tracker.

· Where: As documented in monthly CAP reporting tracker.

· When: Monthly.

· How: 100% review of CAP reporting tracker. Reported to the COR, in a written documented fashion, on at least a bi-weekly basis when performance thresholds are not met.

Developing and Maintaining Population Completeness Universe of Transactions and Manual Reconciliations (Population Completeness)

Performance acceptable when 95% of population completeness documents are updated within 30 days of any known/key changes, to include business processes and systems capabilities/related requirements development and associated controls.

Performance is acceptable when documented strategies and approaches for data analytics/techniques are presented and initial deployment occurs within 30 days of population or reconciliation gaps.

· Who: Government business process area/task lead.

· What: Validate completeness of Population Completeness and Reconciliation Strategy and Analysis documentation.

· Where: As documented in monthly CAP report (information should be reported as a significant deficiency and identify related CAP development).

· When: First review within 1 month of known population completed or reconciliation gap.

· How: 100% review of CAP and related strategic analysis documentation. Reported to the COR, in a written documented fashion, on at least a bi-weekly basis when performance thresholds are not met.

Provide functional, technical and administrative support services necessary to assist with the development/evaluation of business process requirement policies/processes, system requirements, control requirements, test plans, and mitigation plans for inclusion in updating cycle memos, risk control matrices, and Air Force policy/process requirements (Enduring Audit Readiness-A-123 and Entity Level Controls).

Retained refinement of products, strategy, communication, and training with SAF/FM and office of primary responsibilities on a recurring basis that documents the assessment/evaluation of existing practices, capabilities, policies, best practices, and deficiencies.

Products and strategies must demonstrate compliance with/impacts of noncompliance with Federal Manager’s Financial Integrity Act of 1982, OMB A-123, Federal Financial Management Improvement Act of 1996 and GAO Standards for Internal Controls in the Federal Government Performance acceptable when no more than 5% deviation from the standard.

· Cycle memo document updates are made within 15 days of process and control changes/gaps being identified: properly document the current state environment to include complete identification of control gaps

· Test Plans are provided at least 7 days prior to test date and include detailed risk based assessment information

· Test results metrics: Metrics are provided within 30 days of testing

· Annual Statement of Assurance: accurately reflects all material weakness and significant deficiencies as reported during the SOA period

· Training: adequate and timely coverage of new/revised control assessment/testing approaches, allowing for process owners and test managers to incorporate into annual testing cycle

· Who: Government business process area/task lead.

· What: Validate documented analysis/outcomes of strategy/development/training forums/test plans of the related area.

· Where: During recurring bi-weekly reviews of work stream efforts.

· When: bi-weekly

· How: Review a representative sample of documentation and any known issues/concerns reported to the COR, in a written documented fashion, when performance thresholds are not met.

· Memorandums of agreement/understanding, to include related control information, are updated in “draft ready for coordination” form within 30 days of receiving updates/new controls from service providers. The updated MOAs/MOUs should reflect necessary updates for specific roles and responsibilities and reliance/dependencies (to include specifically addressing related CUECs).

· Who: Government business process area/task lead.

· What: MOU/MOA documentation, and control catalogs

· Where: As documented in MOU/MOA and related control information

· When: During recurring bi-weekly reviews of work stream efforts.

· How: Review a representative sample of documentation. Reported to the COR, in a written documented fashion, on at least a bi-weekly basis when performance thresholds are not met.

· Who: Government business process area/task lead.

· What: Monitoring Reports.

· Where: As documented in monitoring reports

· When: During recurring bi-weekly reviews of work stream efforts.

· How: Review a representative sample of documentation. Reported to the COR, in a written documented fashion, on at least a bi-weekly basis when performance thresholds are not met.

· Who: SAF/FMF government managers and/or program managers.

· What: Meeting minutes documentation from each bi-weekly meeting. Action plans resulting for risk areas and potential issues addressed at related meetings.

· Where: Bi-weekly management meeting.

· How: 100% assessment and review of meeting minutes in comparison to management’s known related issues or areas addressed as concerns by management/others. Issues or concerns regarding performance thresholds not being met are reported to the primary COR in a written documented fashion.

Facilitate proper and timely communication with auditors and stakeholders.

(IPA Audit Liaison Support)

Performance acceptable when 95% of communication to MAJCOMs and auditors is provided within 24 hours of receipt of auditor/stakeholder requests.

Performance acceptable when 95% of audit response tool requirements meet stated objectives of the requirements.

Performance acceptable when corresponding back-up plans for any known functionality issues with automated means/tools are initiated within 24 hours, to include communication of the back-up plan to all stakeholders.

· Who: Government business process area/task lead.

· What: Validate timeliness of response out to stakeholders. Validate audit response tool requirements analysis and tool functionality. Assessment of back-up plan initiation.

· Where: weekly reviews of audit requests. Periodic reviews of audit response tool functional requirements, system development and testing documentation.

· When: Within 2 weeks of auditor requests and completion of functional operational capabilities testing.

· How: 100% review of documentation. Reported to the COR, in a written documented fashion, on at least a bi-weekly basis when performance thresholds are not met.

“Ready for review draft” system change requests identifying key control changes are established within 30 days of identifying control gaps and related necessary systems changes.

Within 30 days of known business process changes impacting IT systems/ initiatives, documents the necessary control requirements and related key information to support the service development and delivery.

· Who: Government business process area/task lead.

· What: System change requests and SDDP process initiative documents

· Where: As documented in critical IT system change information and new IT system initiatives

· When: During recurring bi-weekly reviews of work stream efforts.

· How: Review a representative sample of documentation. Reported to the COR, in a written documented fashion, on at least a bi-weekly basis when performance thresholds are not met.

· What: Accepts completed Monthly Status Report(s)

· When: Upon receipt (NLT COB Eastern Time)

· How: COR and FIAR Support PMO acknowledge receipt of the report via e-mail. The COR reviews report for accuracy and accepts on behalf of the Government via e-mail

Details: Detail trip scope, actions/findings, resources used, and provide a summary to leadership

· What: Accepts completed Trip Reports

· When: Upon receipt (NLT five business days after the Contractor returns from travel)

· How: FIAR Support PMO and COR acknowledges receipt of the report via e-mail. The COR reviews Trip Report for accuracy and accepts on behalf of the Government via e-mail