Attachment_B_-_QASP.docx

Attachment B- Quality Assurance Surveillance Plan (QASP) Department of Education Advisory and Assistance Services For Financial Management and Related Business Process Analysis and Internal Control Work

1. Purpose

1. Performance Management Approach

1. Government Resources Roles and Responsibilities

1. Methodologies to Monitor Performance

1. Quality Assurance Reporting

1. Analysis of Quality Assurance Monitoring Results

1. Failure to Perform

1. Appendix I – Required Performance Metrics (RPM) Table

1. Appendix II – Quality Assurance Monitoring Form

1. Purpose: The purpose of the Quality Assurance Surveillance Plan (QASP) is to describe the systematic methods used to measure performance and to identify the reports required and the resources to be employed. The QASP provides a means for evaluating whether the contractor is meeting the performance standards identified in the Performance Work Statement (PWS).

This QASP is designed to define roles and responsibilities, identify the performance objectives, define the methodologies used to monitor and evaluate the contractor’s performance, describe quality assurance reporting, and describe the analysis of quality assurance monitoring results.

2. Performance Management Approach: This QASP will define the performance management approach taken by the Department to monitor, manage, and take appropriate action on the contractor’s performance against expected outcomes or performance objectives communicated in the task order. The data generated in a performance management approach provides information that indicates whether or not expected outcomes for required services are being achieved adequately by the contractor.

The contractor’s internal quality control system will set forth the staffing and procedures for self-inspecting the quality, timeliness, responsiveness, and other performance requirements in the task order. The contractor will utilize its internal quality control system to assess and report their performance to the Contracting Officer’s Representative (COR).

The COR will monitor performance and review performance reports furnished by the contractor to determine how the contractor is performing against communicated performance objectives. The Government will make decisions based on performance measurement metric data and notify the contractor of those decisions. The contractor will be responsible for making required changes in processes and practices to ensure performance is managed effectively.

3. Government Resources Roles and Responsibilities:

The Contracting Officer (CO) is responsible for monitoring contract compliance, contract administration and cost control; and resolving any differences between the observations documented by the COR and the contractor’s performance.

The CO will designate one COR as the Government authority for performance management. The number of additional representatives serving as Technical Inspectors depends upon the complexity of the services measured as well as the contractor’s performance.

The COR is responsible for monitoring, assessing, communicating the technical performance requirements to the contractor and assisting the contractor. The COR will have the responsibility for completing Quality Assurance (QA) monitoring forms used to document the inspection and evaluation of the contractor’s work performance. Government surveillance may occur under the Inspection of Services clause for any service relating to the contract.

The Project Manager (PM) provides detailed technical oversight of the contractor’s performance and reports his/her findings to the COR in a timely, complete, and impartial fashion to support the COR’s administrative activities. The PM may serve as a direct conduit to provide Government guidance and feedback to the contractor on technical matters but is not empowered to make any contractual commitments or authorize any contractual changes on the Government’s behalf.

4. Methodologies to Monitor Performance

In an effort to minimize the contract administration burden, simplified methods of surveillance techniques shall be used by the Government to evaluate contractor performance. The primary methods of surveillance are reports and customer input/feedback. The Government will use appointed representatives, as well as reports and input from users/customers as sources of comments on the contractor’s performance.

The acceptable quality levels for contractor performance, located in Appendix 1, are structured to allow the contractor to manage how the work is performed while providing negative incentives for performance shortfalls. For 6 of the activities, the acceptable performance level is established at 100%. The other levels of performance are established at percentages somewhat less than 100%. All are keyed to the relative importance of the task to the overall mission performance.

5. QA Reporting

The performance management feedback loop begins with the communication of expected outcomes. Performance standards are expressed in the task order and measured by the required performance metrics in Appendix 1. The Government’s QA monitoring, accomplished by the COR (and others as designated) will be reported using the monitoring form in Appendix II. The form, when completed, will document the COR’s understanding of the contractor’s performance under the contract to ensure that the task order requirements are being met. The COR will retain a copy of all completed QA monitoring forms.

6. Analysis of QA Monitoring Results

The Government shall use the observation methods cited to determine whether the acceptable quality levels have been met. At the end of each month, the COR will prepare a written report for the CO summarizing the overall results of the quality assurance monitoring of the contractor’s performance. This written report consists of the contractor’s submitted monthly progress report and the completed QA Monitoring Forms will become part of the QA documentation.

The CO may require the contractor’s PM, or a designated alternate, to meet with the CO and other Government personnel as deemed necessary to discuss performance evaluation. The COR will define a frequency of in-depth reviews with the contractor, however if the need arises, the contractor will meet with the CO as often as required or per the contractor’s request. The agenda of the reviews may discuss:

· Monthly performance measured by the metrics and trends

· Issues and concerns of both parties

· Projected outlook for upcoming months and progress against expected trend

· Recommendations made by the COR based on contractor information

· Issues arising from independent reviews and inspections

In addition to QA monitoring, the COR will use the information contained in the contractor’s monthly report to assess the contractor’s level of performance for each objective measured in this QASP (detailed in Appendix I). The COR must coordinate and communicate with the contractor to resolve issues and concerns of marginal or unacceptable performance. The contractor will discuss with the CO-COR satisfaction ratings receiving a “less than acceptable” rating. For such cases, the contractor should highlight its perspective on factors driving customer satisfaction and present plans to adjust service levels accordingly to bring the satisfaction rating up to an acceptable level.

The CO, COR and contractor should jointly formulate tactical and long-term courses of action. Decisions regarding changes to metrics, thresholds, or service levels should be clearly documented. Changes to service levels, procedures, and metrics will be incorporated as a contract modification at the convenience of the CO.

7. Failure to Perform

The Department may withhold payment or even terminate the contract based on failure to perform. The following criteria apply for determining appropriate action:

· Notifications. Consistent with FAR Part 49, the CO shall notify the service provider of failure to meet standards though QA monitoring forms, cure notices, or show cause notices and shall inform the service provider PM or designated alternate of such notices.

· Withholding. The Government has the right to withhold payment for any deliverable that fails to meet performance standards.

· Termination. If the CO determines that the contractor has failed to perform to the extent that a termination for default is justified, the CO shall issue a notice of termination, consistent with FAR Part 49.

APPENDIX I

REQUIRED PERFORMANCE METRICS (RPM) TABLE[footnoteRef:1] [1: Service/deliverables not meeting acceptable quality levels for one or more performance standards may impact future contract awards, such as a logical Follow-on Contract and/or additional task orders.]

Documentation (Business Process Cycle narratives, flow diagrams, key internal controls) submitted no later than the due dates in the agreed-upon schedule of deliverables Reports, inspections, and/or evaluations; Periodic status meetings

100% timely, accurate Reports, inspections, and/or evaluations

Reports are 95% timely Reports, flowcharts, inspections, and/or evaluations

· Building interfaces between Department and Treasury systems;

· Automating manual procedures and data interchange;

· Leveraging other Department-owned software and applications to improve and/or standardize financial management and related business activities;

· Identifying other tools used by other Federal agencies that can be implemented at Department; and

· Leveraging knowledge management and records management tools to move paper records into digital format.

· Supporting on-going efforts to either upgrade to Oracle financials version 12 or to migrate to a shared service provider in accordance with the Office of Management and Budget (OMB)/Treasury Office of Fiscal Innovation and Transformation (OFIT) requirements;

· Identifying potential business process, policy, system, and/or organizational impediments that may impact realizing the intended benefits of shared service solutions;

· Identifying opportunities for the Department to realign its business processes, organizational structure, or operating philosophy/internal policies to be more in line with the shared service provider model;

· Assessing ED’s current and planned use of shared services and making recommendations for improvement; and

· Evaluating the existing governance structure and service agreements (i.e. Memorandums of Understanding or MOUs) with shared service providers against applicable best practices and making recommendations for improvement.

· Areas of potential non-compliance with Federal laws and regulations and/or Department policies;

· Inefficient, over controlled, overly complex and/or manual processes;

· Missing KBAs and/or essential controls;

· Undefined or ill-defined process ownership responsibilities;

· Workflow and management oversight issues to include insufficient or unsuitable performance metrics to manage and monitor work;

· Lack of access to data, unstructured, or poorly defined data that hinders report preparation, analysis, and/or the ability of the Office of the Chief Financial Officer (OCFO) to produce quality work products, including financial reports;

· Over-reliance or dependency on other offices to perform key financial management business activities or to obtain financial management data;

· Resource issues, including staff capabilities and over or under staffing of critical financial related activities; and

· Organizational, legal, or policy impediments impacting the capability of Department staff to efficiently perform KBAs.

Analysis Reports are 100% timely.

Recommedations are well-researched, feasible and thorough Reports, flowcharts, inspections, and/or evaluations

· Identify areas where Department directives and other internal policies and procedures do not comply with applicable Federal laws and/or regulations and make recommendations for improvement;

· Identify potential legal or regulatory impediments contained in Department internal policies and procedures precluding effective and/or efficient financial operations and make recommendations to address them;

· Ensure that OCFO policies and procedures align with the end-to-end process mapping documents; and

· Identify opportunities to leverage technology to best align policy and process documentation within a Knowledge Management framework.

· Identifying opportunities to integrate pertinent KPIs and other metrics into the workflow of key business processes to help monitor performance;

· Benchmarking current performance against federal standards and best practices;

· Identifying opportunities to leverage technology to support performance management; and

· Helping to align FM operations to achieve the goals of the Government Performance Results Modernization Act (GPRMA) and align with the Department’s Strategic Plan, and underlying goals and objectives.

· Identifying unnecessary steps or inefficient work methods;

· Identifying the most efficient methods to perform key KBAs;

· Determining the number of staff and the appropriate skills and grade level required to perform KBAs effectively and efficiently;

· Identifying key positions and at risk of staff loss through attrition and developing strategies to address them;

· Determining the full range of skills needed to perform effectively in a more highly automated and complex “to-be” OCFO environment;

· Building a comprehensive career ladder progression plan that includes staff development and training plans to prepare staff for more complex financial management tasks; and

· Recommending suitable performance requirements and measurements for use in evaluating staff performance commensurate with grade.

· Identifying analytic requirements of OCFO internal and external stakeholders through customer surveys, benchmarking, and other means;

· Benchmarking other Federal organizations to identify leading practices related to financial analysis;

· Determining the data required to perform analysis across all major financial management business processes and sources and methods of data extraction;

· Performing specific data analytic related tasks and helping to transition the work to OCFO staff;

· Identifying appropriate tools and methods of performing financial analyses, to include making maximum use of existing Department analytic tools; and

· Determining how best to leverage the resources, data, and people from other Department offices, automated systems, and from shared service providers to support financial analysis requirements.

Analysis Reports are 95% timely.

Recommedations are well-researched, feasible and thorough Reports, flowcharts, inspections, and/or evaluations

· Taking an inventory of all financial reports mandated by external stakeholders, requested by internal stakeholders, or otherwise routinely produced by the OCFO and determining why the reports are being produced and who consumes them;

· Determining the financial reports produced by other Department offices, including the Office of the Chief Information Officer and Budget Services;

· Determining if any of the reports should no longer be produced or can be modified to provide more useful information;

· Assessing the efficiency of the processes used to compile reports and the accuracy of data reported;

· Through benchmarking, customer surveys, and other suitable methods, develop of listing of the current reports that should be sustained, other types of reports typically produced by OCFOs across the federal government, and reports Department customers indicated that they wanted;

· Determining the data sources and most efficient compilation methods to producing reports, including expanding the capacity of the staff to obtain the data and increase their capacity to produce ad-hoc reports across the multiple financial management business processes;

· Helping to improve the quality of the form, content and disclosures in the Annual Financial Report (AFR) and other statutory financial reports; and

· Improving results reflected in Treasury and other internal or external dashboards and scorecards.

· Development of a controls framework that incorporates Enterprise Risk Management (ERM) concepts such as the Committee of Sponsoring Organizations of the Treadway Commission (COSO) and International Organization of Standardization (ISO) standards, and/or applicable federal standards and identifying enterprise risks applicable to financial management and related business practices;

· Alignment of Department business processes with those established by Federal Student Aid at a high level to better integrate and align business processes and controls within a Department-wide framework; and

· Development and implementation of change management strategies.

· Identifying potential duplication, overlap, or missing controls or where there are concerns associated with the organizational placement of responsibility for a key financial control;

· Identifying ways to automated controls, to the extent possible; and

· Identifying poorly designed or documented controls that may jeopardize Department’s ability to operate in a timely, efficient manner and may result in such negative consequences as:

· loss of assets and resources;

· intentional errors, illegal acts, or unauthorized use of resources;

· unintentional errors, inefficiency, or waste;

· mismanagement;

· loss of consumer confidence;

· erroneous reporting of information;

· failure of the Department to meet and certainly exceed management objectives;

· failure to follow management decisions and policies;

· failure to meet regulatory requirements; and

· reportable conditions identified in outside audits and reviews.

· Manual controls,

· General computer application controls,

· Computer application controls,

· Credit modeling,

· Formula and discretionary grant compliance,

· Testing for compliance with other applicable laws and regulations, including improper payments.

For Tasks with Bronze level Testing - Minimum testing; Higher risk processes will be selected based on results of Risk Assessment. Testing will consist primarily of interviews and walk through procedures, with some additional procedures for high risk controls;

For Tasks with Silver Level Testing - Additional testing; “Bronze”-level testing plus risk-based testing of key controls at minimum periods. Moderate to High-risk processes will be selected based on results of Risk Assessment.;

For Tasks with Gold Level Testing - Robust testing requiring advanced technical expertise in non-financial areas. For example, this level would include experts to validate key controls and assumptions in the credit modeling process. Extent of testing requires risk analysis and funding from process owners (i.e. Budget Services, CIO). This would be robust testing intended to determine if controls are well-designed, efficient, and working as intended.

· Help program offices to understand what responsibility they have for key financial management and related business activity controls;

· Consolidate input from multiple process owners throughout the Department needed to provide reasonable assurance that internal controls over financial reporting are in place and operating effectively;

· Provide recommendations to improve the quality of the information provided by the program offices, to include modified statement of assurance letters and more reliance on external audits and other reviews;

· Assist Department staff in producing the annual Statement of Assurance with respect to Internal Controls over Financial Reporting, included in the Department’s AFR; and

· Provide recommendations to improve disclosures in the AFR.

APPENDIX II

QUALITY ASSURANCE MONITORING FORM

SERVICE or STANDARD:

SURVEY PERIOD: ____________________________________________________

LEVEL OF SURVEILLANCE SELECTED (Check):

____ Monthly ____ Quarterly ____ As needed

ANALYSIS OF RESULTS:

OBSERVED SERVICE PROVIDER PERFORMANCE MEASUREMENT RATE = ____%

SERVICE PROVIDER’S PERFORMANCE (Check): ___ Meets Standard ___ Does Not Meet Standard

NARRATIVE OF PERFORMANCE DURING SURVEY PERIOD: ________________________________________________________________________________________________________________________________________________________________________________________________________________________

PREPARED BY:_________________________ DATE:_________________