Att_29_-_Past_Performance_Customer_Satisfaction_Survey.docx

DOS Request for Proposals No. SAQMMA17R0168, Attachment 29

PAST PERFORMANCE CUSTOMER SATISFACTION SURVEY

Instructions to Evaluator: This survey will be used by the U.S. Department of State as part of its source selection for a 5-year (with an available additional 5-year extension), multi-billion dollar, global security engineering and supply chain contract competition. Your diligence in providing the requested past performance information is paramount to the success of the Department and the program and is greatly appreciated. Please read the entire form and complete all highlighted areas. Please submit the completed form by email directly to Mr. Steven G. Haines, Contracting Officer, U.S. Department of State at HainesSG@state.gov no later than [DOS CO to enter due date/time matching SF33 Box 9.]

Contractor firm being evaluated: ___________________________

1. Name of contracting activity (i.e. name of local government agency or commercial customer)

a. Evaluator Name

b. Evaluator Title

c. Evaluator Telephone Number

d. Evaluator Email Address

2. Contract Number

3. Task Order Number, if applicable

4. Contract Type [e.g. FFP, T&M/LH, CPFF, CPIF, CPAF, other (explain)]

5. Total contract value, and subcontract value (if applicable)

6. Performance role: (Prime/Sub-Contractor)

7. Description of work

8. Period of Performance

Contractor Performance Rating: Mark the appropriate block which corresponds to the performance rating.

Factor Rating

QUALITY

SCHEDULE

COST CONTROL

(N/A for fixed-price contracts)

MANAGEMENT

SMALL BUSINESS

(N/A for small business awardees, or for customer entities without small business subcontracting requirements)

REGULATORY

Evaluation Ratings Definitions:

Performance Assessment Considerations and Evaluator Narrative/Comments

QUALITY:

For Systems contracts or orders, consider the following aspects of Contractor performance:

· Product Performance. Assess the achieved product performance relative to performance parameters required by the contract or order.

· Systems Engineering. Assess the Contractor's effort to transform operational needs and requirements into an integrated system design solution.

· Areas of focus should be: the planning and control of technical program tasks; the quality and adequacy of the engineering support provided throughout all phases of contract or order execution; the integration of the engineering specialties; management of interfaces; interoperability, and; the management of a totally integrated effort of all engineering concerns to meet cost, technical performance, and schedule objectives.

· System engineering activities ensure that integration of these engineering concerns is addressed up-front and early in the design or development process. The evaluation should cover these disciplines: systems architecture; design; manufacturing; integration and support; configuration control; documentation, and; test and evaluation.

· The evaluation for test and evaluation should consider successes, problems, and/or failures in developing test and evaluation objectives; planning (ground, air, or sea) test, simulations and/or demonstrations; in accomplishing those objectives and on the timeliness of coordination and feedback of the test results (simulations or demonstrations) into the design and/or manufacturing process.

· Other activities include producibility engineering, logistics support analysis, supportability considerations (maintenance personnel and skills availability, or work-hour constraints, operating and cost constraints, allowable downtime, turn-around-time to service and maintain the system, and standardization requirements), survivability, human factors, reliability, quality, maintainability, availability, inspectability, etc.

· The evaluation of systems engineering needs to remain flexible to allow the evaluator to account for program-unique technical concerns, and to allow for the changing systems engineering environment as a program moves through the program phases, e.g., Engineering and Manufacturing Development, Production.

· Software Engineering. Assess the Contractor’s success in meeting contract or order requirements for all applicable software engineering-based activities and processes.

· Software engineering activities include, as appropriate: software development (design, code, and unit test); application of reuse, COTS, and other non-developmental software components; integration (including software component integration, system integration and test, and acceptance test support); and sustainment. Software processes include, for example: software size, effort, and schedule estimation; requirements analysis, development, and management; software configuration management; software risk identification and management; metrics collection and analysis, technical reviews, decision analysis, and software quality assurance and control, each as they specifically address software engineering activities.

· Consider the Contractor’s success with respect to:

- Planning a software development, integration, and testing effort that includes compatible cost, schedule, and performance baselines

- Delivering expected software driven capabilities on cost and on schedule

- Effective software metrics collection/analysis and status monitoring or reporting that provide the software visibility necessary to identify timely corrective actions and appropriately execute them

- Staffing with the software knowledge, skills, and abilities needed to execute the contract or order across the lifecycle; timely assignment of the appropriate numbers of software staff

- Awareness and control of software size and stability to enable tracking and allowing growth according to vetted enhancements vice scope creep

- Effective testing and integration of developed software within the larger system test and evaluation effort

- Effective processes to acquire, integrate, and test commercial and/or government off-the-shelf (COTS/GOTS) software and to achieve planned software reuse

- Achieving software assurance

- Consistent application of documented software engineering and management processes, including technical reviews, in alignment with contract or order requirements

· Logistics Support. Assess the success of the Contractor's performance in accomplishing logistics planning. For example: maintenance planning; manpower and personnel; supply support; support equipment; technical provisioning data; training and support; computer resources support; facilities; packaging, handling, storage and transportation; design interface; the contractor's performance of logistics support analysis activities, and the Contractor's ability to successfully support fielded equipment. When the contract or order requires technical and/or engineering data deliverables, the cognizant cataloging and/or standardization activity comments should be solicited.

· Product Assurance. Assess how successfully the Contractor meets program quality objectives (e.g., producibility, reliability, maintainability, inspectability, testability, and system safety, and controls the overall manufacturing process). The Project Manager or Contracting Officer must be flexible in how contractor success is measured (e.g., data from design test or operational testing successes, field reliability and maintainability and failure reports, user comments and acceptance rates, improved subcontractor and vendor quality and scrap and rework rates). These quantitative indicators may be useful later, for example, in source selection evaluations, in demonstrating continuous improvement, quality and reliability leadership that reflects progress in total quality management. Assess the Contractor's control of the overall manufacturing process to include material control, shop floor planning and control, status and control, factory floor optimization, factory design, and factory performance.

· Other Technical Performance. Assess all the other technical activity critical to successful contract or order performance. Identify any additional evaluation aspects that are unique to the contract or order, or that cannot be captured in another sub-element.

For Non-Systems contracts or orders, consider the following aspects of Contractor performance:

· Assess the Contractor’s conformance to contract or order requirements, specifications and standards of good workmanship (e.g., commonly accepted technical, professional, environmental, or safety and health standards). Include, as applicable, information on the following:

- Are reports and data accurate?

- Does the product or service provided meet the specifications of the contract or order?

- Does the contractor’s work measure up to commonly accepted technical or professional standards?

- What degree of technical direction from the contracting/ordering entity to the Contractor was required to solve problems that arise during performance?

For Operations Support: Assess how successfully the Contractor meets program or project quality objectives such as producibility, reliability, maintainability and inspectability. The Evaluator must be flexible in how contractor success is measured; (e.g., using data from field reliability and maintainability and failure reports, user comments and acceptance rates, and scrap and rework rates). These quantitative indicators may be useful later, for example, in source selection evaluations, in demonstrating continuous improvement, quality and reliability leadership that reflects progress in total quality management. Assess the Contractor’s control of the overall production process to include material control, shop planning and control, and status.

With regard to quality of products and services under the contract, discuss the level of innovation introduced by the contractor and the level of collaboration the Contractor employed to achieve results as a partner of the customer.

To support the assigned rating, the Evaluator Comments should contain sufficient comments, based on supporting documentation, and include successes and failures, as well as specific corrective actions, as appropriate.

Narrative/comments:

SCHEDULE: Assess the timeliness of the Contractor against the completion of the contract, task orders, milestones, delivery schedules, and administrative requirements (e.g., efforts that contribute to or affect the schedule variance).

This evaluation of the Contractor’s adherence to the required delivery schedule should include the Contractor’s efforts during the evaluation period that contribute to, or affect, the schedule variance. Also, address the significance of scheduled events (e.g., design reviews), discuss causes, and assess the effectiveness of the Contractor's corrective actions. This element applies to contract or order closeout activities, as well as contract or order performance. Instances of adverse actions, such as the evaluation of liquidated damages or issuance of Cure Notices, Show Cause Notices, and Delinquency Notices, are indicators of problems which may have resulted in variance to the contract or order schedule and should, therefore, be noted in the evaluation.

Questions to consider include the following:

- Is the Contractor completing the design/engineering services activities in a timely manner? This includes administrative activities, as well as meeting all scheduled milestones in the design process.

- Did the Contractor adequately schedule the work?

- Has the Contractor met administrative milestone dates?

- Has the Contractor met physical milestone dates specified by contract, or agreed to, in the project schedule?

- If the schedule has slipped through the Contractor’s fault or negligence, has the Contractor taken appropriate corrective action of its own volition?

- Has the Contractor furnished all required deliverables on or ahead of schedule?

- Is the Contractor completing the construction activities in a timely manner? This includes administrative activities, as well as physical construction activities, such as submittal of a management response to Request for Proposal (RFPs), etc.

- Has the Contractor furnished updated project schedules on a timely basis?

Evaluator may provide information on cumulative schedule variance, if used by the evaluator’s organization, using standards such as the American National Standards Institute (ANSI) Electronic Industries Alliance (EIA) Standard for Earned Value Management Systems or the Department of Defense Earned Value Management Implementation Guide.

With regard to achieving timely delivery of products and services under the contract, discuss the level of innovation introduced by the contractor and the level of collaboration the Contractor employed to achieve results as a partner of the customer.

Narrative/comments:

COST: (Not required for Fixed Price type contracts or orders). Assess the Contractor’s effectiveness in forecasting, managing, and controlling contract or order cost. If the Contractor is experiencing cost growth or underrun, discuss the causes and Contractor-proposed solutions for the cost overruns or underruns. For contracts or orders where task or contract sizing is based upon contractor-provided person hour estimates, the relationship of these estimates to ultimate task cost should be assessed. In addition, the extent to which the Contractor demonstrates a sense of cost responsibility, through the efficient use of resources, in each work effort should be assessed. Include, as applicable, the following information:

- Does the Contractor keep within the total estimated cost (what is the relationship of the negotiated costs and budgeted costs to actuals)?

- Did the Contractor do anything innovative that resulted in cost savings?

- Were billings current, accurate and complete?

- Are the Contractor’s budgetary internal controls adequate?

- Has the Contractor provided a design that can be constructed with the available funds?

- Has the Contractor notified the Contracting party and taken necessary corrective actions when the cost estimate exceeds available funds?

Evaluator may provide information on cumulative cost variance, if used by the evaluator’s organization, using standards such as the American National Standards Institute (ANSI) Electronic Industries Alliance (EIA) Standard for Earned Value Management Systems or the Department of Defense Earned Value Management Implementation Guide.

Narrative/comments:

MANAGEMENT: Assess the integration and coordination of all activity needed to execute the contract or order, specifically the timeliness, completeness and quality of problem identification, corrective action plans, proposal submittals, the Contractor’s history of reasonable and cooperative behavior (to include timely identification of issues in controversy), customer satisfaction, timely award and management of subcontracts. Include, as applicable, information on the following:

- Is the Contractor oriented toward the customer? How would you describe the relationship between the Contractor and their customer(s) (including from management to front-line personnel)?

- Is interaction between the Contractor and the contracting party satisfactory, or does it need improvement?

- Include the adequacy of the Contractor’s accounting, billing, and estimating systems and the Contractor’s management of property if a substantial amount of property has been provided to the Contractor under the contract or order.

- Address the timeliness of awards to subcontractors and management of subcontractors, including subcontract costs. Consider efforts taken to ensure early identification of subcontract problems, and the timely application of corporate resources to preclude subcontract problems from impacting overall prime contractor performance.

- Assess the Prime Contractor’s effort devoted to managing subcontracts and whether subcontractors were an integral part of the Contractor’s team. Are the Contractor’s management, on-site, and home office personnel exhibiting the capacity to adequately plan, schedule, resource, organize, and otherwise manage the work? If not, describe and relate to other rated elements.

Consider the following aspects of performance:

· Management Responsiveness. Assess the timeliness, completeness, and quality of problem identification, corrective action plans, proposal submittals, the Contractor's history of reasonable and cooperative behavior, effective business relations, and customer satisfaction. Consider the Contractor’s responsiveness to the program as it relates to meeting contract or order requirements during the period covered by the report.

· Subcontract Management. Assess the Contractor’s success with timely award and management of subcontracts.

- Assess the Prime Contractor’s effort devoted to managing subcontracts and whether subcontractors were an integral part of the Contractor’s team. - Consider efforts taken to ensure early identification of subcontract problems, and the timely application of corporate resources to preclude subcontract problems from impacting overall Prime Contractor performance.

- Consider efforts taken to ensure prompt subcontractor payment.

- Assess the Prime Contractor’s managing of subcontractors to ensure compliance with labor and safety standards at the subcontract level.

- If the contract is set-aside for small business, assess the Contractor’s compliance with any limitations on subcontracting.

· Program Management and Other Management. Assess the extent to which the Contractor: discharges its responsibility for integration and coordination of all activity needed to execute the contract or order; identifies and applies resources required to meet schedule requirements; assigns responsibility for tasks and actions required by contract or order; communicates appropriate information to affected program elements in a timely manner. Assess the Contractor’s risk management practices, especially the ability to identify risks, and to formulate and implement risk mitigation plans. If applicable, identify any other areas that are unique to the contract or order, or that cannot be captured elsewhere under the Management element.

· Integration and coordination of activities should reflect those required by the Integrated Master Plan or Schedule. Also consider the adequacy of the Contractor’s mechanisms for tracking contract or order compliance, recording changes to planning documentation and management of cost and schedule control system, and internal controls, as well as the Contractor’s performance relative to management of data collection, recording, and distribution as required by the contract or order.

· Management of Key Personnel (Applicable when the contract/order contains a Key Personnel clause). Assess the contractor’s performance in selecting, retaining, supporting, and replacing, when necessary, key personnel. For example:

- How well did the contractor match the qualifications of the key position, as described in the contract/order, with the person who filled the key position?

- Did the contractor support key personnel so they were able to work effectively?

- If a key person did not perform well, what action was taken by the contractor to correct this?

- If a replacement of a key person was necessary, did the replacement meet or exceed the qualifications of the position as described in the contract or order schedule?

With regard to managing the program(s) and project(s) supported by the contract, as well as managing the Contractor’s personnel and managing the relationship between the Contractor and the customer(s), discuss the level of innovation introduced by the contractor and the level of collaboration the Contractor employed to achieve results as a partner of the customer.

Narrative/comments:

SMALL BUSINESS: Assess compliance with any terms and conditions in the contract or order relating to Small Business participation. As applicable to contracting activity requirements and the terms and conditions of the contract, assess whether the Contractor provided maximum practicable opportunity for Small Business (including Alaska Native Corporations (ANCs) and Indian Tribes), Small Disadvantaged Businesses (which also includes ANCs and Indian Tribes), Women Owned Small Businesses, HUBZone, Veteran Owned, Service Disabled Veteran Owned Small Business, and ANCs and Indian Tribes (that are not Small Disadvantaged Businesses or Small Businesses) to participate in contract or order performance consistent with efficient performance of the contract or order.

Assess any small business participation goals which are stated separately in the contract or order. Assess achievement on each individual goal stated within the contract or order, or subcontracting plan, including good faith effort if the goal was not achieved.

Narrative/comments:

REGULATORY: Assess compliance with all terms and conditions in the contract or order relating to applicable regulations and codes. Consider aspects of performance such as compliance with financial, environmental, safety, and labor regulations, as well as any other reporting requirements in the contract. Consider questions such as the following:

- Has the Contractor complied with all contract clauses and reporting requirements?

- Has the Contractor complied with Cost Accounting Standards and disclosure statements?

- Has the Contractor complied with reporting requirements relating to recovered material content utilized in contract performance?

- Has the Contractor complied with contractual safety requirements and labor laws?

- Perform engineering surveillance to assess compliance with contractual terms for schedule, cost, and technical performance in the areas of design, development, and production.

- Has the Contractor complied with Hazardous Material Identification and Material Safety Data?

- Has the Contractor complied with specifications or other contractual requirements requiring the delivery or use of environmentally preferable products, energy-efficient products, products containing recovered materials, and biobased products? Monitor the Contractor's environmental practices for adverse impact on contract performance or contract cost, and for compliance with environmental requirements specified in the contract.

- Has the Contractor implemented an effective safety program; one which minimizes/mitigates potential accidents?

- Is the Contractor complying with affirmative action and EEO compliance requirements?

Narrative/comments:

OVERALL ASSESSMENT SUMMARY: A factual, detailed narrative is required for all evaluations regardless of rating (e.g., even Satisfactory ratings require narrative support). Cross-reference the comments in the Evaluator Narrative to their corresponding evaluation area. Each detailed narrative statement in support of the area evaluation must contain clear and concise objective information that accurately reflects the Contractor’s performance under the contract or order. It is also important for the information reported to include current, accurate, and complete statements about the Contractor’s performance because this information will be used to assist, inform, and influence future source selection and award decisions. An exceptional cost performance evaluation could, for example, cite the current underrun dollar value and estimate at completion. A marginal evaluation could, for example, be supported by information concerning personnel changes or schedule delinquency rate. Key personnel familiar with the effort may have been replaced by less experienced personnel. Sources of the data used by the Evaluator for the evaluation may include: operational test and evaluation results; technical interchange meetings; production readiness reviews; earned contract or order incentives; award fee evaluations; customer or field surveys; or evaluation of Contractor reports.

The Evaluator must select the applicable choice to the following statement after the Evaluator Comments: “Given what I know today about the Contractor's ability to perform in accordance with this contract or order's most significant requirements, I (would or would not) recommend them for similar requirements in the future."

Narrative/comments:

Evaluator statement (edit/indicate selection): Given what I know today about the Contractor's ability to perform in accordance with this contract or order's most significant requirements, I [__] would [ __] would not recommend them for similar requirements in the future.

(End of Survey)