PWS_360-Degree Multi-Rater Survey Administration and Reporting Platform_Final.pdf

U.S. Office of Personnel Management (OPM) Human Resources Solutions (HRS)

360-Degree/Multi-Rater Survey Administration and Reporting Platform

Performance Work Statement

November 20, 2023

Statement of Objectives (SOO)

360-Degree/Multi-Rater Survey Administration and Reporting Platform

1.0 Purpose

This purpose of this acquisition is to establish a base contract plus four (4) option years for provision of a 360-Degree/Multi-Rater Survey Administration and Reporting Platform for Human Resources Solutions (HRS) within the U.S. Office of Personnel Management (OPM).

HRS seeks to acquire the self-service use of a 508-compliant survey platform to create, administer, produce reports, and download data for a variety of 360-degree/multi-rater online surveys. Through this action, HRS is not seeking assistance with project management, survey development, survey building, survey administration, statistical analyses, or interpretation. HRS staff will serve as the survey creators and administrators. The Contractor will work directly with HRS and will have no interaction with HRS’s customer agencies. HRS will assign a small number of staff to serve as points of contact with the Contractor.

The self-service ability to create, administer, produce reports, and download data for a variety of 360-degree/multi-rater online surveys is critical in order for HRS to provide reimbursable services that help government agencies meet their missions. Examples of the types of surveys HRS administers include, but are not limited to, the OPM Leadership 360™ assessment and the OPM Leadership Potential Assessment (LPA).

2.0 Background

HRS has experience providing assessment-related reimbursable products and services to agencies and their sub-components across the Federal Government, U.S. Military, and state and local governments. Each year, HRS may administer multi-rater assessments for five thousand or more participants, each with up to twenty (and in some cases, more) raters.

Typical 360-Degree/Multi-Rater Survey Process:

1) A customer agency sends HRS a list of between 10 and 150 survey participants.

Each group of participants is considered a survey session.

2) HRS uploads participant information (names, email addresses, etc.) for a survey session into the survey platform.

3) Participants receive a system-generated invitation email with instructions to select raters, including their supervisor, peers, direct reports, and other relevant coworkers, to assess their leadership skills.

4) Participants access a personalized survey or webpage where they enter each rater’s name, email address, and rater type (supervisor, peer, direct report, or other).

5) Participants submit their rater list, then the survey platform automatically sends survey invitation emails to raters and self-assessment invitation emails to participants.

6) The participant completes their self-assessment.

7) Raters complete their ratings of the associated participant.

8) Throughout the survey administration, participants can monitor survey completion, add new raters, and edit rater information.

9) At periodic points throughout the session, progress reports are sent to participants, and system-generated reminder emails are sent to participants and raters who have not yet completed the survey.

10) At the end of the administration period, HRS produces (a) an individual feedback report for each participant and (b) one or more aggregate reports for all participants combined (or subsets of participants). The individual feedback report compiles the participant’s self-assessment and the associated raters’ assessments.

3.0 Scope

The 360-Degree/Multi-rater Survey Administration and Reporting Platform and all measurement tools will be priced on an annual basis (firm fixed pricing) and provided either as software we may install in our data center or as cloud-hosted Software as a Service (SaaS).

OPM requires unlimited access for approximately 25 government personnel who will serve as survey administrators. Survey administration capabilities must include (1) creating and storing hundreds of survey items/questions and dozens of response scales and (2) the capacity to run several dozen concurrent survey sessions and hundreds of survey sessions during the subscription period. Access is also required for thousands of survey participants and hundreds of thousands of raters for the purpose of completing surveys as assigned.

4.0 Performance Objectives (Required Results)

The 360-Degree/Multi-Rater Survey Administration and Reporting Platform will be 508-compliant and provided either as software we may install in our data center or as cloud-hosted SaaS. For a cloud-hosted SaaS platform, the Contractor must currently hold FedRAMP- Moderate authorization.

In addition, the 360-Degree/Multi-rater Survey Administration and Reporting Platform must also be capable of, but not limited to, the functional, operational, and technical objectives listed below.

4.1 Pre-Existing 360-Degree/Multi-rater Infrastructure

1) The contractor’s platform must have an established, tested, production-ready 360-degree/multi-rater survey infrastructure already in use by customers to:

a. Build 508-compliant 360-degree/multi-rater surveys.

b. Administer 360-degree/multi-rater surveys.

c. Monitor completion status of participants and raters in 360-degree/multi-rater surveys.

d. Export data from 360-degree/multi-rater surveys

2) The system will have the capability to produce results reports for 360-degree/multi-rater surveys.

3) HRS will not be required to create or configure the processes or mechanisms for importing a participant roster, sending rater invitations, viewing completion status for survey takers, exporting data, and producing reports.

4.2 Administrative Access

1) The platform must accommodate Microsoft AzureAD Single-Sign On (SSO) using a Personal Identity Verification (PIV) card.

2) The platform must permit, at a minimum, a superuser admin role that allows the user to create/administer surveys as well as view, edit, and download data from all survey sessions in the platform. This role would receive the highest permission levels allowed instance wide.

3) The platform will also permit a lower-level admin role that allows the user to administer, create, view, download data from, and edit only their own survey sessions or survey sessions they have been given permission to access.

4.3 Survey Building

1) Item Types

a. The platform must support a wide range of standard item types, including

Likert-type, mark-one, mark-all-that-apply, drop-down, ranking, choose a specific number of responses, and text.

b. HRS must be able to set character limits for text items (a maximum of up to at least 4,000 characters). A counter must inform the user of how many characters they have left and/or have used.

c. Mark-one and mark-all-that-apply items must be able to contain an “Other – specify” response option for respondents to type text to explain.

d. Mark-all-that-apply items must have the ability to set a response option as exclusive.

2) Item and Response Validation

a. Administrators must be able to set certain items as required; users must be prompted if they do not provide an answer to a required item.

b. The platform must support response validation for the following:

i. The correct number of “choose a specific number of responses” responses are selected.

ii. Text comments are within the allowable range.

iii. Dates are in the proper format.

iv. Numbers are within the allowable range.

3) Conditioning

a. The platform must allow items to be shown or hidden based on responses to a single item and a combination of items.

b. The platform must allow items to be shown or hidden based on uploaded information about respondents.

c. The platform must allow conditional logic to be applied to entire items (i.e., show/hide an entire item, including associated response options).

d. The platform must allow conditional logic to be applied to specific response options for an item (i.e., show/hide only certain response options within an item).

e. The platform must allow conditional logic to be applied to entire survey pages (i.e., show/hide an entire page).

4) Piping

a. The platform must have question piping capability, which allows one to insert, or "pipe," answer text from a previous question or information in the roster into instructions or a question on a later page in the survey. For example, if the participant identifies a rater as the supervisor, the survey pipes “supervisor” in the rater survey instructions, or the survey pipes in the name of the participant the rater is assessing.

5) Survey customization

a. HRS must be able to create new/custom items.

b. HRS must be able to create new/custom response scales.

c. HRS must be able to be create a new survey by “cloning” or duplicating an existing survey and modifying as necessary.

d. Survey formatting must be flexible and customizable so that HRS can add branding, images, colors, and more without using coding or programming language.

e. HRS must be able to determine page breaks and spacing.

f. Once a user submits their responses, they must be presented with either a customizable “thank you” message in a dialog box or a customizable “thank you” page.

4.4 Participant and Rater Invitations

1) The platform must have a pre-existing mechanism to upload a roster of survey participants in bulk (not one at a time) that does not require HRS to create or configure the process or mechanism or use coding/programming language.

2) HRS must be able to add new participants during an administration.

3) Participants must be able to submit rater information, including first name, last name, email address, and rater category without HRS involvement.

4) The platform must have a pre-existing mechanism to automatically send personalized rater invitation emails in bulk when participants submit their rater information that ideally does not require HRS to create or configure the process or mechanism.

5) Participants must be able to add/edit raters at any point, ideally without HRS involvement.

6) Email invitation and reminder text must be customizable and must include the option for fillable fields (e.g., participant name) and allow for text-only or HTML format.

7) HRS must have the ability to send email invitations and reminders on a set schedule.

8) HRS must have the ability to send/resend email invitations and send reminders on demand, ideally without interrupting future scheduled reminders.

9) HRS must be able to send notification and reminder emails individually to a specific person or in bulk to a group of participants and/or raters.

4.5 Email Transmission and Routing

1) The platform must accommodate custom domains to mask the website URL and survey logon links so that they reflect a specified opm.gov website and opm.gov logon links (not dot.com or other commercial domains).

2) The platform must be able to send emails on behalf of an opm.gov subdomain, such as “USASurveyEx.opm.gov” or similar. (It cannot be the top-level domain opm.gov because OPM has an impediment with adding top level SPF records.)

3) The platform must be able to route outgoing emails (e.g., survey invitations/reminders) through OPM mail servers for processing and delivery.

4.6 Data Collection

1) Survey takers must be able to access the survey using a unique/individual URL.

2) The platform must save responses automatically, either as the survey taker pages forward or with each selection, and survey takers must be able to return to a saved survey.

3) A direct link into a survey must be possible by embedding the passcode into the survey URL so that survey takers do not need to type or copy-and-paste the passcode.

4) Once a survey has been submitted, the associated unique passcode must become disabled.

5) HRS must be able to “unlock” a completed survey to make the survey active for the respondent.

6) The platform must have a pre-existing mechanism for HRS to monitor participant and rater survey completion status (i.e., the list of participants in a particular survey session with their self-assessment completion status, number of raters in each rater group invited by each participant, number of completed ratings in each rater group for each participant, etc.) that does not require HRS to create or configure the process, mechanism, or relevant variables.

7) The platform must have a pre-existing mechanism for participants to monitor their own assessment completion status (i.e., the individual participant’s self-assessment completion status, number of raters they invited, and number of completed ratings they’ve received in each rater group) that does not require HRS to create or configure the process, mechanism, or relevant variables.

8) There must be a search function to locate specific participants and raters in the system by email address that does not require configuration by HRS.

9) The platform must allow HRS to reversibly withdraw and restore survey-taker access to a survey before and after the survey deadline.

10) Surveys must maintain all functionality on phones, tablets, and similar devices.

11) The platform must support dozens of active survey sessions at one time, each of which receives a high volume of responses and requires the sending of a high volume of invitations/reminders.

4.7 Simple and Straightforward Data Export

1) Raw survey data (i.e., response values and coded identifiers) must be available for HRS to access and download at any time during or after a survey administration without contractor assistance and at no additional cost.

2) The platform must have a pre-existing mechanism for HRS to export data in bulk for all items, participants, and raters in a particular survey session that does not require HRS to create or configure the process, mechanism, or relevant survey items.

4.8 Report Production

1) The platform must produce deliverable quality, print-ready color reports.

2) Reports must be producible in batches and for one-off, specific breakouts.

3) The platform must offer a variety of presentations for multiple data types, including:

a. 100% stacked bar charts

b. Line graphs

c. Benchmarks with range indicators

d. Most and least favorable items

e. Change indicators (e.g., up arrow for improvement)

4) HRS must be able to create their own report templates for repeated use or provide the information necessary for the contractor to create report templates at no additional cost.

5) A mechanism must exist for HRS to select which specific template to use when producing reports (e.g., for different types of 360-degree/multi-rater surveys).

6) HRS must be able to set the required minimum number of respondents for each rater group (e.g., 1 supervisor, 3 peers, etc.). If a rater group does not contain the required minimum number of raters, that group’s scores must not be shown in reports. Each rater group must have a default requirement, which HRS can override for a particular session.

7) Reports must be able to include benchmark comparisons.

8) Administrators must be able to upload benchmark data for items, aggregates/scales, and other quantitative results.

9) A mechanism must exist for HRS to select which specific benchmarks or benchmarks to include in reports.

10) The platform must produce reports that include basic introductory and interpretation textual commentary.

11) Reports must be able to include a section that contains open-ended comments.

4.9 FedRAMP Hosting, if Applicable

If the system is offered as a cloud-hosted SaaS, the contractor must provide FedRAMP Moderate authorized hosting that is compliant with FedRAMP Moderate security controls. This includes yearly audit by a Third-Party Audit Organization (3PA), Authority to Operate (ATO) as approved by the Joint Advisory Board, and technical support services which meet the requirements of FedRAMP Moderate controls.

4.10 Training and Support

1) The contractor must provide training to ensure HRS can fully implement survey services.

2) The system’s 360-degree/multi-rater-specific survey processes, features, and instructions will be documented by the contractor in a user guide available to HRS.

3) The contractor must provide responsive support to ensure HRS can fully implement and maintain survey services.

4) The contractor must make available employees that are capable of passing the necessary security clearance to support technical and survey administration needs.

5) HRS must own all data, survey content, survey responses, reports and reporting templates developed as part of this contract. The contractor must not share HRS’s data, survey content, survey responses, reports, reporting templates, or any other aspect of HRS’s instance with customers or potential customers.

5.0 Type of Contract Contemplated

The contract type will be Firm Fixed Price. The use of a per survey or per respondent pricing strategy will not be acceptable.

6.0 Period of Performance

The period of performance of this contract is a one-year base plus four (4) one-year option periods.

7.0 Place of Performance

The Survey Services Platform will be provided from the contractor’s site or the contractor’s hosting provider’s site.

8.0 Government information

N/A

9.0 Security

9.1 IT Security Compliance

The Contractor shall comply with OPM IT Security policy and guidance, to include all instructions, as well as administrative, physical, and technical security controls to ensure that all Government security requirements are met. OPM Specific Clauses relating to Security are incorporated as an attachment to this solicitation.

9.2 Background Investigations Requirements

As necessary, the Contractor shall provide qualified personnel that will be required to undergo a background investigation in accordance with OPM Specific Clause 1752.204-70 Contractor Personnel Security Requirements.

OPM Security Services, at its discretion, may require further or expanded investigations or clearances at any time. These positions risk/sensitivity levels will require, at a minimum, a current Background Investigation of Public Trust Moderate Risk.

10.0 Required nongovernment information/documents

11.0 Other

12.0 Quality Assurance Surveillance Plan.

Note: The following Quality Assurance Surveillance Plan (QASP) is OPM’s codification of its method of implementing FAR 37.601, which requires that performance-based contracts or task/call orders:

(a) Describe the requirements in terms of results required rather than the methods of performance of the work.

(b) Use measurable performance standards (i.e., in terms of quality, timeliness, quantity, etc.) and quality assurance surveillance plans.

(c) Specify procedures for reductions of fee or for reductions to the price of services when services are not performed or do not meet contract requirements; and

(d) Include performance incentives where appropriate.

The Quality Assurance Surveillance Plan (QASP) contained in this document shall be incorporated into and made a part of any resulting award.

Quality Assurance Surveillance Plan (QASP)

Requirement Performance Standard

Acceptable Quality Level

(AQL)

Monitoring Method

Performance Incentive/ Reduction

Quality of Deliverables

Deliverables are of a quality consistent with the requirements presented.

100% Review of products

Incentive: Favorable comments will be documented on Contractor's performance.

Disincentive:

Unfavorable comments will be documented on Contractor's performance.

Project Management

Project Management effectively oversees and directs project.

100% Status Reports on system functionality

Incentive: Favorable comments will be documented on Contractor's performance.

Disincentive:

Unfavorable comments will be documented on Contractor's performance.

Survey Building

System provides all required functionality and is available 24/7.

System accessible 99.5% excluding scheduled system maintenance.

Reported down time and number of helpdesk requests related to unavailability of required functionality

Incentive: Favorable comments will be documented on Contractor's performance.

Disincentive:

Unfavorable comments will be documented on Contractor's performance.

Standard

Acceptable Quality Level

(AQL)

Monitoring Method

Performance Incentive/ Reduction

Data Collection/ Administration

Survey participants are able to access and complete surveys 24/7 using Government-owned and/or personal computers.

System accessible 99.5% excluding scheduled system maintenance.

Reported down time and number of helpdesk requests unrelated to user error.

Incentive: Favorable comments will be documented on Contractor's performance.

Disincentive:

Unfavorable comments will be documented on Contractor's performance.

Analyses/ Presentation of Results

System generated reports are error free and properly formatted.

100% calculation accuracy and 99.5% formatting accuracy, Quality checks and independent verification of calculations

Incentive: Favorable comments will be documented on Contractor's performance.

Disincentive:

Unfavorable comments will be documented on Contractor's performance.

Data Housing Databases configured and secured using best practices. Data are encrypted in transit and at rest.

Data is stored in secure facility with system redundancy and weekly full and daily incremental backups.

100% data security and accuracy in storage and transit

System logs, vulnerability scans, and standard security audits

Incentive: Favorable comments will be documented on Contractor's performance.

Disincentive:

Unfavorable comments will be documented on Contractor's performance.

Administrative Permissions and Access

Administrative users can be assigned role based access by levels of administrative rights

100% Survey and data access is correctly limited to authorized individuals

Access logs Incentive: Favorable comments will be documented on Contractor's performance.

Disincentive:

Unfavorable comments will be documented on Contractor's performance.

Standard

Acceptable Quality Level

(AQL)

Monitoring Method

Performance Incentive/ Reduction

Upgrades and Support

Responsive support is provided to correct system bugs and/or deficiencies. Vendor will notify HRS of scheduled system upgrades or changes

95% System bugs and/or deficiencies are corrected within 24-48 hours

Status report on system functionality

Incentive: Favorable comments will be documented on Contractor's performance.

Disincentive: Unfavorable comments will be documented on Contractor's performance.

IT

Architecture

Meet all required Government security and access standards

System accessible 99.5% excluding scheduled system maintenance.

Status report on system functionality.

Results of penetration testing and vulnerability scans

Incentive: Favorable comments will be documented on Contractor's performance.

Disincentive: Unfavorable comments will be documented on Contractor's performance.

EVALUATION FACTORS

E.1 INSTRUCTIONS FOR PREPARING THE TECHNICAL PORTION OF THE

QUOTATION

(1) The Contractor’s quotation must demonstrate the Contractor’s ability and approach to satisfy the objectives of this solicitation. Quotations must be clear, coherent, and prepared in sufficient detail for effective evaluation of the Contractor’s quotation against the evaluation criteria stated in Section E.2 - EVALUATION.

(2) Volume I – Technical Volume -- shall not exceed 25 pages with text in no smaller than 12-point font. Page count does not include any resumes and the Quality Assurance Surveillance Plan. Do not include cost/pricing information in the Technical Volume. OPM will review the hard copy quotation submissions to assess compliance with page limitations. Quotations that exceed the stated page limits will not be considered and will be withdrawn from consideration.

The technical portion of the Technical Requirements quotation should include, at a minimum:

TECHNICAL APPROACH/UNDERSTANDING OF THE REQUIREMENTS - The quotation must be set forth in detail and present the information necessary to provide a basis for evaluation by OPM of the Contractor’s proposed Technical Approach demonstrating how the proposed solution meets OPM’s requirements as stated in PWS Sections 1 - 4. for provision of the Survey Services Platform. The technical approach shall include a narrative and at a minimum the narrative should address the points below.

1. A full explanation of products, services, techniques, and procedures the Contractor proposes.

2. Indicate the initial plan to gain an understanding of the needs of OPM’s operations.

3. Description of the contractor’s capabilities and plan for obtaining an OPM ATO

(Authority to Operate).

4. Describe the strategy for dealing with potential significant technical, administrative, or managerial problems that could arise, particularly those issues that impact deliverable quality and or deadlines.

5. The Contractor’s proposed language to complete the Performance Requirement

Summary Template of the Quality Assurance Surveillance Plan (QASP).

PAST EXPERIENCE/PERFORMANCE

The Government will determine its level of confidence that the Quoter will successfully perform the requirements based on an assessment of submitted Past Experience information. Looking at examples of real work and actual results, the Government will compare opportunities where the Quoter can bring experience on recent and similar projects to those described in the solicitation. Quoters who demonstrate the ability to bring the following experience may receive a higher confidence assessment rating:

• Experience demonstrating the capability for provision of services similar in nature and magnitude to the services required by OPM as stated in the PWS.

• Experience demonstrating the ability to provide services and assistance for access and navigation of survey services serving global user populations of multiple thousands of concurrent users.

The Government will factor into its assessment whether experience was gained by the Quoter acting as a prime, as a subcontractor or by a subcontractor with whom the Quoter may intend to utilize to supplement recognized gaps in its experience.

E.2 EVALUATION

Evaluation of quotations will be conducted by a technical evaluation panel (TEP) in accordance with the evaluation criteria stated below.

The following factors for each requirement shall be used to evaluate offers and are in descending order of importance:

Factor 1 – TECHNICAL APPROACH/UNDERSTANDING OF THE REQUIREMENTS

Technical Approach/Understanding of the Requirements will be evaluated based on the soundness of the proposed Technical Approach and the extent to which the Contractor’s quotation addresses the description of this factor in the instructions above.

Factor 2 – PAST EXPERIENCE

The Government will assess its level of confidence that the Quoter will successfully perform the requirements based on its demonstrated Past Performance considering matters such as: recency, relevancy, and quality.

The first aspect evaluates the recency of the Quoter’s past performance. Recency is considered work performed within the past three years. Next, relevance is determined by looking at how relevant recent effort accomplished by the Quoter is to the effort to be acquired through the source selection. Common aspects of relevancy include, but are not limited to, the following: similarity of product/service/support, complexity, dollar value, contract type, use of key personnel, and extent of subcontracting/teaming.

The table below shows the ratings the Government will assign in its evaluation of Relevancy Ratings for factor 2 – Past Performance.

Past Performance Relevancy Ratings Rating Description

Very Relevant Present/past performance effort involved essentially the same scope and magnitude of effort and complexities this solicitation requires.

Relevant Present/past performance effort involved similar scope and magnitude of effort and complexities this solicitation requires.

Somewhat Relevant

Present/past performance effort involved some of the scope and magnitude of effort and complexities this solicitation.

requires.

Not Relevant Present/past performance effort involved little or none of the scope and magnitude of effort and complexities this solicitation requires.

Factor 3 - PRICE

While neither a weighted criterion nor a controlling factor, price will be an important evaluation consideration in selecting the Contractor under this solicitation. The Government may award to other than the lowest priced quotation if another quotation's non-priced evaluation factors justify the additional price. In its review, the Government will make a reasonable effort to determine if the prices 1) are realistic for the services proposed, 2) reflect a clear understanding of the contract requirements, and 3) are inclusive of all services requested.

Factor 2 – PAST EXPERIENCE