PWS_Evaluation_Services.pdf

Solicitation Number: 75H70419R00034 IHS, OPHS/DPER Evaluation Services IDIQ 2.0

Section C - Description/Specifications/Performance Work Statement

C.1. Title Indian Health Service – Program Evaluation Services

C.2 Background The Indian Health Service (IHS) is the agency within the Department of Health and Human (DHHS) responsible for providing federal health services to American Indians and Alaska Native (AI/AN) tribal members. As the principal federal health care provider and health advocate for AI/AN people, the goal of IHS is to raise their health status to the highest possible level. Within IHS, the Office of Public Health Support (OPHS) provides essential public health functions of epidemiology, surveillance, planning, performance, evaluation, research and statistical information and analysis. OPHS’s Division of Planning, Evaluation and Research (DPER) provides leadership and coordination of the agency’s strategic planning, program evaluation, and health research activities.

The IDIQ will support evaluations of a variety of IHS Programs in partnership with IHS Program and DPER staff, and in some cases, may involve working with grantees.

C.3.1 General Requirements Independently and not as an agent of the Government, the Contractor shall be required to furnish all the necessary services, qualified personnel, material, equipment, and facilities, not otherwise provided by the Government, as needed to perform the work of the contract.

Under this contract, the Contractor is responsible for the planning, coordination, development, implementation, delivery, and quality program evaluation and evaluation support functions to support IHS in understanding processes, outcomes, and either short- or long-term impacts related to Public Health Programs, Clinical Services and Healthcare Business Operations.

This section describes the general requirements for this effort. The following sub-sections provide details of various considerations on this effort. The following schedule of tasks and deliverables outlines the requirements necessary to plan, organize, develop, implement, operate, and evaluate the tasks within the contract.

C.3.1 Non-Personal Services The Government shall neither supervise contractor employees nor control the method by which the contractor performs the required tasks. Under no circumstances shall the Government assign tasks to, or prepare work schedules for, individual contractor employees. It shall be the responsibility of the contractor to manage its employees and to guard against any actions that are of the nature of personal services, or give the perception of personal services. If the contractor believes that any actions constitute, or are perceived to constitute personal services, it shall be the contractor's responsibility to notify the Contracting Officer Representative (COR) immediately who will then notify the Contracting Officer (CO) immediately.

C.3.2 Business Relations

The contractor shall successfully integrate and coordinate all activity needed to execute the requirement. The contractor is required to provide quality assurance services and effective management of subcontractors. The contractor shall seek to ensure customer satisfaction and professional and ethical behavior of all contractor personnel.

C.3.3 Contract Administration and Management The following subsections specify requirements for contract, management, and personnel administration.

C.3.3.1 Contract Management The contractor shall establish clear organizational lines of authority and responsibility to ensure effective management of the resources assigned to the requirement. The contractor must maintain and ensure continuity of evaluation activities between the contractor's corporate offices and Indian Health Service.

C.3.3.2 Contract Administration The contractor shall establish processes and assign appropriate resources to effectively administer the requirement. The contractor shall respond to Government requests for contractual actions in a timely fashion. The contractor shall have a single point of contact between the Government and Contractor personnel assigned to support contracts or task orders. The contractor shall assign work effort and maintain proper and accurate time keeping records of personnel assigned to work on the requirement.

The Contractor will meet with the Contracting Officer Representative (COR) and IHS officials within 10 working days of the contract award to become acquainted with the IHS staff, review the Performance Work Statement (PWS), discuss timelines and deliverables, and to begin initial planning. The Contractor and COR will hold start up meetings at the beginning of each option year of the contract to revisit and make adjustments (if necessary) to the comprehensive action plan, quality assurance plan, communications plan, evaluation plan, and project plan, to discuss ongoing tasks in the PWS, and deliverables for each option year.

C.3.3.3 Personnel Administration The contractor shall maintain the currency of their employees by providing initial and refresher training as required to meet the PWS requirements. The contractor shall make necessary travel arrangements for employees and maintain adequate infrastructure to support contract tasks.

C.3.4 Subcontract Management The contractor shall be responsible for any subcontract management necessary to integrate work performed on this requirement and shall be responsible and accountable for subcontractor performance on this requirement. The prime contractor will manage work distribution to ensure there are no Organizational Conflict of Interest (OCI) considerations. Contractors may add subcontractors to their team after notification to the CO and COR.

C.3.5 Contractor Personnel, Disciplines, and Specialties The contractor shall accomplish the assigned work by employing and utilizing qualified personnel with appropriate combinations of education, training, licensure/certification and experience. The contractor shall ensure the labor categories as defined in the Labor Categories document, labor rates, and man-hours utilized for the performance under the awarded contract will be maintained throughout the performance period.

C.3.6 Location and Hours of Work Accomplishment of the results contained in this PWS requires work at various Government (IHS) facilities. Normal workdays are Monday through Friday except US Federal Holidays.

Workers typically work eight (8) hours per day, 40 hours per week. Core hours of work are from 0930 to 1500 daily. All employees and required services are expected to be available during core hours.

C.3.7 Travel / Temporary Duty (TDY) All travel requirements (including plans, agenda, itinerary, or dates) shall be pre-approved by the government (subject to IHS and local policy procedures), and is on a strictly cost reimbursable basis. Costs for travel shall be billed in accordance with the regulatory implementation of Public Law 99-234 and FAR 31.205-46 Travel Costs (subject to IHS and local policy & procedures;

may reference FAR).

The Contractor will be responsible for coordinating the logistics of the trainings to include travel, per diem, lodging of non-federal subject matter expert instructors. Travel for training participants will not be covered under this contract.

C.4 Performance Requirements The Contractor shall provide the necessary personal resources, equipment and infrastructure to manage, perform, and administer the contract throughout the performance period. The Contractor shall be responsible to complete submitted Task Order Proposals in all and any of the following domains:

I. Feasibility, Pilot and Evaluation Projects II. Statistical Projects

III. Health Systems Analysis, Economics, and Financing IV. Technical Assistance and Training

C.4.1: Feasibility, Pilot and Evaluation Projects

Feasibility and Pilot Studies The Contractor shall design, conduct, and report results from feasibility and pilot studies, as well as develop tools that increase fidelity to the implementation of successful models of intervention and/or program design. All documents will be reviewed and be approved by IHS prior to implementation by contractor. A feasibility study is usually done to determine whether or not an organizational activity, a service, a therapeutic procedure, a financial intervention, a treatment service, strategy, intervention, technology or a statistical operation model will work as intended.

Feasibility studies are usually small scale and completed over a relatively brief period of time. A pilot study is used to determine whether or not the results of a feasibility study can be successfully replicated under a broader set of circumstances that are reflective of the intended range of applicability.

The Contractor shall identify the key components of the feasibility or pilot study; identify the units that will participate in the study and a procedure by which units will be sampled; invite units to participate in the study; arrange all logistics for the study; provide support to the units as the study is conducted; convene the sampled units as needed throughout the study; collect qualitative and quantitative information on unit performance in the study; and prepare technical reports on the outcome of the study.

Evaluation Projects Evaluation studies can be experimental or non-experimental in design; and can include a range of qualitative and/or quantitative information for a number of subject areas.

IHS conducts evaluation studies of projects or programs in order to understand factors that may impede or contribute to its success; to explain linkages between project or program activities and outcomes; to assess effects beyond intended objectives; to estimate what would have occurred in the absence of a project or program, in order to assess net impact; or to compare the effectiveness of alternative interventions, projects or programs aimed at the same objectives. Evaluations may also focus on: process or implementation to understand if the intervention, project, or program is operating as intended; outcomes to determine if projects or programs achieve intended objectives; cost benefit and/or cost effectiveness to assess the costs to produce positive benefits or identify the least costly method of achieving objectives.

As evaluation design and planning is central to the effectiveness of a program and its evaluation, contractor may be involved in strategic planning of programs and evaluations. IHS evaluations may make use of multiple sources of data, such as: grantee progress reports, grantee local evaluation reports, performance data reported to IHS, customer/client surveys, clinical instruments, field observation, and other sources. The Contractor shall have the capacity to design and/or execute such evaluation projects. In some instances, evaluation studies will have short deadlines and limited resources so careful attention must be devoted to selecting the most appropriate designs and employing appropriate standards for health care research and statistical methodology. The Contractor shall develop a strategic planning approach to programs and their related evaluations. The Contractor shall develop one or more preliminary evaluation designs, which will measure relevant aspects of program performance.

The products of this sub task will include strategic plans, theory of change/logic models, data instruments, technical reports, data collection plans, data analysis plans, and use/dissemination plans. The Contractor shall identify the program and evaluative questions to be addressed by the evaluation study; design an evaluation study capable of successfully answering the questions posed; invite participation in the study; arrange all logistics for the study; convene and conduct a technical support group; provide technical assistance on the evaluation; collect qualitative and/or quantitative information; analyze collected data and prepare a report that summarizes the essential questions, design, results, and discussion. In preparing summary or technical reports, the contractor will work with the Task Manager to identify the target audience (there may be more than one for each program evaluation) and develop an information presentation approach that is focused on conveying data and information visually such that it enhances understanding in intuitive ways for the audience to see and explore.

C.4.2: Statistical Projects

Sampling Designs and Statistical Methods The Contractor shall demonstrate expertise in complex sampling and survey design, in weighting, variance estimation, imputation and analyses of complex survey data, and in a wide range of additional statistical methods suitable for the analyses of survey data through direct staff and/or through consultant experts.

The Contractor must make available staff and/or consultants, through appropriate logistical arrangements, who can provide advice to the Government in these specialized design and statistical areas. Advice may take the form of a consultation that includes design of a complex survey, design of a complex sampling procedure for a survey, design of compound weights and variance estimation for a survey, design of one or more complex statistical analyses, or some combination of these activities.

Acquiring, Developing, and Managing a Collection of National Data Bases The Contractor shall identify Federal and/or non-Federal databases pertinent to the scope of work (e.g., household surveys, facility surveys, employer surveys, census data, health claims databases, surveillance or epidemiological data, etc.) that can provide answers to questions posed by the Government. When such databases are not available, the Contractor shall have the knowledge and capacity to provide advice on ways such databases can be developed. This may include building such data bases from other preexisting data bases or recommending new data collections.

The Contractor shall search Federal, State, Local and private sources, as appropriate, for data bases that contain information that can answer the questions posed by the Government. When such data bases are identified, the Contractor shall determine the quality of the data, completeness, accessibility of the data, their costs, use agreements and fees, required security training, etc. Imputations and other methods to correct sampling error should be used as needed.

Once the Government has concluded that the data bases are appropriate, the Contractor shall acquire and process them through appropriate logistical arrangements to answer the questions posed by the Government. In some instances, the Contractor shall process information for use by others (e.g., processing Federal information for use by the States).

When a new data collection is required, the Contractor shall perform all tasks associated with preparation of an instrument or questionnaire including the qualitative assessment of instruments, e.g., through focus groups and cognitive testing, including but not limited to preparation of necessary Office of Management and Budget (OMB) clearance packages, Internal Review Boards (IRB), sampling design and execution, field data collection and cleanup, and electronic data files and tabular/graphical analyses.

The work plan shall specify methods that will be used to collect data, whether on site, (e.g., face-to-face interview, or telephone) or, for large-scale data collections, via the Internet, diskette, or paper forms. The work plan shall also define site selection criteria, and potential study sites, and indicate options and relative advantages of different sampling plans. The Contractor shall have the capacity to conduct large scale data collections; to collect data through the Internet, via computer-assisted telephone interviews or audio computer-assisted interviews or other electronic means, or on paper forms; to perform complex sampling and weighting procedures for probability samples and adjustment of missing data to account for non-response.

Analyses of National Datasets The Contractor shall process and analyze Federal and/or non-Federal data sets, through appropriate logistical arrangements, that address the questions posed by the Government;

undertake research to determine the quality and characteristics of these data sets; and plan appropriate analyses of the data sets to address specific questions. The Contractor shall prepare analyses of the data sets in the most economical manner and prepare appropriate frequency, percentage, and rate distributions, or other statistical measures, e.g. variances. Special attention shall be devoted to economical procedures of data analysis, and contractor shall identify shortcuts and best practices that will reduce costs to the Government when large data sets are analyzed. In some instances, the Contractor may be required to work with weighted data collected through complex sampling designs or to prepare complex analyses (e.g., logistical regression) using these data bases.

The Contractor will comply with all applicable Federal security, confidentiality, and privacy laws. In preparing summary reports, the contractor will work with the COR to identify the target audience (there may be more than one for each program evaluation) and develop an information design approach that is focused on conveying abstract information in intuitive ways allowing the target audience to see, explore, and understand large amounts of information at once.

Analyses of Complex Quantitative Data If involved in a survey, the Contractor shall prepare analyses of complex data sets, whether for survey data derived from complex sampling designs or for analysis of cost, service utilization and population distinctions. This will entail the preparation of statistics and their associated variances using the SUDAAN, NVIVO, STATA, or other appropriate software. Software shall be provided by the contractor. Such results will also be used to prepare variance curves or design effect tables for inclusion in reports to be provided to users who want to interpret the statistics from a survey. To perform this work successfully, the contractor must have staff and/or consultants with advanced expertise in statistics, econometrics, sampling, weighting, imputations, and variance statistics.

The Contractor shall prepare a summary report that integrates findings from the data analysis and literature review, and as required, identify current gaps in knowledge. In preparing summary reports, the contractor will work with the Task Manager to identify the target audience (there may be more than one for each program evaluation) and develop an information presentation approach that is focused on conveying data and information visually such that it enhances understanding in intuitive ways for the audience to see and explore.

Development, Implementation, and Testing of Quality Tools The Contractor shall conduct a range of activities and make logistical arrangements with respect to topics identified by the Government from the following areas: clinical and system practice guidelines, outcome measures, population and service report cards, and performance indicator systems. This work may include analyses that address the compilation and reporting of GPRA measures employed by IHS programs. These activities shall include a review of current literature and practice; development of structures, processes, measures, criteria, and benchmarks;

consultation with key persons and groups (e.g., a panel of experts to discuss, review, and a critique of indicators of substance abuse and other behavioral health related problems);

preparation of synoptic reports for target audiences; review of developments in parallel fields;

examination of new technology and its potential application to the field; and other tasks that promote the development, testing, implementation, and assessment of quality tools.

The Contractor shall make available direct contract staff and/or consultants who can carry out each of these tasks. The Contractor shall demonstrate both substantive and methodological expertise.

C.4.3: Health Systems Analysis, Economics, and Financing IHS frequently requires economic and finance analysis to assess strengths, gaps, or effectiveness of current or future Health Systems programs, structures, or processes. Such analyses will often involve data from various sources/locations as needed and will include data gathering, compiling, and validating. This may include Systems at the national, IHS or program level which will be useful in identifying trends, clarifying funding or program priorities, developing new approaches to health care delivery, or informing general decision-making.

C.4.4.: Technical Assistance and Training IHS frequently provides access to technical assistance (TA) and training to its clinical and public health programs and grantees to support successful program implementation and enhance program outcomes. Whomever the audience, IHS ‘s experience has shown that technical assistance is most effective when it targets a specific need, is delivered by a content expert who has developed a collaborative relationship with the TA recipient, is delivered when the expertise is needed and in a manner that is culturally appropriate and accessible. Technical assistance needs are identified in a variety of ways—IHS-driven based on strategic priorities and objectives and/or programmatic goals, needs assessments, grantee request, Tribal Organization, community, etc.

The Contractor will work with the COR to identify technical assistance needs, identify appropriate resources and respond to requests in a timely manner. Technical assistance and training delivery should take into account the requestor’s organizational capacity, stage of program planning or implementation, Federal program requirements, cultural or linguistic factors, and program objectives.

Establish and Maintain a Technical Assistance and Training Program The Contractor shall establish and maintain a technical assistance (TA) and training program to provide TA and training in one or more of the programmatic areas targeted by IHS. These areas include, but are not limited to: strategic prevention framework, data-driven prevention planning, building community capacity and coalitions, program development and implementation in a variety of service settings, implementation of evidence-based practices, health reform, parity, recovery supports, and other changes in financing or organization of care at national, State, sub- State, and local service delivery systems.

The Contractor shall have the capacity to provide staff or consultants with the expertise to deliver technical assistance or training on a wide variety of topics to a variety of audiences. This will include providing technical assistance and/or training in a culturally appropriate manner.

Technical assistance and training may be provided by experts to IHS grantees, Tribal government representatives, and others.

The Contractor will be responsible for coordinating the logistics of the trainings to include but not limited to securing a site (with priority given to Federal facilities) for the trainings, payment of non-federal sites, and on-site and web-based registration services.

Provide Technical Assistance and Training The Contractor shall provide technical assistance through a variety of mechanisms that may include: training at geographically distant sites; at regional or national meetings; providing expert consultants or staff to deliver workshops, training at leadership institutes and policy academies; or through webinars, podcasts, teleconferences and other social media vehicles in order to deliver technical assistance and training while minimizing costs. The Contractor shall utilize multi-disciplinary teams to provide technical assistance and training, Training of Trainers (TOT), peer-to-peer, etc. Technical assistance and training may be provided simultaneously to multiple programs or sites or may be individualized to a particular organization, grantee or program.

C.5 Special Requirements This section describes the special requirements for this effort. The following sub-sections provide details of various considerations on this effort.

C.5.1 Security and Safety Contractor staff will be required to follow the procedures for obtaining a full Federal background investigation and preliminary clearance. No 3rd party investigation is allowed. In order to obtain a Personal Identity Verification (PIV) card, contractor staff shall complete and pass a Federal background investigation conducted by the Office of Personnel Management. IHS will process the investigations through the Office of Personnel Management on all contractor staff.

Processing of PIV cards shall be completed by the IHS Headquarters HSPD-12/Suitability Office in conjunction with the physical locations throughout IHS and HHS that have PIV card stations.

The contractors will be required to be physically present at one of the HSDP-12 enrollment and issuance workstations to obtain a PIV card. Biometric identification is required.

Each contractor shall complete each of the requirements below and receive pre-employment clearance from IHS prior to performing services under this contract:

1) Fingerprints must be captured electronically at a HHS enrollment workstation or via hard-copy card then sent to the IHS Headquarters Security Office for digital processing;

2) The OF-306, Declaration for Federal Employment, shall be completed;

3) The Addendum to the OF-306, shall be completed;

4) eQIP Information request must be turned into the HSPD-12/Suitability Office;

5) The eQIP questionnaire must be fully and accurately submitted; and

6) The contractor must be enrolled for a PIV card at a HHS HSPD-12 enrollment workstation. Upon receipt of the card, the contractor must activate it at a HHS HSPD-12 card issuance workstation.

If at any point during the investigation process a contractor staff is deemed ineligible or found to have other than favorable suitability, the individual must be removed from providing services under this contract.

During any type of project under this contract, the contractor shall be prepared to address data access and privacy and human protections. The contractor will be responsible for gathering information for any necessary Office of Management and Budget (OMB) clearance or Internal Review Boards (IRB) packages, as appropriate. These activities should be identified at the beginning of the project and anticipated in the technical (including timeline) and business proposal for each Task Order.

Also, in consideration of these data and human protections, third party use of data collected or used during a project is generally prohibited.

C.5.2 Reporting The contractor shall follow the reporting plan submitted as part of the proposal and keep the Government fully informed of status throughout the contract period. Throughout the contract periods, it is essential that attention be given to minimize interruptions or delays to work in progress that would impact the mission.

The Contractor shall provide to the COR, annual performance reports that include but are not limited to project plan status and accomplishments, fiscal accountability, evaluation data, sub- Contractor performance reports (if applicable) and a Contractor self-evaluation report. The Contractor shall provide a written outline of the annual report to the COR for approval according to the deliverable schedule. The annual reports shall be delivered by e-mail and postal service according to the deliverable schedule before the end of the contract period. The annual report shall be discussed at the Start-up meeting.

C.5.3 Storage and Disposition of Materials Disposition of any national training and marketing materials will be determined in the Start-Up meeting. Storage should be available as soon as national materials are generated. A disposition schedule will be discussed with the COR during the Start-Up meeting.

C.5.4 Environmental Requirements The contractor shall comply with all documents listed below as mandatory and referenced under paragraph 3.0, Performance Requirements. Compliance with documents listed as non-mandatory is the contractors' option.

Mandatory compliance document: None Non Mandatory document: None

C.5.5 Correspondence and Conference Calls

The Contractor shall provide weekly updates and correspondence as determined necessary by the

COR.

The IDIQ will have scheduled not less than monthly conference calls with the COR. The monthly conference call schedule may be adjusted as determined by the COR.

Each Task Order will have scheduled not less than monthly conference call with the COR. The monthly task order conference call schedules may be adjusted as determined by the COR (with agreement from Program) and the specific needs of the task order.

C.5.6 Business Associate and Data Use Agreements Once competed, this contract will include an updated version of the general Business Associate Agreement (BAA) found in Appendix D.

When applicable, contractors and/or sub-contractors may be required to enter into a more specific BAA or Data Use Agreement in order to have access to a specific Program’s data, particularly when that data includes Protected Health Information. This will be determined on a project-by-project basis.

C.6 Deliverables The contractor shall provide deliverables as described in subsequent task orders within the above broad categories. Deliverables shall be specified by the government. Format and delivery schedule for deliverables shall be outlined in CDRLs and/or other means TBD.