ID09180048_-_Draft_PWS.pdf

DRAFT PERFORMANCE WORK STATEMENT

Analytical Support for Office of People Analytics

Centers for Health & Resilience (H&R) and Retention & Readiness (R&R)

1.0 INTRODUCTION

The Office of People Analytics (OPA) Centers for Health & Resilience (H&R) and Retention & Readiness (R&R) conduct both web-based and paper surveys and analysis that supports the personnel information needs of the Office of the Under Secretary of Defense for Personnel and Readiness (OUSD [P&R]). Sponsors include, but are not limited to, the Military Community and Family Policy Office, the Transition to Veterans Program Office, and the Sexual Assault Prevention and Response Office. These surveys assess attitudes and opinions on a wide range of personnel issues across the entire DoD community. An illustrative schedule of OPA H&R and R&R surveys is provided as Technical Exhibit 1. In addition to supporting surveys, H&R and R&R partner with other OPA divisions to analyze personnel record data and statistical information, sometimes in combination with survey data, to provide actionable models and information. Some H&R and R&R surveys may require Computer Assisted Telephone Interviewing (CATI) either as the primary data collection method or as an ancillary methodology for multi-method surveys including screening and studies of response and nonresponse. See https://www.DMDC.osd.mil/appj/dwp/dwp_surveys.jsp for an electronic version of examples of surveys and reports. H&R/R&R also conducts “Quick Compass” surveys on specific topics of interest to policymakers (e.g., financial readiness, retention issues, etc.). OPA also conducts focus group studies to help better understand the meaning of survey results related to topics such as gender relations.

2.0 BACKGROUND

2.1 OPA is part of the Defense Human Resources Activity (DHRA), a DoD Field Activity tasked with enabling DoD policy offices to make decisions based on sound administrative/personnel record data and survey data. OPA H&R and R&R focus on providing these offices with survey data. Sponsoring DoD offices and non-DHRA customers require personnel policy- and program-relevant information: attitudes, opinions, behavioral intentions, and self-reported experiences and outcomes. These data analytics are used to formulate, monitor, and refine policies and programs affecting the morale, health, compensation, benefits, training, selection, assignments, and readiness of DoD personnel and their families. As new policies and programs are developed, DoD requires research and studies that quickly analyze survey and administrative data to assess the expected or actual effectiveness of those policies and programs.

2.2 H&R/R&R staff design survey procedures, sampling and weighting plans, and questionnaires, and through analyses, generate documentation including datasets, tabulations, briefings, infographics, and white papers.

2.3 H&R/R&R use four Government-owned, Government developed software packages. 1) The SRT is a visual basic user-interface that uses Microsoft products including Excel, Word, Access, and PowerPoint to develop survey questionnaires and produce automated, publication-ready tabulation volumes, briefing charts, and trend charts. SRT-developed survey questionnaires can be converted into a number of formats (.txt, .doc, etc.) and imported into survey software to create paper- and web-based surveys. The SRT allows the analyst to document the specification of data coding and cleaning in the production of Statistical Analysis Software (SAS) datasets, for specification of variables. Upon initial entry of survey questions in the SRT there is never again a https://www.dmdc.osd.mil/appj/dwp/dwp_surveys.jsp need to retype (risking errors) for creation of the survey on the Web, for creating labeling data variables and values in the dataset, and for including the wording of the items in the automated production of tabulation volumes and briefing charts. 2) The Survey and Variable Authoring System (SURVANT), is an Internet-based tool that is used to develop surveys (web/paper survey creation, testing, editing), for survey database development (variable name, format and value creation), and for documentation (dataset and codebook creation). SURVANT is used to store survey information (variable names, formats, etc.), to test the survey instrument during quality control procedures, to store analysis recodes as well as maintain and produce survey documentation. It is also used to create basic-use (public) and confidential datasets. 3) The Sampling Tool provides efficient samples for H&R/R&R surveys, using Microsoft Excel and Access. The Sampling Tool allows RSSC to estimate the minimum sample size needed to achieve desired precision and confidence goals, given population size, cost, and response rate constraints. 4) The Survey Analysis Macro (SAM) produces estimates and tests of significance within one survey administration. SAM uses SAS, SUDAAN and Microsoft products. 5) H&R/R&R currently use SAS in database creation but also create final datasets in IBM Statistical Package for the Social Sciences (IBM/SPSS) software. SUDAAN, M-Plus, and/or STATA and/or other software packages may also be used to handle the correct estimation of variances.

3.0 SCOPE

The Office of People Analytics (OPA) requires analytical support to plan and analyze Department of Defense (DoD)-wide surveys and focus groups and to conduct in depth analysis using various data sources. The Contractor shall provide all personnel management, equipment and supplies as necessary to perform the requirements as detailed within this PWS. Support will generally be performed in Government office space at the Mark Center in Alexandria, Virginia and via telework.

4.0 REQUIREMENTS. The Contractor shall:

4.1 Prepare and submit a project plan for each task order with milestones and timelines for deliverables for government approval. Revise the plan as necessary and report any issues to the Contracting Officer’s Representative (COR) in a timely manner.

4.2 Provide research, survey, and/or statistical consultation, advice and support to OPA and other OSD and Secretary of Defense stakeholders (e.g. Under Secretary of Defense for Personnel and Readiness, Defense Civilian Personnel Advisory Service (DCPAS), and Sexual Assault Prevention and Response Office (SAPRO)). Anticipated products include briefings, infographics, technical reports, information papers, management (executive) reports, overview reports and papers for presentation at professional conferences or submitted to professional journals containing analysis relevant to DoD actions, policies, or procedures. Other products include user manuals for personnel or manpower policy/program implementation, documentation of surveys and codebooks, and maintenance/enhancements of computer program/web site applications supporting the OPA mission.

4.3 Provide statistical support by designing complex, stratified samples, and weighting survey data to account for differential selection probabilities and response propensities, and post-stratification.

Use software (e.g., SUDAAN) to appropriately make variance estimates from complex sample designs. Use and modify H&R/R&R statistical software programs to perform appropriate statistical tests to determine whether differences in estimates across subgroups (e.g., Service) or time are statistically significant. Perform nonresponse bias analyses and serve as technical expert on nonresponse bias methods. Work with government staff to develop an automated production system to impute for missing data, and serve as technical expert on missing data methods. In addition, disclosure analysis shall be conducted on datasets to be delivered outside of OPA.

4.4 Support survey design, construction, and execution of survey studies using self-administered web-based and paper and pencil questionnaires, as well as local, nation-wide, or world-wide focus groups; world-wide computer-assisted telephone interviewing (CATI); nation-wide in-person interviewing protocols; and pretesting protocols including those requiring cognitive pretesting.

This can include the analyses and reporting of existing and/or new data and/or survey methodologies.

4.5 Assist in the design and development of specifications used in final products including tabulation of responses, briefings, management (executive) reports, overview reports, and survey notes.

4.6 Assist with preliminary and complex analyses of data including univariate and multivariate statistical analyses (e.g., logistic regression, path analysis), and structural equation modeling (SEM) using SAS and other statistical analysis package skills.

4.7 Assist in designing content for focus groups, conducting focus groups, and conducting analysis of focus group sessions by identifying major themes and ideas conveyed across the sessions. For each theme, include supporting comments from the focus group participants.

4.8 Assist with quick turn ad hoc analyses and requests for information regarding OPA research findings form OPA customers.

4.9 In support of OPA’s statistical group, provide expert advice and programming for performing analyses, technical notes or memorandum documenting the statistical issues considered, and recommendations made. Such reports are to be complete with equations, citations, results of analyses performed, and copies of analysis statements prepared for execution in SAS, SUDAAN, or other government approved software packages.

4.10 In support of OPA’s data science group, provide expert advice (e.g., analysis of alternatives, etc.)

and programming (i.e., documented with R markdown and Python Notebook or similar platforms) on diverse data science projects including data wrangling (e.g., extract, transform, and load operations), statistical and mathematical algorithm deployment (e.g., structured and unstructured data), document algorithm performance (e.g., hold out sample validation), using R, Python, and Spark analysis platforms on both single and multiple node platforms.

4.11 Support the review and approval of internally and externally generated surveys and compliance with laws, instructions, and directives (e.g,, DoDI 1100.13; DoDI 8910.01, etc. on use of surveys for data collection, use of survey data for secondary analyses, and use of human subjects in research.

4.12 Attend conferences and meet with federal, state, and local officials as well as members of the public and constituent groups in order to remain current with industry trends relative to government surveys.

4.13 Update and document procedures for automating survey operations including questionnaire development, briefing/trend charts, and analyses. Provide internal training for any new processes.

4.14 Develop coding schemes. Validate and document survey findings in tabulation volumes, trend reports and briefings, and management reports. Document datasets in codebooks. See http://www.dtic.mil/dtic/tr/fulltext/u2/a476557.pdf, http://www.dtic.mil/docs/citations/ADA514130 for examples of codebooks.

4.15 Conduct quality assurance tasks related to survey operations. Quality assurance checks and data validation should include dataset/product/deliverable quality checks, inter-rater reliability for qualitative data, and frequency runs to verify sample and population characteristics.

4.16 Develop and maintain a system for the tracking of both internal and external H&R/R&R milestones.

4.17 Distribute datasets in response to requests for current and historical data in compliance with

Freedom of Information Act requirements, Privacy Advisory or Privacy Act statements, and consistent with OPA procedures to ensure integrity and security of data.

4.18 Provide project management capabilities including scheduling meetings (e.g Inter Service Survey

Coordinating Committee-ISSCC), developing agendas, documenting meeting minutes, and coordinating survey review packages.

4.19 Coordinate the completion and dissemination of research results within OPA and to external clients through Web publication and email notification.

4.20 Tracks the details of all OPA data requests.

4.21 Provide technical support and maintenance for OPA in-house survey tools. OPA currently uses

Survey Reporting Tool (SRT), Sampling Tool, and Statistical Analysis Macros (SAM).

4.22 Be proficient in the use of Microsoft Office ® software applications (Excel, Access, Word, Outlook, and Project) and SharePoint®. Contractor personnel shall possess strong writing and analytic skills and be well versed and able to apply knowledge of DoD personnel management policies as they relate to statistical analysis.

4.23 Provide Monthly Progress Reports to the COR that address the extent to which any problems or circumstances will cause conflicts with program schedules. Present and discuss the report in quarterly Senior Management Reviews (SMRs). Reports and discussions shall include (1) summary of activities, actions, and progress for the prior quarter; (2) status of each deliverable;

(3) financial review for billing purposes, (4) pending issues; (5) analysis of any potential problems, risks/challenges, and risk mitigation plan; and (6) summary of projected activities for the next period. Submit minutes of the SMR to the COR within 5 days following each meeting.

4.24 Apply all applicable professional styles and standards of Offices of the Secretary of Defense and

USD (P&R), and OPA to technical documentation and survey notes produced for evaluations, studies, and analyses, as well as memos, information papers, and briefings related to DoD personnel policies and programs. All editing shall be done according to the publication manual of OPA, “Guidelines for Preparing and Submitting Technical Documentation for Publication.” In addition, technical editing by the contractor must be consistent with the latest edition of the Publication Manual of the American Psychological Association (APA). Maintains OPA Word and PowerPoint templates and maintains the OPA report tracking system.

4.25 Appoint a Program Manager. The Program Manager shall be the Contractor's primary point of contact for the government on all areas of performance under this requirement.

http://www.dtic.mil/dtic/tr/fulltext/u2/a476557.pdf http://www.dtic.mil/docs/citations/ADA514130

4.24 Participate in a Post-Award Conference. This meeting provides an introduction between the Contractor personnel and Government personnel who will be involved with the contract and will aid both parties in achieving a clear and mutual understanding of all requirements and identify and resolve any potential issues. However, this meeting is not a substitute for the contractor fully understanding the work requirements at the time offers are submitted, nor is it to be used to alter the final agreement arrived at in any negotiations leading to contract award. The Contractor shall be prepared to discuss any items requiring clarification and gather information as necessary to support each deliverable and shall submit a written summary of the Post-Award Conference to the

COR.

4.25 Contractor Manpower Reporting

4.25.1 The contractor shall report ALL contractor labor hours (including subcontractor labor hours) required for performance of services provided under this contract for OPA via a secure data collection site. The contractor is required to completely fill in all required data fields using the following web address: http://www.ecmra.mil/.

4.25.2 Reporting inputs will be for the labor executed during the period of performance during each

Government fiscal year (FY), which runs October 1 through September 30. While inputs may be reported any time during the FY, all data shall be reported no later than October 31 of each calendar year. Contractors may direct questions to the help desk at: http://www.ecmra.mil/

5.0 DELIVERABLES

The specific deliverables will vary given the specific task order. Following is a listing of the types of work products/deliverables that the Contractor may be required to provide in performance of the PWS under resulting task orders: Task Order Project Plan, develop and maintain a system for the tracking of both internal and external H&R/R&R milestones, monthly progress reports, quarterly Senior Management Review (SMR) report, SMR minutes, Post-Award Conference, Post-Award Conference Written summary, Contractor Manpower Reporting, Quality Control Plan (Final), demonstration of capability to take over functions of SRT, Sampling Tool, and SAM2, and demonstration of capability to use SURVANT. One or several may be required under any given task order.

6.0 QUALITY CONTROL

The contractor shall implement and maintain a Quality Control Plan (QCP) to ensure work performed conforms to the scope of work and meets the requirements under this PWS. The QCP shall, at a minimum provide a method for performing inspections; identifying, correcting and preventing problems/defective service; addressing customer complaints, and improving the quality of services over the life of the contract. The final QCP shall be delivered 30 days after award.

7.0 QUALITY ASSURANCE

7.1 The Government reserves the right to perform inspections and surveillance to evaluate the

Contractor’s compliance to the contract terms and performance of the requirements in the PWS.

The Government will make every effort to ensure that the surveillance methods described below are conducted in an objective, fair, and consistent manner.

7.1.1 Periodic Surveillance. This action occurs when the COR or other Government official observes a deficiency. Examples include evidence from errors, incidents, or delays. Regardless of where in http://www.ecmra.mil/ http://www.ecmra.mil/ the line-of-duty the COR observes contractual procedures not being followed, he/she has an obligation to document and report the deficiency to the Contracting Officer.

7.1.2 Customer Complaint Surveillance. This action is instituted when the COR receives a complaint from a stakeholder regarding contractor service. The COR will obtain the complaint in writing and then conduct an investigation to determine its validity. If the complaint is deemed valid, the COR will immediately notify the contracting Officer for action. The COR will notify both the Contract Manager and the complainant of the Government’s response to their complaint.

7.1.3 Contract Discrepancy Report (CDR) (Technical Exhibit 2). In the event of contractor’s unsatisfactory performance, the COR will issue a CDR that will explain the circumstances and findings concerning the incomplete or unsatisfactory service. The contractor shall acknowledge receipt of the CDR and respond in writing as to how he/she shall correct the unacceptable performance and avoid a recurrence. The Government will review the Contractor's corrective action response to determine acceptability and will use any completed CDR as part of an overall evaluation of contractor performance when determining present or future contractual actions.

8.0 SECURITY CLASSIFICATION

8.1 Comply with DoD Directive 8500.1 Information Assurance (IA), DoD Instruction 8500.2

Information Assurance (IA) Implementation, DoD Directive 5400.11 DoD Privacy Program, DoD 6025.18-R DoD Health Information Privacy Regulation, DoD 5200.2-R Personnel Security Program, and Homeland Security Presidential Directive (HSPD) 12 for all contractor personnel with access to or responsibility for nonpublic Government data.

8.2 Establish appropriate administrative, technical, and physical safeguards to protect any and all nonpublic Government data to ensure the confidentiality, integrity, and availability of government data.

8.3 Contractor personnel with access to or responsibility for nonpublic Government data under this contract must comply with HSPD-12 Personal Identity Verification (PIV) issuance requirements, known as the Common Access Card (CAC) for OPA and must:

8.3.1 Be CAC or PIV ready prior to reporting for work. At minimum all contractor personnel must obtain/maintain a favorable FBI National Criminal History Check (fingerprint check), two forms of identity proofed identification (I-9 document), and submit a National Agency Check and Law Credit (NACLAC) vetting package for processing. Obtaining CAC or PIV ready status is the responsibility of the contracting agency. It is the responsibility of the contracting agency to notify OPA when this is complete.

8.3.2 Be citizens of the United States.

8.4 If at any time, any Contractor person requiring a CAC is unable to obtain/maintain an adjudicated

NACLAC, the Contractor shall immediately notify the OPA Information Systems Security Group (DISSG) remove such person from work under this contract.

8.5 The Contractor shall report immediately to the OPA CIO / Privacy Office and secondly to the COR discovery of any privacy breach.

9.0 GOVERNMENT FURNISHED EQUIPMENT OR INFORMATION

Up to seven workstations will be provided at the Mark Center in Alexandria, Virginia. Computer equipment and software will be provided for on-site use by contractor employees.

10.0 TRAVEL

Contractor costs for Government authorized travel are included in this contract. All travel shall be in accordance with joint travel regulations (JTR). Contractor payment claims shall include applicable documentation to support actual costs incurred (e.g. airfare and hotel/lodging receipts) as well as any receipts valued at or above $75.00. Failure to provide appropriate documentation may result in loss of reimbursement of travel expenses.

11.0 TRANSITION PLAN

Provide Transition of Contract Services - The contractor shall provide a plan for transition in and out services to ensure minimum disruption to vital Government business. This plan shall address how the Contractor will work with the incumbent and Government personnel to ensure that there will be no service degradation during and after the transition-in changeover (initial thirty (30) day period after date of contract award) and during the transition-out changeover (ninety (90) day period prior to date of contract expiration). During transition-out (1) demonstrate OPA software tools or similar contractor-provided tools and provide instruction on maintenance, operation, and training for the Survey Reporting Tool (SRT), Sampling Tool, SURVANT, and Statistical Analysis Macros (SAM) or similar contractor provided tools.

12.0 APPLICABLE DOCUMENTS

DoD Directive 8500.1 Information Assurance (IA) http://www.dtic.mil/whs/directives/corres/pdf/850001_2014.pdf

DoD Instruction 8500.2 Information Assurance (IA) Implementation http://www.cac.mil/docs/DoDD-8500.2.pdf

DoD Directive 5400.11-R DoD Privacy Program http://www.dtic.mil/whs/directives/corres/pdf/540011r.pdf

DoD 6025.18-R DoD Health Information Privacy Regulation http://www.dtic.mil/whs/directives/corres/pdf/602518r.pdf

DoD 5200.2-R Personnel Security Program http://www.dtic.mil/whs/directives/corres/pdf/520002r.pdf

Homeland Security Presidential Directive (HSPD) 12 http://www.dhs.gov/homeland-security-presidential-directive-12

Protection of Human Subjects and Adherence to Ethical Standards in DOD-Supported Research

DOD 3216.02, 32 CFR 219

http://www.dtic.mil/whs/directives/corres/pdf/321602p.pdf

DoD Instruction 1100.13, Surveys of DoD Personnel http://www.dtic.mil/whs/directives/corres/pdf/110013p.pdf

DoD Instruction 8910.01 Information Collection and Reporting http://www.dtic.mil/whs/directives/corres/pdf/891001p.pdf

DoD Manual 8910.01, volume 1 Procedures for DoD Internal Information Collections http://www.dtic.mil/whs/directives/corres/pdf/891001m_vol1.pdf

DoD Manual 8910.01, volume 2 Procedures for DoD Public Information Collections http://www.dtic.mil/whs/directives/corres/pdf/891001m_vol2.pdf

Publication Manual of the American Psychological Association, Sixth Edition http://www.dtic.mil/whs/directives/corres/pdf/850001_2014.pdf http://www.cac.mil/docs/DoDD-8500.2.pdf http://www.dtic.mil/whs/directives/corres/pdf/540011r.pdf http://www.dtic.mil/whs/directives/corres/pdf/602518r.pdf http://www.dtic.mil/whs/directives/corres/pdf/520002r.pdf http://www.dhs.gov/homeland-security-presidential-directive-12 http://www.dtic.mil/whs/directives/corres/pdf/321602p.pdf http://www.dtic.mil/whs/directives/corres/pdf/110013p.pdf http://www.dtic.mil/whs/directives/corres/pdf/891001p.pdf http://www.dtic.mil/whs/directives/corres/pdf/891001m_vol1.pdf http://www.dtic.mil/whs/directives/corres/pdf/891001m_vol2.pdf http://www.apastyle.org/ http://www.apastyle.org/

TECHNICAL EXHIBIT 1

Anticipated H&R/R&R Survey Schedule for 2019, etc.

Survey Name Frequency Data Collection

● Active Duty SOFS (SOFA) 1 per year September

● Reserve SOFS (SOFR) 1 per year June

● Reserve Component Gender Relations Every 2 years 2019-usually July

(WGRR)

● Active Duty Gender Relations (WGRA) Every 2 years 2020-usually July

● Reserve Equal Opportunity (WEOR) Every 4 years 2019-usually November

● Active Duty Equal Opportunity (WEOA) Every 4 years 2021-usually November

● Service Academy Focus Groups(SAGR) Every 2 years 2019 March/April

● Service Academy Survey (SAGR) Every 2 years 2020 March/April

● Living Patterns (LPS) Every 3 years April 2020

● Active Duty Spouses (ADSS) Every 2 years 2019

● Reserve Spouses (RCSS) Every 2 years 2020

TECHNICAL EXHIBIT 2

CONTRACT DISCREPANCY REPORT (CDR)

1. Contract Number: <insert number>

2. TO: (Contractor’s Manager or on-site representative) <insert name>

3. FROM: (Name of COR) <insert name>

4. Date and time observed discrepancy:

5. DISCREPANCY OR PROBLEM:

<Describe in detail. Identify any attachments.>

5. Corrective action plan:

A written corrective action plan < is / is not > required.

< If a written corrective action plan is required include the following. > The written Corrective Action Plan will be provided to the undersigned not later than < # days after receipt of this

CDR. >

Prepared by: <Enter COR’s name>

Signature – Contracting Officer’s Representative Date

Received by:

Signature – Contractor’s Manager or on-site representative Date