PERFOR~2.DOC

PART 1: GENERAL INFORMATION

1.1 Description of Services/Introduction

The Contractor shall provide all personnel, equipment, supplies, facilities, transportation, tools, materials, supervision, and other items and non-personal services necessary to perform the development of a NCCC Impact Studies as defined in this Performance Work Statement (PWS) except for those items specified as government furnished property and services. The Contractor shall perform to the standards referenced in this PWS.

1.2 Background Information

The Corporation for National and Community Service (CNCS) is a federal agency that engages more than 5 million Americans in service through its core programs. The mission of CNCS is to improve lives, strengthen communities, and foster civic engagement through service and volunteering. AmeriCorps National Civilian Community Corps (NCCC) is a ten month, full-time, team-based residential national service program for young men and women ages 18-24. The program is built on the belief that civic responsibility is an inherent duty of all citizens and that a national service program can effectively work with local communities to address pressing needs. The specific mission of AmeriCorps NCCC is to strengthen communities and develop leaders through direct team-based national and community service.

AmeriCorps NCCC has four regional residential campuses[footnoteRef:2] located in Sacramento, California; Vicksburg, Mississippi; Aurora, Colorado; and Vinton, Iowa. Members complete service projects in all 50 states and the U.S. territories in partnership with nonprofit organizations, state and local agencies, and community and faith-based groups. AmeriCorps NCCC maintains two programs: 1) Traditional NCCC; and, 2) FEMA Corps. [2: NCCC had a Baltimore campus that was closed in June 2018.]

Traditional NCCC is a program where members perform a variety of service projects, such as leading youth development activities, constructing and rehabilitating low-income housing, performing environmental clean-up, helping communities develop emergency plans, and addressing other pressing local needs. AmeriCorps Traditional NCCC members are assigned to one of the four regional campuses, placed into teams of approximately ten members, and complete approximately three to four projects, of six to eight weeks in length, responding to local community needs in every state. In each regional campus, there are two type of NCCC assignments: team leaders or Corps members. Team leaders are assigned a group of about 10 members and work as a team for the 10-month period.

FEMA Corps is an AmeriCorps NCCC program for young adults developed in partnership with the Federal Emergency Management Agency (FEMA) who want to gain professional skills in emergency management while serving with FEMA on disaster response and recovery efforts. The members work solely on emergency management and long-term recovery activities within FEMA helping coordinate services for disaster survivors. More information on NCCC is on the CNCS website: https //www.nationalservice.gov/programs/americorps/americorps-nccc.

In fiscal year 2019, three impact studies on Leadership Development, Member Retention, and Strengthening Communities were developed during the planning and design phase of the NCCC national impact studies. CNCS is seeking to conduct these study designs to identify how participation in NCCC promotes leadership qualities and skills among its members, what factors affect retention, and how NCCC impacts the communities in which NCCC members served. These national impact studies are closely linked to the CNCS’s strategic plan which aims to strengthen national service so CNCS program participants consistently find satisfaction, meaning, and opportunity (Goal 1) and increase the impact of national service in CNCS-served communities (Goal 2).

1.3 Objectives

The purpose of this requirement is for the Contractor to conduct three pre-designed impact studies that include the following:

1.4 Scope

· The Contractor shall conduct three impact studies on Leadership Development, Member Retention, and Strengthening Communities by implementing three study designs previously developed in the planning phase.

· The Contractor shall populate the NCCC Service Project Database with quantitative and qualitative project data on NCCC service projects completed between summer 2019 and 2020 fiscal year.

· Contractor shall secure human subjects approval for the studies from an accredited Institutional Review Board (IRB).

· The Contractor shall establish a Field Working Group (FWG) to provide programmatic expertise to help conduct the studies.

· The Contractor shall establish two Technical Working Groups (TWGs), one for youth leadership development/retention, and a second for strengthening communities (i.e., conservation, disaster services, community development). Each TWG should have sufficient technical expertise on the subject area to advise the Contractor on data collection, sampling, and analysis.

· The Contractor shall submit an annual report describing the progress and findings of the three studies for each contract year.

· The Contractor shall prepare and present a webinar to report on the progress and findings of the three studies at the end of each contract year.

· The Contractor shall submit three final reports for the Leadership Development, Member Retention, and Strengthening Communities studies that is suitable for public dissemination. Each report shall detail the steps taken to conduct the evaluation, present the findings by research questions, and provide recommendation relevant to program improvement and enhancement.

1.5 Period of Performance

The period of performance for the project shall be a 12-month base contract with three 12-month option years. The project shall begin upon award of the contract in 2019. Each option year will be contingent on funding availability, contractor performance, and findings from the prior contract or option year.

1.6 General Information

1.6.1 Quality Control

The Contractor shall develop and maintain an effective quality control program to ensure services are performed in accordance with this PWS. The Contractor shall develop and implement procedures to identify, prevent, and ensure non-recurrence of defective services. The Contractor’s quality control program is the means by which the Contractor assures that the work complies with the requirement of the contract. The Contractor shall deliver the quality control plan to the CO and COR via e-mail as a PDF within 15 days after contract award. After acceptance of the quality control plan the Contractor shall receive the contracting officer’s acceptance in writing of any proposed change to Contractor Quality Control (QC) system.

1.6.2 Quality Assurance

The government shall evaluate the Contractor’s performance under this contract in accordance with the Quality Assurance Surveillance Plan (QASP), see Attachment 1. This plan is primarily focused on what the Government must do to ensure that the Contractor has performed in accordance with the performance standards. It defines how the performance standards will be applied, the frequency of surveillance, and the minimum acceptable defect rate(s).

1.6.3 Recognized Holidays

The Contractor is not required to perform services during the following holidays

1.6.4 Hours of Operation

The Contractor is responsible for conducting business, between the hours of 8:00 am and 5:00 pm Monday through Friday except Federal holidays or when the Government facility is closed due to local or national emergencies, administrative closings, or similar Government directed facility closings. For other than firm-fixed price contracts, the Contractor will not be reimbursed when the government facility is closed for the above reasons. The Contractor must at all times maintain an adequate workforce for the uninterrupted performance of all tasks defined within this PWS when the Government facility is not closed for the above reasons. When hiring personnel, the Contractor shall keep in mind that the stability and continuity of the workforce are essential.

1.6.5 Place of Performance

The work to be performed under this contract will be performed at Contractor facilities, or any suitable location selected by the Contractor.

1.6.6 Type of Contract

The government anticipates a firm-fixed contract.

1.6.7 Special Qualifications

The contractor shall provide documentation of human subjects training for project personnel. In addition, the contractor shall comply with CNCS privacy policies and procedures, adhere to CNCS standards for the protection of personal identifiable information, and undergo CNCS privacy and network training, as appropriate.

1.6.8 Post Award Conference/Periodic Progress Meetings

The Contractor agrees to attend any post award kick-off meeting convened by the contracting activity or contract administration office in accordance with Federal Acquisition Regulation Subpart 42.5.

1.6.9 Contracting Officer Representative (COR)

The (COR) will be identified by a separate designation letter. The COR monitors all technical aspects of the contract and assists in contract administration. The COR is authorized to perform the following functions assure that the Contractor performs the technical requirements of the contract, perform reviews necessary in connection with contract performance, maintain written and oral communications with the Contractor concerning technical aspects of the contract, issue written interpretations of technical requirements, including Government drawings, designs, specifications, monitor Contractor's performance and notifies both the Contracting Officer and Contractor of any deficiencies; coordinate availability of government furnished property, and provide site entry of Contractor personnel to CNCS offices. A letter of designation issued to the COR, a copy of which is sent to the Contractor, states the responsibilities and limitations of the COR, especially with regard to changes in cost or price, estimates or changes in delivery dates. The COR is not authorized to change any of the terms and conditions of the resulting order.

1.6.10 Key Personnel

The following personnel are considered key personnel by the government: Project Manager, Senior Researchers, Mid-Level Researchers, and Subject Matter Experts. The Contractor shall provide a project manager who shall be responsible for the performance of the work. The name of this person and an alternate who shall act for the Contractor when the manager is absent shall be designated in writing to the contracting officer. The Project Manager or alternate shall have full authority to act for the Contractor on all contract matters relating to daily operation of this contract. The Project Manager or alternate shall be available between 8:00 pm and 5:00 pm, Monday through Friday except Federal holidays or when the government facility is closed for administrative reasons. Qualifications for all key personnel are as follows

· The Project Manager must have the following minimum education and experience

· Ph.D. or equivalent

· At least 4 years of relevant experience managing a diverse pool of technical personnel with social science expertise performing work on a contract

· At least 4 years of experience managing a large federal government contract

· Conducting social science research including longitudinal and mixed-method studies

· Demonstrated knowledge of NCCC AmeriCorps program mission and structure

· Writing quality and clear research reports for public dissemination

· The Senior Researchers must have the following minimum education and experience

· Ph.D. or equivalent

· At least 4 years of relevant experience managing a diverse pool of technical personnel with social science expertise performing work on a contract

· At least 4 years of experience managing a large federal government contract

· Conducting social science research including longitudinal and mixed-method studies

· Demonstrated knowledge of NCCC AmeriCorps program mission and structure

· Writing quality and clear research reports for public dissemination

· Other key personnel such as Mid-level Researchers and Subject Matter Experts shall possess expertise in the following areas:

· Conducting qualitative and quantitative research studies including survey research, focus groups, document review, key informant interviews, and others

· Constructing samples, conducting large scale surveys with different modes for administering surveys (i.e. telephone, online)

· Expertise in using both quantitative and qualitative data to conduct mixed-method data analysis

· Expertise in qualitative data analysis – including document review, thematic coding, inductive and deductive data analysis – and developing processes for inter-rater reliability

· Building databases, data coding, data storage, data analysis and reporting

· Writing quality and clear research reports for public dissemination

· Proposed technical personnel must show experience of building strong collaborative working relationships and the ability to be responsive to the needs of multiple stakeholders

1.6.11 Travel

The Contractor’s Project Manager and Key Personnel shall travel to complete this contract.

· The Project Manager and other Key Personnel (up to 3 staff) shall travel to NCCC HQ for the yearly Project Planning Meetings (up to 4 meetings).

· The Project Manager and/or Key Personnel (up to 3 staff per campus) will travel to NCCC campuses and project sites to conduct. The travel will be authorized and approved by the COR. The cost of this travel will be reimbursed by the federal government in accordance with the Federal Travel Regulations (FTR).

1.6.12 Other Direct Costs

All travel costs including transportation, lodging and miscellaneous costs must be pre-approved by the COR.

1.6.13 Data Rights

The Government has unlimited rights to all documents/material produced under this contract. All documents and materials, to include the source codes of any software, produced under this contract shall be Government owned and are the property of the Government with all rights and privileges of ownership/copyright belonging exclusively to the Government. These documents and materials may not be used or sold by the Contractor without written permission from the Contracting Officer. All materials supplied to the Government shall be the sole property of the Government and may not be used for any other purpose. This right does not abrogate any other Government rights.

1.6.14 Organizational Conflict of Interest

Contractor personnel performing work under this contract may receive, have access to or participate in the development of proprietary or source selection information (e.g., cost or pricing information, budget information or analyses, specifications or work statements, etc.) or perform evaluation services which may create a current or subsequent Organizational Conflict of Interests (OCI) as defined in FAR Subpart 9.5. The Contractor shall notify the Contracting Officer immediately whenever it becomes aware that such access or participation may result in any actual or potential OCI and shall promptly submit a plan to the Contracting Officer to avoid or mitigate any such OCI. The Contractor’s mitigation plan will be determined to be acceptable solely at the discretion of the Contracting Officer and in the event the Contracting Officer unilaterally determines that any such OCI cannot be satisfactorily avoided or mitigated, the Contracting Officer may effect other remedies as he or she deems necessary, including prohibiting the Contractor from participation in subsequent contracted requirements which may be affected by the OCI.

1.6.15 Phase In /Phase Out Period

There is no requirement for the Contractor to have personnel on-board during phase-in/phase-out periods.

PART 2: DEFINITIONS AND ACRONYMS

2.1 Definitions

2.1.1 CONTRACTOR.

A supplier or vendor awarded a contract to provide specific supplies or service to the government. The term used in this contract refers to the prime.

2.1.2 CONTRACTING OFFICER.

A person with authority to enter into, administer, and or terminate contracts, and make related determinations and findings on behalf of the government. Note: The only individual who can legally bind the government.

2.1.3 CONTRACTING OFFICER'S REPRESENTATIVE (COR).

An employee of the U.S. Government appointed by the contracting officer to administer the contract. Such appointment shall be in writing and shall state the scope of authority and limitations. This individual has authority to provide technical direction to the Contractor as long as that direction is within the scope of the contract, does not constitute a change, and has no funding implications. This individual does NOT have authority to change the terms and conditions of the contract.

2.1.4 DELIVERABLE.

Anything that can be physically delivered but may include non-manufactured things such as meeting minutes or reports.

2.1.5 KEY PERSONNEL.

Contractor personnel that are evaluated in a source selection process and that may be required to be used in the performance of a contract by the Key Personnel listed in the PWS. When key personnel are used as an evaluation factor in best value procurement, an offer can be rejected if it does not have a firm commitment from the persons that are listed in the proposal.

2.1.6 PHYSICAL SECURITY.

Actions that prevent the loss or damage of Government property.

2.1.7 QUALITY ASSURANCE.

The government procedures to verify that services being performed by the Contractor are performed according to acceptable standards.

2.1.8 QUALITY ASSURANCE SURVEILLANCE PLAN (QASP).

An organized written document specifying the surveillance methodology to be used for surveillance of Contractor performance.

2.1.9 QUALITY CONTROL.

All necessary measures taken by the Contractor to assure that the quality of an end product or service shall meet contract requirements.

2.1.10 SUBCONTRACTOR.

Subcontractor Utilization is applicable to this requirement. However, the government intention is to award this contract to the Prime Contractor without their intention to hire a subcontractor to fulfill the contract requirements.

2.1.11 WORK DAY.

The number of hours per day the Contractor provides services in accordance with the contract.

2.1.12 WORK WEEK.

Monday through Friday, unless specified otherwise.

2.2 Acronyms.

PART 3: GOVERNMENT FURNISHED PROPERTY, EQUIPMENT, AND SERVICES

3.1 Services

The Government will provide scheduling services for conference rooms and work space, at the government facilities when required.

3.2 Materials

The Government will provide access to AmeriCorps NCCC project data, standard operating procedures and policies.

PART 4: CONTRACTOR FURNISHED ITEMS AND SERVICES

4.1 General

The Contractor shall furnish all supplies, equipment, facilities and services required to perform work under this contract that are not listed under Section 3 of this PWS.

4.2 Materials

The Contractor shall furnish materials, supplies, and equipment necessary to meet the requirements under this PWS

4.3 Equipment

The Contractor shall furnish equipment to meet the requirements under this PWS.

PART 5: SPECIFIC TASKS

5.1 Basic Services

The Contractor shall provide all personnel, equipment, supplies, facilities, transportation, tools, materials, supervision, and other items and non-personal services necessary to conduct three impact studies as defined in this Performance Work Statement (PWS) except for those items specified as government furnished property and services. The Contractor shall perform to the standards referenced in this PWS. All the tasks in the contract shall be performed and supervised by the primary Contractor.

5.2 Administrative Activities

5.2.1 Kick-off Meeting

A kick-off meeting shall occur no later than one week after the contract award. During the meeting, the Contractor’s project manager and key project staff shall meet with the COR and other appropriate CNCS staff to discuss the tasks outlined in the proposal, project timeline, and other key issue related to the contract. This meeting may be conducted via telephone conference and/or web conference.

The Contractor shall work with the COR to develop an agenda for the meeting. A draft of the agenda shall be delivered to the COR three business days prior to the kick-off meeting for review and approval. The agenda shall include, but may not be limited to, discussion of tasks and deliverables outlined in the specified tasks. The Contractor shall also come prepared to identify any areas of concern and to suggest ways of responding to these concerns.

The Contractor will be responsible for taking detailed notes during the meeting. Within five business days following the meeting, the Contractor shall prepare and deliver a memo that summarizes the discussion, suitable for broad distribution. The memo shall include key issues and concerns raised at the meeting, and how each will be addressed in the study. The memo shall also include any decisions reached, justification for the decisions, and next steps.

5.2.2 Project Meetings

The project manager and additional key staff (as needed) shall meet weekly with the COR and other appropriate CNCS staff during the life of the contract. Additional meetings shall be scheduled on an as-needed basis. In general, these meetings shall be conducted via telephone conference call, although circumstances may require in-person meetings periodically during the contract performance period. The purpose of these meetings is to discuss details regarding the tasks outlined for the project, the schedule of activities, and other issues related to the contract. If additional staff members are needed to attend a meeting, the Contractor and COR shall keep each other informed with adequate timing so that the appropriate staff can be invited. The Contractor shall come prepared to identify any areas of concern and to suggest ways of responding to these concerns.

The Contractor shall work with the COR to develop an agenda for each weekly project meeting. A draft of the agenda shall be delivered to the COR one business day prior to the weekly meeting for review and approval. The Contractor shall work with the COR to determine whether other written materials will be needed for the meetings and identify the appropriate format and timeline for developing and disseminating the materials prior to the meeting. The Contractor will be responsible for taking notes during the meeting and send to the COR a memo summarizing any key issues, discussions, and decisions within one week of the meeting.

5.2.3 Monthly Progress/Exception Reports

The Contractor shall prepare monthly progress reports, due at the same time as the monthly invoice. The monthly progress reports shall summarize the major activities and accomplishments for the reporting period. In addition, they shall provide information for each project task regarding significant findings and events, problems encountered, and staff use. The reports shall also specify the extent to which the project is on schedule, briefly describe the activities planned for the next month, identify and discuss significant deviations from the requirements in the project work plan, and identify and discuss any decision that may be needed from CNCS. If there are no exceptions, the reports shall state that there are no exceptions. If there are exceptions to the work plan, the Contractor shall describe the plan for resolving the problems.

5.2.4 Project Planning Meeting

At the start of each contract year, the Contractor shall be available for an on-site planning meeting of 4 to 8 hours at HQ with NCCC and ORE staff. The purpose of the planning meeting is to discuss

· each task in detail,

· identify any issues,

· discuss potential solutions, and

· draft a Project Work Plan (Subtask 5.2.6).

The Contractor shall draft an agenda and submit to the COR three business days prior to the meeting and shall work with the COR to determine whether other written materials will be needed. The Contractor will be responsible for taking notes during the meetings and send to the COR a memo summarizing any key issues, discussions, and decisions within one week after the meeting.

5.2.5 Project Work Plan

At the start of the contract, based on discussions in the Project Planning Meeting (Subtask 5.2.4), the Contractor shall develop a workplan to carry out the activities for the tasks included in the PWS. The workplan shall be written with sufficient detail, and with performance markers, so that it can be used to monitor progress. The workplan shall describe in detail, the tasks required, the sequence of tasks, the staff that will be performing each task, including key personnel, the resources and tools needed to complete the task, the expected timeline, potential obstacles, and strategies for overcoming those obstacles. The Contractor shall, with the coordination and approval of the COR, continue to revise the workplan as needed. Changes to the workplan shall be documented by the Contractor with justifications, and approved by the COR and the Contractor Officer. The schedule for delivery of the workplan will be determined at the kick-off meeting and updated at the Project Planning Meeting.

5.2.6 Establish Field Working Group of NCCC Staff

With the assistance of the COR and NCCC, the Contractor shall establish a Field Working Group (FWG) to provide programmatic expertise to help conducting the studies. The FWG shall consists of NCCC program directors and/or staff and Research and Evaluation Office staff. Based on CNCS’ recommendation, the contractor shall confirm the membership and submit a memo with a list of FWG members and a schedule of FWG meetings to the COR within one month after the contract is awarded.

The Contractor shall provide FGW members with all documents at least one week prior to each FW meeting, and shall submit a memo to the COR summarizing each FWG meeting with two weeks after each meeting. The Contractor shall provide CNCS with all questions, comments, concerns, and recommendations from FWG members.

5.2.7 Establish Technical Working Groups of Experts

In consultation with CNCS, the Contractor shall establish two Technical Working Groups (TWGs), one for youth leadership development/retention, and a second for strengthening communities (i.e., conservation, disaster services, community development). Each TWG should have sufficient technical expertise on the subject area to advise the Contractor on data collection, sampling, and analysis.

Each TWG would be comprised of up to 6 individuals, not including CNCS staff. Both CNCS and the Contractor shall gather a list of potential individuals. The Contractor shall submit a list of proposed TWG members to the COR for review and approval by CNCS. The Contractor shall evaluate the potential for conflict of interest by assessing whether a potential TWG member has bias or financial interest in the subjects under study or the outcome of the study. The Contractor shall draft and obtain confidentiality agreements with all TWG members. The Contractor shall contact each proposed member and submit letters of commitment to the COR.

The Contractor may convene up to two TWG meetings per year for each of the TWGs. The meetings shall be conducted via telephone conference call. If TWG members are unable to attend the meeting, the Contractor shall plan on briefing the TWG members separately. The Contractor shall pay the honorarium expense of the TWG members.

The Contractor shall develop an agenda and related to materials for TWG meetings. The materials shall have content sufficient to maximize the discussion. The Contractor is responsible for producing and distributing the materials for these meetings; the COR is responsible for approving the materials. A draft of the agenda and materials shall be delivered to the COR by no fewer than 20 business days prior to the TWG meeting. A final agenda and materials shall be delivered to the COR TWG by no fewer than 10 business days prior to the TWG meeting. After each meeting, the Contractor shall submit a written summary of the meeting to the COR by no fewer than five business days, detailing suggestions and recommendations along with the Contractor’s proposed actions. Proposed actions cannot go forward unless approved by the COR.

5.2.8 Prepare and present a Webinar

Working with NCCC and ORE, the Contractor shall prepare and present a webinar to report on the progress and findings of each of the studies at the end of each contract year. The audience will be at a minimum all NCCC staff from HQ and the four campuses. The Contractor shall submit a draft and final PowerPoint presentation for review by the COR.

5.3 Conduct Three Studies and Update Service Project Database The following section provides an overview of the three pre-designed study designs on Leadership Development, Member Retention, and Strengthening Communities. Upon award of the contract in 2019, the evaluator shall begin by performing tasks needed to prepare for data collection – conduct the kick-off meeting, conduct the project planning meeting, establish Field Working Group and Technical Working Groups, and secure Human Subject Approval. The Leadership Development and Member Retention studies will share the same pre-designed survey questionnaire, use the same participants, and follow similar data collection timeline. The Strengthening Communities study’s data collection will start in the first option year and will use a separate set of data collection instruments. For detailed information on timeline, study participants, data collection instruments, participant retention strategies, data analysis and reporting plans, refer to Attachment 2 - Research Designs to Conduct Three Studies for AmeriCorps NCCC.

Performance Work Statement (PWS) AmeriCorps NCCC National Civilian Community Corps (NCCC) Impact Studies

Figure 1 below shows the data collection timeline for all three studies. Assuming enrollment into the study begins with Class 26 (2020 Winter cycle), the data collection period for the baseline and first follow-up surveys will span 32 months. The baseline survey data collection period is proposed from December 2019 to September 2020. The first follow-up survey is proposed from November 2020 to August 2021. The second follow-up survey is proposed from November 2021 to August 2022. The member early exit survey will be from March 2020 to June 2021. The qualitative data collection for the leadership development and member retention studies is proposed from September 2020 to September 2021. The first data collection point for the strengthening communities study is proposed from January 2021 to September 2021; the second data collection is proposed from January 2022 to September 2022. The analysis and report period spans 12 additional months from October 2020 to November 2023.

Figure 1 Data Collection Timeline for All Three Studies

Each research design relies on NCCC’s service projects database, which will provide information on the characteristics and outputs of all service projects since 2012. Upon award of the contract in 2019, the evaluator will populate the NCCC service project database, currently containing project data from 2012 to spring of 2019, with quantitative and qualitative data on NCCC service projects completed between summer 2019 (approximately 400 projects) and fiscal year 2020 (approximately 900 projects). The evaluator will merge the data from the service projects database with Corps members responses in the leadership development survey to examine the relationship between service projects’ characteristics and leadership qualities and skills, and retention. The service projects database will serve as the sampling frame to select projects for the exploratory strengthening-communities study.

5.3.1 Conduct Leadership Development Study

The Contractor shall conduct a longitudinal, mixed-method, quasi-experimental study on leadership development, guided by the following research questions

1. What is the profile of NCCC members, including their demographics, prior participation in service, sources to learn about NCCC, and motivations for serving? How do members perceive the NCCC experience prior to the start of their service?

2. How do NCCC members differ in interpersonal skills, life skills, including attitudes and behavior in group settings prior to the start of service compare to youth of similar background who do not enroll in NCCC? How do these skills change at the end of their service and one year following the end of service?

3. What is the impact of NCCC participation on members’ professional skills, life skills, teamwork, and civic engagement? How do these skills change at the end of their service and one year following the end of service compare to youth of similar background who did not enroll with NCCC? How does the impact vary for Traditional Corps and FEMA Corps?

4. How do members’ service projects experiences contribute to their professional skills, life skills, teamwork, and civic engagement? How does the contribution of service experiences on professional skills, life skills, teamwork, and civic engagement vary for Traditional Corps and FEMA Corps?

5.3.1.1 Research Design

The study will use a matched quasi-experimental design (QED), and will enroll participants from the incoming NCCC Class 26 (2020 Winter and Summer cycles) and NCCC Class 27 (2020 Fall cycle). The spread of service for Class 26 and Class 27 is illustrated in Figure 1. Enrollment through graduation for three cycles spans from February 2020 to July 2021. There will be data collection with a survey during baseline, the completion of the service year, and a one-year follow-up.

Figure 1. NCCC Class 26 and Class 27 Months of Service

The Contractor shall administer a survey to participants from both the treatment and comparison groups. Applicants who have cleared the background check and have agreed to serve will constitute the treatment group. Applicants who have cleared the background check and been accepted but have declined to join will constitute the comparison group.

To construct matched comparison group data, the study will rely on member survey and program administrative data that will provide rich information on the characteristics of all applicants who applied to serve with either Traditional NCCC or FEMA Corps.

The qualitative research component for the Leadership Development study will complement the quantitative survey results by using focus groups and semi-structured in-depth interviews to further explore members’ leadership qualities and skills. The data collection instruments are currently undergoing OMB clearance process and expected to be approved prior to the scheduled time period for each data collection.

5.3.1.2 Participants

The Contractor shall work with campus Point of Contacts (POCs) to acquire adequate number of participants for the study. Based on the most recent available data on applications received and acceptance rate, it is estimated that the incoming members from the 2020 Winter and Summer cycles from Class 26 and the 2020 Fall cycle from Class 27 will consist of approximately 1,600 applicants who accept to serve. The comparison group participants will be recruited from the applicants list from Class 25 (Fall, Winter, Summer), Class 26 (Fall, Winter and Summer), and two cycles from Class 27 (Fall and Winter). Across these eight cycles during the enrollment period, it is expected that the total number of accepted applicants who declined to serve will be approximately 3,800. Among the comparison group the acceptance rate to participate is expected to be 30 percent, yielding an enrollment of 1,140 comparison participants at baseline.

The Contractor shall work with campus POCs to acquire a list of 10 to 12 Sponsors/FEMA POCs that have volunteered to participate in the study, in which the Contractor shall confirm with six to eight participants to attend or participate in the discussion groups.

5.3.1.3 Sample Retention

The Contractor shall monitor differential rates of attrition between treatment and comparison participants for Traditional and FEMA Corps programs. The Contractor shall use multiple communication approaches to retain study participants, including contacting key friends and family members, relying on a multimodal survey, and contacting participants up to four times (initial contact and three reminders).

Incentive Offers Corps members will not receive any monetary incentive. Once members leave or complete their service, they will be more difficult to reach and remain connected to the study. The contractor will offer a $20 Visa gift card to members once they either end or complete their term of service, and comparison group participants for each survey completed (a total of three surveys). Respondents will be asked to provide their address where the gift card can be sent upon completion of the survey.

5.3.1.4 Mixed Methods Data Collection

Prior to the start of service for each cycle, before Corps members arrive at their assigned NCCC campus, Contractor shall send an individualized link to each incoming Corps member (treatment) and individuals in the comparison group inviting them to complete the baseline survey.

Administer Survey Survey Administration Timeline The Contractor shall administer the leadership development survey (included in the appendix) at baseline (prior to training before members report to their campus), first follow-up (at end of service), and second follow-up (approximately one year following the end of the term of service).

Measures The initial baseline survey will include questions on demographic characteristics, motivation to serve, and baseline outcome characteristics. Linked with program administrative data on applicants, these data will be used to form the matched samples and will be included as controls in the impact analysis regression models. Outcomes will be assessed in professional skills, life skills, team behavior, community and civic engagement. Questions will assess changes in desired outcomes in each of these areas. Survey responses will also be merged with the service projects database maintained by NCCC on the characteristics of each member’s service deployment during his or her term of service.

Focus Groups and Semi-structured Interviews The Contractor shall collect qualitative data through focus groups and semi-structured interviews from active members (Corps members and Team Leaders), NCCC staff, and NCCC Sponsors/FEMA POCs at each of the four campuses with all three enrollment cycles—Winter Class 26, Summer Class 26, and Fall Class 27. The focus groups and interviews will occur near the end of members’ service year. For the Winter Class 26, the qualitative data collection period is October to November of 2020; for the Summer Class 26, the qualitative data collection period is March to April 2021; and for the Fall Class 27 the qualitative data collection period is June to July 2021.

The Contractor shall conduct onsite focus groups or semi-structured interviews with active members (Corps members and team leaders), NCCC staff, project sponsors, and FEMA POCs. Each focus group will consist of six to eight participants. The focus groups and interviews will allow members to retrospectively assess their NCCC experiences and their development.

5.3.1.5 Analysis

Descriptive Analyses Prior to conducting any impact analyses, the Contractor shall conduct descriptive analyses using treatment and matched comparison group participants based on data from the baseline survey and available administrative data. These analyses will provide an overview of the baseline background and demographic characteristics of treatment and matched comparison group participants, and outcome characteristics prior to the start of service.

Once the matched sample is constructed, the Contractor shall do further analyses to determine whether the matching was effective in identifying comparison cases that are observationally equivalent to the treatment cases. The Contractor shall repeat descriptive statistics comparing baseline equivalence of treatment and matched comparison group outcomes at the end of the program participation period (first follow-up) and one year after the end of program participation period (second follow-up).

Impact Analyses To estimate the impact of NCCC on members’ leadership qualities and skills, the Contractor shall examine outcome differences between the treatment group and the matched comparison cases. To estimate the impact with increased statistical efficiency, the Contractor shall use multivariate multilevel regression models using the post-match sample, controlling for baseline characteristics. The dependent variable in this model will be each participant’s leadership qualities and skills measured by professional skills, life skills, team behavior, community and civic engagement. The parameter constituting the ‘impact estimate’ in this model will be the regression-adjusted treatment effect of NCCC participation on each outcome of interest.

Thematic and Content Analysis The Contractor shall triangulate the quantitative analysis with focus groups and semi-structured interview data. The Contractor shall use a thematic and content analysis that focuses on the research questions. Concepts, categories, and assumptions will be continuously compared and refined as new data, findings and themes emerge. The Contractor shall analyze the focus groups and semi-structured interviews data collected from members, NCCC staff, and project sponsors and FEMA POCs to determine what factors contributed to leadership development in members while serving with NCCC. The narrative responses gathered from members will be used to complement the outcomes from the quantitative leadership surveys.

5.3.1.6 Addressing Non-response

Non-response at the baseline survey CNCS expects that all incoming members will participate in the study but that only 30 percent of the comparison group will accept the invitation to participate in the study. The Contractor shall use available application data to determine whether comparison group members who agreed to participate differ significantly from those who decline to participate. The Contractor shall apply matching methods to establish baseline equivalence between treatment and comparison group respondents that should minimize any perceived bias in the differential enrollment rate between the two groups.

Non-response at the Follow-up Surveys All completion rates at follow-up are conditioned on the respondent having been a baseline respondent, since follow-up data collection will be attempted only for those who responded at baseline. If the completion rate is less than 80 percent at either the first follow-up or second follow-up survey, the Contractor shall conduct a non-response bias analysis to determine whether the follow-up respondents differ significantly from the baseline treatment and comparison group members that enrolled in the study, and whether attrition differs for the two groups.

Since almost all questions on the survey are going to be required of respondents, nonresponse will likely be very minimal and will result primarily from early termination of the survey. These will be classified as incomplete, but included, cases and non-sufficient responses. The Contractor shall assess whether both partial survey completion and answers to survey items differ significantly by survey modality (online or phone survey).

5.3.2 Conduct Member Retention Study

The Contractor shall conduct a longitudinal, mixed-method, quasi-experimental member retention study, guided by the following research questions

1. What is the profile of NCCC members who remain in service, including their demographics, prior participation in service, sources to learn about NCCC, and motivations for serving compared to members who leave? How do members who remain in service differ in their perception of the NCCC experience prior to the start of their service compared to members who leave?

2. How do members who remain in service differ in interpersonal skills, life skills, including attitudes and behavior in group settings prior to the start of service compare to members who leave early and compare to youth of similar background who do not enroll in NCCC?

3. How do the characteristics of service projects (e.g., duration, perceived team accomplishment and opportunities to engage with the community) affect member retention? How does the association between service project characteristics and retention differ for Traditional Corps and FEMA Corps?

5.3.2.1 Research Design

The research questions for the Member Retention study focus on understanding how members’ personal attributes and their service experience together contribute to completion of the term of service. Similar to the Leadership Development study, the Member Retention study will use a QED design. The treatment group will be Corps members who complete their term of service; the comparison group will consist of members who leave early and do not complete their term of service.

The Contractor shall conduct an initial impact analysis with the matched sample at the end of the program participation period, which will occur 10 months after program enrollment for Traditional NCCC and 12 months for FEMA Corps. The Contractor shall conduct a second analysis one year following the completion of service.

Since members are nested within campuses and within service projects a multilevel analysis will be conducted. The macro-level unit of analysis will be the campus and service projects, while the micro-level unit of analysis will be the individual members.

The Contractor shall implement the study design as previously described under the Leadership Development study with slight modification

1. Collecting survey from treatment and comparison group participants at four time points; baseline prior to the start of service, at the time of exit, at the end of service, and one year after Corps member complete their term of service. The measures to be collected through survey that include background and demographic characteristics, prior volunteer service, reasons for leaving, and leadership qualities and skills.

In addition, outcome analysis to assess the relationship between characteristics of service projects and likelihood of retention will be conducted. The outcome analysis for Corps members is a mixed-methods approach that involves

1. Merging Corps members survey responses with administrative data on projects the members were assigned to during their term of service.

2. Triangulating Corps members survey responses with focus groups and semi-structured interviews data.

Similar to the qualitative data collection for the Leadership Development study, the Member Retention study includes in-person focus group discussions and semi-structured interviews to explore a wide-range of NCCC stakeholders’ views on member retention.

5.3.2.2 Participants

It is anticipated that about one-third of Corps members who enrolled in the study will end their service early, thus the expected sample size of members who remain will be larger than those who leave.

5.3.2.1 Mixed Methods Data Collection

Survey Administration Timeline The survey administration timeline is the same as for the Leadership Development study. Both studies will use the same questionnaire and participants. In addition, the Contractor shall administer a short survey to members who leave early. The survey includes four questions to measure their reasons for leaving as well as their perception about the service experience and how this experience may have contributed to their decision to leave.

Measures Since the Leadership Development and Member Retention studies share the same questionnaire, the measures for both studies are similar, except for reasons for leaving which will be measured for those who leave service early.

Focus Groups and Semi-structured Interviews The Member Retention study will contain the same participants as in the Leadership Development study, and it will be performed at the same time period when the qualitative data collection is conducted for the Leadership Development study. In addition to the groups included in the Leadership Development study (active Corps member, team leader, staff, and sponsors/FEMA POCs), the Member Retention study will include online focus groups or telephone interviews with members who leave service (early exit member) before graduation.