95332418Q0118_-_Morocco_II_Workforce_Development_Solicitation.pdf

PAGE 1 OF

1. REQUISITION NO.

2. CONTRACT NO. 3. AWARD/EFFECTIVE DATE 4. ORDER NO. 5. SOLICITATION NUMBER 6. SOLICITATION ISSUE DATE

a. NAME b. TELEPHONE NO. (No Collect Calls) 8. OFFER DUE DATE/LOCAL

10:00 A.M. ET TIME

9. ISSUED BY CODE 10. THIS ACQUISITION IS UNRESTRICTED OR SET ASIDE: % FOR:

SMALL BUSINESS

HUBZONE SMALL

BUSINESS

SERVICE-DISABLED

VETERAN-OWNED

SMALL BUSINESS

WOMEN-OWNED SMALL BUSINESS

(WOSB) ELIGIBLE UNDER THE WOMEN-OWNED

SMALL BUSINESS PROGRAM

EDWOSB

8(A)

NAICS:

SIZE STANDARD:

11. DELIVERY FOR FOB DESTINA-

TION UNLESS BLOCK IS

MARKED

SEE SCHEDULE

12. DISCOUNT TERMS

13a. THIS CONTRACT IS A

RATED ORDER UNDER

DPAS (15 CFR 700)

13b. RATING

14. METHOD OF SOLICITATION

RFQ IFB RFP

15. DELIVER TO CODE 16. ADMINISTERED BY CODE

17a. CONTRACTOR/OFFEROR CODE FACILITY CODE 18a. PAYMENT WILL BE MADE BY CODE

TELEPHONE NO. DUNS: DUNS+4:

PHONE: FAX:

17b. CHECK IF REMITTANCE IS DIFFERENT AND PUT SUCH ADDRESS IN OFFER 18b. SUBMIT INVOICES TO ADDRESS SHOWN IN BLOCK 18a UNLESS BLOCK BELOW IS CHECKED

SEE ADDENDUM

19. 20. 21. 22. 23. 24.

ITEM NO. SCHEDULE OF SUPPLIES/SERVICES QUANTITY UNIT UNIT PRICE AMOUNT

(Use Reverse and/or Attach Additional Sheets as Necessary)

25. ACCOUNTING AND APPROPRIATION DATA 26. TOTAL AWARD AMOUNT (For Govt. Use Only)

27a. SOLICITATION INCORPORATES BY REFERENCE FAR 52.212-1, 52.212-4. FAR 52.212-3 AND 52.212-5 ARE ATTACHED. ADDENDA ARE ARE NOT ATTACHED.

27b. CONTRACT/PURCHASE ORDER INCORPORATES BY REFERENCE FAR 52.212-4. FAR 52.212-5 IS ATTACHED. ADDENDA ARE ARE NOT ATTACHED

28. CONTRACTOR IS REQUIRED TO SIGN THIS DOCUMENT AND RETURN _______________ 29. AWARD OF CONTRACT: REF. ___________________________________ OFFER COPIES TO ISSUING OFFICE. CONTRACTOR AGREES TO FURNISH AND DATED ________________________________. YOUR OFFER ON SOLICITATION DELIVER ALL ITEMS SET FORTH OR OTHERWISE IDENTIFIED ABOVE AND ON ANY (BLOCK 5), INCLUDING ANY ADDITIONS OR CHANGES WHICH ARE ADDITIONAL SHEETS SUBJECT TO THE TERMS AND CONDITIONS SPECIFIED SET FORTH HEREIN IS ACCEPTED AS TO ITEMS:

30a. SIGNATURE OF OFFEROR/CONTRACTOR 31a. UNITED STATES OF AMERICA (SIGNATURE OF CONTRACTING OFFICER)

30b. NAME AND TITLE OF SIGNER (TYPE OR PRINT) 30c. DATE SIGNED 31b. NAME OF CONTRACTING OFFICER (TYPE OR PRINT) 31c. DATE SIGNED

AUTHORIZED FOR LOCAL REPRODUCTION (REV. 2/2012)

PREVIOUS EDITION IS NOT USABLE Prescribed by GSA - FAR (48 CFR) 53.212

7. FOR SOLICITATION

INFORMATION CALL:

STANDARD FORM 1449

OFFEROR TO COMPLETE BLOCKS 12, 17, 23, 24, & 30

SOLICITATION/CONTRACT/ORDER FOR COMMERCIAL ITEMS 141

95332418Q0118 07/06/2018

REGINA MAXWELL maxwellre@mcc.gov 08/03/2018

Millennium Challenge Corporation Contracts and Grants Management Division 1099 Fourteenth Street NW Suite 700, Attn: Regina Maxwell Washington DC 20005

X

X

N/A

X

Millennium Challenge Corporation

CGM

1099 Fourteenth Street NW, Suite 700 Washington DC 20005

Millennium Challenge Corporation Contracts and Grants Management Division 1099 Fourteenth Street NW Suite 700 Washington DC 20005

Interior Business Center M/S D-2773 7301 West Mansfield Avenue Lakewood CO 80235-2230

This is a combined synopsis/solicitation for commercial items prepared in accordance with the format in FAR Subpart 12.6 and conducted per the procedures of FAR Subpart 13.5, as supplemented with additional information included in this notice.

MCC seeks a Contractor to provide Independent Evaluation Services in support of the Morocco II Workforce Development Activity.

See enclosed full and open competitive solicitation and associated attachments for all details.

See CONTINUATION Page

X X

Jacqueline H. Naranjo

95332418Q0118 – IE Services in Support of the Morocco Workforce Development Activity Page 2

Table of Contents

SECTION B – SUPPLIES OR SERVICES AND PRICE/COSTS

B.2 FAR 52.216-1 -- Type of Contract (APR 1984) B.3 Accounting and Appropriation Data B.4 Funding Type B.5 Limitation of Funds – Incrementally Funded Contracts B.6 Labor Rates & Other Direct Costs (ODCs)

SECTION C – DESCRIPTION/SPECIFICATIONS/STATEMENT OF WORK

C.1. Introduction and Background C.2. Scope C.3. Evaluation Questions and Guidance on Methodology C.4. Roles and Responsibilities C.5. Tasks

SECTION D – PACKAGING AND MARKING

D.1. Disclaimer for Deliverables

SECTION E – INSPECTION AND ACCEPTANCE

E.1. Inspection of Reports E.2. Quality Control Plan

SECTION F – DELIVERIES OR PERFORMANCE

F.1. Period of Performance F.2. Place of Performance F.3. Deliverables F.4. Reporting Deliverables F.5. Deliverable Format F.6. Acceptance Criteria

SECTION G – CONTRACT ADMINISTRATION DATA

G.1. MCC 52.201-70 Contracting Officer’s Representative (COR) and/or Project Monitor (PM) (JULY 2012) .. 27 G.2. MCC 52.232-70 Invoice Instructions (MAY 2017)

SECTION H – SPECIAL CONTRACT REQUIREMENTS

H.1. MCC 52.203-70 Contractor Non-Disclosure Agreement (MAY 2017) H.2. MCC 52.204-70 Security Requirements for Contractor Personnel (MAY 2013) H.3. MCC 52.209-70 Organizational Conflicts of Interest: Preclusion from Implementation Contract (JULY 2012) H.4. MCC 52.232-72 Limitation of Funds – Incrementally Funded Contracts (NOV 2006) H.5. MCC 52.232-73 Travel Reimbursement (MAY 2017) H.6. MCC 52.232-74 ODC Reimbursement (JAN 2007) H.7. MCC 52.242-70 Contractor Performance Assessment Rating System (CPARS) Registration (AUG 2011) .. 38 H.8. MCC 52.243-70 Increase in Services (OCT 2006) H.9. MCC 52.245-70 Contractor Acquired MCC Government Property Overseas (JULY 2012) H.10. Peer Review and Posting of Reports H.11. Insurance Maintenance

95332418Q0118 – IE Services in Support of the Morocco Workforce Development Activity Page 3

H.12. Stakeholder Input for Evaluation Products H.13. Human Subjects Protection H.14. Personally Identifiable Information and Confidentiality H.15. Delivered Work Product Free from 3rd Party Rights H.16. Key Personnel H.17. Team & Key Personnel Qualifications

PART II – CONTRACT CLAUSES

SECTION I – CONTRACT CLAUSES

I.1. FAR 52.212-4 Contract Terms and Conditions — Commercial Items/Alternate I (JAN 2017) I.2. FAR 52.212-5 Contract Terms and Conditions Required to Implement Statutes or Executive Orders – Commercial Items (JAN 2018) I.3. FAR 52.214-34 Submission of Offers in the English Language (APR 1991) I.4. FAR 52.214-35 Submission of Offers in U.S. Currency (APR 1991) I.5. FAR 52.217-9 Option to Extend the Term of the Contract (MAR 2000) I.6. FAR 52.252-2 Clauses Incorporated by Reference (FEB 1998)

PART III – LIST OF DOCUMENTS, EXHIBITS AND OTHER ATTACHMENTS

SECTION J – LIST OF ATTACHMENTS

SECTION K – REPRESENTATIONS, CERTIFICATIONS AND OTHER STATEMENTS OF OFFERORS OR

RESPONDENTS

SECTION L – INSTRUCTIONS, CONDITIONS, AND NOTICES TO OFFERORS

L.1. Inquiries/Questions Concerning the Solicitation L.2. Exclusion of Quotes and Pre-Award Communications with Offerors L.3. Quotation Due Date L.4. Quotation Response L.5. Quotation Volumes L.6. Quotation Format L.7. Blocked and Unacceptable Electronic File Types L.8. Page Limits L.9. Preparation of Volume I – Technical Quotation L.10. Preparation of Volume II – Cost Quotation

SECTION M – EVALUATION FACTORS FOR AWARD

M.1. FAR 52.212-2 Evaluation – Commercial Items (OCT 2014) M.2. Evaluation Factor 1 – Technical Capability M.3. Evaluation Factor 2 – Past Performance M.4. Evaluation Factor 3 – Price M.5. Basis for Award

95332418Q0118 – IE Services in Support of the Morocco Workforce Development Activity Page 4

SECTION B – SUPPLIES OR SERVICES AND PRICE/COSTS

B.1. Price/Cost Schedule

CLIN Description of Supplies/Services Qty U/I Unit Price Amount

0001 BASE PERIOD

DIRECT LABOR & OTHER DIRECT COSTS (ODCs)

Period of Performance: TBD Estimated 12 months

PoP Start: TBD PoP End: TBD

1 LO ______________ ______________

1001 OPTION PERIOD I

DIRECT LABOR & OTHER DIRECT COSTS (ODCs)

Period of Performance: TBD Estimated 12 months

PoP Start: TBD PoP End: TBD

1 LO ______________ ______________

2001 OPTION PERIOD II

DIRECT LABOR & OTHER DIRECT COSTS (ODCs)

Period of Performance: TBD Estimated 12 months

PoP Start: TBD PoP End: TBD

1 LO ______________ ______________

3001 OPTION PERIOD III

DIRECT LABOR & OTHER DIRECT COSTS (ODCs)

Period of Performance: TBD Estimated 12 months

PoP Start: TBD PoP End: TBD

1 LO ______________ ______________

4001 OPTION PERIOD IV

DIRECT LABOR & OTHER DIRECT COSTS (ODCs)

Period of Performance: TBD Estimated 12 months

PoP Start: TBD PoP End: TBD

1 LO ______________ ______________

5001 OPTION PERIOD V

DIRECT LABOR & OTHER DIRECT COSTS (ODCs)

1 LO ______________ ______________

95332418Q0118 – IE Services in Support of the Morocco Workforce Development Activity Page 5

Period of Performance: TBD Estimated 12 months

PoP Start: TBD

PoP End: TBD

TOTAL ESTIMATED PRICE: _______________

B.2 FAR 52.216‐1 ‐‐ Type of Contract (APR 1984)

The Government contemplates award of a Time-and-Materials (T&M) contract resulting from this solicitation.

(End of Provision)

B.3 Accounting and Appropriation Data

[RESERVED]

B.4 Funding Type

This contract will be funded, at time of award, with no-year Due Diligence funding. As such, funding obligated to this contract may be used during the life of the contract.

B.5 Limitation of Funds – Incrementally Funded Contracts

In accordance with clause MCC 52.232-72, “Limitation of Funds – Incrementally Funded Contracts,” funds in the amount of [TBD] have been allotted to this contract. It is contemplated that funds now obligated under this contract will cover the work to be performed until [TBD]. If the contractor considers the funds obligated under this contract to be insufficient to cover the work to be performed until the date specified herein, or another date agreed to by the parties, the contractor shall notify the contracting officer in writing and indicate the date on which it expects expended funds to approximate 75 percent of the total amount obligated. The notice shall state the estimated amount of additional funds required to continue performance through the date for which incremental funds are provided.

The Contractor shall not perform work on the contract which exceeds the total amount actually allotted by the Government to the contract. The Government is not obligated to reimburse the Contractor for costs incurred in excess of the total amount allotted by the Government to this contract. The contractor assumes the risk for any increased costs beyond what the Government obligates.

B.6 Labor Rates & Other Direct Costs (ODCs)

Labor rates at the inception of a contract period will carry through as “Fixed-Price” without adjustment for the life of that contract period.

Hours by labor category and pricing under ODCs are estimates only and may be increased/decreased as necessary for successful completion of the contract requirements and as approved by the COR/PM. However, the contractor’s expenditures at any given point shall not exceed the total obligated amount or the total ceiling price for this contract as specified below.

Using the template below, Offerors must include all proposed labor categories, fully-burdened hourly rates and Other Direct Costs (ODCs) for each contract year identified in B.1.

PRICING SUMMARY PER CONTRACT PERIOD

Base Option 1 Option 2 Option 3 Option 4 Option 5 TOTAL

Total Labor Hours Total Labor Costs $ - $ - $ - $ - $ - $ - $ - Total ODCs $ - $ - $ - $ - $ - $ - $ - Total Est. Cost $ - $ - $ - $ - $ - $ - $ -

95332418Q0118 – IE Services in Support of the Morocco Workforce Development Activity Page 6

Base Period – Period of Performance TBD (est. 12 months) Item # Description(2) U/I Qty Rate Total

LABOR

0001 hr.

hr.

hr.

hr.

Labor Sub-Total ODCs(1)

ODC Sub-Total Grand Total (Base Period)

Option Period I – Period of Performance TBD (est. 12 months) Item # Description(2) U/I Qty Rate Total

LABOR

1001 hr.

hr.

hr.

hr.

Labor Sub-Total

1001 Lot

Grand Total (Option Period I)

Option Period II – Period of Performance TBD (est. 12 months)

LABOR

2001 hr.

hr.

hr.

hr.

Labor Sub-Total

Grand Total (Option Period II)

95332418Q0118 – IE Services in Support of the Morocco Workforce Development Activity Page 7

Option Period III – Period of Performance TBD (est. 12 months) Item # Description(2) U/I Qty Rate Total

LABOR

3001 hr.

hr.

hr.

hr.

Labor Sub-Total

Grand Total (Option Period III)

Option Period IV – Period of Performance TBD (est. 12 months)

LABOR

4001 hr.

hr.

hr.

hr.

Labor Sub-Total

4001 Subcontract(s) – Data Collection (optional) Lot $350,000.00

Grand Total (Option Period IV)

Option Period V – Period of Performance TBD (est. 12 months)

LABOR

5001 hr.

hr.

hr.

hr.

Labor Sub-Total

Grand Total (Option Period V)

95332418Q0118 – IE Services in Support of the Morocco Workforce Development Activity Page 8

TOTAL COST SUMMARY

Description(2) U/I Qty Rate Total

LABOR

hr.

hr.

hr.

hr.

Total Labor Hours - - Labor Sub-Total

Total Estimated Cost

NOTE (1): MCC anticipates approximately $350,000.00 (Option Period 4) worth of data collection subcontracting costs for which the requirements are unknown at this time. To be fair to all vendors providing a quotation, a plug number is utilized for data collection subcontracting costs in the respective year(s) in which data collection is expected to occur. However, if you believe this plug number is inadequate, please advise during the Question and Answer period of the solicitation. “Data Collection Subcontract(s),” as used in the tables above, refers only to the cost of subcontracting a data collection firm and any associated indirect costs (e.g., subcontract handling fees). It does not include any prime contractor labor associated with overseeing/managing the data collection effort; this time should be incorporated into the level of effort proposed under the respective Labor CLIN. Subcontracting costs for data collection are subject to the requirements of FAR 52.244-2, Subcontracts.

NOTE (2): A job description shall be included for each labor category proposed. The job description must identify essential and non-essential tasks that are assigned to a specific position. It must also identify reporting relationships and shall also describe required qualifications, minimum requirements, working conditions and desirable qualifications.

(End of SECTION B)

95332418Q0118 – IE Services in Support of the Morocco Workforce Development Activity Page 9

SECTION C – DESCRIPTION/SPECIFICATIONS/STATEMENT OF WORK

C.1. Introduction and Background

On November 30, 2015, the United States of America, acting through the Millennium Challenge Corporation (“MCC”), signed the five-year, $450 million Employability and Land Compact with the Government of Morocco (“Government” or GoM). The compact entered into force (EIF) on June 30, 2017, thus beginning the five-year window to execute the investments. This compact supports two Moroccan Government priorities that have posed binding constraints to economic growth and investment in Morocco: high-quality education and land productivity.

The compact investments are composed of two projects:

I. Education and Training for Employability Project ($220 million)

i. Secondary Education Activity

ii. Workforce Development Activity

II. Land Productivity Project ($170.5 million)

i. Governance Activity

ii. Rural Land Activity

iii. Industrial Land Activity

More information and relevant documents, including the compact agreement, are available at:

https://www.mcc.gov/where-we-work/program/morocco-employability-and-land-compact. The Monitoring & Evaluation (M&E) Plan for the compact is available at: https://www.mcc.gov/where-we-work/m-and-e/morocco-employability-and-land-compact.

The Workforce Development Activity aims to increase the employability of Moroccans by: improving the quality and relevance of, and equitable access to, private sector-driven technical and vocational education and training (TVET) programs that respond to private sector needs; providing technical assistance to develop and implement demand-driven TVET sector policy; providing effective employment services to help unemployed or economically inactive women and unemployed, at-risk urban and peri-urban youth obtain quality jobs; and supporting the operationalization of a labor market observatory (the “Labor Market Observatory”) to provide a broad range of stakeholders with dynamic labor market information to improve decision-making for public policy and private investment.

The Workforce Development Activity is composed of four sub-activities:

I. Private Sector-Driven TVET ($103 million): A Private Sector-Driven TVET grant facility, the Charaka Fund (https://charaka.mcamorocco.ma), will be used to support selected private sector-driven training centers.

Grants will support: (1) establishing new TVET centers run by public-private partnerships (PPPs), or expanding existing TVET centers currently managed through PPPs; and (2) transforming existing public TVET centers from a traditional government-driven model to a private sector-driven model. Eligibility requirements, and the operational processes for administering this grant facility, will be set forth in an operations manual approved by MCC. Among other requirements, potential grantees must agree to participate in the Ministry of National Education and Vocational Training’s monitoring and evaluation systems. The Government will directly contribute to the Private Sector-Driven TVET grant facility and take steps to ensure that the grant facility will survive the compact Term, using Government funds. MCC Funding will finance the majority of the first round, and the minority of the second round, of grants issued by the Private Sector-Driven TVET grant facility.

II. Operationalizing TVET Sector Policy Reform ($6.7 million): Technical assistance and limited material support will be provided to the Ministry of National Education and Vocational Training and related agencies to develop and implement private sector demand-driven TVET sector policy.

95332418Q0118 – IE Services in Support of the Morocco Workforce Development Activity Page 10

III. Results-Based Financing (RBF) for Inclusive Employment ($13 million): In order to catalyze a market for improved employment outcomes, results-based financing mechanisms, such as Social Impact Bonds or performance-based contracting, will be used to replicate or expand proven programs that provide integrated job placement services. The services for job placement will target unemployed or economically inactive women and unemployed, at-risk urban and peri-urban youth. MCC Funding will also support the development and implementation of rigorous impact evaluations to test promising labor market interventions, not limited to those interventions funded by this sub-activity.

IV. Labor Market Observatory ($14 million): Technical assistance will be provided to strengthen the Labor Market Observatory to ensure coordinated policy and promote joint activities among relevant Government agencies for analyzing the labor market. MCC Funding will also support the development and rollout of an information dissemination platform. Finally, MCC Funding will support the development and implementation of key labor market studies.

C.2. Scope

Provide Independent Evaluation Services in support of the Workforce Development Activity of the Morocco II Compact.

C.3. Evaluation Questions and Guidance on Methodology

The evaluation will seek to understand the degree to which planned investments are implemented, the validity of the program logic (see below) and its assumptions, output and early outcome results attributable to the activity, and lessons learned from implementation.

95332418Q0118 – IE Services in Support of the Morocco Workforce Development Activity Page 11

Private Sector-Driven TVET and Operationalizing TVET Sector Policy Reform

Are graduates from TVET centers funded by the Charaka Fund more competitive (more likely to be hired) than graduates who receive similar training from comparable, traditional TVET centers?

o What are the insertion rates of graduates of compact-supported centers into jobs relevant to their training?

Average salary of graduates of compact-supported centers? (findings to be disaggregated by sub-groups of interest) o Do the compact-supported centers succeed in attracting and maintaining women and other sub-groups of interest to non-traditional fields?

o Are there differential impacts (insertion rates, average salaries, etc.) between the creation of new TVET centers and the conversion of existing TVET centers?

Are compact-supported TVET centers more responsive to the demands of the private sector and changes in the economy and the labor market?

o Has the Charaka Fund incentivized TVET centers to improve and innovate?

o What are firms’ perceptions of and/or level of satisfaction with graduates of compact-supported TVET centers?

Are compact-supported centers maintained post compact? Are the quality and relevance of the trainings offered by compact-supported centers sustained over time? Does the financial autonomy of compact-supported centers increase over time?

Have the results of the Charaka Fund investment influenced the Government of Morocco to expand the number of TVET centers managed through PPPs?

Results-Based Financing for Inclusive Employment

What are the various results-based financing mechanisms employed by the sub-activity? What are the approaches used in the sub-activity to deliver improved job placement services to women and at risk youth?

Has the sub-activity improved the incentive structure for achieving labor market results among service provides to the “Agence Nationale de Promotion de l’Emploi et des Compétences” (ANAPEC)?

Has the compact’s RBF initiative demonstrated a more effective and sustainable model for job placement services?

Do compact-supported job placement services achieve better outcomes than the traditional services provided through ANAPEC, particularly for women and at-risk youth?

o What are the insertion rates and labor market outcomes of compact-supported job placement services?

(findings to be disaggregated by sub-groups of interest) o Are providers of job placement services properly incentivized to achieve the outcomes of interest?

Labor Market Observatory

Has the National Labor Market Observatory (OMT) been empowered to bring coordination to the labor market information systems in Morocco? Has the OMT improved the coherence and integration of labor market data from different sources?

Have the interventions successfully addressed, in part or in whole, the legal, structural, and technical constraints identified in the diagnostic reports prepared by the Consortium for International Development in Education (CIDE)?

Has the sub-activity adapted existing data into formats that are more accessible to intended users? Has the sub-activity facilitated and promoted access to labor market data?

What aspects of the sub-activity are expected to be sustainable? Are those investments sustained?

Millennium Challenge Account-Morocco (MCA-Morocco) and MCC have defined underlying assumptions linking the selected investment to changes in employment outcomes, and ultimately increased household income. The current program

95332418Q0118 – IE Services in Support of the Morocco Workforce Development Activity Page 12 logic and key assumptions have informed the list of proposed evaluation questions above. However, the Contractor is expected to explicitly assess the program logic, key assumptions, and evaluation questions as part of the Evaluability Assessment (see Task 1) and Evaluation Design Report (see Task 2), recommending revisions, as necessary. The list above represents a broad set of potential questions of interest to MCC and MCA-Morocco. As part of Task 2, the Contractor shall develop a final list of research questions to be explored through the independent evaluation. As part of this work, the Contractor shall determine, based on a review of project and compact documents, the value of including standalone evaluation questions for the Operationalizing TVET Sector Policy Reform sub-Activity and the promoting gender-equitable workplaces component of the Results-Based Financing for Inclusive Employment sub-Activity.

The evaluation of the Workforce Development Activity will be a performance evaluation that includes analysis of longitudinal data on employment rates, salaries, and placements, as well as rigorous qualitative research. The evaluation should assess the outcomes and impacts of each of the four components listed above. The evaluation will need to account for changes in the labor market, the Moroccan economy, and other exogenous factors that may affect outcomes and impacts of interests. Although the timelines for each component may not match up perfectly, the Contractor should propose an innovative strategy that incorporates all of the components in one evaluation, and ultimately one final report.

As the largest component of the activity, the evaluation of the Private Sector-Driven TVET sub-Activity should include a facility-level evaluation and grant-level evaluations of a subset of the TVET programs that receive grants. The facility-level evaluation should include an assessment of the quality, success, and sustainability of compact-funded programs with regards to private sector engagement, provision of skills demanded by the labor market, and employment outcomes of graduates. The grant-level evaluations should include more in-depth monitoring of the progress of the students enrolled in the programs that receive grants, including post-graduation achievements and employment outcomes. The sub-activity anticipates providing approximately 25 grants. While final evaluation design will be determined as part of Task 2, for the purposes of developing cost proposals, Offerors can assume between seven (7) and nine (9) grant-level evaluations.

Outcomes should be monitored through both quantitative and qualitative methods to follow a sample of those students who enter the compact-funded TVET programs, and those that enter other TVET programs. Quantitative methods would be best applied to measure the job placement outcomes and salaries. The evaluation should also include qualitative analysis that assesses the needs of employers and their experience with TVET graduates. The Contractor shall propose a rigorous qualitative methodology as part of Task 2, including sampling methodology and rationale.

The level of effort devoted to each sub-activity should be proportional to the size of the sub-activity, indicated in Section C.1 by funding provided for the implementation of each. Similar methodologies to those cited in the previous paragraph may be appropriate for the evaluation of the Results-Based Financing for Inclusive Employment sub-Activity.

The anticipated timeframe for this contract is September 2018 to September 2024. During compact implementation, data collection for the evaluation will be funded by the compact, at the discretion of MCA-Morocco and MCC. However, it is anticipated that the evaluation will require follow-up data collection after the compact closes in July 2022. The Contractor will be responsible for procuring and managing the data collection firm for any post-compact data collection.

C.4. Roles and Responsibilities

A. Independence:

MCC’s Policy on Independent Evaluations. Based on the 1991 DAC Principles for Evaluation of Development Assistance, MCC’s Evaluation Policy requires independent evaluations of each MCC project. MCC’s implementation of this independence policy adheres to these principles, especially the following: “Impartiality and independence will best be achieved by separating the evaluation function from the line management responsible for planning and managing development assistance.” (P.6.)

95332418Q0118 – IE Services in Support of the Morocco Workforce Development Activity Page 13

Organizational Conflicts of Interest. MCC-M&E is responsible for adhering to our policy-driven requirements for independence. These requirements are distinct from Organizational Conflicts of Interest (OCI) proscribed in the Federal Acquisition Regulations. Interpretations and enforcement of the OCI statutes are the remit of our Office of General Counsel.

Independence of Evaluation Team. Evaluation team members must be impartial and therefore should not have contributed to the design or implementation of the project being evaluated. In cases where individuals or firms that have worked on the project are proposed to be part of the evaluation team, a justification and independence risk mitigation plan must be provided to MCC for consideration.

The Independent Contractor’s Role. To remain consistent with the principles of independent evaluations during project planning and implementation, the independent Contractor’s feedback is restricted to reporting evidence and findings.

Recommendations shall not be made before completing a final evaluation as they can influence project planning or implementation, thereby jeopardizing the independence an evaluation. Prescriptions and recommendations may be included in final evaluations.

Real-time feedback to project implementation. MCC supports monitoring and evaluation activities intended to provide feedback on how to improve program planning and implementation. But this support shall be separate from the independent Contractor. Any contribution of the Independent Contractor to this feedback shall be limited to reporting evidence and findings. Any exceptions to this rule must be approved in advance by the EMC.

B. MCC & MCA Responsibilities: The MCC, MCA, along with the Independent Contractor, play critical roles in designing, implementing, and disseminating the independent evaluations. The MCC and MCA roles are as follows (For further detail see Section J: Evaluation Management and Review Process):

1. Millennium Challenge Corporation (MCC): MCC is responsible for oversight of the Independent Contractor and quality control of evaluation activities, including the following specific responsibilities:

a. Assess when a program is ready for evaluation planning through evaluability assessment;

b. Determine what program components (Project(s), Activities, Sub-Activities) will be covered by the evaluation;

c. Set the evaluation questions to achieve accountability and learning objectives;

d. Build buy-in and ownership of the evaluation;

e. Contract and Supervise the Independent Contractor;

f. Conduct quality reviews of all evaluation products (reports, questionnaires, etc);

g. Facilitate public dissemination efforts to inform decision-makers on learning generated by the evaluation;

h. Oversee preparation of public-use data that ensures appropriate balance of enabling verification of analysis and broad use of the data with adherence to promises of confidentiality to survey respondents;

and

i. Determine the evaluation budgets.

2. Millennium Challenge Account (MCA) and country partners: MCA and country partners are responsible for building local ownership and commitment to the evaluation, oversight of the data collection firm, and quality control of evaluation activities, including the following specific responsibilities:

a. Assess when a program is ready for evaluation planning through evaluability assessment;

b. Set the evaluation questions to achieve accountability and learning objectives;

c. Build buy-in and ownership of the evaluation;

d. Manage the data collection firms during the lifetime of the compact;

e. Conduct quality reviews of all evaluation products (reports, questionnaires, etc); and

95332418Q0118 – IE Services in Support of the Morocco Workforce Development Activity Page 14

f. Facilitate public dissemination efforts.

C.5. Tasks

Independent Contractor: The Independent Contractor is responsible for the overall design, implementation, and dissemination of the evaluation, including the following specific responsibilities:

a. Develop the most rigorous evaluation design feasible given rules of program implementation (Tasks 1-2)

b. Support MCC and MCA to build buy-in and ownership of evaluation. (Task 2 and Task 7)

c. Develop evaluation materials that are held to international standards (Task 3 and Task 8)

d. Ensure Institutional Review Board clearance of survey protocols (Task 4 and Task 9)

e. Prepare and supervise data collection (during the compact) and manage the data collection firms (post-compact) (Task 4 and Task 9)

f. Ensure data quality during collection and entry through supervisions and management (Task 4 and Task

9)

g. Lead data cleaning, analysis, and interpretation of results (Task 5 and Task 10)

h. Produce high quality, credible, transparent evaluation reports (Task 5 and Task 10)

i. Lead preparation of public-use data that ensures appropriate balance of enabling verification of analysis and broad use of the data with adherence to promises of confidentiality to survey respondents (Task 6 and Task 11)

j. Lead public dissemination efforts (Task 6 and Task 11)

Task 1: Assess Evaluation Plan

The Contractor is responsible for conducting an Evaluability Assessment on all existing documentation related to the program and proposed evaluation plan provided by MCC and MCA using the MCC Evaluability Assessment Tool (See Section J) as a guide.

MCC and the Contractor should identify all MCC and MCA self-evaluation activities and determine how the independent evaluation will (i) validate findings of self-evaluation(s), (ii) use self-evaluation(s) to inform the independent evaluation, particularly around understanding what was designed, what was implemented, and understanding any necessary course corrections, (iii) complement the self-evaluation(s) by expanding on self-evaluation activities.

The Contractor will produce an Evaluability Assessment Report that provides an independent assessment of the program being evaluated. This assessment should be guided by the Evaluability Assessment Tool and should assess the quality, completeness, and/or appropriateness of the Problem Diagnostic, Program Logic, Cost-Benefit Analysis, Risks & Assumptions, Monitoring Indicators, and Proposed Evaluation Questions. The report should aim to convey the Contractor’s overarching assessment of the coherence of the program design, the underlying economic analysis, and the M&E approach.

Develop a Work Plan with expected deadlines for deliverables. Any revisions to the Work Plan will be submitted in the Contractor’s Progress Reports.

Task 2: Develop Evaluation Design Report

The Contractor will conduct missions to the field as needed to discuss program design and implementation issues to inform the evaluation design, as well as engage local stakeholders (MCA and Implementing Entities, as well as local government) on the proposed evaluation. This includes hosting local stakeholder workshops to develop/refine research questions, assess appropriate and feasible quantitative and qualitative evaluation methodologies, develop evaluation timeline, and build local stakeholder commitment to the evaluation design.

95332418Q0118 – IE Services in Support of the Morocco Workforce Development Activity Page 15

The Contractor will develop the Evaluation Design Report in both English and French, including all technical analysis required for assessing and confirming (as necessary) internal validity, power calculations, and required sample sizes. The Evaluation Design Report outline will be developed in consultation with the MCC M&E lead.

Please reference the Evaluation Design Report Outline (See Section J) as a guide.

As per MCC’s Gender Policy and Gender Integration Guidelines, the Contractor will ensure that evaluation design, data collection requirements, and analysis address gender and other social differences as appropriate. This involves including research questions that address social and gender dimensions of the project logic, assumptions, and project design; disaggregating impacts by gender, age, income, and other social difference (e.g. ethnicity) as appropriate; including special modules or studies as needed.

The Contractor will estimate the project’s post-compact economic rate of return (ERR):

o In the Evaluation Design Report, the Contractor will specify which key input, output and outcome parameters modeled in the MCC CBA will be measured as part of the proposed evaluation design. If the Contractor believes it is infeasible or prohibitively expensive to measure a benefit stream in the MCC CBA, this should be made explicit and a justification provided for MCC to consider.

o The Contractor should propose a CBA methodology to estimate the project’s post-compact ERR. The methodology may be based on an existing MCC model or developed by the Contractor. It should be documented in the EDR, citing sources where relevant. The Contractor may recommend to MCC that it is not feasible or worthwhile to build a CBA model of the project. A justification for this recommendation should be provided in the EDR for MCC’s consideration.

The Contractor will document feedback from local stakeholders and the response to that feedback in an annex to the final report. In order to obtain feedback, the Contractor may lead local workshops and/or virtual reviews of the Design Report by MCA and Implementing Entities.

o In accordance with Section H, the Contractor is responsible for allowing any key local stakeholders the opportunity to submit a public Statement of Difference or Support, as agreed with MCC and the MCA/designated representative.

The Contractor will submit the Evaluation Design Report to MCC in both English and French and may be asked to present to the Evaluation Management Committee (See Section J). All feedback and response to feedback will be documented by the Contractor in an annex to the final report.

o In accordance with Section H, MCC may: (i) select to obtain external peer review; and (ii) make the Report and peer reviews public on the MCC website.

The Contractor will produce a Nesstar Metadata file (See Section J) summarizing the final Evaluation Design Report for dissemination on the MCC Evaluation Catalog. Prior to publication, the Contractor should review the current version of MCC’s Evaluation Microdata Guidelines (See Section J) for guidance on preparation of public documentation that links to microdata to reduce re-identification risk.

The Contractor will continue to maintain the Evaluation Design Report as the evaluation develops. The Evaluation Design Report will remain a living document that is updated by the Contractor as necessary and resubmitted to MCC as necessary.

Task 3: Develop Evaluation Materials

The Contractor will provide technical support on development of data collection firm TORs, depending on the funding arrangements for data collection.

The Contractor will lead development of any necessary survey instruments for primary data collection (including questionnaires and accompanying enumerator manuals) or a data extraction approach for existing sources.

The Contractor will develop survey materials in English, unless otherwise agreed with MCC. When necessary, the Contractor will utilize existing instruments to improve coordination across MCC independent evaluations for

95332418Q0118 – IE Services in Support of the Morocco Workforce Development Activity Page 16 meta-analysis. The Contractor is responsible for pretesting1 survey instruments during their development and reviewing translations and back-translations conducted by data collection firms prior to data collection start (as necessary).

The Contractor is responsible for ensuring household surveys last no longer than 120 minutes without written consent from the Evaluation Management Committee.

The Contractor will document feedback from local stakeholders and response to the feedback. In order to obtain feedback, the Contractor may lead local workshops and/or virtual reviews of the Evaluation Materials by MCA and Implementing Entities.

The Contractor will submit the Evaluation Materials to MCC and may be asked to present to the Evaluation Management Committee. All feedback and response to feedback will be documented by the Contractor.

o The Contractor is responsible for the quality of the English and local language survey instruments.

Task 4: Prepare and Supervise baseline data collection

The Contractor is responsible for developing and overseeing all data collection and entry protocols and implementation. This requires reviewing the current version of MCC’s Evaluation Microdata Guidelines (See Section J) and ensuring appropriate Institutional Review Board (IRB) review and approval of research protocols, survey instruments, and informed consent procedures (see MCC Informed Consent Statement Template (See Section J)).

o In accordance with Section H, the Contractor should assess the local IRB requirements and determine whether or not additional, international-level IRB approval is required in order to ensure adherence to international standards.

If there are local requirements, the Contractor must abide by them.

If the local IRB is HHS registered, no additional IRB is required.

If the local IRB is not HHS registered, the Contractor also needs to sub-contract an HHS registered IRB.2

The Contractor is responsible for overseeing the training of data collection staff, field work, and data entry. The Contractor is responsible for pilot testing the survey instruments and field procedures as a part of enumerator training. This must be managed in coordination with MCC and MCA. The Contractor is responsible for reporting on the average survey time resulting from the pilot test and from field work.

The Contractor is responsible for conducting and documenting quality control checks of data as it is collected and entered in order to reduce data collection and entry error.

The Contractor is responsible for establishing a system for frequent communication with the data collection firm throughout data collection and entry, through email, phone, and site visits.

Task 5: Develop Baseline Report and Data Documentation Package

The Contractor will review raw files from the survey firm and provide any critical feedback on production of the survey firm’s Data Collection Report.

1 Note that MCC differentiates between the “pre-test”, defined as the testing of the instrument for content and flow, versus the “enumerator pilot”, an exercise completed during training for enumerators to practice with the instrument, and for the survey team to practice data flow, but NOT to result in major changes to the instrument.

2 HHS registered IRBs are found here - https://ohrp.cit.nih.gov/search/search.aspx?styp=bsc

95332418Q0118 – IE Services in Support of the Morocco Workforce Development Activity Page 17

Before conducting baseline analysis, the Contractor will review the current version of MCC’s Evaluation Microdata Guidelines (See Section J) for guidelines on management, documentation, and dissemination of the microdata.

The Contractor is responsible for developing the Baseline Report in both English and French. The Baseline Report outline will be developed in consultation with the MCC M&E lead. Please reference the Baseline Report Outline as a guide (See Section J).

The Contractor will ensure baseline analysis including gender and social analysis is part of the evaluation design as described in the evaluation design report (task 2), including examining differences in baseline characteristics by gender, age, income and other social variables as appropriate.

The Contractor will document feedback from local stakeholders and response to feedback in an annex to the final report. In order to obtain feedback, the Contractor may lead local workshops and/or virtual reviews of Baseline Report by MCA and Implementing Entities.

o In accordance with Section H, the Contractor is responsible for allowing any key local stakeholders the opportunity to submit a public Statement of Difference or Support, as agreed with MCC and the MCA/designated representative.

The Contractor will submit the Baseline Report to MCC in both English and French and present to the Evaluation Management Committee. All feedback and response to feedback will be documented by the Contractor in an annex to the final report. Any revisions to the report deemed necessary by the Contractor will be incorporated into the final draft.

o In accordance with Section H, MCC may: (i) select to obtain external peer review; and (ii) make the Report and peer reviews public on the MCC website

Task 6: Disseminate Baseline Package

Once the baseline report is reviewed by the Evaluation Management Committee, the Contractor is responsible for:

o Presentation at MCC headquarters o Presentation at MCA headquarters

The Contractor should review the current version of MCC’s Evaluation Microdata Guidelines (See Section J) for guidance on preparation of public documentation that links to microdata. All baseline materials (questionnaire(s), informed consents(s), research protocol(s), updated Metadata (as needed), internal and/or public-use Baseline Report, and Data Package Worksheet (See Section J – Evaluation Microdata Guidelines link)) should be submitted as a deliverable package.

The Contractor will update the Nesstar Metadata as appropriate for the MCC Evaluation Catalog.

The Contractor may also be responsible for reviewing any materials developed by MCC public relations for dissemination on the MCC website for quality assurance.

The Contractor may be asked to participate in other MCC-financed dissemination and training events, such as MCC M&E College and MCC Evaluation Summits.

Task 7: Monitor program implementation and conduct risk assessment

The Contractor is responsible for continuous monitoring of program implementation to ensure implementation occurs according to design and any deviations from design are properly documented. This requires collaboration with MCC, MCA sector leads and implementing entities to ensure continuous support for and commitment to the evaluation design. This may require collaboration with MCC/MCA to set up a monitoring or information-sharing system. Risks and implementation updates should be reported regularly by the Independent Contractor as part of the Monthly Progress Report.

Task 8: Revise Endline Evaluation Materials

95332418Q0118 – IE Services in Support of the Morocco Workforce Development Activity Page 18

The Contractor will develop the data collection firm TORs and undertake the hiring of the data collection firm.

Contractor is responsible for any necessary updates and revisions to the Endline Evaluation Materials, including questionnaires and enumerator training materials, based on lessons learned.

The Contractor is responsible for ensuring household surveys last no longer than 120 minutes without written consent from the Evaluation Management Committee. The Contractor is responsible for reporting on the average survey time resulting from the pilot test and from field work.

The Contractor will document feedback from local stakeholders and any response to feedback. In order to obtain feedback, the Contractor may lead local workshops and/or virtual reviews of the Evaluation Materials by MCA and Implementing Entities.

The Contractor will submit the Evaluation Materials to MCC and present to the Evaluation Management Committee. All feedback and response to feedback will be documented by the Contractor.

o The Contractor is responsible for the quality of the English and local language survey instruments.

Task 9: Undertake Endline data collection

The Contractor is responsible for developing and managing all data collection and entry protocols and implementation. This requires reviewing the current version of MCC’s Evaluation Microdata Guidelines (See Section J) and ensuring appropriate Institutional Review Board (IRB) review and approval of research protocols, survey instruments, and informed consent procedures (see MCC Informed Consent Statement Template (See Section J)).

o In accordance with Section H, the Contractor should assess the local IRB requirements and determine whether or not additional, international-level IRB approval is required in order to ensure adherence to international standards.

If there are local requirements, the Contractor must abide by them.

If the local IRB is HHS registered, no additional IRB is required.

If the local IRB is not HHS registered, the Contractor also needs to sub-contract an HHS registered IRB.3

The Contractor is responsible for the training of data collection staff, field work, and data entry. This must be managed in coordination with MCC and MCA.

The Contractor is responsible for conducting and documenting quality control checks of data as it is collected and entered in order to reduce data collection and entry error.

The Contractor is responsible for establishing a system for frequent communication with the data collection firm throughout data collection and entry, through email, phone, and site visits.

Task 10: Develop Final Report and Data Documentation Package

The Contractor will review raw files from the survey firm and provide any critical feedback on production of the survey firm’s Data Collection Report.

Before conducting analysis, the Contractor will review the current version of MCC’s Evaluation Microdata Guidelines (See Section J) for guidelines on management, documentation, and dissemination of the microdata.

The Contractor is responsible for developing the Final Report in both English and French, including an Evaluation Brief. The Final Report outline will be developed in consultation with the MCC M&E lead. Please reference Final Report Outline (See Section J) as a reference.

3 HHS registered IRBs are found here - https://ohrp.cit.nih.gov/search/search.aspx?styp=bsc

95332418Q0118 – IE Services in Support of the Morocco Workforce Development Activity Page 19

The Contractor will ensure that the final analysis includes gender and social analysis as described in the evaluation design report (Task 2), including examining differences in characteristics by gender, age, income and other social variables as appropriate.

The Contractor should provide an explanation for why the Contractor’s estimate of the ERR differs from the MCC ERR estimate, if applicable. MCC EA will provide the appropriate original CBA model for this comparison, where available.

The evaluation results should be interpreted in relation to the CBA as part of the final report, and vice-versa.

Note that all work done by the Contractor in relation to the CBA is considered part of the independent evaluation and the same standards of independence apply.