MCC-17-RFQ-0103_Mongolia_RSPS_Evaluation_-_Approved.pdf

PAGE 1 OF

1. REQUISITION NO.

2. CONTRACT NO. 3. AWARD/EFFECTIVE DATE 4. ORDER NO. 5. SOLICITATION NUMBER 6. SOLICITATION ISSUE DATE

a. NAME b. TELEPHONE NO. (No Collect Calls) 8. OFFER DUE DATE/LOCAL

TIME 08/21/2017

9. ISSUED BY CODE 10. THIS ACQUISITION IS UNRESTRICTED OR SET ASIDE: % FOR:

SMALL BUSINESS

HUBZONE SMALL

BUSINESS

SERVICE-DISABLED

VETERAN-OWNED

SMALL BUSINESS

WOMEN-OWNED SMALL BUSINESS

(WOSB) ELIGIBLE UNDER THE WOMEN-OWNED

SMALL BUSINESS PROGRAM

EDWOSB

8(A)

NAICS:

SIZE STANDARD:

11. DELIVERY FOR FOB DESTINA-

TION UNLESS BLOCK IS

MARKED

SEE SCHEDULE

12. DISCOUNT TERMS

13a. THIS CONTRACT IS A

RATED ORDER UNDER

DPAS (15 CFR 700)

13b. RATING

14. METHOD OF SOLICITATION

RFQ IFB RFP

15. DELIVER TO CODE 16. ADMINISTERED BY CODE

17a. CONTRACTOR/OFFEROR CODE FACILITY CODE 18a. PAYMENT WILL BE MADE BY CODE

TELEPHONE NO. DUNS: DUNS+4:

PHONE: FAX:

17b. CHECK IF REMITTANCE IS DIFFERENT AND PUT SUCH ADDRESS IN OFFER

18b. SUBMIT INVOICES TO ADDRESS SHOWN IN BLOCK 18a UNLESS BLOCK BELOW IS CHECKED

SEE ADDENDUM

19. 20. 21. 22. 23. 24.

ITEM NO. SCHEDULE OF SUPPLIES/SERVICES QUANTITY UNIT UNIT PRICE AMOUNT

(Use Reverse and/or Attach Additional Sheets as Necessary)

25. ACCOUNTING AND APPROPRIATION DATA 26. TOTAL AWARD AMOUNT (For Govt. Use Only)

27a. SOLICITATION INCORPORATES BY REFERENCE FAR 52.212-1, 52.212-4. FAR 52.212-3 AND 52.212-5 ARE ATTACHED. ADDENDA ARE ARE NOT ATTACHED.

27b. CONTRACT/PURCHASE ORDER INCORPORATES BY REFERENCE FAR 52.212-4. FAR 52.212-5 IS ATTACHED. ADDENDA ARE ARE NOT ATTACHED

28. CONTRACTOR IS REQUIRED TO SIGN THIS DOCUMENT AND RETURN _______________ 29. AWARD OF CONTRACT: REF. ___________________________________ OFFER

COPIES TO ISSUING OFFICE. CONTRACTOR AGREES TO FURNISH AND DATED ________________________________. YOUR OFFER ON SOLICITATION

DELIVER ALL ITEMS SET FORTH OR OTHERWISE IDENTIFIED ABOVE AND ON ANY (BLOCK 5), INCLUDING ANY ADDITIONS OR CHANGES WHICH ARE

ADDITIONAL SHEETS SUBJECT TO THE TERMS AND CONDITIONS SPECIFIED SET FORTH HEREIN IS ACCEPTED AS TO ITEMS:

30a. SIGNATURE OF OFFEROR/CONTRACTOR 31a. UNITED STATES OF AMERICA (SIGNATURE OF CONTRACTING OFFICER)

30b. NAME AND TITLE OF SIGNER (TYPE OR PRINT) 30c. DATE SIGNED 31b. NAME OF CONTRACTING OFFICER (TYPE OR PRINT) 31c. DATE SIGNED

AUTHORIZED FOR LOCAL REPRODUCTION (REV. 2/2012)

PREVIOUS EDITION IS NOT USABLE Prescribed by GSA - FAR (48 CFR) 53.212

7. FOR SOLICITATION

INFORMATION CALL:

STANDARD FORM 1449

OFFEROR TO COMPLETE BLOCKS 12, 17, 23, 24, & 30

SOLICITATION/CONTRACT/ORDER FOR COMMERCIAL ITEMS 70

MCC-17-RFQ-0103 07-21-2017

David F. Fuller 202-521-3622 Noon Eastern

Millennium Challenge Corporation

Contracts and Grants Management Division

1099 Fourteenth Street NW

Suite 700, Washington DC 20005

X

X

N/A

X

Millennium Challenge Corporation

CGM

1099 Fourteenth Street NW, Suite 700

Washington DC 20005

Millennium Challenge Corporation

Contracts and Grants Management Division

1099 Fourteenth Street NW

Suite 700

Washington DC 20005

Interior Business Center

M/S D-2773

7301 West Mansfield Avenue

Lakewood CO 80235-2230

This is a combined synopsis/solicitation for commercial items prepared in accordance with the format in FAR Subpart 12.6 and conducted per the procedures of FAR Subpart 13.5, as supplemented with additional information included in this notice.

MCC seeks a Contractor to provide Independent Evaluation

Services in support of Mongolia Registry System Process Survey

(RSPS) Evaluation.

See enclosed solicitation and all attachments for pertinent details.

See CONTINUATION Page

X X

Contracting Officer

Regina E. Maxwell 7/21/2017

MCC-17-RFQ-0103 (Mongolia Registry System Process Survey (RSPS) Evaluation Page 2

Table of Contents

SECTION B – SUPPLIES OR SERVICES AND PRICE/COSTS

B.1. Price/Cost Schedule

B.2. FAR 52.216-1 -- Type of Contract (APR 1984)

B.3. Accounting and Appropriation Data

B.4. Funding Type

B.5. Limitation of Funds – Incrementally Funded Contracts

B.6. Labor Rates & Other Direct Costs (ODCs)

SECTION C – DESCRIPTION/SPECIFICATIONS/STATEMENT OF WORK

C.1. Introduction and Background

C.2. Scope

C.3. Roles and Responsibilities

C.5. Tasks

SECTION D – PACKAGING AND MARKING

D.1. Disclaimer for Deliverables

SECTION E – INSPECTION AND ACCEPTANCE

E.1. Inspection of Reports

E.2. Quality Control Plan

SECTION F – DELIVERIES OR PERFORMANCE

F.1. Period of Performance

F.2. Place of Performance

F.3. Deliverables

F.4. Reporting Deliverables

F.5. Deliverable Format

F.6. Acceptance Criteria

SECTION G – CONTRACT ADMINISTRATION DATA

G.1. MCC 52.201-70 Contracting Officer’s Representative (COR) and/or Project Monitor (PM) (JULY 2012) .. 25

G.2. MCC 52.232-70 Invoice Instructions (Custom)

SECTION H – SPECIAL CONTRACT REQUIREMENTS

H.1. MCC 52.203-70 Contractor Non-Disclosure Agreement (MAY 2017)

H.2. MCC 52.204-70 Security Requirements for Contractor Personnel (MAY 2013)

H.3. MCC 52.209-70 Organizational Conflicts of Interest: Preclusion from Implementation Contract (JULY 2012)

H.4. MCC 52.232-72 Limitation of Funds – Incrementally Funded Contracts (NOV 2006)

H.5. MCC 52.232-73 Travel Reimbursement (MAY 2017)

H.6. MCC 52.232-74 ODC Reimbursement (JAN 2007)

H.7. MCC 52.242-70 Contractor Performance Assessment Rating System (CPARS) Registration (AUG 2011) .. 36

H.8. MCC 52.243-70 Increase in Services (OCT 2006)

MCC-17-RFQ-0103 (Mongolia Registry System Process Survey (RSPS) Evaluation Page 3

H.9. MCC 52.245-70 Contractor Acquired MCC Government Property Overseas (JULY 2012)

H.10. Provision on Peer Review and Posting of Reports

H.11. Provision on Insurance Maintenance

H.12. Provision on Stakeholder Input for Evaluation Products

H.13. Provision on Human Subjects Protection

H.14. Personally Identifiable Information and Confidentiality

H.15. Provision on Delivered Work Product Free from 3rd Party Rights

H.16. Key Personnel

H.17. Team & Key Personnel Qualifications

PART II – CONTRACT CLAUSES

SECTION I – CONTRACT CLAUSES

I.1. FAR 52.212-4 Contract Terms and Conditions — Commercial Items/Alternate I (JAN 2017)

I.2. FAR 52.212-5 Contract Terms and Conditions Required to Implement Statutes or Executive Orders –

Commercial Items (JAN 2017)

I.3. FAR 52.214-34 Submission of Offers in the English Language (APR 1991)

I.4. FAR 52.214-35 Submission of Offers in U.S. Currency (APR 1991)

I.5. FAR 52.217-9 Option to Extend the Term of the Contract (MAR 2000)

I.6. FAR 52.252-2 Clauses Incorporated by Reference (FEB 1998)

PART III – LIST OF DOCUMENTS, EXHIBITS AND OTHER ATTACHMENTS

SECTION J – LIST OF ATTACHMENTS

SECTION K – REPRESENTATIONS, CERTIFICATIONS AND OTHER STATEMENTS OF OFFERORS OR

RESPONDENTS

SECTION L – INSTRUCTIONS, CONDITIONS, AND NOTICES TO OFFERORS

L.1. Inquiries/Questions Concerning the Solicitation

L.2. Exclusion of Quotes and Pre-Award Communications with Offerors

L.3. Quotation Due Date

L.4. Quotation Response

L.5. Quotation Volumes

L.6. Quotation Format

L.7. Blocked and Unacceptable Electronic File Types

L.8. Page Limits

L.9. Preparation of Volume I – Technical Quotation

L.10. Preparation of Volume II – Cost Quotation

SECTION M – EVALUATION FACTORS FOR AWARD

M.1. FAR 52.212-2 Evaluation – Commercial Items (OCT 2014)

M.2. Evaluation Factor 1 – Technical Capability

M.3. Evaluation Factor 2 – Past Performance

M.4. Evaluation Factor 3 – Price

M.5. Basis for Award

MCC-17-RFQ-0103 (Mongolia Registry System Process Survey (RSPS) Evaluation Page 4

SECTION B – SUPPLIES OR SERVICES AND PRICE/COSTS

B.1. Price/Cost Schedule

CLIN Description of Supplies/Services Qty U/I Unit

Price Amount

0001 BASE PERIOD

Time and Materials (T&M)

Period of Performance: TBD

Estimated 6 months

1 LO US$___________ US$___________

1001 OPTION PERIOD I

Time and Materials (T&M)

Period of Performance: TBD

Estimated 12 months

1 LO US$___________ US$___________

TOTAL ESTIMATED PRICE:* US$_______________

*MCC reserves the right to set the contract ceiling above the total estimated price to account for potential deviations due to unforeseen circumstances. However, the Contractor will be expected to perform in good faith in accordance with their proposed cost estimates.

B.2. FAR 52.216-1 -- Type of Contract (APR 1984)

The Government contemplates award of a time-and-materials (T&M) contract resulting from this solicitation.

(End of Provision)

B.3. Accounting and Appropriation Data

[RESERVED]

B.4. Funding Type

This contract will be funded, at time of award, with no-year Due Diligence funding. As such, funding obligated to this contract may be used during the life of the contract.

B.5. Limitation of Funds – Incrementally Funded Contracts

In accordance with clause MCC 52.232-72, “Limitation of Funds – Incrementally Funded Contracts,” funds in the amount of [TBD] have been allotted to this contract. It is contemplated that funds now obligated under this contract will cover the work to be performed until [TBD]. If the contractor considers the funds obligated under this contract to be insufficient to cover the work to be performed until the date specified herein, or another date agreed to by the parties, the contractor shall notify the contracting officer in writing and indicate the date on which it expects expended funds to approximate 75 percent of the total amount obligated. The notice shall state the estimated amount of additional funds required to continue performance through the date for which incremental funds are provided.

The Contractor shall not perform work on the contract which exceeds the total amount actually allotted by the

Government to the contract. The Government is not obligated to reimburse the Contractor for costs incurred in excess of

MCC-17-RFQ-0103 (Mongolia Registry System Process Survey (RSPS) Evaluation Page 5 the total amount allotted by the Government to this contract. The contractor assumes the risk for any increased costs beyond what the Government obligates.

B.6. Labor Rates & Other Direct Costs (ODCs)

Labor rates at the inception of a contract period will carry through as “fixed-price” without adjustment for the life of that contract period.

Hours by labor category and pricing under ODCs are estimates only and may be increased/decreased as necessary for successful completion of the contract requirements and as approved by the COR/PM. However, the contractor’s expenditures at any given point shall not exceed the total obligated amount or the total ceiling price for this contract as specified below.

Using the template below, Offerors must include all proposed labor categories, fully-burdened hourly rates and Other

Direct Costs (ODCs) for each contract period identified in B.1.

PRICING SUMMARY PER CONTRACT PERIOD

Base Option 1 TOTAL

Total Labor Hours

Total Labor Costs $ $ $

Total ODCs $ $ $

Total Est. Cost $ $ $

Base Period – Period of Performance TBD (est. 6 months)

Item

Description(2) U/I Qty Rate Total

LABOR

0001 hr.

hr.

hr.

hr.

Total Labor Hours - -

Labor Sub-Total

ODCs(1)

ODC Sub-Total

Total Estimated Value (Base Period)

MCC-17-RFQ-0103 (Mongolia Registry System Process Survey (RSPS) Evaluation Page 6

Option Period I – Period of Performance TBD (est. 12 months)

Item # Description(2) U/I Qty Rate Total

LABOR

1001 hr.

hr.

hr.

hr.

Total Labor Hours - -

Labor Sub-Total

Total Estimated Value (Option Period I)

TOTAL COST SUMMARY

Description(2) U/I Qty Rate Total

LABOR

hr.

hr.

hr.

hr.

Total Labor Hours - -

Labor Sub-Total

Total Estimated Cost

NOTE (1): MCC anticipates approximately $250,000 worth of data collection subcontracting costs for which the requirements are unknown at this time. To be fair to all vendors providing a quotation, a plug number is utilized for data collection subcontracting costs in the respective years in which data collection is expected to occur. However, if you believe this plug number is inadequate, please advise during the Question and Answer period of the solicitation. “Data Collection Subcontract(s),” as used in the tables above, refers only to the cost of subcontracting a data collection firm and any associated indirect costs (e.g., subcontract handling fees). It does not include any prime contractor labor associated with overseeing/managing the data collection effort; this time should be incorporated into the level of effort proposed under the respective

“Direct Labor” CLIN. Subcontracting costs for data collection are subject to the requirements of FAR

52.244-2, Subcontracts.

NOTE (2): A job description shall be included for each labor category proposed. The job description must identify essential and non-essential tasks that are assigned to a specific position. It must also identify reporting relationships and shall also describe required qualifications, minimum requirements, working conditions and desirable qualifications.

(End of SECTION B)

MCC-17-RFQ-0103 (Mongolia Registry System Process Survey (RSPS) Evaluation Page 7

SECTION C – DESCRIPTION/SPECIFICATIONS/STATEMENT OF WORK

C.1. Introduction and Background

Introduction

The Millennium Challenge Corporation (MCC) is seeking independent evaluation services in support of the Registry

Systems Process Study (RSPS) Evaluation, which assesses the performance and results of the Land Privatization and

Registration System Activity that was implemented in Mongolia’s eight (8) regional centers and capital city of

Ulaanbaatar between 2008 and 2013. This scope of work covers the follow-up evaluation activities. A baseline evaluation was conducted in 2013 by Innovations for Poverty Action (IPA). The Evaluation Design Report, Baseline

Report and Banking Questionnaires can be found online on MCC’s Evaluation Catalogue:

https://data.mcc.gov/evaluations/index.php/catalog/134 .

The Land Privatization and Registration System Activity provided institutional strengthening of Mongolia’s General

Authority for State Registration (GASR) in an effort to improve Mongolia’s formal system of registering rights, including decreasing the time and cost associated with conducting a formal land transaction and increasing demand for formal land registration services. Sub-activities included identifying and addressing obstacles to efficient and affordable land privatization and registration, capacity building for government offices, upgrading physical infrastructures of state registry offices and the geospatial infrastructure and establishment of an electronic registration system.

Background

The Mongolia Compact

On October 23, 2007, the Government of Mongolia (GoM) and the Millennium Challenge Corporation (MCC), a United

States Government agency, signed a five-year $285 million Compact to reduce poverty in Mongolia through economic growth. The five year Compact entered into force on September 17, 2008 and ended on September 17, 2013.

The Compact consisted of five projects: the Property Rights Project, the Vocational Education Project, the Health Project, the North-South Road Project, and the Energy and Environment Project. The Property Rights Project was composed of two sub-projects: the two urban activities—Privatization of Ger Area Land Plots Activity1 and Improvement of the Land

Privatization and Registration System Activity—were commonly referred to as the Property Rights Project (PRP), while the Peri-Urban Land Leasing Activity was known as the Peri Urban Rangeland Project (PURP).

PRP aimed to address the prohibitively long, costly and complicated process of obtaining formal land rights through initiating various institutional strengthening measures to facilitate citizens’ access to obtaining private land titles in

Mongolia’s 8 regional centers and capital city, as well as by providing direct assistance with the privatization and registration of land plots in informal settlement areas called ger districts. Together, these activities were expected to increase the security and capitalization of land assets.

1 There was a RCT evaluation of the privatization of ger area land plots activity via the Special Hashaa Plot Survey (SHPS) evaluation, but the SHPS evaluation was discontinued due to loss of evaluation power. Reports from the SHPS baseline survey and tracking survey can be found online on

MCC’s Evaluation Catalogue.

https://data.mcc.gov/evaluations/index.php/catalog/134

MCC-17-RFQ-0103 (Mongolia Registry System Process Survey (RSPS) Evaluation Page 8

Land Context in Mongolia

From 1990 to 2010, urban areas experienced explosive growth with the population in Ulaanbaatar doubling from 536,600 to 1,161,800. Many migrants settled in underdeveloped, unplanned urban areas, called ger districts. The influx of migration to ger areas highlighted the need for strong and cohesive property rights regulations and more accessible pathway to property ownership. Although the possibility of property ownership in urban areas was formally established over ten years ago, securing ownership of property remained challenging for residents of lower-income ger districts as they dealt with a complex registration process.

Land Privatization and Registration System Activity

The Land Privatization and Registration System Activity aimed to strengthen GASR by: 1) supporting a commission of stakeholders and technical experts (Legal and Regulatory Commission) to study the obstacles to land registration, make recommendations towards alleviating them, and work with the relevant parties to implement the recommendations; 2) upgrade the geospatial infrastructure necessary for accurate land parcel mapping; 3) build capacity for GASR offices and

4) upgrade IT systems and physical infrastructure of GASR’s central office, 8 regionalized offices and establishment of 4 new decentralized UB district offices. See Annex 1, Section C of the Mongolia Compact for additional information:

https://assets.mcc.gov/content/uploads/2017/05/compact-mongolia.pdf

By improving legal regulations and strengthening the state institutions charged with overseeing property related transactions, the PRP expected to lower the transaction time/costs required to register property and increase use of the formal land system. The PRP also funded an outreach campaign to educate citizens on their property rights, and pushed for legislative and regulatory changes affecting property rights. The outreach campaign also encouraged residents to obtain ownership of their land and property with a particular focus on encouraging women to register land in their own names.

The main expected result for Mongolians was a simplified and easier registration process with reduced transaction cost.

Usage of modern technology via an electronic Property Registry System (ePRS) was expected to enable better services, including possibilities to use electronic means (internet web-portal, email). The cooperation and agreements with commercial banks on electronic reference letters and registration of mortgages were expected to directly reduce the cost of mortgage related services to clients.

Beyond the provision of computer hardware/equipment and construction services, there were two implementing contractors key to the strengthening of GASR’s property registry services:

1) SK C&C (Korean Firm)2, who built GASR’s electronic Property Registration System (e-PRS) and carried out related training and capacity building to GASR officials3.

2) Orgut-COWI consortium4 who assessed and implemented suggested improvements to GASR property registration procedures, including training, for GASR’s Property Rights Registration Department (PRRD) in order to make registration processes more efficient and effective for the public and private sector. There were 3 phases of the consultancy, as well as training completed across all phases to the 338 GASR officers. Phase details follow, below:

2 Replaced Thomson Reuters whose contract was terminated.

3 Evaluator will receive ePRS reports on training, pilot testing and program installation.

4 Replaced BlomInfo, whose contract was terminated and worked from March 2010-June 2011 to put in place recommendations of the Legislative and Regulatory Commission (LRC).

https://assets.mcc.gov/content/uploads/2017/05/compact-mongolia.pdf

MCC-17-RFQ-0103 (Mongolia Registry System Process Survey (RSPS) Evaluation Page 9

a) Assessment Phase: review of business processes for property rights registration, including archiving, human resources and financial management functions of GASR;

b) Strategic Management Phase: preparation of a Strategy and Sustainability Plan with Action Plan for 10 and 5 years, an Annual Business Plan and an Operations Manual for Registration; and

c) Sustainable Implementation: drafting of revisions to the Law on Registration of Immovable Property Rights, a set of draft Policies and a proposal on future organizational options.

By the end of the Orgut-COWI consultancy, about 60% of the recommendations provided in the Assessment Phase were adopted or were under preparation and all 4 main recommendations were included in the draft Law on Registration of

Immovable Property Rights, which once enacted, was supposed to enable full implementation of the reforms, including single authority for property rights registration, modern concept of unified property object (land and attachment to land), utilization of unique parcel identification number and improved access to information. The final report from the consultancy, “The Sustainable Implementation of the New Property Registry System” can be found in Section J. During the post Compact period, MCC understands from the Government of Mongolia that although the draft Law on

Registration of Immovable Property Rights was discussed at the Parliament standing committee and working groups in summer 2013, by summer 2014, the Prime Minister recalled the package law from Parliament because of public concerns around controversial issues around land ownership. As such, key expected legal and regulatory changes to allow one stop shops, streamline certain land privatization and registration procedures and allow banks to access ePRS does not seem to have occurred. Although there were no legal changes to allow the banks to directly access ePRS records, in 2013, GASR signed an MOU with the Bankers Association which enabled the banks to send and obtain land title inquiries via emails.

This allowed the banks to no longer contact GASR with such requests. Verification of what PRP recommendations were adopted and sustained, including legal, regulatory and operational changes and related effects on transaction time would need to be reviewed by the Evaluator.

Table 1 below outlines the timeline of completed PRP activities through Compact end:

Table 1. Implementation Timeline

Activities Date Completed

Legislative and Institutional Strengthening

Legislative and Regulatory Commission (LRC) is formed 7/2008

LRC presents a comprehensive set of recommendations 12/2009

Developed draft law for recommended legal reforms 7/2013

Piloting and establishing a centralized electronic registration system 8/2013 – 12/2013

Digitalization of 354,224 property registration archive folders or 27,000,000 pages 8/2013

Support changes in organizational structure and HR practices to be consistent with streamlined process and ePRS 8/2013

Collaborated w/ ALAGaC and banks to make ePRS data specification compatible with other agency data and systems 8/2013

Upgrading Geospatial Infrastructure

Two experts from National Geodetic Survey (NGS) of National Oceanic & 2/2009

MCC-17-RFQ-0103 (Mongolia Registry System Process Survey (RSPS) Evaluation Page 10

Atmospheric Administration (NOAA) visit Mongolia

6 CORS and 16 GPS worth one billion MNT are installed and delivered to beneficiaries 12/2010

High resolution satellite imagery is captured for Ulaanbaatar and eight regional centers 6-9/2010

Satellite imagery of hashaa plots is acquired and consolidated, covering a total of 190,000 ha of land encompassing Ulaanbaatar and eight regional centers

1-3/2011

Supplied IT equipment and standard software. Supplied equipment, hardware, software and related services

9/2011 – 1/2013

Capacity Building for Land Offices

MCA-M hires and trains thirteen Land Market Specialists (LMS) in order to train registry officers and land officers as well as to reach closer to citizens and educate them 1/2010

Upgrading Registry Offices

Four property registration offices in Ulaanbaatar are newly established and upgraded and one office in each of the eight regional centers are upgraded 2/2010 – 12/2011

Eleven registry buildings are refurbished, furnished, and fully equipped 6/2011 – 5/2012

The Registry Systems Process Study (RSPS) Evaluation

RSPS is a pre-post performance evaluation, which aims to capture changes in time to conduct a formal land transaction and related changes in demand and volume of formal land transactions. The pre-post comparison also implicitly captures the impact of any legal amendments on access to property records and other capacity building/institutional strengthening of the Registry. Specifically, RSPS assesses the effects from the subactivities which aimed to improve the legislative and regulatory environment to register land, GASR IT and electronic registry system, operational procedures and physical infrastructure.

Innovations for Poverty Action (IPA) conducted an initial baseline assessment in 2010, surveying GASR customers.

However, there were flaws in the design and survey tools, which led to a revised evaluation design and new baseline in

2013. The revised baseline evaluation used three different survey instruments and collected data on roughly 900 individuals and commercial entities engaged in the completion of banking related transactions that involve GASR, as well as data on the back office processing time at GASR for close to 3,000 land transactions.

RSPS components:

1) A bank customer survey to capture changes in time to access a bank loan using land as collateral

2) Review of administrative data records to capture back office processing times at GASR

3) Informational interviews with GASR stakeholders

MCC-17-RFQ-0103 (Mongolia Registry System Process Survey (RSPS) Evaluation Page 11

Further details on each of the three components follow, below:

RSPS Banking Survey

• Purpose: Measure the impact of registry system process upgrade on time and cost involved in obtaining mortgage or land-based loan

• Evaluation Design: Interview bank customers to capture formal and informal costs and time spent obtaining mortgages or land-based loans at banks before and after registry system process upgrades and banks and GASR offices start sharing information via ePRS

• Data Collection:

o Baseline data collection carried out mid-September 2013 until end of November in 15 banks o Target to collect 900 mortgage and land-based loan transactions

RSPS Back Office Time Tracking Survey

• Purpose: to evaluate any changes in the efficiency of the privatization and registry process at GASR offices, including time savings of carrying out various primary and secondary land transactions

• Evaluation Design: capture procedural time associated with eight key transactions at GASR before and after the full implementation of ePRS, with the baseline data collection occurring in July-August 2013

• Data Collection:

o Contractor observed general transaction process at GASR and made general transaction diagrams by mapping out all steps of each transaction o Barcodes were attached to documents and scanned at specific stages to track their progress through

GASR offices o Once procedure was finalized and tested, contractor collected data on the key transactions for a month in six district offices of UB o 2,906 total transactions were tracked and timed to completion in GASR offices

RSPS GASR Informational Interviews

• Purpose: to understand the GASR registry workers perception of the GASR process and their awareness of the

PRP

• Survey conducted: in four GASR offices in UB and one GASR office in Darkhan in July and August 2013. Two or three registry officers who work directly with customers gave interviews in each of the offices

• Focused on collecting GASR officers’ perception of:

o Changes in time, cost, and number of step in transactions o Changes in registration rates and drivers of registration o Changes brought on by GASR office technology upgrades by PRP o Outreach efforts witnessed o Impact of Building construction, upgrades, or refurbishment of GASR offices on their workload or work process o Knowledge of the electronic property registration system o General knowledge of the PRP activities

IPA also considered a fourth component, a public awareness survey to capture whether the PRP public outreach campaign might have had any effect on registration rates. However, initial analysis of GASR registry records showed this was unlikely and, at any rate, would also be hard to isolate the effect given other donor efforts, climbing property values, introduction of infrastructure in the gers, and uncertainties surrounding Mongolia’s promise of free land. The chart, below, shows the volume of land registrations across time.

IPA’s RSPS Baseline Report and anonymized data can be found on MCC’s Evaluation Catalogue:

MCC-17-RFQ-0103 (Mongolia Registry System Process Survey (RSPS) Evaluation Page 12 https://data.mcc.gov/evaluations/index.php/catalog/1345. The report describes the RSPS evaluation methodology and the survey instruments and summarizes the result of baseline data collected. The report also outlines possible avenues for post-intervention data collection and expands on some of the risks of drawing conclusions from this methodology.

Prior to follow-up evaluation data collection, IPA recommended that the Government of Mongolia pass related land legislation and finish the digitization of land records. Banks could not access ePRS to expedite the mortgage and loan approval process until PRP-supported legislation was passed by the Parliament and GASR and banks established protocols to share information on ePRS. Similarly gains in efficiency for banking processes were not expected to be realized until digitization was completed in GASR offices. Although it seems key land law reforms were not enacted, Banks were in theory given expedited access to records via email to GASR through the MOU with the Bankers

Association referenced above.

C.2. Scope

MCC requires an independent evaluator to conduct the follow-up RSPS evaluation of the performance and outcomes of the Land Privatization and Registration System Activity.

MCC’s primary research questions are:

1) Did PRP lead to increases in demand and volume of formal land transactions, including land registration and related transfers at GASR and mortgages at the banks?

2) What were the characteristics of those who conducted formal land transactions before and after the Compact?

3) Did PRP result in changes in land transaction costs/time for land transactions?

4) Were there any differences in results for parcels held by women/men?

The Offeror should consider the evaluation design developed by IPA; however, the Offeror must assess the current environment and make an independent decision based on their respective analysis on the best evaluation strategy. For example, although the baseline evaluation captured data records in process, GASR records have since been largely scanned and digitized in the ePRS, which could allow the evaluation design to capture data volumes and transaction times based on submission and approval dates from these scanned historical land records in the ePRS.

C.3. Roles and Responsibilities

A. Independence

MCC’s Policy on Independent Evaluations. Based on the 1991 DAC Principles for Evaluation of Development

Assistance, MCC’s Evaluation Policy requires independent evaluations of each MCC project. MCC’s implementation of this independence policy adheres to these principles, especially the following: “Impartiality and independence will best be achieved by separating the evaluation function from the line management responsible for planning and managing development assistance.” 6

Organizational Conflicts of Interest. MCC-M&E is responsible for adhering to our policy-driven requirements for independence. These requirements are distinct from Organizational Conflicts of Interest (OCI) proscribed in the Federal

Acquisition Regulations. Interpretations and enforcement of the OCI statutes are the remit of our Office of General

Counsel.

Independence of Evaluation Team. Evaluation team members must be impartial and therefore should not have contributed to the design or implementation of the project being evaluated. In cases where individuals or firms that have

5 The winning bidder will have access to the non-anonymized data files.

6 1991 DAC Principles for Evaluation of Development Assistance, Pg 6

MCC-17-RFQ-0103 (Mongolia Registry System Process Survey (RSPS) Evaluation Page 13 worked on the project are proposed to be part of the evaluation team, a justification and independence risk mitigation plan must be provided to MCC for consideration.

The Independent Evaluator’s Role. To remain consistent with the principles of independent evaluations during project planning and implementation, the independent evaluator’s feedback is restricted to reporting evidence and findings.

Recommendations shall not be made before completing a final evaluation as they can influence project planning or implementation, thereby jeopardizing the independence an evaluation. Prescriptions and recommendations may be included in final evaluations.

Real-time feedback to project implementation. MCC supports monitoring and evaluation activities intended to provide feedback on how to improve program planning and implementation. But this support shall be separate from the independent evaluator. Any contribution of the Independent Evaluator to this feedback shall be limited to reporting evidence and findings. Any exceptions to this rule must be approved in advance by the EMC.

B. MCC and GoM Responsibilities

The MCC, GoM point of contact (POC), along with the independent evaluator, play critical roles in designing, implementing and disseminating the independent evaluations. The MCC and GoM POC roles are as follows:

MCC is responsible for oversight of the independent evaluator and quality control of evaluation activities, including the following specific responsibilities:

Set the evaluation priorities and evaluation questions: Using the program logic as the foundation MCC, in coordination with the GoM, developed the evaluation priorities and evaluation questions based on existing evidence, the evidence gap, and the learning priorities for the sector and agency.

Build buy-in and ownership of the evaluation: MCC will work with the independent evaluator and the GoM POC to ensure the evaluation design is based on feasible methodologies considering past evaluation data collection and program implementation. MCC will work with the GoM POC to ensure commitment to the evaluation work plan by local stakeholders, and if is not possible to adhere to the work plan, will inform the independent evaluator in order to adapt the evaluation design.

Manage the independent evaluator: MCC will develop Terms of Reference for the independent evaluator and manage the contracting process, including Request for Proposals, Proposal Review, selection of the independent evaluator, development of a work plan, and supervision of the independent evaluator in line with the work plan and agreed deliverables.

Quality review of evaluation materials: Using the program logic as a foundation, MCC is responsible for reviewing evaluation materials, including the design report and survey materials, for feasibility, technical and factual accuracy, and adherence with international standards.

Quality review of evaluation reports: MCC is responsible for reviewing independent evaluators’ reports to ensure technical and factual accuracy. While MCC will provide feedback, the independent evaluator is ultimately the author of the evaluation reports and may or may not take into consideration MCC’s comments. MCC therefore develops an Official Statement for each evaluation report to confirm the technical and factual accuracy of the report, or clarify any differences on technical or factual issues between MCC and the independent evaluator.

MCC-17-RFQ-0103 (Mongolia Registry System Process Survey (RSPS) Evaluation Page 14

Through this review process, MCC’s role is to enable the independent evaluator’s independence in order to produce a high quality, unbiased evaluation of the program. MCC should ensure that various stakeholders understand that while MCC, the GoM POC, and other stakeholders may provide guidance and leadership on evaluation questions and feasibility of methodology, once an approved evaluation design is in place, feedback on analysis and interpretation of results is limited to statements on the continued technical and factual accuracy of the analysis.

Lead final public dissemination efforts. MCC is responsible for leading public dissemination of results, in particular ensuring a feedback loop internally in order to inform future decision making on similar program investment, design, and implementation. This requires ensuring the evaluation results are disseminated on the

MCC website, and results are disseminated to appropriate stakeholders in a timely fashion.

The Government of Mongolia POC (GoM POC): After the end of a Compact, the Millennium Challenge Account –

Mongolia (the entity responsible for implementation of the Compact) was disbanded, but a Post-Compact Monitoring &

Evaluation representative of the GoM POC) within the Cabinet Secretariat was appointed. The GoM, through this POC, is responsible for building local ownership and commitment to the evaluation, oversight of the data collection firm, and quality control of evaluation activities, including the following specific responsibilities:

Build buy-in and ownership of the evaluation. The GoM POC will work with the independent evaluator, GASR and MCC to ensure the evaluation design follows the rules of program design and implementation. When possible, the GoM POC will work with the independent evaluator and MCC to develop a program implementation work plan that enables a rigorous evaluation design. The GoM POC will work with MCC to ensure commitment to the work plan by local stakeholders, and if it is not possible to adhere to the work plan, will inform MCC in order to request the independent evaluator adapt the evaluation design.

Quality review of evaluation materials. Using the program logic as a foundation, the GoM POC in coordination with local stakeholders such as GASR is responsible for reviewing evaluation materials, including the design report and survey materials, for feasibility, technical and factual accuracy, and adherence with international standards.

Quality review of evaluation reports. The GoM POC, or a designated representative, is responsible for reviewing independent evaluators’ reports to ensure technical and factual accuracy. While the GoM POC will provide feedback, the independent evaluator is ultimately the author of the evaluation reports and may or may not take into consideration the GoM POC’s comments. The GoM POC therefore develops an Official Statement for each evaluation report to confirm the technical and factual accuracy of the report, or clarify any differences on technical or factual issues between the GoM POC and the independent evaluator.

Facilitate public dissemination efforts. The GoM POC, or a designated representative, is responsible for facilitating public dissemination of results and coordination with GASR, in particular ensuring a feedback loop in-country in order to inform future decision making on similar program investment, design, and implementation.

Independent Evaluator Roles and Responsibilities: The independent evaluator is responsible for the overall design, implementation and dissemination of the evaluation, including the following specific responsibilities:

Assess the existing evaluation design given the history of implementation, sustainability of results and feasibility of options. Using the program logic as the foundation, independent evaluators must assess the program design, implementation and current status and environment to develop the most rigorous evaluation design feasible. The independent evaluator is responsible for developing an evaluation design report, in consultation with MCC, GASR and the GoM POC, which summarizes the evaluation design, sampling, analysis plan, and other critical

MCC-17-RFQ-0103 (Mongolia Registry System Process Survey (RSPS) Evaluation Page 15 design elements of the evaluation.

In the case of RSPS, the independent evaluator should review the existing RSPS methodology and design report, the implementing PRP contractor reports, as well as the operations of GASR and related ePRS in the post compact period. If any changes are suggested to improve the quality of the evaluation, these should be discussed with MCC and documented in the evaluation design report. The evaluator should assess whether all or parts of the quantitative data collection can be completed simply by extracting data from the ePRS system, or whether data collection by a different method is required.

Support MCC and the GoM POC to build buy-in and ownership of evaluation, including of GASR—a key stakeholder in the evaluation. The independent evaluator will meet with MCC and local stakeholders to develop the evaluation design based on program design and implementation. In addition, the independent evaluator will continuously present the evaluation objectives, materials, and results for MCC and local stakeholders to maintain commitment to the evaluation.

Develop evaluation materials that are held to international standards. Using the program logic, the RSPS baseline

(and accompanying lessons learned) and status of the Land Privatization and Registry System Activity recommendations and outputs as a foundation, the independent evaluator will define how key outcomes will be measured and develop the survey instruments or other data collection tools in order to answer critical evaluation questions. This applies to both quantitative and qualitative instruments.

Ensure appropriate review of evaluation materials and research protocols. Independent evaluators are responsible for ensuring all evaluation materials, including the evaluation design, survey instruments, sampling strategy, data collection and entry protocols are reviewed by an institutional review board (IRB). IRB review and approval is required before any data collection commences. The independent evaluators will document all approvals and informed consent procedures throughout the evaluation time period.

Procure and manage the local data collection firm(s). While much of the data may be extracted directly from the ePRS system, additional data collection may be required, such as qualitative data collection. As appropriate, the evaluator will develop any required Terms of Reference for the data collection firm(s) and is responsible for procuring and managing the contracting process, including Request for Proposals, Proposal Review, selection of the data collection firm, development of a work plan, and supervision of the data collection firm in line with the work plan and agreed deliverables. If able, it would be helpful for the evaluator to identify the local data collection firm(s)/subcontracted firm(s) as part of its proposal. If not, a request for subcontract approval will be required during the contract period.

Supervise data collection. The independent evaluator is responsible for quality control over data collection activities, including interviewer training, pre-testing, and implementing data quality review checks during data collection.

Lead data cleaning, analysis, interpretation of results. The independent evaluator is responsible for leading any necessary data cleaning and consultation with the data collection firm to produce final analysis files. The independent evaluator is responsible for leading all analysis in line with the agreed analysis plan developed with

MCC, and consulting with MCC should the analysis plan need to be revised or adapted. The independent evaluator must also consider how to facilitate verification/replication of their analysis through appropriate documentation and dissemination of the underlying microdata, while adhering to promises of confidentiality.

Produce final evaluation report. The independent evaluator is responsible for sharing the draft evaluation report with local stakeholders and MCC for review and feedback. However, given the independent evaluator is the author of the evaluation reports, local stakeholder and MCC feedback may or may not be taken into consideration for the final report. The independent evaluator is responsible for documenting all feedback and evaluator

MCC-17-RFQ-0103 (Mongolia Registry System Process Survey (RSPS) Evaluation Page 16 responses.

Through this review process, the independent evaluator’s role is to assert independence in order to produce a high quality, unbiased evaluation of the program. The independent evaluator should ensure that various stakeholders understand that while MCC, the GoM POC, and other stakeholders may provide guidance and leadership on evaluation questions and feasibility of methodology, once an approved evaluation design is in place, feedback on analysis and interpretation of results is limited to statements on the continued technical and factual accuracy of the analysis.

Lead public dissemination efforts. The independent evaluator is responsible for leading public dissemination efforts facilitated by the GoM POC and MCC (such as local workshops and conferences), but also additional conferences or other opportunities to publicly disseminate the results of the evaluation. The evaluator will advise

MCC on any public dissemination opportunities and collaborate as appropriate. As per Section H, the independent evaluator is responsible for ensuring that all public release documentation is reviewed to minimize any risk to respondent confidentiality. This may require submission of internal use only and public release reports.

C.5. Tasks

The SOW is divided into a base period for the review of PRP and RSPS materials and revision of the Evaluation Design

Report, along with an option period of up to one year for implementation of the MCC approved evaluation design.

Specific tasks follow, below:

Base Period

Task 1: PRP Review

The Evaluator will review PRP design, implementation and evaluation materials and status of PRP outputs, including PRP outputs during compact, implementation of recommendations post compact and sustainability of PRP outputs. This review should inform the evaluation design, particularly around understanding what was designed, what was implemented and what was completed and sustained post compact. This includes a review of PRP documentation, including contractor reports, M&E Plans, Project Logic, Compact and Post Compact reporting, and liaising with key stakeholders to understand the current status of land administration and governance, especially privatization and registration of land.

Based on this review, if there are key flaws in project logic and related assumptions or problematic environmental factors that could affect the evaluation feasibility and design, these should be flagged to MCC. A kick-off meeting with MCC stakeholders will occur within the first month to answer initial questions, as well as arranging virtual discussions with

Mongolian stakeholders.

Task 2: Revise Evaluation Design Report

The Evaluator will review historic evaluation materials as part of its assessment, including raw data files from the baseline data collection, as well as the evaluation design report, data documentation package and baseline report from IPA. The evaluator will provide any critical feedback on the data quality, project logic, ERR, evaluation design and baseline report as part of its update to the Evaluation Design Report.

The Evaluator will review and revise the Evaluation Design Report, including all technical analysis required for assessing and confirming internal validity, power calculations, and required sample sizes. The Evaluation Design Report outline will be developed in consultation with the MCC M&E lead. Please reference Attachment Section J Evaluation Design

Report Outline as a reference. In the case of RSPS, the independent evaluator should review the existing RSPS methodology. If any changes are suggested to improve the quality of the evaluation, these should be discussed with MCC and documented in the revised Evaluation Design Report. The Evaluation Design Report should include the evaluation timeline, as well as associated budget. The budget may not go beyond the plug in figure of the Contract, which was

MCC-17-RFQ-0103 (Mongolia Registry System Process Survey (RSPS) Evaluation Page 17 partially based on the baseline data collection effort. If there might be concerns with the plug in data collection figure provided, the Evaluator should raise the concern in their bid.

The Evaluator will conduct missions to the field as needed to discuss program design and implementation history to review and inform the evaluation design, as well as engage local stakeholders (the GoM POC and past Implementing

Entities, as well as local government, especially GASR) on the proposed evaluation. This includes discussing PRP design, implementation and performance, as well as engaging local stakeholders highlighted above on the proposed evaluation methodology. The Evaluator will use the field mission to refine research questions, assess appropriate and feasible quantitative and qualitative evaluation methodologies, develop evaluation timeline, and build local stakeholder commitment to the evaluation design.

As per MCC’s Gender Policy and Gender Integration Guidelines, the Evaluator will ensure that evaluation design, data collection requirements, and analysis address gender and other social differences as appropriate. This involves including research questions that address social and gender dimensions of the project logic, assumptions, and project design;

disaggregating impacts by gender, age, income, and other social difference (e.g. ethnicity) as appropriate; including special modules or studies as needed. The Evaluator will also specify in the Evaluation Design Report, which key input, output and outcome parameters modeled in the MCC ERR/Cost Benefit Analysis (CBA) will be measured as part of the proposed evaluation design.