SOL-520-14-000002_-_AMD1_FINAL.pdf

2. RFP cover letter and Standard Form 33, Box 9, change the proposals due date and time to “July 7, 2014”, at “2:00 p.m. Guatemala time”.

3. Section B.8 PAYMENT OF FIXED FEE, is deleted in its entirety and substituted with the following:

“B.8 PAYMENT OF FIXED FEE

In accordance with FAR Clause 52.216-8, payment of fixed fee shall be made as specified in the Schedule.

In accordance with FAR 52.216-8, the Contracting Officer withholds a reserve not to exceed 15 percent of the total fixed fee or $100,000, whichever is less, to protect the Government’s interest. The Contracting Officer shall release 75 percent of all fee withholds under this contract after receipt of an adequate certified final indirect cost rate proposal covering the year of physical completion of this contract, provided the Contractor has satisfied all other contract terms and conditions, including the submission of the final patent and royalty reports, and is not delinquent in submitting final vouchers on prior years’ settlements. The Contracting Officer may release up to 90 percent of the fee withholds under this contract based on the Contractor’s past performance related to the submission and settlement of final indirect cost rate proposals.”

4. Section C.1 OBJECTIVE, delete in its entirety the last paragraph and substitute with the following:

“Therefore, to integrate the various components and activities included in this SOW, as well as reduce costs and promote competition, USAID would prefer a single Contractor rather than a consortium composed of various Cooperating Agencies (CA).”

5. Section E.3 MONITORING AND EVALUATION PLAN, add the following at the of this section of the RFP:

“Attachment No.2 of this Amendment includes “USAID/Guatemala Performance Management Plan FY 2012-2016” and “USAID Project M&E Plan”, which will help Offerors to develop their proposed M&E Plan for this project.”

6. Add the following clause by reference to Section I:

FAR 52.203-17 “Contractor Employee Whistleblower Rights and Requirement to Inform Employees of Whistleblower Rights” (April 2014).

7. Section L.7 A.(b), delete last sentence and substitute with the following:

“Do not use a size of font smaller than 12.”

Same section, add the following to the list of documents not included in the technical proposal page limitation:

(j) Cover Page.

8. Key Individual Certification included in Attachment No. 3 shall be submitted along with the proposal for all identified proposed key personnel.

-- END OF THIS AMENDMENT --

PAGE NO.

SOLICITATION/CONTRACT NO. ORDER NO. AMENDMENT NO.

14. DESCRIPTION OF AMENDMENT/MODIFICATION (CONTINUED)

STANDARD FORM 30 (CONTINUATION)

AMENDMENT OF SOLICITATION/MODIFICATION OF CONTRACT

CONTINUATION PAGE 2 of 2

SOL-520-14-000002 01

PAGE NO.

SOLICITATION/CONTRACT NO. ORDER NO. AMENDMENT NO.

14. DESCRIPTION OF AMENDMENT/MODIFICATION (CONTINUED)

STANDARD FORM 30 (CONTINUATION)

ATTACHMENT 1

SOL-520-14-000002

Amendment 1

QUESTIONS AND ANSWERS

RFP NO. SOL-520-14-000002

USAID/GUATEMALA QUALITY IN HEALTH

1) Section B.8 indicates “Fixed Fee, shall be allocated based upon the proportion of the labor provided over the period covered by the invoice. In the event that the contractor does not provide the total Level of Effort stipulated in the contract budget, the total amount of fixed fee will be reduced in similar proportion.” This language is more appropriate for a cost-plus-fixed-fee term type contract. Per Section B.2 the contract type for this project is a completion type contract, under which the Contractor will be held accountable for achieving results. Section B.7 references FAR 52.216-8 which outlines the payment of fixed fee in line with a completion type contract. We therefore respectfully request that USAID consider removing Section B.8.

R.1) Section B.8 of this RFP is being modified under this Amendment No. 1.

2) Page 7, Section C.1, “Objective” last paragraph states, “…USAID would prefer a single Contractor rather than a consortium composed of various Cooperating Agencies (CA) and sub-contractors.” Page 138, Section M.5, “Contracting with Small Business Concerns and Disadvantaged Enterprises” states, “USAID encourages the participation of small business concerns and disadvantaged enterprises in this project…Accordingly, every reasonable effort will be made to identify and make use of such organizations. All evaluation criteria being found equal, the participation of such organizations may become a determining factor for selection.” Please clarify if the proposal should include a small business concern.

R.2) Per Section L.5.(a), the U. S. Government anticipates awarding approximately one contract as a result of this RFP. However, and in accordance with Section M.5. , USAID encourages the participation of Small Business Concerns and Disadvantaged Enterprises as prime offerors/contractors. Per Section M.2. 3.1.d) (6), if the prime offeror/contractor is not a Small Business Concern, USAID will evaluate their utilization of Small Business concerns as subcontractors, including efforts in achieving small business participation goals, as part of evaluating the Past Performance Criterion.

Based on the above, Section C.1. is being modified as part of this Amendment No. 1 as shown in page No. 2 of this Amendment.

3) C.1 states that USAID would prefer a single contractor, but L.7 states USAID would prefer the use of a local subcontractor. Can USAID please clarify whether use of subcontractors, international or local, is permitted or if USAID prefers no formal subcontractors at the proposal stage?

R.3 Per Section L.5.(a), the U. S. Government anticipates awarding approximately one contract as a result of this RFP. In accordance with Section M.2. 2.B)a., the use of local subcontractors is required for evaluation under the RFP. Also, per Section M.2. 3.1.d) (6), if the prime offeror/contractor is not a Small Business Concern, USAID will evaluate their utilization of Small Business concerns as subcontractors.

4) Can USAID please clarify which technology costs (project website, patient records system, quality management system) of this project will be assumed by the Offeror and which will be covered by the Ministry of Health?

R.4) Technology costs specifically related to the QIH project, such as computers, website etc. will be assumed by the successful Offeror. However, the technology costs related to implementing what the MOH will assume, such as the patient record system and the quality management system will be assumed by the MOH. The Offer will have to include costs associated with any technical assistance provided to MOH in its proposal.

5) In section C.4 the RFP states “The Offeror should submit a gender integration plan that responds to the recommendations included in the gender analysis.” Can USAID please confirm that the gender integration plan required with the Technical Proposal is not included in the 40-page limitation?

R.5) The gender analysis should be included and integrated in the technical proposal, in the 40-page limit.

6) On page 27, section E.3 “The Contractor will provide a proposed Monitoring and Evaluation Plan as part of the response to this RFP”; and on page 29, section F.7.3 Performance Monitoring, Evaluation & Managing for Results, it is indicated:

“The Contractor/Awardee is required to develop a Performance Monitoring and Evaluation (M&E) Plan in English, in line with USAID/Guatemala’s Mission wide Performance Management Plan (PMP) and the corresponding USAID Project M&E Plan (these documents will be provided by COR), for the complete award period.”

Question: The “USAID/Guatemala’s Mission-wide Performance Management Plan (PMP)” and the corresponding “USAID Project M&E Plan” are not included as attachments to the RFP, can these documents be provided to prospective Contractors in order to propose a Monitoring and Evaluation Plan as part of the response to this RFP that is aligned with USAID Guatemala Mission’s performance management?

R.6) USAID/Guatemala’s Mission-wide Performance Management Plan (PMP) and USAID Project M&E Plan to be used to develop the proposed M&E Plan for this Project are included as Attachments to this Amendment No. 1.

7) In section H.20 on page 151 the RFP states “The contractor shall develop an Environmental Mitigation Plan

(EMP) detailing how the implementing partner will handle medical waste issues, to be submitted to the COTR for approval by the Mission Environmental Officer (MEO) and Regional Environmental Advisor (REA) prior to implementation.” Can USAID please confirm that a separate Environmental Mitigation Plan is not required with the technical proposal, and will be completed at the time of contract award? If the Environmental Mitigation Plan is required with the technical proposal can USAID confirm that it is not included in the 40-page limitation?

R.7) The information required in Section H.20 on page 51 of the RFP refers to the Environmental Mitigation Plan that the resulting contractor will have to submit to the COR of the contract after award is made. For more details of what needs to be submitted in your technical approach, refer to Section L.7 of the RFP.

8) On page 48 H.18. Personnel Compensation, the RFP states that “Nor may any individual salary or wage, without approval of the Cognizant Contracting Officer, exceed the employee’s current salary or wage, or the highest rate of annual salary or wage received during any full year of the immediately preceding 3 years.”

Could USAID please clarify the following: If the Offeror names a current employee, is the Offeror prohibited from giving a salary increase for this proposal? If the position has additional responsibility or is a promotion from their current role, may a salary increase be given in the proposal? If the Offeror is recruiting someone outside the organization for a position, is the Offeror permitted to provide a salary increase above any of the candidate’s salaries in the last 3 years?

R.8) Proposing a current Offerors’ employee in the proposal should not automatically increase the employees’ current salary. Salary increase to a current Offeror’s employee shall be in effect at the anniversary date of the employee. If a position has additional responsibilities, proposed salary shall be reviewed in accordance with market value. If a promotion of a current role/position is proposed, the promotion needs to be justified and in accordance with Offeror’s standard practices. Salary increases above any of the candidate’s salaries in last 3 years shall be reasonable per Offeror’s policies and practice, and in accordance with market value.

9) Appendix J.5 has the guidelines for implementing partners on the USAID LAC Environmental Mitigation Plan and Report (EMPR). During the acquisition phase, the contractor submits a suggested/draft EMPR. Once the implementing partner is chosen, a revised initial EMPR is submitted to the COTR, Mission Environmental Officer and Regional Environmental Officer for approval before beginning activities. At the end of each year of implementation, the EMPR is resubmitted with the same information provided initially, plus a component reflecting the status of implementation and effectiveness. The formats and tables to be completed are in the appendix. Could USAID please clarify if the Offeror is required to submit a draft Environmental Mitigation Plan and Report (EMPR) with the proposal?

R.9) No. For more details of what needs to be submitted in your technical approach, refer to Section L.7 of the RFP.

10) Does USAID/Guatemala require an executive summary for the Quality in Health proposal? If so, is there a page limit envisioned for the executive summary and will it count against the 40 pages allocated to the technical proposal?

R.10) No. According to Section L.7 “Instructions for the Preparation of the Technical Proposal”, it does not require an executive summary to be submitted with the proposal.

11) Could USAID/Guatemala please clarify if the cover page is counted against the technical proposal page limit?

R.11) In Amendment 1 to this solicitation USAID will add the cover page to the list of documents, in Section L.7 that are not counted against the technical proposal page limit of 40 pages.

12) Page 120, L.5, General Instructions to Offerors (3) states that “The RFP Section K Certs/Reps need not be completed, upon electronic completion of ORCA prior to submission of proposals; and the prospective contract.” Please confirm that this includes all parts of Section K that require certification (K.1 through K.9).

R.12) Certifications and Representations will be requested/checked if the Offeror is the presumed successful Offeror. Offerors are responsible for having an updated certification in the ORCA system at the time of submission of their proposal.

13) Delivery Instructions. Please confirm that the electronic files must not exceed the 20 MB limit (per e-mail).

R.13) Correct (page 121).

14) Could USAID/Guatemala please clarify the definition of 12 “pitch” for the font type mentioned on page 122 of the RFP?

R.14) What we mean to say is “Do not use a size of font smaller than 12”. That will be clarified in Amendment 1 to this RFP.

15) On page 122 L.7, A. General d) Past Performance Information/Summary Table and Past Performance Short Forms. Can USAID please clarify what type of past performance information should be captured in the past performance information/summary table vis-à-vis the Past Performance description on page 126 which seems to indicate the requirements of a Past Performance Short form?

R.15) Information required on page 122 L.7 A. (b) d), “Past Performance Information/Summary Table”, shall include the information described in Section L.7 B. 3.(a)2. With regards to the “Past Performance Short Form”, the information is obtained from the CPARS/PPIRS system which applies only to Contractors. There is no past performance system yet that applies to Recipients. Therefore, if an organization has only received assistance awards (Cooperative Agreements and Grants), they will not be required to submit the Past Performance Short Form.

16) For specialized charts in the technical volume and spreadsheets in the cost volume, may we use 11 x 17-inch paper folded to 8½ x 11-inch paper?

R.16) No, per Section L.7 A.(b) of the RFP No. SOL-520-14-000002. Also, see response to Question No.14) above.

17) Please confirm the degree of Medical Doctor and Surgeon (obtained after successful completion of six years of university coursework) in addition to three years of post-graduate practical training at a teaching hospital, meets the education requirements for the COP position outlined on page 125 of the RFP.

R.17) No. A Master’s Degree is required and a Doctoral Degree is desirable per requirements stated in page 125 of the RFP.

18) On page 126, section L7. B4. –Past Performance requires the Offeror to list in an annex to the technical proposal up to five of the most recent and relevant contracts for efforts similar to the work in the subject proposal. Is it acceptable to USAID the submission of up to five grants and cooperative agreements or subcontracts past performances for efforts similar to the work in the subject proposal?

R.18) Section L.7 B. 3. PAST PERFORMANCE. USAID requires submission of up to five contracts of similar type of work, scope of work and complexity of tasks and how recently they were performed. However; if the firm has not had contracting experience recently, submission of the required information based on Cooperative Agreements and Grants will be accepted.

19) Section L.7.3(d)(B) (page 127) instructs Offerors to provide “a list of the last three years contracts for which you submitted subcontract reports to eSRS (FAR 52.219-9(d)(10) and a copy of any similarly recent subcontracting reports if they were not submitted to eSRS.” Since we submit hundreds of reports per year, this could mean submission of hundreds of reports. We respectfully request the option — in addition to providing the list of submitted reports — of selecting 10-15 relevant SF 294 forms for USAID’s review.

R.19) That request is reasonable and acceptable to USAID.

20) Section L.8(b)(1) on page 128 mentions that the detailed budget submitted by the Offeror must reflect the budget propose for each one of the four components of the program. Section L.8(b)(2)(O) on page 131 outlines the distribution of program element funds, and Section F.7 on page 28 mentions that the Contractor will need to track and report the level of funds disaggregated by program element (FP/RH, MCH, Nutrition, and HIV/AIDS) according to the line items included in the general budget. Please clarify whether USAID would like (1) the Offeror to submit a budget broken down by the four components of the project detailed in Section C (similar to contract line item numbers, or CLINs), or (2) by the four funding sources, or (3) whether CLINs will not be required, allowing the Offeror to provide a summary budget in the cost proposal outlining proposed funds disaggregated by funding source and/or program component.

R.20) In accordance with Section L.B.1, USAID requires the submission of the following budgets:

1. A summary budget for the contract period.

2. A detailed version of its summary budget for the contract period. The detailed budget must reflect every estimated cost item, broken out. The detailed budget must reflect these individual cost items for each of the contract years and then offer the program total for the entire contract, item for item in the last column of the spread sheet.

3. A budget reflecting each one of the four components of the program.

4. A second summary budget including the following line items, which will be included in Section B.4 entitled “Line Items” of the resulting Contract:

Direct Labor $____________ Other Direct Costs $____________ Indirect Costs $ ___________ Fixed Fee $ ___________

5. Total Estimated Cost plus Fixed Fee $____________ The information provided in Section L.8.(b) (2) (O) entitled “Distribution of the Program Element funds for the QIH Project” is provided as information for the resulting contractor to track the expenditures of the resulting contract by source of funds. USAID does not require the Offerors to submit a budget based on source of funds.

21) L.8 (A) states that biodatas should be presented for all proposed staff while I.25 states that biodatas should be provided for local long-term staff that are designated as “key personnel” and Expatriates/TCNs. Please confirm that biodatas should be presented only for expatriates/TCNs and local long-term staff that are designated as “key personnel” per I.25.

R.21) Even though AIDAR 752.7001 requires that biodata forms be submitted only for (1) all individuals to be sent outside of the United States, or (2) any employees designated as "key personnel, this RFP requires Biographical Data Sheets to be submitted for all long-term and short-term personnel.

22) On the Salaries & Wages (page 128), USAID requests Offeror(s) to include a certification as to whether the individual is a full-time, intermittent employee, or a consultant. Would USAID elaborate more on this particular requirement?

R.22) This certification is required to verify the level of effort for proposed staff.

23) Please confirm that the list included (A-H, page 128) for the different line items are of illustrative nature and that Offeror(s) may use their own templates that reflect their cost structure.

R.23) According to Section L.8 (b) (2) the line items provided in the RFP are the required line items to be included in the cost proposal.

24) On page 129 L.8. b) Part 2 Proposed Costs/prices, the RFP mentions that CCNs and TCNs that are hired under USAID direct contracts, generally be extended the same benefits, and be subject to the same restrictions as TCNs and CCNs employed as direct hires by USAID Mission. Can USAID clarify if the Offeror is required to modify its current established country benefits to mirror USAID’s benefit package? Or may the Offeror maintain the continuity of its established country benefits package?

R.24) The paragraph in question refers to AIDAR 722.170(a) which applies to CCNs and TCNs who are hired abroad for work in a cooperating country under USAID-direct contracts. If the Offeror proposes this type of personnel under this solicitation, they should be extended the same benefits, and be subject to the same restrictions as TCN’s and CCN’s employed as direct hires by the USAID Mission unless a waiver to this policy is approved by the USAID Mission Director. If the Offeror proposes to hire TCNs and CCNs locally, the Offeror will not be required to modify their established country benefits package to mirror USAID’s benefit package.

25) Procurement. Offerors are requested to provide the manufacturer for the equipment. At the pre-award stage, Offerors may not know with certainty this type of information. Would it suffice to include the usual breakdown of the non-expendable and expendable list?

R.25) Goods and services provided by the Offerors under this award shall be within the authorized Geographic Code 937. Offerors shall make its best effort in identifying up front the proposed non-expendable property and all equipment to be purchased under this contract, including the manufacturer if known at the submission of proposal. However, if this is not known at this stage, Offerors shall include the type of equipment, the unit cost, and the number of units to be purchased.

26) Page 132, Section L(d), Part 4 – Policies and Procedures. If the Offeror does not have prior USAID contracting experience, but does have prior USAID subcontracting experience, is it required that the listed policies and procedures be submitted?

R.26) If it is a U.S. Organization, the Offeror shall submit its approved NICRA and confirm that their policies and procedures have been accepted by a U.S. Government Agency. If they do not have an approved NICRA and their policies and procedures have not been approved by a U.S. Government Agency, then the Offeror shall submit its policies and procedures along with its proposal. In the case of a local organization, the Offeror shall submit its policies and procedures along with its proposal.

27) Can USAID please provide Offerors with a copy of the most recent USAID Partner Information Form 500-

13 or clarify that the version that is available online that expired 8/31/2011 is still valid?

R.27) Please note that we are not requiring the Partner Information Form No. AID 500-13 at this stage. It will be required at the competitive range stage.

28) Can USAID please explain if any equipment, furniture, and vehicles will be available from past projects for use on this QIH project?

R.28) No equipment furniture or vehicles are expected to be available from past projects for use on the QIH project.

END OF QUESTIONS AND ANSWERS

May, 2014

USAID/GUATEMALA

PERFORMANCE MANAGEMENT PLAN

FY 2012 – 2016

ACRONYMS

CDCS Country Development Cooperation Strategy

CLA Collaborating, Learning and Adapting

DO Development Objective

ENCOVI National Survey of Conditions of Life (acronym is in Spanish)

ENSMI National Maternal Child Health Survey (acronym is in Spanish)

FFP Food for Peace

FTF Feed the Future Initiative

FY Fiscal Year

GCC Global Climate Change

GDP Gross Domestic Product

GHG Greenhouse Gas

GHI Global Health Initiative Plus

GOG Government of Guatemala

HIV/AIDS Human Immunodeficiency Virus/ Acquired Immune Deficiency Syndrome

IM Implementing Mechanism

IP Implementing Partner

IR Intermediate Result

LEDS Low Emission Development Strategy

M&E Monitoring and Evaluation

MO Mission Order

MOE Ministry of Education

MOH Ministry of Health

PEPFAR U.S. President's Emergency Plan for AIDS Relief

POC Point of Contact

PPSO Planning and Program Support Office

SJSI Security and Justice Sector Institution

Sub-IR Sub-Intermediate Result

USAID United States Agency for International Development

USG United States Government

SECTION 1 – INTRODUCTION

A. USAID Performance Management Plan

This Performance Management Plan (PMP) is the tool USAID/Guatemala uses to assess progress toward achievement of Country Development Cooperation Strategy (CDCS) results.

This PMP provides the empirical grounding and organizational structure to determine the

Mission's progress towards achieving the ambitious goals in its 2012-2016 CDCS. Building upon the Collaborating, Learning and Adapting (CLA) principles laid out in the CDCS, the PMP outlines how these approaches are linked to the performance management practices and processes of USAID/Guatemala and its partners. As per ADS 203, the PMP establishes performance management approaches, roles and responsibilities in three main areas:

Learning: Identify and address gaps in the evidence base through collaborative analysis of available information and when necessary, conducting special studies, assessments and surveys.

Evaluation: Plan, design and conduct program evaluations per USAID requirements as well as when necessary to inform programmatic decisions.

Monitoring: ensure and refine routine data collection, analysis and review.

The PMP was developed with input and contributions from all offices and coordinated by the

M&E POC in the Program Office. Through a series of facilitated meetings, the Mission has identified a list of performance indicators and evaluation questions associated with each Project.

There are three major occasions during which Mission staff (DO teams, Program Office staff, and others) will be engaged in developing, updating, and revising the Mission-wide PMP:

1. After the CDCS is approved, including alignment of the Mission’s existing portfolio;

2. During Project Design and project implementation; and,

3. Following Mission Portfolio Reviews.

During CDCS implementation there are other critical opportunities to update and revise the

PMP, including but not limited to: as data is collected and analyzed (e.g., which might suggest a need to revise targets); as evaluation findings are operationalized (e.g., which might trigger discussion on whether and how to change the Project LogFrame or DO Results Framework);

based on management decisions (e.g., budget requirements); and as new learning occurs. At a minimum, the PMP will be reviewed and updated semi-annually, as a result of the Mission’s portfolio review and reviews of projects’ Monitoring and Evaluation Plans. All Projects teams conduct ongoing reviews of Project Monitoring and Evaluation (M&E) plans to ensure they align with the Mission’s strategy. Some performance indicators and their targets may be adjusted

ADS 203.3.3: A PMP is a mission-wide tool to plan and manage the process of monitoring, evaluating, and analyzing progress toward achieving all results identified in a Country Development Cooperation Strategy (CDCS) and project Logical Framework (LogFrame) in order to inform decision-making, resource allocation, learning, and adapting projects and programs.

through a formal process outlined in the Mission Order on Performance Monitoring as projects are designed and implemented. The development of some indicators requires specialized technical assistance to be completed.

The PMP should be considered a living document to guide activity design and implementation, and is adjustable to align with future decisions and changes in the operating environment. This document includes Development Objectives’ (DOs) and Intermediate Results (IRs) outcome and output indicators (ultimate goal progress vs. monthly/yearly progress), the latter to be reported on by the Mission’s implementing partners and serve as the starting point for more in-depth project and activity-level M&E plan to be developed in conjunction with partners.

The USAID/Guatemala Results Frameworks for each DO are shown below. For additional information on the Results Frameworks, associated development hypotheses or other elements of the framework, please review the USAID/Guatemala CDCS 2012-2016.

B. Results Framework

Figure 1 is the results framework for USAID/Guatemala’s CDCS, 2012 – 2016.

The three DOs and their related IRs and sub-IRs were determined based on extensive consultations within the USAID Guatemala Mission, among other USG agencies in Guatemala, with various GOG entities, donors, partners, academics, civil society, private sector, and indigenous leaders. As such, the final product represents the consolidation and integration of the most pressing development priorities in Guatemala – all of which move the country towards the

CDCS Goal. Taken separately, each Development Objective leads to partial achievement of the goal, but without all three Development Objectives, the Goal cannot be fully realized and sustained.

The results framework for each DO is further elaborated in Section 2 below.

Figure 1: CDCS Results Framework, 2012 - 2016

GOAL: A More Secure Guatemala that Fosters Greater Socio-economic Development in the Western Highlands and

Sustainably Manages its Natural Resources

DO1: Greater Security and

Justice for Citizens

DO2: Improved Levels of

Economic Growth and Social

Development in the Western

Highlands

DO3: Improved Management of Natural Resources to

Mitigate Impacts of Global

Climate Change

IR1: Improved effectiveness and efficiency of security and justice sector institutions (SJSIs)

IR2: Reduced levels of violence in targeted communities at risk

IR1: Broad based economic growth and food security improved

IR2: Access to and use of sustainable quality health care and nutrition services expanded

IR3: Education quality and access improved

IR1: Market-driven conservation and management strategies implemented

IR2: Vulnerability to the effects of global climate change reduced

IR3: Environmental governance strengthened

SECTION 2- DO - RESULTS FRAMEWORKS

DO1: Greater Security and Justice for Citizens

A. DO1 Development Hypothesis

“Increasing the demand for police and justice reform, strengthening national level government capacities to include administrative, financial and strategic planning, institutionalizing crime prevention strategies, targeting pilot projects in key areas with high homicide rates, strengthening local governments, and improving transparency and accountability of key institutions will lead to a reduction in crime and an increase in the number of prosecutions with final verdicts, thus breaking the vicious cycle of impunity.”

B. DO1: Performance Indicators

Context Indicator: Homicide Rate in Target Areas

Homicide Rate National Level

Performance Indicators:

Degree of Citizen Security in Targeted Areas (DO 1)

Citizen Satisfaction with Service Delivery by Target SJSIs (IR 1.1)

Clearance Rate of Target Pluripersonal Courts (IR 1.1)

Crime Victimization Rates in Target Communities (IR 1.2)

Degree of Community Resilience in Target Areas (IR 1.3)

Planned and Actual Values are listed in Project Performance Reporting Table ANNEX 1

C. Evaluation questions for DO1

In accordance with USAID’s evaluation policy to integrate evaluation concerns into project and program designs, DO 1 plans to address the following indicative key performance and impact evaluation questions.

1. Did the establishment of a career path for police improve recruiting and retention of a skilled workforce?

2. Which approaches to address at-risk youth and gang prevention were effective and why?

3. How do the civic responsibility interventions change the target population’s culture of lawfulness?

4. (Potential Impact Evaluation Question) Did Security Sector Reform project interventions increase convictions and otherwise improve the outcomes of criminal cases? If so, what interventions of the project created the impact?

D. Performance and Impact Evaluation Plan

The list below will inform any planned impact assessment/review, and will also be consulted before project/activity design exercises and in planning new evaluations.

DO1 Evaluation Plan Summary

Evaluation Title/Questions

Point of

Contact

Project/ activity/ program (P/A/P) to be evaluated

P/A/P

Start/

End Dates

Required(and reason required) or

Optional

Evaluation

Type

(performance or impact) and

Projected Use

Internal or external

Estimated

Evaluation budget

Eval.

Start/ End

Dates

Local Governance Project (LG) Evaluation Andrew

Lucas

Local

Governance

(LG)

Sep, 2014/

Aug. 2019

Learning and

Management

Purposes

Mid-term performance learning and management purposes

External $350,000 Baseline:

Oct 2014

– Dec

Mid-term:

Mar 2016

- Jun

Security and Justice Sector Reform Project

(SJSRP) Evaluation

Cassandra

Wheeler

SJRSP Aug, 2013/ Jun, (base p.)

Learning and

Management

Purposes

Mid-term performance – to obtain inputs for decision making process to exercise option period

External $350,000 Mid-term:

Mar, 2015/

May.

Violence Prevention Project (VPP) Impact

Evaluation

Vanessa

Reilly

VPP Mar 2010

- Sep

Optional -

Learning and

Management

Purposes

Impact

Evaluation

/Baseline

External $2,000,000

Baseline:

Jan 2010 -

Mar 2010

Mid-term:

Dec 2012

– Feb

Final:

Mar 2014

- May

Community Strengthening Project (CSP)

Evaluation

Carlos

Rosales

CSP Jul 2014 –

Jul 2016

(Base

Period)

Required under USAID

Evaluation

Policy

Impact/ to measure change in development outcome.

Baseline, Mid-term and Final

Evaluation

(Projected)

External $2,000,000 Baseline:

Oct 2014 -

Dec 2014

Mid-term:

Apr 2016

- Jun 2016

This impact evaluation is managed by LAC Bureau , USAID/Washington

VPP/CARSI impact evaluation is managed by USAID/Washington. No cost to USAID/Guatemala involved.

Figure 2: DO 1 Results Framework

Development Objective 1:

Greater Security and Justice for Citizens

IR 1.2: Reduced Levels of Violence in Targeted Communities at Risk

Indicators:

Sub-IR 1.2.2: Trust between police and community in target areas improved

Sub-IR 1.2.3: Civic responsibility increased

Sub-IR 1.2.1: Vulnerability of at-risk communities to gangs and DTOs decreased

IR 1.1: Improved effectiveness and efficiency of security and justice sector institutions (SJSIs) Indicators:

Sub-IR 1.1.1: Strengthened prosecution and adjudication of crime

Sub-IR 1.1.2: Improved management, administration, and coordination among SJSIs

Sub-IR 1.1.3: Strengthen accountability and transparency of key SJSIs

DO 2: Improved Levels of Economic Growth and Social Development in the Western

Highlands

A. Development Hypothesis

“Parallel improvements in the income, health and nutrition, and education of

Guatemala’s rural poor will foster the sustainable rural development necessary for social and economic equity. Improved income alone is insufficient to reduce poverty, illness and malnutrition in Guatemala’s rural poor communities; therefore, USAID will work with the GOG in the Western Highlands to improve economic opportunities and the access and utilization of quality health, nutrition, and education services.”

B. DO 2: Performance Indicators

Context Indicators:

Human Development Index

National Prevalence of Extreme Poverty

National Prevalence of Stunting in Children under Five

Performance Indicators:

Prevalence of Poverty (percentage of people living on less than Q14.41 /day) in Target

Regions (DO 2) – by gendered household type

Prevalence of Stunted Children Under 5 Years of Age in Target Regions (DO 2)

Total Fertility Rate in Target Regions (DO 2)

Under 5 Mortality Rate in Target Regions (DO 2)

Failure Rate in First Grade in Target Regions (DO 2)

Per Capita Expenditures (as a proxy for income) of USG-targeted Beneficiaries (IR 2.1)

Prevalence of Underweight Children Under 5 Years of Age in Target Regions (IR 2.2)

Modern Method Contraceptive Prevalence Rate in Target Regions (IR 2.2)

Percent of Births Attended by a Skilled Doctor, Nurse or Midwife in Target Regions (IR

2.2)

Proportion of students, who, by the end of two grades of primary schooling, demonstrate that they can read and understand the meaning of grade level text, in target regions

(Sub-IR 2.3.1)

Number of Learners Enrolled in Secondary Schools or Equivalent Non-School Based

Settings with USG support (IR 2.3)

Planned and Actual Values are listed in Project Performance Reporting Table ANNEX 1

C. DO2 Evaluation Questions

In accordance with USAID’s evaluation policy to integrate evaluation concerns into project and program designs, DO 2 plans to address the following indicative key performance and impact evaluation questions.

1. What are the effects of mobile clinics on the unmet demand for family planning services?

2. To what extent has community-based management of pneumonia and diarrheal diseases affected chronic malnutrition levels?

3. Does improved reading in early grades impact overall student performance at the Primary

Level?

4. Is there a secondary effect on out-of-school youth returning to the classroom?

5. What are the elements that need to be incorporated into income generating activities to achieve a reduction of malnutrition rates in targeted communities?

6. Does the combination of income generation and nutritional behavior change interventions contribute to reduction of poverty and malnutrition? At what level: beneficiary households, communities and/or municipalities?

7. (Potential Impact Evaluation Question) Do interventions of improved productivity and rural employment expansion combined with access to markets (value chains) have an impact on poverty reduction of target households? Does this impact spill over to the community?

The list below will inform any planned impact assessment/review, and will also be consulted before project/activity design exercises and in planning specific project new evaluations.

DO2 Evaluation Plan Summary

POCs Project/ activity/ program to be evaluated

P/A/P

Start/

End Dates

Required or Optional

Evaluation

Type

(performance or impact), and

Projected Use

Estimated

Evaluation budget

Eval. Start/

End Dates

Western Highlands Integrated

Program Evaluation (IR1 and

IR2)

Tom DiVincenzo Provide updated health sector data and statistics, especially related to Feed the Future and the Global

Health

Initiative.

Apr 2012 -

Aug 2013

Required under

USAID

Evaluation

Policy

Impact

Evaluation with counterfactual comparison.

Baseline, Mid-term and Final

Evaluation .

$3,110,000 Baseline:

Aug 2013 –

Feb 2014

Mid-term:

Aug 2015 –

Feb 2016

Final:

Aug 2016 –

Feb 2017

Alianzas II Final Evaluation Angel Lopez

Alinazas II

Project

Optional.

Learning and

Management

Purposes

Final

Evaluation to obtain best practices and lessons learned.

$300,000 Final:

March, 2014/

Sept. 2014

Figure 3: DO 2 Results Framework

IR 2.3: Education quality and access improved

Sub IR 2.3.2 Access to quality non-traditional education programs increased

Sub IR 2.3.3 GOG institutions and systems strengthened

Sub IR 2.3.1 Reading skills improved

IR 2.2: Access to and use of sustainable quality health care and nutrition services expanded

Sub-IR 2.2.1: Adoption of improved health practices expanded

Sub-IR 2.2.2: Availability of sustainable and culturally-adapted health care and nutrition services increased

Sub-IR 2.2.3: Quality of sustainable health care and nutrition services improved

Development Objective 2:

Improved Levels of Economic Growth and Social Development in the Western

Highlands

IR 2.1: Broad based economic growth and food security improved

Sub-IR 2.1.1: Agricultural productivity and rural employment expanded

Sub-IR 2.1.2: Access to markets expanded

Sub-IR 2.1.3: Resiliency of vulnerable communities and households increased

DO 3: Improved Management of Natural Resources to Mitigate Impacts of Global Climate

Change (GCC)

A. Development Hypothesis

“Community-based economic incentives for conservation and protection of biodiversity, coupled with improved enforcement of existing environmental regulations and improved national policies, will safeguard the resource base necessary for sustainable rural development and thus contribute to Guatemala’s increased prosperity. Concurrent reduction of vulnerability to natural disasters will help mitigate loss of life and livelihoods due to climate change related events.”

B. DO 3: Performance Indicators

Context Indicators:

Proportion of land area covered by forest

Greenhouse Gas Emissions Inventory

Climate Change Adaptation Index

Performance Indicators:

Number of Hectares in Areas of Biological Significance and/or Natural Resources

Showing Improved Biophysical Conditions as a Result of USG Assistance (DO 3)

Quantity of Greenhouse Gas Emissions, Measured in Metric Tons of Carbon Dioxide

Equivalent (CO2e), Reduced or Sequestered as a Result of USG Assistance

Value of Incremental Sales of Timber and Non-timber Forest Products Attributed to USG

Assistance (IR 3.1)

Number of Stakeholders with Increased Capacity to Adapt to the Impacts of Climate

Variability and Change as a Result of USG Assistance (IR 3.2) – by gender

Number of Stakeholders Using Licenses or Permits to Legally Manage and Use the

Forest (IR 3.3)

Planned and Actual Values are listed in Project Performance Reporting Table ANNEX 1

C. DO 3 Evaluation Questions

In accordance with USAID’s evaluation policy to integrate evaluation concerns into project and program designs, DO 3 plans to address the following key performance evaluation questions:

1. Does USAID support for sustainable forestry management effectively maintain and/or improve forest cover?

2. Does USAID support for sustainable forestry management increase incomes?

3. Do USAID adaptation interventions increase the capacity of communities (institutions, municipalities?) to respond to climate change impacts and reduce overall vulnerability to climate change?

4. Are local NGOs strengthened by USAID interventions to the point of being able to directly receive funds from USAID or other donors?

5. How did CNCG perform against its management targets for results and budget?

6. What are particular environmental sector problems that must be addressed by CNCG or future projects in order to meet DO3 objectives?

The list below will inform any planned impact assessment/review, and will also be consulted before project/activity design exercises and in planning specific project new evaluations.

DO3 Evaluation Plan Summary

Point of

Contact

Project/ activity/ program

(P/A/P) to be evaluated

P/A/P

Start/

End Dates

Required

(and reason required) or

Optional

Evaluation Type

(performance or impact) and

Projected Use

Internal or external

Estimated

Evaluatio n budget

Eval. Start/

End Dates

CNCG Performance Evaluation Angel Lopez

(POC)

Climate Nature and

Communities in

Guatemala -

CNCG-

February, 2013/

February, Required under USAID

Evaluation

Policy.

Large Project

Performance

Evaluation.

Baseline, Mid-term and Final

External $350,000 Baseline:

Oct 2013 –

May 2014

Mid-term:

Apr 2015 –

Aug 2015

Final:

Apr 2017 –

Aug 2017

Figure 4: DO 3 Results Framework

Development Objective 3:

Improved Management of Natural Resources to

Mitigate Impacts of Global Climate Change

Sub-IR 3.1.2: National and/or sub-national forest and land use strategies adopted

IR 3.1: Market-driven conservation and management strategies implemented

IR 3.2: Vulnerability to the effects of Global Climate

Change reduced

IR 3.3: Environmental governance strengthened

Sub-IR 3.1.1: Market for community-based sustainable forest products promoted

Sub IR 3.2.1 Improved technologies to mitigate the effects of climate change adopted

Sub-IR 3.3.1: Local institutions strengthened to improve environmental management, protection and enforcement of regulations

Sub IR 3.2.2 Local institutions with access to risk identification and mitigation tools

SECTION 3 – MONITORING AND EVALUATION

APPROACH

A. Monitoring

For USAID/Guatemala it is a priority to proactively engage in assessing and evaluating performance of CDCS implementation in accordance with USAID’s Evaluation Policy, ADS

203, and the MOs on Monitoring and Evaluation.

Each Project Team is responsible for developing and regularly updating a plan to verify monitoring data as part of a Project’s M&E Plan in order to ensure effective management of their portfolio. These plans include the schedule and responsible person for the following activities:

Data collection

Site Visits

Partner Meetings

Data Analysis and Review

Completed M&E plans, for each Project Team, are stored in the “P” drive and will be updated as needed.

At the mission PMP-level, the procedures outlined below will improve and enhance the quality and reliability of information collected during project implementation and intends to support

USAID/Guatemala and local partners in the process of decision making and learning:

Selection & Review of Performance Indicators

Collection of Performance Information

Reporting of Project Performance

Analysis of Performance Data and Other Information

Utilization of Findings

Selection & Review of Indicators:

Performance and Context Indicators:

A performance indicator measures a particular characteristic or dimension of strategy, project, or activity-level results based on a Mission’s CDCS Results Framework or a project’s Logical

Framework (LogFrame). Performance indicators are the basis for observing progress and measuring actual results compared to expected results. Performance indicators help answer the extent to which USAID is progressing towards its objective(s), but alone cannot tell the manager why such progress is or is not being made. Selection of indicators to be included in the PMP and

Project M&E plans should be a participatory process that follows USAID guidelines, best practices as well as the standards established by the MO on Performance Monitoring.

During the CDCS development process, Mission staff identified critical assumptions and assessed the risks associated (either positively or negatively) with successful achievements.

While not all assumptions and risks require context indicators in order to track them, this logic from the CDCS and LogFrames should be the basis for selection of context indicators as required by ADS 203. Context indicators measure conditions relevant to the performance of the

CDCS and projects, such as macroeconomic, social, and political conditions. However, they do not directly measure results, but rather the factors that are beyond the management control of the

Mission and its partners.

Target Setting and Review:

One of the challenges with the PMP is to set targets. The premise with targets is that these are initially grounded in the baseline values and/or other information, and then determined based on the level of effort attached. At the same time, the context and the pace of change can determine how closely targets are achieved. Hence, targets do require an annual review based on the results from the previous year’s reporting period; targets can be adjusted upwards or downwards much more easily than other elements in a project. Note that this is not an action that should be taken unilaterally, but only after discussion and consultation (see MO on Performance Monitoring).

Targets that are now much too high (because of changes in the operating environment) should be reduced for out-years, and those changes documented clearly.

1. Each COR/AOR should (and already do) review their portfolio with an eye towards how, and how well, different targets have been achieved by the implementing partners (IPs). In addition, consultations with the IPs on revising targets can accompany feedback on the annual report and thus provide more guidance about subsequent year’s annual workplans, as well as modifications to the IPs’ M&E plans. This target review can be done by the different

IPs reporting to one COR/AOR, or in a wider setting such as all IPs under individual intermediate results or even at the Project level.

2. At the Mission level, targets based on Mission-level data collection processes should be reviewed initially upon completion of each of these different data collection mechanisms, and notes made about potential changes (as well a rationale for the changes). These changes should be presented during the Annual Portfolio Review so that modifications can be made to the Mission PMP in a consolidated manner.

Assessing Data Quality:

The purpose of a Data Quality Assessment is to ensure that the USAID staff are aware of the strengths and weaknesses of the data they obtain about project and activity performance, as determined by reviewing actual data on indicators against USAID’s criteria for quality: validity, integrity, precision, reliability and timeliness. CORs are therefore made aware of the extent to which the data can be trusted as a basis for management decisions.

A critical, and often overlooked, component in the PIRS is the timing and outcome of data quality assessments (DQAs). USAID/Guatemala will adhere to ADS 203 requirements on

DQAs, and update them throughout the year as noted in performance reference sheets (PIRS).

DQAs must be conducted within six months of reporting data to Washington for new indicators.

Collection of Performance Information:

There are several major sources of data for the Mission, for example third party evaluations, IP performance management reporting, Mission-funded surveys, other third party information (e.g., MOH/ENSMI), and direct sources (e.g., Communications). In addition, different topics, sectors, or IPs can also produce their own internal evaluations (AGEXPORT and ANACAFE). This creates a necessity for a more organized and systematic approach to data collection to ensure data quality is maintained.

Making that information both more systematic and organized may help Project-teams use the information more effectively in management decisions. At the same time, this will also help the

IPs use their own monitoring and evaluation systems more effectively, and allow them to access higher-level information (such as evaluations).

In order to move forward, there are a few processes that the Mission will adopt to ensure a more integrated approach to data collection and improve data quality:

1. Clear guidance as to the components of an M&E plan (narrative, indicators, targets, PIRs;

updated as much as possible) should be provided by the Project Teams so that the IPs will submit this information in a consistent and accessible format.

2. Harmonization of indicators is designed to ensure that similar indicators are collected in roughly similar fashions to allow for aggregation where possible and/or a time or cost savings to USAID/Guatemala. For some portfolios, different IPs may report on indicators that are almost, but not quite, identical. Recognizing which indicators are similar can help ensure that these indicators can align where possible.