Attachment J.12 Final OVC_MIS Assessment Report_1_May 2015.pdf

THE REPUBLIC OF UGANDA

Ministry of Gender, Labour and Social Development

ASSESSMENT OF THE UGANDA NATIONAL

ORPHANS AND OTHER VULNERABLE CHILDREN

MANAGEMENT INFORMATION SYSTEM (OVCMIS)

March 2015 i

FOREWORD

The Ministry of Gender Labour and Social Development values the power data/information in programme planning and decision making. The Orphans and other Vulnerable Children Management Information

System (OVCMIS) presents an opportunity to have data/information for planning and decision making for issues that affect vulnerable children. That is why in 2005, the Ministry with the support of the USAID-funded CORE Initiatives mooted the idea of putting in place the OVCMIS.

The OVCMIS prototype was then developed between 2007 and 2009, and over the years, the system has been operationalized and rolled out to all the 112 districts, with more than 90% of the districts regularly submitting data on a quarterly basis.

This OVCMIS assessment is a timely and welcome initiative and is about what has worked well, what has not worked and what needs to be improved on. The Ministry therefore extends sincere thanks to USAID for funding this exercise and the team of consultants under the leadership of Mr. Xoli Mahlalela for carrying out this exercise and putting together this report. The Ministry also appreciates the OVCNIU, USAID, UNICEF, CDC, SUNRISE/TSOs, Mildmay Uganda, district local governments, CSOs and others who have invested their funds and energies in strengthening the capacity of OVCMIS at National and district levels.

My Ministry is committed to having all the aspects of the recommendations in this report implemented in order to improve the current version of the OVCMIS, and bring it to a world-class standard. The Ministry will continue lobbying the government of Uganda and development partners to commit more funds and technical support for children activities including strengthening OVCMIS and supporting districts to respond to children issues.

I therefore call upon all stakeholders to read and internalize this report and act on specific areas assigned to them.

For God and My Country

PIUS BIGIRIMANA

Permanent Secretary ii

ACKNOWLEDGEMENT

On behalf of the independent OVCMIS Assessment Team, I wish to extend our profound gratitude to all the actors who have supported us in the last three months as we undertook the assignment to do a rapid assessment of the functionality of the OVCMIS.

The assignment would not have been possible without the immeasurable support from the Ministry of

Gender Labour and Social Development, the OVCNIU, the United States Agency for International

Development (USAID), OVC M&E TWG as well as USAID projects MEEP, ASSIST, and SUNRISE. We also acknowledge the immense value of contributions and support from UNICEF as well as other OVC implementing partners such as CHILD FUND, Save the Children, World Vision, to name a few.

We appreciate the support received from district, sub-county and CSO staff during field visits to the districts to learn more about OVC activities and assess the OVCMIS.

We hope that this report meets your expectations and will prove useful in guiding the improved implementation of the OVCMIS at all levels.

We pledge our commitment and availability for any further support you may need in your pursuit of a fully functional OVCMIS.

Yours truly, XOLI MAHLALELA

M&E Specialist and Team Leader: OVCMIS Assessment Team

FIFI MANUEL - M&E Specialist

JOSEPH KIGGUNDU - MIS Expert

ENGINEER BAINOMUGISHA - IT Expert iii

TABLE OF CONTENTS

ABBREVIATIONS AND ACRONYMS v, vi

EXECUTIVE SUMMARY 1

A. INTRODUCTION 3

B. METHODOLOGY 8

C. FINDINGS 12

1. OVC MIS Governance and Human Resources 13

2. Data Management, Reporting and Utilization 18

3. Information, Communication Technology 24

4. OVC M&E Data Needs 28

5. Technical Review of OVCMIS 30

D. GENERAL RECOMMENDATIONS 35

E. OVCMIS IMPROVEMENT PLAN 38

F. APPENDICES 56

G. REFERENCES 143

iv

ABBREVIATIONS AND ACRONYMS

CAO Chief Administrative Officer

CBSD Community Based Services Department

CDO Community Development Officer

CORE Community Responses to HIV/AIDS

CPAs Core Program Areas

CIS Community Information System

CSO Civil Society Organization

EMIS Education Management Information System

DCDO District Community Development Officer

DHIS District Health Information System

DOVCC District OVC Coordination Committee

DQA Data Quality Assessment

DSC District Services Commission

FBO Faith Based Organization

GIS Geographic Information System

HMIS Health Management Information System

ICT Information and Communications Technology

IFMS Integrated Financial Management System

IPPS Integrated Personnel and Payroll System

IT Information Technology

KCCA Kampala Capital City Authority

LG Local Government

LoGIS Local Government Information System

LQAS Lot Quality Assurance Sampling

MDA Ministry Department and Agency

MEEPP Monitoring & Evaluation Emergency Plan Progress

MGLSD Ministry of Gender, Labour and Social Development

MoFPED Ministry of Finance, Planning, and Economic Development

MOVCC Municipal OVC Committee

MoICT Ministry of Information and Communications Technology

MoLG Ministry of Local Government

MoPS Ministry of Public Service

M&E Monitoring and Evaluation

NCC National Council for Children

NGO Non-Government Organization

NITA-U National Information Technology Authority Uganda

NIMES National Integrated Monitoring and Evaluation Strategy v

NOP National OVC Policy

NOSC National OVC Steering Committee

NSPPI National Strategic Program Plan of Intervention for OVC

OVC Orphans and other Vulnerable Children

OVCMIS OVC Management Information System

PEPFAR Presidential Emergency Plan for AIDS Relief

PSWO Probation and Social Welfare Officer

RDQA Routine Data Quality Assessment

REPSSI Regional Psychosocial Support Initiative

SDS Strengthening Decentralised Systems

SMS Short Message Service

SOP Standard Operating Procedures

SOVCC Sub County OVC Coordination Committee

SUNRISE Strengthening Uganda’s Response for Implementation of Service for OVC

TOR Terms of Reference

TSO Technical Service Organisation

TWG Technical Working Group

UBOS Uganda Bureau of Statistics

UDHS Uganda Demographic and Health Survey

UNHS Uganda National Household Survey

UNICEF United Nations Children's Fund

USAID United States Agency for International Development vi

EXECUTIVE SUMMARY

The Management Information System for Orphans and Other Vulnerable Children (OVCMIS) was designed as a central hub for OVC related data in Uganda, and developed mainly for OVC program planners and implementers both within and outside government. It is freely accessible to public users via the MGLSD website www.ovcmis.mglsd.go.ug

It was first conceptualised in 2007, and the current version has been rolled out to all 112 districts and extended to reporting at sub-county level. The system allows for the capturing of quarterly data (i.e. every 3 months). Despite the progressive improvement in districts capturing data on the

OVCMIS, the system still faces some challenges. It is against this background that MGLSD with support from USAID commissioned a detailed assessment of the OVCMIS to be conducted, validate the challenges identified, identify other challenges, and then come up with recommended workable and cost effective strategies to address the challenges and improve the overall functioning of the OVCMIS.

Over a period of just over eight weeks, a team of four consultants conducted interviews with key stakeholders, reviewed numerous documents, and conducted field visits to all eight regions and selected districts around the country to collect and validate information on the OVCMIS.

The critical strengths, weaknesses, opportunities and threats identified during this assessment are summarized in the table below.

SWOT ANALYSIS

STRENGTHS

Good M&E Framework and Plan with data-collection tools developed and in use;

Highly committed OVCMIS staff, especially the M&E

Officer;

Significant reporting rate;

Development partners see OVCMIS as the ‘go-to’ system for OVC related data.

Existence of sufficient bandwidth (4Mbps upload/download) at MGLSD that is used for access and provision of other services.

Capability of in-house hosting of information systems at MGLSD.

National electricity grid, with some districts using generators and solar power.

The advantages of having in place a web system, if properly managed data security/backup is assured.

WEAKNESSES

The system currently does not meet all expectations, provides limited feedback, and inbuilt reporting capabilities are limited;

No IT governance structure at the center for

OVCMIS;

System dependent on donor-funds – not sustainable;

Lack of structured training plan for OVCMIS key users at the districts;

Inadequate IT equipment at the center for in-house hosting of the OVCMIS.

Lack of customizable reports and dashboards.

No clear backup plans for OVCMIS data;

Presence of test data and incidences of exaggerated percentages/numbers

Weak security and user permissions.

http://www.ovcmis.mglsd.go.ug/

OPPORTUNITIES

Ongoing initiatives by NITA-U to harmonize different information systems & reduce data duplication.

Interest by different stakeholders that have own information systems to interface their systems with

OVCMIS to provide or extract data

Increasing Internet coverage and infrastructure e.g., ongoing implementation of the National Data

Transmission Backbone Infrastructure (NBI) and e-

Government Infrastructure (EGI).

Increasing use and availability of mobile technologies e.g., tablets, smartphones, and SMS.

Most CSOs have adequate functional computers, reliable Internet access and receive training support from bigger organizations.

THREATS

OVC MIS is not enlisted among priority government systems by NITA-U hence not able to attract government funding;

Poor funding of OVCMIS, including salaries for staff;

Non-adherence and implementation of the M&E Plan;

Unreliable internet services in districts;

Lack of financial capacity by districts to pay for internet services & to provide support supervision to sub-counties;

No dedicated budget line for IT at the center;

Potential for loss of dedicated staff at national level due to poor funding of salaries and uncertainty;

A major concern by the Assessment team was that a number of the recommendations made in this report had already been made in February 2012 in the OVC M&E Framework and Plan, but little had been implemented since then, mainly due to insufficient funding and skilled human resources to accomplish the task.

As the key recommendation, the assessment team proposes an urgent high-level summit on the

OVC Programme and OVCMIS to be convened. MGLSD has to seriously show leadership of the programme, and high-level political stakeholder buy-in from the Office of the Prime Minister or even Presidency is needed. The summit would serve to validate and endorse urgent recommendations that can be actionable within the short term of six to twelve months. Technical support would need to be provided to MGLSD during this period. The intermediate to long term recommendations would then be implemented during the period of the new NSPPI and updated

M&E Framework and Plan 2016/2017 going forward.

Other recommendations include:

• Improving efficiency and effectiveness for reporting and improved use of the OVCMIS;

• Ensuring sustainability of OVCMIS by, for example, developing and implementing a comprehensive resource mobilization strategy for the OVC Programme as a whole and

OVCMIS in particular, as well as strengthening linkages with Child Protection and Social

Protection management information systems. MGLSD also has to improve its level of accountability when it comes to donor-funding;

• Improving and ensuring compatibility, linkages and integration between OVCMIS and other systems;

• Improving data analytical power at both the MGLSD and the Districts to increase OVCMIS usability. This should be done through an elaborate capacity building plan that takes care of continuous training of all actors on the OVCMIS.

A. INTRODUTION

This section provides the background as well as an overview of the approach employed by the

Consulting Team hired by USAID to assess the implementation of the Uganda National OVC

Management Information System (OVCMIS). The assessment was done and completed over a period of 8 weeks between November 2014 and January 2015.

1. BACKGROUND

Uganda’s vision for orphans and other vulnerable children (OVC), spelt out in the country’s

National OVC Policy, is:

“A society where all OVC live to their full potential and their rights and aspirations are fulfilled.”

The NOP identifies a number of key interventions which are outlined in detail in the National

Strategic Programme Plan of Interventions for OVC (NSPPI). The NSPPI, revised and updated recently, provides a frameworks for strategic interventions, implementation & Coordination, monitoring & evaluation, and costing of the interventions.

A Monitoring and Evaluation Plan provides a systematic approach to tracking & measuring performance of interventions of NSPPI-2 strategic areas. Specific objectives of the M&E Plan are:

Guide data collection, management, analysis & utilization through defined & harmonized set of indicators & data-collection tools;

Guide flow, analysis & consolidation of OVC data across key line ministries involved in services for OVC;

Provide clear roles & responsibilities of key actors/entities/government agencies & NGOs;

Promote the importance of routine M&E to inform decision making;

Guide the review & assessment of progress & processes in achieving outputs & outcomes of

OVC Programme implementation, identify bottlenecks.

It is with this background that the OVCMIS was conceptualized in 2008 under the USAID-funded

CORE Initiative Project as a web based management information system to support the monitoring of implementation of OVC interventions, measure progress and evaluate performance in the effective implementation of NOP and NSPPI. The OVCMIS was designed to be a central hub for all OVC related data in Uganda.

Housed within Ministry of Gender Labour and Social Development (MGLSD) where it is coordinated by the OVC National Implementation Unit (NIU), the OVCMIS was developed mainly for use by program planners and implementers both within and outside government. It is freely accessible to public users via the MGLSD website www. ovcmis.mglsd.go.ug

The current version was switched on in April 2013 and has been rolled out to all 112 districts and extended to reporting at sub-county level. The system allows for the capturing of quarterly data

(i.e. every 3 months). The OVCMIS requires all service providers to register within the districts where they operate; complete the quarterly OVCMIS summary forms, and then forward them to the district OVC focal person (in most cases the probation and social welfare officer) for online data entry.

According to a report on the status of OVCMIS reporting, compiled by the OVCNIU, the number of districts reporting increased from 76 (68%) in Jan-Mar 13, 85 (76%) in Apr-Jun 13, 93(83%) in

Jul-Sep 13, 100 (89%) in Oct-Dec 13, 108(96%) in Jan-Mar 14, 106 (95%) in Apr-Jun 14 and

108(96%) in Jul-Sep 14. Despite the progressive improvement in districts capturing data on the

OVCMIS, the system still faces some challenges as identified during support supervision visits to the districts, including:

Unstable/lack of internet services and other Information Communication Technology (ICT) resources;

Inadequate district staff skills to enter, analyse and share data

Lack of commitment and sense of ownership in local districts;

Delayed and/or inconsistent reporting or no reporting at all;

Poor data quality;

Organizational bureaucracies making it difficult for some district level players to disclose information.

It is against this background that MGLSD and USAID commissioned a detailed assessment of the

OVCMIS to be conducted, validate the challenges identified, identify other challenges, and then come up with recommended workable and cost effective strategies to address the challenges and improve the overall functioning of the OVCMIS.

2. OBJECTIVES OF THE CONSULTANCY

The overall objective of the consultancy was to critically assess the current OVCMIS and come up with recommended strategies on how to improve the system’s processes for improved and sustained use of data by various stakeholders to successfully monitor progress towards achievement of the

National OVC Policy (NOP) and NSPPI goals.

The SPECIFIC OBJECTIVES were to assess the following components of the OVCMIS:

1. Critically assess the OVCMIS governance and ownership at all levels (National, Local government, lower local government and service provider) and recommend on how best the

OVCMIS can be sustained;

2. Study the current data management processes including; Data flow, collection, entry, cleaning, update, analysis, utilization, sharing, backup and security and advice on how these can be improved;

3. Critically assess the existing Information Communication Technology (ICT) (Human resources, hard ware and software) at all levels and recommend improvements;

4. Assess the level of data analysis, sharing and utilization and recommend improvements;

5. Assess the existing data quality assurance and data validation processes and suggest ways to improve them; and

6. Conduct a quick appraisal of the gap between data needs and what the OVCMIS is providing and make appropriate recommendations to reduce the gap.

3. SCOPE OF WORK

The following tasks were identified and performed by the Consulting Team working under the technical guidance of the MGLSD’s OVC National Implementation Unit and reporting to the national OVC M&E Technical Working Group (TWG):

1. OVC MIS Governance

Validation of the current governance responsibilities and processes and proposal of appropriate revisions, as appropriate, to improve efficiency and effectiveness for reporting and improved use of OVCMIS. The revised governance responsibilities and processes have been designed to take full advantage of e-governance aimed at increasing accountability, participation and information sharing.

2. Data Management

Review and/or validation of the current OVCMIS data collection tools and data management procedures to identify policy, process and procedure gaps critical for improved efficiency, reliability and credibility of the system;

Conduct a rapid review of three other relevant information systems (Education MIS, Health

MIS and Community Information Systems) and propose feasible procedures to improve complementarity and/or interface between the systems;

Conduct a rapid assessment of selected service provider M&E systems and determine the degree to which they are compatible with the OVCMIS.

3. Information Technology

Assess the adequacy of IT equipment, hosting arrangements, power backup, data backup mechanisms, security and internet services at the center to facilitate data reporting and use

Assess the adequacy of IT equipment, power backup and internet services at districts to facilitate data reporting and use;

Assess data processing procedures and access rights to allow data quality checks and data modification, and to manage data integrity;

Review user and administration manuals of the OVCMIS;

Assess appropriateness and user friendliness of the system-generated standard reports and processes for customized reports.

4. Human Resource Capacity

Review the training plan and strategies for supporting current and potential OVCMIS users and stakeholders. Validate or propose feasible and sustainable revisions, as appropriate;

Assess the extent to which the availability or lack of appropriately trained human resource affects the proper use of the OVCMIS at national, district, sub-county and service provider levels.

5. Data Reporting and Use

Review and validate the current information flow of data to OVCMIS and report generation from the OVCMIS, including system generated standard reports and customized reports;

Identify the challenges faced by districts and service providers in reporting to OVCMIS and come up with recommendation on how to address them;

Identify challenges in report generation and data utilization and propose feasible and low cost actions for improving data utilization at national, district and sub-county levels.

B. METHODOLOGY

The approach employed by the consulting team in conducting the assessment of the OVCMIS, was multi-faceted and employed mainly qualitative data collection and collation. The consulting team also captured salient information as indicated against each specific objective in the Terms of

Reference.

REVIEW OF KEY OVC PROGRAMME DOCUMENTS

As a starting point, the team reviewed a number of documents in order to understand the OVC in its entirety. These full list of documents reviewed appears in the references page.

CONSULTATIVE MEETINGS

Meetings were held with key stakeholders in order to arrive at a common understanding of the terms of reference as well as better understand (and manage) the expectations of stakeholders from the OVCMIS assessment. The stakeholders included Uganda Government Ministries and

Departments or Units within Ministries; Development Agencies, and Technical Service

Organizations (TSOs).

See appendix F (5) for the full list of stakeholders met.

DISTRICTS, SUB-COUNTIES AND OVC SERVICE PROVIDERS VISITED

The methodology used in the selection of districts was purposive and convenience sampling targeted towards generating relevant and useful critical information for the assessment. All eight

(8) TSO zones were assessed, with at least one district supported by SUNRISE Project and one

NEW district per zone visited. For those districts that were too far and could be reached due to the limited time for the assessment, the consulting team gathered the information from the national level partners (i.e. MGLSD, SUNRISE, other non-USAID partners, and other secondary sources) and also followed up with telephone calls to the respective districts.

The 2013 reporting rates in the OVCMIS Status Report were used to categorize districts as follows:

Districts with highest reporting rates;

Districts with lowest reporting rates;

Districts that have ever reported;

Districts that have never reported.

The above categories were then considered in the selection of districts in each zone that are supported by SUNRISE and those that are new.

For New districts, the following additional factors from the Status Report were considered:

• Number of functioning computers

• Power source

• Internet access

Across these districts various personnel were covered by key informant interviews and focus group discussions as follows:

1. Chief Administrative Officer (CAO);

2. OVC Focal Persons;

3. Probation & Social Welfare Officers;

4. District/Sub-County Community Development Officers;

5. Sub-county Chiefs; and

6. CSOs / NGOs

ZONE TSO DISTRICT

Visit or

Phone

STATUS MOTIVATION

Central

Save the

Children

Kalungu Visit New National grid, fair internet, 71% reporting rate

Kampala Visit SUNRISE In Kampala but poor reporting rate

South

West

ACCORD

Mitooma Visit New Highest average reporting rate

Kiruhura Phone SUNRISE Highest average reporting rate

Western AFRICARE Kyegegwa Visit New Highest average reporting rate

Kamwenge Visit SUNRISE Lowest reporting rate

Eastern Child Line Bulambuli Visit New

No computer; national grid; fair internet, submitted some reports

Manwafa Visit SUNRISE Lowest reporting rate

Eastern

Central Bantwana

Buikwe Visit New Good resources but poor reporting rate

Namutumba Visit SUNRISE Highest average reporting rate

North

Central

AVIS

Agago Visit New Better resourced than unsatisfactory reporting

Pader

Lira

Visit

Phone SUNRISE Lowest & Reasonable reporting rate, respectively

West

Nile

TPO

Zombo Phone New Only new district

Moyo Visit SUNRISE

Highest reporting rate

Koboko Visit 2nd Highest average reporting rate in region

North

Eastern

FOCREV

Ngora Visit New Under-resourced district

Nakapiripirit Phone SUNRISE Highest average reporting rate

DATA COLLECTION TOOLS

The various data collection tools developed and adapted by the consulting team for the purpose of the assessment were used to guide all the data collection exercises. This ensured that comparable data were gathered for analysis and where necessary the team probed further to gain insights into the various component areas of interest.

The following relevant tools with both structured as well as open-ended questions were utilized where applicable:

1. Tool #1: M&E Governance and Strengthening Tool

2. Tool #2: Data Quality Assessment Tool (District Level)

3. Tool #3: Data Quality Assessment Tool (Sub-County Level)

4. Tool #4: Data Quality Assessment Tool (Service Provider Level)

5. Tool #5: ICT Capacity and Processes (National Level)

6. Tool #6: ICT Capacity and Processes (District Level)

7. Tool #7: OVC M&E Information Needs

TECHNICAL REVIEW OF OVC MIS AND OTHER SYSTEMS

The OVCMIS was technically interrogated, assessing data-entry and analysis and report-generation.

Other systems such as the Health Management Information System (HMIS), Education

Management Information System (EMIS), and the Community Information System (CIS) were also assessed.

OTHER METHODS FOR DATA-COLLECTION

Other methods for data collection included focus group discussions, key informant interviews, as well as observations and physical checks during field visits to service providers in the districts.

DATA COLLECTION TEAMS

The team of four consultants comprising the M&E Specialist (Team Leader), a second M&E

Specialist, an MIS / IT Specialist, and an IT Specialist, were split into two fieldwork teams each led by an M&E specialist. Each fieldwork team was then joined by a member of staff from MGLSD

OVC NIU. In addition, one of the teams was also joined by the USAID /QED Public Health M&E

Specialist.

C. FINDINGS

1. OVCMIS GOVERNANCE & HUMAN RESOURCES

In this section, a critical assessment of the OVCMIS governance and ownership at all levels

(National, Local government, lower local government and service provider) was made. This included validation of the current governance responsibilities and processes, as well as consideration of e-governance to increase accountability, participation and information sharing.

The adequacy of human resources for M&E and OVCMIS was also assessed.

FINDINGS - GOVERNANCE

The NSPPI-2 has a detailed Implementation and Coordination Framework which identifies the roles and responsibilities of different stakeholders, with MGLSD taking the lead role in providing overall direction and guidance. The National Council for Children (NCC) is mandated to carry out advocacy for children and monitor compliance with international and national standards. Other stakeholders identified include other government ministries and public institutions, local government, faith-based and civil society organizations, the private sector, as well as development partners. Coordination and collaboration mechanisms such as the National OVC Steering committee and thematic working groups, were also spelt out.

The M&E Framework and Plan for the NSPPI-2 identifies specific roles and responsibilities of stakeholders, starting with the National Implementation Unit (NIU) at MGLSD, the OVC M&E

Technical Working Group comprising members from different agencies, right down to district and sub-county staff (PSWO, District Planner, CDO, M&E Focal persons, etc.), to caregivers and children.

Despite the well-articulated governance responsibilities and processes in these important documents, this assessment found that:

• There was lack of strong adherence to the stated OVC and OVCMIS governance processes at national and district levels. MGLSD which is mandated to guide and regulate activities in the social protection sector didn’t seem to be in much control of the OVC subsector, which is largely donor-driven;

• The existence of the various governance structures (committees and working groups) at the national and district levels was however noted;

• Despite the fact that development partners see OVCMIS as the ‘go-to’ system for OVC related data, with the exception of a few such as UNICEF, most have not been properly and adequately engaged to support the OVCMIS and cater for their expectations and/or needs. The coordination of development partners was found wanting;

• There was lack of strong leadership at the national and district levels to ensure:

• Continuous registration and update of OVC service providers;

• Compliance – with service provision and quality standards, and reporting;

• Full and undivided support for one OVC M&E system - the OVCMIS;

• Some key stakeholders were not represented at high-level technical working groups and steering committees (e.g. NITA-U not represented at OVC M&E TWG);

• OVC programming at national level was fairly strong. However, at lower levels, more clarity was needed especially where DOVCCs and SOVCCs were inactive;

• There was reported lack of efficiency and accountability for OVC funding within government at all levels. This was the general feeling amongst donor agencies interviewed;

• There were inadequate M&E plans covering OVC at the district level. Districts did not mention having M&E plans in place that guide data collection, analysis, reporting, dissemination, feedback and information utilization at their level;

• District funding was a challenge and compromises ability to provide support to sub-counties and

CSOs for OVCMIS;

• OVC Programme performance at Local Governments was not part of the Local Government performance assessment criteria and this bred laxity when it came to OVCMIS at local levels;

• Presence of unregistered CSOs was noted. This was due to lack of stringency in the registration and regulation practices. Some CSOs saw it as not compulsory to submit or share service statistics with sub-counties or districts. Sub-counties and districts also did not exercise their authority to compel CSOs to submit data;

• Most CSOs were not willing to reveal their funding sources, budgets, and actual expenditures;

• A few districts took into consideration reporting when renewing CSO certificates;

• There were limited resources specifically for OVC to organize regular meetings with sub-county CDOs and CSOs;

• There was lack of commitment and sense of ownership of OVCMIS in some districts.

FINDINGS – HUMAN RESOURCES

• Under-staffing of the OVCNIU and districts impacted negatively on the effective implementation of OVCMIS;

• There was no training plan in place for OVCMIS users and staff at all levels. A draft training plan was being developed for 2015. This training plan did not include findings from this assessment.

• There was no continuous training of staff at district and sub-county levels to cater for those that leave, die, or vacate their postings for various reasons;

• Most TSOs had dedicated M&E staff and provided technical support to districts and CSOs.

With the TSOs ending in 2015, the impact will be negative as skills transfer to permanent staff in districts has not been optimal;

• There were knowledge and skills gaps at district level especially with regards to data entry, data analysis, and report-generation, as well as information dissemination. Lack of dedicated

OVCMIS personnel resulted in delays in capturing of forms submitted by CSOs; TSOs tried to fill this gap by offering some basic training;

• The responsibility of handling OVCMIS data at the districts was not clear. In some districts the assumption was that OVCMIS data was supposed to be input by PSWOs. However, the reality on the ground was that PSWOs were overloaded;

• A number of districts had PSWO’s who had been acting in their positions for over 4 years, which was demotivating;

• In some districts & sub-counties, there were office space challenges. There was no dedicated office for the PSWO and DCDO. As a result there were privacy and confidentiality challenges, e.g. for those that needed counseling or reporting cases;

• PSWO and DCDOs were poorly supported and funded, even with locally generated revenue.

RECOMMENDATIONS

MGLSD should continuously promote and advocate for the M&E Framework and Plan at all levels and amongst all stakeholders, and ensure adherence to guidelines in the M&E Plan;

The issue of limited staffing of the OVCNIU should be addressed as a matter of urgency. All staff working for the OVCNIU should be made permanent employees of MGLSD;

Under-staffing of districts with respect to OVCMIS implementation should also be addressed as a priority, especially in light of the TSO contracts coming to an end;

MGLSD should issue Operational Guidelines for all players in the OVC subsector. This will lead to a more streamlined environment with a clearer operational framework regarding the relationship with MGLSD, reporting, registration of service providers, information sharing, scheduling committee meetings, and any other sticky issues that would improve coordination and performance of the OVC subsector;

OVCMIS should be part of a larger (yet-to-be-developed) Social Protection Management

Information System for purposes of consistency in information systems management at the

MGLSD and to allow for consolidation of efforts to mobilize resources for critical information systems at MGLSD; It is also easier to market a social protection MIS than a smaller OVCMIS;

MGLSD should take advantage of the current opportunities (go-to system and timing of assessment) to engage all the relevant stakeholders to ensure that their needs and/or expectations are adequately covered.

As a matter of urgency MGLSD should strictly enforce compliance to standards of service delivery and reporting and institutionalize continuous monitoring of the OVC program implementation. Non-compliance should attract sanctions;

MGLSD should follow up and manage perceptions among some development partners about

Accountability concerns.

There is an urgent need to include the OVC Performance Criteria into the larger Local

Government Performance evaluation criteria; this will mean that the OVC performance will affect district performance, and therefore the Chief Administrative Officers (CAOs) will prioritize OVC activities at both performance management and resource allocation levels;

The National Information Technology Authority of Uganda (NITA-U) should be represented at the technical working groups of the OVCMIS so that information technology related issues of the OVCMIS are carried and supported at the highest levels in Government. This will also

Raise Awareness about the OVCMIS as a critical Government of Uganda information system;

There is need to Strengthen linkages, reduce growth of parallel OVC Database systems and concentrate on Utilization of Data from Existing Systems. This would in turn reduce on the duplications in data collection. This assessment recommends the design and implementation of an OVC data warehouse with secure open interfaces between existing OVCMIS subsystems and other relevant systems like HMIS, EMIS, and CIS;

MGLSD should work closely with MoICT (for policy related aspects), NITA-U (at implementation level), and MoFPED (for funding) to ensure that the OVCMIS is implemented in line with current GoU information system guidelines, and also that it receives the necessary funding like other critical GoU systems like IPPS, IFMS, HMIS, etc. This will ensure

Sustainability of the OVCMIS;

MGLSD working closely with the Ministry of Public Service should ensure that Job

Descriptions for posts relevant to OVCMIS are clear and the workload is well balanced. This will improve performance accountability and hence performance improvement at the various levels.

A Comprehensive Training Plan for OVCMIS capacity building need to be developed and adhered to; this will ensure a reliable supply of well-trained OVCMIS staff at both Central and

Local Government levels. A more comprehensive OVCMIS training needs assessment will add value to the impact of the training;

MGSLD through its mandate for occupational safety and standards, working together with

MoLG, should ensure that the Offices of the PSWO and that of the District Community

Development Officer (DCDO) have enough privacy, space, power, and records storage facilities; This would reduce on the reported cases of infringement of privacy, confidentiality, loss of OVCMIS records, poor reporting due to inadequately resourced facilities;

2. DATA MANAGEMENT, REPORTING & UTILIZATION

In this section, we assessed the current data management processes including data flow, collection, entry, cleaning, updating, analysis, reporting, utilization, sharing, backup and security. The

OVCMIS data collection tools were also reviewed.

FINDINGS

OVC MIS Data Collection Tools and Data Management Procedures

Several data collection tools for gathering comprehensive data were developed and/or adapted for collecting data for various OVC program indicators. This was positive in that there existed a collection of tools that had been developed and adapted and hence minimized the need to reinvent the wheel. However, there was an efficiency issue here as there were cases of duplication and therefore some tools could be eliminated;

Revisions to data collection tools were made without the relevant consultations and this led to the modification of tools without any prior notice or proper briefing of data sources. This had implications on the reliability of data as the same data collection tools were not being used.

The underlying problem is continuously changing reporting requirements from donors which leads to changing of indicators very frequently, leading to the need to revise tools frequently, with which comes a host of issues;

Repetitiveness of data elements between registration forms and summary forms that the database can handle and need not be summarized or aggregated. This was introducing transcription errors into the system. There were issues of redundancy and minimal use of the database for management of OVC program data;

Despite guidance in the M&E Framework and Plan for conducting Routine DQAs, the assessment found no evidence of national or district processes to perform routine Data Quality

Assessment (DQA);

Double-counting of OVCs served was a problem. This stems from the lack of a unique identifier for the OVCs served; collection of aggregated data; and system validation checks were not comprehensive to prevent double counting;

Adherence to data management procedures in most districts visited was not happening (delayed data entry, data loss, date stamps, etc.); Copies of data submitted were not kept on file at data sources; Service providers/CSOs working in more than one sub-county did not share or submit copies of their reports to the sub-counties and report directly to the district;

Most data elements required for deriving the relevant indicators could be found on the data collection forms. Instead of providing for disaggregation, the use of the district summary form aggregated the data and as a result vital critical elements were missing and got left out as aggregated data does not allow to slice and dice data. (For example, UNICEF wants to be able to analyze the various types of OVCs. However, these are lumped together and hence this is currently not possible to do. Also, age groupings are fixed and the current design does not allow to recode data or modify groups as may be required to meet different stakeholder needs).

The various data collection forms and the need to transcribe (or copy) data from one to another to generate summaries resulted in transcription errors that may account for the inaccuracies in the data reported;

Although data collection tools had some instructions, they were not comprehensive and service providers complained that they sometimes found the instructions confusing and/or not comprehensive;

Awareness of reporting schedule was inconsistent across districts and among service providers.

DATA QUALITY ISSUES (i.e. validity, completeness, reliability and timeliness) were observed across district, sub-county and CSO levels;

Data validity:

There was missing data due to incomplete filling of forms: the quarter, financial year and respective quarters were sometimes left blank;

There had been occasions where forms submitted to the district were misplaced;

There was potential for transcription errors emanating from transferring data from registers onto the district summary form. (An example is where the focal person in Moyo District reported that they sometimes identify transcription errors but are unable to rectify this);

Some DQA issues - e.g., parent details were captured instead of the child details in the service register;

Date stamps were not indicated on all forms;

Some data were collected over the phone at district level from sub-counties. Data transcription errors may have occurred through this process as there was no verification done or attempt to request the sub-counties to submit the hard copy forms for the files;

OVCMIS forms did not provide for automatic duplication (self-inking), so CSOs could not easily retain a copy of data submitted.

Data reliability issues:

All districts were found to be using the same District Summary Report form;

There were cases of exaggerated reporting figures by some CSOs. This was because some CSOs thought that over-reporting would lead to better government funding;

There were incidences where different officers from the same CSO gave conflicting figures;

Some districts mentioned that they had tried to use the system to generate reports, however some reports had exaggerated figures; There was a possibility that these were system generated computational errors;

There was a challenge of having different OVCMIS data collection tools. This caused confusion, as it was not clear which one was the right one. Moreover, they were not informed of the changes in the data collection tools;

Double-counting was reported as a major problem. The source of this error was attributed to the lack of unique identifiers for individual OVCs; Also, system validation checks were not comprehensive to prevent double counting;

District staff responsible for data entry were not aware of procedures to edit or clean data. In some cases they were aware of data entry errors but had not followed up to get these rectified.

Timeliness issues:

There were delays in the submission of data by some CSOs - some CSOs attributed the delay in reporting to distance challenges (i.e. challenges with transport costs to deliver the filled forms to the district); coordination meetings were not taking place due to facilitation challenges.

There also delays in entering data. The district officers took long to enter the submitted forms into the OVCMIS;

Some districts did not have the report for the most current period. They were not aware of this and realized this at the time of the visit by assessment team;

Timelines for reporting given by districts was not consistent across all districts. Districts may have modified this reflect their local contexts. It is was not clear if there was no confusions as a result;

• CSOs with strong international parent organizations tended to have higher data quality standards than the district level. This was reflected in the existence of registers and in some cases they had well developed stand alone or web-based databases.

Data Reporting & Utilization

• CSOs working in multiple sub-counties did not share data with sub-counties and submitted data directly to districts, as a result sub-counties were not informed about activities of CSOs within their sub-counties;

• In the majority of cases, data was only reported and not used; Only where there was good leadership and active DOVCCs, OVCMIS data was often used for planning and influencing policy,

• There were data submission delays by some CSOs - some CSOs attributed the delay in reporting to distance challenges (i.e. challenges with transport costs to deliver the filled forms to the district); In other cases, there was inconsistent reporting or no reporting at all;

• Some districts regularly invited CSOs to information sharing meetings and also to discuss challenges and propose action plans;

• Some districts and CSOs used OVCMIS data for resource mobilization and in determining where to locate some projects e.g., the Child Labour Project, MildMay;

• No feedback and analysis from districts and national office (OVCNIU) was received after data was submitted;

• Districts supported by SDS had resources to facilitate CDOs, therefore reporting in SDS-supported districts was higher than those not supported by SDS;

• Collection of reports from the various sources was not consistent across districts. While some districts reported as required into the system, others were either reporting inconsistently or defaulting with their reports;

• Districts reported having entered data into the OVCMIS for the first quarter of the 2014/15 fiscal year; there were no supporting documentation in the form of completed District Summary forms to confirm this.

Some of the recommendations from this assessment have already been made in the M&E

Framework and Plan, but not implemented. The first recommendation therefore, is ensuring political and management commitment to implementing all recommendations arising from this and the earlier assessments, otherwise this exercise will have been futile.

Recommendations from this assessment include the following:

• MGLSD should develop standard operating procedures (SOPs) for the OVC program and

OVCMIS, which should include among other topics covered in other sections of the report the following:

o SOPs should be developed to guide the revision of data collection tools; Change of

OVCMIS data collection tools should be done through consultative meetings with all key stakeholders;

o SOPs should clearly spell out reporting schedule for all data sources;

o SOPs should be developed for performing routine data quality assessments (RDQA) as an integrated part of support supervision at all levels (national to district, district to sub-counties and sub-counties to service providers);

o SOPs should also guide data collation and aggregation (primary register to summary tool etc.);

o Reporting arrangements for CSOs who work in more than one sub-county to share data with sub-county offices even when they submit directly to the district office;

o Filing requirements when data is collected via other modes e.g. over the phone. Step by step instructions regarding registration of data forms received to have records of submissions with date stamps, signatures and other identifiers properly completed. This should cover both data suppliers and receivers;

o Proper records keeping of OVCMIS data.

• Streamline data collection tools to eliminate duplications; This should be guided by the SOPs for revising data collection tools; Revisions to data collection tools should involve IT and/or database developers to ensure that revisions take into account design implications and guidance on how to eliminate redundancy and minimize data collection points and tools.

• MGLSD should engage respective players to resolve the issue of unique identifiers for OVCs;

Consideration should be given to the number issued to new born by the health facilities; Explore

UNICEF initiatives in this area as they are working with the national identification authorities to ensure that the relevant variables are captured and that there is complementarity between the systems;

• There is need for carbonated (or self-inked) OVCMIS reporting tools to ensure that there is an audit trail in place;

• Consideration should be made for CSOs to capture data directly into the system;

• Districts should play a support-supervision and validation role. This would improve accountability, participation and information sharing;

• There is need for provision in OVCMIS to import data from other systems or formats e.g., Excel. This will facilitate export and import of data from service provider systems and minimize the burden of having to fill forms over again after they have entered data into their own systems.

• Steps should be taken to address database related issues impacting the quality of the data in the

OVCMIS. The following are some pointers to system areas that require immediate attention:

o Clean up test data and resolve inconsistencies with drop down lists;

o Bring to the table data mining experts – middle level statisticians and IT experts who are more hands on and are involved in fixing the bolts and nuts so they can ensure that critical elements are fully covered to ensure that data collection processes are aligned with the

OVCMIS database application. They should work closely with OVC practitioners;

o Introduce data validation checks to minimize data entry errors and prompt users for errors in data at the point of entry;

o Introduce approval module into the OVCMIS as a system data quality check; When data is entered, an email is sent to a reviewer who checks the data and if the data is satisfactory approves else rejects the entries and this triggers an email to the data entry person who corrects and resubmits in an interactive cycle until the data is of good quality for approval;

Data does not get added to results until approved;

• Develop and implement a training plan to address knowledge and capacity gaps of all actors at the various levels:

o Address knowledge and skill gaps with regards to data entry, data quality assurance and data cleaning;

o The Training Plan should include Refresher Training;

• Institutionalize routine data quality assessments (RDQA) as part of national, district and sub-county level support supervision to supplement high level data quality assessments from the national level.

• Resources should be budgeted to provide facilitation for OVCMIS data management functions.

• There’s need to develop an elaborate OVCMIS information dissemination strategy to improve on usage of OVCMIS data. This will increase awareness around the overall OVCMIS effort, and also encourage CSOs to report;

• Smart technologies should be utilized to facilitate real time reporting of data where possible, to reduce the current latency of 3 months introduced by quarterly reporting;

There’s need to integrate information gathered through the Child Helpline into the OVCMIS.

This will make the OVCMIS more inclusive in terms of coverage, which in turn improves usage;

The OVCMIS GIS capability should be augmented. This will greatly improve capacity to support the…