END USER MONITORING

Three Stones led an End User Monitoring (EUM) evaluation for UNICEF in Mahama Refugee Camp. This evaluation focused on assessing the end users of UNICEF WASH, education, and health supplies to the camp. Given the quality of the assignment and the thoroughness of the approach, UNICEF requested the report and findings be published and circulated as a UNICEF global best practice. In addition, Three Stones conducted a Sustainability Review of the USAID-funded Higa Ubeho project for Catholic Relief Services and conducted additional surveys and evaluations for USAID-funded projects in Rwanda, including: the CMM-Duhuze (“Connect”) Project in seven districts; thee end-line evaluation of Peaceful Rural Transformation Project in four districts, and; a KAP Survey for Women’s Leadership, Shared Decision-Making and Joint Control Over Land Rights in Rwanda in four Districts. We offer comprehensive expertise in third party/supply EUM, social research and program evaluation to help governments, donors, and organizations effectively design, implement and assess interventions.

 

EUM enables a more comprehensive approach to monitoring and documents and gauges program impact effectiveness and efficiency down to the beneficiary. As well, EUM serves as a risk management tool highlighting issues and gaps that can then be considered and rectified.

Three Stones uses EUM as a part of our outcome monitoring approach, so countries and managers better understand programming progression towards the intended impact as well as the satisfaction of the beneficiary. It helps monitor behavioral changes, (i.e. beneficiaries’ use of the supply items by monitoring if the supply items have reached the intended users, beneficiary and those providing the services) and if the supplies or intervention are used or stored in correct ways and if they are still functioning. Three Stones’ EUM experience:

  • Determines efficiency and effectiveness of the procured supply components by identifying existing gaps and generate recommendations and necessary follow-up actions; 

  • Examine if the supported supply components have been benefiting intended populations;

  • Tests the 3rd party supply end-user monitoring and generate lessons learned for potential larger scale interventions  

  • Verifies if:

    • the supply delivered to end-user/partners according to the plan?

    • are the procured items stored, maintained and/or still functioning well?

    • are end users satisfied with the supply items (in quality and quantity)?

    • is the supply management record, stock inventory and other relevant documentation for the supply items accurate and consistent?

 

Assessing human behavior

Three Stones uses a highly participatory EUM approach. We work with the client, service delivery company and the client to obtain in-depth understanding of whether the end user received the ‘right product’, ‘right quantity’, ‘right quality’, ‘right location’ and ‘efficient management.’ Our approach also allows for triangulation of results.

 

Three Stones uses EUM to:

  • capture the end user perspective

  • improve the quality of programs and maximize impact

  • make mid-implementation course correction
     

The use of tablets for data collection allows real time data flow, which improves data analysis and data security thwarting potential errors which may be derived from paper-based data collection. Field data collection includes warehouse visits and onsite inspections (observation). Quantitative data is collected using tailored survey tools and key information interviews and focus group discussion questionnaires. All tools are translated into the local language and back translated. Tools are pilot tested during the enumerator pre-training.  Our approach includes the following:

  • review of primary and secondary data

  • primary data collection and analysis through 

  • administering the standard questionnaires

  • undertaking key informant interviews (KII)

  • facility observation

  • focus group discussions (FGD)

  • sensitive approaches to interviewing and working with children

Three Stones provides expertise in collection, analysis, and reporting of data along with training modules for capacity building including sessions to share the findings, experience, and lessons learned from the monitoring exercise.

Collaborating closely with clients and local partners to deliver high-impact analysis, results and recommendations, Three Stones has successfully conducted research, evaluation/assessment or EUM in Burkina Faso, Democratic Republic of Congo, Ghana, Mozambique, Rwanda and Uganda.

 

Three Stones has a multidisciplinary team with technical expertise in the intended field of research. We have completed single sector or integrated assignments in gender, disability and inclusion, early childhood development, education, agriculture, rural livelihoods/farming, governance, land reform, health, nutrition, and WASH.

 

We have significant experience conducting and managing quantitative and qualitative field research (e.g. surveys, in-depth KII, FGD, case studies, household KAP studies and researcher training). Our team has carried out interviews at every level of civil society and completed collaborative research with government ministries and international donors.

 

Research Design and Tools Development

Three Stones designs high quality data collection instruments and data collection guidelines to safeguard the integrity and accuracy of data collected. Special attention is paid to a review of research tools to ensure that the research team has technical understanding and mastery of the tools. Where translation is required, Three Stones utilizes a double-blind translation technique for improved accuracy of the translation.

 

Three Stones liaises with national stakeholders so research proposals and tools are acceptable and in line with client or national guidelines. We are adept at ushering research protocols through host country Review Boards. 

Data Collection, Data Analysis and Data Protection

Data collection and management plans are developed at the beginning of each project and applied throughout the duration of the assignment—protocols and software are tailored accordingly. Three Stones uses software to maximize the efficiency, security, and quality of data collection, data processing, analysis, results visualization, and dissemination. When applicable, a tablet-based approach for quantitative data collection is used and we are expert in a wide range of tools and software packages, including:

  • CS Pro & CS Entry

  • Dedoose

  • KoBoCollect

  • Magpi

  • NVivo

  • Microsoft Access and SQL Server

  • Open Data Kit (ODK)
     

Receiving Real Time Results: Upon data collection, quantitative data from the surveys are made available in real-time and disaggregated. Mobile data collection provides more effective surveys with cleaner and richer data (GPS coordinates, photos, videos), but also faster and more secure data transmission and management.

Training

Three Stones sources local staff so work can begin with short notice. To prepare data collectors for each assignment, we have a formalized training package which helps with standardization and quality in approach. The systematic and formalized training includes an introduction to the client and the overall program design, aims and objectives and, a detailed briefing of the context for the research study. Prior to conducting field research, Three Stones’ data collectors and enumerators are trained on and must pass a research ethics short course. Protocols and policies regarding data collection are observed throughout the fieldwork. Given the global expertise and network of the Three Stones team, we have tested and proven our abilities to quickly recruit, select, train and carry out data collection in multiple countries.

 

Logistics support

Three Stones remains in contact with client representatives through regular calls to provide status updates as well as through agreed reporting. In addition to communication, Three Stones provides administrative and logistics support for safe and reliable transportation and arrange lodging, as required, for the research team and client representatives where necessary.

© 2018 Three Stones International

  • Three Stones LinkedIn
  • Three Stones Twitter
  • Three Stones YouTube