Democratic Republic of the Congo: OFDA/FFP Program Evaluation Consultant

“Emergency Response and Economic Recovery for Eastern DRC”

Independent Final Evaluation

Democratic Republic of Congo

Application deadline: September 14, 2016

Start date: October 01, 2016 (earlier start date preferred)

A.Program/Project Description

In August 2014, Samaritan’s Purse (SP) began an emergency response program, jointly funded through a cooperative agreement with USAID’s Office for Foreign Disaster Assistance (OFDA) and Food for Peace (FFP), to meet the emergency needs of conflict-affected households (HHs) in the Orientale and North Kivu Provinces of eastern Democratic Republic of Congo (DRC) as well as promote the rapid recovery of affected livelihoods and markets. This project completed its first year in August 2015 and renewed for a second year through a cost-modification process. At the end of the project in September 2016, this project will have distributed food and non-food item (NFI) kits in emergency situations to assist 142,800 war-affected individuals cope with displacement and loss through either direct distributions or through a voucher-fair system.[1]

Instead of responding to just one type of need of displaced HHs—e.g., food or NFIs—this innovative project seeks to respond holistically to the needs of conflict-affected populations by pairing the provision of food with that of essential NFIs such as household, kitchen, and hygiene supplies. By responding to the multi-sectoral needs of displaced HHs, this program seeks to ensure that all of the needs of the beneficiaries are met in a timely and effective manner.

The project is not only multi-sectoral, but also has flexible modality, allowing for intervention via either direct distribution or via voucher fairs. The decision of which modality to be used is dependent on the results of a multifaceted market analysis designed to ensure that local markets will not be negatively impacted and that the proposed activity will not pose a security risk to the beneficiary community. In order to reinforce local economies, Samaritan’s Purse prioritizes the use of voucher fairs where possible. These activities are organized in closed market settings, following a careful vendor-selection process which involves, among other things, the fixing of price ceilings for commodities to be sold. Vouchers are custom designed cash vouchers, which give freedom to beneficiaries to choose freely from the food and NFI items brought to voucher fairs.

The purpose of this final methodology evaluation is to determine whether the innovative aspects of this multi-sectoral, flexible modality, emergency programming effectively met the needs of conflict-affected populations and markets in a way that should be replicated in the future. Additionally, the consultant will review the appropriateness of Samaritan’s Purse’s method of monitoring and evaluating project outcome indicators, which essentially compares vulnerability information captured in targeting exercises with that collected during post-distribution monitoring surveys for each project location.

B.Purpose/Reason for the Evaluation

This evaluation is not required by the donor; however, due to the innovative nature of the emergency response part of the programming, Samaritan’s Purse—the implementing partner—is soliciting services for this external evaluation. Samaritan’s Purse is interested in understanding the effectiveness of meeting the immediate needs of HHs displaced by rapid onset emergencies through direct distributions or voucher-fairs that provide both food and NFIs at the same time. Samaritan’s Purse is also interested in understanding the effectiveness of its methodology in assisting conflict-affected markets, specifically in terms of its rapid market analysis methodology and vendor selection process. Finally, Samaritan’s Purse seeks recommendations on the appropriateness of its monitoring and evaluation methodology, specifically in the calculation of project outcome indicators, and would like input regarding the scalability and replicability of this model for future emergency responses in eastern DRC.

As part of the effort to reduce bias, the contracted evaluator will be expected to determine whether Samaritan’s Purse’s intervention methodology met project objectives and whether their monitoring and evaluation systems, were appropriate and results accurately represented.

C.Specific Objectives of the Evaluation and Evaluation Questions

The objective of this final performance evaluation is to understand the effectiveness of pairing food and NFI distributions in order to meet the immediate needs of conflict-affected HHs.

Specific evaluation questions include:

· Was this program an effective way to the meet the most immediate needs of conflict-affected HHs in eastern DRC?

· Was the voucher fair methodology used, including market analysis and vendor selection strategies, an effective way to promote the recovery of local market economies in eastern DRC?

· Did the programming activities meet the objectives laid out in the proposal?

· Was SP’s program monitoring and evaluation methodology for project outcome indicators appropriate and did it provide an accurate representation of project success?

D.Scope of Evaluation

The scope of the evaluation will occur for a maximum of 30 days following the end of project activities in September 2016 and cover the life of the project, including both the first and second years. The evaluation will cover, in particular, the emergency response sector methodology, including the provision of both food and NFIs as implemented in the former Province Orientale[2] and North Kivu since August 2014.

This evaluation, which comes at the end of two years of emergency response project activities, will engage all relevant stakeholders, including a representative number of HHs that benefitted from the project activities. Target groups include beneficiaries of the emergency response programming, including IDPs, returnees, and host families. Vendors that participated in food and NFI voucher fairs will also be contacted.

During the prescribed timeframe, it will not be necessary to engage research and data collection regarding the overall change in the economic, security, or social situation of the majority of people residing in eastern Congo as this may be difficult to analyze during the evaluation.

E.Evaluation Methodology

The evaluation will primarily address the successes and failures of the intervention and monitoring methodologies used in this project, rather than the success of indicators, which has been achieved in the 14 different project locations over the life of the project. In case a statistical evaluation of any sort is proposed for evaluating the effectiveness of program methodology, it should be noted that both LQAS and systematic random sampling have been used and these would therefore be the preferred methods. However, the evaluator may propose alternative methodologies with a strong justification for how the intended evaluation results will be achieved.

The methodology for the study will be further developed by evaluator at the outset of the work and will be presented to Samaritan’s Purse in detail upon inception. The study must employ both qualitative and quantitative data collection methodologies. Additionally, the chosen methodologies should be appropriate for rural areas as well as for illiterate, low-income populations and should take into account that sanitation and hygiene practices are often a culturally–sensitive topic.

All data collected under this study shall be disaggregated by gender and displacement status. Parallel or stratified sampling will be used to include a representative sample of both IDP and host target populations. All data will be owned by SP and USAID OFDA and FFP. All data will be owned by SP and USAID OFDA and FFP; all raw data will collected from the consultant and analyzed data will be provided to both SP and USAID OFDA and FFP.

The evaluator is responsible for ensuring informed interviewee consent prior to beginning any surveying activities. The evaluator will also attempt to ensure the privacy and confidentiality of beneficiaries’ interviews.

Specifically, the following strategies for data collection and analysis shall be applied:

1. Desk review of relevant project documents, partner’s assessments and databases;
2. Discussion with senior management of SP and if necessary with donor representatives (OFDA and FFP);
3. Interview with key informants;
4. Field visits to distribution and fair sites;
5. Guided and structured interviews and focus group discussions with beneficiary communities including women, men, children, consumers and other key stakeholders;
6. Recruitment and training of enumerators

F.Stakeholder Involvement

Stakeholder participation is fundamental to SP evaluations. The consultant is expected to conduct a participatory evaluation providing for meaningful involvement by project partners, beneficiaries and other interested parties. Stakeholder participation is to be an integral component of evaluation design and planning; information collection; the development of findings; evaluation reporting; and results dissemination.

G.Deliverables

Inception Report and Presentation

An inception report with an oral presentation to SP including:

· Overview of Program/Project

· Expectations of Evaluation

· Roles and Responsibilities

· Evaluation Methodology

· Evaluation Framework

· Information Collection and Analysis

· Reporting

· Work Scheduling

· Detailed Evaluation Budget

Note that research protocol and survey instruments will need to be approved by SP before going to the fieldwork stage.

Oral Presentations of Top Line Findings

This report must be presented to SP DRC in-country leadership and, if necessary, to SP partners and stakeholders. Findings to be included in top line presentations will be agreed upon ahead of time with the SP.

Final Report

The final written report will include detailed results from data analysis and key recommendations to inform the communication campaign and marketing strategy. The final report will include the following chapters:

· Acknowledgements

· List of Abbreviations and Acronyms

· Executive Summary

· Background and Context

· Methodology

· Evaluation Framework

· Aim

· Objectives

· Limitations

· Results

· Discussion

· Recommendations

· Conclusions

· References

· Annexes

o Summary indicator table (including confidence intervals where appropriate)

o Data collection tools

o Consent form

A final presentation of results will be presented to SP technical advisors and program leadership, either in SP’s headquarters in Boone, NC, or remotely via internet conference call. The donor USAID will be invited to participate, either in-person or remotely.

In addition, the final report will be made available to Samaritan’s Purse and will include:

• Clean and fully referenced electronic data sets for the quantitative position in an agreed format with copies of the original data collection forms in English;
• Basic transcripts of all interviews and focus group discussions, all research tools used in the Annex, and focus group discussions in an electronic format;
• All outputs will be presented in both electronic form (programs to be agreed with SP) and paper copies.

All deliverables submitted to SP shall be submitted in electronic form where feasible, or in hardcopy as necessary or required. Deliverables will be considered drafts upon initial receipt. Drafts will be reviewed and comments provided within two weeks of receipt, unless otherwise specified. The selected consultant shall appropriately address Samaritan’s Purse’s concerns and provide final deliverables within one week of receiving SP’s response. All reports and datasets will be shared with relevant stakeholders.

All products arising from this research will be owned by Samaritan’s Purse. The researcher will not be allowed, without prior authorization in writing, to present any of the results as his or her own work or to make use of the evaluation results for private publication purposes.

H.Evaluator Qualifications and Competencies

· Demonstrable expertise in monitoring and evaluation methodology

· Track record in developing, conducting and reporting on various types of evaluations including qualitative and quantitative data collection for international NGOs

· Experience in statistical analysis of complex datasets using SPSS or similar statistical software

· Strong oral and written communication skills with an ability to respond to comments and questions in a timely, appropriate manner

· Fluent in English

· Fully acquainted with the specific donor requirements

· Experience in the region/sector of programming,

· Prior experience in evaluating Emergency Response Projects (Food assistance and NFI) and Cash based programming in complex humanitarian contexts.

The working language in DRC is French, and the contractor will be responsible for hiring their own professional translator. Familiarity with Swahili or any of the other local languages will be preferred.

Due to the short timeframe, consultant availability will also be preferred. The ideal start date will be during the last week of September 2016 on the ground in Beni, DRC. The latest acceptable start date will be October 1, 2016.

I.Evaluation Budget and proposed Payment Schedule

Samaritan’s Purse will cover the cost of international airfare, in-country accommodation and food, and in-country travel. A maximum of 30 working days will be included in this contract, inclusive of up to 4 days of travel to/from Beni. This number is negotiable, and will be influenced by 1) the number of consultants firms wish to commit to this project, and 2) the number of days firms estimate they will need in DRC to complete the data collection.

The proposed payment schedule details are as follows:

· 10% upon signature of contract

· 20% on acceptance of Inception Report and completion of its oral presentation

· 40% on acceptance of top line results

· 30% on acceptance of the final report

J.Data Ownership and Confidentiality

All datasets and documents passed to the consultants remain the property of SP and are required to be considered under the strictest confidence. The datasets and documents must not, under any circumstance, be shared with a third party without the prior agreement of the SP Director for International Projects.

[1] The project also contains a separate agricultural livelihoods sector which serves different beneficiaries, however this Terms of Reference document is only for the evaluation of the Emergency Response sector of the project.

[2] As of 2015, Province Orientale has been dismembered into 4 smaller provinces; emergency response activities primarily took place in the area covered by the newly established Ituri Province.

Please submit a CV/resume and Cover letter detailing your qualifications to Danielle Bullock at dbullock@samaritan.org by September 14th.