Consultancy for Real Time Evaluation of the Armenia Population Movement Operation

The real time evaluation (RTE) will assess the ongoing IFRC response to the Armenia Population Movement to inform the continued response as well as future global emergency relief operations.

Location of consultancy: remotely and 10 days in Armenia for field visits and interviews with ARCS staff and volunteers, as well as with ICRC, IFRC Country Delegation and Partner National Societies (PNS), external stakeholders and affected persons (as appropriate). Primarily, the RTE will geographically focus on the areas of the capital (Yerevan) and the districts with the highest density of affected persons (at the time of writing, the regions [Marzes] of Ararat, Kotayk, and Armavir).

Evaluation purpose & scope
This RTE is being carried out with the purpose to evaluate the response operation to date and will take stock of lessons, in order to learn and to inform the ongoing response and recovery activities, and to provide
recommendations for the ARCS and IFRC systems in place. Given the pilot character of the NWoW setup, IFRCwide recommendation will be taken into account.

The RTE will not at this point focus on the impact of the operation, nor will it look into the details of the work of each sector of the operation, but will rather focus on the overall timeliness, efficiency, effectiveness, relevance and connectedness of the work done so far. The RTE team will focus on an overview of the operational areas, to see what has worked well, what has not worked well and what lessons can be learned for the future of the operation, including for the (early) recovery work.

Geographically, the RTE will focus on the areas of the capital (Yerevan) and the districts with the highest density of affected persons (at the time of writing, the regions [Marzes] of Ararat, Kotayk, and Armavir). Although other regions can be included, depending on the time available, the focus of the evaluation and the access to those areas.

The RTE is expected to take place in February-mid April2024, lasting 30 working days with 30% of the time of
the evaluation team to be spent in the field (in February 2024).

Evaluation criteria and key questions
The evaluation will focus on three objectives as below:
Objective 1: To focus on the combined Movement response (ARCS, IFRC, Partner NS and ICRC), and the effectiveness and efficiency of Movement coordination and collaboration, from the onset of the operation up to the time the evaluation team arrives in Armenia:
1. To what extent is an effective management structure in place[timeliness, effectiveness and efficiency of internal management and response], providing direction, clarity and wellcommunicated decision-making, within IFRC and ARCS?
2. Are there any specific recommendations to improve coordination levels?
3. How well does the Movement coordination setup correspond to SMCC and S2.0 approach? - To what extent were roles and responsibilities of Movement partners in country fulfilled vis-à-vis the respective guidance?
4. To what extent has the response demonstrated learning from previous responses of population movement from Karabakh in 2020 and 2022?
5. Are constraints and risks regularly identified and analyzed, and plans adapted accordingly?
6. How well are the M&E and learning processes used in the response and can they be improved?

Objective 2: To evaluate the connectedness of the operational specifically with systematic longer-term approaches and initiatives to strengthen the National Society systems, capacities and structure, both at the HQ and branch level
1. Have the capacity strengthening initiatives / activities provided within the operation, been
affective?
2. How well is the operation aligned with the longer-term NSD approaches and established structures, particularly related to Unified Planning process and New Way of Working (NWoW), integrating the EA into longer-term NSD plans?
3. How does the systems' agility that is in place from the outset, adapt as the context / needs evolve and how they support the assessment, planning and follow up of response delivery?
4. What organizational systems strengths can we build on, and what areas of improvement need to be addressed, specifically on ARCS and IFRC side?

Objective 3: To examine the relevance and appropriateness of the assistance provided to the affected population within the given context, building on organizational systems strengths, and to evaluate areas of improvement which need to be addressed on both ARCS and IFRC side:
1. Do the programme objectives correspond with identified needs? Do the prioritized intervention choices meet the most urgent needs of the affected population?
2. Did the response adapt to changes in need, capacities and context?
3. Have protection concerns been adequately addressed in the design of the response?
4. How can the provision of humanitarian assistance be improved to better respond to the needs, vulnerabilities and capacities of girls, boys, women, men, persons with disability and marginalized groups?
5. To what extent is assistance being delivered in a way that is cost effective?

Evaluation methodology & process
The evaluation will be managed according to the draft IFRC Real Time Evaluation Guide and Procedures. The methodology will adhere to the draft IFRC Management Policy for Evaluations, with particular attention to the processes upholding the standards of how evaluations should be planned, managed, conducted, and utilized. An IFRC / ARCS Evaluation Management Team (EMT) will oversee the evaluation.
The RTE team will consist of 3-4 people: 1 external evaluator, and 1-2 internal IFRC / Partner National Society evaluators, as well as an ARCS evaluator.
The specific evaluation methodology will be detailed in close consultation between the RTE team and IFRC evaluation management team, but will draw upon the following primary methods:
1. Desktop review of operation background documents, relevant organizational background and history
and any relevant sources of secondary data.
2. Field visits/observations to selected sites such as Yerevan and surrounding areas, Kotayk, Ararat and
Armavir.
3. Key informant interviews including those from ARCS, IFRC and Stakeholders.
4. Focus group discussions.

Evaluation deliverables & illustrative timeline

The following will be developed and delivered in line with the draft IFRC Real Time Evaluation Guide and Procedures:
• Evaluation work plan/ inception report will be submitted prior to the RTE team being deployed to the field.
• Validation workshop in the field to take place prior to the RTE team leaving the field and verbal debriefs with key stakeholders.
• Draft RTE report will be submitted within two weeks of the consultants’ return from the field.
• Final RTE report will be submitted one week after receipt of the consolidated feedback from IFRC and meet the requirements outlined in the IFRC RTE Guide and Procedures.

All products arising from this evaluation will be owned by the Federation. The evaluators will not be allowed, without prior authorization in writing, to present any of the analytical results as his or her own work or to make
use of the evaluation results for private publication purposes.

The following is an illustrative timeline that will be revisited and refined with more detail during the inception
stage of the RTE.

Evaluation quality & ethics
The evaluators should take all reasonable steps to ensure that the evaluation is designed and conducted to respect and protect the rights and welfare of the people and communities involved and to ensure that the evaluation is technically accurate and reliable, is conducted in a transparent and impartial manner, and contributes to organizational learning and accountability. Therefore, the evaluation team should adhere to the evaluation standards and applicable practices outlined in the IFRC Framework for Evaluation and respect the Red Cross Red Crescent 7 Fundamental Principles.

Team member will collectively have the following skill and experience; this set of skills also provide the basis for the selection of the external evaluation consultant: 
1) Demonstrable experience in leading evaluations of humanitarian programs responding to major disasters / crises, with specific experience in RTEs preferred.
2) Knowledge of strategic and operational management of humanitarian operations and proven ability to provide strategic recommendations to key stakeholders.
3) One team member with a good knowledge on RCRC Movement coordination mechanisms (SMCC, Seville 2.0).
4) One team member with knowledge / experience in Population Movement Operations.
5) One team member with strong National Society Strengthening background.
6) Preferably at least one team member with knowledge / experience in Southern Caucasus, or in other post-soviet countries.
7) Strong analytical skills and ability to clearly synthesize and present findings, draw practical conclusions, make recommendations and to prepare well-written reports in a timely manner.
8) Experience in qualitative data collection and data analysis techniques, especially in emergency operations.
9) Knowledge and experience working with the Red Cross Red Crescent Movement preferred
10) Demonstrated capacity to work both independently and as part of a multi-discipline, multi-national team.
11) Excellent English writing and presentation skills in English, with relevant writing samples of similar evaluation reports; Russian or Arminian language skills are an asset.

Application procedures

Interested candidates should submit their application material by 11 January 2024 via IFRC HR system or to the
following email: PMER.Europe@ifrc.org with the title Real Time Evaluation of the Armenia Population Movement Operation. Application material is non-returnable, and we thank you in advance for understanding that only short-listed candidates will be contacted for the next step in the application process.

Application materials should include:
1. Curricula Vitae (CV).
2. Cover letter clearly summarizing your experience as it pertains to this RTE, and three professional
references.
3. Financial proposal outlining an all-inclusive fee required to undertake this assignment including daily
consultancy rate and all costs for data collection and travel (based on the IFRC standard per diem rate
in Armenia).
4. At least one example of a previous evaluation report authored by the applicant most similar to that
described in this TOR.