International Consultant for the Terminal Evaluation of full-sized UNDP-GEF project

Background

Introduction:

In accordance with UNDP and GEF M&E policies and procedures, all full- and medium-sized UNDP-supported GEF-financed projects are required to undergo a Terminal Evaluation (TE) at the end of the project. This Terms of Reference (TOR) sets out the expectations for the TE of the full-sized project titled “Supporting Climate Resilient Livelihoods in Agricultural Communities in Drought-Prone Areas of Turkmenistan” (PIMS#5459) implemented through the Ministry of Agriculture and Environmental Protection of Turkmenistan. The project started on 15 June 2016 and is in its 5th year of implementation. The TE process must follow the guidance outlined in the document ‘Guidance for Conducting Terminal Evaluations of UNDP-Supported, GEF-Financed Projects: (http://web.undp.org/evaluation/guideline/documents/GEF/TE_GuidanceforUNDP-supportedGEF-financedProjects.pdf).

Project Description:

Turkmenistan is a water stressed country and has one of the harshest climates in the Central Asian region. Climate change modeling indicates significant increases in temperature and reduction in rainfall. Temperatures are expected to increase by 2°C by 2040, with precipitation declining across all agro-ecological zones by 8-17 percent between 2040 and 2100, which coupled with increase in temperature, will lead to a decrease in total volume of water availability that is likely to have a profound impact on agricultural production systems and local farmers. The long-term solution envisaged by the Government of Turkmenistan is to mainstream climate change adaptation at the community, district, provincial and national levels in order to secure climate resilient livelihoods in agricultural communities.

The project, ‘Supporting climate resilient livelihoods in agricultural communities in drought-prone areas of Turkmenistan’ started in June 2016 and is scheduled to finish in March 2022. The project is financed by the Special Climate Change Fund (SCCF), managed by the Global Environment Facility, and implemented through the United Nations Development Programme. Through knowledge management, investment, and policy reform, this project seeks to promote an integrated approach to efficient water management and climate resilient practices, reduces root causes of land degradation, and enhances local livelihoods. Grant financing from the SCCF amounts of US$ 3.046 million. Co-financing of US$20.0 million has been committed from various sources.

Through various interventions, the project aims to directly strengthen the adaptive capacity and reduce the vulnerability of around 40,000 to 50,000 persons (of which around 51.2% would be women) in the Lebap and Dashoguz velayats (provinces). Improved water efficiency and crop production systems will bring approximately 20,000 ha of agricultural and 500,000 ha of pastoral lands under climate resilient technologies resulting in a real net household income increase of at least 15 % for participating households.

The project’s activities are organized into three components:

• Component 1 will improve climate related socio-economic outcomes in targeted agricultural communities in Lebap and Dashoguz velayats through the implementation of community-based adaptation solutions;
• Component 2 will mainstream climate adaptation measures in agricultural and water sector development strategy and policy;
• Component 3 will strengthen national capacity for iterative climate change adaptation planning, implementation and monitoring in the country.

Note: During the COVID pandemic lockdown in 2020, on April 6, 2021 the UNDP GEF Executive Coordinator approved the request of a six-months no-cost project extension considering the impact to the project pace. The closing date of the project shifted from September 17, 2021, to March 30, 2022.

TE purpose:

The TE report will assess the achievement of project results against what was expected to be achieved and draw lessons that can both improve the sustainability of benefits from this project, and aid in the overall enhancement of UNDP programming. The TE report promotes accountability and transparency and assesses the extent of project accomplishments.

The final evaluation will focus on the delivery of the project’s results as initially planned (and as corrected after the mid-term evaluation). The final evaluation will look at impact and sustainability of results, including the contribution to capacity development and the achievement of global environmental benefits/goals.

The Terminal Evaluation should also provide recommendations for follow-up activities, which require a management response prepared by the project team, which should be uploaded to PIMS and to the UNDP Evaluation Office Evaluation Resource Center (ERC). 

TE Approach and Methodology:

The TE report must provide evidence-based information that is credible, reliable and useful.

The TE team will review all relevant sources of information including documents prepared during the preparation phase (i.e. PIF, UNDP Initiation Plan, UNDP Social and Environmental Screening Procedure/SESP) the Project Document, project reports including annual PIRs, project budget revisions, lesson learned reports, national strategic and legal documents, and any other materials that the team considers useful for this evidence-based evaluation. The TE team will review the baseline and midterm GEF focal area Core Indicators/Tracking Tools submitted to the GEF at the CEO endorsement and midterm stages and the terminal Core Indicators/Tracking Tools that must be completed before the TE field mission begins. 

The TE team is expected to follow a participatory and consultative approach ensuring close engagement with the Project Team, government counterparts (the GEF Operational Focal Point), Implementing Partners, the UNDP Country Office(s), the Regional Technical Advisor, direct beneficiaries and other stakeholders.

Engagement of stakeholders is vital to a successful TE. Stakeholder involvement should include (where possible, given the COVID situation) surveys/questionnaires, focus group meetings and interviews with stakeholders who have project responsibilities, including but not limited to executing agencies, senior officials and task team/component leaders, key experts and consultants in the subject area, Project Board members, project beneficiaries, academia, local government and CSOs, etc. Additionally, the TE team is expected to conduct field missions to Turkmenistan, including the project sites in the Danew and Gorogly districts of Lebap and Dashoguz provinces, to meet with the local communities and beneficiaries if permitted by UNDP as per the SOPs under COVID-19 pandemic as well as COVID-19 regulations by the Government of Turkmenistan.

The specific design and methodology for the TE should emerge from consultations between the TE team and the above-mentioned parties regarding what is appropriate and feasible for meeting the TE purpose and objectives and answering the evaluation questions, given limitations of budget, time and data. The TE team must use gender-responsive methodologies and tools and ensure that gender equality and women’s empowerment, as well as other cross-cutting issues and SDGs are incorporated into the TE report.

As of 11 March 2020, the World Health Organization (WHO) declared COVID-19 a global pandemic as the new coronavirus rapidly spread to all regions of the world. Travel to Turkmenistan is restricted and travel in the country is also restricted, to follow UNDSS SOPs for domestic travels which may not permit UNDP contract holder to travel. If it is not possible to travel to or within the country for the TE mission then the TE team should develop a methodology that takes this into account the conduct of the TE virtually and remotely, including the use of remote interview methods and extended desk reviews, data analysis, surveys, and evaluation questionnaires. This should be detailed in the TE Inception Report and agreed with the Commissioning Unit. In case that the international consultant cannot travel to Turkmenistan, he/she will work remotely from his/her home country and provide guidance to, and receive support from, the National Consultant, who will do the field visit to the sites (if the travel is permitted). However, if the international consultant manages to come to Ashgabat, Turkmenistan, then his/her travel to other cities or project regions may or may not be materialized keeping current COVID-19 scenario in mind. The final decision to be taken at an appropriate time; the dates closer to the mission dates. No stakeholders, consultants or UNDP staff should be put in harm’s way and safety is the key priority.

If all or part of the TE is to be carried out virtually then consideration should be taken for stakeholder availability, ability, or willingness to be interviewed remotely. In addition, their accessibility to the internet/computer may be an issue as many government and national counterparts may be working from home. These limitations must be reflected in the final TE report.

The final methodological approach including interview schedule, field visits and data to be used in the evaluation must be clearly outlined in the TE Inception Report and be fully discussed and agreed between UNDP, stakeholders and the TE team.

The final report must describe the full TE approach taken and the rationale for the approach making explicit the underlying assumptions, challenges, strengths and weaknesses about the methods and approach of the evaluation. The final report must also describe any limitations encountered by the Evaluation team during the evaluation process, including limitations of the methodology, data collection methods, and any potential influence of limitation on how findings may be interpreted, and conclusions drawn. Limitations include, among others: language barriers, inaccessible project sites (due to travel restrictions because of COVID), issues with access to data or verification of data sources, issues with availability of interviewees to engage in in- person meetings or virtually in the evaluation process, methodological limitations to collecting more extensive or more representative qualitative or quantitative evaluation data, deviations from planned data collection and analysis set out in the TOR and Inception Report, etc. Efforts made to mitigate the limitations should also be included in the Interim Evaluation report.

Duties and Responsibilities

Detailed Scope of the TE:

The TE will assess project performance against expectations set out in the project’s Logical Framework/Results Framework (All relevant templates will be shared with winner). The TE will assess results according to the criteria outlined in the Guidance for TEs of UNDP-supported GEF-financed Projects http://web.undp.org/evaluation/guideline/documents/GEF/TE_GuidanceforUNDP-supportedGEF-financedProjects.pdf .

The Findings section of the TE report will cover the topics listed below. The asterisk “(*)” indicates criteria for which a rating is required.

Findings:

i. Project Design/Formulation:

• National priorities and country driven-ness;
• Project Strategy and objectives;
• Gender equality and women’s empowerment;
• Social and Environmental Safeguards;
• Analysis of Results Framework: project logic and strategy, indicators;
• Assumptions and Risks;
• Lessons from other relevant projects (e.g. same focal area) incorporated into project design;
• Planned stakeholder participation;
• Linkages between project and other interventions within the sector;
• Management arrangements;

ii. Project Implementation:

• Adaptive management (changes to the project design and project outputs during implementation);
• Actual stakeholder participation and partnership arrangements;
• Project Finance and Co-finance;
• Monitoring & Evaluation: design at entry (*), implementation (*), and overall assessment of M&E (*);
• Implementing Agency (UNDP) (*) and Executing Agency (*), overall project oversight/implementation and execution (*);
• Risk Management, including Social and Environmental Standards;

iii. Project Results:

• Assess the achievement of outcomes against indicators by reporting on the level of progress for each objective and outcome indicator at the time of the TE and noting final achievements;
• Relevance (*), Effectiveness (*), Efficiency (*) and overall project outcome (*);
• Sustainability: financial (*), socio-political (*), institutional framework and governance (*), environmental (*), overall likelihood of sustainability (*);
• Country ownership;
• Gender equality and women’s empowerment;
• Cross-cutting issues (improved governance, climate change mitigation and adaptation, capacity development, knowledge management, etc., as relevant);
• GEF Additionality;
• Catalytic Role / Replication Effect;
• Progress to impact;

iv. Main Findings, Conclusions, Recommendations and Lessons Learned:

• The TE team will include a summary of the main findings of the TE report. Findings should be presented as statements of fact that are based on analysis of the data;
• The section on conclusions will be written in light of the findings. Conclusions should be comprehensive and balanced statements that are well substantiated by evidence and logically connected to the TE findings. They should highlight the strengths, weaknesses and results of the project, respond to key evaluation questions and provide insights into the identification of and/or solutions to important problems or issues pertinent to project beneficiaries, UNDP and the GEF, including issues in relation to gender equality and women’s empowerment;
• Recommendations should provide concrete, practical, feasible and targeted recommendations directed to the intended users of the evaluation about what actions to take and decisions to make. The recommendations should be specifically supported by the evidence and linked to the findings and conclusions around key questions addressed by the evaluation;
• The TE report should also include lessons that can be taken from the evaluation, including best practices in addressing issues relating to relevance, performance and success that can provide knowledge gained from the particular circumstance (programmatic and evaluation methods used, partnerships, financial leveraging, etc.) that are applicable to other GEF and UNDP interventions. When possible, the TE team should include examples of good practices in project design and implementation;
• It is important for the conclusions, recommendations and lessons learned of the TE report to include results related to gender equality and empowerment of women;

The TE report will include an Evaluation Ratings Table.

Expected Outputs and Deliverables:

The TE Consultant shall prepare and submit:

• TE Inception Report: TE team clarifies objectives and methods of the TE no later than 2 weeks before the TE mission or online interviews with relevant stakeholders. TE team submits the Inception Report to the Commissioning Unit and project management. Approximate due date: (19/10/2021);
• Presentation: TE team presents initial findings to project management and the Commissioning Unit at the end of the TE mission or online interviews with relevant stakeholders and counterparts. Approximate due date: (09/11/2021);
• Draft TE Report: TE team submits full draft report with annexes within 3 weeks of the end of the TE mission or online interviews with relevant stakeholders. Approximate due date: (01/12/2021);
• Final TE Report* and Audit Trail: TE team submits revised report, with Audit Trail detailing how all received comments have (and have not) been addressed in the final TE report, to the Commissioning Unit within 1 week of receiving UNDP comments on draft. Approximate due date: (10/12/2021);

*The final TE report must be in English. If applicable, the Commissioning Unit may choose to arrange for a translation of the report into a language more widely shared by national stakeholders. All final TE reports will be quality assessed by the UNDP Independent Evaluation Office (IEO). Details of the IEO’s quality assessment of decentralized evaluations can be found in Section 6 of the UNDP Evaluation Guidelines. http://web.undp.org/evaluation/guideline/ .

TE Arrangements:

The principal responsibility for managing the TE resides with the Commissioning Unit. The Commissioning Unit for this project’s TE is the UNDP Country Office in Turkmenistan.

The Commissioning Unit will contract the evaluators and ensure the timely provision of per diems and travel arrangements within the country (if applicable) for the TE team. The Project Team will be responsible for liaising with the TE team to provide all relevant documents, set up stakeholder interviews, and arrange field visits.

Duration of the Work:

The total duration of the TE will be approximately 26 working days over a time period of 9 weeks starting on 10/10/2021. The tentative TE timeframe is as follows:

• 10/09/2021: Application closes;
• 05/10/2021: Selection of TE Team;
• 10/10/2021: Preparation of the TE team (handover of project documents);
• 10/10/2021: 4 days: Document review and preparing TE Inception Report;
• 19/10/2021: 1 day: Finalization and Validation of TE Inception Report- latest start of TE mission;
• 28/10/2021: 10 days: TE mission or online consultations: stakeholder meetings, interviews, field visits or online interviews with relevant stakeholders and counterparts;
• 09/11/2021: 1 day: Mission wrap-up meeting & presentation of initial findings- earliest end of TE mission;
• 15/11/2021: 8 days: Preparation of draft TE report;
• 01/12/2021: 1 day: Circulation of draft TE report for comments;
• 10/12/2021: 1 day: Incorporation of comments on draft TE report into Audit Trail & finalization of TE report;
• 15/12/2021: Preparation & Issue of Management Response;
• 06/01/2022: Expected date of full TE completion;

The expected date start date of contract is 10/10/2021.

Duty Station:

The TE assignment is expected to be home-based in case if current travel restrictions due to COVID-19 pandemic are not eased. However, in case if travel restrictions are eased it is absolutely essential that shortly after the start of the assignment the international consultant travels to Turkmenistan for a period of 2 weeks (10 working days, not including weekends) to meet with all relevant stakeholders and conducts field missions to Gorogly and Danew project sites.

Travel:

• International travel might be required to Turkmenistan during the TE mission;
• The BSAFE course must be successfully completed prior to commencement of travel;
• Individual Consultants are responsible for ensuring they have vaccinations/inoculations when travelling to certain countries, as designated by the UN Medical Director;
• Consultants are required to comply with the UN security directives set forth under: https://dss.un.org/dssweb/ ;
• All related travel expenses should be included to the Letter of Confirmation of Interest and Availability.

Evaluator Ethics:

The TE team will be held to the highest ethical standards and is required to sign a code of conduct upon acceptance of the assignment. This evaluation will be conducted in accordance with the principles outlined in the UNEG ‘Ethical Guidelines for Evaluation’. The evaluator must safeguard the rights and confidentiality of information providers, interviewees and stakeholders through measures to ensure compliance with legal and other relevant codes governing collection of data and reporting on data. The evaluator must also ensure security of collected information before and after the evaluation and protocols to ensure anonymity and confidentiality of sources of information where that is expected. The information knowledge and data gathered in the evaluation process must also be solely used for the evaluation and not for other uses without the express authorization of UNDP and partners.

Payment Schedule:

• 20% payment upon satisfactory delivery of the final TE Inception Report and approval by the Commissioning Unit;
• 40% payment upon satisfactory delivery of the draft TE report to the Commissioning Unit;
• 40% payment upon satisfactory delivery of the final TE report and approval by the Commissioning Unit and RTA (via signatures on the TE Report Clearance Form) and delivery of completed TE Audit Trail;

Criteria for issuing the final payment of 40%:

• The final TE report includes all requirements outlined in the TE TOR and is in accordance with the TE guidance;
• The final TE report is clearly written, logically organized, and is specific for this project (i.e. text has not been cut & pasted from other TE reports);
• The Audit Trail includes responses to and justification for each comment listed;

Competencies

A team of two independent evaluators will conduct the TE – one team leader (with experience and exposure to projects and evaluations in other regions) and one local team expert. The team leader will be responsible for the overall design and writing of the TE report. The team expert will assess emerging trends with respect to regulatory frameworks, budget allocations, capacity building, work with the Project Team in developing the TE itinerary, etc.

The evaluator(s) cannot have participated in the project preparation, formulation and/or implementation (including the writing of the project document), must not have conducted this project’s Mid-Term Review and should not have a conflict of interest with the project’s related activities.

The selection of evaluators will be aimed at maximizing the overall “team” qualities in the following areas:

Corporate Competencies:

• Demonstrates integrity by modeling the UN’s values and ethical standards;
• Promotes the vision, mission, and strategic goals of UNDP;
• Displays cultural, gender, religion, race, nationality and age sensitivity and adaptability;
• Treats all people fairly without favoritism;
• Fulfills all obligations to gender sensitivity and zero tolerance for sexual harassment.

Functional Competencies:

• Competence in adaptive management;
• Knowledge of and work experience in the climate change adaptation related water and agriculture projects, including those funded by the GEF;
• Excellent training, facilitation and communication skills;
• Results driven, ability to work under pressure and to meet required deadlines;
• Excellent communication and presentation skills;
• Demonstrable analytical skills.

Required Skills and Experience

Education:

• Master’s degree in natural sciences, natural resource and environmental management, development studies, economics, climate change mitigation and adaptation or other closely related field;

Experience:

• Relevant extensive (at least 10-year) work experience and proven track record with policy advice and/or project development/implementation in climate change and or water efficiency (including at least some experience with climate change and/or water projects) in transition economies is required;
• Experience working with the GEF or GEF project evaluations within the past seven years including experience with SMART based indicators is required;
• Experience working with international technical assistance projects in the Eastern Europe countries or CIS region in the past seven years (experience in Turkmenistan will be an asset) is required;
• Demonstrated understanding of issues related to gender and Climate Change; experience in gender responsive evaluation and analysis;
• Project evaluation/review experience within United Nations system will be considered an asset;
• Experience with implementing evaluations remotely will be considered an asset.

Language requirements:

• Fluency in written and spoken English is required, knowledge of Russian will be an asset.

Application Process:

Interested individual consultants must submit the following documents/information to demonstrate their qualifications:

All experts applying for this position are required to provide:

• Brief description of approach to work/technical proposal of why the individual considers him/herself as the most suitable for the assignment, and a proposed methodology on how they will approach and complete the assignment (max 1 page);
• Financial Proposal with the references to (1) the daily rate for the assignment and within the timing scale indicated in the present TOR, and (2) any other expenses (including transportation costs, accommodation costs, the possibility of vaccination and etc.). Template of the Letter of Confirmation of Interest and Availability can be found at: http://www.tm.undp.org/content/turkmenistan/en/home/procurement.html. The UNDP will enter into an Individual Contract based on a lump sum amount. The financial proposal shall represent a detailed, justified and “all inclusive” amount.  In order to assist UNDP in the comparison of financial proposals, the financial proposal shall include a breakdown of this lump sum amount, including: a daily fee for the tasks and an estimated duration as specified in this announcement, travel (to and from the missions), per diems, any other possible costs (including vaccinations, dwelling, communication etc.);
• Cover letter explaining why they are the most suitable candidate for the assignment;
• Resume /CV.

Note (Conflict of Interest): Any individual who participated in the project preparation, formulation, and/or implementation (including the writing of the Project Document) is ineligible to participate in this bidding.

Only those applications which are responsive and compliant will be evaluated. Offers will be evaluated according to the Combined Scoring method – where the educational background and experience on similar assignments will be weighted at 70% and the financial proposal will weigh as 30% of the total scoring. The applicant receiving the Highest Combined Score that has also accepted UNDP’s General Terms and Conditions will be awarded the contract.

When using this method, the award of the contract should be made to the individual consultant whose offer has been evaluated and determined as:

• Responsive/compliant and having received the highest score – out of 100 points.

Out of the maximum score, the score for technical criteria equals 70% - maximum 70 points, and for financial criteria 30%.

The technical evaluation will take into account the following as per the scoring provided:

• Educational background (Advanced University degree, Masters or preferably a PhD, in Natural sciences, Natural resource and environmental management, development studies, economics, climate change mitigation and adaptation or other closely related field) – 10 points max; (PhD related to Natural Sciences/Environment Mgt/Natural Resources/Climate Change = 10 points, PhD related to other relevant topic = 8 points, Masters related to Natural Sciences /Environment = 6 points, Masters related to other relevant topic = 4 points, combined (2 or more) Masters related to relevant topics = 8);
• Extensive (at least 10-year) work experience and proven track record with policy advice and/or project development/implementation in climate change adaptation and/or sustainable use of land and water resources (including at least some experience with climate change adaptation and/or sustainable use of land and water resources projects) in transition economies – 20 points max (more points if experience specifically includes experience related to both climate change and/or sustainable use of land and water resources; more than 20 years = 17 points, 15-20 years = 12 points, 14-10 years = 7 points, 6-9 years = 2 points.) The consultant shall score +3 points if they have specific work experience related to other projects dealing with the issues of both climate change and also specifically related to sustainable use of land and water resources. If the consultant has only specific experience related to one of these two areas then they shall score +1 point;
• Experience working with the GEF or GEF project evaluations within the past seven years including experience with SMART based indicators (Project evaluation/review experiences within United Nations system will be considered an asset) – 20 points max (excellent evidences of the required experience = 20 points (3 assignments or more); very good evidence (2 or more assignments) = 14 points satisfactory evidences (1 other relevant GEF evaluation experience) = 8 points; no evidence of ever having evaluated a GEF project = 0 points);
• Experience working with international technical assistance projects in the Eastern Europe countries or CIS region in the past seven years (experience in Turkmenistan will be an asset and persons who have worked in Turkmenistan before on technical assistance projects will score 10 points) – 20 points max (strong experience (4 assignments or more or at least 1 prior assignment in Turkmenistan) = 20 points; very good experience (3 other assignments or more) = 14 points, good experience (2 assignments or more) = 8 points, satisfactory experience (1 assignment or more) – 6 points, no experience = 0);
• Methodology on how IC will approach and complete the assignment – 10 points max;
• Interview – 10 points max;
• Language skills (English required, knowledge of Russian will be an asset) – 10 points max (10 points for superior writing and oral skills in English + at least some knowledge of Russian; 7 points for superior writing and oral skills in English but no Russian, 4 points for average English and satisfactory writing skills, 1 point for poor English fluency and poor writing skills). Writing skills will be judged by the quality of the 1-page cover letter with the brief description of the approach to the work to be carried out to be sent with this application.

Maximum available technical (education, experience and competencies) score – 100 points.

Templates of the following forms will be shared with the winner:

• ToR Annex A: Project Logical/Results Framework;
• ToR Annex B: Project Information Package to be reviewed by TE team;
• ToR Annex C: Content of the TE report;
• ToR Annex D: Evaluation Criteria Matrix template;
• ToR Annex E: UNEG Code of Conduct for Evaluators;
• ToR Annex F: TE Rating Scales and TE Ratings Table;
• ToR Annex G: TE Report Clearance Form;
• ToR Annex H: TE Audit Trail template;

Additional requirements for recommended contractor:

Recommended contractors aged 65 and older, and if the travel is required, shall undergo a full medical examination including x-ray, and obtain medical clearance from the UN-approved doctor prior to taking up their assignment. The medical examination is to be cleared by the UN physicians, and shall be paid by the consultant.