Terminal Evaluation

Background

Introduction

In accordance with UNDP and GEF M&E policies and procedures, all full- and medium-sized UNDP-supported GEF-financed projects are required to undergo a Terminal Evaluation (TE) at the end of the project.  This Terms of Reference (ToR) sets out the expectations for the TE of the medium-sized project titled Strengthening targeted national capacities to improve decision-making and mainstreaming global environmental obligations into national development priorities (PIMS#00108967/00109706) implemented through UNDP Timor-Leste. The project started on 6 December 2018 and is in its fourth year of implementation.  The TE process must follow the guidance outlined in the document ‘Guidance For Conducting Terminal Evaluations of UNDP-Supported, GEF-Financed Projects’ http://web.undp.org/evaluation/guideline/documents/GEF/TE_GuidanceforUNDP-supportedGEF-financedProjects.pdf

Project Description

The overarching goal of this project is to strengthen capacity of Timor-Leste to meet their obligations under the three Rio Conventions.  This requires the country to have the necessary capacities to coordinate efforts as well as best practices for integrating global environmental priorities into planning, decision-making, and reporting processes.

The objective of the project is to strengthen institutional capacity to effectively manage information systems for national MEAs obligations and monitoring impact and progress. The project has four outcomes: (i) Enhanced institutional and technical capacities to use data and information for planning and decision-making on cross-cutting global environmental issues; (ii) Coordination of technical directorates, policy, planning, and programming (iii) Global environmental obligations mainstreamed into select sectoral policies, legislation, plans and programs; and (iv) Enhanced awareness and value of the global environment to meet socio-economic priorities.

The commitments Timor-Leste has made in signing the Rio Conventions are aligned with this constitutionally mandated mission and provide an overall framework for the country to achieve environment sustainability through application of international best practices.  Timor-Leste has taken additional steps to incorporate the Rio Conventions into domestic legal, policy, and regulatory frameworks in the relevant sectors.

To further assist Timor-Leste in effectively integrating the Rio Conventions into its legal, regulatory, and governance frameworks, the Government, in partnership with UNDP, conducted the National Capacity Self-Assessment (NCSA) to identify specific action items. The NCSA was a Global Environmental Facility (GEF) program intended to assist countries in assessing their environmental management capacities with a specific focus on the implementation of the Rio Conventions. Since completion of the NCSA, the Government of Timor-Leste has taken significant steps to integrate environmental and biodiversity considerations into its overall strategic development and legal frameworks. In 2010, Timor-Leste approved its Strategic Development Plan (SDP 2011-2030) which provides fundamental guidance for the country’s development stakeholders and is considered as the country’s sustainable development strategy. Soon after, the National Report on Sustainable Development in Timor-Leste (2011) was published. The Sustainable Development report concluded that though a widely accepted national census on the importance of sustainable development exists in the country, a sustainable development pathway can only be achieved with a strong political will, financial commitment, and collaborative stakeholders’ involvement.  

 

Timor-Leste also adopted the 2030 Agenda for Sustainable Development and the SDGs in 2015, which has several goals which are directly and indirectly relevant to the Rio Conventions, including: 

• SDG13: Take urgent action to combat climate change and its impacts
• SDG14: Conserve and sustainably use the oceans seas and marine resources for sustainable development; and 
• SDG15: Protect, restore and promote sustainable use of terrestrial ecosystems, sustainably manage forests, combat desertification, and halt and reverse land degradation and halt biodiversity loss.

This project further contributes to Timor-Leste’s United Nations Sustainable Development Cooperation Framework (UNSDCF/CPD) Outcome 6: “By 2025, national and sub-national institutions and communities (particularly at-risk populations including women and children) in Timor-Leste are better able to manage natural resources and achieve enhanced resilience to climate change impacts, natural and human-induced hazards, and environmental degradation, inclusively and sustainably”. The project specifically contributes to output 2.1 (Sustainable management of natural resources and ecosystems promoted through policies, guidelines, information systems, knowledge, and community-level conservation) and indicator 2.1.2 (Number of policies, strategies, plans, guidelines, systems related to sustainable environmental and marine ecosystem development prepared and submitted for approval) of the UNSDCF/CPD.

At the end of the project, the implemented activities are expected to have resulted into a set of targeted capacities in improving national stakeholders’ understanding of the three Rio Conventions, with a particular emphasis on good practices for planning and decision-making. Taking into account that a major barrier to Timor-Leste’s ability to meet and sustain Rio Convention obligations arises from inadequate access to best practices due to lack of technical language skills and opportunities to apply these within the framework of planning and decision-making, this project is strategically structured to overcome this underlying challenge. Towards this end, the project will enhance existing institutional structures and mechanisms to absorb new and improved data and information. Social actors working on various aspects of environmental governance, e.g., government staff and Non-Governmental Organization (NGO) representatives, among others, to better apply improved knowledge and skills to integrate global environmental objectives in national planning frameworks.

The COVID-19 pandemic and its associated restrictions in international and national travels have hampered project implementation. Due to airport closures around the world, the project could not engage international consultants. Furthermore, the State of Emergency and Health Fencing in Timor-Leste also caused delays in project implementation as movements were highly restricted. Implementation of project activities involving meetings, workshops, and training has been delayed due to travel restrictions and observance of social distancing. The project conducted a few meetings online and its international consultants are working on a remote basis assisted by the project team and national consultants. 

 

TE Purpose

The Terminal Evaluation (TE) will assess the achievements of the project against the expected objectives and outcomes as specified in the Project Document’s Strategic Results Framework and establish the project’s success or failure in meeting its goal and draw lessons that can both improve the sustainability of benefits from this project, and aid in the overall enhancement of GEF and UNDP programming. The TE will also review and document project results and their contribution towards achieving GEF strategic objectives aimed at global environmental benefits.  The findings of the TE will also contribute to strengthening the promotion of transparency and accountability for GEF-financed projects and help to synthesize lessons that can help to improve the selection, design and implementation of future UNDP-supported GEF-financed initiatives; and to improve the sustainability of benefits and aid in overall enhancement of UNDP programming. The outcome of the TE will be further used to gauge the extent of project convergence with other priorities within the UNDP country programme, including poverty alleviation, strengthening resilience to the impacts of climate change, reducing disaster risk and vulnerability, as well as cross-cutting issues such as gender equality, empowering women2 and supporting human rights.E

Duties and Responsibilities

TE consultant and/or team will:

1. Review all relevant sources of information including documents prepared during the preparation phase (i.e. PIF, UNDP Initiation Plan, UNDP Social and Environmental Screening Procedure/SESP) the Project Document, project reports including annual PIRs, project budget revisions, lesson learned reports, national strategic and legal documents, and any other materials that the team considers useful for this evidence-based evaluation;
2. Be required to review the baseline and midterm GEF focal area Core Indicators/Tracking Tools (i.e., project Results Tracker) submitted to the GEF at the CEO endorsement and midterm stages and the terminal Core Indicators/Tracking Tools (project Results tracker), if applicable, that must be completed before the TE field mission begins;
3. Be expected to follow a participatory and consultative approach ensuring close engagement with the Project Team, government counterparts (the GEF Operational Focal Point), Implementing Partners, the UNDP Country Office(s), the Regional Technical Advisors, direct beneficiaries and other stakeholders;
4. Review technical and managerial aspects and consider issues of effectiveness, efficiency, relevance, impact and sustainability. This review should be conducted throughout project’s components, strategy and approach against its objectives, output and outcome targets.
5. Identify factors that have facilitated and/or impeded the achievement of the project objectives and provide recommendations and lessons learned that will help project managers/project team in reorienting and re-prioritizing future and/or similar project and the managerial arrangements for future project designs. The TE should also consider all relevant factors including social and/or gender factors that could have hindered the achievement of the expected project results. Hence, gender lens should be applied in the whole approach and methodology of the evaluation.
6. Make recommendations for the project team, donors, government and partners and provide them with a clear understanding of the major outcomes and with a strategy and policy options for future projects to achieve expected results more effectively and efficiently.

Engagement of stakeholders is vital for a successful TE interviews will be arranged to include, but not limited, to the following stakeholders who have responsibilities to the project:

• Secretary of State for Environment, key partner: Directorate General for Environment, National Directorate for Biodiversity, National Directorate for Climate Change, National Directorate for Centre of Environmental Information and Education, National Directorate for Pollution Control, and National Agency for Environmental Licensing)
• Ministry of Agriculture and Fisheries (MAF): Directorate General for Forestry, National Directorate for Research and Geographic Information.
• Ministry of State Administration (MSA); National Directorate for Urbanization
• Ministry of Public Works (MPW); National Directorate for Meteorology and Geophysics
• Ministry of Commerce, Industry and Tourism (MCIT); Directorate General for Tourism
• Ministry of Health (MoH); National Directorate for Public Health-Department of Environmental Health
• Ministry of Finance (MoF)- Directorate General for Statistics

 

Expected Outputs and Deliverables

	Deliverables	Due Dates
D1	TE Inception Report: TE team clarifies objectives and methods of the TE no later than 2 weeks before the TE mission. TE team submits the Inception Report to the Commissioning Unit and project management. Approximate due date 6 June 2022	June 6, 2022
D2	Presentation: TE team presents initial findings to project management and the Commissioning Unit at the end of the TE mission.	July 1, 2022
D3	Draft TE Report: TE team submits full draft report with annexes within 3 weeks of the end of the TE mission. Approximate due date: 22 July 2022	July 22, 2022
D4	Final TE Report and Audit Trail: TE team submits revised report, with Audit Trail detailing how all received comments have (and have not) been addressed in the final TE report, to the Commissioning Unit within 1 week of receiving UNDP comments on draft.	August 12, 2022

 

INSTITUTIONAL ARRANGEMENTS

The principal responsibility for managing the TE resides with the Commissioning Unit.  The Commissioning Unit for this project’s TE is the UNDP Country Office in Timor Leste.

 

DUTY STATION AND DURATION OF THE ASSIGNMENT:

The duty station of the assignment: Dili, Timor-Leste. The duration of the assignment is 35 days over a period of 4 months (May – September 2022)

Competencies

-

Required Skills and Experience

Education

• Recognized university degree (minimum Bachelor’s degree level) in a subject related to environmental education, environmental law, environmental science, environmental management, sustainable development, international development, or a closely relevant field.

Experience

• Minimum of 5 years of progressively demonstrable relevant experience with results-based management evaluation methodologies;
• Experience applying SMART indicators and reconstructing or validating baseline scenarios;
• Competence in adaptive management, as applied to cross-cutting capacity development with particular emphasis on Rio Conventions
• Experience in evaluating projects;
• Experience working in Timor-Leste;
• Experience in relevant technical areas for at least 3-5 years;
• Demonstrated understanding of issues related to gender and cross-cutting capacity development with particular emphasis on Rio Conventions, experience in gender-responsive evaluation and analysis;
• Excellent communication skills;
• Demonstrable analytical skills;
• Project evaluation/review experience within United Nations system will be considered an asset;

Language

• Fluency in written and spoken Tetun and English;
• Ability to communicate in Portuguese, Bahasa Indonesia and/or other local languages considered an asset.

Scope of Price Proposal and Schedule of Payments

Financial Proposal:

• Financial proposals must be “all inclusive” and expressed in a lump-sum for the total duration of the contract. The term “all inclusive” implies all cost (professional fees, travel costs, living allowances etc.);
• For duty travels, the UN’s Daily Subsistence Allowance (DSA) rates are (N/A) , which should provide indication of the cost of living in a duty station/destination (Note: Individuals on this contract are not UN staff and are therefore not entitled to DSAs.  All living allowances required to perform the demands of the ToR must be incorporated in the financial proposal, whether the fees are expressed as daily fees or lump sum amount.

Schedule of Payment

• The lump sum is fixed regardless of changes in the cost components.

• 20% payment upon satisfactory delivery of the final TE Inception Report and approval by the Commissioning Unit
• 40% payment upon satisfactory delivery of the draft TE report to the Commissioning Unit
• 40% payment upon satisfactory delivery of the final TE report and approval by the Commissioning Unit and RTA (via signatures on the TE Report Clearance Form) and delivery of completed TE Audit Trail

 Final payment of 40%

• The final TE report includes all requirements outlined in the TE TOR and is in accordance with the TE guidance.
• The final TE report is clearly written, logically organized, and is specific for this project (i.e. text has not been cut & pasted from other MTR reports).
• The Audit Trail includes responses to and justification for each comment listed.

 

Recommended Presentation of Proposal

1. Letter of Confirmation of Interest and Availability using the template provided by UNDP;
2. CV and a Personal History Form (P11 form);
3. Brief description of approach to work/technical proposal of why the individual considers him/herself as the most suitable for the assignment, and a proposed methodology on how they will approach and complete the assignment; (max 1 page)
4. Financial Proposal that indicates the all-inclusive fixed total contract price and all other travel related costs (such as flight ticket, per diem, etc.), supported by a breakdown of costs, as per template attached to the Letter of Confirmation of Interest template. If an applicant is employed by an organization/company/institution, and he/she expects his/her employer to charge a management fee in the process of releasing him/her to UNDP under Reimbursable Loan Agreement (RLA), the applicant must indicate at this point, and ensure that all such costs are duly incorporated in the financial proposal submitted to UNDP.

 

Criteria for Selection of the Best Offer

Only those applications which are responsive and compliant will be evaluated. Offers will be evaluated according to the Combined Scoring method – where the educational background and experience on similar assignments will be weighted at 70% and the price proposal will weigh as 30% of the total scoring. The applicant receiving the Highest Combined Score that has also accepted UNDP’s General Terms and Conditions will be awarded the contract.

Detail Evaluation Criteria

Technical Criteria	Maximum obtainable points 70	Weight Percentage 70%
Education:
Recognized university degree (minimum Bachelor’s degree level) in a subject related to environmental education, environmental law, environmental science, environmental management, sustainable development, international development, or a closely relevant field.	10	10%
Experience
Minimum of 5 years of progressively demonstrable relevant experience with results-based management evaluation methodologies;	5	55%
Experience applying SMART indicators and reconstructing or validating baseline scenarios;	5
Competence in adaptive management, as applied to cross-cutting capacity development with particular emphasis on Rio Conventions	5
Experience in evaluating projects;	10
Experience working in Timor-Leste;	5
Experience in relevant technical areas for at least 3-5 years;	5
Demonstrated understanding of issues related to gender and cross-cutting capacity development with particular emphasis on Rio Conventions, experience in gender-responsive evaluation and analysis;	5
Excellent communication skills;	5
Demonstrable analytical skills;	5
Project evaluation/review experience within United Nations system will be considered an asset;	5
Language
Fluency in written and spoken Tetun and English; Ability to communicate in Portuguese, Bahasa Indonesia and/or other local languages considered an asset.	5	5%
Total technical score		70%
Financial	30	30 %
Final Score	100	100%

Annexes (click on the hyperlink to access the documents):

• Annex 1 - IC Contract Template (for information);

• Annex 2 -  Financial Proposal Template using the template provided by UNDP (to be completed at later stage upon request by UNDP);
• Annex 3 – IC General Terms and Conditions (for information);
• Annex 4 – RLA Template (if consultant wishes to be recruited through an employer) - (for information).
• Annex 5 - ToR

Incomplete applications will not be considered and will be disqualified automatically.