Independent Evaluator of GGGI’s Mexico Country Program: Transport Specialist

Contract
Project	Independent Evaluation of GGGI’s Mexico Country Program: Transport Specialist – Evaluator Consultant
Expertise
Category	Advisory Support: level F
Location
Duty Station	Global
Contract Length
Start Date	23 October 2017
End Date	31 January 2018
Part-time/Full-time	Part time consultant
Contract Value
Daily Rate	NA
Days Estimated	35.5 days
Total Fees	Up to USD 26,625 (dependent on experience)
Specifics of Recruitment
Closing date: 8 October 2017
Apply at: ##

Introduction:

The Global Green Growth Institute (GGGI) was established in 2012 to support countries to pursue a model of economic development known as “green growth”, which aims to simultaneously promote economic growth, poverty reduction, social inclusion and environmental sustainability. To achieve this, GGGI works with partner countries to design and deliver programs that demonstrate practical pathways to achieve green growth.

The Institute is overseen by an Assembly, comprising 27 members, a Council which serves as the executive organ of GGGI and the Management and Program Sub-Committee (MPSC) which advises the Council on financial and programmatic matters. GGGI is headquartered in Seoul, Republic of Korea, with operations in over 25 countries.

In November 2014, the Council approved the GGGI Strategic Plan 2015-2020, which sets strategic priorities and directions for the organization over a 6-year period. Based on the theory of change identified, 3 strategic outcomes were identified for the organization:

• Outcome 1: Strengthened national, sub-national, local green growth policy planning, financing and institutional frameworks;
• Outcome 2: Increased green investment flows; and
• Outcome 3: Improved multi-directional knowledge sharing and learning between countries on green growth.

The Strategic Plan also prioritized 4 thematic areas considered key to enabling countries to pursue green growth: energy, water, land-use and green cities.

To implement the Strategic Plan, a biennial Work Program and Budget (WPB) is developed and approved by the Council. This outlines a portfolio of country and global programs, and supporting corporate reforms, to be implemented over a 2-year period. The first WPB period covered 2015-2016 and GGGI is currently implementing its second biennial WPB during 2017-2018.

Implementation of country and global programs under the WPB is largely undertaken by two divisions who work in a complementary and integrated way:

• Green Growth Planning & Implementation division (GGPI), which leads the development and implementation of GGGI’s in-country green growth programs; and
• Investment & Policy Solutions[1] division (IPSD), which designs and delivers specialist products and services to support the development and financing of green growth policies.

In addition, GGGI’s Office of the Director-General (ODG) and the Operations Enabling Division (OED)[2] manage a range of functions and initiatives in non-programmatic and corporate areas.

More information about GGGI is available at: www.gggi.org.

The Impact and Evaluation Unit (IEU) is responsible for commissioning country and thematic evaluations to assess the impacts of GGGI’s work and to inform improvements to program design and delivery. In 2017, an evaluation of GGGI’s Mexico country program was commissioned as part of IEU’s workplan.

 

Overview of the Mexico Country Program:

Mexico is one of GGGI’s founding member countries and a contributing member. GGGI’s country team is based in Mexico City and has been working with the Government of Mexico (GoM) since 2013 in the assessment, development and implementation of public policies that promote green growth. GGGI has been involved in the provision of both technical assistance to specific projects, as well as the support of knowledge sharing between Mexico and countries in the region. GGGI works closely with national and sub-national governments and other strategic partners through its principal counterpart in the GoM, the Secretariat of the Environment and Natural Resources (SEMARNAT).

An outline of the main program areas of work since the inception of the country program is highlighted below:

1. Green Growth Planning (2013-Ongoing):
   1. Technical inputs provided to the National Climate Change Strategy and Special Climate Change Program. The inputs included an approach to determining the level vulnerability and prioritization of municipalities to implement adaptation measures.
   2. Development and implementation of the Green Growth Strategy for the State of Sonora is an ongoing project.
   3. Air quality (2013-2015):
      1. Supported the establishment of the Environmental Commission of the Megalopolis (CAMe) and various activities to improve the management of air quality in the Megalopolis region^[3]. This included strengthening air quality monitoring networks and reporting platforms, air quality index, contingency plans and vehicle emission testing program among others.
      2. Specific studies to evaluate economic instruments to promote the use of public transport and cleaner and more efficient vehicles (specifically through fuel surcharges) were commissioned by GGGI in partnership with GoM and United States Agency for International Development (USAID). 
      3. Public transport (2015-Ongoing):
         1. Compiled best practice concession models from 7 international cities along with key lessons to establish a concession model to transform the bus transport system in the State of Morelos.
         2. Provided recommendations on technical aspects of a new mobility law in the State of Morelos which provides a legal framework to introduce a new concession model for bus operations.
         3. Developed a tool for economic options modelling for the State of Morelos to determine the break-even point for bus operations and facilitate decisions on the type of operating model and tariffs that can be charged for the services.
         4. Provided recommendations to leverage an existing megalopolitan trust fund to finance bus fleet renewal in the Megalopolis region.
         5. Developed a transport externalities calculator to determine the environmental, economic and social externalities of public transport under different intervention scenarios. A separate tool to measure health externalities was also developed.
         6. Utilizing the previous studies and tools, the program has planned to support one local government to improve its public transportation system.
      4. Climate technologies (2015-2016):

The following activities were completed in partnership with the United Kingdom’s Foreign & Commonwealth Office:

1. Assessed the innovation ecosystem in climate technologies and issued recommendations for improvement.
2. Identified technologies with the greatest potential for GHG mitigation and generation of economic opportunities for the country.
3. Assessed in greater detail a priority technology (solar PV) and developed recommendations to promote the implementation of the technology.
4. Knowledge sharing to promote South-South Cooperation (2016-Ongoing):
   1. GGGI, the Government of Mexico, and other international partners organized a Regional Forum on Technologies for Mitigation and Adaptation to Climate Change held in Mexico City in February, 2016. This was attended by government officials from Chile, Colombia, Costa Rica, Dominican Republic, El Salvador, Guatemala, Honduras, Mexico, Nicaragua, Peru and Uruguay.
   2. GGGI, SEMARNAT and the Mexican Agency for International Development Cooperation (AMEXCID) organized the 1st Regional Forum on Green Cities: Achieving Growth through the Management of Climate Change and Air Quality in Mexico City in August, 2016.
   3. GGGI, the Government of Mexico, and other international partners organized a Regional Forum on Long-term Green Growth strategies for the implementation of the Paris Agreement held in Mexico City in June 2017. This was attended by government officials from Brazil, Chile, Colombia, Costa Rica, Dominican Republic, Guatemala, Mexico, Paraguay, Panama and Uruguay.

During the early stages of the program, GGGI provided demand-driven support to the government. A 5‑year in‑country delivery strategy is currently being developed in close consultation with the GoM. This Country Planning Framework (CPF) 2017-2021 will prioritize future areas of intervention, building on GGGI’s comparative advantage and in response to green growth challenges and opportunities in Mexico. Thus, GGGI does not expect to work across all 5 of the above program areas moving forward. 

 

Objectives/Purpose of the Assignment:

The overall objective of the evaluation is to gather evidence on the results of the GGGI Mexico Country Program that have been achieved to date and to inform improvements to future program design and delivery. In 2018, the country team will begin planning and designing its work program for the next biennium (2019-2020). Within the scope set by the Country Planning Framework, GGGI faces a number of strategic choices about how best to take the program forward, and this evaluation will be used to help inform those decisions through clear and actionable recommendations.

 

Scope of Work:

Based on GGGI’s evaluation policy, the first stage of the evaluation involved an internally-led process to develop an Evaluation Approach Paper to document the proposed design and workplan and gather background information on the country program. The Evaluation Approach Paper for the Mexico Country Program is under development and will be provided to the selected evaluator for discussion and finalization of the evaluation approach and methodology. This section provides a brief overview of the key evaluation questions which defines the scope of work.

The Independent Evaluator is expected to evaluate GGGI’s Mexico Country Program by assessing the following three key evaluation questions (KEQs), along with the indicative issues to be considered as part of each question. The questions selected are a combination of descriptive, cause and effect and normative questions[4] that aim to analyse all 5 program areas and deep dive into the transport program area. A more detailed design matrix (connecting the questions and specific sub-questions to the approach and methodology) will be included in the Evaluation Approach Paper which will be discussed and finalized with the selected evaluator.

1. What are the main outcomes (intended/unintended/positive/negative) achieved by GGGI from 2013-present, across all 5 program areas?

The aim is to capture supporting evidence on the outcomes that have been achieved so far by the country program. This will also help determine the level of quality of outputs delivered, partnerships developed and operational processes in place which have helped achieve the program outcomes. The question is critical to better understand how the design of the country program aligns with GGGI’s Strategic Plan 2015-2020 and is contributing to the achievement of green growth outcomes in Mexico. 

1. Under the transport program area, how much progress has been made against the program logic to enable the development of clean, high quality and efficient public transport in Mexico? Is there sound evidence to support the theory of change underpinning this logic?

This question seeks to better understand the program logic underpinning GGGI’s transport work and determine the robustness of the causal pathways to longer term outcomes. In addition, the aim is to gather adequate evidence and determine the likelihood that program outputs and medium term outcomes under the transport program area will contribute to the intended longer term outcomes.

1. How can GGGI build on previous results to scale, replicate and maximize impacts in the transport program area?

An analysis of the forecasted theory of change is required to provide actionable recommendations on how to scale, replicate and maximize the intended impacts of the program. The most critical issues in the public transport sector in Mexico need to be analyzed to determine how the program can contribute to transformational change.[5] The time period to be covered by the evaluation is 1 January 2013 to 31 August 2017.

 

Proposed Approach and Methodology:

The detailed approach and methodology will be included in the Evaluation Approach Paper and will be discussed and finalized with the selected evaluator. A combination of approaches and mixed methods (mainly non-experimental) is proposed to answer each of the key evaluation questions.

1. Assessment of program outcomes and the process to achieve them (Question A)

The program theory of change identifies these outcomes and the focus will be to gather evidence on the ones that have been achieved so far across all 5 program areas.

The methodology will include gathering credible and reliable evidence to validate the achievement of intended/unintended/positive/negative outcomes. It will also be necessary to develop strong arguments of how the program has contributed to specific outcomes. Evidence will be gathered through primary data collection (key informant interviews) and review of relevant program documents.

1. Assessment of longer term program outcomes (Question B)

The approach is to assess the transport program theory of change based on completed outputs and test the robustness of the causal pathways to the achievement of longer term outcomes. The focus will be to provide a reasonable level of assurance that the forecasted longer term outcomes are likely to be achieved. This will help validate the assumptions underpinning the program logic.

The methodology will include a review of relevant secondary literature (research or evaluations) on similar interventions in other geographic locations to validate the possible longer term outcomes that the transport program contributes to.

1. Assessment of program design and theory (Question C)

The approach is to assess the transport program theory of change based on forecasted activities / outputs to test whether the theory of change appears to be working and which causal pathways will help to maximize the intended program outcomes.

The methodology will include an assessment of the political economy of the transport sector[6], maturity of enabling policies and institutions, opportunities for financing and scale up and technical considerations to enable the development of efficient, clean and high quality public transport services. Evidence will be gathered through primary data collection (key informant interviews) and review of relevant secondary literature.

Timeframe and Deliverables:

The expected deliverables and corresponding timelines for this evaluation are elaborated below. The timelines are indicative and will be finalized in consultation with the selected evaluator.

Timeframe	Activity / Deliverable	Payment percentage
Late-October, 2017 – Mid-November, 2017	Inception phase: includes an inception meeting and discussions to finalize the approach, methodology and workplan for the evaluation. A desk review of key program documents will be conducted. Following this, an inception report will be developed. Majority of the content of the inception report will build on the Evaluation Approach Paper. Deliverable 1 – Inception report with final evaluation methodology, workplan, primary data collection tools and an agreed outline for the evaluation report	20%
Mid-November, 2017 – End-November, 2017	Data collection phase: includes a visit to Mexico City to meet with GGGI’s country team and key stakeholders. A list of stakeholders will be included in the Evaluation Approach Paper and the schedule for the visit will be developed beforehand in consultation with the country team.	n/a
December, 2017 – Mid – January, 2018	Analysis phase: appropriate qualitative and quantitative data analysis will be conducted with the aim of triangulating to arrive at reliable and credible findings, conclusions and recommendations. Reporting phase: development of the draft report. Deliverable 2 – Draft evaluation report	70%
Mid-January 2018 – End-January, 2018	Presentation to GGGI’s Mexico Program Team followed by presentation to GGGI’s Management Team. This can be done remotely depending from where the evaluator is based. Revise report based on feedback (if required) and incorporate management responses. Deliverable 3 – Final evaluation report submitted	10%

All reports must be in English, in accordance with GGGI’s formatting requirements, and submitted in a digital format.

The final report will be shared with the Secretariat of the Environment and Natural Resources (SEMARNAT), other relevant strategic partners in Mexico, GGGI’s members and staff, and published on GGGI’s website.

All air travel undertaken in relation to this assignment shall be undertaken at economy class and reimbursed by GGGI. 

 

Management arrangements:

This evaluation will be managed in accordance with GGGI’s evaluation policy. The key aspects are:

Evaluation manager

Within GGGI, the evaluation will be commissioned and managed by the Impact and Evaluation Unit (IEU). This will include:

-          Develop and finalize the evaluation TOR;

-          Develop the Evaluation Approach Paper;

-          Recruit, contract and manage contractual issues for the independent evaluator;

-          Supervise the delivery of the evaluation deliverables by the independent evaluator;

-          Support the independent evaluator with organizing or coordinating activities during the visit to Mexico;

-          Accompany the independent evaluator to meetings with key stakeholders in Mexico; and

-          Help facilitate the involvement of the Evaluation Advisory Group (see below) in this evaluation.

Evaluation Advisory Group (EAG)

GGGI will establish an Advisory Group for this evaluation. The Group will be coordinated by IEU and comprised of representatives from GGGI member countries who are interested to ‘opt in’ to participate in this evaluation. The EAG’s role is to assist in the oversight of the evaluation and review and comment on key evaluation deliverables. More specifically, the role of the EAG will include:

-          Advise on the finalization of the Evaluation Approach Paper based on which the evaluation will be implemented;

-          Review and provide comments on the deliverables of the evaluation.

 

Required skills and competencies:

Based on the scope of the evaluation, the primary requirements of the evaluator are demonstrated transport sector expertise, experience in conducting evaluations, and an understanding of public policy development. The specific requirements are as follows:

• Postgraduate degree in engineering, economics or a related discipline;
• Minimum of 15 years of experience in development, financing, implementation and/or assessment of land transport / public transport policy in multiple countries;
• Experience in Latin America, preferably Mexico, would be an advantage;
• Experience in conducting evaluations of international development programs; and
• Working level knowledge of Spanish is desirable but not mandatory.

A lower qualification can be accepted if substantial evaluation experience or experience in the transport sector is demonstrated above the required 15 years.

 

Application process:

To apply for this position, candidates must submit a detailed CV outlining relevant experience and a cover letter (maximum 2 pages) describing the candidates’ suitability to conduct this evaluation.

The deadline to apply for this position is 11 October, 2017. Shortlisted candidates will be interviewed and requested to provide referees and a sample of writing.