Consultancy – Mid-term Evaluation of the project: “Enabling responsive, coherent and inclusive support to the implementation of the 2030 Agenda for Sustainable Development” (Project)

Background

In September 2015, Member States of the United Nations adopted a new global development framework entitled “Transforming Our World: 2030 Agenda for Sustainable Development.” This framework came into effect upon expiry of the Millennium Development Goals (MDGs) on 1 January 2016 and runs through 2030. The ambition and scope of the 2030 Agenda is reflected in its 17 Sustainable Development Goals (SDGs) and their 169 targets that will be the road map for the efforts of Members States and the United Nations system until 2030. The Agenda seeks ‘to realize human rights of all and to achieve gender quality and the empowerment of all women and girls. They are integrated and indivisible and balance the three dimensions of sustainable development: the economic, social and environmental.’ (UN Resolution 70/1).

The new universal agenda calls for an integrated approach to sustainable development and collective action, at all levels, to address the challenges of our time, requiring coherent integrated support from the United Nations system.

Following the adoption of the Agenda, demand was real and urgent. At the time of the project’s inception - even before the official entry into force of the 2030 Agenda - 95 UN Country Teams had already received requests from governments to support its implementation[1]. To respond to these requests, UNDP conceived the Project under review - “Enabling responsive, coherent and inclusive support to the implementation of the 2030 Agenda for sustainable Development”. The project design was based heavily on UNDP’s experience with and lessons learned from a predecessor project - ‘Building the Post-2015 Development Agenda: Open and Inclusive Consultations’ - which had facilitated a multi-agency global conversation to inform the drafting of the 2030 Agenda and of supporting Member States with the Millennium Development Goals framework.[2]

Thus, this Project represents a swift response by UNDP to put in place a formal structure with resources and partnerships to operationalize the 2030 Agenda, making it actionable. The Project’s implementation timeframe (from 1 January 2016 to 31 December 2020) also coincides with a time of change within the UNDP and broader UN context, including the Quadrennial Comprehensive Policy Review (QCPR) and UN System Reform, and a post-Agenda adoption landscape that was quite unique and arguably more complex to deliver results during the initial years of the Agenda’s adoption.

At its core, the Project aims to help apply ‘MAPS’ (Mainstreaming, Acceleration and Policy Support), adopted by the UN Development Group (UNDG) in October 2015 as a common approach to support the implementation of the 2030 Agenda.[3] Guided by the MAPS approach, the project aims to deliver the 2030 Agenda through assisting the mainstreaming of the 2030 Agenda at the national and local levels; offering an SDG-based analytical framework to accelerate progress in tackling obstacles to development; facilitating access to policy support and thematic expertise available throughout the UN system; foster citizens’ engagement and multi-stakeholder partnerships to progress sustainable development; advocate a bottom-up approach to monitoring and reviewing progress in achieving the SDGs; improve data availability; and generate and disseminate knowledge from implementation.

Recognizing the shared responsibility to act on this mandate, this project is consistent with the evolving work of the UNDG on SDG implementation and aims to contribute towards the following result: Effective, coherent, inclusive and context-sensitive implementation of the 2030 Agenda in countries around the world. The project has activities, deliverables and results at the global level (e.g. inter-agency coordination, global research, advocacy and public engagement, knowledge generation and exchange, south-south and triangular cooperation), as well as components for regional support and country support. For instance, the Project supports the application of the MAPS approach through “MAPS missions” at the country-level, the development of analytical tools, support on data, partnerships, innovation, and addressing the needs of the furthest left behind. The output under the SDG Action Campaign focuses on awareness raising and citizens’ engagement, to support citizen and multi-stakeholder action in order to realize the SDGs.

The initial experience since the adoption of the SDGs has brought additional information about partnership and resource opportunities, as well as constraints. Similarly, it also brought to the fore new initiatives to further strengthen the support package for country-level SDG implementation: for instance, in 2016 UNDP initiated ‘MAPS missions’ as a way to help countries operationalize a more effective cross-thematic approach to integrated SDG implementation (reflected in Outputs 1-5 of the Project). These missions were intended to assist in assessing alignment of the SDGs with country priorities, identify areas most likely to speed up (‘accelerate’) overall progress, and make integrated policy recommendations moving forward. The 26 missions undertaken in 2016-2017 have been tailored to specific country contexts and entry points for SDG engagement. (The first nine in 2016 were UNDP missions, while the 2017 missions were organized as interagency initiatives.) National priorities and characteristics define the substance and scope of each mission as well as the composition of the mission team.

In November 2017, the Project took stock of accomplishments to date and reflected upon lessons and challenges to assess ways forward, presenting these findings to the Project Board. This led to a revision of the Project Document in March 2018, which continues to be structured upon the UNDG three broad approach of Mainstreaming, Acceleration, and Policy Support, allowing maximum flexibility to ensure the Project prioritizes country-level demand as its core offering. The November 2017 Project Board meeting approved the revision of the project document to take into account new realities: specifically, it adjusted the multi-year budget downwards given resource constraints, and also incorporated a new output on addressing SDGs in contexts of fragility). The new output, SDG implementation in fragile and conflict affected settings seeks to ensure that the project takes a risk-informed and fragility-sensitive approach, aligned with the Secretary General’s prevention agenda and World Humanitarian Summit commitments. All of the above components are reflected in the Project’s eight outputs (below), and compose the entire Project’s integrated approach to support SDG implementation through the MAPS approach.

Output 1: Mainstreaming

Agenda 2030 for Sustainable Development mainstreamed at national and local levels

Output 2: Acceleration

Analytical tools used by policy makers to identify synergies and trade-offs in the implementation of the SDGs

Output 3: Policy Support

Countries draw on expertise & joined-up capacities of UN entities to support national governments with SDG implementation

Output 4: Accountability and Data

Capacities of countries to produce data for SDG monitoring and reporting are strengthened

Output 5: Learning and Integration

Evidence-based knowledge products on issues relevant to SDG implementation developed and shared

Output 6: The UN SDG Action Campaign

Cutting-edge advocacy tools made available, effective multi-stakeholders mechanism for implementation and citizen- driven review and follow-up process established (This output is not included in the scope of this midterm evaluation, since it has recently conducted a midterm evaluation separately.)

Output 7: MAPS Team[4]

Output 8: SDG implementation in fragile and conflict affected settings (This output is not included in the scope of this midterm evaluation, since it has been added in 2018.)

MID-TERM EVALUATION PURPOSE AND OBJECTIVES

This mid-term evaluation, which will be taking place during the third year of the project’s implementation (2016-2020), in 2018, will be used not only to assess this project but also to inform UNDP’s ongoing strategy for SDGs implementation at the global, regional and country-level. The evaluation will analyze the progress towards the achievement of the project goals and outcomes as specified in the revised Project Document, and assess early signs of project success or failure with the goal of identifying the necessary changes to be made in order to set the project on-track to achieve its intended results by 2020. It will examine factors that facilitated or constrained the achievement of intended project objectives, against the backdrop of the 2014-2017 and 2018-2021 UNDP Strategic Plans, and the ongoing UN Reform. It will make forward-looking recommendations on the revision of the project to align more closely with the current UNDP and UN vision.

The midterm-evaluation will also review the project’s strategy and sustainability.

The evaluation findings and recommendations will be critical to guide UNDP in further refining appropriate strategies, policies and programme approaches to strengthen UNDP's support to SDG implementation. The evaluation will strengthen UNDPs accountability to national programme partners and donors of the project. 

[1] Together Possible: Gearing Up for The 2030 Agenda, UNDG, Page 9, https://undg.org/wp-content/uploads/2016/12/UNDG-results-report-2015.pdf

[2] Outcome Evaluation of the UNDP Project, “Building the Post-2015 Development Agenda: Open and Inclusive Consultations” https://erc.undp.org/evaluation/evaluations/detail/7735

[3] UNDG MAPS concept note: https://undg.org/document/maps-mainstreaming-acceleration-and-policy-support-for-the-2030-agenda/ 

[4] The staffing support included as a separate output in the original project document has now been dispersed across the other outputs as key components of “development effectiveness,” in keeping with current UNDP policy.

Duties and Responsibilities

The evaluation will cover the period 1 January 2016 to 30 April 2018 and Outputs 1-5 and 7 of the MAPS Project Document; Output 6 does not require an evaluation because it has just been reviewed and Output 8 will not be reviewed because it has been added to the project outside the review period. 

Given the complexity and integrated nature of the 2030 Agenda, the early implementation stage and the speed at which UNDP was required to develop this Project, it is understood that the midterm evaluation may not be able to isolate UNDP's unique contribution. Yet the midterm evaluation will attempt to take stock of UNDP’s role and contribution to support countries to engage with the 2030 Agenda during the early stages of implementation.

This includes understanding how through the Project, UNDP has taken steps to facilitate inter-agency collaboration and serve the role as convener with other UN entities, national governments, other intergovernmental bodies, academia, foundations, private sector and CSOs. The midterm evaluation will focus on documenting UNDP's contribution to date, in particular at the outcome and output level within UNDP's sphere of influence, while acknowledging other intervening factors and the contribution of partners.

EVALUATION CRITERIA AND QUESTIONS

The evaluation will be guided by the following criteria, to help focus the evaluation objectives above by defining the standards against which the initiative will be assessed. The evaluation criteria to help focus the evaluation objectives include: relevance, effectiveness, efficiency, and sustainability[1].

The final areas to evaluate and the questions will be determined, in reference to the below list, by the evaluator and MAPS Project Team. The evaluation will be carried out within the overall provisions contained in the UNDP Evaluation Policy and evaluation questions will be aimed to address and assess the four evaluation criteria. A list of initial questions for potential consideration is proposed below:

RELEVANCE

• What role has UNDP played through this project in the provision of "thought leadership", and in coordinating within the UN system andbeyond? Please provide concrete examples of UNDP’s coordination role through this Project in the roll-out and early implementation of the 2030 Agenda. Please make concrete suggestions on how to enhance these roles moving forward.

• Assess the funding and technical support provided toCountry Offices. In what ways can support be provided more systematically and relevant for Country Offices?

• Was UNDP’s work relevant to Member States, addressing countries' needs, and consistent with the organization's mandate and focus as illustrated in the 2014-2017 UNDP Strategic Plan?

• Please assess the impact of reports, research paper, workshops, online and offline platforms, tools and guidance that have been produced under the Project, including those produced in collaboration with other agencies. What has been the demand, supply and usage of such projects? Which tools and products have been particularly well received and which have been received to a lesser degree?

EFFECTIVENESS

• How has UNDP addressed the needs of diverse groups, including marginalizedpeoples and groups, to leave no one behind in SDGs implementation? In what ways could the needs of different stakeholders have been addressed more directly? What has been done well?

• How has the project addressed cross-cutting issues such as gender? Which cross-cutting issues could be more emphasized by the Project?

• What are some of the key achievements the Project in aligning the SDGs with the national development plans or identifying resource gaps, etc.?

• What lessons can be learned to improve the Project’s delivery and how could the Project evolve to meet the demands of countries and stakeholders more effectively?

• What is UNDP’s role in facilitating the implementation of the 2030 Agenda, at the local, national, regional and global level, responding to the demands from UNCTs and Country Offices?

• What is the extent to which the Project, via MAPS, has informed UNDP’s ongoing strategy for SDGs implementation? Please provide recommendations for the Project to guide the way forward, in relation to UNDP and UN System.

• How has UNDP contributed to achieving stronger and more coherent integration in the implementation of the 2030 Agenda through MAPS missions/engagements? Have the different components of the project (outputs) worked complimentary to one another?

EFFICIENCY

• Were stated outcomes on outputs achieved to date? Please describe the progress, including to what extent have UNDP outputs and assistance contributed to outcomes.

• What factors have contributed to achieving or not achieving intended outcomes?

• Is the project’s governance mechanism (i.e., the project board or equivalent) functioning efficiently?

• What are the resource mobilization strategy for this Project to date? Please identify and make recommendations on possible avenues for future resource mobilization.

• What are factors that have facilitated or constrained UNDP’s contribution or performance in supporting the early implementation of the 2030 Agenda? Please identify risks and opportunities, including which decisions, strategies and approaches worked well.

• Please provide a comprehensive set of recommendations to improve the quality of the project going forward, both in terms of the project’s strategic focus and UNDP’s management role.

SUSTAINABILITY

• Which partnerships, tools, new innovations and new technologies developed and used by UNDP could be expanded upon (i.e. go to scale) moving forward, in view of working with countries to mainstream, accelerate and provide policy support for the 2030 Agenda?

• Which solutions were developed and drawn from the country-level in collaboration with Country Offices? What strategies should be kept and what should be changed?

• What are the partnerships developed by the project to promote the MAPS approach both within and outside the UN system?

• What has been the Project’s ability to adapt and evolve based on continuous lessons learned and the changing development landscape? Please account for factors both within the UN and external.

• Is the project sufficiently at scale? Is there potential to scale up the project?

EVALUATION METHODOLOGY

The evaluation will rely on multiple sources of information for analysis, validation and triangulation of evidence against the evaluation questions. Sources of data and methods of collection could include (final methodology to be determined jointly with the evaluator):

Project monitoring documents

• Synthesis Reports
• Mission Reports
• Feedback from stakeholders
• Project Board documents and minutes
• Quality Assurance and peer reviews
• Global corporate reporting (i.e. ROAR and Global Programme Results Framework)

Output Reports

• Thematic focus reports (i.e. Data, Ageing)
• Guidance papers and reports
• Substantive reports and papers

Information collection from key personnel

• In person and virtual interviews with key people from Country Office, Regional Bureau, HQ, may include UNDP, UN Agency, Government, CSO, Academia, Private Sector and others
• Written follow-up with key personnel
• Web-base and other types of assets

1. Online surveys and polling for campaigning
2. Data visualization tool for follow-up and review
3. Virtual Reality and others

Expected outputs and deliverables

EVALUATION PRODUCTS (DELIVERABLES)

1. Evaluation inception report. An inception report should be prepared by the evaluator before going into the full-fledged data collection exercise. It should detail the evaluator's understanding of what is being evaluated and why, showing how each evaluation question will be answered by way of: proposed methods, proposed sources of data and data collection procedures. The inception report should include a proposed schedule of tasks, activities and deliverables. The inception report should respond to each of the following questions:

• What evidence is needed to address the evaluation questions?
• What data collection method(s) will be used to address the evaluation criteria and questions? Why were these methods selected? Are allocated resources sufficient?
• Who will collect the data?
• What is the framework for sampling? What is the rationale for the framework?
• How will the information collected be analysed and the findings interpreted and reported?
• What methodological issues need to be considered to ensure quality?

2. Draft evaluation report. UNDP and key stakeholders will review the draft evaluation report to ensure that the evaluation meets the required criteria.

3. Final evaluation report and presentation by evaluator and report shared publicly.

EVALUATION ETHICS Evaluations in UNDP will be conducted in accordance with the principles outlined in the UNEG ‘Ethical Guidelines for Evaluation’ . Critical issues the evaluator must address in the design and implementation of the evaluation, including evaluation ethics and procedures to safeguard the rights and confidentiality of information providers must be upheld. For example: measures to ensure compliance with legal codes governing areas such as provisions to collect and export data, particularly permissions needed to interview or obtain information about children and young people, provisions to store and maintain security of collected information; and protocols to ensure anonymity and confidentiality, etc. 

Time Frame for the Evaluation Process:

The evaluation will be undertaken over the period May through August 2018 based on the following schedule. To complete the deliverables, the evaluator will be expected to present a daily rate for a maximum of 35 days total, this includes roughly:

• 7 Days: Review literature, prepare evaluation design, and produce inception report
• 6 Days: Interviews
• 6 Days: Analysis
• 7 Days: Produce Draft Report and presentation to Project Board
• 5 Days: Revision based on comments
• 3 Days: Finalization
• 1 Day: Presentation

Activity	Proposed period
Review reports and desk review	May- June,  2018
Finalize evaluation questions once contract is signed	June, 2018
Finalize evaluation design and methods, submit inception report	June, 2018
Conduct on-site evaluation (interviews, questionnaires)	June-July, 2018
Prepare the draft report	June-July, 2018
Submit 1st Draft of Report to be presented for Board Meeting	July, 2018
Stakeholder meeting and review of the draft report (for quality assurance)	Late July, 2018
Incorporate UNDP management comments	Late July, 2018
Submit Final Mid-term Evaluation	August 10, 2018
Editing, design and printing	August 10-15 2018
Present report	Mid-August 2018

*Note- the above timeline is tentative and may be subject to change depending on real delivery timeline.

[1] For details please refer to Page 168-170 of the Handbook on Planning, Monitoring and Evaluating for Development Results http://web.undp.org/evaluation/handbook/documents/english/pme-handbook.pdf

Competencies

The Evaluator should comply with the following UN Core Values, including:

• Professionalism
• Planning and Organizing ability
• Accountability: takes ownership of responsibilities and honours commitments.
• Communications: speaks and writes clearly and effectively; listens to others, correctly interprets messages from others and responds appropriately; asks questions to clarify, and exhibits interest in having two-way communication; tailor language, tone, style and format to match audience. Keeps confidential information undisclosed.
• Innovator: learn, share and acquire new competencies and seek new challenges by exploring new approaches
• Performer: works against an agreed outcome and priorities and seeks performance feedback from supervisors and support staff in the performance review in a constructive and objective manner.

Required Skills and Experience

Academic qualifications:

• Master’s degree in social, development or behavioral science is required

Years of experience:

• Over 12 years of international development experience is required
• Substantive knowledge of the 2030 Agenda implementation, understanding of the core principles of the 2030 Agenda including national ownership, universality, leave no one behind, human rights, and integration and indivisibility of the SDGs and development is required.
• Extensive experience and knowledge in the area of 2030 Agenda implementation is required;
• Extensive experience in applying evaluation techniques, and demonstrable experience in conducting evaluations of projects that focus on sustainable development, human development, the Post-2015 era, MDGs and/or the 2030 Agenda is required;
• Qualitative data collection and analysis skills;
• Process management skills such as facilitation skills and ability to negotiate with a wide range of stakeholders;
• Prior experience in working with multilateral agencies;
• Knowledge of UNDP’s role, and UN programming at the country level and regional levels is an asset.
• Understanding of the UN ongoing reform processes, and experience in working with other relevant regional and/or continental entities and international donors is an asset.

Language:

• Fluency in written and spoken English is required;
• Working knowledge of another French and Spanish would be an asset

Application Procedure

The application package containing the following (to be uploaded as one file):

• Online application with brief description of why the Offer considers her/himself the most suitable for the assignment; and
• Personal CV or P11, indicating all past experience from similar projects and specifying the relevant assignment period (from/to), as well as the email and telephone contacts of at least three (3) professional references.
• A two-page methodology on how you will conduct the work including a Work Plan and approach in delivering the required outputs within the assignment period. Please also provide samples/links to previous evaluations, applications without this will not be considered.

Note: The above documents need to be scanned in one file and uploaded to the online application as one document.

Shortlisted candidates (ONLY) will be requested to submit a Financial Proposal.

• The financial proposal shall specify a total lump sum amount, and payment terms around the specific and measurable deliverables of the TOR. Payments are based upon output, i.e. upon delivery of the services specified in the TOR, and deliverables accepted and certified by the technical manager.

• The financial proposal must be all-inclusive and take into account various expenses that will be incurred during the contract, including: the daily professional fee; cost of travel from the home base to the duty station and vice versa, where required; living allowances at the duty station; communications, utilities and consumables; life, health and any other insurance; risks and inconveniences related to work under hardship and hazardous conditions (e.g., personal security needs, etc.), when applicable; and any other relevant expenses related to the performance of services under the contract.

• This consultancy is a home-based assignment, therefore, there is no envisaged travel cost to join duty station/repatriation travel.  For the two in-person consultations in New York, travel will be arranged by UNDP.

  In the case of unforeseeable travel requested by UNDP, payment of travel costs including tickets, lodging and terminal expenses should be agreed upon, between UNDP and Individual Consultant, prior to travel and will be reimbursed. In general, UNDP should not accept travel costs exceeding those of an economy class ticket. Should the IC wish to travel on a higher class he/she should do so using their own resources.

If the Offeror is employed by an organization/company/institution, and he/she expects his/her employer to charge a management fee in the process of releasing him/her to UNDP under a Reimbursable Loan Agreement (RLA), the Offeror must indicate at this point, and ensure that all such costs are duly incorporated in the financial proposal submitted to UNDP.

The Financial Proposal is to be emailed as per the instruction in the separate email that will be sent to shortlisted candidates.

Evaluation process

Applicants are reviewed based on Required Skills and Experience stated above and based on the technical evaluation criteria outlined below.  Applicants will be evaluated based on cumulative scoring.  When using this weighted scoring method, the award of the contract will be made to the individual consultant whose offer has been evaluated and determined as:

• Being responsive/compliant/acceptable; and
• Having received the highest score out of a pre-determined set of weighted technical and financial criteria specific to the solicitation where technical criteria weighs 70% and Financial criteria/ Proposal weighs 30%.

Technical evaluation - Total 70% (700 points):

Criteria 1. Understanding of the 2030 Agenda implementation and experience at the global, regional and/or country level on Post-2015,era, MDGs and/or the 2030 Agenda;  Weight = 20%; Maximum Points: 200;

Criteria 2. Proven experience in applying evaluation techniques, and demonstrable experience in conducting evaluation of projects that focus on sustainable development, human development, the Post-2015 era, MDGs and/or the 2030 Agenda Weight = 20 %; Maximum Points: 200;

Criteria 3. Understanding of UNDP's role and UN programming, in relation to the UN Development System and UN Agencies, including in the context of the ongoing reform process, show proven work in these areas. Weight = 15 %; Maximum Points: 150; and

Criteria 4. Evaluation of submitted samples/links to previous evaluations. These will be evaluated in terms of quality. Weight =5 %Maximum Points: 50

• Interview: Weight = 10 %; Maximum Points: 100.

Having reviewed applications received, UNDP will invite the top three/four shortlisted candidates, with minimum scores of 420 points (70% of 600 points) for interview. Please note that only shortlisted candidates will be contacted.

Candidates obtaining a minimum of 70% (490 points) of the maximum obtainable points for the technical criteria (700 points) shall be considered for the financial evaluation.

Financial evaluation - Total 30% (300 points)

The following formula will be used to evaluate financial proposal:

• p = y (µ/z), where
• p = points for the financial proposal being evaluated
• y = maximum number of points for the financial proposal
• µ = price of the lowest priced proposal
• z = price of the proposal being evaluated

Contract Award

Candidate obtaining the highest combined scores in the combined score of Technical and Financial evaluation will be considered technically qualified and will be offered to enter into contract with UNDP.

Institutional arrangement

The evaluation will be directly managed by the Project Manager of the Project, who is the Programme Specialist of the 2030 Agenda team, located in BPPS Strategic Policy Unit and the Sustainable Development Cluster in New York, USA, with activities at all levels. The final evaluation will be presented to the Project Board. This evaluation will be a home-based assignment for an evaluator with two in-person consultations in New York.

The Consultant will be responsible for providing her/his own laptop.

Payment modality

• Payments are based upon output, i.e. upon delivery of the services specified above and deliverables accepted and upon certification of satisfactory completion by the manager.

• The work week will be based on 35 hours, i.e. on a 7 hour working day, with core hours being between 9h00 and 18h00 daily.

Click here for important information for US Permanent Residents ('Green Card' holders).