By continuing to browse this site, you agree to our use of cookies. Read our privacy policy

International Consultancy for Technical Peer Reviewer Evaluating the Attribution and Contribution of UNICEF LAC in High/Middle Income Countries

Panama City (United States of America)

Under the overall supervision of the Regional Monitoring and Evaluation Specialist, UNICEF LAC RO is looking for qualified and experienced professional to provide Technical Peer Review to an Evaluation Technical Note to conduct evaluations that measure the attribution and contribution of UNICEF work in high and middle-income countries.

THE UNITED NATIONS CHILDREN’S FUND (UNICEF)

TERMS OF REFERENCE FOR INDIVIDUAL CONTRACTOR/ CONSULTANCY

FOR

INTERNATIONAL CONSULTANCY FOR TECHNICAL PEER REVIEWER EVALUATING THE ATTRIBUTION AND CONTRIBUTION OF UNICEF LAC IN HIGH/MIDDLE INCOME COUNTRIES

Level of experience:   Junior [   ] 1-3yrs        Medium [   ] 4-10yrs   Senior [X] more than 10yrs

Location: Regional Office for Latin America and the Caribbean [LACRO]

Language(s) required: Fluency in English (oral and written) is required

Closing date for applying to this consultancy: August 8, 2018

Duration: 4 months

Estimated start date of this consultancy: August 15, 2018           Estimated end date: December 31, 2018

BACKGROUND:

The majority of countries where UNICEF LAC functions are high-middle income countries. Country offices have strategically designed their country programmes focused on upstream work, often through a combination of policy and programme advocacy as well as modelling interventions and contributing to pilot projects to be scaled up, targeting institutional and behaviour changes in societies towards better results for children. Country programmes are typically designed to create the most appropriate enabling environment, through supporting adequate child focused policies and programmes, and promoting legal frameworks laws in line with international standards and improving learning and capacity of government institutions. Piloting and modelling are implemented in the field to create evidence and engage local partners to identify solutions for children to be replicated or scaled up. In addition, UNICEF is frequently invited to provide top-expert advice on child-related policies without a local or demonstrative project. For example, on social protection, child-focused budgeting, child protection legislation, systems strengthening, etc. This type of efforts is complex and highly demanding in terms of time, investment and capacity, considering that it usually requires an inter-sectoral approach to address key systemic barriers that impede the full realization of children’s rights.  This remains a challenge to demonstrate results that can be directly attributed to UNICEF and measure the actual contribution of UNICEF to high level results.

Results for children is the main ingredient for the definition of UNICEF country programmes. For policy work, the time span of five years of a country programme is usually not enough to demonstrate results at the level that it is required. Implementing policy work is a long-term investment to achieve results at large scale and often the influence of different factors may overshadow the casual relationship between the actual achieved changes and the changes in the life of children.

A critical but frequently unanswered question remains – are these programmes actually achieving changes in children’s lives? Regular monitoring is often focused on reporting indicators at the output level, which aim at reducing bottlenecks and barriers, such as generation of evidence policy reports, or elaboration of a toolkits and guidance documents for policy makers, that are impeding institutional or behavioural changes. At the outcome level, results reflect contributions to strengthening systems, and results reports are focused on institutional changes, such as monitoring system in place, or a draft law/policy. Results also include the generation of models, which are the systematic representation of lessons learned and recommendations to scale up validated pilot projects to be used by government officials for national or sub-national programs design and implementation. Whether or not these outputs and outcomes are changing the lives of children as intended by the UNICEF strategic plan is not evident through the current monitoring and evaluation mechanisms in practice. Outputs (attribution) and outcomes (contribution) are part of a theory of change that in the long run is expected to change the lives of children which will eventually translate into children survive and thrive, all children learning, all children protected from violence and exploitation, a fair chance for all children, and all children live in secure environments.

The Role of Evaluation

In line with the UNICEF’s Evaluation Policy (2013), the evaluation function in UNICEF LAC aims at helping to improve its performance and results, by supporting organizational learning and accountability.

UNICEF LAC seeks to enhance the strategic role of Evaluations to understanding the context in which advocacy, policy advice, and models and scaled up pilots can reach results at scale. Policy advocacy is present at all levels.

In the last two years, UNICEF LAC evaluations have been mainly focused on assessing interventions at the outcome and output level. Although, some evaluations had an indication of impact results, evaluation analysis of long term and sustained results has been restricted. There is limited evidence that can validate the hypothesis (theory of change) that the activities, outputs and outcomes of a UNICEF LAC intervention is reaching (or contributing to) long term and sustained changes that can have an impact. Even when evaluations have assessed national policies and programmes (jointly with Government counterparts), the contribution of UNICEF to impact level results has been hardly assessed.

Evaluating the attribution of specific interventions and its contribution of UNICEF at the impact level is of high importance, as UNICEF is accountable to bring results at scale and contribute to high level results described in the new UNICEF Strategic Plan, and ultimately to the Sustainable Development Goals. In the context of high/middle income economies, the contribution of UNICEF may even be more “diluted” leaving the question of the value-added role of the organization in these contexts. Evaluation, through adequate evaluation methods, can help clarifying this role, making the causal inferences clear, even for advocacy strategies, and identifying the linkages all the way from activity to output to outcome to impact (which in many cases is not linear as it is portrayed) - and therefore offering a response to the question: “are these programmes actually achieving changes in children’s lives?”, “What are the conditions that caused reaching those changes?”

Considering the nature of the upstream work in the Latin America and the Caribbean region in middle/high income countries, the evaluation methods package UNICEF LAC is looking for is threefold: i) to assess the attribution of the effect of pilot / demonstrative projects; ii) to assess the contribution of those pilots / demonstrative projects to scaled up interventions; and iii) to assess policy advocacy and technical assistance at high levels.

Evaluation needs to assess attribution/contribution of UNICEF work in Middle / high Income countries in the LAC Region

Evaluation methods such as those based on causal inference of Theory of Change (theory-based approaches to evaluation) can show the complete spectrum of how a single advocacy intervention and/or a combination intervention that includes advocacy, technical advice, and implementation pilots is making a difference in children’s lives – where other stakeholders are also contributing. Specific evaluation methods can assess the effects of different upstream components a combination of often non-linear actions directly attributable to UNICEF and others pitched as contribution as part of a broad spectrum of a policy advocacy work, where UNICEF is one in many other stakeholders.

As part of the results-based management approach to design, implement and report results, UNICEF requires programmes to use a “theory of change” to present the broad vision of what results will be achieved and how they will be achieved. The theory of change articulates the theories and assumptions that underpin the anticipated change process through which inputs and activities are converted to or lead to identify results at the output, outcome and impact level (Results-Based Management Handbook, UNICEF, 2017). To design a proposed methodology for this specific assignment, theory-based approached to evaluation, which use Theory of Change to assess with evidence whether and how an intervention contributed to observed results, would be prioritized.

Evaluation Methodology Goals

UNICEF LAC has hired a group of consultants to contribute to improve organizational learning through an evaluation framework that can assess if an intervention has produced or influenced high level results and determine the role of UNICEF in these contexts. 

The evaluation framework intends to provide a guide for country offices and the LAC regional office (or other regional offices) to design, implement, and report results for children resulting from UNICEF LAC interventions, including one or a combination of these interventions: i) evidence generation through modelling and implementation of pilot projects, ii) policy advocacy of bring models to scale; and iii) Policy advocacy / technical assistance upstream work.

The evaluation framework will include a technical evaluation note that includes a proposed methodology to assess the attribution/contribution of UNICEF. The proposed methodology will then be tested in a country office – in an area of work related to the LAC Regional Priorities. Specifically, one country office evaluation will be used as case study to test the proposed methodological framework to assess the attribution/contribution of UNICEF in a specific context. The case study will help be used to fine-tune the technical note.

The Evaluation Technical Note will be composed by a proposed evaluation methodology for upstream work, including i) assessing pilot/demonstrative projects; ii) advocacy work to bring models up to scale; and iii) advocacy / technical assistance upstream work. The note will go beyond impact evaluations approaches to assess attribution/ contribution to impact level results: When to apply the methodology, what are the conditions needed to apply the methodology. The framework will be refined after its application in a country case study.

The case study for testing will be based on the theoretical, the proposed methodology will be tested/piloted in one country office evaluation. The case study will assess the three parts the evaluation framework (assessing pilot/demonstrative projects; advocacy work to bring models up to scale; and advocacy / technical assistance upstream work). The country office evaluation will be selected according to the country evaluation plan, capacity to participate in this process, and available resources. The topic of the evaluation will need to be one directly linked to one selected regional priorities. The evaluation will assess the contribution of UNICEF in a specific context. Lessons learned from this experience will allow the proposed methodology to be refined.

PURPOSE:

Under the overall supervision of the Regional Monitoring and Evaluation Specialist, UNICEF LAC RO is looking for qualified and experienced professional to provide Technical Peer Review to an Evaluation Technical Note to conduct evaluations that measure the attribution and contribution of UNICEF work in high and middle-income countries.

EXPECTED RESULTS (measurable results):

Specifically, technical reviewer will commit to the following tasks:

  • First Review to the Technical Note: Review, give feedback to, and identify practical recommendations for the draft Technical note, including a thorough review of the methodology, instruments, the applicable ethical procedures, according to UNICEF standards (https://www.unicef-irc.org/research/ethical-research-and-children/ ). It should also include the identification of potential conflicts with UNEG evaluation standards and norms (http://www.unevaluation.org/document/detail/1914 ) and the UNICEF Global Evaluation Reports Oversight System GEROS.
  • Face-to-face presentation of feedback and contribute to technical discussions
  • Review to the Case Study for Testing: Review, give feedback to, and identify practical recommendations for the Case Study, including a thorough review of the methodology, instruments, the applicable ethical procedures, according to UNICEF standards (https://www.unicef-irc.org/research/ethical-research-and-children/ ). It should also include the identification of potential conflicts with UNEG evaluation standards and norms (http://www.unevaluation.org/document/detail/1914 ) and the UNICEF Global Evaluation Reports Oversight System GEROS. Review of the case-study will be done in two moments, at draft stage and final stage.
  • Final Review to the Technical Note: Once the recommendations have been included, review the quality of the revised version of the Technical Note once again and give final recommendations.

Reviews and written comments will be submitted by email to the manager of the consultancy “Evaluating the Attribution and Contribution of UNICEF LAC in high/middle income countries”. Every time feedback is submitted, the Peer Reviewer will participate in a teleconference organized by UNICEF LAC to discuss feedback to the intermediate and final products of the Evaluation Framework.

The peer reviewer is expected to participate in face-to-face mid-term review of the process to discuss with the Researchers/Evaluators. The consultant is This task may involve traveling if the consultant is not based in the designated location.

Peer Reviewer will be acknowledged throughout the process and mentioned as technical contributor in the final product.

 DELIVERABLES:

Product

Percentage of the Total Budget

Estimated Delivery Time

First Review to the Technical Note and Online meeting to discuss methodology and way forward

40%

September 2018

Face-to-face meeting and presentation of feedback and comments

10%

September/October 2018

Review to the Case Study for Testing and Online meetings (at least 2) to discuss case and way forward

30%

November 2018

Final Review to the Technical Note and Proposed Evaluation Framework

20%

December 2018

Adjustments to the time-frame will be communicated in advance to Peer Reviewer.

KEY QUALIFICATIONS, TECHNICAL BACKGROUND, AND EXPERIENCE REQUIRED:

Education

At least a master’s degree or equivalent in Social Sciences, Evaluation or another related technical field.

 Skills and Experience

  • At least 10 years’ experience in designing and conducting evaluations
  • Excellent and proved knowledge of Evaluation methods and approaches, with especial focus on Theory-based evaluation methods
  • Publications in evaluative methods, including theory-based evaluation
  • Proved experience in peer review of methodological approaches for evaluations
  • At least 10 years of proven professional experience in designing and conducting evaluations
  • Previous experience with Evaluation Quality Assurance systems is a plus
  • Excellent analytical and writing skills in English. Working knowledge of Spanish is a plus.
  • Excellent interpersonal skills with ability to work in a multi-cultural environment
  • Working knowledge in the LAC region is a plus
  • Familiarity with UNICEF programmes in middle and high-income countries is a plus

Other competencies

  • Demonstrated ability to work in a multicultural environment and establish harmonious and effective relationships with national partners;
  • Demonstrated leadership, managerial and supervisory ability.

MANAGEMENT ARRANGEMENTS

This consultancy will be managed by the Regional Monitoring and Evaluation Specialist of LACRO.

INDIVIDUAL CONTRACTOR’S WORKPLACE:

The individual contractor will be required to work home-based and establish periodic communications/virtual meetings with the Regional Office.

The individual contractor will be required to conduct one field mission to LACRO Panama to present feedback and provide technical commits as a peer review of the work assigned.

TRAVEL (IF APPLICABLE):

Travel will be covered by UNICEF as per policy.

Travel costs will be estimated and added to the contract once they are determined based on UNICEF Financial Rules and Regulations.

Please note, for travel to countries - it will be decided based on agreements with UNICEF and contractor/consultant. For agreed country visits, the contractor/consultant will be responsible in administering its own travel. UNICEF will reimburse travel related expenses based on actual costs or on the below criteria whichever is lower and upon presentation of receipts.

Any travel involved should be budgeted according to UN Travel Standards as a ceiling.

 -Travel: http://www.un.org/esa/cdo/hr/CIRCULARS/STAI20064%20- %20OFFICIAL%20TRAVEL%20(SECTION%2010).pdf Section 4, paragraph 4.2 , numerals (d) and (e)

-Accommodation (Daily Subsistence Allowance, DSA): http://icsc.un.org/ (all countries and destinations can be found by navigating on the map)

FEES:

Payment will be made in US dollars. Payment distribution will be done according to the table above. A total fee/amount for the deliverables will be proposed by the consultant in the financial proposal. The contract will have a duration of 6 months after the contract is signed.

PAYMENT PROVISIONS:

UNICEF's policy is to pay for the performance of contractual services rendered or to effect payment upon the achievement of specific milestones described in the contract. UNICEF's policy is not to grant advance payments except in unusual situations where the potential contractor, whether an Individual consultant, private firm, NGO or a government or other entity, specifies in the bid that there are special circumstances warranting an advance payment. UNICEF will normally require a bank guarantee or other suitable security arrangement.

Payments will be made upon delivery and approval of deliverables by UNICEF. UNICEF reserves the right to withhold all or a portion of payment if performance is unsatisfactory, if outputs are incomplete, not delivered of for failure to meet deadlines.

Details of payments will be agreed in advance and individually between the consultant and UNICEF.

EVALUATION METHOD AND CRITERIA:

Consultants and/or Individual Contractors (CIC) will be evaluated based on a cumulative methodology, being the award of the contract made to the CIC whose offer has been evaluated and determined as:

  1. Responsive/compliant/acceptable, and;
  2. Having received the highest score out of a pre-determined set of weighted technical and financial criteria specific to the solicitation.

Technical component score

Max. 70 points

Economical component score

Max. 30 points

TOTAL score

Max. 100 points

Only candidates obtaining a minimum of 60 points (of the total technical points) would be considered for the Financial Evaluation.

Technical Proposal

 The technical proposal should include: cover letter referring to the required qualifications, timeline, workplan, proposed methodology, and 3 contact references from previous work experience.

 
 

Financial Proposal

 In addition, separately from the technical proposal (as in another document), a financial proposal detailing the total costs of the assignment (not including costs for travel yet) must be submitted.

The total amount of points allocated for the economic component is 30. The maximum number of points will be allotted to the lowest price proposal and compared among those consultants which obtain the threshold points in the evaluation of the technical component. All other price proposals will receive points in inverse proportion to the lowest price; e.g:

                                                Max. Score for Financial proposal * Price of lowest priced proposal

Score for price proposal X = ---------------------------------------------------------------------------

 Price of proposal X

HOW TO APPLY

To apply, click on the Apply link found either at the beginning or at the end (see example below)

Submit:

  • Technical proposal: cover letter (referring to the required qualifications, timeline, workplan, and proposed methodology, as well as contact references from three (3) previous work experiences)
  • Financial proposal
  • CV

In case of doubts please send your comments to lacro-ops@unicef.org  with the subject: Peer Review – Contribution/Attribution at least 3 working days before the deadline.

Example:
The deadline to receive applications is August 8, 2018. The pre-selection of candidates will be done after closing date. Please note only selected candidates will be contacted.

Important: In the selection of its consultants, UNICEF is committed to gender equality and diversity, without distinction as to race, sex, or religion, and without discrimination to people with disabilities.

Before applying, please make sure that you have read the requirements for the position and that you qualify.
Applications from non-qualifying applicants will most likely be discarded by the recruiting manager.
Apply

What does it mean?

Click "SAVE JOB" to save this job description for later.

Sign up for free to be able to save this job for later.

  • Organization: UNICEF - United Nations Children’s Fund
  • Location: Panama City (United States of America)
  • Grade: International Consultant - Internationally recruited Contractors Agreement - Consultancy
  • Occupational Groups:
    • Monitoring and Evaluation
    • Children's rights (health and protection)
  • Closing Date: 2018-08-09

What does it mean?

Click "SAVE JOB" to save this job description for later.

Sign up for free to be able to save this job for later.