Consultancy for Evaluability Assessment of Social Protection in the CEE/CIS Region

Duty Station: Home-based Duration: 42 working days Time Frame: 1 September - 31 December 2017. Social Protection Section in Regional Office for CEE/CIS is seeking for a consultant to undertake the Evaluability Assesment of Social Protection in the CEE/CIS Region with the following principal objectives to assess: • The evaluability of national social protection policies and programmes for children in the region; the extent to which the progress of national social protection systems and their evolution towards the realisation of children’s rights can be evaluated, as well as UNICEF’s contribution to any changes • The availability of relevant information from international and national sources, together with UNICEF internal documentation on its programming. • The practicality and utility of an evaluation to UNICEF and to national partners, given the nature of UNICEF’s work in this area (geographically and substantively); • The potential for applying robust impact evaluation measures in a small number of countries where UNICEF is working.

If you are a committed, creative professional and are passionate about making a lasting difference for children, the world's leading children's rights organization would like to hear from you.

For 70 years, UNICEF has been working on the ground in 190 countries and territories to promote children's survival, protection and development. The world's largest provider of vaccines for developing countries, UNICEF supports child health and nutrition, good water and sanitation, quality basic education for all boys and girls, and the protection of children from violence, exploitation, and AIDS. UNICEF is funded entirely by the voluntary contributions of individuals, businesses, foundations and governments.

Terms of Reference for an Evaluability Assessment (EA)

of Social Protection in the CEE/CIS Region

1. Why and for whom the evaluability assessment is being done

Social protection has become a key area of engagement and investment for UNICEF in the Europe and CIS region.  A Regional Knowledge and Leadership Area on ‘A Child’s Right to Social Protection’ (RKLA 8) was established in 2012, with the aim of bringing together the work of UNICEF’s Regional office and country offices to share knowledge and approaches, and drive common results for children.

UNICEF’s Social Protection Strategic Framework defines social protection as the set of public and private policies and programmes aimed at preventing, reducing and eliminating economic and social vulnerabilities to poverty and deprivation^[1]. UNICEF's focus is on strengthening national social protection systems towards ensuring they are child sensitive, and is built on the following four blocks of policies and programmes in particular:

• Social assistance cash benefits or in-kind transfers and social insurance benefits. This is typically the core domain of UNICEF’s work in social protection in the CEE/CIS region.
• Programmes which facilitate children, adolescents and families' use of health care, education, early learning, and other services. In the CEE/CIS region, these programmes include a variety of interventions many of which are addressed through different components of UNICEF programmes:  
• Interventions to enhance education enrolment, attendance and attainment: school-fee abolition, school transport or transportation allowance for children with disability, free textbooks and free school meals, and education services, partial/total financing of crèches and kindergartens for disadvantaged children, Roma education mediators, social pedagogues in schools.
• Interventions to enhance access to health care: health insurance, waivers on health expenditures for children and pregnant women, measures to fight corruption and to eliminate informal payments in health, Roma health mediators, home visiting, social workers in maternity wards.
• Other: social canteens, fuel and other utilities subsidies.
• Social services, which provide care and support to families and children at-risk. In the CEE/region, these services include a variety of interventions, many of which are being addressed through different components of UNICEF programmes:
• Interventions to address social vulnerability and provide additional social support:  social counselling including mobile units in rural and highland areas, home and respite care, day and week care centres, home stays for children with disabilities, early intervention services, shelters, hostels and emergency assistance, family assistance including consultation family centres (including family education, parental skills, periodically organised childcare, assisting in resolving family problems), centres for social work and social workers, outreach programmes, special social services for children who require rehabilitation or intensive treatment by skilled professionals, administration assistance and conflict resolution.
• Interventions to prevent, care and treat children affected by HIV and AIDS: contact centres for drug addicts, counselling for young people, home nursing and home social care, centres for HIV positive children and youth.
• Other: recreation centres, counselling for employment/job search centres/services.
• Legislation and policies that address structural causes of discrimination and exclusion (e.g., maternity and paternity leave, inheritance rights, etc.) and which help remove inequalities in access to services or livelihoods/ economic opportunities.

UNICEF’s work in the region in support of social protection includes work in the majority of countries, with a strong focus on assessments of social protection benefit systems, support to revision of targeting and eligibility of benefits, supporting the development of social support services and enhancing system building by pursuing integrated approaches.  A further strong focus has been on outreach and extending access to social protection benefits and services to children from excluded groups.  In 2016, UNICEF CEE/CIS published the ‘Social Monitor 2015: Social protection for child rights and well-being in Central and Eastern Europe, the Caucasus and Central Asia’, which highlighted a number of ongoing issues in connection with social protection for children in the region.   This led to the development of a full-fledged regional strategy on ‘A Child’s Right to Social Protection’, which was finalised and adopted in 2016. The strategy serves as a common framework for social protection interventions in the region, underpinned by six key outcome level results, adopted in particular by eight UNICEF Country Offices of the region.

1.1 Purpose(s) of the Evaluability Assessment

The principal objectives of the evaluability assessment is to assess:

• The evaluability of national social protection policies and programmes for children in the region; the extent to which the progress of national social protection systems and their evolution towards the realisation of children’s rights can be evaluated, as well as UNICEF’s contribution to any changes
• The availability of relevant information from international and national sources, together with UNICEF internal documentation on its programming.
• The practicality and utility of an evaluation to UNICEF and to national partners, given the nature of UNICEF’s work in this area (geographically and substantively);
• The potential for applying robust impact evaluation measures in a small number of countries where UNICEF is working.

There are also a number of supplementary objectives of the evaluability assessment: 

• To assess the validity of UNICEF’s Theory of Change on social protection for the CEE/CIS region, and propose refinements of the regional strategy, including for key indicators.
• To assess the adequacy of the RKLA 8 Strategy/common results framework and designated metrics for evaluating UNICEF’s contributions to national SP policies and programmes
• To identify areas where national monitoring systems could be strengthened to improve the availability of information for evaluating social protection for children
• To propose options for filling critical data gaps and for an eventual evaluation design, including

1. Proposed timing for an evaluation
2. Potential evaluation questions
3. Proposed Evaluation methods, including impact assessments if feasible
4. Resources and expertise for an evaluation

1.2 Primary intended users and uses of the Evaluability Assessment

The primary intended users of the EA are the Regional Management Team (RMT), and all stakeholders of this workstream for the region, the RKLA Steering Group (SG) for RKLA 8 on A Child’s Right to Social Protection.  In addition it is expected the EA will be of interest to specific country offices in informing their programming and dialogue with national counterparts. 

The primary use of the EA is to assist in determining whether to conduct a full evaluation of UNICEF’s work in this thematic area, and if so to assist in defining the scope, timing and steps towards an evaluation. 

A further intent is to consider the potential and possible designs for impact evaluations of social protection in one or two countries in the region, and to recommend further steps towards this if feasible.

1.3 Key Evaluability Assessment questions

The EA will consider the following key questions related to the design of UNICEF’s social protection programming, availability of information, and the institutional context for an evaluation.

(a) Design

Are the long-term impact and outcomes of programming clearly identified and are the proposed steps towards achieving these clearly defined?  Is the target group clearly identified and is programming clearly relevant to the needs of the target groups, based on situation analysis or baseline study?  Is there a clear causal chain connecting UNICEF’s interventions with the proposed outcomes and impacts?  Does UNICEF programming use valid, measurable, and reliable indicators for monitoring its programming, outcomes and impacts?  Are assumptions about the roles of other actors explicit and monitored?  To what extent are UNICEF’s objectives shared with those of other stakeholders in social protection?

(b) Information availability

What information is available for assessment of UNICEF programming and is it complete (eg: country plans and annual workplans, concept notes, progress reports, evaluations)?  Is baseline data available or if not is there a plan to collect it?  Is baseline data of suitable quality and sufficiently disaggregated, including by sex, to assess the intended outcomes or impacts?  Is there data on a control group, and if yes is it clear how the control group compares to the intervention group?  Is data collected for all indicators, with sufficient frequency, and are the measures reliable?  Are the reports of reviews or evaluations available?  Where data is not yet available, do the existing systems have the capacity to provide required data in the future, are responsibilities, sources and time periods defined and appropriate, and is the budget for data collection adequate? 

(c) Institutional context

Is an evaluation relevant to UNICEF’s work in the region?  Would an evaluation be likely to influence social protection policies and programmes, including UNICEF programming, in the region?  Has UNICEF accumulated enough implementation experience to enable useful conclusions and lessons to be drawn? Which stakeholders should be involved in an evaluation and what forms and fora for coordination could be utilised?  Who would be the primary users of an evaluation, and how would they participate in the evaluation process?  What evaluation issues are of interest and to whom?  How would the results of an evaluation be used?  Are there any ethical issues in an evaluation and if yes, how could these be managed?  What would be the likely cost of an evaluation and would funding be available?

 2. How the Evaluability Assessment will be accomplished

2.1 Overall scope and approach

The main objects of interest are national social protection policies and programmes for children in the region, the coverage and adequacy of social benefits, and their impact on child poverty.  It is not expected that the entirety of national social protection systems is considered, only those most relevant to addressing children’s economic and social vulnerability, principally social cash benefits covering children and social care and support services.  UNICEF’s contributions to system level changes in social protection are also of importance.

UNICEF has been engaged in supporting Social Protection to a limited extent and in a limited number of countries[2] for 4-5 years. The UNICEF Regional Knowledge and Leadership Agenda (RKLA), which includes social protection results for children, was formally agreed by the Regional Management Team in 2012. UNICEF is commissioning this evaluability assessment to understand better if an evaluation of work under the RKLA for Social Protection would be useful at this time - or if, in fact, further work needs to be done to capture baseline and early results.

The evaluability assessment has been initiated by the Regional Office of CEE/CIS (Social Policy and Monitoring & Evaluation Sections in coordination with the RKLA 8 Steering Group Chair and Members) after consideration by the RMT and the Regional Office committee for research, studies and evaluations (RSE).  The results of the evaluability assessment will feed into decision-making about the timing and scope of further data gathering and evaluative work, as inputs into the rolling regional evaluation workplan 2017/18.

2.2 Evaluability Assessment (EA) Methodology

The evaluability assessment will involve extensive desk review (see bibliography) and a cross section of light interviews at regional (RO) and UNICEF country office (CO) level, the suggested number of interviews is six (four country offices, RO and HQ).  No interviews with Government are envisioned at this time, although UNICEF CO staff should use the EA process as an opportunity to discuss the interest of their counterparts to engage in a systematic review process - to promote engagement at an early stage in an eventual evaluation process.

Similarly both UNICEF Programme Division and the Office of Research in Headquarters will be stakeholders in any eventual evaluation exercise. The responsibility for keeping these two stakeholder groups informed about the EA process and results will be with the EA Manager, the SP Advisor.

A key risk is that funds will not be available for recommended additional evidence generation/evaluative work.  The EA is timed to coincide with RO annual work plan and budgeting processes - and the RO is looking into pooled funding arrangements that will allow COs to participate in any multi-country initiatives. It is anticipated that partnerships around SDG1 (poverty and social protection) and new UNICEF investments associated with the roll-out of UNICEFs Data Strategy will create an enabling environment for stronger monitoring systems.

2.3 Evaluability Assessment Outputs

• Assessment of UNICEF social protection intervention monitoring data (Country Programme Documents, Country Annual Work plans) including recommendations that will make UNICEF work more evaluable (in terms of measuring contribution to national results);
• Assessment of data availability (from national and international sources), baseline and trend data, and the systems & capacities to make it available, including recommended changes that will improve evaluability;
• Assessment of the context and trends in CEE/CIS with respect to Social Protection 2000-2016.
• A judgement as to whether the results of social protection and UNICEF’s contribution to them can be evaluated at this stage.  Recommendations on what is needed to make it so (including timing, people & resources required).
• Determination of the likely level of demand for an evaluation, based on the desk review and interviews. Which stakeholder interests can or should be addressed? What evaluation questions and designs could meet their needs, given national policies and programmes and likely data availability? Judgement on the usefulness and feasibility of conducting more systematic impact evaluations in selected countries.

 3. EA Services and Management

The Evaluability assessment will be undertaken by a service provider with relevant expertise both in EA and in social protection. 

3.1   Roles and responsibilities

The Evaluability Assessment will be jointly managed by the Monitoring and Evaluation Advisor and Social Policy Regional Advisor. The RSE Committee Chair and RKLA 8 Steering Group will act as reviewers and the final assessment outcome will be endorsed by the Regional Director. 

In order to avoid any actual or perceived conflict of interest the Evaluability Assessment includes not only the two RO staff mentioned but also a staff member from the Office of Evaluation.  The contracted service provider for the EA will be ineligible for any follow up UNICEF data and evaluation related work created as a result of the EA.

4. Milestones, deliverables and timelines

UNICEF is committed to diversity and inclusion within its workforce, and encourages qualified female and male candidates from all national, religious and ethnic backgrounds, including persons living with disabilities, to apply to become a part of our organisation.