Consultancy (thematic expert) for Impact Feasibility Assessment (IFA) of UNICEF Mental Health & Psychosocial Support (MHPSS), 7 months, 50 days, Evaluation Office, NYHQ

UNICEF Evaluation Office is seeking to recruit an experienced impact evaluation expert (mental health and psychosocial support) to lead the impact feasibility assessment as a part of multi-country evidence strategy to build rigorous evaluation evidence base on Mental Health and Psychosocial Support services in countries where UNICEF is taking a substantive role in supporting relevant interventions. The work will contribute to the strategic global effort of building rigorous evidence base at the outcome and impact level to improve UNICEF programming.

UNICEF Evaluation Office is seeking to recruit an experienced thematic expert (mental health and psychosocial support) to contribute to the impact feasibility assessment as a part of multi-country evidence strategy to build rigorous evaluation evidence base on Mental Health and Psychosocial Support services in countries where UNICEF is taking a substantive role in supporting relevant interventions. The work will contribute to the strategic global effort of building rigorous evidence base at the outcome and impact level to improve UNICEF programming.

For every child, evaluate

One in four children or adolescents have a caregiver with a mental disorder. Nearly one billion people throughout the world live with a mental health condition and more than 80 percent of them reside in low- and middle-income countries. In those settings, between 76 per cent and 85 per cent of people with mental health conditions receive no treatment for their condition. Poor mental health is both a cause and a consequence of poverty, compromised education, gender inequality, ill-health, violence and other global challenges. People living with mental health conditions experience disproportionately higher rates of disability and mortality. It impedes the individual’s capacity to work productively, realize their potential and make a contribution to their community. The risk for mental health conditions and psychosocial problems among children and adolescents is exacerbated when facing poverty, violence, disease or humanitarian crises.

UNICEF has scale up investments in Mental Health and Psychosocial Support (‘MHPSS’) in the Strategic Plan 2022-2025. The UNICEF’s MHPSS direction is articulated across three Goal Areas: health, education and child protection. Such multi-sectorial integration is an important step towards strengthening institutional capacity and accountability to respond to the MHPSS needs of children, adolescents and families around the world. UNICEF’s MHPSS multilayered and multisectoral interventions focus on support to children, adolescents, caregivers, families and the wider community. UNICEF deepened its commitment to deliver MHPSS in 2019 by providing community based MHPSS to more than 3.7 million children and adolescents (up three per cent compared to 2018) across 60 countries and almost 517,000 caregivers in 41 countries. MHPSS activities are implemented through:

(a) child protection and social protection services, including in response to child protection concerns such as issues of abuse and violence.

(b) education and socioemotional learning focused activities in school and out of school programming.

(c) health- and HIV-focused programmes that support mental health and psychosocial wellbeing of adolescents.

In addition to these programmes, UNICEF has made important investments in evidence-generation for MHPSS work.  In the last years and in collaboration with different partners, UNICEF took steps to develop measurement standards and generate data on mental health. Those include:

• The Module on Child Functioning for use in censuses and surveys captures information on anxiety and depression.
• The Measurement of Mental Health Among Adolescents at the Population Level (MMAP) suite of tools and standard procedures which guide data collection at the population level for anxiety and depression, functional impairment due to mental health conditions, suicide ideation and attempt and psychosocial support.
• The Multiple Indicator Cluster Surveys (MICS) which generate data on key indicators on the well-being of children and women largest source consider anxiety and depression as functional domain and provide estimates.
• The evidence gap map of child and adolescent mental health and psychosocial support interventions provides an overview of child and adolescent MHPSS intervention research in low- and middle-income countries since 2010.

How can you make a difference? 

The purpose of this assignment is to conduct an independent assessment of the opportunities and limitations for designing and conducting rigorous impact evaluations of selected scalable and innovative interventions on MHPSS where UNICEF is taking a substantive role in supporting an intervention. The scope of work shall neither favor nor preclude any of the possible options but rather create the opportunity to systematically assess all expertly defined feasible and rigorous alternatives.  

This impact feasibility assessment (IFA) will provide input to the evaluation plan of MHPSS 2022-2030 and recommendations to the global MHPSS team. It will propose methodological approaches and adaptations that would be required to achieve robust and coherent evaluation designs across multiple countries and regions. A set of transparent criteria for the selection of appropriate methodological approaches should be finalized at the inception phase of this assignment.

Scope

This assignment will focus on the two outcome areas of the ToC with reference to indicator framework of the MHPSS Global Framework (2022). 

• Assess identified global evidence gaps at outcome and impact levels in reference to the programmatic focus defined in the MHPSS Global Framework (2022), making use of existing synthesis and maps of evidence on MHPSS such as the evidence gap map of child and adolescent mental health and psychosocial support interventions elaborated by the UNICEF Office of Research.
• Drawing on the typology of interventions identified in the MHPSS evaluative baseline 2018-2021 elaborated by the UNICEF Evaluation Office, select the most innovative, potentially scalable interventions (or their combination) and assess the plausibility of change at outcome and impact results (based on global evidence, perceptions of programme staff as well as data from monitoring and reporting systems).
• Conduct cross-country comparisons to identify the most appropriate and feasible (sub)national and programmatic contexts for measuring effects at outcome and eventually impact levels and discuss requirements and conditions imposed by humanitarian and fragile contexts.
• Propose the evaluation designs (if any) to measure attribution (if deemed feasible and appropriate) of the selected interventions. The proposed design/s should: 
  • Identify design aspects of MHPSS interventions at country level in relation to the global theories of change, their outcome priorities and pathways and current programme characteristics.
  • Mention the suitability of available sources of data proposed for use: existing monitoring and reporting systems at country level, programmatic surveys, etc. to measure relevant indicators and overall fit for purpose and able to provide granular data at the community and sub-national levels. 
  • Articulate the steps, budget and timeline to integrate any identified impact evaluation components into the current programming and already planned evaluations (considering that primary data collection may need to be collected, etc.).
• Identify methodological elements relevant to the MHPSS evaluation plan, to achieve a greater evaluative focus on impacts and outcomes.

Methodology

Stage 1. Stock taking on ‘what works?’ with reference to the Theory of Change, the UNICEF global outcome indicators framework and countries of the MHPSS Evaluative Baseline 2018-2021. The objective of this stage is to conduct rapid review of the most recent rigorous evidence to identify which UNICEF MHPSS interventions have the potential the most scalable and transformative. The IFA team will build on the wealth of information generated and systematized by the organization, including the 2022 MHPSS Evidence Gap Map from UNICEF Office of Research – Innocenti, and also information available externally in peer review journals and other organizations. The findings of this stage will be aligned with topics from the MHPSS, in particular the outcomes from the ToC of the MHPSS Global Framework. This work will be done through a desk review of the identified literature and consultations (remote) with a selected number of experts and UNICEF programme staff.  

Stage 2. Mapping and selection of interventions and narrowing down the set of candidate interventions in the light of country contexts (e.g. fragility, conflict, prevalence rate of mental health issues), and intervention characteristics based on a number of technical criteria which has to be presented and discussed in detail. The most innovative, scalable interventions and plausible to achieve outcome and impact level change should be selected for further investigation. The results from the stage 1 (stock taking on what works), and particularly the gaps in certain programmatic areas, should be considered as a relevant criterion for the final selection of interventions. This stage will combine a desk review on the country-level data with the first round of consultations with all country offices to verify intervention modalities, contextual conditions, prioritization of MHPSS by national partners, etc.

Stage 3. Impact Evaluation design (programmatic ‘deep dive’). This stage of the assessment focuses on the selected (as per stage 2) subset of interventions from the initial list of nineteen countries mentioned in the scope section. The objective is to better understand the specific programme/intervention logic (TOC) and modalities, design, implementation conditions, geographic coverage across communities, and the timeline.  This is critical to make a reasonable judgement (based on selected technical criteria) on whether we can achieve internal validity by accurately estimating the counterfactual through a valid control/comparison group, use natural experiment or adopt a theory-based approach to assess ‘contribution’ rather than attribution of the programme.[1] The result of this stage will be an expert agreement on the feasibility of constructing a rigorous counterfactual in selected countries and recommendation on the design options (or a combination of approaches).

The IFA team will consult (remotely) with the staff from selected COs, the corresponding Regional Offices, and the MHPSS global team, to obtain adequate information and understand details of specific interventions, including geographic distribution and targeting, available data, timeframe, scalability plans. Diagnostics of available data sources is an important aspect of analysis at this stage and should include the country (CO and government) monitoring and situational data with the focus on their suitability to be used credibly (as an alternative to primary data collection or as a complementary source). The staged will be entirely based on more detailed cross-sectorial consultations at the country level (done remotely).

Stage 4. Integration into the MHPSS evaluation plan 2022-2030. The objective of this stage is to assess implications, including methodological and financial ones, of integrating the ‘rigorous impact component’ into the MHPSS evaluation plan 2022-2030. The IFA will generate a report indicating concreate recommendations on the following elements:

• The evidence-based rationale for integrating/adding a rigorous impact component into the global evaluation plan demonstrating how proposed evaluations contribute to and advance programmatic learning and improve effectiveness of UNICEF global efforts in MHPSS
• Operationalized outcome indicators in application to specific countries context 
• Data requirements.
• Conditions for methodological coherence and complementarity of different evaluation approaches and designs.
• Cost and time implications; and
• Any other key issues that might arise.

To qualify as an advocate for every child you will have… 

The IFA will be led by a team of two/three experts with extensive knowledge and experience in conducting mixed methods impact evaluations and thematic experience on mental health and psychosocial support. The thematic expertise should include MHPSS and could be linked more strongly with a particular area (e.g. adolescent mental health, psychosocial support in educational settings, psychiatry, psychology, etc.).

UNICEF reserves the right to change the distribution of days between thematic and impact evaluation experts based on the complementary profiles of selected experts. The consultants must demonstrate a clear understanding of the UN system and ensure that the feasibility assessment is conducted in line with the UNEG Norms and Standards for Evaluation in the UN System and abides by UNEG Ethical Guidelines and Code of Conduct. UNEG guidance on Integrating Human Rights and Gender Equality in Evaluation should also be reflected throughout the evaluation.

Required qualifications and expertise of thematic evaluation expert:

• Academic qualifications. Graduate degree in Social Science or relevant discipline (psychology, psychiatry. sociology, economy, or related field).
• Extensive experience in conducting complex, thematic, multi-sectoral evaluations for international development organizations with in-depth knowledge of evaluation methodology and mix-method approaches, particular for multi-country, global evaluations.
• In-depth knowledge and proven expertise in areas of child and adolescent MHPSS wellbeing, MHPSS in child protection settings or in health or education settings, psychology, psychiatry.

The expert should demonstrate the following skills and competencies

• Proven experience with the ethics of evidence generation; experience collecting data from vulnerable groups; familiarity with ethical safeguards.
• Familiarity with the UNICEF programmatic mandate in child and adolescent MHPSS wellbeing and understanding of evidence generation process in this area is an asset.
• Delivering evaluations and research projects with tight deadlines, complex national contexts and multi-stakeholder consultative process.
• Applied knowledge and application of UNEG norms and standards.
• Excellent abilities in presenting technical information to a non-technical audience, including excellent drafting and presentation skills in English. 
• Strong ability to interact with a wide range of stakeholders, particularly on issues that are politically sensitive. 
• Proficiency in English is required. French is an advantage.

How to apply

Interested candidates must submit the following documents:

1. CV and cover letter.
2. The financial proposal should indicate consultant’s daily rate and expected total budget with a breakdown cost for each stage of the work. 
3. The application should be accompanied by short examples (through links provided or attached documents) of analyses that show experience and competence to undertake this consultancy in line with the required qualifications described above.
4. A consultant/consultants can apply alone or as a team. The consultant can subcontract part of the work to complement the expertise (e.g. provision the work of research assistant). A clear explanation has to be given in the proposal on how the skills and experiences of a sub-contractor or a co-investigator will benefit the process and the quality of the deliverables. 

For every Child, you demonstrate… 

UNICEF is here to serve the world’s most disadvantaged children and our global workforce must reflect the diversity of those children. The UNICEF family is committed to include everyone, irrespective of their race/ethnicity, age, disability, gender identity, sexual orientation, religion, nationality, socio-economic background, or any other personal characteristic.

UNICEF has a zero-tolerance policy on conduct that is incompatible with the aims and objectives of the United Nations and UNICEF, including sexual exploitation and abuse, sexual harassment, abuse of authority and discrimination. UNICEF also adheres to strict child safeguarding principles. All selected candidates will be expected to adhere to these standards and principles and will therefore undergo rigorous reference and background checks. Background checks will include the verification of academic credential(s) and employment history. Selected candidates may be required to provide additional information to conduct a background check. 

Remarks:  

Only shortlisted candidates will be contacted and advance to the next stage of the selection process. 

 ToR MHPSS Impact Feasibility Assessment - Thematic expert.pdf