Consultancy: Evaluability Assessment of Country Programme Document

The Programme Section of UNICEF Thailand Country Office is seeking an individual consultant to conduct an Evaluability Assessment of the UNICEF Country Programme Document (CPD) 2017-2021 for Thailand.

If you are a committed, creative professional and are passionate about making a lasting difference for children, the world's leading children's rights organization would like to hear from you.

 

INTRODUCTION : In September 2016, UNICEF’s Executive Board approved the Country Programme Document 2017-2021 (CPD) for the Kingdom of Thailand[1]. After completing 6 months of implementation, the Thailand Country Office (TCO) is planning to conduct an Evaluability Assessment of the CPD to determine if the results for children to be achieved at the end of the Country Programme are logic from the design perspective, measurable and evaluable.

The recommendations derived from the Evaluability Assessment (EA) will be used, if necessary, to further improve the implementation, monitoring and evaluation of the results. This would include the preparation of the systems and tools to conduct the planned evaluations especially the evaluation of the CPD at the end of the cycle (2021). It is also expected that the EA will contribute to strengthen the Result Based Management and Programming skills among internal staff and key stakeholders.

 

PURPOSE AND OBJECTIVES

The primary purpose of this Evaluability Assessment is to determine whether the causal pathways towards results for children are logic and to review the adequacy of the mechanisms to track performance and demonstrate results for children at different levels. The secondary purpose is to provide recommendations to the management and relevant stakeholders on how to improve the tools and systems to better implement, monitor and evaluate those results.

 

Specifically the exercise will include:

1. Assessment of the logic and  alignment of the planned results based on a desk review of the planning documents and information collected through an online survey and interviews with staff and key stakeholders  (vertical assessment)
2. Assessment of the adequacy of the mechanisms and systems in place to collect, analyse and use data to track performance and demonstrate the results. This will be reviewed by looking into the existing M&E frameworks for each Country Programme (CP) outcome area, the Evaluation Plan and other support documents as well as information collected through strategic interviews with staff and key stakeholders (horizontal assessment).
3. Recommendations to further improve the design, data collection systems and evaluation readiness of the CP outcomes. This would include the preparation of the systems and tools to conduct an evaluation of the CPD at the end of the cycle (2021).

 

---

 

SCOPE

The scope of the Evaluability Assessment will be limited to the Country Programme (2017-2021), the results that aimed to be achieved, as well as the M&E framework associated to these results. The Country Programme comprises four components covering seven outcomes and a mix of strategies. Each of the result was designed under the Theory of Change that explains the pathways towards changing the lives of children, starting from a situation analysis. The provided documentation will be also framed on the CPD, unless is considered background documentation.

The analysis should be framed in the overall goal of the Country Programme: to contribute to national efforts to enable all children in Thailand, especially the most disadvantaged and excluded, to have their rights progressively fulfilled and to develop to their full potential in an inclusive and protective society.

 

The evaluability assessment will:

- Determine the clarity of intent of the CP in general and of its components to understand whether the planned results for children are logic and achievable. Assessment of the financial resources will be excluded from the analysis.

- Review the availability and validity of quality indicators, baselines and targets. This will determine whether the progress of different outcomes and outputs is adequately measured and will allow to identify any significant gaps in coverage. This will also provide inputs to improve the development of the monitoring and evaluation frameworks. The assessment of the availability and frequency of data to monitor indicators will be an important part of the exercise, based on the provided documents and results of the interviews.

- Review the evaluation readiness of the planned evaluations by mapping the current situation in terms of alignment with the CP outcomes, availability of information, definition of a methodology, timing, conduciveness of the context, and accessibility to and availability of stakeholders etc. This will be based on the planning documents and the knowledge and perception of internal staff and key stakeholders.

- Provide an indication whether an evaluation of the CPD at the end of the cycle (2021) would be feasible, credible and useful. It will identify what overarching questions the evaluation could focus on.

- Provide recommendations to adapt the CP design, monitoring and evaluation systems M&E systems and capacity development

- The assessment will be conducted mainly remotely, with a one-week country mission.

- Agreement on methodology for the Evaluability Assessment will be detailed in the inception report. However, it will include: 

• Desk Review of the planning documents (CPD, Integrated Monitoring and Evaluation Plan, Strategic notes, Work plans and Programme Cooperation Agreements etc.) as well as others as agreed.
• Online consultations
• Online  Survey
• One-week Country Mission involving meetings and interviews with internal staff and key stakeholders

 

EVALUABILITY ASSESSMENT QUESTIONS

Assessment of the design of the planned results for children

Logic

- Does the CPD have gaps in the causal pathway that would affect the likelihood of UNICEF achieving the planned outcomes?

- Have risks and assumptions been adequately identified? Is the design flexible and responsive to external factors?

 

Alignment

- Are the Country Programme’s outcomes aligned to the country’s priorities and based on a Situation Analysis of Children?

- Does the CPD have the potential to be aligned with the new UNICEF Strategic Plan and the Regional Headlines?

- To what extent does the CPD address equity (including gender issues) and will progress towards removing bottlenecks to achieving outcomes for children?

- Agreements (Workplans PCAs etc) are aligned with the CPD and directly contribute to achieve results for children?

 

Assessment of the mechanisms and systems to monitor and evaluate results

Measurability

- To what extent are there sufficiently measurable indicators for each programme component expected output and outcome? Do they adequately capture the expected change?

- To what extent are there reliable sources of information?

- Are there SMART performance indicators with clearly identified means of verification?

- Are there adequate baselines and targets?  Are the formulated targets plausible and realist?  What is the strategy to obtain the baselines where they have not been formulated?

- Do the performance indicators have the potential to be monitored regularly on quality and on time?

- Are the monitoring systems of the partners in place to track the progress of the indicators according to the knowledge and perception of UNICEF staff and stakeholders?

- Are sufficient human capacities in place to monitor and evaluate progress towards the CPD?

- Are cross-cutting themes adequately measured?

 

Evaluation Readiness

- Is the necessary documentation, baselines and indicators available to conduct the planned evaluations? If not, are the mechanisms in place to collect the relevant information?

- Are the planned evaluations aligned to the outcomes and respond to the Theory of Change?

- Has the initiative accumulated enough implementation experience to enable useful lessons to be extracted?

- Have the primary users of the evaluation been clearly identified?

- Is the environment to implement the evaluation conducive in terms of availability and accessibility of data and stakeholders?

 

Recommendations

- Which tools and institutional arrangements should be considered to improve the implementation, monitoring and evaluation of the results?

- What are the main considerations and recommendations to be taken into account when planning an evaluation including the CPD evaluation at the end of the cycle?

 

WORK ASSIGNMENT:

Under the supervision of Monitoring and Evaluation Specialist, the consultant will perform the following tasks:

 Inception Phase:

1. Inception consultations (online);
2. Initial desk review;
3. Drafting of the inception report including methodology, data collection tools, list of interviews, outline and work plan.

 Development/Data collection:  

4. Desk review and analysis;
5. Internal/external meetings (in country);
6. Interviews with internal staff and key stakeholders (in country).

Validation and finalization:          

7. Presentation to internal staff and key stakeholders (online);
8. Final report writing including comments and feedback.

 

WORK SCHEDULE:

Phase: Inception

Activity: 

-  Inception consultations (online)

- Initial desk review

Timeframe:  4 days

 

Phase:  Development/ Data collection

Activity:

-  Desk review and analysis

- Internal/external meetings (in country)

- Interviews with internal staff and key stakeholders (in country)

Timeframe:  17 days + 3 days for internal review

Phase: Validation and finalization

Activity: 

-  Presentation to internal staff and key stakeholders (online)

- Final report writing including comments and feedback

Timeframe: 6 days

 

PRODUCT(S):

• Inception report, including methodology, data collection tools, list of interviews, outline and work plan;
• Draft of the Evaluability Assessment report with preliminary findings and recommendations;
• PowerPoint Presentation with preliminary findings and recommendations;
• Final Evaluability Assessment report.

All materials produced or acquired under the terms of this assignment shall remain the property of UNICEF. UNICEF will retain the exclusive right to publish or disseminate such materials even after the termination of this consultancy.

 

ESTIMATED DURATION OF CONTRACT:  30 working days, preferably before mid-September 2017.

 

OFFICIAL TRAVEL: 

One-week official travel to Bangkok, Thailand to conduct face to face consultations and interviews with UNICEF internal staff and key external stakeholders.

 

QUALIFICATIONS OR SPECIALIZED KNOWLEDGE/EXPERIENCE REQUIRED:

• Master’s degree or equivalent in Development Studies, Economics, Social Studies, International Relations or other related field;
• At least 10 years of experience in conducting or managing evaluation assessments or evaluations at the strategic and policy level, or developing Result Based Management (RBM) planning documents and strategies;
• Experience working with UNICEF and/or UN agencies; 
• Strong understanding on RBM, planning, monitoring, evaluation, UNICEF’s policies and programming practices, equity, gender equality and child rights;
• Commitment to equity and gender sensitive analysis.

 

APPLICATION PROCESS:

Interested applicants are requested to submit Application Package and Expression of Interest by 25 August 2017 detailing:

• Qualification and technical capacity (CV/P-11);
• 2/3 pages proposal including :

           a) Timeline;

           b) Methodology;

           c) Two relevant reports of evaluability assessments or evaluations conducted in the past;

           d) Financial proposal with: a) Daily professional fee in USD. b) All-inclusive travel cost (ticket and other incidentals) from home base to the UNICEF Office in Bangkok.  Please note the air tickets under UNICEF contract shall be re-routable, refundable type (economy class and most direct route). The consultant will be responsible for own travel insurance, visa fee and terminal expenses.

--------------------------------- 

Only short listed candidates will be notified.

 

 VA2017-58 Consultancy - Evaluability Assessment of CPD.pdf

 FORM P11.doc