Consultancy International and National, Evaluation of Mali Country Office Field Delivery Structure

Evaluation Team composed of : Senior Team Leader P5 Evaluation expert P4 National Consultant NOC

Overview of the current field offices structure:

Coordinated from the principal office in Bamako (the Country Office), the UNICEF MCO operation functions through 5 zonal offices (Zos) - Kayes, Sikasso, Mopti, Gao and Timbuktu – which are located at distances between approx. 250 and 1100km from the capital. Cumulating 65 staff across all functions, including administration and operations support, they vary in staffing numbers from 5 (Timbuktu) to 20 staff (Mopti) and expertise for each programme component (Health, Nutrition, WASH, Education, Child Protection) and only the two largest ZOs have outposted monitoring and evaluation capacities. Two northern ZO, namely Gao and Mopti, respectively have dedicated security positions at specialist and assistant levels. The main roles of the ZO include, among others, the following:

1. Implementation of UNICEF programme as per the CP
2. Ensuring UNICEF resources are optimally utilized for intended purposes.
3. Coordination of recovery and development activities in collaboration with the government and other partners
4. Providing technical assistance to decentralized government structures and other partners
5. Support to emergency preparedness and response
6. Monitoring and evaluation of UNICEF-supported programmes
7. Representation at State and local levels

Rationale for evaluating the ZOs

This evaluation, included in MCO’s Costed Evaluation Plan 2015-2019, is part of the strategic MTR and, by reviewing how UNICEF does business through the ZOs and effects change for children, will inform potential improvements/strengthening of systems and processes in the Country Office/zonal offices structures and the relationships between them to the effect of enabling the most efficient, cost-effective, sustainable and equitable delivery of results for the children of Mali.

Evaluation purpose

An external evaluation of the ZOs of the UNICEF MCO is being undertaken for the purposes of accountability and learning.

• The measurement and analysis of the originally intended impacts of the UNICEF MCO decentralisation process (i.e., accountability to report on the impact of a management strategy).
• To encapsulate learning and provide recommendations for a more efficient and effective:
  • management strategy of UNICEF programme implementation
  • representation and advocacy capacity at the field level
  • knowledge management and sharing between the Country Office and zonal offices

These recommendations will inform decision-making on strategic positioning and organisational effectiveness of the UNICEF MCO, with possible uptake in other similar operational contexts. The evaluation will also help clarify lines of accountability in a ZO business model and will highlight the value added of field presence for Cos where there is currently no field presence

Evaluation objectives

The evaluation is primarily summative in function and will assess the impact, effectiveness, efficiency, and relevance of the ZO business model on programme delivery and results in achievement for the period of January 2015-17. The evaluation will provide an assessment of the current situation, structure, and systems of the ZOs in the UNICEF MCO, including in the emergency response, through the coverage and coordination.

Evaluation scope

The evaluation will cover: 

• Rationale and strategic vision of the ZO structure and system of functioning; thereby determining current needs of the MCO CP and CO and recommending plans for subsequent ZO structures
• The management strategy of choosing the ZO modality for programme implementation in the MCO by assessing the efficacy of operations in areas/states with no zonal offices;
• The current delegation of authority, distribution of responsibilities and accountabilities;
• The structure and oversight arrangements, including internal communication;
• The costing structure and funding modalities of the ZO set up to informing a cost-benefit analysis;
• The current management strategy to address risks and mitigation measures;
• The current ZO closure and exit strategy.

The evaluation will identify, in each of the sub-categories listed above, areas in need of review/strengthening and propose an action plan addressing identified gaps and weaknesses. The evaluation will provide several suggested courses of action, which also take into account the evolution of the social, political and security situations. Suggested scenarios should also take into consideration low, adequate, and high resource availability projections to inform management decision-making and provide action plans with detailed proposed time frames.  

Evaluation framework

While the contractors should refer to frameworks of strategic and importance to UNICEF, such as the Strategic Plan 2014-2017, the MDG/SDG objectives, the human-rights based approach to programming (HRBAP) and Core Commitments for Children in Humanitarian Action (CCC) for a general framing of the proposed evaluation instruments, the main framework for the external evaluation will be the UNEG evaluation criteria aiming to measure the impact, relevance, effectiveness, efficiency and sustainability of the zonal offices roles/functions with regard to achieving CP results for the population UNICEF serves. Given the humanitarian conditions prevailing throughout the period of evaluation, it will also be assessing coverage, coordination with regard to zonal offices emergency response with all relevant partners, given that they are involved in the management of potential risks identified in the emergency preparedness plan (including armed conflict, floods and epidemics).

For each of the above criteria, the external evaluation may consider providing answers to the following indicative questions, which – for many of them – derive directly from the functions/roles of the zonal offices. More specific questions, for each of the functions of the zonal offices and generally, will be developedby the evaluation team as part of their methodology and workplan and refined in conjunction with the UNICEF staff overseeing the evaluation and assuring quality of process and output milestones.

Evaluation Criteria and Questions

Impact: While the evaluation will be responding to each of the UNEG criteria below, it must also provide a comprehensive assessment of the impact of the ZO business model on the delivery of results by the UNICEF MCO towards the most vulnerable, girls, boys, and women of Mali.

Relevance: Assessing the extent to which the structure is suited to the priorities and policies of the country’s regional realities, children needs, target groups.

In evaluating the relevance of the field structure, it would be useful to consider: 

• To what extent is the current structure/set up of Zos still valid?
• Are the ZO assigned tasks, organised functions, and existing resources consistent with the overall goal of UNICEF Mali and the implementation of its mandate?
• Are the tasks, functions and existing resources consistent with the intended impacts, outcome and outputs results (CPD, sector strategies, global commitments)?
• What are, if any, specific comparative advantages for ZO delivery on parts of the UNICEF mandate and Work-plan?

Effectiveness: Assessing the extent to which the field delivery structure is the most effective for UNICEF Mali in order to attain its objectives.

In evaluating the effectiveness, it would be useful to consider the following questions:

• To what extent are the ZO’s contributing to the achievement of the programme results of the UNICEF MCO?
• What were the major factors influencing the contribution of the ZOs in the CP to the achievement or non-achievement the CP results?
• To what extent does the current setup for programme planning, implementation monitoring, and the division of roles and responsibilities enable UNICEF to achieve its programme results by ZO?
• To what extent is the level of delegated authority (including financial management) appropriate for ZOs to effectively fulfil their mandated role and responsibilities? Are the responsibilities sufficiently clear to all stakeholders?

Efficiency: Measures the outputs -- qualitative and quantitative -- in relation to the inputs. It is an economic term, which signifies that the field delivery structure uses the least costly resources possible in order to achieve the desired results. This generally requires comparing alternative approaches to achieving the same outputs, to see whether the most efficient process has been adopted.

When evaluating efficiency, it is useful to consider the following questions:

• To what extent  the current activities, processes, functions are the ZOs cost-efficient?
• Can objectives and activities be achieved on time?
• How does the current set up of ZOs provide efficiency in comparison to other alternatives?
• What would be the most cost-effective way to obtain the expected CPD results?
• How effectively have internal and external resources been harnessed to achieve results?

Sustainability: Measuring whether the field delivery structure is sustainable in the current context while taking into account financial, political, and programme trends (funds are expected to shrink while needs will be similar or increasing etc.). It also refers to the review of a framework of accountability between the different structures; and matching authority vs. responsibility.

When evaluating the sustainability, it will be useful to consider the following questions:

• To what extent, is the current structure sustainable considering current and projected developments in context, needs, and financial support?
• What were the major factors, which influenced the sustainability of the ZO structure?
• Which environmental threat analysis mechanisms were used to assess the safety and security of the staff, the vulnerability of operations and UNICEF’s physical assets to natural and man-made hazards in the regions covered? How do such mechanisms and mitigation measures weigh into the cost-effectiveness of the ZO business model?

Coverage: How well is the zonal office emergency response reaching the affected population in the current context of access constraints,  in terms of total coverage and vulnerable groups within affected populations?

• Do all groups have equal access to services provided?
• Are human rights, protection, gender issues and social inclusion adequately addressed?
• Are beneficiaries safeguarding, are staff and partners’ security adequately addressed in the set-up, in respect of do-no-harm principle?
• Are zonal offices able to identify most vulnerable groups within their area of operation?
• How do zonal offices collect, manage, analyze and share data and information?
• What linkages are being made to disaster (floods and epidemics) and conflict prevention activities and other cross-cutting issues such as gender and human rights?
•  

Coordination : How well are the emergency response and general programme delivery coordinated, with partners from government, civil society and communities, as well as within sectors and cross-sectorally?

• What other coordination mechanisms exist that have facilitated response?
• How well is communication between the zonal and central mechanisms integrated and conducive to achieve targeted results?

Methodology

The Evaluation Team will first undertake a review of UNICEF field operations structure in Mali through review of documentation, administrative data, organigrams, consultations with staff, partners and other key stakeholders in Mali and the UNICEF Regional Office. Drawing from best practices and comparison with models used elsewhere, the consultants will provide a series of specific and actionable recommendations, tailored to the needs and realities of MCO through the following additionally suggested methods:

Indicative data collection and analysis methodology:

1. Literature review: The exercise should be informed and initiated by a literature review of relevant reports and assessments:
   1. External factors including analysis of context – main trends affecting operations and capacities both at national and Regional/locality level as well as a review of best practices in other UNICEF country offices in similar settings, other agencies’ practices and systems in Mali and key partners structures (government in particular). Particular attention should be paid to identifying potential innovative and creative approaches;
   2. Internal aspects: UNICEF requirements, procedures and internal factors affecting the structure and set up for the country, including general vision and CDP, global policy and process issues, identified tasks etc. The consultant should also review key planning, monitoring and reporting documentation pertaining to UNICEF Mali operation. This should include CO documents in addition to relevant materials from Regional and HQs offices. The desk review should look at broad contextual issues and trends affecting the needs and challenges of UNICEF CO and its zonal offices.
   3. The literature review should set the chronology, essential events and associated actions and measures taken as recorded. The CO will locate and assemble primary materials in advance to expedite this aspect of the exercise. Analysis of recent directives in relations to effectiveness and efficiency in particularly the creation of a business center for operations (similar to GCC) and its implications for country offices.
2. Monitoring data (financial and programmatic) and process analyses: The results of the evaluation will be informed by monitoring data and process analyses, such as cost-benefit analysis, workflow and bottleneck analysis, strengthening the analytical basis for conclusions and recommendations.
3. Staff survey: The initial desk review will be complemented with a staff survey. The staff survey will look at perception of roles and responsibilities within the country office, including delegation of authority, areas requiring strengthening, work load, team management, work processes (select those most in need of review), perceptions and internal communication. A detailed analysis of the survey results will help evaluate areas that require attention as well as level of clarity in distribution of tasks and overall understanding of the structure by CO staff members. Results of the survey and analysis will inform discussions and interviews below.
4. Interviews: The exercise should be informed by interviews with all key staff involved in UNICEF field operation as well as selected outside key informants so as to inform the understanding essential to the drawing of initial findings. This will include Senior Management, Section Chiefs, Key Section Staff at the Central Office and zonal offices levels. In addition, key partners may be interviewed from within the UN, NGOs, donors, key central and Regional counterparts and ideally local communities. This highlights the importance of and value added of field visits to qualify the participatory aspect of the review. The CO is advised to draft an essential list of key partners and counterparts and support organization of discussions around sector and thematic aspects relevant to the review.
5. Stakeholder workshop(s) a stakeholder discussion(s) could be considered to share preliminary findings with key partners and to obtain feedback with view to rough validation. These could be organized at the Regional level and/or Bamako. They will be organized in an informal manner to prompt discussion and further strategic inputs to the exercise. CO support will be provided in locating a venue and related logistical and admin assistance.
6. Direct observations in the field of ZOs will help gather additional information from field visits. Data could be collected while activities are ongoing and also on infrastructures already constructed. Regarding the security situation on the field and access constraints, mainly in the north and the center of the country, data collection from direct observations should be relatively limited.

Expected Background and Experience

International and Mali’s are encouraged to apply for this evaluation exercise. Individual experts are also encouraged to apply, either directly for any of the below positions or as a team with other candidates:

Senior Team Leader: a senior expert at the P5 level at least with extensive experience in running/managing country operations with field offices/structures. Knowledge of and experience in operating in Mali would be an advantage. Strategic thinking, facilitation and communication skills, familiar with the UN system (with UNICEF an advantage). The team Leader will be in charge of designing and leading the evaluation process, oversight of the experts in the team, consultations with experts, partners and counterparts, facilitation of discussions and responsible for the inception and final reports.

The consultant to undertake the work will have the following profile:

• Advanced university degree in Political Sciences, Sociology, Law or related field
• At least 10-12 years’ experience in international development, including management experience of offices/programmes and/or organisational development.
• At least 3 years’ experience in Monitoring and Evaluation
• Experience in context affected by complex emergencies, including conflict/post conflict with security constraints.
• Leadership skills, strong communication and presentation skills
• Excellent analytical, conceptual and writing skills
• Excellent command of Franch both written and oral
• Knowledge of English an asset
• Knowledge of Mali an asset
• Familiar with the UN system, in particular UNICEF would be an asset

Evaluation expert: an international expert at the P4 level familiar with organizational/institutional evaluations, preferably in complex fragile contexts. Experience with and deep knowledge of UNICEF internal processes and systems a requisite. The evaluation expert will report to the team leader and be responsible, inter alia, for the methodological design and implementation of the process, learning and linking with UNICEF internal practices and requirements and assisting in the evaluation as required by the team leader. It is expected for the evaluation expert to play a key role in the inception report and have major contributions to the final report.

The consultants to undertake the work will have the following profile:

• Advanced university degree preferably in Public Administration or Business Management or/and advanced degree in Political Sciences, Sociology, Law or related field
• At least 5-7 years’ experience in conducting evaluations in the area of international development and/or emergencies, and related Planning, Monitoring and Evaluation frameworks
• At least 2 years’ experience in efficiency and effectiveness consulting and in conducting operational audits and information system/human resources reviews
• Experience in context affected by complex emergencies, including conflict/post conflict settings with security constraints
• Excellent analytical, conceptual and writing skills
• Excellent command of French, both written and oral
• Knowledge of English an asset
• Knowledge of Mali an asset
• Familiar with the UN system, in particular UNICEF, would be an asset

National consultant: a senior Malian with consultancy, preferably with international/UN experience and exposure, good knowledge of Mali, its actors and dynamics at the national and local level. The national consultant will be fundamental in providing contextualization and general advise on Mali, key stakeholders etc. He/she will also facilitate organization of interviews, support facilitation and translation when necessary, and play a key role in consultations overall. The consultant is expected to contribute to both inception and final reports under the supervision of the Team Leader.

The consultant to undertake the work will have the following profile:

• Advanced university degree in Political Sciences, Sociology, Law or related field
• At least 10-12 experience in development/humanitarian assistance
• Excellent analytical, conceptual and writing skills
• Excellent command of French, both written and oral
• Extensive knowledge of Mali
• Familiar with the UN system, in particular UNICEF would be an asset

Management and Technical Guidance

The exercise will be undertaken and led by the Mali CO based on the day-to-day supervision of the Chief, Field Operations, in coordination with the Deputy Representative and the Chief of Operations and technical oversight from the Chief, Planning, Monitoring and Evaluation. Additional technical support will be provided by RO and as need by HQ if/as required.

The evaluation will follow requirements of the UNICEF organization-wide system Global Evaluation Reports Oversight System (GEROS), whereby the quality assessment of final evaluation reports is managed by the Evaluation Office’s Systemic Strengthening unit. An external independent company will assess the final evaluation report and published in UNICEF’s repository of evaluation reports, Evaluation Database at https://www.unicef.org/evaldatabase/. Applicants are requested to gain knowledge of the GEROS Methodology online in more detail.

General Conditions

• The consultants will be responsible for drawing as much information out of UNICEF programme staff as possible and will work independently, though in consultation with the programme group.
• State if the SSA consultancy will include other benefits: No
• State if consultant has right to stay in UNICEF property: Yes
• Should consultant provide his/her materials: The consultant should be responsible for needed materials(Laptop computer)
• Is he/she authorized to have access to UNICEF transport: Yes
• Under the consultancy agreements, a month is defined as 21 working days, and fees are prorated accordingly. Consultants are not paid for weekends or public holidays.
• State if flight costs would be covered and at what standard: Should be included in cost of field work
• Consultants are not entitled to payment of overtime. All remuneration must be within the contract agreement.
• No contract may commence unless the contract is signed by both UNICEF and the consultant or Contractor.
• Consultant will be required to sign the Health statement for consultants/Individual contractor prior to taking up the assignment, and to document that they have appropriate health insurance.
• The Form 'Designation, change or revocation of beneficiary' must be completed by the consultant upon arrival, at the HR Section.