DVV International
Terms of Reference
Evaluation of the work in the country projects Malawi and Mozambique
| Project title: | Promotion of integrated adult education systems in Africa (in general) |
| Countries: | Malawi and Mozambique (sub – projects of the overall continental project) |
| Project number: | PN 2022.2607.4/DVV0005 |
| Funding agency: | BMZ |
1. Introduction
DVV International is the Institute for International Cooperation of the Deutscher Volkshochschul-Verband e.V. (DVV), the German Adult Education Association. DVV represents the interests of approximately 920 adult education centres (Volkshochschulen) and their state associations, the largest service provider of continuing education in Germany. As the leading professional organisation in the field of adult education and development cooperation, DVV International has committed itself to supporting lifelong learning for 50 years now. DVV International provides worldwide support for the establishment and development of sustainable structures for a lifelong Adult Learning and Education (ALE).
Funded in a large part by the BMZ for the creation/strengthening of social structures (Sozialstruktur-förderung – SSF), the Institute cooperates with more than 200 partners in more than 30 countries in Africa, Asia, Latin America and Europe. DVV International promotes, together with national, regional and global associations of adult education (AE), lobbying and advocacy activities for the human right to education and lifelong learning. In doing so, DVV International contributes to the United Nations Sustainable Development Goals (SDGs), the Global Agenda for Education (Education 2030) and the UNESCO International Conferences on Adult Education (CONFINTEA).
DVV International supports the exchange of information and interaction between professionals at European and global level through conferences, seminars and publications. DVV International’s interventions cover three levels: first level – contribution to policies, strategies, standards and norms, as well as funding on national level (macro level), second level – institutional capacity building and capacity building of teaching staff (meso level), and, third level – the basic model interventions for the population with potential for replication at national, regional, and international level (micro level).
2. Background of the project
After a certain history of cooperation between DVV International and its partner countries including Mozambique (cooperation since 2008) and Malawi (where the cooperation took off only in 2017), the donor (BMZ) and DVV International agreed on the formulation of a new project with new timelines.
The projects to be evaluated are part of a continental project which comprises 11 country projects and a continental project (a digital platform for information, exchange and (blended) learning). They are embedded in a timeframe of three 3- year -long phases, with the first-one from 2020 to 2022.
3. Objectives of the programme
In general, the general objectives of the work of DVV International in the context of creation, respectively strengthening of social structures are as follows:
- The technical and institutional capacities of the cooperation partners for carrying out practical activities and for the management of a system are reinforced (macro level);
- The inclusion of adult education in policies, poverty reduction strategies and local and national budgets has increased (meso level).
- Adult education offers are improved and multiplied (micro level);
Yet, more specifically, the projects to be evaluated work under one general overall objective on continental level:
Overall objective 9 years: An adequately resourced, professionally managed adult education system across different sectors provides good quality services to youth and adults in a demand-driven manner, thus contributing to the SDGs.
Moreover, each country project has an even more specific objective for each of the 3 – year phases:
Overall project objective phase 2 (2023 – 2025) for Malawi: The launch of the National Adult Education Strategy strengthens the coordination and implementation of integrated and cross-sectoral adult education in Malawi and contributes to the achievement of the SDGs
Overall project objective phase 2 (2023 – 2025) for Mozambique: Adult education is strengthened through improved funding, management, and coordination, as well as the expansion of innovative approaches to contribute to the achievement of the SDGs.
Project concept Southern Africa region
The continental project proposal was elaborated by DVV International in a participatory way with its regional and country offices and approved by BMZ. It had an introductory chapter for each region to contextualize approach and corresponding log-frames. Here the relevant part for the evaluation:
The continuation proposal comprises three sub-projects in Southern Africa within the framework of a differentiated planning arc per sub-project with a total duration of nine years until 2028. The projects are managed by country offices in Mozambique and South Africa as well as a regional office in Malawi, which steers the work in all three countries.
In the first phase (2020-2022), in line with the System Building Approach (ALESBA) developed by DVV International, the Adult Learning System (ALE) in all three countries was analysed in a participatory manner and system building was supported on a country-specific basis. Due to the Covid-19 pandemic, this could not be done in the required depth everywhere and will require follow-up work. However, the main parameters of the results to be achieved in phase 2 in terms of the medium-term overall objective were identified.
The common denominator in all three countries is the work on the following key issues:
- Lobbying for the recognition of a multi-sectoral approach to ALE that includes literacy, including the development of a strategy to mobilize resources for ALE at the macro level, as ALE is critically underfunded in all countries;
- Capacity development for strategic individuals and institutions in ALE at the meso level, with a key technical area being the respective management information systems for monitoring ALE;
- Providing evidence-based examples of micro-level ALE offerings and approaches that respond to the respective strategic national ALE frameworks, particularly for vulnerable and marginalized groups and communities with demonstrated development ambitions and initiative.
National approaches will be discussed and further developed in terms of cross-cutting themes and methodologies both in regional exchanges and in collaboration with the continental sub-project, including through the use of the MOJA digital platform.
In addition, the country-specific focus will be on the following aspects:
- In Malawi, the focus will be on supporting the ALE policy developed in the last phase. The project will support the national ALE thematic group. Another key element is the implementation of the first university course in Adult Education and Development at the Catholic University, which will start before the end of 2022.
- In Mozambique, there is a focus on the integrated program for ALE – developed with DVV International and recognized in the Strategic Education Plan 2020-2029. The work focuses on its application in more and more regions. This involves other ministries at all levels in addition to the Ministry of Education.
4. Activities of the programme
With the objective to sustainably develop social structures for adult education, DVV International’s programs aim to have an impact on three levels (micro, meso and macro) in accordance with the above-mentioned objectives. The indicators to measure the achievement of the objectives are part of the log-frame (attached to these ToR). The project proposal is also accompanied by a catalogue of activities. The choice of action areas depends of prior project activities and the continuous dialogue between all stakeholders of the respective country projects.
5. Evaluation goal and objectives
DVV International understands itself as a learning organization. In order to improve further project activities, DVV International, as the implementing agency, would like to learn more about achievements, challenges and best practices of the above-mentioned project.
The evaluation’s main goal is to document project achievements towards the objectives set in the initial project proposal. Furthermore, the evaluation should review and assess the relevance of the project activities, the achievement of the expected results, and their perspectives for sustainability. It should include a reflection about lessons learnt and make recommendations for the project counterparts regarding methodologies, strategies and approaches for possible further activities. The evaluation is aimed at revealing problems and barriers in the project implementation, as well as outlining how they were overcome. The objectives and impacts will be evaluated on the basis of the information laid out in the Logical Framework Matrix.
The evaluation and its recommendations shall be a useful tool of information for the project implementing agency – DVV International and its partners[1].
6. Main Evaluation questions
As a learning evaluation, the work should take into account
- the overall project logic of the 9 – year project for the continent as proposed to BMZ and approved;
- the observations and recommendations of the prior evaluations 2021;
- The system diagnosis according to the ALESBA approach done in 2021 and 2024;
- the impressions gathered during the investigations in 2024;
- additional questions and suggested ideas for the further work (see table below in green).
In this context, Criteria for Evaluation Development Assistance of the DAC of the OECD serve as a guide. The evaluation should give clear statements regarding the following categories:
6.1 Relevance
The extent to which the aid activity is suited to the priorities and policies of the target group, recipient and donor. In evaluating the relevance of a programme or a project, it is useful to consider the following questions (compare also chapter 6.6.):
- To what extent are the objectives of the programme still valid?
- Are the activities and outputs of the programme consistent with the overall goal and the attainment of its objectives?
- Are the activities and outputs of the programme consistent with the intended impacts and effects?
6.2 Effectiveness
A measure of the extent to which an aid activity attains its objectives. In evaluating the effectiveness of a programme or a project, it is useful to consider the following questions:
- To what extent were the objectives achieved / are likely to be achieved?
- What were the major factors influencing the achievement or non-achievement of the objectives?
6.3 Efficiency
Efficiency measures the outputs — qualitative and quantitative — in relation to the inputs. It is an economic term which signifies that the aid uses the least costly resources possible in order to achieve the desired results. This generally requires comparing alternative approaches to achieving the same outputs, to see whether the most efficient process has been adopted. When evaluating the efficiency of a programme or a project, it is useful to consider the following questions:
- Were activities cost-efficient?
- Were objectives achieved on time?
- Was the programme or project implemented in the most efficient way compared to alternatives?
6.4 Impact
The positive and negative changes produced by a development intervention, directly or indirectly, intended or unintended. This involves the main impacts and effects resulting from the activity on the local social, economic, environmental and other development indicators. The examination should be concerned with both intended and unintended results and must also include the positive and negative impact of external factors, such as changes in terms of trade and financial conditions. When evaluating the impact of a programme or a project, it is useful to consider the following questions:
- What has happened as a result of the programme or project?
- What real difference has the activity made to the beneficiaries?
- How many people have been affected?
6.5 Sustainability
Sustainability is concerned with measuring whether the benefits of an activity are likely to continue after donor funding has been withdrawn. Projects need to be environmentally as well as financially sustainable. When evaluating the sustainability of a programme or a project, it is useful to consider the following questions (compare also chapter 6.6.):
- To what extent did the benefits of a programme or project continue after donor funding ceased?
- What were the major factors which influenced the achievement or non-achievement of sustainability of the programme or project?
6.6 Additional evaluation questions
- With regard to chapter 6.1. on „Relevance“ and specific reference to the macro level: To which extent do the governmental partners see the relevance of ALE as a pillar of their education and training systems in the light of the existing challenges in their formal education and training systems?
- With regard to chapter 6.6. on „Sustainability“ and specific reference to the macro level: As a pre-requisite for sustainability: Which level of ownership is measurable in the responsible governmental partner-structures in terms of
- Associating enough qualified staff to the respective units?
- Mobilizing resources within the governmental channels, and
- Supporting the process of resource mobilization from funding partners, technical partners (locally, internationally, govern-mental and non-governmental?
- With regard to specific technical aspects in the country projects in Malawi respectively Mozambique:
- Can further organizational and institutional development of NACLAE (Malawi) be a road towards sus-tainability and a tool for cross-sectoral pooling of resources towards an integrated system for ALE?
- Comment on the quality and sustainability of the management structures of Community Literacy Fora (CLF) in Mozambique and Community Learning Centres (CLCs) in Malawi
- To which extent can the users of more qualified ALE personnel (government agencies and NGOs) fund or co-fund the upgrading of their knowledge and skills? (exemplary case: CUNIMA / Malawi)
7. Evaluation approach and methods
The evaluation will consist of both desk and field study. A combination of primary data collection and secondary data review is expected during the evaluation.
- Project proposal
- Interim project reports
- Mid-term and other kinds of (self)evaluations referring to the project
- Partner documents of project implementation (as far as available)
It should be checked if standardized quantitative approaches might be applicable.
8. Indicative Timetable and work input
The maximum number of working days foreseen is 29 days (3 days for preparation including inception report, 3 days for the workshop to present and discuss findings and 3 days for reporting plus 20 days for field work (10 for Malawi and 10 for Mozambique including travel days). The field phase could be covered by two persons, e.g. one for Malawi and one for Mozambique, in this case, up to 5 more days can be granted for the double – coverage of the workshop to discuss the findings and to coordinate planning and reporting.
The contractor to make alternative suggestions as long as the overall time – budget line is not exceeded.
[1] The work of the project during its first phase in the 3-phase cycle (meaning between 2020 and 2022) had been evaluated by Naturalive Consulting for Malawi and by RL CONSULTORES for Mozambique.
Get detailed TORs and templates below
ToR + plan for a joint evaluation work concerning Malawi and Mozambique
Template for financial offer for evaluation work DVV International