Internal Evaluation of the UNCDF Participatory Eco-Development Programme

5. Methodological tools and approaches

5.1 General considerations

The main programme phases and their functions, as well as the methodological tools, are summarized in Table 2 below (UNCDF, 1997a).

Our analysis will deal first of all with general aspects (overall methodology), before looking more closely at certain tools which are apparently more complicated to use.

Table 2: Summary of the main phases, functions and methodological tools of eco-development projects

Phases and Functions

Methodological Tools

1. Start-up Phase

Expected duration:(18) 6-12 months

* establishing logistical arrangements, personnel and equipment

* Knowledge-acquisition on the local area

- Summary of existing documents

- Communication strategy (consultant

- Typology of agro-pastoral systems (researchers)

- Micro-regional study (consultants)

* Mobilizing partners/information and training

- Start-up seminar

- Workshops/training courses


2. Knowledge-acquisition on communities and their environment

Expected duration: variable in accordance with size of the terroirs; average 1-2 months


* Socio-tenurial interpretation of terroirs (repeated annually)

- Socio-tenurial mapping

- Socio-tenurial survey

- Setting up socio-tenurial database

* Analysis of production systems (repeated annually)

 

- Dynamic analysis of natural environment (village assembly discussion)

- Analysis of production systems (village assembly discussions)


3. Appraisal, strategy and programming phase

Expected duration: about 15 days


* Consultation within project team

* Participatory appraisal of the development potential and constraints in the terroirs

- Internal meeting

- MEGEVE, supported by the results of the socio-tenurial interpretation of terroirs and the micro-regional study (group and village assembly meetings)

* Definition of strategic choices

- Village assembly discussions, based on the results of the environmental studies and negotiations in accordance with the eco-swap concept

* Programming (to be updated every year)

 

- Typology of activities and possible funding methods

- Village development programme (VDP)

- Annual village development programme (AVDP)


4. Implementation and monitoring of village development programme

Expected duration: continuous depending on programme


* Negotiation and signature of agreements

- Written agreements

* Putting agreements into practice

* Monitoring of agreements

SEGESTE, comprising:

- Monitoring forms

- Database on agreements

- Thematic studies (if necessary)


5. Impact evaluation

Expected duration: 4 weeks for the consultancy; 3-4 weeks per quarter for implementation


* Setting up the impact evaluation system

- Use of specialist consultant/

* Implementing system (3-4 weeks every quarter)

- Systematic approach

- External evaluation of physical impacts and process

- Self-evaluation (by the villagers)

- Impact assessment database

* Feed-back and decision-making

- Meetings with villagers and communes



(Source: UNCDF, 1997a)

Examination of the various documents relating to methodological tools reveals a possibly excessive concern for exhaustive descriptions of how to use the tools.

Furthermore, examination of Table 2 and documents relating to the methodological tools clearly shows two substantial imbalances in relation to the vocation of eco-development projects:

* on the one hand, the bulk of the tools relate to activities at terroir level. There is practically no mention of tools for developing support to local governance and negotiating with the services and authorities which will remain in place. This backs up the previous observation that a substantial gap exists here; while

* on the other hand, great importance is given to acquiring knowledge about the environment and to appraisal, whereas there currently seems to be unanimous agreement that GT type projects erred in giving too much attention to this type of activity (Demante et al. 1995). The same need for simplification of procedures was expressed at the seminar in Senegal (UNCDF, 1997b, p. 1).

These considerations lead us to put three simple but nevertheless fundamental questions regarding methodological tools:

We give below a more specific analysis of the tools, guided by these questions.

5.2 Methodology of the pre-formulation and formulation phases

The first remark relates to the fact that Table 2, like the operational guidelines, contains no indications on the pre-formulation and formulation phases. Our evaluation is therefore based on the few references found in evaluation reports (Jenny et al. 1997), oral testimony collected during the mission and two documents designed to improve these approaches (Luche, 1996 & 1997a).

5.2.1 Pre-formulation/identification of projects

According to the sources quoted above, it seems that, until recently, the pre-formulation phase was conducted in a rather conventional manner: a team of experts collected information essentially through discussion with government officials, academics and the main donors.

With a view to broadening the range of information sources, it has recently been suggested that representatives of civil society (journalists, NGOs, interest groups, etc.) should be used as additional sources (Luche, 1997a). Moreover, it has been suggested that this process should be made more open, organizing a two day workshop for representatives of the bodies consulted, so as to improve information and communication on the political and operational aspects of the projects. This proposal deserves to be made part of the recommended procedure. In fact, it should reduce the risks of basing project strategy on incorrect assumptions – about the capacity of the DPDRE in Guinea, for example (Diakité et al., 1996, p. 30) – thereby making overall programming more realistic. One may wonder, however, whether two days are really necessary at this stage.

5.2.2 Project formulation – institutional arrangements
  1. Procedure
  2. With regard to the procedure for formulating projects, this consisted until recently mainly of an exchange of information on local conditions and the main themes of the development programme, by means of separate meetings between consultants and government authorities, some donors and communities. There was no feed-back/information workshop in the future project area, but provision has recently been made for this in the joint eco-development/LDF project in Senegal, as well as in the new LDF projects in Gambia and Burkina Faso. It would seem advisable to use this procedure systematically, especially from the point of view of more sustained effort in supporting local governance. In fact, this type of workshop could then form a first stage in setting up a forum for dialogue worthy of the name.

    Questions could be raised, however, about the usefulness of the start-up seminar as planned at the end of the start-up phase, if a broadly based information workshop were incorporated as a feature of the project formulation procedure. Considering that there would be few concrete achievements between the two events, it should be possible to dispense with the start-up seminar. It is important to avoid succumbing to the "2/3 day workshop syndrome" as a basic tool in communication.

    Finally, some of the practical methods suggested by Luche (1996) should be called into question:

    The number of participants

    The first workshops of this type were attended by some 120—130 people (Jenna Luche, pers. comm.). Experience with workshops of this nature does, however, show that the number should not exceed 50-70 participants; this enables the richness of debate to be retained, while making logistics and the task of mediating the discussion easier and less costly.

    Representation of the various interested parties

    According to Luche (1996, p. 7), there should be 30% representation for government structures and NGOs and 70% for village communities. Although one must ensure that the local people are properly represented, there is a risk, if they have a majority in debate, of compounding the current failing of concentrating much more on the terroir level, to the detriment of the local governance dimension.

  3. Products – project documents – institutional arrangements
  4. The formulation of most project documents, especially those relating to the first PED projects, follows a fairly "classic" pattern:

In view of this, innovative capacity is weak and projects too often function "in isolation", with no guarantee of continuity. By way of comparison, Box 2 describes the institutional arrangements of two Natural Resource Management (NRM) /GT projects, located close to the PED project in Seno, Mali, which are more conducive to a programme approach and where activities are more likely to be put on a permanent footing.

Box No. 2: Two examples of institutional arrangements conducive to the programme approach in Mali

In Mali, the eco-development project shares the same geographical area as several other World Bank, IDA and GTZ natural resource management/village development projects.

* The GTZ project (grant) is implemented at local level by the Local Development Committee (LDC), which brings together all the technical services and is chaired by the Circle Commander. This committee is assisted by a single resource person, the technical support officer, recruited by the project. Relying exclusively on MARP, the team works out a technical and financial implementation plan for the villages selected within the Circle and passes it on to the project's national office. This office is responsible for identifying shortcomings in field operations and engaging in policy dialogue with the LDCs and the line ministries.

* Conversely, the IDA and World Bank projects (loans) are led at local level by a co-ordinating team restricted to a facilitator, a specialist in natural resource management, a financial administrator and extension staff working on a contractual basis in the target villages. The level of investment per village amounts to FCFA 15 million. Here again, planning and authorization of expenditure is a matter for the LDC, under the supervision of the co-ordinating team.

These two forms of funding are managed by the same project office at national level, co-operating with the various ministries and public services, thus fostering better co-ordination between donors as well as policy dialogue at national level. By way of example, GTZ actively seeks to develop the best possible relationships between the LDC and management committees at village level, based on transparency in management and control of activities. In particular, resources made available to the public services must be monitored at village level, where records are kept of technical operations carried out by the services. This monitoring is carried out both by the co-ordinating structure and the village committees.

The effective participation of the public services is not just requested, but checked and criticized at several levels. Clearly, the success of the project depends on genuine transparency in management and motivation of the local authorities and services.

The NRM/GT projects described above have some interesting characteristics in terms of formulation, which differentiate them from PED projects:

Despite the difficulties mentioned above, the Mission recommends that PED projects should draw inspiration from the above examples of the programme approach. It would thereby be possible to avoid formulations which have shown a low level of sustainability and replicability and to take part in the ongoing experimentation of new types of institutional arrangements. There are several possible options and, as often happens in the development field, one should not be afraid to proceed by trial and error, while learning from agencies which have more experience in this regard (especially IDA, GTZ and the Dutch Co-operation Service).

5.3 Knowledge-acquisition tools

5.3.1 General remarks

The methodological tools whose systematic, sequential use is suggested in the operational guidelines include:

The eco-development methodology aims thereby to combine a scientific approach using the Geographical Information System (GIS) and socio-economic database with a participatory approach based on consultation and dialogue with the communities and their involvement at all stages.

Each stage is backed up with a series of external consultancies employing skills in GIS, developing the database and defining monitoring criteria. The results of each stage are also fed back to the village communities.

The important place given to knowledge acquisition has been pointed out elsewhere in this report. This aspect and especially the respective tools are currently the subject of lively debate amongst the staff of the eco-development programme at all levels. These instruments need to be analyzed from the qualitative and quantitative points of view:

The need to simplify the overall procedure for knowledge-acquisition is clear from many documents (Diakité et al., 1996, p. 54; UNCDF, 1997b, p. 1), as well as field visits. By way of example, the project team in Seno, Mali, suggests that the typology of agro-pastoral systems should only be used in reference villages and that a less cumbersome toolkit be used elsewhere.

With these concerns, as well as the three key questions mentioned in Section 5.1, in mind, recommendations concerning knowledge-acquisition tools are as follows:

* Not seeking to "re-invent the wheel". In other words, project and development agencies involved in similar activities should be consulted to find out what instruments they use. One current example involves local development projects (or second generation GT projects);

* Combining external and local know-how, whether the latter be national or within the project. This would decentralize the development of such tools, bringing in the field dimension, whereas hitherto these tasks have mainly been handled by external experts;

* Finally, tools should be perfected gradually and follow an iterative process of applied research.

5.3.2 Specific considerations relating to certain tools

(i) Typology of agro-pastoral systems

With more specific reference to this tool, in Niger, the only instance where it has been developed, the purpose was mainly academic and the results were not fed into the project (Jenny et al., 1997, p. 16). The intellectual advantage of the study is not reflected within the programming which is essentially based on a mental picture of village land management (GT) projects. The range of activities is not structurally difference from those conducted as part of the latter type of projects. It does seem here that development, in this case the UNCDF project, has been placed at the service of research, whereas the converse should, in principle, be expected!

(ii) Socio-tenurial mapping The maps certainly have the advantage of combining physical (land-use) and social (lineage groups) data. However, to a greater extent than other instruments, this type of mapping tool tends to "freeze" situations which are by nature dynamic and difficult to mark out in space, such as pastoralism, changing land-use strategies and so on.

Moreover, one should not fall into the error of giving more importance to spatial aspects than the resource aspect of local production strategies. The documents are rarely used to raise crucial questions in relation to land tenure, especially in view of the need to analyze the latter from a social and political point of view (its reflection of the existing power structure, for instance).

In fact, it is hard to see the link between the data produced using this tool and the strategies mentioned in the VDPs and AVDPs.

One is led, therefore, to query the relevance of these tools as they stand at present. They should perhaps be used only as and when required, for instance when substantial development of village lands is feasible and planned.

(iii) GIS (Geographical Information System) The following points most commonly arise in this regard:

* Incomplete training of national experts on the development and possible fields of application of the geographical information system (GIS). A GIS is inconceivable unless it allows cross-referencing of spatial information with socio-economic information in order to general new information which can be used to support decision-making. This is what makes it an information system. There is a degree of incompatibility between socio-tenurial mapping and the database, which was developed separately without taking into account the cartographic units identified by the land tenure study. Putting the GIS to full operational use depends on staff having a good grasp of the software and information management. It should be mentioned in passing that the software used by the project is unknown to other local GIS users so that on-going training by local human resources is not possible. In Niger, the project team acknowledges these shortcomings, noting that procedure between the MEGEVE and MADRAS stages was not made very explicit, meaning that the approach did not hang together very well.

* It is rare for even 40% of the GIS potential to be used, in that once the maps have been produced, they are no longer used: