1998 Project Evaluation Results
Prepared by the Policy, Planning and Evaluation Unit (PPEU)
Perspectives
Thematic Analysis: Capacity Building
One way UNCDF provides capacity building is via technical assistance [TA] and training. Overall the TA evaluated in 1998 was of good quality. Two main, recurring problems affected it: Delays in recruiting technical advisors in the field and significant amount of turn-over. Slow recruitment in Mauritania, Niger, Sao Tome & Principe, Viet Nam, Zambia, and Laos caused project implementation delays. Reasons for tardy recruitment varied. In most cases, the UNDP/UNCDF/UNOPS recruitment procedures were lengthy. Occasionally the candidate profiles were difficult to match with the resources available; MAU/92/C05 was delayed a year because no suitable technical advisor could be found.
The turn-over of technical staff posted to the field affected many projects evaluated in 1998. Some candidates left when their profiles no longer met project needs after re-design; others went because roles and responsibilities were unclear and frictions arose within the project support units or elsewhere.
| The Laos Nam Tan Irrigation Project (LAO/89/C02) was originally formulated to have UNOPS execute the project. After start-up the project switched to national execution without discussing and agreeing on the roles, responsibilities, and reporting of the TA and the Government counterparts, which would have permitted a smoother transition. However, the local management of the irrigation scheme, although overly complex and overstaffed, was able to distribute water equitably, rehabilitate the irrigation scheme, and promote a successful participatory methodology for group formation of farmers. The latter was accomplished largely with the assistance of international TA. |
Turn-over had both negative and positive implications for institutional building and project performance. Certainly some continuity was lost and project efforts were redirected when staff changed. But from an institution-building perspective, in most cases where projects were redesigned or components were added, a change of staff benefited the project. A clearer understanding of project strategy and the roles and responsibilities of project partners resulted. This served to reduce technical and managerial obstacles to implementation, encourage adoption of more participatory approaches, and elevate the priority of capacity-building in project staff activities.
In general TA targeted infrastructure development rather than institution-building. There were noteworthy exceptions to this trend, where new TA inspired a more participatory approach to project implementation with favorable implications for institution-building. Where this occurred there was generally greater ownership of the project by the local population, notably women.
Evaluations provided more information on training than on technical assistance, but data remained limited. On the whole, evaluations described a spotty record on training. While effective, there were problems with the impact, content, and delivery of training.
But training had a limited impact on staff ability to manage larger, technically more complex projects. This synthesis concludes that project designs should factor in the need to reinforce the capacities of engineers working at more skill-intensive levels. The VIE/95/C01 (RIDEF) evaluation suggested establishing linkages between projects and higher level training institutes; another advocated contracting out training to other, more locally experienced, partners (e.g., NGOs). Several projects opted for contracting rules that encourage awards not to "the least costly" entrepreneur, but rather to "the least costly while best qualified."
Training was not adequate in all instances. While training generally met project requirements as perceived by designers, focusing on contractual and accounting procedures and on quality control of infrastructure, a technical approach to project design encouraged a technical "bias" in the training. Such training met the requirements of neither the population nor the technical staff. At times, after infrastructure had been developed, the population did not know how to exploit it for maximum benefit. The NER/87/C02 and STP/92/C02 project evaluations showed that training benefited the population most when it imparted practical knowledge aimed at improving existing practices such as farming. Hence projects attached insufficient weight to understanding the cognitive aspects of the cultural contexts in which projects were implemented.
Training must be tailored and made more flexible and modular to answer better the needs in the communities and government circles where it is administered. The NIC/92/C03 evaluation observed: "training needs and the definition of subject areas must be identified by the persons themselves who are to be trained."
Because training needs were not always properly assessed, important topics were, generally, found wanting. These topics included: Organizational skills and organizational management, programming, oversight, project management, leadership, community mobilization, participatory planning methods, gender-awareness, networking, and coordination. In addition, IRD evaluators pointed out the continuing need to reinforce and extend training in infrastructure maintenance, especially with regard to wells and boreholes. Evaluations also showed the need to train trainers in maintenance and, more importantly, to train women in maintenance techniques.
Although the delivery of training was generally appropriate, it could be improved. There were a few instances of "in-class" training only, and where this method was used impact was lower. There were two noteworthy training innovations: First, there was a growing shift away from classrooms to more experiential forms of "learning by doing." This format was exemplified in VIE/95/C01 (RIDEF) where observation of practices in other districts was an integral part of the training process. As evaluators observed, "The [RIDEF] project has set in motion an effective training process for planners, project implementers and accountants. This includes a cascading system where district staff train each other." Similar examples appeared in LAO/89/C02 and NER/87/C02 both of which used agricultural demonstration plots and other models of production. These were more effective than lectures because they offered observable and convincing examples of how new work methods can improve productivity in a variety of settings and they complemented the learning process. Second, whereas training continued to be administered through project staff and/or technical assistants, more projects used a "training-of-trainers" approach. In Viet Nam, Niger, Yemen, Sao Tome and Principe, as well as Laos there was a growing trend towards "peer-to-peer" training. For example, the best performing farmers in Laos acted as extension messengers for the training. Then, too, women played a bigger role in training other women. The woman-to-woman approach was consistently cited as the most effective way of transferring skills and knowledge to that segment of the population.
However, the evaluations stressed that these new forms of training required that beneficiaries take special care to identify their training needs. They also emphasized that the trainers be carefully chosen and prepared. Finally, they underscored the importance of shaping the training materials according to the trainers' own experience.
One evaluation warned of over-specifying the training program. The STP/92/C02 project team tried this at first and failed: Farmers did not show up for sessions since neither the training content nor its schedule appealed to them. Responding to this setback the project team reacted by consulting with the farmers on how best to accommodate their farm production schedule. They also decided the content of the training sessions in a participatory manner and administered it in groups of like-minded people. This dramatically increased the interest and number of participants.
Several other observations on training are worthy of mention: First is the case for having high intensity, international technical assistance to improve training at the project start and at timely moments throughout the project's lifecycle. This could facilitate a training needs assessment based on beneficiary input; select and train trainers; prepare training support materials, and ensure quality control of training activities. Viet Nam highlighted the need to invest heavily in capacity and institution-building in order to ensure that infrastructure development was properly carried out and that the local population knew how to access funds and plan and use its investments.
Second, it was clear from STP/92/C02 and NER/87/C02 that sub-contracting private firms to deliver training had advantages. It reduced the size and administrative costs of the project support unit by relieving it of its training burden. On the other hand, project evaluations showed that the use of private companies to deliver training limited development of indigenous capacities.
Third, the VIE/95/C01 (RIDEF) evaluation showed that no matter how much training was done, or how good it was, training could not substitute for effective supervision and oversight by qualified professionals. Projects need either to provide additional training in supervision and oversight to existing staff or to extend the network of specialists capable of providing such oversight.
Fourth, when project documents specified training activities, financial provisions were often insufficient to carry them out fully. This is changing in more recently designed projects, where capacity-building is a strong component. But older project designs lacked strategic perspective on the use of training. Absent organizational and institutional performance goals, sufficient funds were not allocated to reinforce training capacity and training was conducted without a clear strategy. While training of project support teams was quite successful in honing staff skills in participatory planning, training had a much more limited effect on community-based organizations. The evaluations show that where training funds are insufficient, projects may not survive for lack of institution-building, especially at the community level.
Fifth, popular education — mass training — affected institution-building by discussing good governance and public use of public services, and by encouraging appropriate attitudes and behaviors in support of good local government. Nevertheless most of the UNCDF projects evaluated were weak on mass training: As of late 1998 there was no comprehensive strategy to communicate with stakeholders during the design, implementation, monitoring, and evaluation of the projects. Thus people were not properly informed of ways to influence decision-making or how to access capital investment funds. As a result behavior supportive of good governance was not adopted in a widespread fashion and institution-building did not meet its full potential.
Lessons learned include:
• Training should be tailored, more flexible, and modular to address needs in the communities and government circles where it is administered.
• Projects should anticipate the need to train trainers in maintenance and, more importantly, to train women in maintenance techniques.
• The woman-to-woman approach is the most effective way of transferring skills and knowledge to that segment of the population.
• Training can not substitute for effective supervision and oversight by qualified professionals.
• Where training funds are insufficient, projects may not survive for lack of institution-building, especially at the community level.
• Project "exit strategies" need to consider the consolidation and sustainability of training, particularly in the field of rural development and participatory planning.
• More and better linkages must be established with training institutions and with Government departments above the local government units to ensure permanent capacity-building.