Project Evaluation Summaries
Prepared by the Policy, Planning and Evaluation Unit (PPEU)

---

Bhutan

---

I. Basic Project Data

Project Number:	BHU/92/C03
Project Title:	Permanent Works for Feeder Roads (Eastern Government Zone)
Government Executing Agency:	Ministry of Communications, Public Works Dept.
UN Cooperating Agency:	UNOPS
Sector:	Transport/ Communications
Sub-Sector:	Road Transport
UNCDF Budget:	US$     3,324,690
UNDP Budget:	135,000
WFP Budget:	160,000
Gov't. Budget:	291,000
Total Budget:	3,910,690
Actual Expenditures:
UNCDF:	3,628,350
UNDP:	135,502
WFP:	160,000
Government:	1,447,737
Total Expenditures:	5,371,589
Date Project Approved:	14 July 1993
Date Project Began:	1 October 1993
Date Project Completed:	31 October 1997
Date of Evaluation:	15 October 1997
Type of Evaluation:	Final

II. Background

Bhutan is a small landlocked country in the Eastern Himalayas with an area of approximately 46,000 square kilometers and an estimated population of about 600,000 people. The lack of a sufficient transport network is a major constraint to socio-economic development. The majority of the population still lives half a day’s walk from a road, while some communities are four to five walking days away. These people rely on mule tracks and foot trails and suspension bridges for access to markets and social services. The government has accorded considerable importance to the road infrastructure in its successive five-year Development Plans (FYP). At the start of the 7th FYP (1992-1997), the Government gave priority to the maintenance and rehabilitation of existing roads over the construction of new ones.

III. The Project

  UNCDF was approached by the Royal Government of Bhutan (RGoB) for assistance during a Planning and Project Identification Mission fielded to Bhutan in February 1992. The project was well suited to the area-based approach UNCDF had adopted at the time: it was designed to strengthen the rural transport infrastructure in support of agriculture (primary health care and education). The project was conceived jointly with another UNCDF project, “Construction of Primary Schools and Basic Health Units in Eastern Bhutan” (BHU/92/C01)

UNCDF at the time was also implementing two related roads projects, BHU/85/C01, Establishment of Maintenance and Repair Workshops for Road Construction, and BHU/85/C02, Rural Feeder Roads for Integrated Rural Service Centers. Considering the paltry progress shown by the latter project, it was decided to design the new project to consolidate the works done so far, and to complete the permanent works still to be done. BHU/85/C02 was closed in May 1996, and a final evaluation undertaken in September 1996.

The primary objectives of this project (BHU/92/CO3) were to: (i) enhance living conditions in rural areas through a better distribution and marketing of agricultural production, and better access to public social services such as health and education; (ii) help the government balance its regional development and decentralization policies; (iii) consolidate the network of feeder roads, (iv) preserve the investments already done (under the earlier project) on the roads, (v) lower maintenance costs and ensure the upkeep of the roads, and (vi) build the capacity of the Trashigang Road District to carry out maintenance in the future.

Main outputs were to be (i) permanent works on the five links (52.8 km), (ii) base courses on all the links, and bituminous sealing and resealing works on two of them, (iii) formation cutting on two of the links, and (iv) maintenance works as required during the construction period.

IV. Findings of the Evaluation

The mission concluded that the project did not reach its stated objectives. The main reason was insufficient strategic pre-project planning, an issue that had already been present in the earlier BHU/85/C02 project. While the project’s blueprint, the Joint Project Agreement, indicated that some of the shortfalls resulting from the limited time for preparation were to be remedied during implementation (in particular preparation of detailed working drawings, and determination of the resulting final quantities), they were not done. Consequently, the project proceeded without detailed working drawings and without a proper technical and financial sanction.

>Assessment of results achieved

(i) Construction

The evaluation compared the amount of work anticipated under the project (as revised in March 1996) with estimated achievements for 31 October, 1997, the project’s closing date. It was calculated that formation cutting would be exceeded by more than 8%. In addition, the evaluation team determined that the bailey bridge and all culvert repair and construction, as well as the box drain construction would be completed before the project termination date. However, only 30% of river training works, 40% of the slab culverts, 90% of the causeways, 60% of the breast and retaining walls, and 50% of the base courses would likely be completed by 31 October. In addition, none of the bituminous sealing and resealing, nor any of the slope protection works would be done. On average, the mission estimated that only about 70% of the works that were supposed to be carried out had been realized.

(ii) Equipment

Most of the equipment that was purchased for the project was the right type; however, there were a few exceptions, such as a flat bed trailer that could not be used because it did not suit the mountainous terrain conditions. In addition, there were two cement mixers and a cement pump that were inappropriate. The mixers (agitators) were too large for the work and terrain conditions; when fully loaded they weighed too much for the trucks on which they were mounted (and thus could not be driven to the project sites). The cement pump, a high output piece of equipment, was inappropriate for the type of work as well and was used only once. These cases are obvious situations of applying inappropriate technology. Instead of adapting to the local conditions, sophisticated modern, high yield, equipment was purchased that turned out to be less than fully utilized.

(iii) Participation

The local communities should have been fully involved in decision-making processes early during the preparation of this project. The communities, however, were not consulted. While the project does not seem to have had negative social effects (quite the contrary), a larger impact could have been achieved if community participation had been sought; for instance, on the positioning of cross drains. If this had been discussed with the affected people, not only would the sustainability of the roads have been improved, but also the compounding of the slippage problems might have been avoided. Water is a critical resource and the villagers will make it flow wherever they need it, regardless of whether it means blocking drains or cutting ditches across the road. Proper account should be taken of the needs of local farmers and communities in future projects. Similarly, the project contained no specific actions to improve the conditions of women. For instance, short side lanes for women to dry crops and produce (from rice to chilies) would have been helpful; or paving flat areas where the roads cross villages to help promote the development of better, cleaner, more attractive markets.

(iv) Sustainability and Maintenance

The long-term sustainability of the works carried out so far is a critical issue. While the quality of the work is average, the roads are incomplete and hardly maintainable. The main problems with the roads include a lack of retaining walls, an insufficient quantity of gabion protection walls, a complete absence of slope protection, insufficient base course, and the absence of weather-protection sealant. To keep them open and passable, in the rainy season particularly, will demand a considerable and expensive maintenance effort. The mission prepared and estimated costs for a simple maintenance program for the project roads, which, it is estimated, would amount to Nu 30,000 per km/per year. Unless the roads are regularly maintained, they will deteriorate quickly and their future rehabilitation will be much costlier.

(vi) Socio-economic Impact

The project has produced some beneficial outcomes. Thanks to the ongoing works, the roads have been kept open, particularly in the rainy season. In addition, people have been able to move more rapidly to and from the larger center of Trashigang. However, while there are some benefits, they fall short of the goals anticipated in the project document. Agricultural output has remained, by and large, the same as it was before the works started. This is because basic access to the affected areas had already been there for some time, built under the earlier project. This project was supposed to complete the work that was not done under the earlier project, making the roads accessible by motor vehicles year-round. This, however, was not achieved.

The mission measured how much traffic actually traveled the roads and found traffic volumes to be quite low, but consistent with traffic volumes on the main road network of the area. Thus, if one considers that the expected rates of return were marginal, it can be deduced that the rate of return of the project is quite low (especially since the costs of the earlier project and this one should be considered together for purposes of the economic evaluation). If inflation is taken into account (that costs will have increased by about 40% when all project works are completed), the rate of return is even lower.

The project’s main benefits were supposed to be non-quantifiable social gains. A total population of about 20,000 people lives in the road’s direct area of influence. Since the roads were opened under the earlier project (albeit accessible year-round), living conditions have improved. People report easier access to health and schooling facilities, more frequent visits by agricultural extension officials, and a closer and more frequent contact with political authorities. The field interviews conducted by the mission show that the opening of the roads has helped them to reduce their physical and social isolation. The villagers are now better able to voice their concerns through their Block representatives, the Gups, and make their views known to the District Dzongdag (Administrator). The current project has not, however, significantly helped to improve the situation beyond what it was at the end of the earlier project.

The most frequently mentioned source of improvement detected from the field interviews is access to better and cheaper building and construction materials. Indeed new houses are being built close to the roads as people have moved to them in the expectation of better accessibility.

Assessment of project design

(i) Pre-project planning, design

An in-depth review was ordered in late 1991 to find out why the works to be done under the earlier project could not be completed as proposed. The first and most important issue raised had to do with lack of strategic, forward planning. The review and later the evaluation of the project brought up the effects of the lack of surveys and working drawings, of not studying properly the alignment, and of not specifying fully the scope of works to be done. In other words, it warned of the ill effects of proceeding without proper final engineering, and its associated physical and financial sanction. Despite the earlier project’s experience, this project was carried out in much the same way.

PWD has a perfectly competent Planning and Design Division capable of preparing geometric working drawings, drawings for bridge sub- and superstructures, RCC slab culverts, and road pavement drawings, as well as drawings for B/R walls and causeways. However, as pointed out, no detailed working drawings were prepared.

(ii) Institutional Arrangements

While the Project Document provided a clear assignment of responsibility for the execution and control of the works to the NPD, the process of recruiting the RCM led to misunderstandings, and eventually to the acceptance by both the donors and the Government, of a situation in which there was dual responsibility. Such cases should be avoided; split responsibilities spell no responsibility. The RCM was first appointed in July 1994 but repatriated a month later – a new RCM was not appointed until July 1995. Unfortunately, the project suffered as a result of this situation. The project probably would have been more efficient, and higher achievement levels could have been reached had this situation not existed.

V. Recommendations

1. To UNCDF
   1. Project Monitoring: UNCDF should monitor projects closely, not only financially but also physically. This is critical to detect early warning signs of impending difficulties. Within the context of this project, progress reports should have been designed to provide a balanced picture between financial and physical advancement of projects.

   2. Equipment Disposal: The mission proposed the following three alternatives:

   (i) Transfer all equipment to the new UNCDF feeder roads project. If the equipment stays where it is now, it may not last long –there is a serious risk of pilferage and cannibalization for parts.

   (ii) Hand over the equipment to PWD. This should be done at the equipment’s residual value, which should be substantial because of its low utilization, and the proceeds would be credited as part of UNCDF’s contribution to the new project.

   (iii) Sell the equipment to private contractors. This alternative would boost the local contractor’s capabilities and encourage their increased participation in road works, a declared policy of the RGoB. The sale should fetch good market prices, and could be organized through UNOPS. The proceeds would be credited as part of UNCDF’s contribution to the next project.
   
   

2. To the RGoB, in particular the Ministry of Communication’s Public Works Division (PWD):

   Maintenance: Carry out a review and assessment of the PWD’s maintenance organization and needs. Maintenance of the main road network is generally poor in Bhutan, and for feeder roads practically non-existent. A simple maintenance program should be developed for the project roads to prevent losing much of the work that was done under the project.

VI. Policy Implications and Lessons Learned

The following summarized the key lessons learned from this project:

Project design

It is critical to the successful management and implementation of a feeder roads project to provide ample time during project preparation to meet adequate strategic, pre-project planning requirements. This applies particularly to preparing detailed engineering blueprints, and to enacting a physical and financial sanction [project incorporated into the national budget] before initiating the execution of a project. The physical and financial sanction is a legal sanction, obtained from the recipient country’s Ministry of Finance, that incorporates the project into the national budget in terms of financial funds and in terms of expected physical output. This establishes budgetary and physical limits of a project and is crucial to project accountability.

Community Participation

More local participation should be incorporated into the planning process for feeder roads in order for the roads to address all the community’s needs and concerns. Local farmers and communities (the direct beneficiaries) should be consulted and their immediate needs and concerns investigated. In addition, extensive studies should be carried out to determine how the community will be affected by the proposed project. In rural areas, this applies particularly to women, who make up a large proportion of the population because men usually leave to work in towns and cities.

Environmental Concerns

Environmental issues concerning roads in mountainous terrain must be taken into account early in the project preparation in order to avoid potentially negative environmental impact and damages (i.e. slides and unprotected slopes and earth cuts) that may be too costly to mend later. Mitigation and remedial measures must be estimated and included in project costs and budgets; leaving environmental protection unfunded is equivalent to not carrying out the necessary actions for mitigating any possible damage.

Project Management

Dual or shared responsibilities in roads project management can have a negative impact on effective project management and decision-making. The project agreement must therefore set clear actionable terms of reference for project staff, especially expatriate staff posted in critical positions in the project. In many cases it is preferable to hire short-term inspections by specialized technical professionals than posting permanent staff that may be submerged in day-to-day problems of projects.

Technology

Appropriate technology must be incorporated into the execution of works. Furthermore, techniques that are unnecessarily sophisticated and complicated must be avoided. For instance, sophisticated concreting equipment should not be used when simple masonry construction done by hand would suffice. Close and frequent supervision as well as frequent technical quality control tests during project execution are critical to minimize the potential use of substandard materials and poor quality work.

Use of Private Contractors vs Use of Force Account

Experience has demonstrated that significant gains in productivity and efficiency can be gained from using private contractors rather than carrying out works by force account (road construction managed by Government line ministries). However, project designers must address the fact that contractors have to be hired, managed and supervised and that these are complex and specialized matters on which expertise should be available. The capacity-building for such expertise, if not available, should be incorporated into future project design.

Training and capacity-building

Long-term sustainability of development assistance also depends on building proper training and human resource development into projects. Perhaps as important as the civil works themselves is the transfer of know-how and the training that is associated with internationally assisted projects, especially technical training for field engineers, and managerial and contact management training for supervising staff. A strong training component should be built into all future UNCDF projects.

Operations & Maintenance

When no specific funds are provided for maintenance during construction, there is a risk that shortcomings will either be made up for by detracting from the construction itself, or that no maintenance will be done at all. If no funds are provided after construction, the roads will deteriorate and the investment will quickly lose its value. Furthermore, it will be costly to repair.

VII. Evaluation Team

The evaluation team was comprised of Renato Schulz (economist, team leader), R.L.Kapoor (highway engineer), and Karma Jimba (environmental specialist).

Addendum

In response to the findings of the evaluation (that took place in October, 1997), UNCDF requested the Royal Government of Bhutan (RGoB) to field a Quantities Assessment Team comprising of two PWD engineers, engineering geologist and an environmentalist to assess the additional work quantities and funds required to complete the project. The Field Survey Report (FSR) produced by the assessment team in January 1998 is comprehensive and clearly indicates the extent of works which remains to be carried out with full costing analysis. The FSR focused generally on the major cost items proposed for the most difficult and significant problem areas and suggested a minimal amount of additions and/or cancellations of works.

UNCDF subsequently fielded a Supervision Mission by an independent consultant (Mr. Peter Benthall) in February-March, 1998 to review the outcome of the survey and construction drawings and to verify the quality and quantity of the works to be undertaken.

The report of the Supervision Mission report highlights the following key insights:

• The Supervision Mission is in agreement with the recommendations of the FSR, except for minor variations to the report proposals;

• The peculiarities and difficulties of road engineering in Bhutan are such that a balance needs to be struck between trying to predict the largely unpredictable landslides and slips, and awaiting events and reacting to them through ongoing maintenance;

• The standard of works completed was judged as very satisfactory with some parts being excellent;

• The measures proposed for each site situation were judged to be appropriate and adequate given the circumstances and the nature of the roads as low-traffic feeder roads; and

• A detailed schedule of works is required (not detailed designs) so that a realistic estimate of costs can be made before works commence.

A technical review mission was conducted in March, 1998 by UNCDF’s Principle Technical Advisor (for infrastructure), Mr. Antonio Cittatti. Mr. Cittatti subsequently confirmed the reports of both the Quantities Assessment Team and Mr. Peter Benthall.