Appendix 6

NORAD	Norwegian Agency for Development Cooperation
NPA	Norwegian People's Aid (Norsk Folkehjelp)
NCA	Norwegian Church Aid (Kirkens Nødhjelp)
Røde Kors	Norwegian Red Cross
Redd Barna	Norwegian Save the Children
CARITAS	Catholic Development Organisation

This case is part of the overall synthesis study of NGO evaluations which seeks 'to provide an analysis and assessment of the impact, including efficiency and effectiveness of NGO development interventions, and secondly to provide an analysis and assessment of evaluation methods and approaches used' (Inception report, p.1).

In trying to achieve these objectives the study will undertake a synthesis of the major donor-initiated evaluation studies of Northern NGOs and a number of major sectoral, thematic and discrete evaluation studies received from donors, NGOs and NGO umbrella organisations in the North and the South.

In parallel with this synthesis process based on available documents, the study will also gather information about approaches and methods used in evaluations in donor-initiated studies and among Northern and Southern NGOs.

Norway is one of the ten Northern countries selected for the synthesis study. A major donor-initiated evaluation of Norwegian NGOs was finalised in 1995 and is summarised in the main report. This case study focuses on Norwegian NGOs' own evaluations. Several of the NGOs have gained considerable experience with this and have carried out an increasing number of evaluations to assess the relevance, effectiveness and impact of their development interventions.

This case will describe Norwegian NGOs' evaluation policies and practices, approaches and methods used, including perceived constraints and opportunities based on interviews with and review of documents from the organisations. The purpose is not to summarise the findings of individual organisations or assess the quality and impact of NGO studies, but to present and analyse trends and status in how Norwegian NGOs work with evaluations and possibly to increase the understanding of the extent to which evaluations are used to enhance organisational learning. Changes in perceptions and utilisation of evaluations among Norwegian NGOs will also be discussed.

A group of Norwegian NGOs were selected for this case study. It has not been possible to provide a complete overview which covers and is valid for all Norwegian NGOs. Time and resources were constrained and this is only one part of a major study. There is a conglomerate of NGOs in Norway with important differences in size, professionalism, available resources, number of staff and, consequently, evaluation policies and practices. But in spite of important differences and organisational idiosyncrasies, there are common trends and concerns which are shared among the NGOs.

The case is based on a purposive sample of Norwegian NGOs providing a mix of both the large 'mainstream' NGOs and the smaller, more specialised groups. It is not a representative sample of Norwegian NGOs, and organisations with relevant and acknowledged experience with evaluations were deliberately preferred. A total of seven NGOs were interviewed covering small and large so-called 'framework organisations'¹. None of the NGOs currently has special evaluation staff or units, so in most cases relevant Programme Officers or Heads of Departments were interviewed. To complement and validate findings from the interviews some NGOs were also consulted. In addition to interviews, documents were collected and reviewed and the NORAD NGOs office was consulted.² Time constraints have not allowed for sufficient discussion of the draft with the NGOs involved. Subsequent changes may, therefore, be required.

Interviews were structured around eight key questions addressing methodological evaluation issues as defined in the Inception Report (p.18f). All questions were not sufficiently covered in all interviews and several of the answers were incomplete, but in most cases an evaluation profile for the organisations emerged.³

The report is organised thematically. All questions will not be discussed for each NGO. After a brief introduction to the Norwegian NGO landscape, the study uses the questions as the point of departure and the description and analysis seek to extract and address important trends and critical evaluation issues defined by the NGOs during the interviews and in available documents. Most of the NGO profiles and specific examples are presented in text boxes.

In 1994 NORAD's NGO Department provided NOK 578.8 million in support to approximately 56 Norwegian organisations for projects and programmes in third world countries þ an amount which represented about ten per cent of total bilateral aid (not including emergency aid) ⁴.

The NGO aid community has emerged over the last twenty years⁵ and the current group is very diverse. The international development NGOs are made up of a variety of secular and church affiliated organisations, as well as domestic organisations that have an interest in development issues.⁶ The humanitarian and mission organisations are the most stable and in 1993 65.4 per cent of the total grants allocated by the Norwegian state went to the ten largest NGOs. ⁷

NGOs receiving support from NORAD fall into two broad categories: those receiving support on an annual project-by-project basis, and those with 'framework agreements'.⁸ In 1995 there was a total of fifteen framework organisations. Four stand out as significantly larger than the others: The Missionary Council, Norwegian Church Aid, Redd Barna (Norwegian Save the Children Fund) and Norwegian Peopleþs Aid. In the next group comes Care, Red Cross, and the organisations for the disabled (FSB).

In 1990, the total number of positions in Norway associated with NGO aid was about 240. About two- thirds of the organisations from which the Tvedt study collected information, maintain a paid, home based staff.⁹ Only the traditional solidarity organisations are run on a voluntary basis.

With the escalation of funding several of the large NGOs have increased and professionalised their staff. Norwegian Church Aid increased its headquarters from 21 in 1991 to 71 in 1990. Redd Barna increased from 35 in 1981 to 90 in 1991, of which 21 worked directly with aid activities.

An overview and analysis of NGOs' evaluation activities have so far not been carried out. The Tvedt study prepared a NGO bibliography also consisting of evaluation reports from Norwegian NGOs, but only in four selected countries and not for all NGOs.¹⁰

¹ Five of the framework organisations were interviewed (Missionary Council, CARITAS, Red Cross, Redd Barna, Norwegian Church Aid, Norwegian People's Aid and Strømme Memorial Foundation).

² Due to a heavy turnover in the NORAD NGO office the institutional memory is negatively affected, but former staff were also contacted.

³ The author has more than ten years' working experience with three Norwegian NGOs, and is currently a consultant to several NGOs and the NORADs NGO department - a background which contributed to and influenced the case.

⁴ If all contributions are included the NGOs take approx. 25 per cent of the total bilateral budget.

⁵ In 1963 the state for the first time channeled NOK 3 million to seven Norwegian organisations, but the NGO support 'exploded' first from 1980. The Tvedt report mentions three phases: an establishment phase (1963-1978), an expansion phase 1978-1992, and a consolidation phase 1992 onwards (Tvedt 1995, p.1).

⁶ Tvedt is using the following categories: a. humanitarian, b. political, c. missionary, d. special interest groups and e. occupational associations and trade unions (Tvedt 1995, p.5)

⁷ One of these, Missionary Council, is an umbrella organisation, representing 13 smaller missionary organisations.

⁸ A framework agreement is a five-year agreement with broad guidelines and indicative budget figures which allows the NGOs to plan its activities based on a reasonable level of financial assurance.

¹⁰ Barved, Einar (1995), Bibliografi over evalueringsstudier av norsk privat utviklingssamarbeide og generell litteratur om privete organisasjoner med vekt på einskilde utvalde land. Most of the reports are included in Appendix 1.