Telephone counselling service evaluation : an evaluation of the Dunedin Emergency and Citizens' Advice Service Inc. development of a reliable records system, extraction of service-operation statistics, and community survey appraisal of the service.

Abstract

Evaluation of the Dunedin Emergency and Citizens’ Advice Service (Inc.) was accomplished in a research programme consisting of four major phases of study. The research was conducted over a three-year period, from late 1977.

1. Modifications of the call-record system employed by the Service involved the compilation of a manual to assist in the classification of problems presented in calls to the Service. The manual, the Otago Index of Community Problems, enables the categorisation of problems according to three levels of specificity: problem area; problem type; and specific problem definition. Substantial changes were also made in the call-record form.

2. A revised records sytem was tested in operation from January, 1978, with analyses of records data performed on a monthly basis and results of these were disseminated to a variety of social service agencies in Dunedin and elsewhere.

Reliability amongst users of the system was found to be considerably enhanced by the modifications introduced, this being assessed in an inter-rater agreement exercise which yielded kappa values of .8 for problem-area agreements and .77 for more specific classifications.

3. Data from records of calls to the Service over the 24-month period (from April 1, 1978, to March 31, 1980), were subjected to detailed analyses to investigate trends in, and relationships amongst: demographic caller-variables; caller-chronicity; temporal variables; call "nature" variables; categorization of calls; and action or referral options used.

Results are also reported from analyses conducted after selection of cases on the basis of chronicity and other confounding variables which otherwise limit the extrapolation of trends to community problems.

4. Information on public levels of awareness of the Service, perceptions of Service functions, and community preferences regarding social services, was obtained from the community survey component of this evaluation. The results of this survey suggest that members of the Dunedin public tend to misunderstand the functions of the Service and that awareness of the Service is lowest in the less-privileged groups.

The investigation provides supportive evidence for efficiency and effectiveness criteria as they relate to the Service studied. Implications of the findings are detailed in the Discussion section and 11 major conclusions and recommendations are presented.