Related Case Studies

 Overview 

These materials support a discussion of four case studies that are drawn from the experiences of open and distance learning providers in implementing quality assurance schemes in their institutions.
The practice exercise consists of a list of questions to be used as guidelines for participants as they work in small groups to develop a presentation of the case study their group has been assigned.

Source materials for this topic

Guri-Rosenblit, S. Quality assurance procedures at the Open University of Israel. In A. Tait, (ed.), Quality assurance in higher education: selected case studies, pp. 29-41. Vancouver: The Commonwealth of Learning, 1997.

Koul, B. Quality assurance practices and principles: the case of Indian distance education. In A. Tait (ed.), Quality assurance in higher education: selected case studies, pp. 42-56. Vancouver: The Commonwealth of Learning, 1997.

O'Shea, T. and A. Downes. The roots of quality assurance at the British Open University. In A. Tait (ed.), Quality assurance in higher education: selected case studies, pp. 57-66. Vancouver: The Commonwealth of Learning, 1997.

Seaborne, K. Quality assurance in the provision of library services in British Columbia. In A. Tait (ed.), Quality assurance in higher education: selected case studies, pp. 77-88. Vancouver: The Commonwealth of Learning, 1997.

Tait, A., (ed.) Quality assurance in higher education: selected case studies. Vancouver: The Commonwealth of Learning, 1997.

 Quality assurance procedures at the Open University of Israel 

By Sarah Guri-Rosenblit

Introduction

Many of the full-fledged, autonomous distance teaching universities that were established in the early 1970s, following the model of the United Kingdom Open University (UKOU), have initiated special quality mechanisms in designing their learning materials, monitoring their learning and teaching process, and assessing their students. These quality assurance procedures purported both to improve the quality of academic teaching in a distance learning setting, and to ensure its respectability and credibility in the context of the national higher education system.

This case study examines the quality assurance procedures exercised by the Open University of Israel (OUI) in relation to comparable and different procedures employed by some other large distance teaching universities. With a small nucleus of full-time academic faculty and heavy reliance on external colleagues from traditional universities, the OUI has adopted unique quality mechanisms in its everyday operation and organisation.

This case study analyses the special quality assurance processes used by the OUI for course approval, for course development, and for monitoring the learning, tutoring, and teaching functions. The merits and the problems associated with employing these quality procedures are elaborated.
Before presenting and examining the quality assurance mechanisms at the OUI, the meanings attached to quality in higher education settings, in general, and in distance teaching universities, more specifically, are discussed.

Quality Assurance in Higher Education

The pursuit of quality is an important issue in higher education world-wide. In the last decade both national governments and international organisations have shown a growing interest in defining and assessing quality in higher education institutions (Vught and Westerheijden 1993; Westerheijden et al. 1994). But quality is a vexed, complex, and relative term. Vught claims that "The concept of quality is as elusive as it is pervasive. Universal agreements on the meaning of quality or a final answer regarding the definition of this concept seems impossible to reach" (Vught 1994: 36). Harvey and Green in trying to define what quality is in higher education have reached the conclusion that "it means different things to different people, indeed the same person may adopt different conceptualisations at different moments ... There is a variety of 'stake holders' in higher education, including students, employers, teaching and non-teaching staff, government and its funding agencies, accreditors, validators, auditors, and assessors ... Each have a different perspective on quality. This is not a different perspective on the same thing, but different perspectives on different things with the same label" (Harvey and Green 1993: 10). Harvey and Green had grouped the various definitions of quality into five categories:

· quality as exceptional;

· quality as perfection or consistency;

· quality as fitness for purpose;

· quality as value for money; and

· quality as transformation.

Traditionally the notion of quality in higher education has been associated with meritocracy, with belonging to a small, privileged, and exceptional class. Education at Oxbridge or at the Ivy League universities in the United States, for example, implies distinctiveness and exclusiveness. All of the distance teaching universities (DTUs) have been established on a totally different philosophy. They were meant to open up the higher education systems to diverse and heterogeneous clienteles, to widen access to university studies, and to contribute to the democratisation of higher education systems. Concurrently, they have had to prove that the education they offer is valuable and on a high level, and in some aspects of an even higher quality than their traditional counterparts.

Excellence is often used interchangeably with quality. Excellence sees quality in terms of high standards (Moodie 1988; Reynolds 1986). Aspiring for excellence in research is a common goal of all universities, but excellence in teaching is rarely rewarded. Many scholars consider instruction as a compulsory task that they have to perform, and devote most of their efforts and energy to their scholarly studies. But DTUs by their very nature pay special attention to teaching. By establishing high standards in distance teaching, the DTUs have ensured themselves a unique and respectable position in the academic world. Clearly, one of the important areas in which DTUs choose to excel is the development of high quality learning materials, which are designed to facilitate and enhance self study. The preparation of the materials is at the centre of the learning and teaching system at all DTUs, taking up the largest proportion of academic faculty time. The course materials produced by DTUs are "transparent" in the sense that they are open to scrutiny and criticism, and thus there is control over the quality of instruction by virtue of its public nature (Perry 1976; Reddy 1988; Guri-Rosenblit 1993).

The fact is that DTU students are not the only beneficiaries of high quality learning materials. Faculty and students at most traditional universities use extensively the materials developed by the DTUs as university textbooks, because of their clarity, integrated structure, and overall appealing design (Guri 1987; Guri-Rosenblit 1990). The learning materials are routinely scavenged by faculty at traditional universities for ideas and content presentation. Some DTUs have become the biggest academic publishing houses in their countries. In this sense, many DTUs have contributed most conspicuously to the production of high level university textbooks, mainly at the undergraduate level. Furthermore, since the textbooks or study units are written in the spoken language of each national setting, many DTUs, such as those in Spain, Israel, and Germany, have played a crucial role in assisting all first degree students to overcome some of the difficulties they encounter when assigned to read mainly in English.
In order to achieve high quality learning materials, each DTU had to define special quality assurance procedures. It is important to distinguish between these and quality control procedures. The latter refers to a set of operations that measure, and if necessary adjust, a product's appropriateness according to a set of predetermined required criteria. Quality control in relation to academic teaching entails considerable conceptual and practical difficulties (Alexander and Adelman 1982; Guri-Rosenblit 1993). It is particularly complicated to define exact standards against which it is possible to evaluate the fitness of an academic course for its purpose-whose purpose, and how should fitness be assessed?
Quality assurance does not purport to clarify the standards or specifications against which to measure or control quality. "Quality assurance is about ensuring that there are mechanisms, procedures and processes in place to ensure that the desired quality, however defined and measured, is delivered. The assumption implicit in the development of quality assurance is that if mechanisms exist, quality can be assured" (Harvey and Green 1993: 19-20).

In the context of this case study, it is important to state that most of the procedures employed at DTUs for monitoring the quality of the self-study courses are mainly quality assurance procedures, rather than quality control mechanisms that specify exact criteria against which the quality of the final products such as study units, textbooks, and readers can be assessed.
Course Approval Procedures.

In most traditional universities, course proposals are rarely subject to a thorough evaluation. Frequently the appraisal process is no more than a "rubber stamping formality at faculty or Senate level which ensures that outward forms of appraisal-title, syllabus and examination arrangements are consistent with prevailing practices" (Adelman and Alexander 1982: 9).

In DTUs, mainly at those that focus their activity on the development of learning materials, course approval is usually subject to more stringent quality assurance procedures as compared to traditional universities. The need to employ special control regulations for course approval at the OUI stemmed mainly from its unique model of operation. From the outset, the OUI was planned to base its academic work upon a nucleus of internal faculty members working together with outside contributors from other major Israeli universities. The OUI was established in 1974 by the Government of Israel and by the Rothschild Foundation, and it has been greatly inspired by the remarkable success of the UKOU. Interestingly the OUI has in some respects followed the recommendations of the UKOU's planners more closely than the UKOU itself. When the UKOU was initiated, both the planning and the advisory committees had recommended that it be based on a small internal faculty. "The Advisory Committee envisaged that the University would require a central professional staff of between forty to fifty. This total included not only academic staff, but also the administrative and operational staff that would be required" (Perry 1976: 77). But the first vice-chancellor, Walter Perry (now Lord Perry), invested a huge effort to alter this decision and to mobilise a substantially greater number of academics. He insisted that the academics engaged in developing a course be full-time members of the institution: "I am quite sure that we were right to employ as our main course creators full-time academics of the Open University, and eschew the original idea of the Advisory Committee, and indeed of the Planning Committee, of using mainly consultants or people of secondment from other universities" (Perry 1976: 92).

Naturally, the size of the academic faculty influences the quality assurance procedures employed by each DTU in approving course proposals and in developing its courses. The German Fernuniversität had, in 1991, 430 professors with around 53,000 students (EADTU News 1993). In 1992, the UKOU had an academic staff of 810 with nearly 100,000 students (Open University 1992), while the Spanish Universidad Nacional de Educacion a Distancia (UNED) had over 800 professors with nearly 100,000 students (UNED 1992). In contrast, in 1994, the OUI had only 32 full academic faculty members with over 23,000 students (Open University of Israel 1995).

Obviously, a DTU like the OUI, which depends heavily on the employment of external academics from other universities, faces specific problems and has to establish totally different quality assurance procedures compared with a DTU that possesses a large staff and develops its materials mainly with its internal faculty.

As a consequence of the small internal faculty at the OUI, several hundred scholars from the seven traditional Israeli universities are employed on short-term contracts to consult, write, and re-write varying portions of its courses (Guri 1987). The co-operative work between internal and external faculty and the continuous recruitment of outside contributors have led to the creation of special course approval rules, described schematically in Figure 1.

Figure 1. Course approval procedures at the Open University of Israel Figure 1. Course approval procedures at the Open University of Israel 

 

 

The recruitment of external academics is performed in several ways, which range from personal acquaintance to a systematic search for experts. Guidance is crucial for outside contributors, since most of them are not aware of the special characteristics of the OUI's learning system. Jenkins (1983) specified several models for training writers at DTUs:

· training by correspondence;

· self-tuition;

· intensive workshops; and

· in-service training involving an editor and a writer working closely together.

The OUI uses an alternative model in which the external academics are personally matched and guided by the internal academic faculty. Obviously, some course teams also combine both internal and external authors. All of the external contributors receive guidance on the elements that compose a self-study written course, and are advised on how to prepare the course proposal and write the sample material.
The decision to request sample materials from tentative authors resulted from a series of previous failures, in which many of the written study units did not meet the basic requirements of an instructional text, and had to be dumped. Even brilliant scholars with an outstanding reputation in the academic world can fail to transfer their knowledge and expertise into an instructional written discourse. Writing texts with a didactic apparatus suitable for self-study settings requires special skills that are quite often different from those required for writing a scholarly article or lecturing. Some of the external professors were either surprised or shocked when presented with challenges to their assumptions about teaching and were forced to rethink how they should present their subject matter. A few resisted training and criticism and resigned from writing for the OUI.

The need to institutionalise quality mechanisms for approving and writing learning materials at the OUI results partially from the fact that part-time external faculty lack the institutional commitment of full-time permanent staff. "Conflicting loyalties among part-time writers may result in the production of material which is unsuitable for distance learners. Also, because of the academic freedom they have previously experienced, they may be reluctant to allow any modifications to be made to their work by the permanent staff" (Carr 1984: 17). Naturally, the contracted authors are much more concerned with the students in their own institution than with the needs of students learning at a distance. Many of the authors may be too busy to devote sufficient time and energy to the development of self-study courses.

It is important to note that the prevailing ethos about academic autonomy in each national setting is a crucially important variable that influences the nature of the quality mechanisms employed in each DTU. The concept of academic autonomy assumes that what goes on in a particular classroom is the sole responsibility of the professional scholar concerned, which rests on a view of the academic "as professionally competent over the full range of activities he [or she] undertakes, and this competence includes the necessary knowledge and skills to make or seek insightful and valid appraisals of his work and act on these appraisals" (Adelman and Alexander 1982: 16). In Germany and Spain, for example, where the ethos of academic autonomy is most valued and sacred, it was unthinkable to develop formal compulsory quality assurance procedures for evaluating the learning materials written by professors at UNED or at the Fernuniversität. Both of these DTUs employ the author and editor model (Smith 1980). In this model the materials written by a faculty member are edited by a professional editor. The work of the editor may be limited to proofreading and assisting with graphics and layout, or it may involve substantial restructuring of the author's work. But even the editor's employment at UNED and at the Fernuniversität is not mandatory, but just available and recommended.

At the UKOU, on the other hand, the evaluation of writing by colleagues is an integral ingredient of the course team approach. It seems that in the United Kingdom it was easier to implement quality assurance of teaching as compared to DTUs in continental Europe. The external examiner system in the United Kingdom attempts to ensure comparability of standards across higher education institutions. The external examiner system, by its very nature, puts limits to the practice of academic freedom, by appointing external academics to evaluate and even change the content and structure of the final exams, in courses taught by fully established academics. In such an environment, it might be speculated that it was simpler to implement the UKOU's course team approach.

The OUI faced the most complicated situation. The ethos of academic autonomy is most guarded and valuable in Israeli universities, comparable to the situation described in Germany and in Spain. But the OUI had decided to employ stringent quality procedures for developing its courses. As a result, some scholars from the Israeli traditional universities had felt reluctant to undergo procedures of inspection and criticism, and did not co-operate with the OUI. Some even claimed openly that assessment contradicts the very basic concept of academic autonomy (Guri-Rosenblit 1993).
Those contracted scholars who are willing to write for the OUI are asked to submit a detailed course proposal (the same procedures apply also to the OUI's internal faculty). In the proposal they must specify the following details:

· the theme of the course;

· its place in the disciplinary programme;

· the level of the course (introductory, intermediate, or advanced);

· the course's prerequisites;

· the course's overall goals;

· the structure of the course and its division into study units;

· specific objectives of each learning unit;

· a brief description of each study unit;

· audio-visual material to accompany the written learning materials;

· prospective radio, television, computer, or satellite programmes to be developed;

· the academic staff who will comprise the course team;

· proposed academic consultants to whom the study units will be sent for evaluation; and

· a tentative timetable for completing the writing.

The detailed course proposal is usually sent for evaluation to as few as three and as many as seven experts in the field, who work at other Israeli universities or, in a few cases, abroad. Some of the evaluators may be internal faculty, who may also comment on the sample material, which is usually a study unit or a substantial part of it, in relation to the clarity and pedagogical quality of the written text. Comments are asked from both an academic and an editor.
The course proposal evaluators are asked to comment on the following matters:

· the scope of the course's content as reflected in the proposal;

· the structure of the course and its division into sub-themes;

· the appropriateness of the content to the course's specified goals; and

· the update of the course's content.

The comments of outside experts are needed to gain a variety of possible perspectives on a given topic from several professionals. It enables the OUI's subject committees, which are composed of academics coming from different disciplines to discuss each proposal constructively. The area committee of social sciences, for example, is composed of psychologists, economists, sociologists, educators, and political scientists. In such a structure in which very few experts represent any given discipline, it is impossible to evaluate a course proposal without the assistance of expert outsiders.
Subject area committees then discuss each proposed course in the presence of the author or authors. In such meetings three alternative decisions might be reached:

· to approve the development of the course as it was proposed;

· to approve the course proposal with restrictions, in which case revisions are needed; or 

· to reject the course proposal.

Between thirty to forty proposals are discussed in the various subject committees each year. Approximately 20% are rejected on the grounds of inadequacy or poor quality material, and another 20% require redrafting. Decisions are taken by a majority vote of the subject committee's members. If corrections are needed, the author is advised how to go about them. Sometimes the subject committee will reassemble to discuss a revised proposal. Minor corrections are sent to the committee's members individually for comments. If the course is approved without restrictions, the subject committee submits its decision to the academic committee, the OUI's higher academic authority (comparable to the senate in other universities).

The academic committee, composed of the OUI's academic faculty and professors from other Israeli universities, is chaired by the OUI's president. Only rarely does the academic committee refute the decisions of the subject committees; however, its members might ask for clarifications and require changes or reconsideration of various elements. The fact that the academic committee is comprised of scholars from a broad spectrum of disciplines, working at different universities, enables it to perceive the development of a specific course in the context of the total higher education system. This perspective is particularly beneficial when dealing with an interdisciplinary course. After the academic committee approves a proposed course, a "green light" is given, and it enters the development phase.

Course Development

Course development at the OUI takes between three to five years, the greatest proportion of that time being devoted to the writing and rewriting of learning materials. Most of the course materials are approximately 1,000 pages, divided into ten to twelve study units. Eighteen to twenty-four courses are required to complete a bachelor's degree. The development of a course involves a team of professionals and costs over $250,000.

A given course may be written by several authors or by just one, but there is always a course team chair who is responsible for the scope and content of the total course. The course team chair determines the course's structure and methodology and is responsible for directing the other authors, if any.
The course team includes all of the academics involved in writing the study units and a course co-ordinator, whose task is to regulate the team's work and to provide a link between the authors of the study units and the others participating in the course production process, such as academic consultants, editor, graphic designer, media specialist, librarian, publishing office, and so on. Course co-ordinators are usually doctoral students or they hold a master's degree in the specific course area. The course's chair and the co-ordinator constitute the "nuclear course team".

Successful co-operation between the "nuclear course team" and the other members of the team depends greatly on the management skills and the personality of the course 
co-ordinator, especially when an external academic is responsible for the development of a course. Around 15% of approved courses are dropped in mid-production due to a variety of problems, the most common of which is the submission of poor quality texts or the failure of external academics to devote enough time to completing the study units according to the timetable they had promised. In order to encourage external authors to submit their texts according to a given schedule, the OUI has initiated a special incentive. The authors are promised to be paid twice the amount of money for writing a study unit if they complete it by the deadline agreed upon between themselves and the head of the course development administration. A course chair is paid twice the sum for developing a course if the development of the whole course is accomplished according to the deadline specified in the initial contract. This specific policy, which has been in practice only since the end of the 1980s, has resulted in a significant improvement in the pace of course development.
Whether the writer is an external academic or an internal faculty member, the first draft of each study unit is addressed to the nuclear course team. If the chair or the co-ordinator do not recommend any drastic changes, it is sent for comment to two to five outside and internal experts in the appropriate field (Guri 1987).

Evaluation of the study units is performed on the basis of some definite questions, the most common of which are (the order does not imply any priority):

· Is the material up to date and accurate?

· Are the explanations clear and fluent, and do they meet the standards of self-study

  materials?

· Are the presentations interesting and stimulating?

· Do the learning activities and assignments enhance learning and assist the student to

  comprehend the   main points and critical issues?

· The study unit is designed for 15 to 20 learning hours. Does the scope of the unit meet

  this criterion? Is it too overloaded, or is it too limited?

The goal of the evaluation at this stage is to analyse the content and instructional quality of the learning materials carefully and critically. By using the intellectual resources of other universities, the OUI tries to upgrade academic instruction through collective criticism and intensive brainstorming. Obviously, collaboration with other universities is essential in the OUI, which, as stated before, is based on a tiny internal staff.

If most of the critical comments recommend a revision, the nuclear course team meets with the author, and the required revisions are discussed. Some authors take it upon themselves to revise and even to rewrite their first draft, while others prefer to pass the mandate to the course team, mainly to the course co-ordinator. Occasionally, a unit has to be rewritten by a different author. In all cases, the nuclear course team has a crucial role in deciding on the instructional design of each unit.
The OUI's quality assurance mechanisms in the course development process may be analysed in relation to some other models. Smith (1980) classified course development procedures in distance teaching universities into five broad categories:

· the course team model;

· the author/editor model;

· the contract author/faculty model;

· the educational adviser model; and

· the intuition model.

The OUI's model combines some components of the "course team model" and the "contract author/faculty model", adding to them its own unique elements (Guri 1987). The concept of a course team, consisting of academics, television and radio experts, editors, graphic designers, educational technologists, tutors, and others as needed, has been developed by the UKOU. The OUI is based on a scaled down version of the course team approach, because it relies on a small internal faculty. The OUI invests great energy in guiding external contracted scholars, and in ensuring a flowing communication among team members. Naturally, in teams consisting largely of outside contributors "communication among them and the co-ordination and integration of their individual contributions in terms of content, pedagogy, etc., are more difficult to achieve" (Carr 1984: 18).

The contract author/faculty model is used by universities that rely on external contributors. According to this model, outside experts are contracted to write a course or a unit, "but the material is vetted by the full-time faculty of the University" (Smith 1980: 65). Smith indicates that this "model enables courses to be developed much more quickly than would happen under a course team arrangement, but significant amendments and revisions to the original drafts are much more difficult to effect under this system" (Smith 1980: 65).

The basic elements of the contract author/faculty model exist in the OUI, but with substantial modifications. The drafts written by external authors are subject to the same quality assurance procedures as those written by internal faculty. Quite often the external work requires more substantial revision than the materials written by the OUI's faculty. Thus the development of courses based heavily on outside contributors takes more time than those produced by internal academics.
Halperin argues that the involvement of external academics in writing the OUI's courses has had an impact on improving the academic teaching at the traditional universities. He claims that the OUI has contributed to Israeli higher education "by paying pre-eminent attention to effective pedagogy and by addressing the crucial question, 'what is quality in higher education?' (Halperin 1984: 99). As hundreds of professors from the traditional universities have come to be involved in the OUI's work, "the spin-offs for improved teaching elsewhere cannot be doubted" (Halperin 1984: 99).
Monitoring the learning, tutoring, and teaching processes
Developing high quality learning materials does not suffice for assuring the quality of the learning and teaching process. The DTUs that have adopted an open admission policy, like the UKOU and the OUI, had to invest special efforts to ensure that their "exit requirements" are stringent, in order to establish their credibility in the academic world.

Open admission, by its very nature, attracts heterogeneous student clienteles, some of whom lack basic study skills or are unable to cope with academic studies for a variety of reasons. On one hand, the DTUs have had to construct special support devices to assist those who might benefit from university level education with appropriate help and encouragement. On the other hand, they have had to define high learning and teaching standards, which naturally result in a relatively high drop-out rate, especially at the initial stage of the studies. Opening the university gates to anyone who wishes to pursue academic studies constitutes a most advanced and liberal admission policy. Those who are capable of coping with the study requirements irrespective of their prior formal qualifications succeed and progress; those who find it too difficult, drop out.

Special quality assurance regulations to ensure the quality of the learning, tutoring, and teaching processes have been constituted at the OUI. Some mechanisms refer to the unique interaction between the teaching course co-ordinators and the senior academic faculty, and others relate to the monitoring of tutors by the teaching course co-ordinators.

The reliance on a small academic faculty has dictated the transfer of most of the tutoring and teaching responsibilities to junior academic staff, who include the teaching course co-ordinators and the student support team. At the OUI in 1995, 192 teaching course coordinators were responsible for the instruction of 339 courses, studied in 2,359 study groups held in over 100 study centres (Open University of Israel 1995). The teaching course co-ordinator is responsible for the whole range of activities that are part of the learning and tutoring processes: preparing the assignments and final exams; recruiting and guiding the tutors; preparing instructional aids for the intensive tutorials held once a week; monitoring the tutors' work; checking the final exams; and recruiting external examiners, if needed. Clearly, the teaching course co-ordinators possess a significantly wider and greater academic responsibility compared to teaching assistants at traditional universities. Some of the teaching course co-ordinators hold a doctoral degree, but most of them are either doctoral students or hold a master's degree in the specific relevant areas of the course or courses they are responsible for. Ideally, the course co-ordinator, who had been responsible for the development of a given course, becomes the teaching course co-ordinator after the development phase has been completed. But it is not always the case.

Special mechanisms were established so that senior academic faculty could monitor the teaching course co-ordinators. Those who have not completed their doctoral studies must submit all of the final exams for evaluation by the senior faculty. In areas in which no internal expert exists at the OUI, external academics are contracted for that purpose, usually those who were in charge of developing the relevant courses. Once a year all of the assignments and exams prepared by the teaching co-ordinators are evaluated by senior academic faculty, and submitted to the head of the teaching administration (who is an academic faculty in charge of the overall teaching operation). The teaching course co-ordinators are promoted professionally on the basis of their performance's evaluation.

In several areas, such as in education, mathematics, and computer sciences, teams composed of both senior and junior faculty are responsible for teaching courses in the relevant disciplines. In 1995, a special revision committee, headed by Professor Ginzburg, who was the OUI's president from 1977 until 1987, has recommended that the OUI's activities be restructured on disciplinary departments, and that the division between course development administration and teaching administration be abolished. This committee stresses that the current volume of the OUI's operation justifies the enlargement of its senior academic staff and the enhancement of the collaboration between the senior and junior academic faculty in the teaching phase.

The teaching course co-ordinators are responsible for mobilising the tutors and for guiding them in their responsibilities and teaching activities. Each tutor is assigned a certain number of students (from 15 to 40) with whom he or she interacts in writing (checking assignments); orally (counselling via the phone); and in a tutorial group meeting. It is important to note, that currently over 80% of the OUI's student body prefer to study in the framework of intensive tutorials, which means meeting with the tutor once a week for three hours in each course. Since distance does not constitute a real obstacle in Israel, the OUI has gradually moved towards providing more face-to-face tutorials, at the students' request. Such a situation, puts a heavier role on the tutor's shoulders, and threatens the centrality of the learning materials in the learning and teaching process. Thus monitoring the tutoring sessions is crucially important to both ensure comparability of standards in hundreds of study groups spread across Israel, and to guarantee the centrality of the instructional test in the OUI's operation.

The intensive tutorials had led to the creation of staff development training, in which newly recruited tutors are initiated to the unique elements inherent in a distance teaching university, like the OUI. They are encouraged to elaborate various topics presented in the study units. But it is emphasised strongly that the written text with its didactic apparatus is in place of the professor in a regular university, and by no means are the tutors expected to lecture or teach the subject matter instead of the textbooks. They are to facilitate the students' learning, and assist in comprehending difficult and complicated issues.
In order to ensure common standards, the teaching course co-ordinators are urged to prepare general syllabi for the tutorials, and to provide slides, overhead transparencies, and television and satellite programmes, if relevant. The quality of the tutorials and their effects are evaluated and researched constantly by the evaluation division, resulting in recommendations for ongoing changes and improvements. Each semester, the teaching course co-ordinators are submitted a report as to the students' achievements in each study group, as well as the results of evaluation questionnaires reflecting the students' evaluation of the learning materials and the tutorials. In addition, the teaching course co-ordinators must visit at least once per semester each of the study groups they are responsible for, and to submit a summary report to the head of the teaching division. Furthermore, samples of the assignments, checked by the tutors, are evaluated each semester by the teaching co-ordinators, in order to guarantee as much as possible common evaluation measures.
An additional quality assurance procedure relates to the final exams, which refer only to the content presented in the study units. In order to ensure comparability of standards, the final exam is checked not by the group's tutor, but by either the teaching course co-ordinator or external examiners contracted for that purpose by the OUI.

Concluding Remarks

All of the quality assurance procedures at the OUI discussed in this case study provide an example of how a small internal academic faculty can manage quite efficiently the operation of a relatively large scale distance teaching university. The OUI has succeeded in doing it by tapping the intellectual resources of other neighbouring universities, and by defining special quality assurance regulations to ensure the quality of its learning materials and of the learning and teaching processes.
Such an example might be of special interest to both developing and small size developed countries, in which a large DTU like the UKOU, based on a large academic staff, is not feasible.
However, it is worth mentioning that the insistence on assuring quality in the contest of the Israeli OUI has its pitfalls and prices. The fact is that the internal academic faculty is "over-worked", and torn between many responsibilities. It is difficult to recruit on an ongoing basis external academics for writing and revising the learning materials. It is already clear at this stage of the OUI's development, that there is a real need to extend the size of the academic faculty. The move towards the departmental structure described here will ease the burden on the internal faculty. Compared to some other large DTUs, however, even if the OUI's internal academic faculty doubles or triples, it will still constitute a small faculty.

References

Aldelman, C., and R. Alexander. 1982. The Self-Evaluating Institution. London: Methuen.
Carr, R. 1984. "Author Employment in Distance Teaching Universities", ICDE Bulletin, 6: 16-20.
EADTU News. 1993. FernUniversïtat - Gesamthochschule Hagen, Special Institutional Insert, 12 (January).
Guri, S. 1987. "Quality Control in Distance Learning", Open Learning 2 (2): 16-21.
Guri-Rosenblit, S. 1990. "The Potential Contribution of Distance Teaching Universities to Improving the Learning/Teaching Practices in Conventional Universities", Higher Education, 19: 73-80.
Guri-Rosenblit, S. 1993. "Quality Assurance Procedures in Developing Academic Courses: A Comparative Study of Five Distance Teaching Universities". In A. Tait (ed.), Quality Assurance in Open and Distance Learning: European and International Perspectives. Collected papers from the Cambridge International Conference on Open and Distance Learning, 99-113. Cambridge: The Open University.
Halperin, S. 1984. Any Home a Campus: Everyman's University of Israel. Washington: The Institute for Educational Leadership.
Harvey, L., and D. Green. 1993. "Defining Quality", Assessment and Evaluation in Higher Education, 18 (1): 9-34.
Jenkins, J. 1983. "Tell Me How to Write". In D. Sewart, D. Keegan, and B. Holmberg (eds.), Distance Education: International Perspectives, 319-320. London: Croom Helm. 
Moodie, G. C. 1988. "The Debates about Higher Education Quality in Britain and in the USA", Studies in Higher Education, 13: 5-13.
Open University. 1992. Pocket Guide to OU Figures. Cambridge: The Open University Press.
Open University of Israel. 1995. President's Report. The Open University of Israel Press.
Perry, W. 1976. Open University: A Personal Account of the First Vice-Chancellor. Cambridge: The Open University Press.
Reddy, G. R. (ed.) 1988. Open Universities: The Ivory Towers Thrown Open. Sterling Publishers Ltd.
Reynolds, P. A. 1986. Academic Standards in Universities. London: Committee for Vice-Chancellors and Principals of Universities in the United Kingdom.
Smith, K. C. 1980. "Course Development Procedures", Distance Education, 1 (1): 61-67.
UNED. 1992 Memorio Curso Academico. Universidad Nacional de Educacion a Distancia.
Vught, F. A. van, and D. F. Westerheijden. 1993. Quality Management and Quality Assurance in European Higher Education: Methods and Mechanisms. Luxembourg: Office of Official Publications, EC.
Vught, F. A. van. 1994. "Intrinsic and Extrinsic Aspects of Quality Assessment in Higher Education". In D. F. Westerheijden, J. Brennan, and P. A. M. Maassen (eds.), Changing Contexts of Quality Assessment: Recent Trends in West European Higher Education, 31-50. Utrecht: Uitgeverij Lemma B.V.
Westerheijden, D. F., J. Brennan, and P. A. M. Maassen (eds.). 1994. Changing Contexts of Quality Assessment: Recent Trends in Western European Higher Education. Utrecht: Uitgeverij Lemma B.V.

 Quality assurance practices and principles: the case of Indian distance education 

By Badri N. Koul

Introduction

At one time, standards of education in India were maintained through a prescribed syllabus and a final examination to evaluate student performance. Even though this approach to quality control did not quite serve the purpose, most of the efforts to improve it came to naught. Then, after independence in 1947, a conceptual shift from quality control to quality assurance was noticed. Efforts made to improve the quality of education through inputs like streamlined funding, appropriate infrastructure, and relevant guidelines pointed to a new guiding assumption, that if the quality of the inputs improved, then the quality of education stood assured. Reality, however, did not seem to confirm this stimulus-response relationship. In 1982, debate on quality issues became more intense when the first open university in the country was established. This debate has led to yet another shift-to total quality management in which the attention has shifted from inputs to the processes of education. Such a close parallel between education and industry is obvious today in the context of distance education, where the phenomenon of total quality management is just emerging. The next ten years will see how it shapes and influences education in India.

Quality Consciousness in Indian Education: The Background

Ever since university education was introduced in India (1857 to be exact), its quality has remained generally intractable. It is true that well-meaning educators have, from time to time, expressed their concerns about its quality, but, by and large, the overall dynamics of education have been governed by market forces-mainly the vast gap between supply and demand:

..hitherto Allahabad has conformed to the practice of the three original universities, and confined itself to conferring degrees on candidates who pass its examinations …" (Quinquennial Review of 1897-1902, 22)
The ever increasing demand for personnel to support a rapidly expanding bureaucracy for the education of an exponentially rising population allowed us to accept mediocrity even a century ago as we do today-no wonder that we have become hoarse talking about falling educational standards for decades now.
In spite of the various significant steps towards reform taken in the late 1920s, the scene at the end of the first quarter of the century did not show any appreciable improvement:
... the theory that a university exists mainly, if not solely, to pass students through examinations still finds too large acceptance in India, ... They have been hampered in their work by being overcrowded with students who are not fitted by capacity for university education and of whom many would be far more likely to succeed in other careers. (Simon Commission Report of 1929, 29-30)
If the theoretical standards of the curriculum were really enforced, the elimination would be much higher, but the University of Calcutta has depended for the finances of its post-graduate work on Matriculation fees, and for financial reasons among others has kept its standards low. This low standard in the end-examination means lax promotions, and lax promotions means ill-graded classes, and ill-graded classes mean an impossible task for the teacher and consequently worse teaching; ..a vicious circle ... (West, 111)

By the time India became independent, the weaknesses of the institution of Indian education were quite conspicuous:

Anyone who studies the story of universities in India since 1857 cannot escape the conclusion that the system of higher education inherited at independence from the British Raj was dangerously weak in three ways: (i) During the British rule we failed to set and maintain the quality of teaching and the standards of achievement essential to a university, (ii) We failed to devise, and to persuade Indians to accept, a content of higher education suited to India's social and economic needs and (iii) We failed to establish patterns of academic government and relations between universities and state, which would accord to university that degree of autonomy without which they cannot serve society properly. (Ashby and Anderson, 138)

Efforts to overcome these weaknesses were started in earnest with the Education Commission (1948-49), which assumed that quality assurance was a consequence of adequate inputs. Some of the major steps taken are manifest in the following:

· the establishment of the University Grants Commission (UGC) in 1956;

· the Association of Indian universities;

· the various State Boards of secondary and higher secondary education;

· the Central Board of secondary and higher secondary education;

· the National Council of Educational Research and Training (1961);

· the Asian Institute of Educational Planning and Administration (1962); and

· the establishment of Academic Staff Colleges (in the mid 1980s) to provide orientation and 

  refresher programmes for the junior academic staff of colleges and universities.

At the level of higher education, the following measures were seen as sufficient to ensure quality in education:

· prescribing qualifications for various personnel to be employed;

· fixing the minimum levels of infrastructure in terms of land, buildings, classrooms,  

 

  laboratories, furniture, equipment, libraries, and so on;

· providing guidelines for the content and conduct of syllabuses and examinations;

· providing funds and grants in aid for the promotion and development of new subject areas;

· establishing new institutions.

At the secondary and higher secondary levels, the key to quality assurance continued to be seen in reforming, upgrading, and updating the syllabuses and examinations, as well as enforcing strict adherence to recruitment norms. These measures instilled confidence and for a while it was felt that the quality of education in India had been secured for the future.

The corresponding expansion of education and the speed with which students entered at all levels of admission, together with the rising aspirations of newly awakened populations of diverse learner communities, however, opened the way for independent institutions and unrecognised universities that would neither depend on the University Grants Council for grants, nor observe the necessary conditions to make their operations legally valid.

This contemporary scenario recalls a mirror-image in what has been reported about the situation around a century ago:

The organisers of Bengal High Schools were discovering that these schools could be run on a self-supporting basis without Government grants, and they need not therefore submit to the conditions which the department imposed. (Government of India, 22)
Obviously, the steps taken since 1947, though based on a different paradigm, have not improved the situation to the extent expected.

More recently, with the establishment of open and distance education, the issue of quality has become more serious. Partly because distance education institutions launched courses without any reasonable understanding of the distance education system, very often without sufficiently preparing for effective distance education transactions, the questioned perception that distance education could provide an effective teaching-learning environment and the issue of falling standards came to the fore when the practices and principles of conventional and distance education were compared. The initial reaction of distance educators in India manifested in their building a defence for distance education as an effective educational system, as if quality was an issue only for distance education systems and not for the conventional system. The perceived effectiveness of conventional education was used as a standard measure for the effectiveness of distance education. Distance education expanded with an overwhelming momentum, both in its use and misuse, as teacher training and science-based programmes began to multiply rapidly on the one hand and professional-vocational and awareness-extension programmes expanded on the other. The debate become broad based as the validity of using conventional notions of effectiveness as a measure for the effectiveness of distance education came to be questioned. The attention of those concerned shifted slowly but steadily from mere perception to the reality-hardly anything in concrete terms could be characterised as a quality assurance mechanism in the Indian educational system, be it conventional education or distance education. The UGC has realised as well that they play a recommendatory and funding role and that, if an institution can mobilise funds independently, it will not bother about UGC recommendations and guidelines. It was time the issue of quality in education was addressed afresh.

This brings us to the latest thinking on and the steps taken to achieve quality in education. Rather than depending on the assumption that quality is assured by ensuring inputs, the paradigm of the 1990s is that quality can be assured only when all the processes involved are ensured through appropriate management-not assumptions but appropriate activities ensure quality. Accordingly, the National Assessment and Accreditation Council (NAAC) was established (in 1995) within the UGC to plan and implement schemes for ensuring quality in higher education. For distance education operations in India, the Distance Education Council (established in 1992) within Indira Gandhi National Open University (IGNOU) is expected to take this responsibility and work in close collaboration with NAAC. Both bodies, however, have as yet taken only what may be called the initial steps.
Quality Assurance: The Case of Higher Distance Education

At present seven open universities operate in India. Of these, two state open universities have yet to start operating; another two have started operating but they have not reached a stage when their quality assurance practices and principles could be looked into. The remaining three universities are Dr. B. R. Ambedkar Open University (BRAOU) of Andhra Pradesh (1982), Indira Gandhi National Open University (IGNOU) of New Delhi (1985), and Yashwantrao Chavan Maharashtra Open University (YCMOU)) of Maharashtra (1989).

It is clear from the organisational structures of BRAOU and IGNOU that they have no mechanisms provided explicitly for quality assurance. The tasks involved, however, make it necessary to incorporate various mechanisms which may be labelled quality assurance scheme at the conceptual level and quality assurance operations at the implementation level. YCMOU, on the other hand, explicitly provides for quality assurance in its management plan. The relevant details of each university's quality assurance activities are as follows.

Dr. B. R. Ambedkar Open University
The main function of BRAOU is the preparation and delivery of educational programmes. Accordingly, quality assurance measures are related to the following six processes:

· planning academic programmes;

· developing curricula and learning materials;

· producing learning materials;

· implementing programmes;

· reviewing programmes; and

· developing human resources.

Planning Academic Programmes

New academic programmes are usually identified with the vice-chancellor through informal discussions with and suggestions or guidelines received from the state government, the UGC, and other national bodies. Faculty may also initiate proposals. Proposals initiated by either the vice-chancellor or faculty are placed before the co-ordination committee, which is a non-statutory advisory body made up all senior functionaries like the deans of the various faculties and directors of the various service units. After the co-ordination committee approves the proposal, the faculty or department prepares a detailed proposal for the consideration of the academic senate, the highest statutory academic authority of the university that is empowered to approve academic programmes. The detailed proposals approved at this stage are submitted to the executive council, the highest policy making and administrative authority of the university, for administrative sanction to launch the programmes.
The vice-chancellor, the faculty, the co-ordination committee, the academic senate, and the executive council, all serve as a means of assuring the relevance and quality of the programmes at the planning stage.

Developing Curricula and Learning Materials

After a programme is finally approved, an expert committee made up of the internal faculty members and external subject experts is constituted to prepare the curriculum. The committee may meet once or twice for this purpose, and the curriculum thus designed is sent for approval to the academic senate. Thus, it is the subject experts and the academic senate who look into the quality of the programme at this stage.
The approved curriculum and syllabus are passed on to the course team, which consists of subject experts (both from the university and from outside institutions) and an audio-visual producer, who is designated for the programme or course. The team is headed by a subject expert, who is designated the editor and is responsible for content editing and the quality of presentation. Language is edited by language experts. The course team also identifies the audio-visual components of the materials, which are developed by the audio-visual centre with the help of the producer and subject experts. Depending on the nature of the course or programme, field practitioners are also associated with the development of learning materials as course writers, editors, or audio-visual programme developers. In a few cases, the print materials are sent to external assessors for their comments, which are subsequently used to improve the materials. The audio-visual materials are previewed by the internal faculty before they are duplicated for use by students.

The quality assurance mechanisms that function at this stage include:

· editing learning materials for different purposes (content, format, and language);

· co-ordinating with the producer of audio-visual materials and previewing the

  audio-visual materials before they are duplicated; and

· orienting the course writers to make them familiar with the requirements of quality.

Producing Learning Materials

The learning materials in manuscript form are printed by the material production division of the university with the help of private printing agencies. They follow a style manual that the university prepared with the help of printers and internal faculty members. Internal faculty read the final proofs to ensure error free publications. Similarly, the audio-visual unit is responsible for producing the audio-visual tapes in accordance with an audio-visual manual prepared by the university.
Thus the internal faculty, the audio-visual unit, and the printing division are collectively responsible for the quality of print and media learning materials at the production stage.

Implementing Programmes

When a course is implemented, the three main processes are distributing learning materials, providing student support services, and issuing examinations. Learning materials are distributed from the material distribution division at headquarters, and they are responsible for preparing and adhering to despatch schedules. Student support services are provided by a network of study centres at different locations. The directorate of student services located at headquarters is responsible for ensuring the quality of these services by determining the norms and patterns of support services required at different study centres. The examination branch of the university is responsible for the reliability and validity of evaluation. The schemes of evaluation and the conduct of examinations are developed by the faculty with the help of external experts wherever necessary, and are approved by the academic senate and the executive council before implementation.

Reviewing Programmes

Learning materials are reviewed on the basis of feedback from learners, counsellors, and subject experts and they are updated by internal faculty. The university has set up a separate system evaluation unit, which conducts regular studies on the different aspects of learning materials and implementation processes. The feedback from these studies is used to revise materials and improve practices.

Developing Human Resources

The university has established a separate unit for staff training and development. This unit undertakes orientation and training programmes for the academic and administrative staff to improve the quality of their services.

Indira Gandhi National Open University
At IGNOU, quality assurance activities focus on processes quite similar to those at BRAOU, but the measures taken to achieve the desired quality differ considerably.

Existing built-in quality assurance mechanisms are as follows:

· planning the course or programme;

· developing the course or programme;

· producing the learning materials;

· implementing the course or programme;

· reviewing the courses and follow up activities; and

· developing human resources.

Planning the Course or Programme

Each school or division is expected to present a perspective plan covering a period of about five years. This perspective plan must be approved by the co-ordination committee and, subsequently, after incorporating the modifications the committee suggests, the perspective plan must be approved by the planning board. Within the approved perspective plan, the school or division is expected to prepare a project concept pertaining to each course or programme that they want to launch. This project concept is developed through the services of experts in the relevant field or any other means such as workshops, brainstorming sessions, and so on. The project concept is submitted to the co-ordination committee and then to the planning board for their approval.

After the project concept is approved by the planning board, the school or division develops a detailed outline of the curriculum components of the course or programme. The project design must be enriched in consultation with an expert committee, made up of subject experts and instructional designers. It is then submitted to the school board for approval.

After looking into various aspects of the project design and enriching it through interaction with experts, it is finalised as a project report, which presents the total instructional design of the course or programme for the academic council to consider.

While the project report is being shaped, another document called the launch document is developed in co-operation with the admission and evaluation division and the communication division. The launch document outlines relevant schedules as well as the infrastructure needed and the kind of services required to implement the course or programme. The launch document is also submitted to the co-ordination committee for approval.

At this point the planning stage is over. So far the mechanisms for quality assurance at the concept and design stages lie with the co-ordination committee, the planning board, the school board, and the academic council. Following these steps meticulously, the quality of the course or programme is sure to meet the objectives of the university, social relevance, economic viability, and operational feasibility.

Developing the Course or Programme

Using the approved project report and the launch document, the school or division moves on to the stage of course or programme development. Usually, the task of co-ordinating development is given to a course or programme co-ordinator, who assists the course contributors, content editors, language editors, instructional designers, audio-visual producers, and personnel involved in the project.
Course contributors, the main constituent of this team, are usually identified with the help of an expert committee. They undergo a two-day orientation programme so that they may design materials in accordance with the house style adopted by a particular school.
First drafts received from the course contributors are passed on to content editors (who are subject experts), language editors, and format editors (who are instructional designers). They look into the pedagogic and presentational attributes of the materials and make changes. The final draft is prepared and thoroughly proofread before amera-ready copies are prepared.

Besides developing the learning materials, course contributors are expected to suggest topics that need audio-visual support. These suggestions are written up as academic briefs, which are further developed as academic notes, which outline in detail the expected academic content of the audio-visual materials to be prepared. Sometimes the academic briefs and the academic notes are prepared by the course contributors; other times they may be prepared by the internal academics themselves. The academic note is passed on to the communication division, who assign a producer to produce the programme. The producer and the academic work together to build both the script and the production script. The producer plans and arranges for the technical facilities and technical support required to produce the materials. Rushes collected from outdoor as well as indoor shooting are edited at the post production centre to finalise the first version of the audio-visual programme, which is previewed by a committee of internal academics and technical personnel. After approval the programme is passed on for duplication and packing.

The quality assurance of course or programme development, and print and audio-visual materials, therefore, relies on the following mechanisms:

· orienting the course contributors;

· content editing by subject specialists;

· language editing by language experts;

· format editing by instructional designers;

· collaborating to identify themes for audio-visual materials;

· combining academics with audio-visual producers and providing high-level technical

  facilities; and

· previewing the audio-visual programmes before finalising them.

Producing the Learning Materials

The quality of learning materials production depends on the following:

· the quality of the paper on which they are printed;

· the cards in which they are bound;

· the quality of the printing itself;

· the appropriateness of layout;

· accuracy in typography; and

· the placement of diagrams.

Similarly, in the case of audio-visual materials, quality depends on the quality of blank tapes and duplication, which is carried out at the university.

The mechanisms to ensure the quality of paper, card, and blank tapes are the advisory technical committees, which set the minimum standards for the quality of the materials the university is to use. The general quality of printing is assured by penalty clauses in the agreements with the printers. Duplicated copies of audio and video tapes are also randomly checked.

Implementing the Course or Programme

A course or programme is implemented through many processes: the despatch of materials; counselling, tutoring, and practical work at study centres; assignment handling; query handling; feedback; and evaluation. The quality of these processes is assured by ensuring the prerequisites which support them as follows:

· Despatch of materials: availability of materials in the warehouse, availability of schedules, 

  adherence to schedules, quality of packing;

· Counselling, tutoring, and practical work: availability of schedules, adherence to    

  schedules, punctuality, regularity, availability of facilities, attendance of learners, quality of

  learners, quality of counselling or tutoring, practical work and interaction, use of

  audio-visual materials, learner satisfaction;

· Assignment handling: availability of schedules, adherence to schedules, short   
  turn-around time, quality of assessment;

· Query handling: pre-admission services, on-course services, and post-course services;

· Feedback on: quality of print materials, quality of audio-visual materials, quality of

  counselling, tutoring, and practical work, quality of assignments, quality of support

  services in general; and

· Evaluation: availability of schedules, adherence to schedules, conduct of examinations,

  turn-around time in the case of assessment, handling of appeals made by the students

  and discipline issues, results (time taken for declaration) and their accuracy, regular and

  timely certification.

At study centres and regional centres, quality assurance mechanisms comprise the various monitoring schemes and related feedback mechanisms that originate there. Regional centres are expected to monitor study centres for the quality of their management, tutoring, and counselling, and their general function and facilities, while study centres provide feedback on their infrastructure, the functioning of tutors and counsellors, and learner behaviour.

Reviewing the Courses and Follow Up Activities

Course review activities depend mainly on the feedback made available to regional centres and, in turn, to headquarters. Along with the project report at the start of the course, a review document is prepared, which outlines a timetable for receiving feedback as well as the various areas under which feedback is to be received. Feedback informs decisions about whether the course should continue (entailing preparation of new assignments and programme guides for every year), be modified (entailing maintenance of courses through supplements and minor revisions), expanded (entailing restructuring of courses and also increasing the content by means of revisions), or withdrawn (if the course is no longer relevant).

Developing Human Resources

The Staff Training and Research Institute of Distance Education (STRIDE), an IGNOU constituent, provides staff development programmes for the academic and non-academic staff to improve the quality of their input.

Yashwantrao Chavan Maharashtra Open University
The approach followed at YCMOU is quite scientific in the sense that total quality management was planned for as a significant concern right from the beginning. Accordingly, YCMOU made arrangements to:

· develop a quality system for the university;

· develop a total quality management model for the university;

· implement the quality system in the university; and

· develop and establish a mechanism to perpetuate total quality management in the

  university.

The quality system in conceptual terms has already been articulated in a preliminary quality manual, indicating clearly the dimensions of quality that will be attended to (namely the time a function will be executed, the product or service specified, and the costs involved) as well as the administrative and operational mechanisms needed to materialise the system. The work done so far includes a policy decision to: 

· appoint management representatives to co-ordinate the overall quality assurance activities;
· set up a quality advisory council consisting of officers and experts who will be responsible

  for implementing this quality assurance programme;
· establish a quality assurance centre for total quality management-such a centre would be

  entrusted with the responsibility of supervision and verification of the quality standards laid

  down; and 
· organise training programmes for in-house staff to implement the quality system.

Further additional work has been completed:

· detailed documents on the interface of departments; the responsibilities of every division,

  centre, and section of the university; and procedures for reviewing the feedback received

  from various units, as well as those for revising the details and distribution of the reports; 

 

· procedures for documentation, both at the institutional and the departmental levels,

  including those required to materialise the quality system. 

 

Accordingly, departmental processes have already been analysed and component activities have been identified. More work needs to be done to make the system completely functional.

Immediate requirements

In the context of quality assurance, IGNOU and BRAOU have operated and continue to operate on more or less similar conceptual lines. Differences at the operational level relate to differences in their administrative structures and the powers of various counterpart authorities and officers.

In actual practice problems that hamper quality assurance occur at various stages. The major problem areas, which provide directions for immediate research work, are as follows:

· It is not unusual to ignore some of the steps at the planning stage. We need to identify the

  steps that are ignored, the circumstances in which they are ignored, and the reasons they

  are ignored; and then we need to identify ways to improve the planning process.

· Once the first draft of the printed material is available, it is not subjected to all the checks

  described here. In many cases, if the various types of editing are actually effected, the

  proposed modifications are not incorporated in the final versions. Even proofreading does

  not come up to the mark. Set norms are needed for getting quality work done by course

  contributors, artists, and so on, as at present their relationship with the co-ordinator

  remains a long drawn tug-of-war. Consequently, our print materials display inadequacies

  only after improvement is possible. We need to locate the weak spots in print material

  production early on, and find their causes and ways to overcome them.

· For audio-visual materials, the only effective quality assurance mechanism is previewing,

  yet even that does not appear to be sufficient. Another way to improve the quality and

  utility of these materials must be found and put in place.

· As far as the quality of printing and duplication of audio-visual materials is concerned,

  various mistakes are made; for example, blank tapes may be sent to study centres; a

  diagram that should be printed on page 5 appears on page 32; or diagrams are lost at the

  printers and need to be redone. The reason for such mistakes needs to be found and

  mechanisms to block them must be developed and used.

· Feedback at the implementation stage is still minimal or unavailable even though

  procedures to obtain it are in place. Also, the turn-around time for assignments continues 

  to be unreasonably long. To reduce the average turn-around time and to get regular

  feedback and use it purposefully, we need to find the causes of slackness and tighten the

  schedule.

· We need a system of post-implementation activities; for example, surveying the types of

  employment graduates have been able to find. 

· Materials intended to be used outside India need special attention to the content

  presented (it has to be free from Indian bias), the language used (it must have international

  acceptability on issues like sexism in language), the raw material used (quality of paper,

  audio-tapes, and so on), presentation (quality of printing, packaging, and so on), and

  support services. Norms have yet to be developed for this purpose; but beforehand we

  need to sensitise ourselves to international needs and develop mechanisms for their

  quality assurance.

· Although stride offers staff development programmes, including long term programmes like

  the post-graduate Diploma in Distance Education and Master of Arts in Distance

  Education, a mechanism is needed to ensure the utilisation of these programmes at

  operational levels.

· Some of these problems have procedural solutions available in the powers or function of

  school boards. However, school boards do not seem to function as they should (for

  example, to evaluate educational material and to make suitable recommendations to the

  academic council, to review the facilities of the study centres, and so on). The implication

  is that we need to review the function of the school boards and other authorities.

  Restructuring may already be overdue.

· Exploration into, experimentation with, and implementation of advanced technologies may

  solve most of these problems. How do we proceed in this case?

Research into solutions to these problems deserves immediate attention, whatever the priorities of university management. 

Concluding Remarks

The YCMOU experience may provide useful insights and also guidance, but it must be obtained and detailed before it can be used by others. Obviously, we have a long way to go; total quality management is still a distant goal.

References

Ashby, E., and M. Anderson. 1966. Universities: British, Indian, African: A Study in the Ecology of Higher Education. London: Weidewfeld and Nicolson.
Government of India. 1950. The Report of the University Education Commission,
1948-49, Vol. 1. Delhi: The Manager of Publications.
Prasad, V. S. 1995. Quality Assurance Measures at Dr. B. R. Ambedkar Open University, Hyderabad. Personal communication.
Yashwantrao Chavan Maharashtra Open University. 1995. Quality Manual. Nashik, Maharashtra.
Quinquennial Review of 1897-1902, quoted in The Report of the University Education Commission, 1948-49, Vol. 1.
Simon Commission Report of 1929. Quoted in The Report of the University Education Commission, 1948-49, Vol. 1.
West, M. 1926. Bilingualism. Occasional Reports, No. 13. Calcutta: Government of India, Central Publications Branch.

 The roots of quality assurance at the British Open University 

By Tim O'Shea and Anne Downes

Introduction

The founders of the British Open University did not use the term quality assurance but they were very much concerned with the design and implementation of robust educational systems. In this case study we will argue that quality assurance principles and processes are inherent in the Open University's approach to supporting distance learning and were demanded by the educational mission of the university.

We take the view that there are three key quality assurance principles for an educational institution:

· that the institution have an educational mission that can be related in a tangible positive    

  way to the educational well being of society at large or some particular community;

· that it is possible to measure success in achieving that mission by focusing primarily on

  the quality of the student learning experience; and

· that at any time the institution should have explicit goals for the further enhancement of 

  the quality of student learning experience.

Quality assurance processes vary depending on the educational design and delivery methods of the institution but must fundamentally be concerned with the iterative use of feedback information from a range of sources, including admissions data, examinations data, student progress statistics, survey data, interview data, tutor data, graduate employment information, and employer views. Such feedback data can be used to inform changes in provision at a whole range of levels, from small elements of courses and course design to the broader curriculum and arrangements for local or institution wide tutorial or pastoral support. To achieve greatest value, feedback driven processes can also be embedded in hierarchical networks for multiple use. That is to say lower levels of feedback will be aggregated and compared with higher levels of feedback to form synoptic views of the educational provision and the feedback information can also be used to determine what feedback collection processes should be initiated in the future.
Quality assurance has been the subject of much debate in British higher education over the past four years and universities have been subject to both external assessments of teaching quality focused on particular subjects and external academic audits, which address the workings of universities as a whole. The Higher Education Funding Council for England, which is responsible for funding the Open University, has just embarked on a new round of subject-based assessments of the quality of education, and a quite polarised debate about the future of external quality audit and assessment continues (O'Shea, Bearman, and Downes 1996). However, from the perspective of the Open University, concern for the quality of educational provision is not new, and striving for excellence has always been an intrinsic part of the academic professionalism of our staff. So a focus for us has been explicating and communicating our quality assurance processes to external assessors and auditors.

Below, we look at the extent to which quality assurance processes were built into the Open University and how well those measures have served the university in a changing environment. We hope to demonstrate that, as an institution that was established with a modular curriculum, many of the quality assurance processes are embedded in the structure of the institution, and have been evident throughout its life. At the same time, we recognise that current thinking within the institution is much influenced by the methodologies and guidelines of external audit and assessment. We consider whether these influences are directing us towards genuine enhancement of quality, looking as examples at the development of a mission statement, curriculum design and approval processes, and teaching and learning.

Determination of Mission and Objectives

Current audit and assessment methods depend upon a clear statement of the aims and objectives of the provision in the relevant subject area. The Open University has in recent years restated its mission, as follows:

The Open University is - 

Open as to people
It will play a leading role in the move to mass higher education by serving an increasingly large and diverse student body.

Open as to places
It will make its programmes and services available to people throughout Europe and beyond.
Open as to methods
It will harness new technologies and educational techniques to serve students in their homes and workplaces.

Open as to ideas
It will be a vibrant academic community dedicated to the pursuit and sharing of knowledge.
These words by no means represent a new strategic approach for the institution. The following extracts from the inaugural address of the university's first chancellor, Lord Crowther, contain a vision of the university which, after nearly 30 years, continues to hold good:

We are open, first, as to people. Not for us the carefully regulated escalation from one educational level to the next by which the traditional universities establish their criteria for admission. The first, and most urgent, task before us is to cater for the many thousands of people, fully capable of a higher education, who for one reason or another, do not get it, or do not get as much of it as they can turn to advantage, or as they discover, sometimes too late, that they need. Men and women drop out through failures in the system, through disadvantages of their environment, through mistakes of their own judgement, through sheer bad luck. These are our primary material. To them we offer a further opportunity…Wherever there is an unprovided need for higher education, supplementing the existing provision, there is our constituency. …We are open as to places. This University has no cloisters - a word meaning closed. … We are open as to methods. … already the development of technology is marching on, and I predict that before long actual broadcasting will form only a small part of the University's output. … Every new form of human communication will be examined to see how it can be used to raise and broaden the level of human understanding … We are open, finally, to ideas. It has been said that there are two aspects of education, both necessary. One regards the individual human mind as a vessel of varying capacity, into which is to be poured as much as it will hold of the knowledge and experience by which human Society lives and moves. … the (other) regards the human mind more as a fire that has to be set alight and blown with the divine afflatus. That also we take as our ambition …

The sentiments of this inspiring speech have undoubtedly attracted many staff and students to the university, and have been echoed by a concern for access, equality of opportunity, use of innovative methods of teaching and learning, and the dissemination of knowledge and good practice, throughout the systems and procedures that have been developed to produce and present courses and learning resources over the years.

In 1969, it was sufficiently unconventional for a university to claim to be open, rather than exclusive, and to offer the flexibility of modularity rather than a three-year programme of learning. However, although the mission of the institution may be widely shared by staff and students alike, critics might ask how far the Open University has achieved its standards of openness and equality of opportunity.

Since 1971 (Perry 1976) the university has used its Institute of Educational Technology to lead and guide a range of monitoring and feedback activities and to measure the real effect of admitting students without educational qualifications to undergraduate level courses. In doing so, the university has been looking for the apparent factors that contribute to student success, in terms of preparedness before entry, progress rate, academic support and guidance, teaching media, and learning resources.

These monitoring and feedback processes provide data on which future policy decisions can be based. Many examples of enhanced provision result from the identification of good and successful practice, over the years. However, external scrutiny of process and content, in the form of audit and assessment respectively, has looked for evidence that all parts of the institution are engaged in structured processes of feedback and review, and that the whole university is constantly improving performance towards achievement of its aims. We quickly recognised that quality improvement had been greatest when enthusiastic individuals and groups had used the available mechanisms to good effect. External scrutiny has been a force for bringing each unit or activity to a minimum level of dialogue about quality assurance, but in some cases the process diverted energy and enthusiasm that might have stimulated excellence on a more piecemeal basis. A fundamental question for the Open University, as an increasing proportion of staff engage in quality assurance related dialogue, is the extent to which we can realistically expect examples of excellence, in our particular disciplines, regions, or courses, to form the baseline quality assurance procedure for all others.

Curriculum Design, Content, and Organisation

The university developed a multi-faceted approach to student learning, which includes teaching materials, correspondence tuition and tutor support, and assessment, each with a range of procedures to assure quality at the outset, and a process of review throughout the life of a course. The system of course planning has been devolved increasingly over the years, and much of the responsibility for carrying forward the development of the course profile now rests with the individual academic units. Nevertheless, the university as a whole maintains oversight of the curriculum profile, and assures that the programme of courses being developed is academically appropriate and that individual course proposals conform to agreed policy and practice. The single most important element of the Open University's approach to quality assurance in this area is the course teams, described below, which collectively have academic "ownership" of individual courses.

The five-year forward plan of each academic unit contains details of the courses, packs, or other provisions that the unit intends to produce and present in each year of that period, given certain assumptions about staffing and resource levels. The plan is approved by the faculty or school board, which comprises all its academic staff, representatives from other academic and service units, full-time research students, and representatives of the student body and tutorial counselling staff. The sum of these unit plans, when approved, represents the university's commitment to curriculum development.
The curriculum is divided into modules or courses, which may contribute to one or more programmes of study leading to an award or may be offered as free-standing courses. 

Academic approval therefore takes place at two levels:

· the award level; and

· the course level.

Award Approval

A new programme of studies leading to an award is sponsored by an academic unit, through its faculty or school board and approved by a curriculum development committee and by the academic board, acting on behalf of senate (though some new proposals require a full senate discussion). Teaching and assessment strategies are considered by the relevant university-wide committees. The university has to be satisfied that support areas can accommodate the new programme within the capacity constraints that exist. Approval for a new programme of studies is sought in the third year before the programme is presented for the first time and before consideration of the constituent courses begins.

Course Approval

If it is agreed within the unit that a course proposal is academically appropriate and that there is a likelihood of staff and resources being available to develop and maintain the course over a specified period, the outline proposal is incorporated within the relevant academic unit's five-year plan. Three years before the course is due to be presented, plans begin to be developed in more detail, a formal course proposal is drawn up for approval by the appropriate academic unit board and the procedure for the appointment of an external assessor begins.

This course approval process includes:

· approving the details of course titles, general subject matter and detailed syllabuses, 

  their objectives and the method of assessment, the structure and relationship of the   

  course components, tutorial, broadcast, and resource requirements, and any relevant

  regulatory requirements;

· appointing members of staff to all course teams, including course team chairs, subject to

  ratification by the curriculum development committee; and

· nominating external course assessors for the approval of the curriculum development 

  committee and appointment by the academic board.

A separate approval and external assessment is required for every new course, as well as for any significant change to an existing course. A decision on what constitutes a significant change in this context is made by the pro-vice-chancellor (curriculum development). Academic units have delegated authority to approve new courses and packs in certain circumstances, where resource and academic issues have already been resolved. So the faculty or school board is the main forum for debating academic and pedagogic issues connected with the course and for approving the details of the course. Referral to university-level committees takes place when any aspects of the course proposal go beyond agreed policy or practice, and may require specific approval by the appropriate committees of any plans to use broadcasts, audio or video cassettes, computing, or residential schools.

The Course Team

The course team is led by a chair, who provides its academic leadership, and the formal responsibilities of this role are specified in the university's government structure. Other members include a course manager, who is responsible for the administrative arrangements of course production; a number of writing academics; possibly one or two academic reading members; a BBC producer if television or radio programmes are included; possibly a member of the Institute for Educational Technology to advise on the delivery of the course; and an editor and designer to assist with the presentation of the final printed texts. Like the course components, the exact mix will depend upon the requirements of the individual course.
The course team carries the academic responsibility for ensuring the quality of the university's teaching of each course. It has a range of tasks:

· the definition and development of the intellectual subject matter of the course;

· the identification and development of the course's teaching strategy, integrating the range

  of resources for teaching and student support that are available; 

· creating and implementing the appropriate assessment strategy for the course; 

· ensuring the production of high quality teaching materials; 

· planning, implementing, monitoring, and reviewing the presentation of the course to

  students. 

Course team members bring with them the knowledge and expertise gained from being involved in the production and presentation of other courses. An editor not only comments on the format of the printed material, but will be able to advise authors on the way the information is presented.
The team's method of operation is intended to embed quality assurance procedures. Throughout the development and production of a course, the peer group constantly monitors, discusses, and revises the draft course material, which is subjected to a process of collective criticism and development. Having work constantly scrutinised by a peer group, and being involved in discussion of teaching and learning strategies for each course, serves as ongoing staff development for all members of staff and produces high quality courses. This process is also applied to the academic content in a wider sense, and to the teaching of the subject matter, and the examination and assessment policy. Integration is crucial in developing a successful course, both in terms of content and the use of different media to provide students with a stimulating learning experience.

When a course is being produced, some or all of it may be developmentally tested on a group of students with appropriate experience and attainments. If the course is to be a re-made version of a course that has been, or is about to be discontinued, notice will be taken of the evaluations of its predecessor, including the views of students, tutors, and external examiners.
Open University courses are divided into "units" (broadly the material required to teach a student for 12 to 15 hours per week) and "blocks" (which are groups of associated units within a course). Before a course is finalised for presentation to students it will have been scrutinised, in part and in whole, by an external assessor and by assessors of individual blocks.

The university encourages innovation in its academic staff, and innovation is one of the criteria for promotion. Innovation is also encouraged through the university's study leave policies. Encouragement to pursue research and participate in course development as inter-relating activities both feed the course development process, and offer opportunities to disseminate research findings widely.

The university has always been proud of its innovation in curriculum and course and programme design, and the quality assurance processes that support them. External scrutiny has usefully encouraged us to be more explicit about these processes, and it has also posed some very real questions about the roles we ask staff to assume, and their preparedness to do so. Although the role of the course team chair has long been established, we are now explicitly documenting what is required to undertake this role effectively, what are the pitfalls in the course production process and how can they be avoided, and how to encourage course teams to access as much information as possible about good practice within and outside the institution. At the same time, tensions develop and trade-offs are made as we attempt to increase the rate of course production, to use new technology to enable more frequent updates of material, and to be more innovative in assessment. The objective has to be to meet these new goals without long-term loss of quality or consistency.

Colling and Harvey (1995) discuss educational teams and the management of quality, and suggest that "Course teams responsible for delivery of academic programmes rarely behave as teams. Experience shows the prevailing culture is one based firmly on individual autonomy, which is often jealously guarded …". In the Open University we would argue that is not the case. Our academic staff are committed to the team approach, and the end product, in the form of tangible multi media course materials with integrated teaching and assessment strategies, can be publicly credited to all members of the team. We would argue however that the collaboration upon which the team depends is hampered by external assessment frameworks, in this case for both research and teaching quality, which still focus on a single discipline or subject area, making no real allowances for the interdisciplinarity that is so well served by a course team approach. If, as has often been convincingly argued (Beecher 1989), that many academic staff have a stronger allegiance to their subject than to their institution, then the current external quality assessment methods are likely to further reinforce this single subject focus within the university to the detriment of interdisciplinary innovation in course content.

Teaching, Learning, and Student Support

Supported open learning combines the provision of high quality multimedia teaching materials with tutorial and counselling support. Together they provide an integrated system of course presentation, incorporating advice on preparation for study, and describing options for the choice of course and award, support, and administrative arrangements. Students receive a study calendar, which helps them to pace their studies and links suggested dates for the study of each unit with corresponding broadcasts and assignments. The study calendar is complemented by details of local tutorial support, where applicable, provided by the regional centre.

The university's 13 regional centres provide a service to enquirers, applicants, and students. Each regional centre operates its own enquiry and advisory service and enquirers about the university's courses and awards are encouraged to seek personal advice in support of the printed material brochures about particular awards, courses, and packs. When an applicant first accepts a place on an Open University course, they are assigned to a tutor-counsellor. On a designated entry course, the tutor-counsellor will act as both tutor and counsellor. For subsequent or higher-level courses, students will be assigned to a tutor for the specific course, while the tutor-counsellor may continue to provide a counselling service, or this may be provided direct from the regional centre.

The university has a large body of around 8,000 associate lecturers (tutors and 
tutor-counsellors), all of whom work part time. Many of them also have full or part time work for other organisations. For 10%, the Open University is their sole employer. They live all over the United Kingdom and the mainland of Europe (including the Channel Islands, and remote Scottish islands), though all are connected to one of the regional centres. Formal responsibility for the appointment of associate lecturers rests with regional directors, advised by regionally based members of academic units (staff tutors and regional managers) and senior counsellors. In addition to recommending associate lecturer appointments, they have a responsibility for supervising their work, and ensuring that tutorial and other support policy and strategy are implemented effectively.

Recruitment needs are therefore assessed annually, and the staff tutors, regional managers, and senior counsellors take responsibility for the recruitment process within their region or academic unit, in accordance with the Guidelines for the Appointment of Tutorial and Counselling Staff, which incorporate internal policy and the principles of fair selection and equal opportunities. The various associate lecturer roles are outlined in a recruitment document Teaching with the OU. Further details of staff roles are set out in other publications which summarise best practice; for example, Supporting Open Learning and Effective Tutorials.

In the last few years about 1,000 new associate lecturers have joined the university each year. Dealing with such a large number of staff, at a distance, most of them only working for the university a few hours a week, demands a rather different approach than that used with full time staff. The university communicates regularly with its associate lecturers through written materials and face-to-face meetings. New staff normally attend at least one induction meeting, and experienced staff on average attend a staff development event every other year. Current topics are covered as necessary and often include the future plans of the university. Written communications cover a wide range, from the newspaper Sesame to individual letters. Some regions produce their own newsletters for associate lecturers.
The tutor is responsible for correspondence tuition, guided by detailed marking schemes for all courses. The marking of tutor marked assignments is monitored by central and regional full time academic staff (and, in some cases, by experienced tutorial staff), and this monitoring process provides feedback to the tutor and the staff tutor or regional manager on the quality of the grading and qualitative feedback awarded to the student.

The course tutor is also responsible for both maintaining personal contact with his or her students and responding to queries and concerns about the course. The course tutor may respond through face-to-face contact, although other factors, such as geographical distribution, may place greater emphasis on contact by letters and telephone. Within the tutorial strategy for each course, tutorial contact hours are allocated, most of which are likely to be concentrated in the form of evening or weekend tutorial meetings or day schools. Because tutorials are non-obligatory, they are regarded as supportive rather than essential to the teaching and learning process. They are usually focused on tackling students' problems and misconceptions and developing observational and interpretative skills, the ability to discuss, and so on, rather than on the transfer of a body of knowledge. The course tutor can play a significant role in "enriching" the presentation of the course, through tutorial meetings and by bringing contemporary developments to the students' attention within the framework of the course.

On some courses, students spend a week, or shorter period, at a residential school to carry out work for which laboratory or group work has no substitute. The residential schools committee, which is responsible for the central co-ordination of this aspect of the teaching provision, has a quality assurance working group, which developed a framework for quality assurance at residential school, both in terms of staff monitoring and to cover wider aspects of the students' learning experience. The university undertakes to take all practical steps to enable everyone to participate as fully as their circumstances allow.

The external audit of institutions, and assessment of the quality of education, claim to respect diversity among institutions, and discipline areas within the same institution, but have caused us to review how far we can expect to provide the same inputs to the student experience across our range of provision. The argument that what is excellent today becomes expectation tomorrow drives efforts to disseminate good practice, but we must now ask what we can say to students and assessors about the standard of provision they will receive in support of their studies, and which we can assure. The formulation of minimum standards, and the quality control processes that monitor them, are not attractive to academic colleagues, who have never seen control and policing as the route to quality enhancement. Yet there are strong pressures from society in general, apart from the development of quality "terminology" in higher education, towards the issue of student charters, articulated learning objectives (which, by implication, will be achieved if the institution is providing input of appropriate quality), and complaints systems. It is critical to balance the extent to which we transmit these pressures, and the need to perform well in external assessments against the extent to which we allow and encourage diversity at the expense of structures and systems, in deciding on, for instance, internal resource flow, the funding of new initiatives, and collaborative arrangements.

Quality Assurance and Enhancement

The quality and quality assurance mechanisms of the Open University are more exposed to scrutiny than those of any other institution of higher education in the United Kingdom, through the size of its student body, its associate lecturers, the wide-spread availability of its course materials, and the visibility and audibility of its television and radio programmes. Accordingly, it is not too surprising that the university acquitted itself very well when formal external quality assessment and quality audit were introduced. Also, in this case study we have argued that given the mission of the university and its approach to both the curriculum and to student support, then quality assurance has always been a necessity for us and that the main development in the last few years has been the communication of the associated processes to external bodies.

The approach the Open University has taken has been to create a small number of new staff roles, such as those held by the authors, with a primary focus on quality assurance and a quality assurance panel that takes a university wide view of this area. Our current concern is to maintain a healthy culture of quality enhancement in the face of resource constraints and the negative side-effects of external quality assessment and audit. While our mission is the secure foundation for applying the principles of quality assurance, we argue that there is a real risk that external assessment processes, especially coupled with institutional success in these arenas, will lead to a stifling of innovation in such areas as interdisciplinary provision and new models of student support and might even result in the creation of bureaucratic procedures that attempt to over-standardise on the perceived critical minimum components that underpinned the past success. In the face of these pressures we need to provide the space in which small and large scale quality enhancement experiments can take place.

Concluding Remarks

If genuine institutional learning is to occur, then a reasonable proportion of these experiments must fail. So we must take calculated risks and at the same time we ensure that we properly document the lessons we learn from both the failures and the successes of different variations on our approaches to supported open learning. These lessons must be shared with the widest possible audience within the university. In tandem we must encourage the development of local groups concerned with quality enhancement. These may be focused on an area of the curriculum, a geographical region in which we operate, or an aspect of the way we support learners. In conclusion, then, we consider that if we are to continue to work towards the Open University's historic mission, then we must support, promote, and gather feedback from quality enhancement driven experimentation, innovation, and diversity in both the curriculum and in learner support.

References

Beecher, T. 1989. Academic Tribes and Territories. Buckingham: Society for Research in Higher Education and Open University Press.
Colling, C., and L. Harvey. 1995. "Quality Control, Assurance and Assessment: The Link to Continuous Improvement", Quality Assurance in Higher Education, 3 (4):30-34.
O'Shea, T., S. Bearman, and A. Downes. 1996. "Quality Assurance and Assessment in Distance Education". In R. Mills and A. Tait (eds.), Supporting the Learner in Open and Distance Learning, 193-205. London: Pitman Publishing.
Perry, W. 1976. Open University: A Personal Account of the First Vice-Chancellor. Cambridge: The Open University Press.

 Quality assurance in the provision of library services in British Columbia 

By Kate Seaborne

Introduction

The University of Victoria offers undergraduate and graduate level distance education courses and degree completion programmes in child and youth care, education, nursing, public administration, and social work for professionals in British Columbia and other western Canadian provinces. Fiscal restraint is forcing a re-examination of the well-established models for the delivery of these programmes. Concerns have commercial on-line data bases; and charges for printing information brochures and administrative forms; as well as travel costs associated with visits to off-campus sites.

By 1990, as concerns about the rising costs of the library service grew, a plan was developed to implement fee-paying services for non-university clients. A set of unit charges was developed for various services; for example, the loan of a monograph from the university libraries, the provision of a copy of a periodical article, a reference, or a literature search.

Since 1990, INFOLINE has negotiated a number of contracts with outside clients, including significant relationships with the provincial government's Ministry of Education and North Island College on Vancouver Island. As anticipated, these contracts have proven to be very beneficial to the university and to the INFOLINE service. The contract revenues have cross-subsidised the traditional services, thereby reducing the university's contribution to INFOLINE's non-salary operating budget. Two-thirds of the non-salary costs of providing the service have, since 1990, been covered by revenue generated from the external contracts and the remaining one-third has been provided through a grant from Continuing Studies. They ensure that INFOLINE's budget can be sustained at a level which enables the unit to continue providing full service to distant learners. In addition, the contract librarian and library assistants who are hired to deal with the new clients provide assistance with regular INFOLINE business as their time permits. Unfortunately, in 1995, the Ministry of Education notified INFOLINE that planned cutbacks in its internal programmes would result in a reduced budget for the library services contract. INFOLINE is also anticipating that the North Island College contract may be reduced as the college increases its permanent collection of library holdings. It is clear that cutbacks in these service contracts will affect the unit's ability to continue to provide the same level of service to distant students. Forecasts of cutbacks in external contracts, coupled with parallel reductions in funding by the Division of Continuing Studies, have forced senior administrators to begin to ask hard questions about the cost and the significance (to students) of a number of services, including INFOLINE.

Quality Assurance at INFOLINE

Library Services

On the face of it, INFOLINE Library Services appears to have provided an innovative and widely appreciated set of services since its inception in 1980. But what procedures and systems did it have in place to provide assurances of service quality?

Documentary data provided a context in which to ground a study of INFOLINE's approach to quality assurance. The co-ordinator of INFOLINE has published several reports describing its service (1993, 1995). They provide historical details and statistical data on the volume of the service. A strategic plan, developed by the staff as part of the Division of Continuing Education's strategic planning process in 1991 and updated in 1995, defines the unit's values and goals. Brochures, request forms, and other informational documents illustrate the ways in which the service is described to users. Surveys of users and interviews with students provided a range of clients' perspectives of the service.
Semi-structured group discussions with the INFOLINE staff provided the opportunity to explore their perspectives on the nature of their service and on elements of quality assurance. In co-operation with the INFOLINE director, I used Robinson's (1994) framework for assessing a quality assurance system to develop a set of interrelated questions to guide four hours of discussion. T

The following questions were posed at two meetings in December 1995:

· Does INFOLINE have an internal policy on quality service? Has this policy been translated

  into a plan? 

· Has INFOLINE set standards for the delivery of library service? Are all the INFOLINE staff

  aware of these standards? Are these standards achievable? reasonable? measurable? Are

  the users (learners, faculty, and external clients) aware of the standards?

· Has INFOLINE identified the key procedures that need to be in place to achieve the

  standards set for library service? Is the learner the starting point for some of them?

· Are the procedures for library service clearly documented? Are they readable and

  user-friendly? Does everyone who needs to know about INFOLINE's procedures have

  access to information about them?

· What kind of monitoring system has INFOLINE designed to reach its standards for library

  service delivery? Do these monitoring systems check whether standards are being met

  and procedures followed? Does monitoring result in improved performance of a review of

  practice or a reappraisal of standards?

· Have the users (learners, faculty, and external clients) been involved in setting and

  monitoring these standards?

Robinson's (1994) framework for examining quality assurance systems includes several additional elements that seemed relevant only to organisations already explicitly committed to the operation of a quality assurance system. For example, the framework recommends that an organisation take into account the degree to which staff are involved in the development of such a system, the type of training provided to staff and how that training is linked to quality assurance, and the costs of implementing and maintaining quality assurance activities. Since INFOLINE has not explicitly adopted a quality assurance system, these elements were not included in the discussion.

The unit's perceptions of how it functions to provide library support to distant learners and external clients is summarised below. Elements of what Robinson (1994) describes as a quality assurance system that have been incorporated into INFOLINE's approach to the delivery of library services are identified. All quotations are drawn from the transcripts of the interviews.

A Quality Service Policy and Plan

Organisations with a quality assurance system will have a policy on quality with which all staff are familiar and will have translated this policy into a practical plan (Robinson 1994).
INFOLINE does not have an explicit, formal policy for "quality service" or a practical plan to guide delivery of service but, based on the staff's collective experiences in dealing with clients of the university library, they share the belief that users want:

· rapid delivery of requested books and other resources; 

· a sufficient amount of material or information to be useful to the user; and

· a quick response to the user on the status of the request, if we are going to have any difficulty in filling it.

Although the unit lacks a formal policy and plan for providing quality service, they say that the goal statements formulated for their strategic plan reflect their implicit policy on quality and provide the framework for their plan of service delivery:

Goals

· To maintain the quality of the distance education programmes by providing or facilitating access to

  appropriate library resources as required.

 

· To provide all users with effective mechanisms to request and receive library materials.

· To enable [University of Victoria] distance students to receive library services comparable to those

  provided to regular students on-campus.

· To provide a level of service that will encourage external clients to continue their contractual

  arrangements with INFOLINE for library services.

· To monitor the library services provided to distance students and fee-based clients and to utilise formal

  and informal evaluation procedures to determine the effectiveness of specific services and the need for

  any change or enhancement in service level.

(Continuing Studies Library Service: Strategic Plan, 1995)
If the unit were to adopt a formal quality assurance system, it could reformulate these components into policy and planning statements.

Standards for Service
An organisation with a quality assurance system specifies and defines reasonable, achievable, and measurable standards for key activities and procedures and communicates them to everyone concerned (Robinson 1994).

INFOLINE has not developed formal standards for service, but staff describe the value statements in their strategic plan as the equivalent of a set of standards:

Values
1. Developing and maintaining credibility with all users by providing prompt and responsive service:

· requests for library materials from distance students and fee-based clients are filled as quickly as possible;
· all enquiries for library information or materials are acknowledged and given thorough attention;
· individual users are always informed of the status of their requests;
· requests from individual users take priority over other duties and tasks.

2. Taking a proactive approach to service:

· library needs are anticipated and mechanisms are developed to fill those needs as required; 
(Continuing Studies Library Service: Strategic Plan, 1995)

The staff believe that their informal standards for practice (as described in the value statements) are reasonable and achievable under normal conditions. When they encounter problems in achieving them, they tend to perceive the problem to be outside the span of their control:

whether we can supply what they want depends on whether we own it, whether it is on the shelf or whether we can get it through the inter-library loan office. And as much as we can hustle at this end, whether we can deliver in good time, in the end, depends on how quickly other library staff wrap the materials for mailing and on the effectiveness of the priority post service. (Staff interviews 1995)

On the other hand, they are not sure how these standards could be easily measured. They recognised that an examination of service delivery cannot be separated from the individuals who take part in the transaction, in this case, the INFOLINE staff and their clients. They also realise that each transaction is unique and is likely to be judged by the individual client's perception of service quality.

INFOLINE communicates its service standards to its student clients through brochures and library services information that is incorporated in course materials. Fee-paying clients are informed of these standards via intra-agency memos and other documentation. This example illustrates how the standard of timeliness is addressed in the brochure for student users:

The INFOLINE staff attempt to provide as prompt service as possible. In most cases, your request will be handled with 48 hours of receipt. …The average time to receive material from INFOLINE is one week. … there may be delays due to the mails or the unavailability of certain items. If you have not had a response from INFOLINE within two weeks of making your request, please call 1-800-563-9494 to check on the status of your requests. (INFOLINE brochure 1995)

The unit has also produced a video for student users, which they hope will dramatise its service standards through a case study of a distant student using INFOLINE to help her research an assignment. The video introduces the library staff and illustrates how requests are handled:

The idea for the video has been developing since we saw from a couple of surveys that new students feel quite shy about requesting library assistance. We hope it will demystify the process of requesting library material so that they will feel very comfortable about calling in a request.

The video will be distributed to new learners in a variety of ways: as part of an introductory package of information from the individual programme areas, as a component of courses with a library assignment, and through INFOLINE on a loan basis.

If INFOLINE chooses to implement a formal quality assurance system, it will need to develop a set of standards that specify and define how library service will be delivered. The first step they would take would be to diagram their procedures for delivering library service. A table or chart helps to depict the complete set of procedures that need to be managed (Robinson 1994; Lovelock 1992). Working from the diagram, the staff can develop specifications and definitions for standards of delivery which they believe are reasonable and achievable given their working situation, and which they also believe can be measured. For example, a standard for service to distance education students might read: "a student's request for library materials will be moved through the INFOLINE office in 48 hours from the date of request, or the student will be contacted". As noted earlier, the INFOLINE staff have indicated that, under normal conditions, they can turn around a student's request within 48 hours of receipt, so this example would be perceived as reasonable and achievable. This standard could also be measured by having staff keep details of each transaction using a management tool such as a record of activity. This tool permits them to measure themselves against the standards they have set for library service and develop their own plans for self-improvement, if required.

Key Functions and Procedures Required for Service Delivery

An organisation with a quality assurance system has identified and analysed the key functions and procedures required to achieve the standards it has set and has established the clients of the service as the starting point in designing at least some of these functions and procedures (Robinson 1994).
Through its strategic planning process, INFOLINE has done a thorough, though informal, job of identifying and analysing its functions and the procedures required to achieve the standards it has set for itself. The key functions are identified in its strategic plan's mandate statement. These are to provide
library support for off-campus and distance credit, certificate and diploma programs sponsored by the Division of Continuing Studies and … fee-based library services for external organizations, institutions and individuals.

INFOLINE's procedures are based on their assumptions about the user's needs, their collective experience in the delivery of library services, and the values they have articulated in their strategic plan. To a considerable extent, the needs of the clients have been taken as the starting point in identifying and analysing the unit's functions and procedures. The staff believe that the procedures they have designed for service delivery reflect an accurate analysis of user's needs for library service. INFOLINE's procedures for service are summarised below.

The telephone is still the principal medium of access to the INFOLINE staff but, over time, the means of access have been expanded to include regular mail, fax transmission and, more recently, electronic mail. Requests received by telephone are recorded on an answering machine in response to a message on the machine that instructs the user to list the library material they require or to describe the type of information needed for research purposes. A library assistant transcribes the telephone messages each working day. Users are only called back if there is a query about their requests. The requests for specific titles are handled by a library assistant, and requests for reference assistance or literature searches are passed on to a librarian. Fax and e-mail requests are passed on to the library assistant or a librarian, depending upon the type of request. As noted earlier, the unit's objective is to successfully address every request within a 48-hour period. Most material is sent by mail or private courier directly to the student's home address. Small quantities of articles are occasionally sent to students by fax.

Any circulating book in the University of Victoria libraries can be loaned to users. Loan periods are the same as those given to on-campus users. INFOLINE also has a large office collection of uncatalogued materials to support the distance programmes, including duplicate copies of books, articles, and audio-visual items designated as supplementary materials for the various courses. Items from this collection are sent on short-term loan to individual distance students as required. Periodical articles are copied on demand for users from the library collections. Interlibrary loan requests are placed on their behalf when necessary.

In response to requests for reference or subject assistance, literature searches are conducted using the appropriate CD-ROM or on-line databases. Due to the short deadlines in most undergraduate courses, the librarian usually selects materials from the search results to be sent to the distant student. Printouts from the database searches are sent directly to graduate students and provincial government clients so that they can select their own references.

The staff believe that most of the procedural problems they experience lie outside the scope of the unit's responsibility. For example, they describe typical student complaints as revolving
around seeing books on recommended reading lists and not being able to get them from us. The typical situation is that students will see a book and call for it and five other students are on the list to get the book ahead of them and there are only two copies. And that's not our fault. The program area is really at fault here because they are responsible for providing us with additional copies of any books (one for every 10 students registered) that they put on the reading list.

In this case, INFOLINE has developed a procedure to deal with the situation but depends on the academic area to ensure that it is followed and the staff must bear the brunt of student dissatisfaction if it is not. An analysis of this problem from a quality assurance perspective might lead the staff to redesign the procedure they have established with the academic units to deal with the provision of multiple copies of recommended readings.

Documentation of Procedures

An organisation with a quality assurance system in place will have documented its procedures for delivery of service. The documentation will be clear and explicit in its description of procedures, it will represent practice, and the information will be presented in a readable and user-friendly format (Robinson 1994).
The INFOLINE staff are currently developing their first procedures manual as a reference resource. They recognise that, while the current staff is familiar with procedures, it would be very difficult to train new staff without a written discussion of procedures for reference. However, they point out that the procedures manual can be only a guide because so many of the decisions they make about trying to address learners' requests are subjective: only from years and years of experience can you learn when you're not providing sufficient materials. It is not that six items are not sufficient and seven are. We use judgments every day in what we provide. We have no hard and fast procedures.

If INFOLINE were to adopt a formal quality assurance system, it would need to design a procedures manual that clearly defined the unit's standards for service delivery and described how these would be addressed and measured.

Monitoring Service Quality

An organisation with a quality assurance system in place has set up systematic monitoring mechanisms to check whether standards are being met and procedures followed. The data collected via the monitoring mechanisms are disseminated to everyone concerned and are used to improve performance, to review practice, or to reassess current standards (Robinson 1994).

INFOLINE does not have a regular monitoring system in place. At present, the only formal monitoring is done with those users who request reference assistance (for example, information about a specific topic). Forms that ask for an assessment of the reference service are sent out with the material and the onus is on the users to return them. Apart from this one monitoring mechanism, the unit relies on users' unsolicited comments on the basic library service (for example, requests for books or articles that are listed as supplementary reading in the course materials) to determine whether they are achieving the informal standards they have set for service delivery:

we assume that if we've given the users what they've been looking for then the only variable is time. Did they get it in a timely manner so that they could use it? …We assume that the user will get back to us if the request is late or if they didn't get what they asked for. And over the years, the number of people who have complained about not having received materials is very small. We do get delays because of the mail service but that seems to be the most troublesome issue for them.
Unsolicited comments come to them in two ways: either directly to the INFOLINE office by note or phone call or indirectly through learners' comments on evaluations of distance education courses and the services that support course delivery.

In 1993, INFOLINE surveyed its student users about the effectiveness of the delivery procedures and found that, on the whole, learners were very satisfied with the way their requests were handled. Student reviews of the INFOLINE service collected for other studies of the University of Victoria's distance education programmes bear out these findings (Seaborne 1995; Brown and Molzahn 1994). A user satisfaction study conducted with employees of the Ministry of Education determined that there were high levels of satisfaction with the contracted services provided by INFOLINE (Ministry of Education 1995).
The unit believes that it would do a more comprehensive job of monitoring the quality of their service delivery if it had more time and more staff. If INFOLINE were to adopt a quality assurance system, it would need to attach monitoring mechanisms to all its procedures and add responsibilities for data collection and analysis to existing job descriptions. The staff would also need to hold regular sessions in which they could review the collected data and use it to assess standards, procedures, and practice.

Involvement of users

An organisation with a formal quality assurance system involves the staff and the clients in setting and monitoring standards for service delivery. Staff are also involved as "internal clients" in setting and monitoring standards for their work environment (Robinson 1994).

Through the strategic planning process, INFOLINE staff have been involved in identifying the values and goals that define the library service and in developing the procedures that guide its practice. In this way, the staff are implicitly involved in setting standards for service delivery and in creating a work environment that supports their beliefs about practice. Clients have not been directly involved in these processes.
If INFOLINE were to adopt a formal quality assurance system, it would need to find strategies for incorporating the views of its clients in developing standards, which could prove challenging for a service unit dealing with clients at a distance. Interactive communications technologies such as teleconferencing or computer conferencing, however, would provide a means by which clients and INFOLINE staff could come together in focus group discussions. These sessions would enable staff to increase their understanding of clients' needs, test assumptions about existing standards, and solicit the clients' views about service quality.

Concluding Remarks

Any organisation that focuses on service quality is likely to have put in place, implicitly, elements of a quality assurance system, as was clearly the case with INFOLINE. The unit has spent considerable time identifying and analysing the procedures required to achieve the standards set out in its strategic plan. And although the unit has not, to date, regularly monitored all procedures to ensure that standards are being met, the solicited and unsolicited feedback it does receive suggests that, overall, users are very satisfied with the quality of service they receive from INFOLINE. The staff discussions about quality assurance systems also illustrate the unit's collective willingness to turn the process of reviewing standards and procedures into an opportunity to look critically at what they do. In a discussion about procedures, for example, they focused on a routine that they have established with academic programmes to supply additional copies of books on recommended reading lists. They rely on the programmes to bring these lists to their attention and to provide required copies and, frequently, the programmes neglect to provide the materials until students call for them. Discussion of this matter has caused the unit to reassess the procedure and to open discussions with the programmes on how to improve communications on this matter. Equally as clear, an organisation like INFOLINE could divert significant resources to the implementation of a full-blown system of quality assurance, with consequent costs and benefits. On the benefit side, a comprehensive monitoring of standards and procedures would increase communication between the unit and its clients as well as prompt regular reviews of practice and reassessment of standards. On the cost side, implementation of a quality assurance system would require a significant reorganisation of the staff's roles and responsibilities in order to incorporate monitoring tasks. Given the small number of staff and the volume of activity, they would be hard pressed to add additional monitoring tasks while maintaining their current standards of service.

References

Brown, M., and A. Molzahn. June 1994. Survey of the Satisfaction of Bachelor's of Science (Nursing) Students with Distance Education. Victoria, British Columbia: School of Nursing, University of Victoria.
Continuing Studies Library Services. 1995. Strategic Plan. Victoria, British Columbia: Division of Continuing Studies, University of Victoria.
Cowan, J. 1994. "How can you assure quality in my support, as a distance learner?" Open Learning, 9 ( 1): 59-62.
Croft, M. 1991. Student Support Services: An Overview. Report of the Roundtable on Student Support Services. Vancouver: The Commonwealth of Learning.
Deshpande, P. M., and I. Mugridge (eds.). 1994. Quality Assurance in Higher Education: Papers presented to a symposium on quality assurance, New Delhi. Vancouver: The Commonwealth of Learning.
Freeman, R. 1991. "Quality Assurance in Learning Materials Production," Open Learning, 6 (3): 24-31. 
INFOLINE Services. September 1995. INFOLINE. Brochure. Victoria, British Columbia: Continuing Studies Library Service, Division of Continuing Studies, University of Victoria.
INFOLINE Services. December 1995. Staff interview. Victoria, British Columbia: Distance Education Services, Division of Continuing Studies, University of Victoria.
Inter-University Library Survey of Off-Campus Students in Western Canada. July 1993. Summary of Results. Victoria, British Columbia: Continuing Studies Library Service, Division of Continuing Studies, University of Victoria.
Lovelock, C. 1992. Managing Services, 2nd ed. Englewood Cliffs: NJ: Prentice Hall.
Planning, Research, and Evaluation Branch. 1995. Report of Findings: INFOLINE Survey. Internal report. Victoria, British Columbia: Ministry of Education.
Moran, L. 1995. "Who Sets the Agenda for Quality in Distance Education?" In D. Sewart (ed.), One World, Many Voices: Quality in Open and Distance Learning. Selected papers from the Seventeeth World Conference of the International Council for Distance Education, held in Birmingham, United Kingdom, June 1995, 88: 158-161. Milton Keynes: The Open University.
Nunan, T., and J. Calvert. 1992. Report of the Project to Investigate Quality and Standards in Distance Education. Victoria, South Australia: University of South Australia, Deakin University.
Robinson, B. 1994. "Assuring Quality in Open and Distance Learning". In F. Lockwood (ed.), Materials Production in Open and Distance Learning. London: Paul Chapman.
Seaborne, K. 1995. A Case Study of Student Support Services for Distance-Based Professional Education. Dissertation in progress.
Slade, A. L. 1993. "Funding Off-campus Library Services Through Alternative Sources: Expanding the Infrastructure to Include Fee-paying Clients.'' In The Sixth Off-Campus Library Services Conference Proceedings. Mount Pleasant, MI: Central Michigan University.
Slade, A. L. 1989. "Establishing an Off-campus Library Service for Remote Educational Centers: Variables and Potentials". In B. M. Lessin (ed.), The 
Off-Campus Library Services Conference Proceedings, Reno, Nevada, October 20-21, 1988. Mount Pleasant, MI: Central Michigan University Press.
Slade, A. L. 1987. "Library Services for Distance Education Courses". In B. M. Lessin (ed.), The Off-Campus Library Services Conference Proceedings, Reno, Nevada, October 20-21, 1986. Mount Pleasant, MI: Central Michigan University Press.
Statistics Canada. 1995. Serving Canadians: Survey of Practices in Support of Quality Services in the Federal Public Service of Canada. Ottawa: Queens Printer.
Warren, J., K. McManus, and R. Nnazor. 1994. "Quality Assurance and Distance Education: A Review of the Literature". In P. M. Deshpande and
I. Mugridge (eds.), Quality Assurance in Higher Education: Papers presented to a symposium on quality assurance, New Delhi. Vancouver: The Commonwealth of Learning. 

 Practice exercise 

Discussing the case studies

Instructions: Divide participants into four small groups. Assign one case study to each group. Ask each group to prepare a 15-minute presentation of the case study, to be made to the workshop as a whole, using the following questions as guidelines for their presentation: 

· What are the primary features of the context, political, economic, sociocultural, and institutional, in    

  which the quality assurance schemes are operating?

· How do the authors of the case study define quality assurance?

· What issues are the institution or institutions described in the case study having to deal with in their   

  quality assurance schemes?

· What methods and mechanisms are they employing in their quality assurance schemes?

· Have they been successful? If so, why? If not, what problems remain to be solved?

Timeframe: Each presentation should take about 15 minutes. Participants ideally should work on this task over the duration of the workshop. Alternately you will need to provide at least one half-day for the groups to prepare their presentations.

Materials required: Flip charts or overhead transparencies and marker pens.