Appendix 5

Quality Assurance in Distance Learning

According to the Department of Education’s policy document Norms and Standards for Educators, quality assurance involves establishing and maintaining self-improving processes and systems in an institution or programme. One measure of an institution’s capacity to maintain educational quality is its ability to anticipate and address problems rather than merely react to them. A second is that quality assurance processes should involve everyone in the organisation. However, all processes must be shown to be contributing to the educational effectiveness of the institution. 

Quality assurance systems have the following essential purposes: 

·     To assist institutions to maintain and enhance the quality of educational provision by reviewing their quality assessment systems and processes, and confirming that their internal arrangements reflect good practice;

·     To assure everyone connected with the institution namely learners, parents, would -be employers, funding agencies, and society in general about the accountability and responsibility of the institution for its own quality and probity;

·     To furnish the ETQA with a general basis for evaluating whether quality assurance in the programme’s learning area is satisfactory;

·     To ensure continuous improvement in the entire operations of the institution;

·     To provide assurance to the public regarding the achievement of the required quality standards; and

·     To assist in institutional and systemic reform efforts.  

To accomplish these, each provider must evolve mechanisms and procedures for ensuring:

·        timely, efficient and progressive performance of academic and administrative tasks;

·        the relevance and quality of the academic programmes;

·        the maximisation and integrity of teaching and learning opportunities;

·        the credibility of evaluation procedures;

·        appropriate evaluation criteria; and

·        the adequacy, maintenance and proper allocation of support structure and services. 

Quality is, however, a highly contested concept. People in different contexts have different understandings of what is meant by quality. For example, the following dimensions have been associated with the meaning of quality: 

·     quality as exceptional, i.e. as something special, distinctive, in some cases linked to notions of excellence;

·     quality as perfection, i.e. as consistent or flawless outcome;

·     quality as value for money, i.e. quality in terms of return on investment or cost-effectiveness;

·     quality as transformation, i.e. in terms of enhancement and empowerment of students, practitioners and system managers, and development of knowledge; and

·     quality as an attainment of standards. 

A conceptualisation of quality which embraces both generic aspects of the educational provision and context specific ones is advocated in the White Paper 3: A Programme for the Transformation of Higher Education (1997: 12).  It refers to the principle of quality as: 

‘... maintaining and applying academic and educational standards, both in the sense of specific expectations and requirements that should be complied with and in the sense of excellence that should be aimed at. These expectations and ideals can differ from context to context, partly depending on the specific purposes pursued. Applying the principle of quality entails evaluating services and products against set standards, with a view to improvement, renewal or progress.’ 

A workable definition of quality within this framework is the notion of quality as ‘fitness for purpose’, i.e. the ability of an institution to fulfil its mission, programme, aims and objectives, determined on the basis of clients’ needs and with reference to specified desired outcomes.  

Most importantly, purpose is the organising principle with reference to which learning areas must be identified, learning programmes and qualifications packaged, and assessment criteria defined. 

Quality assurance is a cyclical process in which an institution should strive for continuous improvement. 

The review cycle 

The provider would need to put in place an internal review system with the following characteristics: 

Defining purpose

The purpose of this phase of the quality assurance process is to come to a common understanding, element by element, of the purpose of implementing a quality assurance strategy in a particular institution or programme. This is not simply a technical exercise. It needs to be designed in such a way that it enables staff (probably only at middle and senior management level at this stage) to come to terms with the very notion of quality assurance and to accept it as a process essential to improving quality. This commitment will be vital as the process continues. 

Planning

The purpose of this phase is to develop plans for phases 3, 4, and 5 of the quality assurance strategy. Again, this is not simply a set of technical activities, particularly if one is running the process for the first time.

It is the point at which all staff members need to be introduced to the quality assurance strategy and convinced of its importance. Here are some useful questions to guide planning and ongoing internal evaluation:

·        What are you trying to do?

·        Why are you trying to do it?

·        How are you planning to do it?

·        Why are you trying to do it that way?

·        Why do you think that is the best way of doing it?

·        How do you know that it works?

·        How do you improve it? 

The planning may also entail: 

·        Defining a set of criteria and performance indicators on the basis of which to measure improvement. These should focus on academic, professional and occupational competences rather than inputs or content taught. A distinction should be made between primary performance indicators (assessed on an ongoing basis) and contributory indicators (assessed randomly). Primary performance indicators are linked to issues such as educational and quality assurance processes, success ratios, and post-programme success and employer satisfaction. Contributory performance indicators are linked to issues such as staff satisfaction and development, staff/learner ratios, learner progress, attendance and counselling. 

·        Defining the process of assessment of institutional strengths and weaknesses. 

·        Identifying areas to be audited and the assistance required from the ETQA to meet its organisational development targets. These may include: (i) functions (like teaching/learning/research, community service); (ii) services (staff and student services; financial services; academic support services); (iii) aspects of functions and services (governance, access, programme planning, funding, financial aid); and (iv) issues such as transformation, diversity, language policy, and affirmative action. 

·        Identifying members of the institution to be involved, and defining how they should be involved and their capacity developed. 

·        Describing the processes for gathering and processing evidence to be used in self-evaluation. 

Gathering information

All necessary information is gathered, mostly through various forms of primary research during this phase. In order to run smoothly and to ensure maximum benefit, this phase should involve all staff in some way or another. It should also incorporate professional development activities designed to introduce staff to new ideas in the field in which they work. This is essential to allow them to assess their work from a new perspective and gain insight into its strengths and weaknesses. This is also the phase during which student responses, peer evaluations and external evaluations are most likely to be sought (although they need not be obtained exclusively during this phase). 

Making judgements

During this phase, identified staff members are expected to consider the various types of information gathered to make judgements about the effectiveness of the particular institutional/programme element under consideration. This information may include for example student questionnaires, internally and externally conducted evaluations, and relevant statistical information.  

Reporting

Having made judgements, the staff members responsible are expected to compile brief reports detailing the strengths and weaknesses of different aspects of a particular institutional/programme element. Importantly, these reports should identify what action needs to be taken to build on the strengths and eliminate the weaknesses detailed in the report. Staff should not do this in isolation; the process of identifying appropriate action to be taken should include all staff at appropriate moments. The report should also indicate how that action would be monitored.  

The report should include at least an introduction (description of quality systems in place); analysis and evaluation (What did we set out to achieve? Have we succeeded? How do we know? Strengths and weaknesses? What will we do next? How will we evaluate this?) and a conclusion (reflection on progress from previous year; suggestions for external review).  

It should be assessed by peers and, if applicable, seconded to the ETQA for purposes of: (i) guiding the external review; and (ii) making decisions about emergency interventions as part of assessing internal capacity of the institution. Important indicators for evaluating the report are:

·           Internal motivation;

·           Commitment of  management and staff;

·           Clear procedures;

·           Combination of both qualitative and quantitative performance indicators, including evidence of a committee leading the process and evidence of ongoing high quality reports. 

Taking action

In this phase, appropriate action is taken to build on the strengths and eliminate the weaknesses of the institution or programme. Of course, in certain cases, staff will start to act as soon as they have identified certain weaknesses or gaps in their work. Thus, the neat chronological order of phases suggested here will be blurred (as it will during phases 1 and 2 and during phases 3, 4, and 5). This is not a problem as long as the progress of each phase is carefully monitored by people assigned to oversee the quality assurance process. It is important to set organisational development targets, processes, and an explanation as to how these will bridge the gap between goals set and institutional capacity. Crucial to making this work is the development of a clear mission statement or vision for the institution followed by equally clear and concrete action plans. While the former is best described as ‘macro scenario sketching’ – for example, ‘what is likely to happen in this field in the next five years’ - the latter must be concrete, ‘do-able’ and contain a number of measurable indicators. 

Monitoring action

The point of this phase is to monitor how effective actions, suggested during the reporting phase and subsequently undertaken, have been in building on the strengths and eliminating the weaknesses of the institution/programme. Thus, it functions as a form of summative evaluation of the entire quality assurance process and is another clear moment at which external evaluation and learner input can be valuable. Although the monitoring process functions as a summative evaluation of the quality assurance process, it should not be seen as a retrospective process. Rather, it can lead to immediate changes in the actions being taken and, more importantly, to re-thinking the quality assurance strategy before the cycle begins again. For this reason, the monitoring action phase is the point at which the planning worksheets for the ‘Defining Purpose’ and ‘Planning’ phases of a new quality assurance strategy will be completed, allowing the process to begin again. 

Re-defining vision, re-planning

As indicated earlier, good quality assurance processes are circular; the end becomes a new beginning. It is, however, important to reiterate that this need not occur in the neat order suggested. It should be an organic process where the institution is able to fairly quickly respond to ‘gaps’ in its quality or to changes in larger, societal needs.

This process is summarised in the following diagram:

 

Reporting on a Commonwealth of Learning workshop on quality assurance in distance education held in Lusaka, Welch (1998), presents the following cyclical model for programme quality assurance in the distance education context: 

 

Central features of the quality assurance method outlined above are:

·     Quality assurance is related to planning and can be reviewed in a cyclical process in a similar way to planning. The cyclical process consists of curriculum planning (or programme planning in NQF terms), admissions/access, delivery (teaching and learning strategies), assessment of programmes (self-evaluation part of the cycle) and revision and improvement.

·     Strategic quality management – selection of aspects of strategic importance for quality assurance – is more important in the African context than total quality management.

·     Rather than specifying beforehand the necessary aspects the quality of which needs to be quality assured, these aspects can be r\derived referred to in order to check for possible critical gaps.

·     When it comes to the process of assessment of programmes/courses, there is a need for thorough self-evaluation, but also for an external review that goes beyond merely the process of external moderation. 

As the programme planning and quality assurance is being carried out, whole institutional structures need to be put in place to support programme quality assurance. It is extremely difficult to sustain quality in an inhospitable institutional environment. 

A useful way to get an understanding of effective institutional quality assurance structures is to look at case studies of credible institutions, for example the Open University in the United Kingdom (UKOU). Besides the sine qua non of a missions statement directed towards the improvement of student learning, selected examples of institutional QA lessons taken from the OKOU include:

·     Monitoring, feedback and review is a complex process probably best carried out by a separate research unit in the institution (in the case of the OKOU, the Institute for Educational Technology).

·     There needs to be special institutional provision made for overall control of the quality assurance system (in the case of the OKOU, there is a pro-Vice Chancellor with responsibility for institution wide quality assurance). Quality assurance is not seen as an add on extra, but as an integral and valued part of all work done in the institution.

·     A strong staff development policy, supported by sound staff appraisal systems, is essential for successful quality assurance.

·     Self-evaluation needs to be firmly established before external evaluation makes its demands, otherwise external evaluation can influence self-evaluation negatively.

·     A team approach to programme and course design, development and delivery fosters the necessary ownership and contributes to quality. 

Tait (1995) provides the following useful key questions which need to be constantly returned to in the quality assurance process.

                                              Who are your learners?

 

              How will you evaluate?                                                         What are their needs?

    

         How much will services cost?                                        How will you meet their needs?

                  

                                                    How will the services be managed?

 It is important to bear in mind that this is not a once-off process. It is likely that the learner profile will change over time and hence so will the needs of learners; equally, the learners and their needs are likely to change if one tries to replicate the programme in another context, e.g. the UOFS: BML programme which is currently offered in Bloemfontein may in time be offered in the USA. Therefore, all the key underpinning questions and assumptions need to be interrogated with each major transition from year to year or from location to location, as indicated below.

Year Two

How will the services now be managed?

 

How will you meet their (changed) needs?                      How much will services cost now?

                                                                                                             

                                              

                 Have their needs changed?                                            How will you evaluate?

                    

Year One

                                       Who are your learners? Have they changed?

                                                   Who are your learners?

 

 How will you evaluate?                                                              What are their needs?

 

 How much will services cost?                                              How will you meet their needs?

    How will the services be managed?