Introduction
1 This is the report of an audit of the quality assurance arrangements for a
partnership between the University of Sunderland and the Micro Center Institute,
Bahrain (the Micro Center). The partnership was entered into for the purpose
of offering a taught postgraduate programme of study, by distance learning, leading
to the University's award of MSc Computer Based Information Systems (MSc CBIS),
together with interim awards of Postgraduate Certificate and Postgraduate Diploma.
2 The audit was carried out by the Quality Assurance Agency for Higher Education (QAA). The audit visit was part of a series undertaken in the Gulf States in October 1997. The quality assurance arrangements of the University, as a whole, were considered at an audit visit by the Higher Education Quality Council (HEQC) in November 1994, leading to a report published in September 1995, and by a further audit of collaborative activity, conducted in 1995 and reported in May 1996.
3 A general description of the University's procedures for the assurance of quality of its collaborative teaching and learning provision is contained in the HEQC's 1996 Report. The present audit of the partnership arrangements between the University of Sunderland and the Micro Center used the 1996 Report as a major source of reference for detailed examination of the policies and procedures used by the University to satisfy itself of the academic quality and standards of the award being offered to distance learning students based in Bahrain.
4 The Agency is grateful to the University and to the Micro Center for
their assistance and co-operation.
Abbreviations used in the report
5 The following abbreviations have been used throughout this report:
1995 Report
- Quality Audit Report: University of Sunderland HEQC, September
1995;
1996 Report
- Quality Audit Report: University of Sunderland Collaborative Provision,
HEQC, May 1996
The University
- The University of Sunderland
Pro Vice-Chancellor
- Pro Vice-Chancellor (Academic Programmes and Quality)
The School - The School of Computing and Information Systems
HEQC
- The Higher Education Quality Council
CBIS - Computer Based Information Systems
QAA - The Quality Assurance Agency for Higher Education
The audit process
6 Prior to the audit visit, the University submitted documentation outlining
its aims and implementation strategies in relation to the collaborative provision
it has established with the Micro Center. This briefing material was mainly drawn
from existing papers. Following its reading of the briefing documentation, the
audit team proposed a programme for its visit to the University. It also sought
additional contextual materials to help it to understand and verify the structure
and processes of this aspect of the University's collaborative quality assurance
arrangements.
7 To prepare for the visit to the Micro Center in Bahrain, the audit team visited the University on 22 July 1997 and held discussions with individuals and groups including the Pro Vice-Chancellor (Academic Programmes and Quality); the Director of the School of Computing and Information Systems; the Director of the Centre for Distance Education in Computing; two course leaders for the MSc CBIS; the Administrator of the Centre for Distance Education in Computing; and a member of staff in the Quality Support Unit.
8 In Bahrain the audit visit took place on 14 October 1997, when the audit team met the President of the Micro Center; the Counsellor/Administrator; the Course Leader; two local tutors; and eight students. At the time of the audit visit, there were eight students in the first cohort, and 10 in the second.
9 After returning to the UK, the QAA Assistant Director clarified some matters of fact with the University by an exchange of letters.
10 The audit team comprised Professor J Cowan, Dr P A J Easy, Professor J Rear, auditors, and Dr K M Clarke, audit secretary. Dr D S Law, QAA Assistant Director, accompanied the team to Bahrain and co-ordinated the audit for the Quality Assurance Agency.
The University context for collaborative provision
11 The University's quality assurance arrangements for its collaborative provision were the subject of the audit described in the 1996 Report by HEQC. At that time, there was some debate within the University as to whether programmes of study on which students were registered at an approved centre, provided with teaching materials devised by University staff and supported by local tutors, should be considered to constitute collaborative activity. Reviewing the terms of this debate, the Report recommended that the University might find it advisable to include such provision within the University's models for programme quality assurance. It also suggested that the University should review its procedures to determine whether or not the mechanisms used to approve such collaborations could be further strengthened.
12 In accordance with that latter recommendation, the University's Academic Board in1996 approved detailed guidelines on International Collaborative Taught Programmes: Planning and Operational Requirements. This document provides a checklist of requirements for the guidance of 'planners and managers of international collaborative programmes'. It sets out in detail the University's requirements relating to planning, validation and approval, programme operation, monitoring and review. A noteworthy inclusion is a requirement for all new proposals for collaboration to include a risk assessment and a business plan. The University has thus taken steps to provide a framework for the introduction of new activities overseas, and has decided that any further extensions to existing programmes should be carried out under these procedures. For example, the audit team heard that the University is planning to establish a new central unit, headed by the Pro Vice-Chancellor, to enhance its capacity to co-ordinate and monitor its overseas partnerships. The team noted the willingness of the University to learn from experience and to translate that learning into revised and more demanding practices. However, since the current audit examined the extension of an existing programme to a new centre, the team was not in a position to confirm the extent to which the University's intentions had been realised.
13 The University and the School of Computing and Information Systems have had considerable experience of international collaborative associations based on supported distance education. Within the School there is a specialist Centre for Distance Education in Computing which administers the arrangements for students based at an overseas study centre. Between 1994 and 1996 three approved sites commenced delivery of the MSc CBIS, Hong Kong, Jeddah and Kuala Lumpur, before the first cohort of students commenced in Bahrain in November 1996.
14 Students registered for the MSc in Computer Based Information Systems in Bahrain undertake studies which are based on teaching materials provided by the University in pre-recorded and print form, and are supplemented by tutorials conducted by local tutors using outlines provided by School staff. The pre-recorded instructional materials are prepared specifically for distance learning by University staff in the School concerned. The School considers the materials to be equivalent in content and standard to those available to students in Sunderland. There is also some face-to-face instruction from members of University staff during their visits to the study centre and at 'summer schools'. The latter are held twice a year, and offer some 12 hours of contact over two or three days, the visitors primarily reacting to immediate learning needs.
Initial approval and validation processes
15 The MSc in Computer Based Information Systems was first approved
as a 'face-to-face' programme in 1986. In March 1993 the School put forward
a proposal to provide the established MSc CBIS at sites overseas, using
distance learning methods, with the support of approved local tutors.
The declared and accepted aim of the course team was to deliver to remote
students a comparable experience to that received by home students, with
access to appropriately qualified local tutors to answer students' questions.
16 This proposal was considered in June 1993 by a panel set up by the then Validation and Review Committee, now the Quality Enhancement Board. It was recommended for approval subject to certain conditions, which were to be met by the end of July 1993. These conditions included the production of a document setting out the administrative arrangements for the delivery of the course in an overseas institution, consideration of the need to free staffing resource in the University to support such schemes, and the updating of the academic regulations in line with the University CAT Scheme. The audit team found evidence that the first two conditions had been met, and were a feature of the working relationship with the Micro Center. The team did not investigate whether there is now full alignment with the University CAT scheme.
17 The Micro Center, which has operated in Bahrain since 1984, is primarily a computer training and software development company. As the 'Bahrain Micro Center (Manama)', it was cited to the collaborative audit team as an overseas partner (as at November 1995), and was so recorded in an appendix to the 1996 Report, although it did not appear to the current audit team that enquiries about the suitability of the facilities and staffing had at that time been completed.
18 The audit team considered the chain of approval which was followed for the collaboration with the Micro Center. In particular it was interested to learn what information had been assembled, and who had evaluated it, noting that this pre-dated the reconsideration of the responsibilities of the Quality Support Centre in the late spring of 1997. At the time when the University approved the delivery of the MSc CBIS in Bahrain, the extension to an additional overseas centre depended only on the ability of the School to demonstrate to the Pro Vice-Chancellor that adequate facilities and resources would be available at that centre.
19 Discussions between the University and the Micro Center commenced in 1995. A former member of staff at the University, in conjunction with an academic colleague who owned the Micro Center, responded to interest on the part of the University which was seeking a partner in Bahrain. The company was visited in 1995 by a Principal Lecturer from the School. In January 1996 a further visit was made, when the Principal Lecturer was accompanied by the Director of the School. The Micro Center then applied to the Ministry of Education for recognition as a training institution so that it could enter into the partnership with the University.
20 The audit team was told by the University that responsibility for negotiation of a new partnership rested with the Director of School and that, once a programme had been validated for distance learning delivery, considerable autonomy in the negotiation of extensions to new sites was normal at that time. The team was also told that the arrangement was seen as equivalent to a collaborative programme in the UK, with the result that the University followed the same quality assurance arrangements, which were largely School-based. The team was told that, in this case, the relatively brief report submitted by the Director of the School, following a visit in January 1996, was the basis for approval by the Pro Vice-Chancellor, who thereby accepted the conclusion of the report: 'the Micro Center offers a good resource centre and access to good quality staff to support the MSc CBIS programme'.
21 Following the HEQC audit of the University's arrangements for collaborative provision, the 1996 Report registered that the audit team had 'encountered some uncertainty regarding approval mechanisms' and called attention to the doubt in the University which seemed to exist at that time about precisely what authority was needed to approve 'the addition of a new overseas partner to an approved supported distance learning programme'. The current audit team learnt that at the same time as the Micro Center in Bahrain was being considered and approved using established procedures, the University was also reacting to the 1996 Report and instituting new procedures for the extension to new sites of validated programmes.
22 The audit team was assured by the University that its new procedures would be more systematic and would require 'inspection of documentation for the agency agreement by an audit team drawn from outside the School'. Whilst this is to be welcomed, the University may also wish to consider the advisability of ensuring that there is an institutional validation process, including a site visit, which includes representatives drawn from outside the School which will be delivering the programme.
Background to the partnership
23 The University requires that each collaborative partnership is
governed by a formal memorandum which sets out the arrangements governing
the operation, the monitoring and review of the programme, and the responsibilities
assigned to the University and the collaborating institution. The audit
team was therefore interested to receive and scrutinise the Memorandum
of Collaboration between the University and the Micro Center. In an agreement
dated 3 June 1996, signed for the University by the Pro Vice-Chancellor,
the details of the collaboration and the responsibilities of both partners
are set out comprehensively and in accordance with the principles to which
the University is committed. The Micro Center is authorised to market and
facilitate recruitment to the MSc CBIS, to provide local tuition and IT
facilities, to arrange pastoral support for students, and to provide administrative
support including arrangements for the issue of assignment tasks, the collection
and return of assignments, and managing the examinations, held annually
during the Diploma stage. The Micro Center is also expected to provide
local counselling, to a University remit which is detailed in the student
handbook and course guide.
24 At the University, there is a Programme Leader in charge of the entire
MSc CBIS programme. Also at the University, there is a course leader for
each distant centre, who handles matters of process, communication, and
quality for all the courses provided at that centre, and who takes responsibility
for the supervision and progress monitoring of its students. The course
leaders communicate with staff at the centres at least once a week, as
well as visiting. In addition each module is the responsibility of a module
leader. The audit team heard a clear description of these arrangements
but was not provided with formal documentation of the respective responsibilities
of staff in the School who are involved in the programme. Whilst the team
appreciates the School's point of view that those concerned are 'a small
group and talk to each other', the University may wish to consider the
advisability of more documentation of roles and responsibilities.
Development of the partnership
25 Eight students began the programme in November 1996 and, by the time of the audit visit, a second cohort of 10 had been recruited. During its visit, the audit team encountered the launching in Bahrain by the School of a second masters degree, the MSc in Management of Information Technology (MSc MIT), with an initial recruitment of 11 students who began their induction programme in late September 1997. The team also heard that there were detailed plans for further provision including doctoral work and an undergraduate degree. Prior to its audit visit to Bahrain, the team had not been made aware of any of these, nor of the possibility that increased student numbers might make it necessary for the Micro Center to move to larger premises. The University informed the team that if proposals were made to run additional programmes at a partner institution, there would need to be further approval by the Pro Vice-Chancellor, acting on advice from an internal panel. The team believes the University will wish to ensure that its own requirements for proper consideration of staffing and other resources are carefully followed.
26 The audit team noted the overlap of modules between the MSc CBIS and the MSc MIT, and found nothing in the information available to it which addressed the possibility of double counting; that is the possibility that a single module might be counted towards two academic awards. The University may wish to consider the issues of credit transfer between two such programmes, and to make its position clear in its publicity materials and regulations.
Monitoring and review arrangements
27 The Memorandum of Co-operation requires that the University's standard
arrangements for quality assurance will be followed. Under these, the School
will submit an annual report to the University's Quality Support Unit,
each report starting with an account of what has been done to rectify previous
deficiencies and to act upon previous decisions. To inform this process,
the University's procedures call for an annual programme report from partner
institutions, which should be considered by the appropriate module and
programme boards before being merged into a joint report to the School
Quality Board. As the team visited the Micro Center within its first year
of operation as a partner institution of the University, it was not able
to see a programme report.
28 The audit team heard that, since the programme began in 1996, there had been five visits by University staff to the Micro Center. In most weeks there have been several communications, generally on student-related issues, between the Micro Center and the Course Leader, who is therefore well informed of activities at the Micro Center. Provision for informal reporting during the year is thus in place and it is reasonable to expect that, in due course, formal reports will be made as required. The audit team commends the close involvement of University staff in the arrangements for monitoring the programme.
29 When in Bahrain, the audit team was told that the first of the reports
from the Micro Center would be prepared shortly by the local Counsellor/Administrator.
The team confirmed that the required form of report had been specified
to the Micro Center, in guidance notes approved by the Pro Vice-Chancellor
after the team's first visit to the University. This working document,
in the version provided to the team, did not directly identify resource
provision as one of the specified report headings. The University may wish
to consider whether it would be advisable to add this item to the specification.
30 The audit team noted that the 1995-96 annual programme report for the MSc CBIS, compiled from reports from the established centres prior to commencement of the collaboration with the Micro Center, refers to consideration of various approaches to collecting student feedback at overseas sites. Experiments had led staff to conclude that the most effective method was to use course representatives, since feedback obtained through the use of forms was seen by students as little more than a 'tick box exercise'. The team was told that there had been a deliberate decision in the School to concentrate on informal systems for feedback.
31 The audit team accordingly enquired about the process of student representation at the Micro Center and was told that one student took responsibility for communicating the views of a cohort to the Micro Center and School. Formal student meetings are not held, as students meet each other regularly in class, and there are small numbers enrolled in each cohort. As a guideline, the representative for the first cohort had been asked to contact the University every two weeks or so, and to do so even if there was nothing to report. The representative for the newly started second cohort, understandably, had not yet been briefed on this matter. The team gained the impression that this was an effective means of securing student feedback, and that it was recognised as such by the students.
32 The University also uses questionnaires. The audit team learnt from the first cohort that feedback forms (in the standard University format) came all together at 'summer school', and not module by module. The extraction of data from these forms, and its consideration within the University system, is handled just as for other programmes at the University. The team heard that, during the course of a staff visit to Bahrain, the School advises students of its response to their feedback.
33 The audit team heard that Bahrain is one of two centres where the School had asked the partner to discontinue the appointment of a local tutor because he was found unacceptable, partly as a result of student evaluation. The team commends the willingness of the University to take prompt action when the monitoring of the quality of provision revealed a deficiency.
34 The audit team concluded that the University has shown a commendable
concern to ensure that its students on this supported distance learning
course have both direct opportunities to give feedback, when University
tutors visit 'summer schools', and indirect opportunities through their
representative and through questionnaires.
35 The University and the School emphasise the need for academic equivalence in the distance learning experience, and they claim this is achieved by applying the same quality procedures for face-to-face provision and collaborative arrangements. The audit team noted that the annual programme report for 1995-96 recorded agreement on the continuing importance of monitoring the operation of the supported distance learning mode for each cohort of students, in order to ensure the general maintenance of standards and to maintain conformity with the validated course. This commendable intention figured explicitly in the action plan at that time. The team suggests that the criteria against which equivalence is judged, and the methods used to obtain data to make that judgement, will be of interest beyond the School, and should be disseminated in due course within the University.
The student experience
Enquiries, admissions and induction
36 The Micro Center places advertisements and arranges promotional seminars. If prospective students apply, they can expect a response from the University within two or three weeks. The audit team satisfied itself that the University has sufficient involvement, in a careful process, to make decisions on admissions.
37 Induction of new students took place at the beginning of the first module
and provided all the materials and information that the students desired
at this time. For subsequent modules, the audit team encountered consistent
accounts of carefully planned delivery of learning materials in due time
and with provision for back-up.
Access to textbooks and libraries
38 The audit team was told that, in the taught modules, students do not need to make much use of a library, since the School depends on set texts. However, the team noted that access to suitable library facilities had been an early consideration by University and Micro Center staff when the possibility of partnership was being explored, and that steps had been taken to make appropriate arrangements.
39 Set texts are purchased by the University and sent to the Micro Center, module by module. Some of these texts go out of date rapidly, calling for changes and updating. School staff assured the audit team that, at the start of the modules, the necessary textbook resources have always been in place. That firm claim was fully endorsed by students and staff whom the team met in Bahrain.
40 The Micro Center holds one copy of each core text, mainly for use by
its tutors and staff, as well as some supplementary textbooks which can
be borrowed by students. For project work, it has been arranged that students
will have adequate access to the libraries of the University of Bahrain,
the Bahrain Centre for Study and Research, and the Bahrain Society of Engineers.
When, in due course, annual programme reports are made (see above, paragraph
27) the University will no doubt wish to ensure that these include comment
on student access to library resources.
41 The audit team discussed with the Micro Center the functions of the post of 'Local Student Counsellor'. The range of duties includes providing first-line advice if students find difficulties in reconciling the demands of their studies with private or employment obligations; facilitation of day-to-day communication between the local student body and the course team; co-ordination of administrative matters relating to receipt of coursework; recording of communications with the University and others; and managing the receipt, storing and issue of materials. The role is, therefore, mainly administrative but is certainly central to the way the Micro Center provides student support in accordance with the Memorandum of Co-operation.
42 The audit team formed the impression that academic counselling makes
an effective contribution to the student experience. The students see it
as their principal channel for liaison with the University, which is indeed
specified by the Memorandum as one of the main functions of the role. The
team was satisfied that the University was in a position to ensure that
arrangements for counselling as described in the Memorandum and in the
course guide were being fulfilled. The team commends the University for
the care it has taken in establishing this system.
43 In each year and stage of the programme, students attend two separate
weeks, which are designated as a 'summer school', and which are also attended
by tutors from the University. On these visits, there is the opportunity
for tutors to revise and consolidate material which has been presented
in the video-taped instruction, deal with aspects of the taught syllabus
which have been difficult for students to grasp, and provide module updating
by enriching and updating the material contained in the pre-recorded materials,
which can date rapidly in this particular subject area. The planning by
University tutors for that visit is influenced by e-mail communications
between the Micro Center and the students, which are copied to them, and
are a useful source of information about areas of academic difficulty.
The content of the summer school programme is such that it can assist students
to prepare for assessed work. Students on the first cohort told the team
that they had valued the summer school experience, but were concerned that
there had been marked differences between the first and the second summer
schools. The University may wish to consider the advisability of clarifying
the purpose of the summer school week, and monitoring its effectiveness
against objectives.
44 The course guide and student handbook both state that marked assignments will normally be returned within three weeks of receipt in Sunderland. The audit team was told that the turnaround of assignments is carefully monitored in the School in line with this promise, and found that students, when asked specifically about this point, stated firmly that they were pleased with both the prompt return of assignments and the punctilious issue of assignment tasks. The team commends the University for monitoring this important aspect of distance learning support.
Academic standards and the assessment of students
45 The Memorandum of Co-operation assigns responsibility for academic
standards to the University and, within the University, to the Academic
Board. It is the University's expectation that a designated link person,
in this case the Course Leader, will play a key role and act as a channel
for communication between the University and the partner institution.
Standards of the student intake
46 The audit team noted that publicity for the programme states that the
'usual' entry requirement is 'a good degree'. However, the University provided
information about enrolments which showed that a large proportion of students
had joined the first cohort without a first degree. The Course Leader explained
that there were many expatriates in Bahrain who had achieved a high employment
status without a first degree, which was the reason for the profile of
entrance qualifications for the first cohort. The team was informed that
the intake standards for the second cohort were more consistent with the
usual requirement and was able to confirm this when in Bahrain. The University
may wish to reassure itself both that admission standards conform to the
stated position and that the needs of non-standard entrants are identified
and that the action then taken is monitored.
47 There are either one or two assignments to be submitted per module, and in some modules there will also be an examination. Students told the audit team that they had received all the information they wished about assessment criteria, whether on paper, from local staff, or from University staff. However, students also reported that there had been a number of occasions when they were unclear about the meaning of particular assignment tasks, and when the clarification provided had not been fully satisfactory. Students informed the team that the experience of the first cohort had been of a pattern of submission dates which had led to a concentration of demand in the closing weeks of the first year. Student feedback on this point had led to a revision in the dates scheduled for the second cohort. Similarly students told the team of several occasions when a student who had enquired about a marking decision had received a full and prompt response from the School. In these and other instances, students volunteered appreciation of the commitment of University staff to respond to communications, and to do so speedily.
48 Many assignments are 'time constrained' in that there is a fixed period of some weeks available for completion after the task has been issued. The assignment tasks are set by University staff, who also mark the submitted work. Arrangements for the receipt and safe onward transmission of student work are agreed, managed and monitored by the School and undertaken by the Micro Center. The completed assignments are handed in to the local tutors. The audit team was told by University staff that these tutors would know of the capability of the students from their work in class, and would notice and report any discrepancy in standard if there was any suspicion that students might have received external assistance. But it also learnt that this informal checking process was not part of a tutor's job description, nor did it occur in every case.
49 The team notes that the Centre for Distance Education in Computing (see above, paragraph 13) has produced a manual, Code of Practice for Distance Learning Assessment, which has advice on collusion and plagiarism. At present, with small cohorts at the Micro Center and relatively frequent visits by University staff, the existing procedures to guard against collusion and plagiarism may well be appropriate. Nevertheless, the team suggests that the University may wish to consider the advisability of monitoring the procedures in place to ensure the integrity of continuous assessment.
50 The audit team recognises that the objective of the course team is to
bring to distance learning students 'as nearly as possible an identical
experience to those students who study in Sunderland'. Nevertheless, it
advises the University that careful consideration should be given to public
perception, as well as the reality, of the integrity of an assessment scheme
which largely depends on the assessment of coursework. During its audit
enquiries in Bahrain and other states in the region, the team was informed
by reliable sources that this issue is of importance to both employers
and authorities in the Gulf area.
51 The assessment decisions for students who are studying on distance learning
options are made for all distant centres by the same examination board
as for face-to-face course students. The same external examiners are involved.
Since the University has not decided to implement distinctive procedures
for examining distance learning candidates, the audit team was interested
to discover how an external examiner would be informed of, and be able
to compare, the quality of work across particular centres. It learnt that,
due to the absence of standard starting and finishing dates, several assessment
boards take place during the academic year to consider the results for
more than 100 candidates. Since the scripts and marks are kept separate
they are available for analysis by the Programme Assessment Board. There
was some evidence in the documents made available to the team that external
examiners were in a position to comment on variability in standards between
centres but, apart from a brief report submitted by an external examiner
who had visited the centre in Hong Kong, there was no clear evidence that
there had yet been any systematic consideration of variability. The University
may wish to consider the advisability of undertaking such an analysis.
52 The audit team was provided with the MSc Computer Based Information Systems Student Handbook which incorporates syllabus, rules and regulations. Several documents from different sources are grouped together and, possibly because the documents were produced at different times by different authorities, there are inconsistencies between them. For example, at one point there is a description of 'Grade Schedule' which states that 'C' is equivalent to 40 per cent to 54 per cent and that this grade is the pass grade; at another point, the same handbook, but in a different document, states that 'Grade D is the minimum requirement' and that 'C' has a percentage equivalence of 53 per cent to 56 per cent.
53 At its initial meeting with the University before the audit visit, the team was informed that the letter grades in the regulations were not used at all in assessment, and that feedback to students was given in the form of percentages. The team was told that the student handbook had not been brought into line with this practice because the School was changing the version of the programme in the University presentation, and would implement that change, as others, one year later in the distance provision. School staff expressed the view that it was not worthwhile correcting the handbook when this was the penultimate cohort of distance learning students on the current version of the programme.
54 Students who met the audit team appeared confident that the pass mark
was set at 40 per cent and that marking used percentages and not letter
grades. They told the team that this information had been given to them
orally by University staff on one of their visits. The team, however, concluded
that oral guidance was not a satisfactory correction to inconsistent handbook
information. The University may therefore wish to consider the necessity
for clear and definitive written regulations to be given to its students
based in distant centres, especially when new centres are brought into
operation.
55 The audit team noted from papers supplied to it by the University that, when the School conducted its annual monitoring exercise for 1996-97, the MSc CBIS was operating in full- and part-time mode and in distance mode at five centres involving some 15 distinct cohorts. In a commendably self-critical report on annual monitoring, the School recorded that 'optimal control' of the inevitable complexity had not yet been achieved by the CBIS Programme Board of Studies. The team enquired about the steps being taken by the University to ensure that a greater degree of control and consistency would be achieved, and learnt that the University's Quality Support Unit would be expected to establish central oversight of arrangements for distance learning. This was reported to the School Quality Board in January 1997 and to the audit team in July 1997. The team regarded the University's intentions as sound but understood that implementation had been delayed for good reasons. The University may now wish to consider the advisability of accelerating its central oversight of distance learning programmes, particularly with respect to assessment procedures and the arrangements for Boards of Assessment.
Staffing and staff development
56 The local tutors whom the audit team met were clear about the limitations
as well as the duties expected of them in their role. They are employed
by the Micro Center to handle what the University described to the team
as first-level questions; second-level questions are to be referred to
School staff. Local tutors attend for scheduled classes where they assist
students on tasks provided for them by the University. They should not
amend material, nor change the assessment schedule; they should not assist
with assignments, though they might clarify what is expected of the student;
they should resist requests to offer an opinion on possible grades for
submitted work, or work about to be submitted. They countersign coursework,
testifying that it is the student's own work. They are clear that they
may discuss with the students the feedback on assignments, and on occasions
will receive telephone calls at home or their place of work, but that is
not encouraged. Other than when the students inform them of their marks,
local tutors receive no information about assessment decisions. Their feedback
on the materials and syllabus is welcomed and considered by School staff.
57 Applicants for the post of local tutor are recruited mainly from the relatively small local academic community, in the first instance by personal contact. The audit team was told that when University staff visit Bahrain they interview prospective tutors, most of whom are drawn from the staff of the University of Bahrain. Since visits occur relatively frequently (see above, paragraph 28) this can be done with little delay.
58 The School informed the audit team that it expected local tutors to have a masters degree and teaching experience. The team was also told by the University it had confidence that local tutors were 'technically and academically able to undertake their support duties' because they were experienced and well qualified. However, the team noted from curricula vitae that not all local staff in post had had experience of teaching at University and postgraduate level. The University may wish to consider the advisability of checking carefully that its expectations are being met.
59 The audit team was informed that initial briefing of local staff at the Micro Center had been the responsibility of the Director of the Centre for Distance Education in Computing, who goes through one module with them, as an example of the School's approach. They receive a full set of materials and textbooks, together with tutors' notes for the activities which they will facilitate. The Course Leader maintains contact with local tutors. The team found these arrangements thorough and satisfactory.
60 The audit team enquired about provision for the staff development of local tutors, for example in respect of tutoring for the transferable personal skills and supervision of dissertation work which the University emphasises as particular features of the MSc CBIS. The team was disappointed to learn that neither matter had been the subject of staff development at the time of their visit. In addition, the team noted that neither Micro Center nor School staff saw it as their responsibility to observe local staff working with students, a practice which could yield valuable insight into development needs.
61 The audit team was told by staff of the University that the link person took responsibility for determining the adequacy of staffing arrangements and student support in an overseas centre. The team was therefore surprised to learn that the Course Leader for Bahrain did not regard observation or supervision of local staff as one of his responsibilities. Local tutors told the team that they believed the University would be aware of their performance through student feedback forms and the communication links between the student representative and Micro Center and University staff. As indicated above (see above, paragraph 33) there has been an instance of a local tutor being removed from duties following critical comment from students.
62 The audit team found that the range of activities undertaken by local tutors had been clearly communicated during induction and was operating as envisaged by the University. The team commends the University for establishing clear expectations and monitoring performance. Nevertheless, the team was not confident that, in this respect, the University was observing, to any significant extent, the HEQC Code of Practice for Overseas Collaborative Provision (1996) second edition, which calls upon UK universities to satisfy themselves that provision is made to provide staff in partner institutions with 'suitable development opportunities'.
Publicity and promotional materials
63 Responsibility for the production of the University's various promotional
materials is vested in the Corporate Affairs (Marketing and Promotions)
Unit; and the appropriate Pro Vice-Chancellor is required to give formal
approval to proposed publicity material prior to publication. The 1996 Report comments
that, at that time, the audit team had been unable to ascertain if a formal
check by the University had taken place in every case. The current audit
team therefore gave careful consideration to the publicity materials prepared
both by the University and the Micro Center and to the procedures for approving
them.
64 The programme is promoted in Bahrain through the use of local advertisements and by the staging of open evenings. When in Bahrain, the audit team found careful record of all communications with the press, including advertisements, and also of the communications with the University through which approval had been punctiliously and commendably obtained.
65 The audit team found nothing questionable in the locally prepared promotional materials. However it was concerned to see a potentially misleading statement, in a leaflet about the MSc CBIS produced at the University, which claimed that the School is 'internationally renowned for research'. This is a wide claim which, in the context of promotional material, should be accompanied by factual evidence. In the absence of this, the University may wish to consider the way in which it monitors the claims it makes in respect of the School's reputation.
Conclusions and points for further consideration
66 In 1993 the University agreed to the adaptation of an existing
MSc programme for overseas delivery in the form of supported distance learning.
The School of Computing and Information Systems has established an ambitious
project. At the time of the audit, five centres were operating: in addition
to Bahrain, the programme was available in Hong Kong, Jeddah, Kuala Lumpur
and Riyadh. Institutional procedures are being extended for formal quality
assurance of distance learning programmes and the University has itself
recognised the advisability of improved central oversight through the Quality
Support Unit.
67 Since 1993, experience has been gained by the University and the School in the quality assurance of distance learning programmes. This has led to greater formalisation of procedure and there may well be areas, such as the provision of programme reports, which would benefit from further attention. However, the audit team was able to conclude that any lack of clarity about responsibilities had not, in fact, prevented necessary action from being taken. The team concluded that student interests had been given high regard.
68 In the further development of its procedures, the University will, no doubt, wish to monitor the measures it has taken to strengthen consideration of proposals both to extend approved programmes to new locations and to develop the scope of collaboration with approved partners. The University may also wish to consider how best to monitor academic standards at its partner centres, particularly with a view to variation between centres and the integrity of student assessment. Nevertheless, the audit team concluded that there is good reason to believe that the programme as delivered in Bahrain is equivalent in academic standard to the MSc CBIS available at the University.
69 The audit team found scope for improvements in documentation, particularly of assessment regulations, and the respective responsibilities of those involved in the programme. Provision for the staff development needs of local staff also appears not to have been provided.
70 The University has ensured that students registered for its distance learning MSc CBIS receive academic counselling at the partner institution. The audit team found that the University was able to monitor this provision appropriately. It also concluded that the University had shown commendable concern to ensure that its students have direct opportunities to give feedback to University staff and indirect opportunities through their representative and through questionnaires.
71 Overall, the audit team concluded that the learning experience which
the School provides for students in Bahrain has, to date, been characterised
by a caring style of support and prompt response to queries and feedback.
The audit team commends those immediately responsible, both in the University
and in the Micro Center. In particular, close interaction has been facilitated
by the visits which University staff have made to the Micro Center and
by good communications, especially on student-related issues, between the
Micro Center and the Course Leader.
Annex
Commentary on the audit report supplied by the University of Sunderland
Developments since the audit visit
1 A new Centre for International Education, reporting to the Pro Vice-Chancellor,
was established by the University with effect from August 1997. It is responsible
for the provision of international outreach programmes currently offered or planned
by the School of Computing and Information Systems and the Business School and
for international recruitment. These programmes will be subject to the normal
quality assurance processes of the University. New outreach programmes will be
developed according to the guidelines for International Collaborative Taught
Programmes: Planning and Operational Requirements.
2 The annual monitoring of overseas provision is now more detailed with individual reports from each site on each programme prepared according to University guidelines.
3 All centres are now operating a common set of regulations. Marking is carried out entirely using a percentage system, with a pass grade of 40%.
4 University Postgraduate regulations state quite clearly that credit from one module cannot be used towards more than one final award. If students do not complete their original award they may be able to transfer to another programme as long as they meet the requirements of the University regulations for Accreditation of Prior Achievement and subject to approval of the relevant Programme Studies Board.
5 Local tutors are expected to have at least a masters degree and some teaching experience. Their role is limited to providing support to tutorials and obtaining answers to students' questions on course material. They do not develop teaching materials, prepare or deliver lectures, prepare tutorial material, nor do they mark coursework or examinations. These latter activities are carried out by Sunderland-based staff.
6 All tutors appointed to supervise masters dissertations are given appropriate staff development, before the commencement of the dissertation.
7 Local tutors are offered a range of staff development opportunities including participation in summer schools and regular meetings with University staff. They also enjoy a one-to-one relationship with the Module Tutor at Sunderland, where any issues or problems pertinent to their role can be explored in depth.
Top