Introduction
1 The Quality Assurance Agency for Higher Education (QAA) is a UK organisation that seeks to promote public confidence that the quality of provision and standards of awards in higher education are being safeguarded and enhanced. It provides public information about quality and standards in higher education to meet the needs of students, employers and the funders of higher education. One of QAA's activities is to carry out quality audits of collaborative links between UK higher education institutions and some of their partner organisations in other countries. In the spring and early summer of 2002, QAA audited selected partnership links between UK higher education institutions and institutions in Denmark, Germany and Switzerland. The purpose of the audits was to provide information on the way in which the UK institutions were maintaining academic standards and quality of education in their partnerships with institutions in these countries.
The process of audit of overseas partnership links
2 In planning the audits of overseas collaborative provision, QAA invited all UK institutions to provide a list of their collaborative links with Danish, German and Swiss partners. On the basis of the information provided a variety of collaborative partnerships was selected for scrutiny. Each of the UK institutions whose collaborative link had been selected for the audit provided a Commentary describing the way the partnership operated, and commenting on the effectiveness of the means by which the UK institution assured quality and standards in the link. In addition, each institution was asked, as part of its Commentary, to make reference to the extent to which the link was representative of its procedures and practice in all its overseas collaborative activity or specific to the partnership being audited or to the country concerned.
3 Audit teams visited the Danish, German and Swiss partner institutions to gain further insight into the experience of students and staff, and to supplement the view formed by the audit team from the institution's Commentary and from a visit to the UK institution. During the visits to Denmark, Germany and Switzerland, further documentation about the partnerships was made available to the team, and discussions were conducted with key members of staff, lecturers and students. The audit team for this partnership comprised Dr R Davison, Professor R J Harris and Professor P D Hartley, auditors. The UK and overseas audit exercise was coordinated for QAA by Dr P J Findlay and Dr C J Haslam, Assistant Directors, Institutional Review Directorate, QAA. QAA is particularly grateful to the UK institutions and their partners in Denmark, Germany and Switzerland for the willing cooperation provided to the teams.
4 Institutions were invited, in their Commentaries, to make reference to the ways in which their arrangements met the expectations of QAA's Code of practice on the assurance of academic quality and standards in higher education, Section 2: Collaborative provision (1999) (QAA's Code), which took full effect in August 2000.
5 This report describes the audit of the collaborative link between the Open University (the OU or the University) and the Kolding Købmandsskole in Kolding, Denmark (the College). The audit was conducted on the basis of visits by an audit team to the institutions concerned and on the scrutiny of documentary evidence made available by both the University and the College. A series of meetings was held on 12 April 2002 at the University between members of the team and senior staff of the University and staff of its Validation Services, and this was followed by a visit to the College on 17 May 2002, when the team met with staff and students at the College.
6 The most recent audit of the Open University at institutional level took place in 1999. The University's general arrangements for its collaborative provision were subject to an audit conducted by the Higher Education Quality Council (HEQC) in 1996. Previous audits of the University's overseas collaborative provision took place in 1997, relating to partnerships in Greece and Hungary. The University's Business and Management provision was assessed as 'excellent' by the Higher Education Funding Council (HEFCE) in 1994.
The background to the collaborative partnership
7 The programmes of study which form the object of the present audit are a Dip HE in International Business (hereafter Dip HE), designed in accordance with the requirements of, and subsequently approved by, the Danish Government, and a 'top-up' BA(Hons) International Business degree. Both programmes are taught at the College and lead to validated awards of the University. The Diploma is of two years duration, is taught and assessed in English, and includes two 15-week placements. These are often but not necessarily undertaken in the United Kingdom, where the College's placement coordinator is based. At the time of the audit, there were 370 students enrolled on the Dip HE. The BA programme, after a very slow start, has 40 registered students, almost entirely on a part-time basis, who are taught by a combination of face-to-face and e-learning methods.
8 Kolding Købmandsskole is a further and higher education business college of some 4,000 students (1,800 FTE) and 73 full-time and 14 part-time academic staff. The large majority of students are Danish, but not necessarily local, the College seeing its catchment area as predominantly national. It also has an increasing but still fairly small number of international students. A legally independent entity, the College's formula funding from the Danish Ministry of Education is overwhelmingly its main income stream. The College is situated on the edge of the town of Kolding in south-west Denmark. A substantial new building programme will, when completed in September 2002, increase available working space by around 25 per cent, further improving the quality of the learning environment.
9 The University's association with the College began in 1994, when the College approached the Open University Validation Services unit (hereafter OUVS) to request validation of its International Business Academy (IBA) award and, subsequently, a third BA year. To this end OUVS, acting on behalf of the University, accredited the College, approving the College's IBA programme as an Open University validated Dip HE. In 1997, OUVS, again on behalf of the University, validated the additional BA year, giving limited (2-year) approval in the first instance, though the follow-up review was in the event postponed to 2001 because of failure to recruit full-time students. Danish students on approved programmes of study receive maintenance grants and a loan facility, and have no fee requirement. However, the Danish Government did not recognise the BA year for such support, so the few students taking the programme were doing so on a part-time basis and would not have graduated by 1999. As noted above, this recruitment problem has now been resolved, with an increase in degree students from single figures to 40 in the 2001-02 academic year. This is partly explained by the College's development of e-learning, and partly by the accumulation of previous years' diplomates among the part-time student population.
The University's approach to collaborative provision
10 The University's Commentary explained in some detail the ways in
which the accreditation and validation arrangements for collaborative provision
adopted by the University centre to a large extent on the work of OUVS. This
unit validates such provision at a range of academic institutions, professional
bodies, companies and other organisations, both in the UK and overseas. Formed
in 1993 and based in London, OUVS initially comprised former staff of the Council
for National Academic Awards (CNAA). OUVS is funded by its services to the
HE sector generally, including validation and accreditation services for organisations
which are offering programmes using the Open University's own courses and materials.
In the University's words, OUVS 'neither draws on nor subsidises the funding
of other areas of the University's work'. While OUVS is part of the University,
the audit team found that it had operated with a degree of independence. The
University itself had described this relationship, in its own documents and
in statements to the team, as having been characterised by a 'Chinese Wall'
separating the unit from the main body of the University's work (a view endorsed
by the team, based on its observations at the College). However, the University
also stressed that a number of significant proposals for changes in the operation
of OUVS, intended to achieve a higher level of integration and co-ordination,
were currently being implemented
(see below, paragraphs 13, 14 and 17). The team warmly welcomed these proposals.
11 The collaborative provision that leads to awards of the University is of two types: first, programmes which involve Open University students in other countries, following the University's own open and distance learning programmes in collaborative partnership with another institution: and secondly those programmes involving the validation of another organisation's programme, wholly owned, designed and delivered by the partner. In such cases, the partner institution is required to achieve accreditation in order to offer programmes leading to OU validated awards. The majority of these latter programmes are mainly delivered on the basis of face-to-face teaching and learning. In the University's view, the arrangement with the College can be seen as representative of this second category. Whereas students registered for the first type of partnership are classified as full students of the University, the University defines students in the second category as students of the accredited institution registered with OUVS for a validated award of the Open University. Hence their award is BA (OUVS) as opposed to BA (OUVA). These students have different rights and entitlements to those of the Open University's other registered students.
12 The University's communication and liaison with accredited and validated institutions occurs in very large part through OUVS, to which is devolved the responsibility for approving, monitoring, supporting and revalidating external programmes. Much of the work of OUVS in relation to the University's accredited institutions is of an administrative character. This includes the setting up and servicing of accreditation and validation panels, the conferring of award certificates and the maintenance of records of graduates, the provision of the regulatory framework for the awards, and assistance with the development of quality assurance systems. Each accredited partner institution has, as its primary point of contact, an OUVS officer (the programme manager) whose role is to oversee the continuation and development of the collaboration.
13 OUVS publishes a Handbook for Validated Awards (the Handbook), in which it describes its principles, regulations and processes, together with the expectations and requirements laid upon accredited institutions and their validated programmes. Together with other information, the Handbook sets out the principles adopted by the University for accreditation and validation, and describes in substantial detail the quality assurance procedures which will be used, the regulations which will apply, the standards of awards and the requirements for external examiners. Copies of the Handbook are provided to institutions and to members of committees and panels involved in OUVS work. The audit team noted that a revised edition of the Handbook, originally to be sent to accredited institutions in September 2002 but subsequently postponed to the first half of 2003, was under preparation. In the meantime, a published consolidation of recent amendments to the procedures was to be distributed to accredited institutions shortly after the team's visit to the College. This represented existing practice and changes made since 1997. The team considered that, subject to the updating now required, the Handbook provides comprehensive and professionally presented information on the arrangements operated by OUVS.
14 Until September 2001, the quality and standards of the University's collaborative provision was overseen by the former Validation Board, which worked separately but in parallel to the University's Academic Board. In September 2000, the University carried out a wide-ranging review of the work of OUVS, and included in its recommendations the view that 'in the future we need to ensure that the aims of OUVS and the rest of the University are aligned so that there are tangible benefits to the whole institution'. Following upon the review, the current arrangements provide for a higher degree of integration of collaborative provision business into the main academic structures of the University. A Validating Committee, which is a part of the Academic Board sub-structure, has replaced the Validation Board. The Validating Committee oversees all validated collaborative provision, and reports to the Curriculum and Awards Board and onwards to Academic Board. The Validating Committee contains a majority of external members from both higher education and the business and industrial world. The audit team was informed by the University that the Curriculum and Awards Board now provides a valuable function in taking a general overview of all collaborative provision, and of its relationship to internally approved programmes.
15 The University regards the locus of responsibility for quality and standards of its validated awards as being shared between the University and the accredited institution, though the accreditation agreement for the partnership confirms the University's ultimate responsibility. An accreditation panel representing the University establishes these matters at initial approval stage and on a six-yearly cycle thereafter. In the interim period the University stresses the 'accredited partners' responsibility for quality assurance of validated programmes'. In that the accredited institution also employs the external examiner, and will receive his or her reports, it can be construed as having a prime responsibility for the maintenance of standards. Such a view is consistent with the University's insistence that the students are primarily those of the accredited institution and not the University, albeit that they are registered for a programme leading to the award of an Open University validated degree.
16 The audit team noted that a feature of the University's arrangements is that a large proportion of those involved in the accreditation and validation procedures, and in the consideration of reports, are members external to the University. The University's view is that this level of externality is indicative of the fact that the quality and standards agenda set for accredited institutions is judged against UK national norms and not primarily against those of the University itself. While the team agreed that there were some clear strengths in this involvement of external expertise, it also found that this approach had the disadvantage of limiting the amount of direct University involvement with its partners. As a consequence, under the University's model of accreditation, there were a number of features which demonstrated different emphases in the relationship between the University and its accredited institutions, when compared with similar arrangements elsewhere. The partner institution and not the University appoints and employs external examiners (subject to the final approval of the University to whom examiners now have a direct reporting duty). The accredited institutions have discretion to make teaching staff changes in respect of validated programmes, subject only to retrospective notification: the University does not approve the curricula vitae of new staff in advance of their taking up their duties. The partner institutions are responsible for maintaining student records, the University maintaining only minimal student data of a kind that precludes value added or comparative analysis. Following discussions at the University and in Kolding, the team was therefore able to substantiate the observation made in the Commentary that '[the OUVS] model of course validation (like that of the CNAA before it but unlike some other validating universities) does not rely on close interactions between staff in accredited institutions and academic staff of the University or its Validating Committee outside the formal interactions of review'. The fact that, at least at the time of the audit, the route for communication between the University and its accredited and validated institutions was almost exclusively through OUVS was, in the view of the team likely to make it more difficult for the University to establish the equivalence between the standards of its own awards and those of its partner institutions.
17 During its visit to the University and in later stages of the audit, the audit team was able to read documentation relating to the internal review of OUVS carried out in September 2000, which indicated an awareness among senior University staff of the need for fundamental change in the structural positioning and work of the unit. This document demonstrated that the University had carried out a major review of its procedures. The team was also able to meet with the recently appointed Director of OUVS and the Pro-Vice-Chancellor to whom that person reports. In that discussion, the team understood that OUVS was now being encouraged to take a more strategic approach to its validation work than hitherto, and that attention had been given as to how OUVS and its area of work could be further integrated into the University. In addition, the University has since stated that it is addressing a number of matters which together will constitute a significant development in the University's management of its validated collaborative provision. These include: the status of students in validated institutions; systematic communication with its accredited institutions and liaison within the University through the appointment of nominated OUVS, Regional and Faculty representatives; more regular monitoring visits to institutions; and a more focused and regular programme of networking and development opportunities. Taken together, these considerations (which the team had not found reflected fully in the University's Commentary) will make a considerable contribution towards addressing some of the concerns identified in this report (see below, paragraphs 25, 29, 32, 35, 36 and 40), thus also ensuring a fuller adherence to QAA's Code. These were, however, relatively recent changes and still remained to be fully realised. Accordingly, many of the developments were not yet found by the team in the evidence relating to the partnership with the College.
The establishment and management of the collaborative partnership
The selection of the partner
18 The University, as represented by OUVS, takes a reactive approach to the initiation of its validated collaborative partnerships, in that it responds to enquiries received rather than initiating associations with institutions. OUVS stated that the rigour of its procedures is such that only 10 per cent of 'serious enquiries' result in a validation agreement. The evidence heard by the audit team generally indicated the careful application of these demanding selection procedures. Nevertheless the team observed that, as a consequence of a responsive rather than strategic approach, the University had accredited a range of geographically dispersed partners whose teaching practices, scale of operation and (in some cases) discipline affiliation differed substantially from its own. The team was encouraged to hear that issues such as this are increasingly being debated within the University. The team would encourage the University, for the future, to reflect on the desirability of identifying, where appropriate, the degree of consonance between its own interests and those of accredited institutions.
The accreditation and validation process
19 Normally, the University will accredit an institution as having the appropriate structure and framework for the delivery of a course of studies before proceeding to the validation of individual programmes. If accreditation of the institution is approved by the Validating Committee, decisions as to the validation of programmes within that institution are then devolved to visiting panels. Initial validation of existing programmes may also be undertaken at the same time and by the same panel as accreditation.
20 The accreditation procedure entails an initial approach followed by a 'senior level' meeting with OUVS staff to determine whether a prima facie case exists for proceeding. The proposing institution then submits an outline submission. This includes a commentary on the nature of the institution, its executive and academic structures, resource information, staffing policies and academic plans. The submission is explored further in an advisory visit by OUVS staff, with a subsequent developmental report and if necessary consultancy support. A final submission document is then produced and an accreditation and validation visit is made. This typically lasts for two days, and panel membership normally includes members of the Validating Committee and OU staff together with external members. Where the outcome of the visit is successful, continuing approval may be granted, subject to review every five to six years. The report of the visit, supplemented by oral contributions from members of the visiting panel, is considered at a full meeting of the Validating Committee. OUVS officers have an organisational and participatory role in all stages of the process.
21 The partnership with the College, and its Dip HE programme were subject to accreditation and validation in 1994, in accordance with the procedures described above. This was followed by the validation of the BA award in 1997. However, the BA award was validated for two years only, subject to certain conditions being met. These conditions were then tracked through the annual reporting mechanism. In 2001 the University carried out a further full review of the BA programme. This was combined with a general institutional review, meeting the University's standard requirement for a six year period between initial accreditation and institutional review. The University maintained some continuity of membership as between the accreditation and review panels, although some original members were excluded from later visits because they had in the meantime been appointed as external examiners and advisers to the College. The audit team noted the involvement in accreditation and validation (though not in continuing support) of one member of the OU Business School.
22 The audit team scrutinised the validation panel reports relating to the College and followed through their consideration by the University's Validation Board (as it then was). The reports were comprehensive in their range of consideration, with appropriate and carefully specified conditions. With some minor exceptions, conditions had been followed up carefully by the College and monitored by the University, either in the annual report correspondence with OUVS or in the context of subsequent visits. On the basis of the evidence available, the team found that the University's procedures for the initial process of institutional accreditation and programme validation were thoroughly specified, supportive of the partner institution, and well-managed and delivered.
Written agreements
23 The partnership is legally defined in an Accreditation Agreement between the University and the College. This lays out clearly and in considerable detail the basis for the operation of the accreditation and validation relationship, including the obligations on both sides. The audit team found the Agreement, which had been renewed in 2001, to be broadly in accordance with the recommendations of the QAA Code. The University may, however, wish to consider the inclusion of statements in its agreement relating to residual obligations to students in the event of termination, and the management of publicity. It will also wish to give strong consideration, in accordance with the relevant precept in the Code, to making the Vice-Chancellor the signatory to the Agreement on behalf of the University.
Certificates and transcripts
24 The audit team found that the University's arrangements for the design and award of degree certificates were in accordance with the precepts of the Code. The University does not provide a transcript; this is done by the College, which produces a transcript in Danish, recording the final results for its own, concurrent, IBA award. These internal College transcripts are not verified by the University, and students do not receive a transcript of their achievements leading to the Dip HE award. At present the University's commitment is only to produce a transcript relating to its own award in response to specific individual requests, and in practice such requests are very rare. It is also likely that the limited student data kept by OUVS would necessitate the University obtaining the relevant information from the College. In view of the possibility of students attaining one award and not the other the University may, for the avoidance of ambiguity, feel it would be helpful to consider issuing a transcript itself for the Dip HE.
Control over publicity and marketing materials
25 The copy of the Agreement between the University and the College seen by the audit team lacked a clear allocation of responsibility for publicity, although the team became aware that additional explanatory information had subsequently been sent to all accredited institutions. The team heard that informal sampling of College publicity is conducted by the OUVS programme manager during visits, and spot checks are also made on the College web site. The College claimed, and the team's observation confirmed, that its use of the University logo had been 'very modest'. While the University intends to regularise the marketing of its collaborative arrangements, and had recently issued accredited institutions with a form of words which could be used without approval, a well-developed strategy and procedures for the control of publicity did not as yet exist, and the University acknowledged that this was an area in which it could experience problems. The team would therefore encourage the University to take forward the planned changes in the management of publicity and marketing in the context of collaborative provision, and in particular to advise validated institutions on the appropriate use of the University's visual identity. This will assist in explaining more thoroughly to prospective students the character of the programme to which they are giving consideration, and status of the validated award to which it leads.
The QAA Code of practice
26 The audit team heard that the University gives careful consideration to the requirements of the QAA Code as a whole within its committees. The University claimed either to be fully compliant with the section of the Code relating to collaborative provision, or to be working towards full adherence to it: different staff members adopted a different emphasis in response to this question. However, as highlighted elsewhere in this report, the evidence available to the team suggested that the level of adherence might be in question. The team's main concerns in this respect related to the rationale for selecting partnerships (see above, paragraph 18), the University's general overview of the partnership (see below, paragraphs 28-30), the formal signing of the agreement (see above, paragraph 23), establishing comparability of standards (see below, paragraph 41) and the line of accountability of external examiners (see below, paragraphs 44 and 45). In addition, it appeared to the team that, even in areas where direct adherence to the Code is not an issue, the University's devolution of responsibility to its accredited institutions may be such that it does not routinely possess the necessary level of information implied by parts of the Code.
Quality of learning opportunities and student support
Liaison and administration
27 The route for communication between the University and its accredited Colleges is almost exclusively through the work of OUVS. The most significant interactions between the University and the College came at the time of initial validation and in the periodic revalidation events. In the intervening periods, communication is mainly administrative, with accredited institutions trusted to deliver a high quality service, but able to contact OUVS for advice and support if necessary. The main focus for such communication is the nominated OUVS programme manager, who would typically be in touch with the partner institution on a monthly basis through correspondence and email. College staff advised the audit team that contact with the programme manager had been intense around the time of revalidation, and, to a lesser extent, with regard to annual reporting, but otherwise sporadic. An important part of the programme manager's role is the receipt of the annual monitoring report from the College and the preparation of a response to it, indicating where actions should be taken. The team noted that the programme manager is not a subject specialist, but a senior OUVS administrator, and it ascertained that little or no programme-level contact based on discipline expertise takes place, although the programme manager is always helpful in suggesting other academics to assist. The College has no relationship with the University's Business School, and the team formed the view that in general the level of any informal day-to-day contact was low. In discussion, management and teaching staff at the College stated that at a number of points such support would have been welcome. This was especially so since, in their view, minor issues which had become problems for predominantly cultural reasons could have been resolved with help from discipline experts. On the evidence available to it and within the defined limits of OUVS procedures, the team found that the work of the programme manager was professional and efficient. The University may nevertheless wish to consider ways in which a greater level of subject-based liaison can be made available to its partner institutions.
Monitoring and review
28 The University's accredited institutions are required to submit an annual monitoring report on the previous academic year's activities to the Director of OUVS by the end of the following January. Information provided in the report covers enrolment and admissions, the minutes and results of the annual examination board meeting, the external examiner's report, a list of current staff, and minutes of the College Academic and Course Committees. It also includes a brief evaluative report by the College course manager. The audit team considered that while the reports included the necessary basic information and statistics, the quantity and quality of analysis and evaluation could usefully have been increased. The team noted that such a development had been encouraged in OUVS responses to reports. On receipt of each annual report, it is read by the OUVS programme manager, who responds on behalf of OUVS. Programme managers for different institutions then meet to compare and discuss the annual reports received, and identify a sample selection to bring to the attention of the University's Validating Committee.
29 The annual monitoring reports are submitted to the Validating Committee,
a sub-committee which each year reads and reviews around four to five of the
circa 30 reports received. These reports are selected for the sub-committee
meeting on the basis of the sample drawn up by a meeting of OUVS programme
managers, both to illustrate good practice and to bring to its attention cases
where specific concerns have been identified. However, the audit team found
that in this sampling process, no monitoring report from the College had thus
far been seen by the sub-committee.
It has, therefore, been the case that the senior University committee with
responsibility for collaborative provision has not, over a period of several
years, given any detailed attention to the reports received from one of the
University's accredited institutions. The team considered that this constituted
a major weakness in the University's monitoring procedures.
30 With regard to the sub-committee's work more generally, the audit team saw evidence of the thoroughness of the discussion on the Committee, but was nevertheless concerned both at the smallness of the sample of reports scrutinised and the variable response of accredited institutions to the Committee's observations. Minutes of successive meetings revealed repeated criticism of the variable quality of reports received from those accredited institutions in the sample: some omitted mandatory information, lacked any reflective dimension or failed to respond to issues raised by programme managers. Since these were repeated concerns, it appeared possible to the team that the follow-up mechanisms in place (for the Validating Committee and for OUVS) to address any action planning for improvement were less than effective. The team did note, however, a reference in the recent Handbook addendum to a proposed annual monitoring visit to report on issues arising and actions taken during the preceding year. Though this intention had been communicated to accredited institutions, it appeared to the team that it had not yet been extensively discussed with them - certainly College managers to whom the team spoke had a low level of awareness as to the current intentions. The University will wish to ensure that its senior committees are in a position to receive and properly to consider periodic reports on all institutions offering awards in its name, and that it can both monitor and respond to the reports in an effective way.
31 The University also has a Quality and Standards Committee, but the responsibilities of this committee relevant to this audit are largely concerned with addressing questions of compliance with the QAA Code of practice, and do not extend to specific issues emerging from the University's experience of validation of external programmes. The attention of the audit team was also drawn to the work of the Curriculum Partnerships Committee, which has the remit to draw together the different kinds of partnership operated by the University, and to provide for the exchange of good practice. The Validating Committee reports to the University's Academic Board through the Curriculum and Awards Board, which has a similar role in bringing together the oversight of all kinds of collaborative provision offered in the University. However, the team saw no evidence from the analysis of the Kolding collaboration that these committees played a role in moderating reports and concluded that, at least so far as the Kolding collaboration was concerned, the University was not well-placed to identify common issues of concern arising from such reports.
32 Currently the annual monitoring report from the College contains no reference to views received on the basis of student feedback, or actions taken as a result. Student feedback on the academic elements of the course mainly takes the form of open meetings with the course leader. Documentation seen by the audit team indicates that College staff believe feedback systems are working better than hitherto, that students are becoming increasingly 'demanding' and that OUVS visiting panels had concurred with this view. On the other hand, in discussion OUVS staff were less sanguine, advising the team that 'it doesn't seem to be working well'. Discussion with College staff confirmed that the system lacks any means of consistently recording comments, deciding on corrective action and reporting such action (or the decision not to take it) back to students. While accepting the view it heard that collective rather than individual feedback is a feature of Danish culture and that anonymity may be a lesser consideration than in the United Kingdom, the team believes a more systematic approach to student feedback would be desirable in the interest of fulfilling the University's own expectations. The team noted that student feedback had been identified as a matter for attention in the OUVS response to the College's annual report in 2002, but considered that more could have been achieved in this area over the six years of the partnership. Accordingly the University will wish to take the necessary steps to ensure that progress towards this goal is more rapid.
33 The audit team noted that annual monitoring reports gave little indication as to the day-to-day process of the partnership or the extent of University involvement in, or support for, programme management. When this point was raised with College managers it was claimed that completing a monitoring report was an unusual task in Denmark and they would have valued more help and advice. On its side, the University's approach to accreditation and validation appeared to the team to provide for limited arrangements of a developmental kind, with only infrequent opportunities provided for its partners which might contribute to the ongoing quality enhancement or staff development within a partnership. In this respect the team agreed with a view put to them in discussions with the University that OUVS procedures had failed to keep pace with developments in validation practice since the demise of CNAA. The team therefore welcomed initiatives being introduced, at the time of the audit, to increase the effectiveness of OUVS's developmental engagements with accredited institutions.
34 As noted above (see above, paragraph 21), the College had received two periodic review visits from the University, in 1997 and 2001. The visit in 2001 had combined institutional and course review and also picked up the requirement to extend the approval of the BA programme which had been subject to conditions. The review was informed by a substantial submission document prepared by the College's IBA department, which was informative and self-evaluative. The audit team was able to see the report of the visiting panel, and this described what appeared to be a critically constructive process of review, which had identified a number of areas for action on the part of the College. These included the function and role of its Academic Committee, the management of the Examination Board, and the use of student feedback. It was noticeable, however, that the review focused very much upon the work of the College, and gave relatively little consideration to issues relating to the working of the partnership - such as the quality of communications and support - matters which had been raised in the College's submission. The report of the panel was then considered and noted by the Validating Committee. While the team judged that the review panel had carried out its work effectively, it also believed that some of the matters raised in the report (for instance those relating to Examination Board minutes) merited a more effective response on the part of the Validating Committee.
Staffing and staff development
35 The qualifications and experience of staff at the College were considered by the OUVS validation panel in 1994. Details of teaching staff currently involved with the programme are included in the College's annual report. However the audit team heard that there is no other formal record kept by the University of staff appointments or changes to the staff establishment at the College. Staff teaching on validated awards in partner institutions are not recognised in any formal way by the University, and have no entitlements. As the University's Commentary pointed out, concerns relating to the extent of research and other forms of staff development have been raised by OUVS visiting panels to the College. The 1994 validation report suggested that the College's appraisal system should form the basis of a development programme geared to the needs of the validated course. The 1997 review panel expressed disappointment at lack of progress and made improvement a condition of approval. It also recommended that the College's research profile (in the broadest sense) should be enhanced. As the College's response to this was judged disappointing, the 2001 panel made such enhancement a condition of continuing validation, and has requested a report on progress for 2003. The 2001 report also referred to problems with the English language competence of some academic staff. The team was advised that the College, which is not funded for research, is responding to the demand for a developed research culture by developmental work in learning and teaching (particularly e-learning), and that English language tuition has been made available to staff who need it. The team was reasonably confident that these issues are being addressed by the College. Nevertheless, in view of the English language problem in particular, the University may wish to reconsider its current approach to the appointment of new academics to teach on programmes leading to its awards. The team also noted as a positive feature that OUVS invited staff from its overseas partner institutions to attend workshops, held in London, relating to the quality assurance and management of collaborative provision, an arrangement that was now being further strengthened.
Information to, and support for, students
36 The University noted in its Commentary that 'the University's relationship is primarily with the accredited institution, not with the student'. This was certainly confirmed by staff as well as by the students whom the audit team met in its visit to the College. Senior staff there informed the team that neither academic staff nor students 'identified' with the Open University, with whom their link was a 'transaction not a relationship'. None of the students at the College had ever met a member of Open University staff, none had received written acknowledgement from the University of their registration fee or information as to their status as students registered for its award. For example, students of validated institutions have 'last resort' rights of complaint and appeal to the University, but the students the team met had been advised of this only by the College (against which such complaint or appeal might hypothetically be made), and not by the University. As noted previously (see above, paragraph 16), the University itself holds only very limited information relating to students on its validated programmes. In relation to this the team appreciated that the University's normal administrative system, geared to its open learning students, is inappropriate for this purpose. Nonetheless the team was surprised that the University retains only nominal data on validated institution students, and is therefore not able to compare their progression and achievement with that of others of its students; it also indicates little interest in communicating concern for the welfare of these students, or in making them feel part of an Open University 'family'.
37 The University's position, as made clear in the Commentary and in discussions with the audit team, is that within an accreditation model type of collaborative relationship, the primary relationship for the student must be with the providing institution. In these circumstances, provision of information to students is substantially devolved to the College, and the team found that OUVS staff were unclear about what information is in fact provided to students on enrolment. In the team's view the BA Course Handbook, which is produced by the College, is a thorough and well-written account of students' forthcoming educational experience, though it contains only limited information about the University and its role in their studies. In discussion OUVS staff acknowledged that there were some shortcomings in the provision of information to students about the University, and discussion with College staff and students strongly reinforced this view. In the team's view it is a duty of a responsible awarding institution to provide the relevant information formally and directly to students. The University will no doubt wish to review the level and quality of its communication with students in its accredited partner institutions; in doing so, it should give high priority to explaining to students following its validated awards such matters as status, rights, and routes of appeal.
38 The audit team noted the importance of the two placements undertaken by students as part of the Dip HE, and that, while the placements are on the whole regarded as successful and well-managed, there were reported instances of a small number of students having 'mundane experiences'. It did not appear that the University had received monitoring information on placements or followed up any concerns. Accordingly, the team was interested to explore placement administration with College staff, including how the system operates, what planning and support are provided and how this element is assessed. Following discussion with both students and the placement coordinator the team was satisfied that mundane experiences are exceptional. Placements are highly valued, students normally receive adequate support when difficulties occur and assessment (mainly through the preparation of case studies), is generally rigorous and practice-relevant. The University may however wish to consider whether the level of information that it receives from the College relating to the management of student placements is sufficient.
39 The three OUVS panels which had visited the College had commented very positively on the quality of the learning environment for students. The audit team found that the University as validating body had taken appropriate steps as a part of its accreditation to satisfy itself of that quality. For example, the College library, which at the time of initial accreditation in 1994 was of some concern, has been brought up to the University's expectations. Book ordering is integrated with teaching needs, and students benefit from a system which makes books in academic libraries across Denmark accessible to all students. Workstations are plentiful, with a reported ratio to FTE students of 1:2.5, and of good quality.
40 Viewed more generally, however, the audit team was not entirely convinced that monitoring mechanisms currently in place permit the University to have a sufficient level of confidence that the quality of learning opportunities and student support provided in the partner institution is satisfactory. Although in the case of the College there had been a number of contact and review visits since the initial validation, the University's procedures would potentially allow a period of six years between reviews of an institution, which would be considerably less secure. The inadequacies in this context are attributable both to the lack of a well-developed evaluative dimension in the monitoring communications with OUVS, and to the failure of the University to consider reports at a higher level. In making this observation, the team is not implying that the College's activities in themselves are in any way open to question (indeed there is reason to believe that the College's standards are high). Rather, the team is concerned that the University may not have the necessary structures and procedures in place to provide it with a firm grasp on information relating to the nature and quality of the provision in accredited institutions, and the effectiveness of the partner institutions' own monitoring arrangements. The University has already identified some of these issues, and the team welcomes and supports the proposal, still in discussion at the time of the audit, that the University introduce annual monitoring visits to its validated programmes.
Assurance of the standards of the awards
41 The University's Commentary did not provide a clear overview of its approach to the maintenance of standards in its collaborative partnerships. In meetings, OUVS staff advised the audit team that academic standards are established through the accreditation and validation procedures and were then reviewed through annual monitoring, external examiners' reports, evaluative feedback received during visits to the College, and the work of programme managers. In addition the team was advised that a number of diplomates have joined other UK universities at Level 3 and performed successfully. The team was able to confirm the success of diplomates in securing good degrees with different UK universities, and this is clearly to the credit of both students and the College. In the OUVS Handbook, the University states that 'each OU validated award...is to be comparable in standard with similar awards granted and conferred throughout higher education in the United Kingdom'. A limited assurance of this is provided by the reports of external examiners and external advisors to the College. However, in taking this approach, the University will find it difficult to meet fully the recommendations of the QAA Code, which expects that the awarding institution should be able to demonstrate that the quality of the collaborative programmes is comparable to the quality of any similar programmes provided by the awarding institution itself. In discussions, the University placed considerable emphasis on the rigorous requirements of its initial validation procedures for establishing and maintaining standards. The team was not fully persuaded by this position, however, taking the view that institutions, their management, teaching staff and funding sources are all subject to unpredictable change, and under the current arrangements it could be over a year before OUVS became aware of any such change, and even longer before OUVS informed the University and it formulated and implemented a response. The University will wish to consider how it can strengthen the information it receives relating to academic standards in validated programmes, and how it can establish a more developed and informed level of comparison with its own provision.
Student admissions
42 The University's Commentary made no reference to admissions policy, although the Handbook makes it clear that approval of admissions policy is a part of the accreditation and validation process, and lays out clear procedures. Following formal approval, admission to a programme is at the discretion of the admitting partner institution. Admission to the validated Dip HE programme is on the basis of the Danish final school qualifications, and decisions on admission of students are delegated entirely to the College. The College is also responsible for enrolment, induction and all aspects of student record-keeping. OUVS receives a list of students enrolled and these names are then entered on the OUVS student records database. Students are therefore formally registered as OUVS students following an OU award. The audit team heard that the University does not carry out any periodic checks or audits relating to admissions. As neither the University nor OUVS has any detailed data related to admissions qualifications and student progression, it is not possible for the University to assess any relationship between admissions and student achievement.
The assessment of students
43 Once the accreditation procedures have recognised the partner institution's management of quality and standards, and assessment regulations have been approved through the validation procedures, the University delegates responsibility for the management of assessment almost entirely to its partner institution. Therefore, assessment is designed and administered by the College, and Boards of Examiners are held there without any University representation, but in the presence of the UK external examiner. There is no moderation by the University, no evaluation of assessment outcomes in the annual monitoring report, no comparison of outcomes with Open University or other UK university results, no double marking or moderation by University staff and no sight of scripts: the University simply receives a mark sheet signed-off by the external examiner.
44 The Commentary explained that all the University's validated programmes are required to have at least one external examiner appointed to them, and that 'OUVS believes that it is important for external examiners to be seen to be working for the accredited partner as well as for the University. External examiners are appointed, briefed and remunerated by the accredited institution, with a formal approval by the University. However, they have a dual reporting line to the accredited institution and the University'. Regulations for external examining and the criteria for appointment are detailed in the Handbook, and OUVS uses external appraisers to review nominations. The University will wish to consider whether the high level of delegation to partner institutions with regard to external examiner appointment, as described, is compatible with its need for independent evidence relating to standards, and with the expectations of QAA's Code.
45 External examiners' reports were until recently annexed to annual monitoring reports, and therefore not seen by OUVS until the end of the January following the June examination period. Now they are sent to OUVS and to the accredited institution simultaneously. The reports from validated institutions are reviewed and filed in OUVS, but not communicated further in the University. Both external examiners thus far employed by the College are academics of professorial rank from UK universities. However, the College confirmed that the time available to the external examiners in which to study scripts is very limited and it is 'impossible' for them to read more than a small sample. The concerns of the audit team in this area were not eased by two further problems only recently resolved. Until recently (when the format was changed following University pressure) the report forms used by the external examiners had a check-list format, with space only for brief additional comments. The team considered that the majority of these reports did not provide sufficient information. The team also noted repeated reference to external examiners not being invited to approve examination papers. Particular concern about this was expressed both by the 1997 review panel and by the Programme Manager following the 1999 assessment, when it was insisted that this should in future be standard practice. Discussion between the team and College staff established that this point and its significance in the UK system were now appreciated and that it was being addressed. While recognising that the University had recently taken steps to strengthen the external examiner's role at the College, the team could not be confident that the examiners' work had facilitated a careful monitoring of standards over the period of validation thus far. This was of all the more concern as the University has no other arrangements for checking on standards.
Conclusions
46 The collaboration between the Open University and Kolding Købmandsskole (the College) began in 1994, when the OUVS Unit, on behalf of the University, accredited the College and validated its two-year IBA award at the level of Dip HE. The collaboration developed further in 1997, when a BA year was added to the Diploma and was also validated by the University, initially for two years and subsequently for a full six-year term. In 2001 the University carried out a full review of both the institutional partnership and the validated programmes. The University regards this partnership as broadly representative of its approach to its accredited and validated programmes.
47 In the audit of this collaboration, of at least equal importance to the association between the University and the College is the relationship between the central functions and committees of the University and its constituent unit, the Open University Validation Services (OUVS). The OUVS, though part of the University, has historically operated at a certain distance from it and is funded by the quality assurance and validation services that it provides to 'academic institutions, professional bodies, companies and other organisations both in the UK and overseas'; these may include organisations which are offering programmes using the Open Uiversity's own courses and materials.
48 With regard to the partnership with the College, OUVS, acting for the University, has had very substantial responsibility for accrediting the College, validating its awards and assuring quality and standards. Outside of the work of OUVS the University's involvement with the partnership has been limited. Its main functions with regard to collaborative provision lie in a committee structure which oversees policy and practice and, on a selective basis, receives reports; in the maintenance of basic student records; in providing a last resort responsibility for hearing complaints and appeals; and in the issuing of certificates to graduates and diplomates. The University has only minimal direct contact with staff or students of accredited partner institutions. Students are deemed students of the accredited institution registered for an Open University validated programme and have different rights and entitlements to those of the University's other registered students. It is the accredited College that manages student admissions, maintains student records, employs external examiners and administers the Board of Examiners at which no University member of staff is present.
49 It follows that any security that the University is justified in feeling with regard to the quality and standards of its validated award is dependent both on the reliability of the means by which OUVS secures and interprets relevant information from the College and on the way in which the University secures it from OUVS. OUVS, on behalf of the University, manages the collaboration by four main mechanisms. A validating panel revalidates each collaboration on a periodic cycle; an annual monitoring report is prepared by the College; external examiners' reports are submitted on all validated programmes; and each collaboration has an OUVS programme manager. Of these mechanisms, the work of validating panels is well-designed and is given careful consideration by the University's Validating Committee. The process of validation and revalidation is conducted professionally and to a high standard. The audit found that the communication of the annual monitoring reports was not a reliable procedure for informing the University about quality and standards on a continuing basis, and indeed no such report from the College has ever been included in the small sample seen by the University committee with the responsibility for oversight in this area. The external examiners' reports reviewed by the audit team were found to be barely sufficient for the purpose of checking standards, and they are not considered in the University beyond OUVS. The work of the OUVS programme manager was found to be professional and efficient within the operating framework of OUVS; however this made no provision for advice and support at the subject level, and involved only limited contact between the annual monitoring reviews.
50 The strengths identified by the audit in the University's arrangements for the quality assurance of its overseas collaborative provision are the high quality of the work of the validation panels and their reports, the substantial involvement of external advisors, the comprehensive and detailed documentation provided by OUVS, and the professional support offered by OUVS officers. The audit also found that the University was engaged in implementing substantial changes and improvements following an internal review (see below). However, the evidence available to the audit team highlighted significant weaknesses in the specific context of this partnership: in the University's internal reporting structures; in the amount and quality of the information that the University relied upon in assuring the standards of its awards; in the regular communication and academic support for the partner, in the management of marketing and publicity; and in the provision of information to students. It should be particularly noted that these weaknesses in no way reflect upon the work of the College or the quality of the learning experience of its students, which were not the subject of this audit.
51 The audit was able to confirm the strength of the initial approval procedures operated by OUVS. The shortcomings identified reflect, however, deficiencies in the ongoing assurance of the University's award through the processes of monitoring and report, liaison and support. The audit found that these derive in large degree from the historical and structural features of these arrangements, and not from any lack of attention to agreed procedures on the part of the University or its officers. Taken as a whole, the evidence suggested that the University's arrangements at the time of the audit were not sufficiently robust to enable the central University authorities to have the necessary level of confidence that if problems arose with quality or standards it would quickly be made aware of them.
52 Many of the key issues outlined in the main body of this report are relevant to the University's general arrangements for accreditation and validation, as exemplified by the partnership which was considered in the audit. The view of the University is that the high level of devolution of responsibility for quality and standards accorded to its accredited institutions is justified in view of the rigorous selection and approval arrangements operated by OUVS in the accreditation process. While the scope of this audit did not provide sufficient evidence to make a judgement on this general view, it is clear that the implementation of the policy with regard to this partnership limits the support available to the College and also presents the University with certain difficulties in relation to adherence to the QAA Code of practice.
53 The University carried out a comprehensive review of the work of OUVS in September 2000 which included wider aspects of its management of collaborative provision. The evidence of that review showed the University's clear awareness that aspects of its validation arrangements could usefully be revised. The review initiated a number of changes which at the time of the audit had either recently been put in place or were planned for implementation in the near future. These measures included in particular, the closer integration of OUVS into the mainstream work of the University, changes to the committee structure which allowed for a higher level of accountability to the University's Academic Board, and improvements in communication and liaison. Positive developments are also planned with regard to the status of students at validated institutions, to the rigour and regularity of monitoring processes, to the publication of procedures and advice to institutions, and to staff development support. These are all positive signs for the future, but nevertheless, in its review of the evidence relating to the University's practice in the context of the partnership with the College, the audit identified grounds for concern.
54 The conclusion of the audit is therefore that the University should now, as a matter of priority and at the highest level, complete the implementation of its own review concerning the management of validation and accreditation, and introduce the planned arrangements for a strengthened level of monitoring and support. The University is also advised to give further careful consideration to its adherence to the QAA Code of practice on collaborative provision, and to reflect on the ways in which it can best achieve the appropriate balance between devolving authority to its accredited institutions and retaining a full responsibility for the quality and standards of its awards. On the basis of the evidence relating to this partnership link, the audit found that until such time as the planned developments in the University's arrangements are fully effective, there can be only a limited confidence in the University's stewardship of the quality and standards of its validated awards.
QAA Audit of Collaborative Partnership with Kolding Købmandsskole
An institutional statement from the Open University for publication as an Appendix to the Agency's report
The University has welcomed the audit by external peers of its arrangements for this collaborative partnership. Some of the issues identified in the report had already been highlighted for review by our own quality review and evaluation. The enhancement and improvement process has therefore been given added emphasis by the audit. Examples of improvements in train or planned include:
- A major review by Validating Committee of key aspects of the quality assurance arrangements such as annual monitoring and external examiners
- Continuing development of OUVS processes for accreditation and validation
- An OUVS publication for students on validated programmes, explaining the relationship between institutions and their relationship to both
IRD 725 9/2002
Top