Introduction
1 The Quality Assurance Agency for Higher Education (QAA) is a UK organisation that seeks to promote public confidence that the quality of provision and standards of awards in higher education are being safeguarded and enhanced. It provides public information about quality and standards in higher education to meet the needs of students, employers and the funders of higher education. One of QAA's activities is to carry out quality audits of collaborative links between UK higher education institutions and some of their partner organisations in other countries. In the spring and early summer of 2002, QAA audited selected partnership links between UK higher education institutions and institutions in Denmark, Germany and Switzerland. The purpose of the audits was to provide information on the way in which the UK institutions were maintaining academic standards and quality of education in their partnerships with institutions in these countries.
The process of audit of overseas partnership links
2 In February 2001, QAA invited all UK higher education institutions to provide information on their collaborative partnerships. Using this information, QAA approached a number of institutions who had indicated that they had established collaborative links with Danish, German or Swiss partners. Following discussion, a variety of collaborative partnerships was selected for scrutiny. Each of the UK institutions whose collaborative link had been selected for the audit provided a Commentary describing the way the partnership operated, and commenting on the effectiveness of the means by which the UK institution assured quality and standards in the link. In addition, each institution was asked, as part of its Commentary, to make reference to the extent to which the link was representative of its procedures and practice in all its overseas collaborative activity or specific to the partnership being audited or country.
3 Audit teams visited the Danish, German and Swiss partner institutions to gain further insight into the experience of students and staff, and to supplement the view formed by the team from the institution's Commentary and from the UK visit. During the visits to Denmark, Germany and Switzerland, further documentation about the partnerships was made available to the team, and discussions were conducted with key members of staff, lecturers and students. The team comprised Professor R J Harris, Mr A Davidson, Dr P D Hartley, auditors. The UK and overseas audit exercise was coordinated for QAA by Dr P J A Findlay and Dr C J Haslam, Assistant Directors, Institutional Review Directorate. QAA is particularly grateful to the UK institutions and their partners in Denmark, Germany and Switzerland for the willing cooperation provided to the teams.
4 Institutions were invited, in their Commentaries, to make reference to the ways in which their arrangements met the expectations of QAA's Code of practice for the assurance of academic quality and standards in higher education, Section 2: Collaborative provision (QAA's Code), which took full effect in August 2000.
5 This report describes the audit of the collaborative link between De Montfort University (DMU or the University) and Niels Brock Business College (the College) in Copenhagen, Denmark. The audit was conducted on the basis of visits by an audit team to the institutions concerned and on the scrutiny of documentary evidence made available by both the University and the College. A series of meetings were held on the 9 April 2002 at the University between the team and senior staff of the University and this was followed by a visit to the College on the 22 May 2002, when the team met with staff and students at the College.
6 The most recent QAA audit of the University at institutional level took place in 2001. The University's overseas collaborative arrangements have been the subject of two previous QAA audits in 1999 (partnerships in South Africa and Malaysia). In 1997, two collaborative partnerships operated by the University in the Netherlands and Germany were subject to scrutiny by the former Higher Education Quality Council. The University's business and management provision was assessed as 'excellent' by the Higher Education Funding Council for England (HEFCE) in 1994, and the quality of its computer science provision was assessed as 'satisfactory' by HEFCE in 1994.
The background to the collaborative partnership
7 The collaborative provision considered in this report comprises two franchises operating under a common arrangement whereby the third year of the DMU BSc Computer Science and BA Business Administration are delivered by the College. In this report, other than where otherwise specified, the two collaborative arrangements are treated as a unified whole.
8 The origins of the agreement lie in earlier relationships between the two institutions. From 1993 to 1997, Datamatics students from the College undertook project work under the auspices of DMU; and, in 1995, DMU began admitting the College's Advanced Computer Studies diplomates to the final year of the University's BSc Computer Science and BSc Software Engineering programmes. In 1996 and 1997, the College approached DMU to explore the possibility of establishing a closer relationship in the areas of business administration and computer science. From these discussions, both institutions concluded that closer collaboration would be strategically beneficial. Among the strategic benefits identified by DMU were heightened visibility for the University in Scandinavia and the aspiration that the College might provide a supply source for on-campus research students. For the College, presently prohibited under Danish law from offering degree level programmes, the relationship was perceived as providing a demonstration of the institution's growing intellectual maturity and status.
9 The College is a large and well-established self-governing institution recognised by the Danish Government as offering business education to diploma level. The College was founded in 1880, its name commemorating Niels Brock, a wealthy merchant who was one of the founders of the Danish vocational business education movement. It is the largest business college in Denmark, with more than 30,000 students and over 800 staff. The College offers a range of undergraduate business programmes, graduate programmes and a range of adult education courses.
Formal arrangements
10 The BSc Computer Science and BA Business Administration franchises commenced in September 1998 and 1999 respectively, and are formalised in a consolidated Memorandum of Cooperation for Collaborative Delivery of Academic Programme(s) (the Memorandum). This Memorandum specifies the roles and responsibilities of both partners, incorporates termination and dispute settlement arrangements and is enforceable under English law. While noting that the Memorandum did not appear to address all relevant precepts in QAA's Code, the audit team learnt that these matters were covered in other formal documents prepared by the University.
11 It is University policy that such memoranda have a maximum duration of three years, but are renewable following triennial review by the attachment of a new financial annex. The present agreement with the College expires in August 2004. Triennial review entails a formal visit to all of the University's partner institutions in a specified geographical region by University representatives. Visiting panels are independent of the agreement under review. All panels have subject representation, but do not typically include external members. The University also conducts an annual financial review of all its collaborative partnerships.
12 The audit team was informed by staff of both institutions that extensive informal contact occurred, that any major difficulties in respect of quality or standards would be quickly addressed, and that there existed a well-understood referral route for problems beyond amelioration at operational level. From its enquiries, the team was able to substantiate that email contact between the partners was extensive. In addition, it was apparent that DMU staff made regular visits to the College.
13 The audit team noted that graduates of the College receive the same certificates as UK-based students, with the location and language of study (which is English) being stated on the accompanying transcript. With effect from the 2001-02 academic year, all certificates will make reference to the existence of the transcript, in line with the precepts of QAA's Code.
Responsibility for quality and standards
14 The Memorandum states that the University's Academic Board retains overall responsibility for the maintenance of standards and for the appointment of management boards and a Programme Representative 'with full authority to make decisions pertaining to the smooth administration of the programme'. In respect of the day-to-day management of the link with the College, the operational significance and personal enthusiasm of Programme Representatives (sometimes referred to as Programme Coordinators) were consistently communicated to the audit team by staff both of DMU and of the College. Responsibility for ensuring that the Memorandum is implemented lies with the relevant Dean at DMU.
15 The University's quality assurance and control framework involves a series of interlinked bodies, the principal elements being the Pro Vice-Chancellor (Learning and Teaching), acting both in an executive capacity and as Chair of the Academic Quality and Standards Committee (AQSC), a standing committee of the Academic Board, and the Academic Planning Sub-Committee for Collaborative and Overseas Provision (APSCOP). While APSCOP is responsible for vetting the legal and financial status of prospective partner institutions, the AQSC is responsible for the maintenance and enhancement of quality and standards. The duties of these bodies are undertaken with reference to the Curriculum Approval, Review and Monitoring Handbook and the guidance notes on the Development and Approval of Overseas Partnerships and Provision. Both of these documents appeared to the audit team to offer appropriate guidance to both University staff and staff in partner institutions. At faculty level, faculty quality assurance committees (FQACs) report to the AQSC on the quality and standards of all faculty provision and are responsible for ensuring intra-faculty dissemination of good practice and for coordinating the work of all Subject Authority Boards (SABs) within their faculty. Careful examination of the minutes of FQACs did not suggest to the team that collaborative arrangements generally constituted major items of business for these committees. On its visit to DMU, the team noted with interest that the franchise with the College was mentioned only briefly in a very limited number of FQAC minutes and, accordingly, wondered whether this level of formal monitoring was adequate in view of the stated importance of the partnership to the University. The University will wish to consider whether FQACs are uniformly fulfilling their responsibilities to monitor the quality and standards of all provision. The team was advised that SABs convene in either 'management' or 'assessment' mode. In the former they have operational responsibility for programme management and quality assurance within broad discipline areas, and have student members in attendance. In addition, the Faculty of Business and Law has established an Off-Site Provision Committee, which is a sub-committee of the FQAC. In both modes, SABs report to their respective FQAC, and thence through the AQSC to the Academic Board.
16 In terms of executive responsibilities, each faculty has a Head of Quality who is charged with advising on University policy, ensuring that agreed policies are implemented correctly and who also acts as a conduit between the faculty and central University functions. While the Deans chair their FQACs, the Heads of Quality are deputy chairs and are primarily responsible for shaping the agendas and advising on quality policy. The Heads of Quality are members of the AQSC and meet periodically on an informal basis in order to promote inter-faculty consistency and good practice. In respect of collaborative activities, all faculties additionally have a Faculty Collaborative Coordinator, and each programme has an Overseas Programme Coordinator as well as several module leaders. Overseas Programme Coordinators, nominated by their SAB, have operational responsibility for programme management, including quality management and enhancement and the maintenance of standards.
17 The audit team's attention was drawn to the use made of subject and course journals. The University's Commentary noted that subject journals were 'used by subject teams to record and monitor issues relating to the management of courses falling within the authority of the subject'. While subject journals are collectively evaluated by the University's Quality Assurance Division they are not, so far as the team was able to ascertain, systematically analysed in relation to overseas programme delivery. Course journals are used by collaborative partners 'to report on their own evaluation of the standards and quality of the delivery of the DMU provision in their location'. The course journal is analysed by DMU staff and incorporated into the appropriate subject journal. Examples of such subject journals were made available to the team. While noting that they did not contain any significant discursive or self-reflective dimension, the team considered the journals to be a potentially useful and task-oriented management tool for identifying and resolving operational issues. Staff at DMU spoke positively of the system and the team noted that the Quality Assurance Division had prepared a helpful document addressing the way in which subject journals might be used in relation to the University's overseas provision. The document did, however, observe that the experience of using journals in relation to overseas provision had been 'variable'.
18 During its visit to the College, the audit team learnt that, at College level, responsibility for monitoring the quality and standards of the provision offered in collaboration with DMU resided with the Programme Leaders. The Programme Leaders met by the team described their coordinating functions and outlined the liaison that took place with their counterpart programme coordinators at DMU. The team also learnt that in most instances there is frequent contact at module tutor level between DMU staff and colleagues in Copenhagen. In discussion with teaching staff at the College, it was suggested to the team that it was not common practice for such staff to be given the opportunity to make an input to the course journal. Similarly, the team could find no evidence of systematic feedback being given to teaching staff on the responses made by DMU in relation to important quality and standards issues. On the basis of the evidence available to it, the team would wish to encourage the University to consider further how course and subject journals might be used to promote the maintenance and enhancement of standards of programmes offered on a collaborative basis.
19 During its meetings with staff at the College, the audit team received repeated assurances that the reporting lines within the College were clearly established, with the Programme Leaders taking overall operational responsibility for effective delivery of the courses offered in collaboration with DMU. The team formed the view that there was a high level of interaction and communication, particularly at an informal level, between staff at the College. Matters of import identified for communication to DMU were channelled through the Programme Leaders, with more routine matters typically being communicated directly between module tutors in Copenhagen and in the UK. The team noted that there was not a general requirement for all matters to be routinely reported to the Programme Leaders and, as a consequence, they did not have a fully accurate log of issues arising at the module level.
20 In reviewing the University's systems for quality management and enhancement, the audit team was clear regarding the responsibilities of the Academic Board and the Pro Vice-Chancellor (Learning and Teaching) in relation to the operation of collaborative programmes. The team was, however, less certain as to where the responsibilities of programme coordinators, chairs of SABs, faculty collaborative coordinators, faculty heads of quality and the Head of the Quality Assurance Division began and ended, and whether these demarcations were cross-institutionally consistent. This apparent lack of operational clarity appeared to the team to be a consequence of the University's move towards devolving a number of administrative functions. While acknowledging the existence of centrally defined policy objectives, the team nevertheless considered that the University's present infrastructure arrangements could lead to duplication and omission, and, in the case of collaborative arrangements, to uncertainty among the staff of collaborating institutions. The University will wish to review carefully whether scope exists for simplifying and clarifying its quality maintenance and enhancement infrastructure, having particular regard to the University's interface with collaborating institutions.
Marketing and publicity
21 In its Commentary, DMU was clear that responsibility for marketing and publicising its courses resides with the University, and that 'enhanced protocols' had been put in place in September 2001 following restructuring of the Marketing Division. The University requires that publicity materials emanating from collaborating institutions are submitted for prior approval, with spot checks being undertaken subsequently on the collaborating partner's web site to ensure that materials accord with DMU protocols. The audit team was made aware of a recent review of the College's web site undertaken by DMU. The review report noted that references to DMU were 'by no means negative' and that 'all comments [were] positive', but contained no recommendations on how improvements might be made to enhance the presentation of DMU's collaborative partnership. During its visit to the College, the team was assured that all publicity material bearing the University's logo was subject to initial approval by DMU and, furthermore, that it would not be permissible for material advertising the franchised courses not to indicate that they were DMU courses. The team was, therefore, interested to note an advertising sheet for an open evening at the College (published in Danish) which listed the BSc and BA franchised programmes without any reference to DMU. While accepting that this might have been an isolated incident, the University will nevertheless wish to keep under careful review the mechanisms it uses to approve and monitor the publicity materials produced by partner organisations, in accordance with University policy.
Quality of learning opportunities and student support
22 The University's Commentary noted that the institution was committed to ensuring education 'of appropriate and comparable quality and standards' for all its students, irrespective of location. The audit team was advised on its visit to DMU that College students completed identical feedback questionnaires to those available to their UK-based counterparts, and that data was analysed and acted upon in exactly the same way. Although there is no student representation system at the College, the team learnt that monthly review meetings take place between staff and students, with any matters of import being communicated to the Programme Coordinator who 'indirectly' represents the students on the relevant SAB. Drawing attention to the mature and assertive character of the students, different cultural attitudes towards student representation, and the fact that issues of concern would be recorded in course journals (see above, paragraph 17), DMU staff defended this procedure as effective in practice, whatever its structural limitations.
23 In discussion with staff and students at the College, the audit team was able to confirm that a student questionnaire system was in operation, but that not all students participated actively. The team learnt from students that completion of questionnaires is encouraged but not mandatory and that the highest response rates are typically achieved when the questionnaires are distributed to a class and then collected at the end of the teaching session. While stressing the ready access they had to teaching staff, students indicated that they did not always receive timely and appropriate feedback on issues they had raised and, accordingly, suggested that improvements could usefully be made to student feedback systems. To this end, the team could find no evidence that student issues were systematically incorporated into the course journal which, as noted previously (see above, paragraph 17) the University regards as being one of the key elements in the monitoring and assurance of standards. While students were, overall, broadly satisfied with their educational experience on the franchised courses, they expressed the view that they would welcome enhanced opportunities for direct contact with staff at DMU. The University will wish to give further consideration to the effectiveness of its systems for securing and then responding to feedback received from its students studying in overseas locations.
24 During its visit to the University, the audit team learnt that the normal International English Language Testing System requirement for on-campus students is waived for Danish students at the College since, having completed earlier teaching in English as part of their school education, they are deemed to have sufficient linguistic competence. The team noted, however, that a number of non-Danish students, especially those from China, had experienced learning problems, particularly with unseen examinations. These were attributed by the Programme Leader at the College to 'very poor linguistic capabilities' in spite of the fact that they had previously been studying in English and had previously strong marks in course work. The team did not have the opportunity during its visit to the College to meet any of the Chinese students who had been experiencing language difficulties. College staff demonstrated a clear awareness of the linguistic difficulties affecting the group of Chinese students (which currently comprise some 35 per cent of the College's DMU student cohort), although no action had been taken to offer additional English language instruction for the students in question. Reviewing the evidence available to it, the team would wish to encourage the University to give further careful consideration to the mechanisms used to assess the English language competence of prospective students, the ways in which the performance of students with acknowledged language difficulties are monitored and the remedial support provided to such students.
Liaison and administration
25 The University's Commentary outlined the various mechanisms employed by both institutions to secure up-to-date information on the operation of the collaborative partnership including, for example, staff visits and email communication. During its meetings with College staff, the audit team learnt that DMU staff made visits to Copenhagen at least twice each year in order to discuss matters relating to course delivery and to meet teaching and administrative support staff at the College. College staff regarded such visits as comprising key elements in the way in which DMU evaluated the operation of the franchised courses. From the evidence made available, the team was, however, unable to determine the extent to which such visits were used to evaluate the strategic development of the partnership and the recurrent themes emerging either from course monitoring or from external examiner's reports. It did not appear to the team, for example, that there was a direct link between the visits and the evaluative function of the course journals.
26 It was evident to the audit team, both from discussions at DMU and the College, that DMU staff visited Copenhagen more frequently than College staff undertook reciprocal visits to Leicester. The team was advised that when selected College staff had visited DMU, little attempt had been made for their visits to coincide with relevant meetings so that College staff could experience the University's deliberative or examining structures at first hand. College staff indicated that they would have welcomed developmental opportunities of this type. The University will wish to give this matter further consideration.
Monitoring
27 As indicated previously (see above, paragraph 11), formal monitoring takes place on a triennial basis by a review team external to the programme but not to the University. The University's Commentary indicated that the approach of reviewing geographically cognate collaborations offered benefits in terms of both effectiveness and economy, and it was noticeable that, like the earlier 1997 review, the most recent review conducted in 2000 had highlighted a number of cross-institutional issues. The audit team was, however, surprised to learn that such reviews lacked institutional externality. While accepting that strategic development rather than quality control issues constitute the principal focus of such review visits and that external examiners' reports would be available to panel members, the team nevertheless wondered whether, at the very least, a formalised dialogue between review panels and external examiners would helpfully augment an interesting and worthwhile process.
28 The audit team noted with interest that the timing of the University's European triennial review visit was somewhat discordant with the approval period as set out in the Memorandum with the College, such that the recommendation of the March 2000 visiting panel for a three year renewal of the Niels Brock franchise became operational only in August 2001. As a consequence, the University's authorisation of the collaboration partnership presently expires some four-and-a-half years after the March 2000 review visit, creating a near 50 per cent increase in the effective approval period over the formal maximum. Such an arrangement was, in the view of the team, especially significant in the case of the University's collaboration with the College, since the European review authorised a new (part-time) mode of study for the BSc Computer Science for, in effect, a five-year period. The team, while appreciating the arguments advanced by DMU staff in support of present practice, formed the view that synchronising the process triennial review and the renewal of the Memorandum would be both achievable and beneficial to the integrity of the collaborative partnership.
29 Reviewing the evidence available to it, the audit team wondered whether the present formulation of the University's journal and SAB systems offered a sufficiently robust and rounded evaluation of programme performance. More specifically, the team wondered whether a more self-reflective report from the collaborating institution, annotated as appropriate by the Programme Coordinator and the SAB Chair, might usefully be made available to the Quality Assurance Division and the AQSC.
Curriculum
30 In discussions with staff at DMU, the audit team learnt that, as a franchise arrangement, the same curricula and syllabi are taught in both Leicester and Copenhagen, as described in the module template. During discussions both with DMU and College staff, it emerged that there had been pressure from Danish students, particularly those studying on the BA Business Administration programme, for modifications to be made aimed at making their programmes of study more relevant to their interests and less specifically UK-focused. In acknowledging the potential benefits of minor programme modulations to suit local circumstances, the team noted that any such changes might, for example, have implications in respect of examination arrangements and the way in which College students might legitimately market themselves to prospective employers in the UK.
31 During its discussions with students at the College, the audit team was informed that, in the students' view, there had been an occasion where there had been a serious mismatch between the taught content of a module and the subsequent examination. The students were of the view that the matter was still under review by the University and the College.
Learning resources, student information and support
32 The Commentary provided by the University did not supply detailed information about the learning resources available to support the collaborative partnership with the College. Accordingly, the audit team was not in a position to undertake a meaningful comparison of the learning resources made available to students studying in Copenhagen and the facilities provided in Leicester. Nonetheless, in the course of its visit to the College the team heard repeated reassurances from staff and some students that the learning resources were appropriate.
33 In discussing student support related matters, the audit team was struck by the commitment of College staff to ensuring high levels of provision. Effective links have been developed by cooperation between the College and DMU library staff, resulting in, for example, the recent establishment of an on-line link to the DMU library catalogue. Students welcomed this development.
34 From the evidence made available, it was evident to the audit team that students at the College were working in a supportive environment where both formal and informal contacts between students and staff were valued and considered effective. A large majority of students commented positively on their overall learning experience and their working relationship with the College's teaching and support staff.
Staffing and staff development
35 The Commentary stated that staff development takes place on visits to Copenhagen by DMU staff and focuses on 'ensuring that Niels Brock staff understand DMU procedures and policies, discussing operating and communication issues, and course administration'. In learning of the communication enhancements that derived from such visits, it appeared to the audit team that the visits focused primarily on handling day-to-day issues of programme management. Reviewing the information made available to it, the team was unclear as to whether the University had a clear policy on its staff development responsibilities in respect of the employees at its partner colleges, and the extent to which these expectations were explored strategically at initial approval stage.
36 From the evidence presented to the audit team, it appeared that staffing matters have presented several challenges for the University. Cultural differences between assessment regimes and practices have occasioned much discussion and the College has also experienced relatively high staff turnover rates. In particular, the team noted that an Assessment Board minute from February 2001 referred to 'a complete change of personnel at [the Niels Brock] site', and was able to confirm with College staff that a completely new cohort of staff had been employed in February 2001 to deliver the BSc Computer Science course. This matter did not appear to have been identified through the University's Course Journal.
37 External examiner reports seen by the audit team suggested that the University's standards of assessment might not yet be fully inculcated among the College's teaching staff, a view supported by the fact that, although the collaboration has been in operation since 1998, a notable number of examination scripts from Denmark continue to be re-marked by staff at DMU. While sympathetic to the cultural differences and noting this particular approach to the maintenance of standards, the team wondered what impact any significant increase in student numbers at the College might have upon resources at DMU. The University will wish to consider ways of ensuring that College teaching staff are provided with appropriate development opportunities to enable them more fully to understand and operate to the University's stated examining norms. As noted previously (see above, paragraph 26), reciprocal visits by teaching staff might constitute one means by which such consistency could be secured.
38 The curricula vitae of College teaching staff are subject to the approval of the University, this responsibility residing jointly with the Programme Coordinator and SAB Chair. It appeared to the audit team that decisions are discretionary and are not criteria referenced. The team was informed during its visit to the College that while it is a requirement that staff possess a masters degree in order to teach on the DMU franchised courses, the University does not require a specified level of competence in English. Although both University and College staff felt that this was not in practice necessary, since all teaching is required to be in English, it did appear to be a slight anomaly to the team. The University will wish to keep this matter under review.
The assurance of the standards of the awards
Student admissions
39 The Commentary noted that admission to the BA Business Administration and the BSc Computer Science franchised courses is automatic for those students who successfully complete a relevant State diploma at the College, or who have completed an equivalent diploma at another Danish institution. Students who do not have a directly relevant diploma may be required to undertake further study prior to entry. The admission of these, or of other non-standard students, is at the ultimate discretion of the University. The audit team was informed during its visit to the College that the standard entry requirement was an average grade of 8 or higher in the Danish State Diploma (on a scale of 1 to 15; 15 being the highest). On checking a number of student records, however, the team noted that a number of students had been admitted with average grades of 7 or lower. It was unclear to the team whether these students had been admitted to the relevant DMU programmes after the exercising of due discretion by the University. The University will wish to keep this matter under careful review.
40 The audit team learnt that students receive a letter from the University admitting them to the relevant programme. The letter of admission will, in the near future, also include the students' registration number to enable them to access the remote log-on to the University's on-line library catalogue (see above, paragraph 33).
Assessment
41 The University's Commentary advised that academic standards at the College 'are directly comparable to the programmes taught at Leicester', that College staff are encouraged to 'participate in the development of assessments, through direct and email contact with module leaders at Leicester', that the administration of assessment was identical in the two institutions, and that assessed work was marked at the College with an agreed sample of the assessments being 'despatched to Leicester for moderation and consideration by the SABs and their respective external examiners'. Reviewing the evidence presented to it, the audit team considered that these claims were justified. While, overall, reassured by the mechanisms that had been put in place by the University and the College to secure student assessment arrangements, a number of matters were drawn to the attention of the team.
42 As noted previously (see above, paragraph 37), cultural differences have resulted in DMU staff having to undertake re-marking of student work in order to ensure that differential standards were not being applied to College students. Several external examiners' reports attributed this to 'confusion' on the part of College staff. The audit team additionally noted the comment of an external examiner that '[College] students...[tended to produce] more superficial and descriptive answers'. Nonetheless, the team was persuaded in discussion with both DMU and College staff that, overall, the contribution the students made in class was considerable, and that the work they produced was, for the most part, satisfactory.
43 In noting that external examiners have been consistently complimentary about the administrative efficiency and professionalism of both DMU and College staff, the audit team identified some key themes emerging from external examiners' reports. In some instances, it was unclear to the team whether these matters had been speedily or effectively addressed in spite of the fact that external examiners, as well as DMU staff, had undertaken visits to the College. By means of example, the team learnt of a complaint made by College students regarding the perceived inadequacy of the time given to answer a series of multiple-choice questions in a formal examination. In discussing the matter, the external examiner had questioned whether the use of multiple-choice questions constituted a suitable assessment methodology for a level 3 module. The team noted that the issue had been debated by the Marketing SAB and that the SAB had felt confident that multiple-choice was appropriate for one module at level 3. Neither was it evident as to why the resit paper had been significantly different in structure from the original paper and whether students had been prepared for this. Importantly, it was not clear to the team how the University had concluded its discussions on this matter.
44 During its visit to the College, the audit team learnt that it is not standard practice for teaching staff to see the University's external examiner reports or for them to receive systematic feedback on the matters that had been raised by external examiners. Notwithstanding the fact that it is possible for visiting DMU staff to draw attention to matters that might have been raised by external examiners, the University will wish to consider whether the systematic copying of external examiner's reports to College teaching staff might usefully fulfil both a staff development and quality assurance dimension.
45 The audit team also took the opportunity to discuss the operation of assessment procedures with students at the College. Students drew attention to differences in the time taken for coursework to be returned to them with the definitive final mark. In some instances, it appeared to the team that delays could be considerable, with some students not receiving their final examination grades until after they had commenced their next suite of modules. While University policy permits College tutors to provide indicative feedback on summative assessments, the students met by the team were not aware that provisional feedback could be given and expressed the opinion that they were not readily able to judge their performance and progress. The University will wish to review this matter.
Conclusions
46 The collaborative provision considered in this report comprises two franchises between De Montfort University (the University) and the Niels Brock Business College, Copenhagen (the College). The College is a large and well-established self-governing institution recognised by the Danish Government as offering business education to diploma level. The College was founded in 1880, its name commemorating Niels Brock, a wealthy merchant who was one of the founders of the Danish vocational business education movement. It is the largest business college in Denmark, with more than 30,000 students and over 800 staff. The College offers a range of undergraduate business programmes, graduate programmes and a range of adult education courses. The collaboration involves delivery by the College of the third year of the University's BSc Computer Science and BA Business Administration programmes. The franchise arrangements were established during 1998 and 1999 and are presently due for review before August 2004.
47 The University's frameworks for managing collaborative activity have received favourable comment in previous QAA reports, with identified points for action receiving conscientious attention. In particular, the University has sought to enhance external examiner involvement with its collaborative programmes and has actively encouraged its examiners to undertake site visits. From the evidence made available, the structures underpinning the University's collaborative partnership with the College are compliant with QAA's Code of practice. As it continues to develop and enhance its collaborative provision arrangements, the University will wish to give further consideration to a number of matters arising from the scrutiny of the partnership with the College.
48 The audit team would wish to commend the University on its strategic approach to triennial reviews. The institution will, however, wish to consider ways in which the timetable for such reviews may be harmonised with the period of programme approval granted to its partner institutions, thereby ensuring that collaborative programmes of study do not receive authorisation for periods in excess of the University's formal three-year maximum. The University may additionally wish to consider whether the activities and outputs of triennial review panels would be enhanced through either the addition of members external to the University or through undertaking discussions with its external examiners prior to the commencement of the review visits.
49 Both an external examiner and an internal commentary have drawn attention to the potential benefits of the University adopting a more proactive and strategic approach to the management of its overseas collaborative arrangements. In reviewing this matter, the University will wish to give further consideration to whether its quite complex, and sometimes seemingly fragmented, quality management and enhancement structure may benefit both from clarification and a stronger central control. While course journals have the potential to be a powerful mechanism in evaluating and enhancing the quality of provision, in respect of the University's partnership with the College, this potential does not yet appear to be fully harnessed. The journals produced by the College lacked sufficient critical detail and there was little evidence that effective feedback was being provided to all relevant College staff following their consideration through the University's quality assurance processes. The University will wish to consider how improvements might be effected to this important aspect of its monitoring processes.
50 External examiners have commented upon the scale and number of adjustments made to student marks by University staff following initial marking by College staff. In exploring this matter, it was evident that a number of College staff are not yet fully aware of the standards of assessment applied by the University. The establishment of a coherent staff development policy targeted at those staff teaching on the University's collaborative programmes might usefully address this, and other matters. At present, much of the responsibility for staff development is devolved to operational staff at the College whose main concerns are, quite properly, with the immediate delivery of the programme. While noting that the University has supported staff and external examiner visits to the College, the number of reciprocal visits made by College staff has been much lower. The University may wish to review the framework for the visits undertaken by University staff and the means by which their impact is evaluated.
51 Available evidence suggested that some students were experiencing difficulties in relation to the study and assessment format of the University's degree programmes. It appeared, for example, that the challenges of sitting unseen examinations might be considerable for students. Similarly, it seemed that a number of students were experiencing problems in being taught and assessed in English. The University will wish to keep these matters under active review.
52 Overall, it is considered that the University's collaborative arrangement with the College is fundamentally sound and that there can be broad confidence in the way that the University is exercising its stewardship of the quality and standards of its awards offered in association with the College. The present effectiveness of the partnership is due in no small measure to the commitment of both College and University staff and the highly motivated, relatively small cohort of students. The University will wish to reflect carefully on whether its current processes and procedures would be sufficiently robust to ensure an equally effective stewardship of a significantly larger scale partnership.
IRD 722 11/2002
Top