Introduction
1 The Quality Assurance Agency for Higher Education (QAA) is a UK organisation that seeks to promote public confidence that the quality of provision and standards of awards in higher education are being safeguarded and enhanced. It provides public information about quality and standards in higher education to meet the needs of students, employers and the funders of higher education. One of QAA's activities is to carry out quality audits of collaborative links between UK higher education institutions and some of their partner organisations in other countries. In the spring and early summer of 2002, QAA audited selected partnership links between UK higher education institutions and institutions in Denmark, Germany and Switzerland. The purpose of the audits was to provide information on the way in which the UK institutions were maintaining academic standards and quality of education in their partnerships with institutions in these countries.
The process of audit of overseas partnership links
2 In February 2001, QAA invited all UK higher education institutions to provide information on their collaborative partnerships. Using this information, QAA approached a number of institutions who had indicated that they had established collaborative links with Danish, German or Swiss partners. Following discussion, a variety of collaborative partnerships was selected for scrutiny. Each of the UK institutions whose collaborative link had been selected for the audit provided a Commentary describing the way the partnership operated, and commenting on the effectiveness of the means by which the UK institution assured quality and standards in the link. In addition, each institution was asked, as part of its Commentary, to make reference to the extent to which the link was representative of its procedures and practice in all its overseas collaborative activity or specific to the partnership being audited or country.
3 Audit teams visited the Danish, German and Swiss partner institutions to gain further insight into the experience of students and staff, and to supplement the view formed by the team from the institution's Commentary and from the UK visit. During the visits to Denmark, Germany and Switzerland, further documentation about the partnerships was made available to the team, and discussions were conducted with key members of staff, lecturers and students. The team comprised Professor R J Harris, Mr A T Davidson, Dr P D Hartley and Professor R H Bryant, auditors. The UK and overseas audit exercise was coordinated for QAA by Dr P J A Findlay and Dr C J Haslam, Assistant Directors, Institutional Review Directorate. QAA is particularly grateful to the UK institutions and their partners in Denmark, Germany and Switzerland for the willing cooperation provided to the teams.
4 Institutions were invited, in their Commentaries, to make reference to the ways in which their arrangements met the expectations of QAA's Code of practice for the assurance of academic quality and standards in higher education, Section 2: Collaborative provision (QAA's Code), which took full effect in August 2000.
5 This report describes the audit of the collaborative link between the University of Birmingham (the University) and the Missionsseminar, Hermannsburg, Germany (the Missionsseminar). The audit was conducted on the basis of visits by an audit team to the institutions concerned and on the scrutiny of documentary evidence made available by both the University and the Missionsseminar. A series of meetings were held on the 14 March 2002 at the University between the team and senior staff of the University and this was followed by a visit to the Missionsseminar on the 15 May 2002, when the team met with staff and students at the Missionsseminar.
6 The most recent QAA audit of the University at institutional level took place in 1999. The University's overseas collaborative arrangements have been the subject of one previous QAA audit, in 1996 (a partnership in Singapore). The University's theology and religious studies provision was 'quality approved' by QAA in 2001.
The background to the collaborative partnership
The collaborative partnership
7 The collaborative partnership that is the subject of this report comprises the validation by the University of a three-year programme of study at the Missionsseminar leading to the award of a BA with Honours (unclassified) in Theology and Mission Studies. The validated programme is entirely taught and assessed in German. The link commenced in September 1999 and was approved initially for four annual student intakes. The agreement is to be subject to formal review during the 2002-03 academic session, some 18 months before its formal expiry. Student numbers on the validated programme are presently small. At the time of the visit, a total of 10 students were enrolled on the programme.
The partner organisation
8 The Missionsseminar is a missionary college or seminary in Hermannsburg, a small village some 100 kilometres to the south of Hamburg. The Missionsseminar, which was set up under the aegis of the Evangelisch-Lutherisches Missionswerk in Niedersachsen, specialises in Lutheranism, missionary enthusiasm and agrarian romanticism. The organisation of the missionary activity itself has resulted in substantial library holdings and a renowned collection of archival materials. Typically, its graduates work as ministers or lecturers in the service of independent churches overseas.
9 The Missionsseminar has on average around 25 to 35 resident students, both female and male. The normal period of study at the Missionsseminar is six years full-time, and seven years for those who do not fulfil the required entrance qualification of the Abitur (the German equivalent to GCE A-Levels) (see below, paragraph 39). The University's BA with Honours (unclassified) in Theology and Mission Studies is awarded to students following successful study in three of the student's six years at the Missionsseminar. The Missionsseminar awards a 'First Theological Examination' which is recognised by the Protestant Landeskirchen as being equivalent to their 'Kirchliches Erstes Examen' taught by German universities. This recognition enables graduates of the Missionsseminar, after their return from overseas service, to be employed as regular pastors by the church.
The history of the collaborative partnership
10 The University was approached by the Missionsseminar in November 1997 concerning the possibility of developing a collaborative association. The Missionsseminar believed that a link with the University would help facilitate enhanced international recognition, assist in the establishment of a more clearly defined career structure and, as a consequence, enhance student recruitment. The University has established a number of long-standing collaborative arrangements with UK institutions in the area of theology (see below, paragraph 15) and staff from the University's Department of Theology explained to the audit team that the link with the Missionsseminar was seen as an extension of the institution's existing activities. The University itself does not offer an undergraduate award in the area of Mission Studies, but does have an MA degree and strong research base. University staff noted that the institution's initial interests in the link were fuelled by the possibility of recruiting some students from the Missionsseminar to undertake research in Birmingham, and the opportunities that would be provided for the University's Department of Theology to access the Missionsseminar's unique library and archive resources.
The University's approach to the management of collaborative provision
11 The University's Commentary stated that the institution's approach to collaborative provision is based on the view that the academic quality and standards of all awards made under a collaborative arrangement must be the same or comparable to those of equivalent awards for programmes delivered directly by the University. The University's policies and procedures in respect of collaborative activity are, in the view of the audit team, clearly set out in the institution's handbook External and Collaborative Programmes: Guidance on the Development and Approval of New Programmes. This document has been in existence for a number of years and was revised in 1999 to reflect the development of QAA's Code.
12 Management of the University's collaborative links is undertaken through the External and Collaborative Programmes Committee (ECPC). The ECPC, which was established in the 1998-99 academic session, is a subcommittee of the University's Academic Board and has overall responsibility for the oversight of quality and standards of the institution's various collaborative programmes. In its discussions with University staff, the audit team learnt that the ECPC has directed its attention to ensuring that partner institutions are of appropriate standing and to scrutinising programme proposals before their approval by the Academic Board. The Undergraduate Committee (of Academic Board) acts as a 'progress board' and, as such, receives examination results determined by school boards of examiners and formally determines progress decisions on individual students studying through collaborative programmes on the basis of school recommendations.
13 While all formal agreements are made with the University (see below, paragraph 17), the focus for the day-to-day management and monitoring of each collaborative programme is at the level of the school or department. Day-to-day management of the University's validated programmes is undertaken by 'visitors' appointed from the relevant University department and its departmental validation board. In the case of the University's collaborative partnership with the Missionsseminar, the visitor is appointed from the University's Department of Theology. Following the initial partnership approval process (see below, paragraph 18), the visitor is responsible for monitoring the continued suitability of the academic environment in the partner institution and for making initial recommendations to the relevant departmental validation board should any concerns be identified with the academic health of the collaborative programme.
14 The University's 2001 continuation audit report encouraged the University to strengthen its systems for monitoring the quality assurance and academic standards of its externally provided programmes of study. The Commentary noted that scrutiny of all of the University's collaborative provision activities is embraced within the institution's annual academic audit and quinquennial review processes. As part of this activity, visitors are required to produce an annual report on the collaborative programme for which they have responsibility that, together with the relevant external examiners' report, is forwarded to the University's Academic Office prior to onward transmission to the ECPC, the relevant dean of faculty and the relevant head of school (see below, paragraph 24). In addition, the audit team learnt that heads of school prepare annual school reports which make mention of off-campus teaching provision. The University made available exemplar information illustrating the work of the visitor appointed to oversee the collaborative programme established with the Missionsseminar. It was evident that, in addition to the formal annual report, a significant amount of informal and constructive interchange took place between the visitor and staff at the Missionsseminar.
The University's collaborative links in theology
15 At the time of the audit team's visit, in addition to the link with the Missionsseminar, the University validated two programmes delivered at two local church-based institutions providing professional training for ordained ministers and lay workers. The validated programmes offered through the church-based institutions are all academic programmes in theology. These academic programmes exist separately from the spiritual and personal formation and other elements that are required by the colleges involved, and the University is clear that it has no formal engagement with these vocational elements. The University also made clear in its Commentary that not all the students who attended the church-based colleges studied on the University's validated programmes and that the admissions requirements for the colleges were distinct from those for the validated programmes. In this regard, the Commentary noted that it would be atypical for students to register for its validated programmes who were not also students at the college, although this is technically possible. None of the church-based validated programmes is funded by the Higher Education Funding Council for England. For this reason, there is no publicity produced by the University that encourages students more generally to register for these programmes. The team noted that the publicity produced by the Missionsseminar drew attention to the University's validated programme only as part of its overall provision.
16 In its discussions with University staff, the audit team learnt that a considerable amount of work had been undertaken by the University to help ensure that there was congruence between the values, aims and objectives of the validated programme offered through the Missionsseminar and the broadly equivalent programme offered by the Department of Theology, especially in the area of academic excellence.
The University's formal agreement with the Missionsseminar
17 The collaborative partnership with the Missionsseminar is governed by a formal collaborative agreement that, in the view of the audit team, appeared generally robust. The team did, however, identify some possible anomalies in the agreement. Under the terms of the agreement, the University retains permanent records of student assessment and completion, and the Missionsseminar has responsibility for issuing student transcripts. While noting that the Missionsseminar has not yet been required to issue any student transcripts, from the available evidence it was nevertheless unclear to the team how the University might control the preparation and issuing of such transcripts. The agreement additionally includes the requirement for the Missionsseminar to be prepared to submit any information required by national or other agencies in the UK.
The establishment and management of the link
Approval process
18 As noted previously (see above, paragraph 10), the Missionsseminar initially approached the University to explore the potential for establishing a collaborative partnership. The University's Commentary observed that particular care had been taken to ensure that the degree programme that was proposed for delivery at the Missionsseminar was equivalent to those of the University's Department of Theology in terms of academic standard and quality of delivery. University staff advised the audit team that it had been recognised early on in the approval process that the academic culture of the Missionsseminar was notably different from that of the University and that a considerable amount of discussion would be needed to help staff at the Missionsseminar understand the processes and principles that would need to be put in place in order to underpin a programme validated by the University. The University accordingly agreed that it would be helpful for staff from the University to visit the Missionsseminar and, later on in the approval process, for a reciprocal visit to be made by a small group of staff and students from the Missionsseminar to observe the way in which programmes were run by the University's Department of Theology and to talk to relevant members of the Department and University. The team learnt from both University and Missionsseminar staff that during these visits specific training had been delivered by staff of the University on matters of academic quality, learning outcomes, modular structures, quality management procedures, student representation and complaints procedures. In addition, detailed subject-based discussions had taken place on the nature of theological education.
19 In the light of these initial interactions and related discussions within the Department of Theology regarding the appropriateness and feasibility of the University recognising the work of the Missionsseminar, in January the ECPC was alerted to the possibility of the University entering into some form of collaborative arrangement with the Missionsseminar. Following approval from the Chair of the ECPC to proceed with further discussions, the University assembled a panel of four academic staff (three of whom were fluent speakers of German) together with the administrative secretary of the ECPC. The panel visited the Missionsseminar in December 1998 and met with academic staff and students, reviewed teaching and learning support facilities, and reviewed the Missionsseminar's administrative infrastructure. The subsequent visit report indicated general support for the proposed collaboration and the Missionsseminar accordingly began work on preparing for the formal submission of a validated BA (Honours) programme to the University for approval. The formal proposal for the BA (Honours) programme was considered by the University in the Summer of 1999 by a review team comprising the Dean of Arts and Social Sciences and two other members of the Undergraduate Committee of the University's Academic Board. In October 1999, the Chair of the Academic Board gave approval, in principle, for the validation of an unclassified BA (Honours) programme. During 1999 and 2000, further development work took place between the University and the Missionsseminar on refining the programme proposal and on drafting a formal agreement. The formal agreement with the Missionsseminar was eventually signed in March 2001.
20 University staff advised the audit team that at various stages during the programme approval process suggestions for changes and alterations had been made. Each of the matters identified by the University had, in the view of the University, been dealt with promptly by the Missionsseminar. University staff additionally stressed that in all matters relating to modular structure, student load, assessment methods and appropriateness of learning outcomes to level, the BA (Honours) programme met the usual requirements of the University. Staff of the Missionsseminar spoke positively of the approval process, noting that it had been a challenging and intellectually stimulating exercise.
21 On the basis of the information made available to it, the audit team considered that the University had approached the approval of the collaborative partnership with due diligence and care.
Monitoring and review
22 Once the structure and content of the programme had been approved, the ongoing monitoring of the delivery of the programme became the responsibility of the visitor and the Validation Board in the Department of Theology. The Board meets three times a year in Birmingham and discusses matters of relevance to the three church-based colleges who run validated programmes in theology. Given the distances involved, the audit team learnt that the University does not require that representatives from the Missionsseminar attend all of these meetings, although all the agendas and minutes are sent to the Missionsseminar. One of the meetings of the Validation Board acts as an examinations board for the programmes from the two UK church-based colleges. The team was advised that it had been agreed that the examinations board for Hermannsburg should take place at the Missionsseminar, with the first examination board meeting being chaired by the visitor on behalf of the University (see below, paragraph 37).
23 All proposals for changes to the validated programmes in the Department of Theology are sent to the Validation Board in the first instance. The Board can approve minor changes to modules with the relevant details being passed to the School Undergraduate Committee and the University's Academic Board for information. Major changes to modules, or changes to the overall structure of the programme, are considered by the School Undergraduate Committee and then the Academic Board. The audit team learnt with interest that members of the Department of Theology's Validation Board had actively sought to make its church-based collaborative colleges aware of changes in procedures within the University and of national developments in relation to academic quality management, benchmarking and academic standards.
24 The audit team learnt that information submitted to the University annually on monitoring and review is expected to include two reports: firstly, a report from the partner institution of its own evaluation and monitoring and, secondly, a report from the University's visitor following a prescribed set of headings addressing both quality and standards. These complement the external examiner's report following the University's standard procedures and headings.
25 The Validation Board expects each of its collaborating institutions to undertake module evaluation at the end of each teaching period and to be involved in some form of programme review at the end of each academic year. The audit team learnt that, to date, the only monitoring report that had been made available to the Department of Theology's Validation Board in respect of the Missionsseminar collaboration had been that prepared by the visitor. University staff confirmed that while the production of an institutional annual report was the 'normal expectation', in the case of the Missionsseminar such a report had not yet been produced. It was suggested to the team that the bedding down of the University's expectations in relation to the assurance of standards and quality had taken the Missionsseminar 'a little time'. The team was, however, made aware that some staff at the Missionsseminar had sought student feedback on the programme (see below, paragraph 32).
26 Recognising the important role played by the University's visitor, the audit team explored in some detail the way in which the monitoring information produced by the visitor was used by the University in monitoring programme performance. In the view of the team, the visitor's report for the 2000-01 academic session tended towards brevity and 'compliance-type' responses. This was perhaps encouraged by the design of the report form. A number of the visitor's responses consisted simply of cross-references to the external examiner's report, which was appended only in the German language (see below, paragraph 27), and there was no reference to student views and feedback. In general, the team considered that the visitor's report provided little information in terms of 'what was going on' in the partner organisation. The University will wish to reflect further on this matter.
27 The audit team noted that the visitor's report for 2000-01 referred to an appended external examiner's report in German (see above, paragraph 26). The team noted with interest that a later English translation of the original German external examiner's report had omitted mention of concerns raised by the external examiner in relation to staffing resources, although the visitor's report did include a reference to the relevant numbered paragraph in the German version. The University will wish to ensure that, in the future, translation discrepancies are avoided and that the Validation Board receives English language versions of all external examiner reports (see below, paragraph 46).
28 Noting that the visitor was involved not only in acting as a critical friend to the Missionsseminar but also in the moderation of student marks and had chaired an Examination Board, the audit team explored with University staff the potential role conflict for the visitor. The University considered that there was not a conflict in the various roles of the visitor and that, in practice, the visitor was not solely responsible for the quality and standards of the collaboration. Particular attention was drawn to the important standards assurance role played by the external examiner (see below, paragraph 44).
29 On the basis of the information made available to it, the audit team considered that the University's formal arrangements for the establishment of the link were effective. However, the team noted that the very small scale and specialised nature of the collaboration, coupled with the fact that the programme is taught and assessed in German, have meant that only a relatively small number of University staff have been closely in touch with the operation of the programme at the Missionsseminar. Specifically, the three key players in the collaboration appear to be the University's visitor, the BA Programme Coordinator at the Missionsseminar and the external examiner. The team came to the view that this situation might create the potential for firstly, a blurring of support and moderating roles and, secondly, a personal closeness and familiarity that might potentially limit the University's ability to assure itself as to the rigour and effectiveness of the implementation of its own procedures. While wishing to emphasise that it saw no evidence of major problems, the team would nevertheless wish to encourage the University to reflect upon how it might best achieve a constructive balance between maintaining a supportive, close working relationship with a small partner and assuring itself regarding the quality and standards of provision in that partner.
Quality of learning opportunities and student support
Liaison and administration
30 The responsibilities and roles of each partner are clearly set-out in the formal agreement and in the University procedural document External and Collaborative Programmes: Guidance on the Development and Approval of New Programmes. Operationally, the key executive in the University procedures is the University's designated visitor. The Department of Theology has set up a separate Validation Board to oversee the collaborative provision in this area and the visitor reports to this Board. Based on its review of available documentation and discussions with staff during visits to both the University and the Missionsseminar, the audit team considered that lines of communication and reporting between the University's visitor and the key contact in the Missionsseminar, the BA Programme Coordinator, were clearly defined and were functioning effectively.
Monitoring and review
31 The general procedures for annual monitoring and review have been discussed previously (see above, paragraphs 22 to 29).
32 The audit team was advised that the support extended to the Missionsseminar by the University in approving the programme included provision of templates for its standard end-of-module student evaluation questionnaires. During discussion, the team noted that staff at the Missionsseminar were unsure about the use of student evaluation questionnaires (for example, whether they were compulsory or advisory), and what they should do with any results. Discussion with both staff and students indicated that neither group considered questionnaires to be appropriate to their close-knit community. The team learnt that existing practice regarding monitoring and review reflected the context and culture of the Missionsseminar, and was based on collective discussion rather than formal procedures with written reports. These discussions took place in two types of 'Seminar Conference', one involving all staff and students and one involving predominantly teachers but with student representation. Based on its discussions with staff and students at the Missionsseminar, the team considered that these arrangements were appropriate for the culture and context of the institution. However, the team did note that no internal documents or reports resulting from these local reviews appeared to be submitted to the relevant body within the University, either by the Missionsseminar or by the University's visitor. Taking into account earlier observations (see above, paragraphs 25 to 29), the University will wish to reflect upon how it might satisfy itself regarding the effectiveness of its annual evaluation mechanisms in ways that respect the culture and traditions of the Missionsseminar.
33 The University's policies and procedures require periodic formal review of all collaborative links at a specified time interval, which varies from between two and six years. The University's Commentary stated that the first review of the operation of the collaboration with the Missionsseminar was scheduled for 2002-03. Discussion with staff at the Missionsseminar gave the audit team the impression of some confusion about the significance of annual programme review as distinct from the annual meeting of the Examination Board. However, the team was advised that staff at the Missionsseminar had started to identify issues for consideration in a review of the programme. The University will wish to consider how it can best support and manage the programme review in the coming academic session.
Staffing and staff development
34 The University's policies and procedures require consideration of the prospective partner's staff resources as part of the approval process. In this context, the small size of the Missionsseminar is significant and the audit team noted comments in the external examiner's report (German language version) for 2000-01 raising concerns about the staff complement to support the width of the full curriculum. The University will wish to regard this particular matter in its planned review of the link.
35 The University's policies and procedures also encourage 'some form of staff review and development' in the partner, noting that the visitor may be able to play a useful role 'in advising the management and individual staff of the partner institution on issues of professional development'. During its discussions with staff from both the University and the Missionsseminar, the audit team noted the significant extent of staff development support provided by the University during the approval and initial development phase, and the considerable value attached to this support by staff at the Missionsseminar (see above, paragraph 18). The team additionally noted the wish expressed by the BA Programme Coordinator at the Missionsseminar for ongoing staff development support and increased contact with staff in the same subject at the University. The University will wish to consider how it might support the continuing development needs of staff at the Missionsseminar.
Provision of information to, and support for students
36 The Commentary noted that the University had provided the Missionsseminar with guidance on the type of information normally provided for students and had supported the development of a student handbook for the validated programme. A copy of this student handbook, written in German, was made available to the audit team during its visit to the Missionsseminar. The handbook provided details of the curriculum and general information for students. Students at the Missionsseminar advised the team that they found the handbook to be helpful and appropriate.
Assurance of the standards of awards
The University's approach to assuring standards
37 The University's approach to collaborative provision is based on the view that academic standards of all awards made under a collaborative arrangement must be comparable to those of equivalent awards for programmes delivered by the University (see above, paragraph 11). The key elements within this approach are: approval of the intended standards as part of the formal approval process; constitution of a formal Examinations Board in the partner organisation; routine moderation of the assessment and examining process by the University's visitor, including attendance at an annual meeting of the Examinations Board; appointment by the University of an external examiner; and the formal reporting of assessment results to the Validation Board.
38 The University has prepared a detailed guidance note on the requirements and role of the visitor. The audit team considered this to be a helpful and appropriate guide, clearly stating the visitor's responsibilities for monitoring standards. Importantly, the University recognises that the visitor must be appropriately qualified and experienced and must be allocated sufficient time to perform the role alongside their other University duties. Having regard to the University's guidance, the team noted that the visitor's report for 2000-01 did not include comments on internal moderation. This may be attributed to the template for the University's visitor's report form, which does not require any explicit comments to indicate firstly, that the visitor undertook internal moderation and, secondly, what the conclusions of any internal moderation were. The University will wish to consider the extent of information regarding internal moderation that it requires in order to assure itself of the standards of its collaborative provision.
Admissions
39 The University's Commentary indicated that consideration of admissions criteria had been included in the approval process. These were based on the German equivalent to GCE A-levels (the Abitur). It was additionally noted that the Missionsseminar offered a preliminary year for applicants who lacked these entry requirements. In discussion, the audit team learnt that the University's visitor monitored the Missionsseminar's student admission arrangements.
Language of delivery and assessment
40 The programme at the Missionsseminar is taught and assessed entirely in German. The University's policies and procedures set out in its handbook External and Collaborative Programmes: Guidance on the Development and Approval of New Programmes recognise the greater challenge to quality management associated with programmes delivered in a language other than English, and set out specific requirements for the justification and approval of such proposals. Documentation submitted by the University enabled the audit team to confirm that the University followed these requirements during the approval process and that it was aware of the implications associated with the programme being delivered and assessed in German. In discussion with staff during the visit to the University, the team was advised that the current visitor was a native German speaker and, additionally, that there were a small number of German-speaking members of staff in the Department of Theology.
41 In discussion during the visit, staff from the Missionsseminar confirmed that although proficiency in the German language was considered in the student application process, there were no defined criteria regarding German language competence. Missionsseminar staff additionally advised that they hoped to increase the number of non-German students in the future. The University will wish to ensure that its admissions criteria to the programme explicitly have regard to expected language competence, thereby enabling students to commence their studies effectively.
The assessment of students
42 The University's Commentary advised that assessment and progression regulations are the same as those required for all other undergraduate degrees within the University. The audit team learnt that staff at the Missionsseminar accordingly had to shift from a grade-based scale of assessment routinely used in Germany to the University's percentage-based scale of reporting. Based on scrutiny of the student handbook and discussions with staff and students at the Missionsseminar, the team noted that both scales were used operationally, but that all results were reported to the University in its standard percentage scale. Students advised the team that they considered they had appropriate information regarding assessment criteria and procedures.
External examiners and examination
43 During meetings with the audit team, staff at the Missionsseminar repeatedly indicated that the Missionsseminar had no tradition of external examining in the UK sense. Provision at the Missionsseminar had been subject to scrutiny by local churches, but this had related to curriculum content and church concerns rather than specific moderation of academic standards. Staff at the Missionsseminar advised that while they had found their initial engagement with the University's external examiner 'a shock', they had been prepared to listen and to respond constructively to the developmental comments made by the external examiner.
44 The University's Commentary explained the processes of appointing the external examiner for the validated programme at the Missionsseminar. The University expects the Missionsseminar to nominate an appropriate external examiner; this nomination is initially considered by the University's visitor who then makes a recommendation to the Head of School at the University. The University, under its normal procedures, then formally appoints the external examiner.
45 Evidence presented to the audit team suggested a somewhat confusing picture regarding how the University assured itself that the external examiner conducted moderation with respect to UK reference points for academic standards. Documentation describing the University's procedures at the time of the approval of the validation clearly recognised the need to relate standards to those within the University, noting additional concerns if the programme was delivered and/or assessed in a language other than English (see above, paragraphs 40 and 41). These included a requirement that 'the visitor should discuss...whether any additional procedures or training are required for the external examiner(s) to ensure that quality and standards are assessed appropriately and match those of a similar or identical programme delivered and assessed here'. However, documentation regarding the actual approval process for the validated programme at the Missionsseminar included the comment to the Missionsseminar regarding the external examiner that 'familiarity with the British system is not required (although there would be no problem if such familiarity did exist)'. Discussion with staff at the Missionsseminar suggested that the current external examiner had no personal experience of higher education in the UK. The team noted that the first external examiner's report (for the 2000-01 academic session) had included a response to a question regarding the consistency of assessment with national standards that the question was 'not applicable, because there are no comparable German programmes'. Having regard to QAA's Code: Section 4: External examining, the University will wish to reflect further upon the ways in which it assures itself that the moderation of standards undertaken by non-UK external examiners has regard to British norms and expectations.
46 As mentioned previously (see above, paragraphs 26 to 27), the audit team noted an additional language related concern associated with external examining in that two external examiner's reports had been submitted to the University covering the 2000-01 academic session. A copy in German, dated August 2001, had been appended to, and cross-referenced in, the visitor's report for 2000-01. An English version, dated December 2001, had been submitted formally to the University and copied to the Dean and School. The team noted some important discrepancies between the reports (see above, paragraph 27). The University will wish to consider, firstly, how it ensures completeness of translation and, secondly, how it ensures that an English language version is available for discussion at all stages of its monitoring and reporting processes.
47 A more general issue that the University may wish to consider in its management of standards in a programme taught and assessed in a language other than English relates to the potentially limited number of visitors and external examiners who are academically qualified in the subject and have appropriate language proficiency.
Conclusions
48 The validation by the University of Birmingham (the University) of the BA (Honours) in Theology and Mission Studies at the Missionsseminar, Hermannsburg (the Missionsseminar) is a relatively recent initiative, commencing in 1999-2000 and with the first cohort of students due to complete at the end of 2001-02. It is small in terms of student numbers, commencing with an intake of five students in 1999 and with a current (2001-02 academic session) total of 10 students enrolled on the validated programme. The two partner institutions are very different in terms of size, mission, culture, tradition, experience of collaborative activities and approaches to quality assurance. The University is a large, research-led institution with a significant range of collaborative activities for which it has appropriate and clearly defined policies and procedures. The Missionsseminar is a very small, specialised and close-knit institution with a very specific mission linking academic, spiritual and vocational concerns. Staff in both partners have recognised the significant differences in tradition and approach to teaching and quality assurance in higher education in the UK and Germany. Staff in both institutions were open and candid in their discussion of the development of the collaboration. Both groups considered that they had embarked 'on a journey' and that significant achievements had occurred. Without exception, they appeared committed to working together constructively to develop further the collaboration.
49 The findings of this audit broadly confirm the University's view of the link as set out in its Commentary. The University's policies and procedures for the approval and management of validated programmes are appropriate, clearly defined and take account of QAA's Code of practice. The University devoted considerable resources to the approval process, including significant development support to staff of the Missionsseminar to help them understand better both the University's and the British higher education system's approach to the management of quality and standards. Staff at the Missionsseminar have valued the support offered by the University and have demonstrated a commitment to fulfilling their responsibilities and a willingness to effect improvements to the collaboration.
50 As it develops its management of this collaboration the University may wish to consider a number of matters: the extent to which monitoring, moderation and reporting are explicit and specifically related to equivalent reference points in UK higher education; the desirability of making monitoring and moderation less dependant on a very small number of individuals; and how the University can monitor and contribute to the development of the staff base at the Missionsseminar. A further series of issues for consideration relate to the circumstance of a validated programme being delivered and assessed in a language other than English: specification of admission requirements covering expected proficiency in the language of teaching and assessment; the size of the pool of potential University visitors and external examiners who are proficient in both writing and speaking in relevant languages; the completeness of translations and the control of documents in order to guard against unintentional changes in content or meaning.
51 The collaborative link is still at an early stage in its development and, accordingly, the evidence base is somewhat limited. However, careful evaluation of the available information indicates that there can be broad confidence in the University's stewardship of the quality and standards of the audited link.
IRD 721 11/2002
Top