Introduction
1 The Quality Assurance Agency for Higher Education (QAA) is a UK organisation that seeks to promote public confidence that the quality of provision and standards of awards in higher education are being safeguarded and enhanced. It provides public information about quality and standards in higher education to meet the needs of students, employers and the funders of higher education. One of QAA's activities is to carry out quality audits of collaborative links between UK higher education institutions and some of their partner organisations in other countries. In the spring and early summer of 2002, QAA audited selected partnership links between UK higher education institutions and institutions in Denmark, Germany and Switzerland. The purpose of the audits was to provide information on the way in which the UK institutions were maintaining academic standards and quality of education in their partnerships with institutions in these countries.
The process of audit of overseas partnership links
2 In February 2001, QAA invited all UK higher education institutions to provide information on their collaborative partnerships. Using this information, QAA approached a number of institutions who had indicated that they had established collaborative links with Danish, German or Swiss partners. Following discussion, a variety of collaborative partnerships was selected for scrutiny. Each of the UK institutions whose collaborative link had been selected for the audit provided a Commentary describing the way the partnership operated, and commenting on the effectiveness of the means by which the UK institution assured quality and standards in the link. In addition, each institution was asked, as part of its Commentary, to make reference to the extent to which the link was representative of its procedures and practice in all its overseas collaborative activity or specific to the partnership being audited or to the country concerned. Institutions were also invited, in their Commentaries, to make reference to the ways in which their arrangements met the expectations of QAA's Code of practice on the assurance of academic quality and standards in higher education, Section 2: Collaborative provision (1999) (QAA's Code), which took full effect in August 2000.
3 In the spring of 2002, audit visits were made to each UK institution to discuss its arrangements in the light of the Commentary. In May 2002 audit teams visited the Danish, German and Swiss partner institutions to gain further insight into the experience of students and staff, and to supplement the view formed by the team from the institution's Commentary and from the UK visit. During the visits to Denmark, Germany and Switzerland, further documentation about the partnerships was made available to the team, and discussions were conducted with key members of staff, lecturers and students. The team for this audit comprised Mr A T Davidson, Professor R Pearce, and Professor M Shaw, auditors. The UK and overseas audit exercise was coordinated for QAA by Dr P J A Findlay and Dr C J Haslam, Assistant Directors, Institutional Review Directorate, QAA. QAA is particularly grateful to the UK institutions and their partners in Denmark, Germany and Switzerland for the willing cooperation provided to the teams.
4 The most recent QAA audit of the University's predecessor (Cheltenham and Gloucester College of Higher Education) took place in 1995. Following a process of scrutiny and recommendation from QAA, the Privy Council confirmed university status in 2001 and the College adopted the title of University of Gloucestershire.
5 This report describes the audit of the collaborative link between the University of Gloucestershire (the University) and the Freie Theologische Akademie in Giessen, Germany (FTA). The audit was conducted on the basis of the visits by the audit team to the two institutions and on the scrutiny of documentary evidence made available both by the University and FTA. A series of meetings was held on 4 March 2002 between the audit team and senior staff of the University and this was followed by a visit to FTA on 13 May 2002 when the team met with staff and students.
The background to the collaborative partnership
6 In June 2000 the University validated a one-year full-time postgraduate programme leading to the award of MA Biblical Studies, to be delivered at FTA (the award title for the programme was subsequently modified to MA in Biblical and Theological Studies). FTA is a private academic institution founded in 1974 to provide an alternative theological education to that provided by the state university system in Germany. It offers an interdenominational four-year programme which provides an academic and professional preparation for various forms of Christian ministry. There is a particular emphasis in the work of FTA on biblical studies. The overall programme provided by FTA does not lead to any formal award recognised within the German state framework of educational awards, but it is considered by the European Evangelical Accreditation Association (EEAA) to be the equivalent of the degree of Master of Divinity. FTA has around 120 students, most of whom are registered on its four-year programme, and 11 full-time teaching staff, together with associate faculty and visiting tutors and a small professional administration. There is a strong research ethos, all of the full-time staff are research active, and most hold doctorates. The validated MA constitutes a fourth year of study following on the first three years of FTA's own four year programme, and it is both taught and assessed in the German language. Nine students were admitted to the programme in October 2000, all of whom graduated the following September. A second cohort of eight students was admitted in October 2001.
7 Through its discussions with the two institutions, the audit team understood that FTA saw a link with a UK institution as reinforcing its international European connections and as offering recognition of the achievement of its students. For its part, the University saw FTA as a strong institution, operating in a field potentially capable of supporting the achievement of the University's mission. The team was told that the University considered that there was potential for developing a stronger relationship with benefits at the subject level.
The University's approach to collaborative provision
8 In its Commentary, the University described its collaborative provision both in the UK and overseas as limited, and made clear that this was a result of a deliberate policy, following an approach characterised by caution in the use of its degree-awarding powers. The University had 'rejected the concept of collaborative work as a profit-making exercise' and now sought to develop only a limited number of partnerships 'in areas which it considers to be particularly congruent with its mission and major strengths'.
9 The University has set out the main features of its approach to international activities in a Policy Statement on International Activities (1999). This promotes an international dimension in the curriculum, the recruitment of international students, and the encouragement of links and exchanges compatible with the University's mission. The principles set out in the policy are reflected in a set of precepts, framed as questions, which form a part of the University's Validation and Review Handbook (the Handbook), published in September 1999 and updated in October 2000 by a revised appendix. The University made clear in its Commentary that 'these precepts form an important context for the initial contacts with, and visit to, potential partners'. The requirements for different types of approval in collaborative partnerships are also set out in the Handbook, as are matters which should be included in a memorandum of cooperation. There are no separately specified procedures set out in the Handbook for the monitoring and review of collaborative programmes, which are therefore governed by the procedures applicable to internal programmes.
10 In its Commentary, the University explained that the management of quality assurance for collaborative links was in general similar to that for all other academic provision. The review of the operation of collaborative partnerships is the responsibility of the University Academic Affairs Committee (AAC), which manages all quality assurance matters on behalf of Academic Board. The main focus for the quality assurance of programmes at departmental level in the University is the course committee or course board. In some cases the course committees for collaborative programmes report to the relevant Faculty Academic Standards Committee (FASC), as is the case for internal provision, and the FASCs in turn report to AAC. However, for the University's more substantial collaborative links, the course board reports directly to AAC and this is the case with the FTA partnership. The University takes the view that this provides for a higher level of control and accountability through a direct line of report. The arrangements for annual review reports to be submitted directly to AAC were approved by that Committee in November 1999. The main quality assurance arrangements relating to the partnership are therefore the initial approval process, the annual course board report to AAC, and the separate report made by the appointed Advisor to the programme (see below, paragraphs 30 and 31). In addition, there are regular advisory and liaison visits undertaken by senior staff, which also address quality assurance matters.
Locus of responsibility for quality and standards
11 The University stated in its Commentary that it accepts responsibility for all academic awards made in its name. This is confirmed in the Handbook and in the Memorandum of Cooperation relating to the partnership, which states that 'the quality assurance of [the award] will be the responsibility of the Academic Board of [the University] which will oversee and maintain academic standards through its Academic Affairs Committee'.
QAA's Code of practice
12 In the Commentary, the University outlined the process it had undertaken to check its adherence to the precepts of QAA's Code. It had undertaken a systematic mapping of its policy and practices against the Code, and considered that it adhered to the precepts relating to collaborative provision, with one minor exception relating to the production of transcripts. The audit team also noted that the current specification set out in the Handbook and the Memoranda of Cooperation did not yet include specific reference to the scrutiny of promotional material, as suggested in the Code. The University intended to revise its specification to address both these points, as a part of the ongoing enhancement and improvement of its systems for collaborative provision.
Representative nature of the University's procedures
13 As a matter of deliberate policy, the University has not sought actively to pursue overseas collaborative partnerships. The University's link with FTA was at the time of the audit its only overseas collaboration, and as such the view of the University in its Commentary was that it 'can safely be considered as wholly representative of the way in which overseas collaborative partnerships are managed'.
The establishment and management of the collaborative partnership
The approval process
14 In its Commentary, the University explained that it was not pro-active in establishing the link with FTA, but gave consideration to a proposal for academic collaboration on the basis of the common commitment to theological research, and the quality of doctoral candidates who had studied in the University following completion of the programme at FTA. The Commentary also made clear that the University requires a formal approach to the Principal (as chair of Academic Board) as the appropriate starting point for any institutional link. Initial, informal discussions between FTA and the University took place in 1997, leading in November 1998 to a formal letter from the Rektor of FTA requesting official consideration of a possible link. In that correspondence, the Rektor named two senior UK academics familiar with FTA who could be approached for information about the institution. In February 1999, having received reports from these senior academics, as well as a briefing note from a University staff member who had previously studied at FTA, the Senior Management Group of the University agreed to initiate formal consideration of the proposed link, subject to simultaneously conducting further enquiries about the standing of the institution, and making a visit. The Senior Management Group noted at this time that the use of German as the language of instruction would need to be particularly carefully addressed in this partnership.
15 The audit team learnt that although the University offers a range of master's degrees in Theology at its UK campus, the programme at FTA does not directly correspond to any of these, even though the proposed award originally carried the same title as one of the University's awards for internal study. The team was told that the University did not seek greater correspondence between the internal programme and that of the partner because the programme at FTA was already running successfully and the University wished to respect the different ethos and scholarly conventions of FTA as well as the different traditions of theological scholarship in the UK and Germany.
16 The approval process followed by the University contains an initial 'monitoring phase' which is described in the Handbook as 'a continuous process which is advisory and developmental as well as judgemental'. A monitoring panel is appointed to assist in this phase. Normally, the relevant FASC appoints the monitoring panel, but because the link with FTA involved an overseas partner, the approval process was directly managed by AAC, and there was no formal committee consideration at departmental or faculty level.
17 The procedures set out in the Handbook require the involvement of senior staff in the development and approval of collaborative arrangements. The Academic Registrar is involved in the production of the Memorandum of Cooperation, which must be signed by the Principal of the University. For this partnership, as with all collaborative arrangements, the formal validation panel had been chaired by the Assistant Director (Academic) (now the Vice-Principal). Following the approval of the programme, there had been a continuing involvement in the monitoring of its operation by the Vice-Principal and Academic Registrar.
18 The monitoring phase involved a number of visits to FTA by senior staff of the University, reporting to its Senior Management Group. Monitoring arrangements to support approval were agreed by AAC in November 1999 with a number of issues being specifically identified as requiring particular attention, including the need for the appointment of an appropriate bilingual external examiner. The monitoring panel worked with FTA staff to develop the proposed programme submission. This panel, following the receipt of independent external advice, provided an interim report to AAC in May 2000. It recommended that, although a number of issues remained to be resolved, a date should be set for the validation event.
19 The validation event then took place in June 2000 in Giessen and this was chaired by the University's Assistant Director (Academic), the standard practice for all collaborative programmes. The University's Commentary pointed out that the main function of the formal validation event was to check and confirm that all necessary issues had been addressed in the monitoring process, and to address any final or outstanding matters. The validation panel included members of the monitoring panel, together with an external member appointed by the University. A significant focus in the event was on those areas where the British and German higher education systems differed, particularly in relation to assessment. The validation panel did not impose any conditions upon approval, but did put forward five recommendations. The validation panel's report was submitted to AAC which recommended to Academic Board that the programme be approved from 1 September 2000. AAC agreed a recommendation that the Head of the School of Theology and Religious Studies should chair the first board of examiners at FTA in September 2001 in order to induct an appropriate FTA member into that role. The recommendation of AAC was approved on behalf of Academic Board by Chair's action which was duly reported to Academic Board.
20 The audit team noted that in addition to the information received from the two UK academics nominated by the Rektor of FTA for reference, the University had involved its own external assessor in the evaluation of the proposed partnership with FTA. This person was a German-speaking academic and therefore able to directly assess the standard of work produced by students at FTA. The University was also able to consider the partner's 1996-97 report to the EEAA (of which FTA is a member) which contained summary financial and resource information. FTA's approval by the EEAA had involved the scrutiny of the curriculum and a three-day visit including observation of classes, a process the University would have been aware of. The team believes that the checks made by the University were appropriate and sufficient; however, the University recognises that in the light of the current recommendations of QAA's Code, more direct checks on the financial status and robustness of a partner institution would now be appropriate. It has already implemented this practice for new UK collaborations.
21 The audit team also observed that although the management of the process of approval by AAC ensured that there was appropriate central involvement in and control over the approval process, involvement at the subject level was less well-defined. While the monitoring and validation panels had both included members with subject expertise, there had been no formal consideration of the proposed programme at either departmental or faculty level. With regard to the task of establishing equivalence of standards with the University's own comparable programmes, a 'mapping exercise' had been conducted by the Head of the School of Theology and Religious Studies. However it appeared to the team that this had been intended primarily to demonstrate the credit value of the subjects studied by students following the proposed MA programme, and had not sought to compare in any detailed manner the outcomes or level as between the degree offered through FTA and similar degrees offered by the University or more widely within the UK.
22 The audit team was provided with evidence that recommendations set at the time of validation had been followed up. The University's procedure for making amendments to the programme other than minor amendments to existing modules is for proposals to be submitted for consideration by AAC, following consideration by the course board. This procedure had been used to amend the title of the degree and to amend the admission requirements to allow greater flexibility in the minimum average mark required of entrants. The procedure does not involve any consideration at the disciplinary level within the University (apart from the University representation on the programme course board), nor does it require draft text to be submitted where, as in the case of the change to the admissions requirements, the programme regulations required amendment. The team was provided with the revised definitive document which incorporated the changes to learning outcomes which had been recommended by the validation panel and which had been agreed under the delegated authority given to the course board at FTA. The admissions-related amendment was not, however, included in the definitive document as revised and reissued in February 2002.
23 The audit team was satisfied on the basis of the evidence presented to it that the University had sound mechanisms in place for supporting and completing the approval process, and for following up recommendations and conditions made at the time of validation, and that broadly appropriate procedures were in place for agreeing and approving amendments to the programme. Strong features of these arrangements were the supportive preparatory monitoring process and the involvement of external advice. The University may, however, wish to consider the extent of subject level involvement in these procedures, particularly in relation to amendments to the programme.
Formal arrangements
24 All collaborative partnerships established by the University are governed by a formal contractual agreement laid down in a memorandum of cooperation. The requirements for this are clearly specified in the relevant Annex of the Handbook. The procedure for the preparation of memoranda of cooperation requires all such documents to be prepared by the relevant departmental or faculty staff operating in conjunction with the Academic Registrar. There is therefore an opportunity for a senior officer of the University, familiar with the requirements of the Code, to ensure that all the elements of the precepts are satisfied. For the partnership with FTA, the Memorandum of Cooperation was revised in 2001 in order to address additional requirements under the precepts of QAA's Code. There was clear evidence from the revised document that the University was seeking to adhere to the guidance contained in the Code relating to written agreements. The audit team noted, however, that there did not appear to be any formal procedure for the approval of the revised Memorandum of Cooperation, even though approval by the validation panel and AAC had been required for the original agreement.
25 The University expressed the view that the revised Memorandum of Cooperation covers the matters specified in QAA's Code with one exception relating to the issue of transcripts, which had already been identified by the University (see below, paragraph 27). Further points of detail that the University might wish to address, and which are identified for consideration in the Code were the identification of the language of instruction and assessment and the requirements for documentation to be supplied in the English language. In overall terms, the audit team found that the University had given careful attention to updating the agreement in the light of the Code, and was managing the programme in accordance with the precepts of the Code.
26 In accordance with the University's standard arrangements, the validation proposal documentation included a copy of the Memorandum of Cooperation signed by the Director on behalf of the then College and by the Rektor on behalf of FTA. The University expressed the view that this was an appropriate procedure since the monitoring process was designed to ensure that all major issues would be resolved prior to the validation event. The audit team noted, however, that in this instance the validation panel observed in its report that the signed Memorandum of Cooperation included in the validation papers required later amendment in order to satisfy the precepts set out in QAA's Code. The team also considered that there might be instances when other amendments to the Memorandum might be required to give effect to conditions or recommendations arising from the approval process. The University may therefore wish to review its practice of signing memoranda of cooperation prior to final approval by Academic Board.
27 The University issues degree certificates on the basis of a pass list supplied by the Board of Examiners and signed by the external examiner and the Chair of the Examination Board. The certificates indicate that the graduate followed a programme of study at FTA and that the language of instruction was German. The certificates therefore comply with the relevant guidance in QAA's Code. Students also receive a final assessment report which is prepared by FTA, containing grades for individual modules derived from the spreadsheets used by the Board of Examiners. The University does not currently verify the accuracy of these reports issued by its partner, nor does it conduct any audit of them. The Commentary made clear that the University is aware that this aspect of its current practice does not meet the guidance in the Code and that it is giving active consideration to introducing an arrangement for collaborative partners to issue more formal transcripts which are verified by a responsible officer of the University before distribution.
28 The Memorandum of Cooperation requires that all publicity and promotional material related to the course is subject to the approval of the University. Publicity is not extensive, as students are drawn almost exclusively from within FTA, and in the first year of running the programme there had been no external promotion. Scrutiny of any publicity was a part of the role of the Advisor, and the Advisor's annual report contained a formal statement relating to the checking of publicity material.
Quality of learning opportunities and student support
Liaison and administration
29 Administrative support for the link with FTA is provided by the University's Academic Registry which provides guidance on institutional and regulatory matters. It is anticipated by the University that an annual visit will be made by the Head of Quality Assurance for this purpose. Support is also provided through the Advisor who is drawn from the School of Theology and Religious Studies. A substantial number of visits had taken place both to FTA and with staff from FTA visiting the University; in this early stage of the link these contacts had in the main addressed general development, validation and quality assurance matters. However, given the University's motivation for entering into the collaboration, which included the promotion of research developments between the two institutions, the audit team expected, even at this point, to find greater contact between staff at subject level than appeared to be the case. Apart from the formal links, staff interaction appeared to be relatively limited and the team heard that staff at FTA would have welcomed more interaction with their counterparts from the University. No doubt the University will encourage such communication as the partnership develops further.
30 The University's Commentary highlighted the role of the University Advisor in providing support and liaison for each of its links. The Advisor's role included developing a culture of evaluation, providing subject advice, monitoring and reporting on quality and standards, and checking the fulfilment of the requirements of validation and review procedures. The Advisor also visited the partner institution periodically. It was recognised by the University during the development of the FTA link that it was important that the person appointed as Advisor to the programme should have fluent German. The first person appointed was the member of staff who had previously been a student at FTA (13 years earlier), and who had participated in the approval process. The University had noted the potential conflict of interest in this appointment, and AAC had accordingly asked the Advisor to include in his first report to AAC a reflection upon whether this had proved to be a significant issue. The role of Advisors for collaborative programmes was approved by the University's AAC in November 1999 and appeared to be well-understood by the Advisor and by FTA. However, there was currently no reference in the University's Handbook to the appointment of Advisors for collaborative programmes, although the audit team was informed that an Advisor is appointed for all collaborative arrangements. The University will no doubt be including relevant guidelines for such appointments in the next review of its Handbook.
31 The evidence seen by the audit team suggested that the appointment had worked well and that the Advisor's familiarity with both the German and the British systems of higher education had been a distinct advantage. Teaching staff at FTA, indeed, expressed the view that familiarity with theological education in both countries was essential for the Advisor to be fully effective. The Advisor had, as part of the annual review procedure, produced a first annual report on the operation of the link which was informative, reflective and where necessary constructively critical. In the view of the team, the Advisor mechanism was a useful and important element in the effective operation of the link. The first Advisor to be appointed had recently taken up a post in another University, but had agreed to continue in his role until the end of the current academic year. The team was assured that this was not because of any difficulty in finding a successor, but because the University wished to ensure continuity during the early stages of the link. It was told by senior staff of the University that there were a number of members of the School of Theology and Religious Studies who had the necessary academic experience and had a reading knowledge of German. The team understood from senior staff at FTA, however, that some consideration had been given to appointing a replacement Advisor who had neither a background in theology and biblical studies nor any German language competence. On the basis of the evidence it reviewed during its enquiries, the team concluded that the operation of the Advisor mechanism ideally required the appointment of a multiskilled individual who was proficient in the subject, familiar with the University's quality assurance procedures and requirements, fluent in the German language, and possibly also had personal experience of studying or working in both English and German higher education institutions. In light of this, it might have been helpful for a successor to the current Advisor to have been identified prior to the end of the academic year in which the current appointment expires in order to provide some measure of continuity. Given the crucial role of the Advisor and the difficulty in identifying appropriately qualified persons for this role, the University is advised to give careful attention to the necessary succession planning.
Monitoring and review
32 The University's Commentary made clear that its basic review requirements were identical, irrespective of the location of the award. The procedures for the monitoring and review of collaborative programmes are therefore broadly the same as those for internal provision. These require the submission of an annual report by FTA to a template issued by the University and to an agreed timescale. The required monitoring processes and reports are, however, supported by the University Advisor.
33 In accordance with the University's quality assurance procedures, a course board comprising all full-time teaching staff and with student representation has been established at FTA with responsibility for the oversight of the programme. The terms of reference of this board reflect those of comparable internal University boards. The board meets three times annually and submits an annual report. The Advisor is the University representative on the board. Unlike annual reports relating to internal provision, which are initially considered by the relevant FASC, the annual report is submitted directly to the University's AAC. The report includes a review of the year, the external examiner and Advisor's reports, responses to them and an action plan. The first report, on the academic year 2000-01 was submitted in early January 2002. It provided all the specified information and evaluation, and was considered by a meeting of AAC in February 2002. The team learnt that FTA had been supported in the production of the annual report both by the Advisor, who had made four visits to FTA during the course of the academic year, and by the comments of a senior university officer on the first draft of the report. The report provided a substantial range of information upon which the University would be able to make judgements concerning the operation of the programme. The minute of the meeting of AAC to which the report was presented indicated that the report had been given due consideration and that a number of issues had been identified for deliberation. Nevertheless, given the level of detail of the annual report the University might wish to consider whether AAC is able to deal fully with all issues potentially arising through the annual reporting mechanism and whether it would be desirable to introduce a more detailed level of preliminary scrutiny, perhaps at the Faculty or departmental level. Although a representative from FTA was eligible to attend AAC (as a full member) for the discussion of the report, this opportunity had not been taken by FTA. Inevitably, AAC did not consider all aspects of the annual report, and the team learnt that in the light of issues arising during the operation of the link, a number of actions were decided upon outside the committee framework and which were not therefore reported to AAC. FTA had received feedback on the annual report, both through a copy of the relevant minute and through notification of actions proposed by senior managers.
34 Student representatives serve on the course board for the programme at FTA. The proceedings of the board are conducted in English, and whilst this might present a problem for some students, the student representatives seen by the audit team were clearly able to represent the views of students through the medium of English. They indicated to the team that issues which they raised in this forum had been properly addressed. In addition, student opinion is sought through questionnaires for each module. The course board minutes noted that there had been some difficulties, which FTA was seeking to address, with the implementation of this source of feedback. The Advisor also meets with students and is thus able to obtain informal feedback. In view of the limited oral fluency in English of some of the students, it was clear to the team that there were benefits for the operation of this means of feedback where the Advisor was a German speaker.
35 The audit team observed that while the annual report clearly contributed to the development of the collaborative link, it was a retrospective evaluation which might only signal difficulties well after the event. In practice the University relied substantially on the role of the Advisor to ensure that it was kept fully informed of unforeseen operational issues. In general, however, the team considered that the combination of the course annual review report and the Advisor's report provided a good level of information to the University on the running of the programme. Overall, the evidence available to the team suggested that the University had supported FTA effectively in introducing the monitoring and review arrangements, that these were comprehensive and beginning to work well.
36 The University operates a system of quinqennial review. The link with FTA was due to be reviewed in 2004-05, and this was recorded in the revised Memorandum of Cooperation. Evidence provided by the University relating to a major review of another partner institution indicated that this process was wide-ranging and appropriate.
Provision of information to students
37 The University ensures that all of its partner institutions receive standard items such the Handbook and the Assessment Handbook. A draft student handbook was supplied to AAC as part of the first annual report on the programme. Since the whole of the handbook (apart from the module descriptions) was written in German, and no translation was provided, it is unlikely that many members of the committee would have been able to comment meaningfully upon it. The audit team was shown an English language version of the handbook during its visit to FTA, but was informed by students that they used the German language version. The team was told that one of the tasks of the Advisor was to inform students generally about the University (although this did not appear in the list of tasks for Advisors approved by AAC). The University has put in place a process for academic appeals by students at FTA which involves a joint process and allows for early local resolution if possible, while retaining the right of final appeal to the University as the awarding body. The team spoke to students at FTA who were aware of the academic appeals procedure. Students at FTA had indicated in a course board that they felt a need for written assessment regulations for the University programme, but this need appeared to have been satisfied by the publication of written course information.
38 The audit team concluded that the information provided by the University to students at FTA did not originally follow fully the guidance given by QAA's Code, which states that the information provided to students on collaborative programmes should be comparable with that given to internal students. For instance, the first cohort of students were not initially informed as to the ways in which assessments for the University programme differed from that for FTA's own programme, nor were they informed of intended learning outcomes (because these at this stage had not been finalised). These matters were addressed in the student handbook issued for the academic session 2001-02, but the handbook did not address the consequences of the programme being a validated rather than a franchised or internal programme concerning matters such as the status of students on the University programme in regard to access to University teaching staff or any rights of access to University facilities. However, the overall quality of information given to students is being addressed and the team is confident that, as the link matures, the University will meet fully the requirements of the Code.
39 With regard to the use of language in the administration of the programme, the audit team was informed by FTA that all formal meetings relating to the programme were conducted in English. There is also an understanding that all key documents, including minutes of the course board, are produced in English. The University appeared, however, to be under the impression that Course Board meetings were held in German, so that no student would be prevented from participating as a representative because of language difficulty. If that were in fact the case, it would reinforce the need for any Advisor to have a high level of proficiency in the German language (see above, paragraph 31). In general the team found no indication that language issues had caused any difficulty in communication.
Staffing and staff development
40 As part of the approval process, the University examined the qualifications and experience of teaching staff at FTA. FTA is required by the revised Memorandum of Cooperation to provide the University with notification of significant staff changes through the annual report. There were no such changes during the first year of the programme. All of the full-time teaching staff at FTA were male, and the University had indicated that when the opportunity presented, it wished FTA to appoint a female staff member in order to provide for the pastoral care of women students.
41 The University's Commentary identified three main ways in which staff development in support of the programme occurred. These were: internal staff development provided by FTA to its own staff; the provision by the University of tailored events for its partner; and the regular work of the Advisor in supporting staff at the partner institution. The University had undertaken a number of activities in support of the development of staff at FTA. For instance during the programme development phase, prior to approval, it had organised a seminar at Giessen on assessment issues and a visit to the University by senior staff from FTA. It had also conducted a further session in Giessen in June 2001 to provide help in the development of assessment criteria and learning outcomes. Formally organised staff development activities at FTA, including annual appraisal, concentrate on research development and have yet to grapple with issues of learning and teaching. However, it was manifest to the audit team that teaching staff at FTA had engaged in extensive informal discussions concerning pedagogic issues, including the significant issues concerning differences in German and UK higher education relating to assessment. The team understood that staff at FTA would have welcomed the opportunity to engage in dialogue over curricular and pedagogic issues with a wider range of staff from the University. In view of the University's expressed objective of seeking to develop links between the two institutions, it may wish to give further consideration to means by which this might be possible. Taken overall, the evidence seen by the team showed that the importance of appropriate staff development support and advice in the early stages of the link had been fully recognised by the University, that a considerable investment had been made in providing this support, and that this was valued by staff at FTA.
Assurance of the standards of the awards
Admissions
42 As the Commentary made clear, the admissions requirements and procedures for the programme were approved as a part of the validation process and subsequently monitored by the Advisor. Admission to the programme is normally restricted to students who have completed the first three years of FTA's four-year programme with a specified minimum mark. The validation report and discussions with University staff indicated that consideration was given by the University to the equivalence of the first three years of study at FTA to a first degree in the UK. The University took into account a number of matters, including a consideration of the curricula, and its own experience in admitting students from FTA to internal research degree programmes. The current Memorandum of Cooperation does not give the University any involvement in decisions relating to the curriculum or standards of the first three years of the four-year programme, but FTA understands that the University should be consulted over any significant changes and it is intended to include such a requirement in the next revision of the formal agreement. Students may also be admitted to the programme who have equivalent qualifications or experience elsewhere (although the definitive document suggests that students who have not completed the first three years of the FTA programme are eligible for entry on the basis of a school-leaving qualification alone, this is neither the intention nor the understanding of FTA and the University). To date the only students to enter the programme are students who have undertaken their previous studies with FTA.
The assessment of students
43 The language of both delivery and assessment of the programme is German. The University's Commentary explicitly addressed this issue, and showed that the implications of delivery in a foreign language had been considered at all stages of the approval process. The University claimed that it 'approved the language arrangement in full knowledge of its implications, the most practical of which are the necessity for fluency in German in both the University Advisor and the external examiner'. The audit team observed that the use of the German language for assessment also limited any opportunity for the University's own staff to participate in the assessment of students or to moderate student assessments, a route which was not in any event envisaged for this link.
44 The Commentary also drew attention to the care given by the University to addressing issues arising from the different academic cultures, assessment regimes and traditions in the British and German higher education systems. These included for instance the uncommon use in Germany of formal examination arrangements at postgraduate level; the unfamiliarity with learning outcomes and criteria-based assessment; different approaches to marking practices; and different understanding of marking scales. German academics were unfamiliar with the use of external examiners and unaccustomed to the practice of second marking; there was also a greater reliance upon oral classroom performance. Considerable effort had been expended on both sides of the partnership to discuss and overcome these differences, but despite this some consequent difficulties had arisen (see below, paragraphs 46 and 50). Nevertheless, the University is to be commended for the steps which it has taken to identify and to address the cultural issues involved. It is also fortunate in having a partner in which the teaching staff are willing to engage fully with a dialogue about different academic traditions. The changes already made by FTA had gone beyond simple compliance and had included some changes to elements of the four-year programme which fell outside the MA programme validated by the University.
45 The regulations and procedures for assessment were approved at validation. The validation panel recommended that the learning outcomes of all modules should be revised in order that they more accurately reflected module content and assessment. It also recommended that detailed assessment criteria were provided for all modules. This was done during the course of the year under authority delegated by AAC to the course board. The audit team noted that staff at FTA had been provided with assistance in developing statements of learning outcomes and assessment criteria, and that this had been provided jointly by the Advisor (who was also a course leader for a comparable internal programme) and a member of the University's Centre for Learning and Teaching. However, staff at FTA were not provided with assistance from a wider range of subject-level counterparts from the University in addressing, at the level of the individual course unit, what was for them a new and unaccustomed task. The team believed that a more developed involvement at the subject level would more effectively meet the University's objective of developing comparability of subject standards.
46 In its Commentary, the University discussed a number of problems that had arisen in the assessment context during the first year of the programme's operation. The University's view was that, notwithstanding the attention paid to this area in the development period, such issues were likely to occur because of different assessment regimes and academic traditions, and that not every consequence of these differences could be foreseen. The issues identified in the Commentary had included marked differences between student performance in dissertations and in unseen examinations, overmarking or other discrepancies in marking, and the failure of some examiners to provide scripts sufficiently in advance of the examination board to enable proper moderation by the external examiner to take place. In addition, the delay in the appointment of an external examiner (see below, paragraph 49) meant that he could not be involved in approving some of the assessments and examinations (the Advisor instead taking on this role), and in the external examiner's view some of the examination questions had sought descriptive answers rather than seeking the kind of analytical and critical skills which would normally be expected at master's level. There were also difficulties in the management of the board of examiners (see below, paragraph 50). As already noted, the University had given considerable attention to preparing its partner for the assessment requirements and procedures, but notwithstanding this it was clear to the audit team that this remained an area of considerable vulnerability where further efforts would be required on both sides. Given the perceptive analysis of the issues in the University's Commentary, the actions which had been taken to address the areas of concern, and the willingness of FTA to participate in this process, the team found a good basis for confidence that the issues will be tackled effectively.
47 The audit team was told that the University considered that it did not have any directly analogous programme available internally which might serve as a benchmark for the MA programme. Nevertheless, QAA's Code, which the University follows, expects that there should be equivalence of standards between the validated programme and any comparable awards offered by the awarding institution, and the University does offer a range of MA awards in the same subject area. Judgments on the equivalence of standards made during the approval process appeared to have relied mainly upon external advice, on the University's experience of admitting students to postgraduate programmes after completion of the four-year programme at Giessen, and the recognition of the four-year programme by the EEAA. No direct comparisons were made with the standards expected by the University for broadly comparable internal awards. Whilst the audit team saw no evidence to question the achievement of master's level standard by students on the FTA programme, it considered that the University could have taken further steps to assure itself through direct evidence that the academic standards in the programme were equivalent to comparable internal programmes. The absence of this consideration was perhaps the more remarkable because of the University's statements relating to compatibility of mission and the closeness of approaches between the two partner institutions at the subject level. The University might therefore wish to consider the advisability of incorporating more formal involvement at the subject level and a clearer and more explicit evaluation of the equivalence of standards in validated programmes to those expected of its internal provision. In particular, the University may wish to reconsider the appropriateness of its decision to exclude FASCs in the consideration of such collaborative arrangements.
External examiners
48 The University appoints external examiners for all of its programmes, internal and external. The external examiners are provided with clear and detailed guidance on their roles by means of a section of the University's Handbook. Reporting arrangements are identical to those for UK based programmes, and the external examiners' reports are received centrally in the University, and read by the Vice-Principal on behalf of Academic Board. The University does not have a policy of appointing external examiners for collaborative programmes who also act as examiners for internal provision, and the examiner appointed in this instance was responsible only for the collaborative programme.
49 The audit team noted that although the importance of appointing a bilingual external examiner had been identified at an early stage in the University's consideration of the link, active steps to recruit an external examiner did not appear to have begun prior to the approval of the link. In the event, the University experienced considerable difficulty in finding an external examiner who had the necessary competence. The team was informed that although there were a significant number of British academics in the field of theology who were sufficiently conversant with German to undertake the task, there had been a number of unforeseeable difficulties in making an appointment. The University itself identified the importance of an appropriately qualified external examiner in the monitoring phase, and no doubt it would have been helpful in this instance if the name of a proposed external examiner could have been presented to the validation panel prior to final approval, thus indicating that it would be possible to recruit to such a pivotal role. The University might wish to review its procedures to ensure that such critical issues are addressed at an earlier stage.
50 As has already been indicated, a number of problems surfaced at the first examination board. Despite the investment made by the University in emphasising the importance of assessment arrangements, five of the 15 staff involved in the programme at FTA were absent from the examination board. Although all marks were available, some scripts were not available to the external examiner who was therefore unable to moderate them. The external examiner therefore expressed the view in his report that the arrangements for the board had been 'haphazard' and 'did not enable the Board to conduct its business with full confidence'. Fortunately, the difficulties were not such as, in the view of either the examiner or the University, to compromise the assessment of students. To its credit, the University had already made considerable efforts to introduce colleagues at FTA to the requirements of the examinations boards, including attendance as observers at similar boards in the University, and it moved rapidly to address the concerns identified by the external examiner. A decision had already been taken as part of the validation process that for the first year the examination board should be chaired by an experienced member of staff from the University, despite the usual practice of examination boards for collaborative programmes being chaired by a member of staff from the partner institution. Events have proved that decision to be a wise one, and the University has sensibly decided to continue to provide the Chair for the Board for at least one further year. In addition, the University is arranging for the external examiner and Chair to be present for an extended period before the Board in order to provide advice on preparation for the Board. The University has therefore shown itself willing to take prompt and appropriate action to address the serious issues raised relating to the conduct of the examination process.
51 The University considers itself fortunate to have appointed an examiner who has acted purposively and robustly in upholding and enhancing academic standards and in offering valuable advice. It seemed to the audit team possible that the limited extent of the subject links between colleagues at FTA and their counterparts at the University had placed a particular burden and reliance upon the external examiner. The team recognises that teething difficulties and unforeseen problems are particularly likely in newly established programmes, and this again underlines the crucial role of both the Advisor and the external examiner during this period. The University will wish to ensure it retains an appropriate level of control over the management of assessment, while continuing to assist and support its partner, and that the appointment and involvement of the external examiner is fully safeguarded.
Conclusions
52 The University of Gloucestershire established its link with the Freie Theologische Akademie, Giessen through a series of validation procedures conducted between 1997 and 2000, leading to the approval of the MA in Biblical and Theological Studies. With an intake of just nine students in its first year, the programme is operating on a modest scale, and at the time of the audit was at an early stage in its development. As this is currently the University's only overseas partnership, it has conducted the quality assurance of the programme mainly through adaptation of the procedures developed for its internal programmes and its UK collaborative provision. These have been operated in a careful, thorough and broadly effective manner, although some problems remain to be addressed in the area of assessment. The University has devoted considerable time and resources to support the development of the link. Of particular note have been: the institutional commitment to the successful development of the link as demonstrated in the management of the process through the AAC of Academic Board and the substantial involvement of senior staff; the commendably careful monitoring and development support that has been provided to the partner prior to formal approval; the substantial involvement of external academic expertise in the approval process; the awareness shown of potential problems likely to arise from the differences of language and pedagogical traditions, together with the support provided to the partner to accommodate the requirements of the UK higher education system; and the willingness to take determined action to address matters which had been identified as problematic. The University has given careful consideration to QAA's Code of practice for the assurance of academic quality and standards in higher education, Section 2: Collaborative provision (1999) and has engaged in a process for ensuring adherence to its precepts. To the extent that the University does not already address all the recommendations of the Code, it has demonstrated its intention to move towards satisfying them.
53 As it seeks to develop further its management of this partnership, the University may wish to give further consideration to several matters. These include: the problems arising as a consequence of different pedagogical and assessment traditions; the limited extent to which the link operates at the subject level; the ways of establishing more effectively the comparability of subject standards; the rationale for signing memoranda of agreement before the conclusion of the approval procedures; the appropriate points in the deliberative structure for the review of annual reports; the appropriate arrangements for managing the conduct and chairing of Boards of Examiners in the partner institution; and the high degree of reliance placed on a small number of individuals with wide-ranging skills who can fulfil the support and advisory roles required.
54 At the time of the audit this partnership was not only recently established, but also constituted the totality of the University's overseas collaborative provision. However the University believed that its approach to this link might be regarded as representative of that which would be taken to any future such development. The Commentary produced for the purposes of the audit provided a good picture of a link which was in the initial years of its operation. It also demonstrated that the University was well-informed and self-critical with regard to management of the programme. The University was open about the strengths and weaknesses of the link, ready to accept that not every aspect of the link was working perfectly, and demonstrated that it was willing to learn from its experiences and to seek to improve its practices. The findings of the audit show a clear commitment by the University to support, enhancement and effective quality assurance in the context of this partnership, but they also identify some areas of potential vulnerability. Given the early stage of the partnership and the relatively limited amount of information available, the evidence of the audit provides grounds for a cautiously positive assessment of the quality assurance of the programme. It therefore supports a conclusion of broad confidence in the University's management of the quality and standards of this overseas collaborative partnership.
Top