Quality Assurance of Overseas Collaborative Provision
The Quality Assurance Agency for Higher Education (QAA) was established in 1997 to contribute to the maintenance and improvement of the quality and standards of all UK higher educational provision, wherever and however this is offered to students. To this end, QAA undertakes regular academic quality audits of individual institutions to review the operation and effectiveness of arrangements for assuring quality and standards.
Quality audits also cover the arrangements which institutions use to assure the quality and standards of their awards and programmes offered in collaboration with other partners, both within and outside the UK. As part of this process, QAA has extended its audit procedures enabling audit teams to visit overseas partners of UK institutions so that the same enquiries can be made of arrangements for safeguarding UK awards and programmes offered to students outside the UK as are made of UK-based provision. This initiative has been designed to help provide enhanced confidence in the work of UK universities and colleges operating overseas.
QAA's enquiries have been assisted by the publication in December 1996 of the former Higher Education Quality Council's revised Code of Practice for Overseas Collaborative Provision in Higher Education. This offers guidance on good practice and a framework within which institutions can review and consider their current and future activities. The Code of Practice has been widely welcomed and has been used as a common point of reference for the programme of overseas visits. Shortly after this overseas audit visit, QAA published a Code of Practice for the assurance of academic quality and standards in higher education: Collaborative provision. This document will form the basis of overseas auditing activities undertaken with effect from Summer 2000.
The UK universities and colleges, with the agreement of their overseas partners, were voluntary participants in the programme of overseas visits. Their collaborative links cover between them a range of programmes and subjects, levels of award and different forms of institutional partnership.
1 This is a report of an academic audit carried out by the Quality Assurance Agency for Higher Education (QAA) of the quality assurance arrangements for a collaborative partnership between The Nottingham Trent University ('the University' or 'TNTU') and Griffith College Dublin ('the College') to deliver a programme of studies, through a validation arrangement, leading to the award of the University's LLB (Hons). QAA is grateful to The Nottingham Trent University and its partner for their willing co-operation.
2 Following receipt of the University's Commentary and additional supporting information, QAA convened a briefing meeting at which members of the audit team discussed the documents and proposed a programme of meetings for visits by members of the team to the University and a visit to Griffith College.
3 The University's Commentary set out to describe and analyse the collaborative link with its partner and to identify 'strengths and weaknesses in the partnership' but stated at the outset that the 'recent nature of this arrangement coming into approval [had given the University] little scope to provide an evaluative commentary'. The information in the Commentary was subsequently amplified by members of the University during the visit in a number of reflective comments on its arrangement with the College.
4 The visit to the University took place on 20-21 April 1999, in the course of which members of the audit team scrutinised the materials made available by the University in a base room set aside for the purpose, and held discussions with members of the University, some on more than one occasion. On their visit to the University, members of the team met: the Deputy Vice-Chancellor; senior members of the University's academic staff who had been involved in the validation of the link, and members of the University responsible for monitoring and administration of the validation arrangement. The members of the team who visited the University were Dr F R Burnet, Mr J C P Raban and Dr D W Cairns.
5 Members of the audit team visited the University's partner Griffith College, Dublin on 20 May 1999. On this occasion the members of the team were: Mr G Clark (audit secretary); Dr F R Burnet and Mr J C P Raban. Dr D W Cairns accompanied the team. In the course of this visit, the team met the Principal of the College; more than 15 members of the College responsible for managing, delivering and supporting the programme leading to the University's award; the external examiners for the award; and a group of students on the first year of the programme.
6 The audit team wishes to record its special thanks to those who travelled long distances to meet it in order to help it to understand the University's arrangements to safeguard the standard of its award.
The institutional context for collaborative provision
7 The University's academic quality assurance arrangements were the subject of a HEQC academic audit report in 1994. Its academic quality assurance arrangements for its collaborative provision were the subject of two audit reports in 1996. The first of these scrutinised its collaborative provision as a whole and the second examined an instance of its overseas collaborative provision in Malaysia. The reports of these audits together provided helpful introductions to the audit team in following the University's development of its academic quality assurance arrangements for its collaborative provision. At the time of the present audit the University was also preparing for a QAA continuation audit to be conducted later in the year.
The University's approach to collaborative provision
8 The University provides students studying under the auspices of its partner institutions with access to its awards in two ways: through franchising and through validation arrangements. The University defines franchising as involving an agreement by which a programme, designed by the University, is operated and delivered by another institution. It describes validation as obtaining where the programme leading to the University's award is designed, operated and delivered by the partner institution. The University has distinct arrangements for franchising and validation consistent with its view of franchising as an activity solely to be carried out within the University's faculties. In franchise arrangements, therefore, the operational responsibility for assuring the quality of educational provision and the standard of the University's award resides at first instance with each faculty, which is then responsible through the Academic Review and Development Committee (ARDC) to the University's Academic Board. Operational matters concerning validated awards in external institutions are conducted by Validation Services, an integral part of the University's Office of Academic Planning and Quality Management (OAPQM), the remit of which embraces all of the University's academic provision. OAPQM is led by the Deputy Vice-Chancellor. Validation Services reports formally to the Validation Services Sub-Committee of ARDC and then to the University's Academic Board. Both the Validation Services Sub-Committee and ARDC are chaired by the Deputy Vice-Chancellor.
9 The University sees Validation Services as providing 'the opportunity for academic institutions, professional bodies, employers - both in the UK and overseas - to offer validated awards which are equivalent in standard to those made to students studying on programmes designed and operated by the University'. In management terms, the Deputy Vice-Chancellor is responsible to the Vice-Chancellor for the operation of Validation Services and he provides the University's governing body with regular reports on its activities.
10 The University's Commentary and its briefing papers established that the number of the University's partnerships operating under the auspices of Validation Services was growing steadily, in the UK and overseas, and that validation was now the University's preferred approach to collaborative provision: the audit team was told that ARDC would now require a persuasively argued justification before it would agree to consider any proposal to franchise provision. At the time of the audit, overseas activities within the ambit of the Validation Services involved 10 countries and 12 different institutions delivering a total of 34 programmes of study ranging from foundation to master's level.
11 In explaining the University's preference for validations to franchises as a mode of collaborative provision, members of Validation Services stated the view that when compared to validations, franchises provided more restricted developmental opportunities for the University's partners. In support of this view the University cited its recent arrangement of, and financial support for, a Validation Services Conference for the University's partner institutions, to which each had been invited to send delegates in order to discuss matters such as 'Quality and Standards in Programme Delivery'; 'Quality and Standards in Assessment' and the University's approach to quality assurance.
12 The University's Handbook for Validation Services (the Handbook) describes three forms of relations between the University and its partners and refers to accreditation, validation and credit-rating. Where the University designates an institution as 'accredited', it advances to the institution full responsibility for the quality assurance of its educational provision, for validating new programmes of study to lead to the University's awards, and for maintaining the academic standards of the relevant University awards under prescribed conditions and subject to periodic reports. At the time of the audit, the University had established one accreditation arrangement and had established no credit-rated centres. The majority of the collaborative arrangements therefore relate to 'validated centres'.
13 An institution which is designated by the University as a 'validated centre', is considered capable of assuring the standard of the University's award on an annually renewable basis, with the University retaining responsibility for the approval of significant modifications to existing programmes, and for the introduction of new ones. For validated centres, the University also undertakes to provide substantial inputs into the quality assurance processes underpinning the programmes of study it validates (see below, paragraph 49). 'Credit-rated centres', are centres which do not offer a full programme leading to an award but have credit points assigned to programmes based on the University's credit rating tariff.
14 Within this typology of arrangements, the University has designated Griffith College as a 'validated centre', and the Commentary stated the University's view that its arrangements with Griffith College were typical of its relations with other validated centres. On the basis of the information provided by the University, the audit team came to the same view.
The Handbook for Validation Services
15 The operation of Validation Services is governed by the University's Handbook, which is a comprehensive document of more than 100 pages, the provisions of which, in many instances, go beyond the suggestions of the HEQC Code of Practice. The Handbook contains guidance and regulations for all aspects of validation relations, including criteria for the selection of appropriate partner institutions; the steps to be taken after an initial approach by a potential partner institution; subsequent contacts; the submission of a programme for validation; the role and remit of key individuals; and specimen copies of such documents as institutional agreements and the standard forms for external examiners' reports. The Handbook is issued in a ring binder format to ease the process of keeping its contents up-to-date. Members of the University and the College told the audit team that they made frequent reference to the Handbook and found it to be a valuable resource.
16 The LLB programme was in its first year of operation at the time of the audit; it was therefore not possible for the audit team to review the outcomes of several of the key procedures that the University has established to assure the quality of the provision leading to its award. The team's enquiries consequently focused on the procedures through which the College had been approved as a partner and the LLB programme had been validated as fit to lead to the University's award, together with the initial steps that the University had taken to assure that the standard of its award was being, and would be, safeguarded.
The partnership between the University and the College
Origins and purposes of the link
17 Griffith College is a private institution established in 1974 which now offers programmes in business, management, finance, media, computing, interior design and law. The College has approximately 2,500 students, mainly studying at undergraduate level and its programmes are validated by Ireland's National Council for Educational Awards (NCEA), and by TNTU, and accredited by a number of professional bodies. NCEA is a statutory authority established in 1979 to validate courses and award qualifications to those who successfully complete degree and other courses in institutions outside the university sector.
18 The College is owned by a private limited company 'Bellerophon Limited', registered in Ireland. It is managed by a Board of Directors, headed by a Chairman who is also the College's Principal and is sometimes referred to in the papers seen by the audit team as its President. Academic matters within the College are handled by a 'Faculty Board' which meets weekly. This Board is attended by deans of faculties and course directors and has a wide remit but no formal terms of reference. Individual programmes of study each have a 'course committee' the membership of which includes student representatives.
19 The University's relations with Griffith College developed following those established by TNTU's accredited partner, Southampton Institute, since 1995-96, to support the franchise of a programme of studies leading to the University's BA (Hons) Business and Law (BABL) award. In 1998, as a consequence of its awareness of the buoyant demand in Ireland for access to Irish Law programmes, the College explored the possibility of offering an LLB (Hons) in Irish Law with Southampton Institute ('the Institute'). This was not feasible within the latter's agreement with the University, however, and the Institute advised the College to approach the University directly. It was as a result of this approach that the University undertook to consider the College's proposal for a validation arrangement.
20 The Handbook states that any institution seeking to have a programme validated by the University must satisfy a number of requirements, including that the prospective institution should have: an appropriate learning environment; an effective organisational structure; a system of quality assurance, and that its arrangements should provide for the incorporation of an external critical perspective. Institutions are expected to demonstrate to the University, through a process of 'institutional review', that 'the academic awards it offers are consistent and comparable in standard with those conferred by the University', that its programmes of study will be maintained at a high academic standard' and that it 'will effectively monitor, sustain and develop the standard of student performance and the quality of teaching and learning'. Satisfactory completion of an initial institutional review leads to institutional approval which, when granted, is for three years with a further institutional review at the end of that period.
21 The HEQC Code of Practice recommends that 'a UK institution should seek the widest advice from reliable sources. In particular, it should look for firm evidence of the financial security and probity of the proposed partner, particularly where the prospective partner is wholly or substantially funded from private sources'. The University's process for institutional review described in the Handbook makes no provision for such inquiries. Members of the University told the audit team that information on the financial situation of the College had been available and that, in this particular instance, the University had been able to supplement such information through the audit it had conducted of the partnership arrangements between Southampton Institute and the College in 1996. Members of the University stated that had it not been for this information, they might well have sought more detailed financial information from the College. The team was also told by the University that its judgements on the stability and reliability of a partner institution were informed by the robustness of developmental and business planning of the latter and its identification of resources to match its commitments. In this particular instance, it was clear to the team that the University's experience of its prospective partner over several years, via their respective relations with Southampton Institute, had allowed it to take an informed view of the College's likely future stability. In circumstances where the University does not have the advantage of such a perspective, however, the grounds for such an informed view might be more slender and, in such cases, more formal inquiries into the financial suitability of a prospective overseas partner might be advisable. In the interests of enhancing the security of its validations, the University may wish to include a formal requirement in its Handbook that financial inquiries be made of prospective partners as a preliminary to institutional review.
22 The University's initial institutional review process for its partnership with Griffith College was undertaken by a visiting University panel which included an external member, resident in Ireland, on the same day as the validation of the College's proposal for a programme of studies leading to an LLB (Hons) in Irish Law. Members of the University who had participated in this twofold event told the audit team that the visiting panel had devoted the greater part of the visit to the validation of the LLB, and that the two components of the event had not been wholly separate.
23 In the context of the focus in the institutional review on the learning environment within the College, the latter's validation proposal provided an essentially descriptive account of its 'library development programme'. The audit team discussed this aspect of the University's institutional review process with members of the College and was told that its preferred approach to planning the allocation of resources to academic programmes continued to be 'incremental' and focused on their immediate needs. The University's record of the institutional review component of the validation event comments on this matter, and notes the need for further development but, in establishing the conditions for validation, it is not addressed further (see below, paragraph 43). The College's validation submission for the LLB programme provided curricula vitae for the prospective course team (many of whose members were employed on part-time contracts) and described its approach to staffing matters in terms of seeking and acting on student feedback on the performance of lecturing staff. The team was told that there were discussions at the validation event concerning the adoption by the collaborative partner of a more strategic and directive approach to staff development than that indicated in the validation submission. The team was informed that the University's panel had been satisfied that the College had accepted the need for a more strategic approach to staff development.
24 The Handbook states the University's requirement that a validated centre should possess 'a forum for planning, target-setting and devising implementation strategies to enhance the quality of the learning environment' and an organisational structure that 'is widely understood within the Institution and which assigns clear executive, administrative and educational responsibilities to individuals and groups for the delivery of its programmes'; 'allows regular opportunities for staff and the student body to contribute in an informed way to the formation of policies, priorities and the discussion of issues affecting the Institution's educational performance and direction'; and 'demonstrates a collective approach to policy-making and encourages the development of an educational community which is capable of operating independently of the participation of particular individuals'.
25 The University's Handbook also states that partner institutions 'must have an organisational structure that is widely understood within the institution and which assigns clear executive, administrative and educational responsibilities to individuals and groups for the delivery of its programmes' and which 'demonstrates a collective approach to policy-making and encourage [sic] the development of the educational community'. The audit team noted that the academic management arrangements for the University's partner in this instance reflected the nature of the College's status as a privately owned institution, the deliberative arrangements for which reflected its comparatively small size, so that the same individuals would often be members of most, and sometimes all, groups and committees with quality assurance responsibilities. The record of the University's institutional review noted that the adoption of a 'formal style of operation ... would be increasingly necessary as the College grew'. The record also noted that, because the College's Academic Board 'was operating in a developmental context, the Board took its role in respect of academic quality and responding to issues very seriously, and had adopted a formal style of operation'.
26 The audit team discussed the quality assurance responsibilities of its senior committees and its management team with members of the College. Within the College the relationship between its various committees and boards charged with managing the delivery of the College's programmes and conducting their quality assurance did not appear sharply defined. Members of the College found it difficult to identify the senior bodies in the College which performed the roles required by the Handbook - for example in encouraging 'the development of an educational community which is capable of operating independently of the participation of particular individuals'. The University might wish to consider whether it had done enough to ensure for itself that the senior bodies of its collaborative partner were sufficiently well-defined to enable them to perform the roles required of them by the Handbook.
27 When discussing its approach to institutional approval under Validation Services with members of the University, the audit team was told of the importance that would be placed on the capacity of the prospective partner to operate a validation arrangement without frequent recourse to the University for support and advice (see below, paragraph 48). It was emphasised that the University expected validated centres to be capable of 'standing alone' and that, although the Handbook emphasises the University's commitment 'to assisting Institutions in the development of their own systems for quality assurance to enable them to function more independently of the University', in most circumstances such developmental activity was to be limited to 'feedback on annual monitoring, periodic review and the role of the Verifier'.
28 The Handbook requires a validated centre to have an institutional system of quality assurance, 'for the regular evaluation and monitoring of its institutional and programme performance'; 'a system that involves informed groups and includes teaching staff, students, external examiners and other external people (eg representatives of professional bodies and employers)'; and a 'system that enables swift action to be taken to tackle issues raised through its monitoring procedures'. The College's validation proposal describes its quality assurance arrangements for its programmes of study chiefly in terms of the establishment of a 'course planning committee'; an annual course review process, the functions of which are described in terms identical to those laid down in the University's Handbook; and 'quality assessment procedures' based on the assessment of the performance of lecturers by their students, using questionnaires and telephone polls. The Commentary also noted that through acting as the awarding body for the BABL programme, franchised by Southampton Institute to the College since 1995, the College had already established a form of relationship with the University (see above, paragraph 17).
29 The University's record of the institutional review component of the validation event makes no specific reference, however, to inquiries by its panel into the scope and effectiveness of the College's quality assurance arrangements. The University informed the audit team that the record of the institutional review and validation event was not a stand-alone document, but had to be read with all the documentation available to the panel. Were this to be the case, it was not clear to the team how committees of the University, charged with overseeing collaborative arrangements, which did not receive the documentation available to the panel, could have been confident that the latter had given adequate consideration to the matter of whether the College's own quality assurance systems were robust enough to underpin a validation arrangement, as distinct from a franchise.
30 The College's validation proposal does not directly address its measures for incorporating external perspectives on its work into its quality assurance and strategic planning arrangements. It refers to the experience of its staff in working with external advisers from a number of universities in the UK and Ireland, although its proposals (subsequently adopted) for its board of examiners for the LLB refer to the role of external examiners and the University's Verifier (see below, paragraph 47). It is a requirement of the Handbook that a centre offering a validated programme should be aware of, and responsive to, national and international (academic) standards. The University's record of the institutional review component of the validation event provides no specific information on the measures taken by the panel to satisfy itself of the College's awareness of national (that is, UK) academic standards, although it could have referred to the College's prior experience of offering degree-level programmes of study in partnership with another UK university, and its existing partnership with Southampton Institute in delivering a programme of studies leading to the University's BABL award.
31 The procedures followed by the University in establishing a validation partnership emphasise the importance of the initial institutional review in establishing the fitness of a potential partner for the development of a successful validation arrangement. In reaching this judgement, members of the panel told the audit team that the University's review of the partnership between the College and Southampton Institute in 1996, and the presence of members of the University on visiting 'franchise validation' panels to the College, organised by the Institute, had provided valuable information that had informed the panel's decision. It seemed likely to the team that the information the University had gathered on the College through its association with the Institute had played a significant part in enabling the visiting panel to approve the proposed relationship between the College and the University, and that the decision of the visiting panel had taken into account the College's development to that point, and its potential for the future.
32 The University's Handbook states 'It may sometimes be appropriate to combine Institutional Approval with a programme validation', but adds that where there is a single validation visit 'this would involve two discreet processes: an Institutional Review Visit ... and the validation of the programme(s) leading to the award(s)'. It appeared to the audit team that the conduct of the visit in this particular instance had not assisted members of the University's panel to establish, as the Handbook requires, a clear separation between the initial institutional review and programme validation components of the visit. The University might wish to consider whether the support and guidance it gives to panels conducting institutional reviews and validations is sufficient to ensure that the requirements of its Handbook are followed. This is of particular importance in the case of an initial institutional review for a new partner institution.
33 The University's Handbook notes that the report of a visit such as that to the College 'is meant to record the panel's findings on the whole event and incorporate comment and observations intended to assist the further development and process of critical reflection within the institution ... [and that] it is not a verbatim report' (see above, paragraph 29). The formal record of the University's visit to the College makes no reference to a number of the criteria laid down by the Handbook for institutional approval and, in addressing others, provides no indication of the grounds on which the panel considered itself able to recommend approval of the institutional relationship. It is difficult to see how members of the University and others, who were not present at such events, and who did not have access to the full range of information available to the visiting panel, could satisfy themselves from such a record that the requirements of the Validation Services Handbook had been met.
34 An Institutional Agreement (the Agreement) for a validation service between the University and the College was completed by the College in November 1998. It establishes a basis for further institutional review visits by the University; the methods by which courses will be reviewed and validated; and specifies the exact nature of the quality assurance records that will be maintained by the College, as well as the undertakings given by the University. The Agreement also sets out the parameters within which the College can publicise its relationship with the University and specifies procedures for commencement and termination of the arrangement which safeguard the interests of students registered for the award.
35 The Agreement is largely consistent with the advice offered by the HEQC Code of Practice and provides a clear framework for the operation of the validation relationship in most respects. It makes no provision, however, for the involvement of the University in the approval of staff to teach on the programme leading to the LLB award. The audit team was told that when dealing with more recently-established or smaller institutions, the University would reserve the right to approve each member of the team delivering a validated (or credit rated) programme and any replacements. In this instance the satisfactory composition of the College's proposed teaching team, and its relative stability over the period that the College had been providing tuition for the University's BABL award, offered evidence which satisfied the validation panel that any additions or replacements to the proposed teaching team would be similarly well qualified. In situations where the University was entering into a wholly new partnership, it would be much less likely that there would be equivalent evidence. The University informed the team that in this matter it had confidence in the efficacy of its normal arrangements, which were for changes in staffing to be reported to it in the annual monitoring report for the course, and the Verifier's separate report. This is, though, to overlook the post hoc nature of such reports, and the potential for the learning experience of students to be put at risk in the interval between the departure of a member of staff (or their replacement) and the University receiving a report on such a matter, which could be more than 12 months after the event. The University might therefore wish to consider the desirability of amending its procedures and its standard agreement with partner institutions to require, as its ordinary practice, notification to the University of relevant staff resignations and replacement staff appointments, together with curricula vitae, until a partnership has been sufficiently well established to provide confidence in the ability of the partner to act without guidance.
36 The Agreement specifies the key items of quality assurance related evidence that must be sent to the University each year, including an Annual Course Report (ACR), and the reports of all external examiners, together with an account of action taken in response to issues raised by the latter. In addition, the Agreement confirms the authority of the University to appoint a member of its own staff to act as a 'Verifier' charged with protecting the integrity of the University's award with the remit to attend the examination boards for the LLB and report annually to the University's Validation Services Sub-Committee. The role of the Verifier is considered elsewhere in this report (see below, paragraph 47).
Quality assurance of academic provision
37 As noted earlier, at the time of the present audit the LLB programme at the College was in its first year of operation and not all the University's procedures to safeguard the standard of its award had yet come into play. For example, the University's Verifier, who had been appointed, had yet to make her first scheduled visit to the College, and the College's first ACR was not due to be received by the University until after the end of the present session. Similarly, whilst the College had nominated and appointed external examiners following their approval by the University, any comments from them on the work of students could only come in the future. The inquiries of the audit team into the University's quality assurance arrangements for the partnership consequently focused on the procedures by which the LLB programme had been validated and those items of evidence it was reasonable to expect might be available to assist the team to assess how the University was monitoring the College's progress towards meeting the requirements of the formal Agreement. In addition to the validation process, therefore, the team considered the steps taken by the University to satisfy itself that the conditions attached by the panel to the validation of the LLB had been met by the required dates and the processes for the appointment and briefing of the University's Verifier and the external examiners for the LLB programme.
38 The Handbook provides an institution seeking the University's validation of a programme of study with detailed guidance on the information to be provided to support the validation process and a prospectus for the conduct of a validation event. For the validation of the LLB, the College provided a bound 150 page document briefly setting out the College's internal management arrangements and, in greater detail, how it proposed to administer the LLB programme if it were approved; quality assurance arrangements; syllabuses for each module; assessment arrangements, including regulations; details of the proposed teaching team, including curricula vitae and a description of the learning environment within which the LLB would operate if approved. Having read this document individually, and following a discussion within the panel, members of the latter conducted a series of discussions with members of the College and chiefly the prospective course team; these covered matters such as teaching, learning, assessment, and learning support.
39 The University's record of the institutional review and validation event indicates that the members of the visiting panel gave careful consideration to the processes through which the College recruited and appointed staff, and to the qualifications and professional development of the team of staff which would be delivering the programme if it were approved. As noted above (see above, paragraph 35), the formal Agreement between the University and the College makes no reference to staffing matters, and the Commentary makes it clear that the College is solely responsible for appointing staff to teach on the validated programme. Many of the staff delivering the LLB are employed by the College on a part-time basis and it was clear from the University's record, and from the audit team's discussions with members of the University and the College, that the visiting panel had spent some time satisfying itself that the College's current arrangements and future plans for staff development applied to both full-time and part-time staff (see above, paragraph 23).
40 The visiting panel agreed to recommend the validation of the LLB (Hons) in Irish Law to the University's Academic Board and attached six conditions to its approval, two of which were formal - the appointment of external examiners and the appointment of a University Verifier - whilst the other four were to be met before the commencement of the programme. The University's Commentary states that all the required conditions were met by the due dates. As with institutional approval, validated programmes are subject to review after three years of operation, or more frequently, if circumstances warrant it.
41 The University's Handbook provides detailed guidance on the conduct of programme monitoring, based on the preparation of the ACR by the course committee, for consideration by the College's Academic Board, prior to being sent to the University. The Handbook suggests the range of information that the course committee should expect to draw on in compiling its report: it specifically mentions student feedback comments and the reports of external examiners and that the ACR for the first year of operation of a new course should be a 'basic', principally factual, report. Members of the University told the audit team that in view of the difficulties experienced by some of its validated centres in completing ACRs to its satisfaction, its advice to such centres had recently been modified. In future, all validated centres would be expected to provide ACRs to a common and more demanding standard. The team noted this evidence of the University's enhancement of its quality assurance arrangements. To assist the College to complete the first ACR for the validated programme, the University had provided the LLB course leader with an example of an ACR produced by another centre. The course leader told the team that she was expecting to write the ACR at the beginning of the next session and that in the interests of enhancing collegial responsibility she intended to involve as many members of the course team as possible in preparing the ACR.
42 The Agreement requires that the College maintains minutes and reports on the evaluation of course monitoring within the College's academic committees. It was too early in the life of the programme for the audit team to verify this directly, although it confirmed that the course committee had met and that its principal business had been in responding to the comments of a student representative. The team considered that the University's programme monitoring arrangements were likely to be effective.
The learning environment for the validated programme
43 The Commentary states that the College's learning resources were inspected at the time of the validation and were found to be 'generally adequate'. In the course of the validation event described earlier, one of the conditions attached by the visiting panel to its approval of the validation proposal related to library provision for Year 1 of the LLB (see above, paragraph 38). Following the validation event, the external member of the panel, who was a university lecturer in Law in Ireland, provided the University, and through it the College, with an indicative specification of the library resources that would be needed for the first year of the LLB. The College subsequently undertook to provide these materials by the date specified in the record of the validation event. In addition, the visiting panel recommended that College plans for the development of library and learning resources should be 'more explicitly developed' and aligned with the delivery of the LLB. The panel allowed the College until September 1999 to respond to this recommendation, which it did not make a condition of validation. Members of the University told the audit team that they considered it would have been unrealistic for the panel which had validated the LLB to have insisted on a three-year development and funding plan for the College's library, as a condition of approving the programme.
44 The University has a number of means available to it to monitor the development of the College's learning environment: the ACR (which will include summaries of student feedback information); the reports of the external examiners; and the report of its Verifier (see below, paragraph 47). Two of these sources provide the University with information which relates to the learning environment in previous sessions: only the Verifier is in a position to provide the University with more immediate information on the learning environment in the College. Under present arrangements, however, the University expects Verifiers to visit annually, on one occasion at the end of each session; information on the maintenance and development of the learning environment in a partner institution might therefore be received by the University up to a year after any relevant development (whether negative or positive). Members of the College told the team that it intended to supplement its library stocks year by year to support the LLB and that it was in the process of producing the development plan for library and learning resources which had been a recommendation of the University's visiting panel. The University is confident that the means at its disposal will allow it to monitor developments satisfactorily throughout the three-year period before the next institutional review; because of the time lag inherent in the University's present annual reporting and monitoring arrangements, described above, the team does not necessarily share this confidence.
The academic standards of credits and awards
45 The Commentary stated that students are admitted to the LLB through admission arrangements operated by the College but governed by the University's admissions policies. The University plays no part in student induction, but LLB students at the College had attended an induction programme during which the nature of the involvement of the University in the programme had been explained.
46 The University requires the regulations for a validated programme to describe the procedures and methods of assessment, and to demonstrate how external examiners will be involved in all forms of assessment which count towards the recommendation of an award. The College's validation proposal sets out detailed procedures relating to student assessment but one of the conditions imposed by the University's visiting panel was that the generic assessment criteria in the document should be more fully set out. The University requires that assessment for its awards be the responsibility of a single board of examiners and that this board shall determine the marks, progression, and awards of all students following the programme. The audit team was satisfied that the University had secured evidence that the necessary assessment procedures had been put in place by the College although, at the time of the audit, many had yet to be brought into operation.
The work of the University's Verifier
47 In developing its arrangements to assure the quality of validated programmes and safeguard the standards of the resulting awards the University states in the Handbook that the Chair of the Validation Sub-Committee should appoint a 'Verifier' to each validated programme of studies. The Verifier is required to attend examinations, assessment and progression boards, and monitor that their conduct is in accordance with the University's requirements; verify the adequacy and accuracy of the ACR; and submit an annual report on these matters to the Validation Services Sub-Committee. The Handbook states that a Verifier would not normally be required to visit a validated centre more than twice per cohort progression, and that an annual visit, timed to coincide with the meeting of the board of examiners charged with recommending awards, would normally be sufficient. This annual visit is normally expected to last no more than two days, during which time, in addition to attending one or more examination boards, the Verifier would also be expected to check that all aspects of curriculum delivery, teaching and learning practice, assessment procedures, learning facilities, quality assurance mechanisms, staffing and promotion and advertising are as specified in the definitive course document, the conditions of validation, and the Institutional Agreement.
48 The Verifier for the LLB programme at the College had recently been appointed but had had extended experience of the College and its learning environment as a representative of the University; she had been a member of a Southampton Institute franchise validation panel which had visited the College in 1997; she had been drawn into the early discussions between the College and the University concerning the proposed LLB in Irish Law; and she had also been a member of the visiting panel which had validated the LLB. At the time of the audit, the Verifier had yet to visit the College in her new role, and her view of how she intended to approach her responsibilities envisaged being a 'critical friend' to the programme team in the College.
49 In its current notes on 'The Role of the Verifier', the University states that the principal function of the Verifier is to protect the 'integrity of The Nottingham Trent University award which has been conferred on a teaching centre as a result of a successful validation exercise'. In one section of these notes, the University states that frequent contact with the validated centre should be interpreted as a sign that the latter 'is not adequately prepared to deliver the approved programme, a matter which should be reported to the Chair of the Validation Services Sub-committee immediately'. Later in the same notes, however, it is suggested that in relation to the development of a centre the Verifier should 'feel able to act as a critical friend and engage in a debate about teaching and learning philosophies, developments and strategies'. The University may wish to reflect upon the possibility that the active engagement of a Verifier in supporting an institution in the development of its teaching and learning strategy might conflict with the same person's responsibilities for acting as the guardian of the University's interests, and whether it might be desirable, in the interests of creating more robust arrangements, to introduce additional points of contact between the University and its partners.
50 The Handbook requires the Verifier 'to check that the validated course is being delivered as specified in the definitive document and as agreed under the terms of validation and that the academic standard of the work is commensurate with the award'. The Handbook warns Verifiers that they may well find themselves 'in conflict with either the internal assessors, or the external examiner or even both'. In these circumstances the Handbook advises that 'the Verifier should establish that he/she is a representative of the Validation Services Sub-Committee which is a sub-committee of the NTU Academic Board and that all awards are made subject to this Board's ratification'. Verifiers are advised that when visiting a partner institution, they should 'offer to clarify issues about assessment procedures and regulations'; 'offer advice on the interpretation of regulations in non-standard situations'; and 'offer advice on procedures for student appeals and to indicate the type and rigour of documentation required from the teaching centre should a student make a final appeal to the award giving body, NTU'. Verifiers are advised to 'refrain from entering into debate about ... [individual student] work and grades unless there is concern about technical irregularities which may cause the board's decision to be deemed invalid at a later date' and to 'ensure that the external examiner fully understands the role and maintains an appropriate standard of academic quality commensurate with the award'.
51 Whilst the University sees the role of the Verifier as 'discrete and independent' from that of the external examiner, it suggests that 'in some arrangements, particularly overseas, neither the teaching centre nor the external examiner will be fully conversant with the respective roles that each should fulfil'. In such circumstances, it adds 'it may be necessary for the Verifier to provide appropriate advice and guidance to the teaching centre and to the external examiner(s) on the external examining role, including reporting requirements, whilst refraining from becoming involved in the moderation process'. The University's advice to its Verifiers is that in order to check academic standards, 'some form of sampling may be necessary [by the Verifier], without offering any academic judgements on submitted work or contributing to the assessment process'.
52 From its discussions with members of the University, including the Verifier for the LLB, and its review of the Handbook and the University's most recent advice to its Verifiers, summarised above, the audit team formed the view that satisfactory performance of the role of the Verifier was of crucial importance for the University's capacity to assure the quality of the educational provision leading to its award. Reviewing the range of tasks that the Verifier is expected to perform, the team found it difficult to envisage all of them being completed satisfactorily during a single annual visit to the College, particularly since there appeared to be no provision within the University's current arrangements for the Verifier to receive any information about the operation of the programme in the intervening period.
The appointment of external examiners
53 The Handbook sets out the powers, duties, and rights of external examiners, the criteria by which their fitness for appointment is to be judged, and the process through which they are to be appointed. External examiners are nominated by the College for the approval of the University's Academic Board through the Validation Services Sub Committee and its Chair. One of the formal conditions attached by the visiting panel to the approval of the LLB (Hons) in Irish Law was the appointment of two external examiners to the programme by the time of its commencement. Subject to the comments made above about the role of the Verifier, external examiners for programmes such as the LLB are expected to assure the University that the level and academic standard of the awards conferred on students are comparable in standard with equivalent awards of the University, and the UK higher education sector more generally, regardless of the location of the programme. The University is aware that the appointment of external examiners possessing the necessary familiarity with academic standards in UK higher education, particularly in the case of overseas partnerships, demands its close attention to their selection and support. The University told the audit team that it declines many of the nominations it receives from its overseas partner institutions and that it regards its approval procedures as a safeguard for the academic standards of its awards.
54 In Autumn 1998, the nominations of the external examiners were sent by the College to the University which, in line with its own procedures, sought the views of the recently-appointed Verifier on their suitability. The University's papers show that the Verifier gave the nominations careful consideration, and that she advised Validation Services to seek further information from the College. The information provided by the University to support the QAA audit showed that the two external examiners submitted by the College for the University's approval had little direct experience of setting and monitoring the academic standards of awards in UK higher education. The University's papers also showed that one of the nominees might have had difficulty in satisfying the University's requirement that its external examiners should be able to demonstrate 'current and recent active involvement in research/ scholarly/ professional activities in the field of study concerned', and that one of the nominees also held more than the maximum of two concurrent external examinerships specified as allowable by the Handbook. The Handbook states that 'external examiners ... should not have previous close involvement with the [partner] Institution or University' and that 'the proposed examiner should not have been ... an examiner on a cognate programme in the Institution'. Both nominees put forward by the College had already held external examining appointments for other programmes offered by it; one for six and one for nine years respectively. These latter attributes do not, however, appear to have been considered by either the College or the University as impediments to the appointment of the nominated external examiners, notwithstanding the requirements of the Handbook.
55 Following the confirmation of their appointments by the University the external examiners for the LLB were provided, via the College, with a number of University documents. These included: the Handbook, a paper relating to the benchmarking of Law degrees and an external examiners' briefing pack.
56 The Nottingham Trent University's recent partnership with Griffith College, Ireland, is managed through its Validation Services, which is part of the Office of Academic Planning and Quality Management, and which reports directly to the Deputy Vice-Chancellor. The College is described as a 'validation centre' by the University, which designation implies a significant degree of delegation of responsibility for quality assurance matters.
57 The University's operations in this area are governed by its comprehensive Handbook for Validation Services. This Handbook is designed to guide the work of Validation Services and other members of the University, its external examiners and its partner institutions, in assuring the quality of validated educational provision and safeguarding the standard of the University's awards. While generally commendable, there are nonetheless some gaps in the guidance it offers. For example, the Handbook establishes no requirement for the University to confirm the good standing and financial stability and resources of a potential partner.
58 The University requires potential partners to have strong internal quality assurance systems and sets out the features which it expects to see in place and operating effectively. The University uses a two-part approach to checking that the proposed arrangements are satisfactory: in the first part the institution itself undergoes a process of approval while in the second the programme of study is subject to a validation procedure.
59 The University's present approval procedures appear to make insufficient allowance for circumstances where it has little direct prior knowledge of its proposed partner. Where this is the case the University may be obliged to rely heavily upon a written description of quality assurance arrangements, which may not yet be tried and tested. Nor, under the procedures currently operated, does the University explicitly envisage an evolutionary relationship, in which it exercises a degree of control over the activities of a newly-approved partner until the latter has demonstrated its ability to operate sufficiently independently to justify a lighter touch. From the start, the University does not expect to have to exercise close day-to-day control over its partners.
60 In this particular instance there appears to have been no evidence available to the University, or to its institutional review and validation panel, which made it clear that the partner was operating quality assurance mechanisms of the type envisaged within the University's Handbook as necessary. It is difficult to be confident, therefore, that the University had established, as fully as its own published procedures required, that its proposed partner possessed the characteristics deemed necessary by the Handbook. This is not, of course, in any way a criticism of Griffith College or its procedures.
61 The University's approval processes, as set out in its Handbook, require a clear distinction between the institutional review and programme validation elements. In this instance, both elements were dealt with on the same day. Although this possibility is envisaged within the University's Handbook, the official record of the events did not make it clear that all the steps prescribed by the Handbook were followed and the two events kept entirely separate. The University will wish to review the specifications for its records of institutional reviews and validation events. The validation process which applied to the LLB programme of studies in the College closely followed the validation process set out in the University's Handbook.
62 Programmes of study are validated for a period of three years, after which they are subject to review. In the interval between validation and review, the University relies on its 'Verifier' and its external examiners to confirm the reliability of the annual course reports it receives on the progress of the validated programmes, and to ensure that the academic standard of its award is being maintained. The terms of reference provided for Verifiers emphasise that they are to attend the awards and assessment boards for the programmes to which they are appointed. Their duty is to assist the University's partners to understand and follow its regulations and to safeguard the standards of the University's awards. They are also expected to provide some developmental support. The University's arrangements do not provide for validated centres to keep Verifiers informed of relevant developments in the interval between their visits, and Verifiers are expected to carry out the range of tasks assigned to them in the course of one annual visit to the validated centre, usually of no more than two days, during which assessment boards are to be held. This makes it difficult for the Verifiers to carry out their responsibilities fully effectively.
63 The University recognises that the reports of the external examiners provide it with the most authoritative evidence that the standard of its validated awards are comparable to that of similar awards attained on its 'home' campuses. For this evidence to be authoritative, though, external examiners need to be familiar with the academic standards current in equivalent programmes in UK universities. They also need to meet the University's normal criteria for appointment. In this particular instance, the University's procedures for scrutiny of the nominations of the external examiners it subsequently approved for appointment do not appear to have been fully effective.
64 The University's validation arrangement with Griffith College is still in its early days and there is no evidence that it is anything other than a sound relationship. Nevertheless there were instances in the course of the institutional review and validation process where procedures were not carried out as specified in the Handbook. This should cause the University to reflect on its confidence in the design and operation of the quality assurance arrangements for its validated provision more generally. In doing so it should also note that there are a number of instances where the procedures laid down in the Handbook, particularly those relating to institutional review and the role of the Verifier, would benefit from further consideration.
Comments on the audit report supplied by The Nottingham Trent University
Commentary
At the time of the audit the University's validation service arrangement with Griffith College Dublin was still in its first year of operation. This meant that, not only had most of the University's quality assurance mechanisms to underpin the arrangement not yet come into full operation, but also that no student work was available for a judgement of attained standards to be made. The extent to which the auditors could therefore make an informed and valid judgement of the collaborative partnership within the context of their brief was thus severely limited. With hindsight the University would urge QAA to reconsider the appropriateness of conducting audits of new relationships in the future. Notwithstanding this, the University is pleased that the auditors conclude that 'there is no evidence that it is anything other than a sound relationship.'
Since the audit visit the University has received and considered, through its formal mechanisms, the Annual Course Report, the external examiners' reports and the Verifier's report in respect of the operation of the first year of the LLB. All four reports provided evidence that enabled the Validation Service & Collaborative Activities Sub-Committee to conclude that the quality and standard of the University's award at Griffith is secure. The College has also prepared and submitted, as agreed, a plan for the development of the library and learning resources and a staff development plan explicitly geared to the needs of the delivery of the LLB, thus justifying the confidence of the University in this respect.
The University's quality assurance arrangements for all its academic provision are constantly evolving and developing. While we welcome the report's observation that the Validation Service Handbook is comprehensive, generally commendable, and in many instances goes beyond the suggestions of the HEQC Code of practice, we also recognise that the reissue of our handbooks is a priority. New editions will take due account of QAA's emerging code of practice, including arrangements for establishing the probity of potential new partners. A revised 'housestyle' for reporting of events and approvals, linked more explicitly to published procedures and quality processes is also being introduced, which should provide more comprehensive documentation of the process and outcomes of quality assurance activities.
The nomination forms submitted to the University for the appointment of external examiners did not include details of previous appointments held by the nominees at Griffith. The form will be amended to ensure the relevant section is more explicit. One examiner was in transition between retirement and assumption of new appointments and did not therefore exceed the maximum number of concurrent appointments specified in the Handbook. The University remains satisfied with the appropriateness of the external examiner appointments for this arrangement, both in terms of their scholarly and professional activities in the fields concerned and in their familiarity with UK academic standards.
The University does not accept the auditors' comment that no evidence was available to the Institutional Review panel on the quality assurance mechanisms in place at Griffith. It does, however, accept that consideration of quality assurance could have been documented more fully in the validation report. The management structures of Griffith College are constantly evolving to support the rapid pace of growth. The College is conscious that, in its next phase of expansion, it will be advisable to formalise the terms of reference of its various academic committees so that the role of each and the interaction between each can be more clearly seen externally.
The University recognises that the role of the Verifier is both critical and difficult. Consequently the effectiveness of the Verifier's role is kept under constant review, including, as discussed with the auditors, a formal annual review with the Verifiers themselves. This annual monitoring has resulted in the continuing development of the quality assurance arrangements governing the validation service.
The University is surprised that the report comments on arrangements for approving partners of which it has little prior knowledge, since these were not the focus of the audit and were not discussed with the audit team. The assumptions made by the auditors are incorrect. The University does not rely on written descriptions of quality assurance arrangements and does indeed operate an evolutionary relationship with new partners where it deems it appropriate. The University has acquired considerable experience of putting new arrangements into place over a ten year period and is confident of the robustness of its procedures.
Top