Preface
Quality Assurance of Overseas Collaborative Provision
The Quality Assurance Agency for Higher Education (QAA) was established in 1997 to promote public confidence that the quality of provision and standards of awards in higher education offered by UK institutions are being safeguarded and enhanced. As part of its activity, QAA undertakes regular academic quality audits of individual institutions to review the operation and effectiveness of arrangements for assuring quality and standards. A brief guide to the various quality assurance processes operating in UK higher education is available from QAA on request.
Quality audits also cover the arrangements which institutions use to assure the quality and standards of their awards and programmes offered in collaboration with other partners, both within and outside the UK. As part of this process, QAA audit teams visit overseas partners of UK institutions so that the same enquiries can be made about how the quality and standards of UK awards and programmes offered to students outside the UK are safeguarded, as are made of UK-based provision. This initiative is designed to help provide enhanced confidence in the work of British universities and colleges operating outside the UK.
QAA's audits of overseas partnerships use as their main reference point the Code of Practice for Overseas Collaborative Provision in Higher Education 1996 (2nd edition), which was published by the former Higher Education Quality Council (HEQC) and which QAA has endorsed for the purpose of the current programme of visits. This offers guidance on good practice, and a framework within which institutions can review and consider their current and future activities. The Code has been widely welcomed by universities and colleges. While UK institutions participating in the programme have not been narrowly 'measured' against the Code (which is not intended to be a definitive check list), their experience of using it, and the findings from the overseas visits in general, will contribute to its revision and further development.
This report is published following an overseas audit visit to Portobello College, undertaken in the autumn of 1998, which looked at collaborative arrangements between the University of Wales and the College. It should be read in conjunction with QAA's and the former HEQC's published audit reports of the University of Wales, details of which can be found in this report.
The UK University and College which participated in this overseas audit visit did so voluntarily.
Introduction
1 This is the report of an audit carried out by the Quality Assurance
Agency for Higher Education (QAA), of the quality assurance arrangements
for a partnership between the University of Wales (also referred to as
'the University' or 'Wales') and Portobello College (also referred to as
'the College' or 'Portobello') in Dublin, the Republic of Ireland. The
partnership is established for the purpose of offering students in the
Republic of Ireland opportunities to study there for an award of the University
of Wales; specifically the three-year and two-year LLB programmes.
2 The quality audit of the University's link with Portobello was undertaken in the Autumn of 1998, as a single overseas visit, partly in response to a request from the University but also following on from earlier Higher Education Quality Council (HEQC) audits of the University's collaborative work. Further information about quality audit and QAA is contained in the preface at the front of this report.
3 The quality assurance arrangements of the University were the subject of an overseas collaborative audit by HEQC resulting in the publication of two Quality Audit Reports on the University's collaborative work in Spain. These reports were published in December 1996, and include a general description of the mechanisms used by the University at that time for the assurance of the quality of its academic provision, (see below, paragraph 11). Fuller explanation and comment on the University's structure is contained in the HEQC Quality Audit Report on the Federal University of Wales published in August 1994, although since that date, there have been some important changes to the internal University structures with more authority having been delegated by the University to its constituent institutions.
4 QAA is grateful to the staff and students of Portobello College and the University for their assistance and co-operation with the audit team.
The audit process
5 Prior to the audit visit, the University provided QAA with briefing
documents, comprising a description of the University's process for the
validation and delivery of award-bearing courses overseas, namely its International
Validation: Handbook of Quality Assurance; the relevant memorandum
of agreement between the University of Wales and Portobello College; documentation
on the approvals and subsequent review of the LLB programmes; the College Prospectus and
the minutes of several meetings of relevant University Committees. The
audit team requested various supplementary documents to clarify its understanding
of the relationship, including the College's annual monitoring reports
to the University, external examiner and moderator reports, and course
and student handbooks. The University was able to provide all of the material
requested.
6 Members of the audit team met with a member of staff from the University Validation Unit and a current external examiner, who had previously been a member of the University's quinquennial review team. Discussions took place between members of the team and staff of the College on 27 October 1998, and meetings were held with some 30 people, including the President of the College, the Head of Law, the Assistant Director Law and Finance, members of the full-time and part-time teaching staff (including internal examiners), administrative and academic support staff and students. The team also had a meeting with the University's moderator. After its return from Dublin, the team clarified some minor points with the University through correspondence.
7 The audit team comprised Professor J Rear, Dr D Timms, auditors, and Ms G Clarke, audit secretary. The audit was co-ordinated for QAA by Miss A R Hynes, Acting Assistant Director for the Institutional Review Directorate, who accompanied the team to Dublin.
The audit context - Republic of Ireland
8 The National Council for Educational Awards (NCEA) is the statutory
body which is a licensing authority and awarding body for non-university
sector higher education in the Republic of Ireland. There are eight universities
in Eire, including one outside the state-funded system, four of which constitute
the National University of Ireland together with three recognised colleges.
There are about 50 colleges or other institutions offering higher education
which are accredited by the NCEA as designated institutions. Portobello
College holds NCEA designated institution status. Private colleges form
a small proportion of the higher education provision in Ireland and are
primarily focused in the business and professional field (eg accountancy,
law, computing). The population of students in higher education has grown
from 20,000 in 1965 to almost 97,000 in 1997 with an expectation that this
will grow to 120,000 by the year 2000. Just over 56 per cent of
higher education students are studying in the universities. At present
students who study in private institutions in Eire are not eligible to
receive public support for study at degree level in contrast to their peers
in the state-funded universities and colleges. This is a position which
is currently the subject of a legal challenge.
9 The Honourable Society of King's Inn and The Incorporated Law Society of Ireland are responsible for professional training in law in the Republic of Ireland. In Northern Ireland, this role is undertaken by the Northern Ireland Institute of Professional Legal Studies. In order to prepare students for the relevant professional examinations, the curriculum of the LLB programme at Portobello College validated by the University of Wales is closely matched to the requirements of these professional bodies (see below, paragraph 13).
10 The audit team had regard to the HEQC Code of Practice for Overseas Collaborative Provision in Higher Education 1996 (2nd edition) when conducting the audit.
Matters given consideration in the audit
The University context
11 At the time of the audit visit the University was supervising about
7,500 students world-wide, following programmes of study leading to University
of Wales' awards, validated or franchised by the University and delivered
through 47 centres abroad. An International Validation: Handbook of
Quality Assurance sets out clearly the policies and procedures that
govern all of the University's overseas collaborative activity. This document,
hereafter described as the 'Handbook' has been subject to a number
of revisions as the University's procedures have developed, and at the
time of the audit the July 1998 version VI was operational. The University
of Wales exercises formal responsibility for the validation of programmes
of study offered through overseas collaborative partnerships through its
Validation Board, a sub-committee of the Academic Board. The Validation
Board was established in 1974 and its membership is drawn from the constituent
and associate institutions of the University. To manage the number of overseas
operations the Board has established a Standing Executive Committee, which
is responsible for the detailed monitoring of quality assurance processes.
The mission statement of the Validation Board commits it to ensuring that
'in matters relating to validation and franchising, the University's academic
standing and integrity is safeguarded at all times'. In addition, the Board
describes its 'working philosophy' as 'demanding excellence by focusing
on quality assurance and appraisal mechanisms alongside regard for appropriate
course content, whilst maintaining a workable and flexible approach to
potential partners'. The Validation Unit services the Validation Board
and administers the programme of international validations. The Unit is
part of the University Registry; it comprises about seven full-time staff
and two part- time staff, and it provides the main security for the University
that effective quality assurance processes are operating across the wide
range and scale of collaborative partnerships. The audit team recognised
the significant informal as well as formal role played by the Unit as indicated
below, paragraphs 23, 45 and 59. However, the team had some concerns as
to the longer term sustainability of the present arrangements should there
be continuing expansion of the University's collaborative operations.
The background to the partnership
12 Portobello College is a private institution which has the objective of providing tertiary-level education primarily in business and professionally-based subjects. It was founded in 1988 as the Dublin Institute of Education Business College and developed into Portobello College. It now provides almost exclusively third level and professional studies. Current enrolment at the College is just under 900 students, of whom over 150 are following programmes leading to the LLB. The College is a designated institution of the Irish National Council for Educational Awards (NCEA), (see above, paragraph 8), and within this relationship awards a range of degrees in business studies. In addition to its partnership with the University of Wales, it has validation arrangements with the University of Glamorgan for a business degree programme. Portobello College occupies a campus in the centre of Dublin, comprising four buildings with tutorial and lecture space, computer workshops, a library and café, and gymnasium facilities. The Law Department of the College has six permanent staff and, according to need, is also supported by part-time staff, often themselves in legal practice.
13 University of Wales collaboration with Portobello College Dublin formally
began in 1992 after the College made an approach to the University. This
approach resulted in the validation of a three-year LLB and a one-year
postgraduate Diploma in Legal Studies, designed for non-law graduates who
wished to prepare for the professional examinations of the Incorporated
Law Society of Ireland (ICLSI). Since that date, the diploma has been discontinued
and has been replaced by a two-year LLB degree, also designed for graduates
with degrees in subjects other than law. Both LLB programmes are in Irish
Law and the medium of instruction is English. The course content of both
the three-year and the two-year programmes covers core subjects required
for application to qualify to enter the legal profession in the Republic
of Ireland: Contract, Criminal Law, Constitutional Law, Property, Tort,
and Company Law. In addition, there are optional subjects, though owing
to time constraints there are fewer options for two-year than for three-year
students. Graduates of the LLB programmes are currently entitled to apply
for admission to the Institute of Professional Legal Studies of Northern
Ireland. Portobello College designed the programmes within the constraints
of the requirements of the qualifying examinations for the Irish legal
profession. From the outset of the development of the programmes, Portobello
had the benefit of advice from a member of the Law School of the University
of Wales, Cardiff, who is an expert in Irish Property Law, who was subsequently
appointed as the University's moderator for the course. He has remained
in this post to date (see below, paragraphs 24 to 30).
14 The original legal agreement of 1992 between the University of Wales
and Portobello College was updated in 1998 and now includes references
to admission requirements (including student numbers); the conduct of examinations;
quality assurance procedures (such as arrangements for review); the information
requirements of the University (such as details of teaching staff); arrangements
for termination of the agreement; and control of promotional material,
as well as stipulations about payments. A guarantee bond with Allied Irish
Banks plc, under which, in the event of the financial failure of the College,
the continued provision of students' courses is assured, covers students
of the College. It appeared from the legal agreement that the University
of Wales had made no commitment to students beyond any termination of the
agreement, and Portobello College's guarantee bond was entered into without
the University's prompting. Recently the University had decided that Portobello
College had met the criteria to merit the title of Affiliated Institution
of the University of Wales.
Initial approval and validation processes
15 The Handbook sets out a standard process by which initial and final approvals are given for a programme validated at an overseas institution. Under the terms of this process, an initial visit is made to the institution and, on the basis of the evidence gathered, a decision whether to proceed to formal validation is made by the Validation Board of the University. If it is determined that the validation may proceed, a full course submission is requested, on the basis of which a panel of assessors appointed by the Validation Board visits the institution. Within its report of this event, the Validation Panel makes a recommendation to the University's Academic Board.
16 The audit team was informed that the provisions of the current procedures were not all in force when the partnership was originally established. The team did not look at the 1992 version in detail, but it appeared that the panel of assessors made its visit on the basis of a document that did not constitute a full description of the proposed programme, since a full document was actually prescribed as a condition of successful validation of the original three-year LLB and Diploma. Nevertheless, the validation panel itself was composed largely of members external to the University of Wales and seemed to the team to have thoroughly considered the programmes and the resources to support them.
17 In May 1993 the College submitted a further document making a proposal
for a two-year version of the LLB degree to replace the Diploma course.
On this occasion the University determined that a postal process was sufficient
to effect validation. The submission document was sent to the assessors
who had validated the three-year degree, plus one additional assessor.
The written responses made by the assessors indicate that the document
was again somewhat deficient, provoking a series of important questions
about the proposal. On the basis of some of the assessors' responses, the
University asked the College to comment. The University provided these
responses to the individual assessors who posed the questions. However,
individual assessors appear not to have been informed of the queries raised
by the others, and the responses of the College did not appear to the audit
team to answer all the assessors' questions. Nonetheless the assessors
appear finally to have accepted the programme proposed and the University
formally approved it on 14 September 1993. The team considered that in
the case of the two-year degree in 1993, while the University went through
the motions of a validation process, it did not treat its findings with
the attention they deserved and the new programme was approved on a less
than secure basis. The University may wish to consider whether, particularly
in the case of postal validation processes, its procedures now ensure that
fully informed judgements are made about new proposals.
18 Modifications to Portobello programmes are proposed to, and considered by, the LLB examinations board, which, having concluded its examinations business, reconvenes itself as an augmented course committee. By this means, the course committee has the benefit of the views of the external examiners and the University's moderator (the University link person with the College) in considering modifications. This process appeared to the audit team to be working satisfactorily, and the close involvement of expert individuals external to the College seemed to the team to give the University reliable assurance that proposed changes made to the programmes would be properly informed by external views.
19 An annual report (called by the University the Annual College and
Course Report Form) is submitted by the partner institution to the
University for the purposes of monitoring the programme and informing the
University of any proposed changes to the programmes of study, including
staffing changes. The forms state that amendments 'must be sanctioned by
the University in advance of implementation'. The only body which sees
the forms is the Standing Executive Committee of the Validation Board and
it presumably must therefore be taken to be this body which sanctions changes
on behalf of the University. However the style of the reports of the Executive
Committee is to refer to 'considering and receiving' annual response forms
and the reports of external examiners and moderators; on the evidence available
to the audit team no explicit mention of the approval of any change to
a scheme of study appears in the reports of Standing Executive Committee
meetings. It was therefore not clear to the team whether the University
does in fact formally indicate its satisfaction or otherwise with proposed
amendments, and given the volume of business of the Standing Executive
Committee it seemed unlikely that it has real opportunities to consider
amendments. On the limited evidence seen by the team, at least in relation
to minor amendments, de facto approval appears to be given by the
Validation Unit on the recommendation of the moderator alone or is simply
inferred from the fact that reports of the moderator and external examiners
approving changes have been considered by the Standing Committee without
comment. Yet the formal statements of the roles of moderators and external
examiners do not in terms allocate such approval functions to these postholders.
The University may wish to consider whether its mechanism for considering
changes to programmes and for signalling its acceptance or otherwise of
proposed changes fully meets its needs. At the least there appears to be
a need to review, and make consistent, the wording of the various regulations,
forms and guidelines which touch this important matter (see below, paragraph
40).
Arrangements for programme review and monitoring
20 All validated programmes of the University are subject to review within a cycle, normally of five years' duration. The Portobello LLB programmes were last reviewed in May 1997. The process for review is closely related to the processes for validation and is conducted on the same basis: a review document is submitted and, on the basis of that document, a panel of assessors, mostly external to the University, is appointed by the Validation Board to make recommendations on the programme. Under the rules for the composition of review teams, the reviewers for the Portobello programmes consisted of a member of the University's Validation Board (as Chair of the panel of reviewers), the external examiners for the programmes, and an additional external expert assessor. The practice at the time was for the review team also to include the moderator. The panel, apparently prompted by the moderator himself, commented that the close involvement of the moderator with the programmes under review might put him or her in a position where his or her own role might be under consideration as a part of the review exercise, and it recommended that the University reconsider whether review panels should include moderators as full members. The University has since changed this policy and in the July 1998 edition of the Handbook, it states that the moderator is now 'not appointed as a formal member of the Review panel'.
21 In addition to review arrangements, monitoring of the LLB programme is undertaken in three related ways. First, external examiners are appointed by the University and are required to report annually. Second, the College is required to make an annual return to the University detailing its internal arrangements for ongoing monitoring of the programmes and also for any changes or developments during the year. This return includes a range of statistical data. Third, a moderator appointed by the University reports on an annual basis. The roles of external examiners are discussed elsewhere in this report (see above, paragraph 20, and below, paragraphs 23 and 46) and detailed consideration of the role of moderator is given below in paragraphs 24 to 30.
22 The annual reports made by Portobello College to the Validation Board were comprehensive documents and followed closely the guidelines indicated by the Handbook. The reports consisted in large measure of extant papers, such as notes of departmental and examination board meetings. However, the audit team considered that the reports were not the product of truly self-critical reflection on the successes or weaknesses of the programmes, and had not been disseminated amongst the staff or students of the College. It was the view of the team that the University might consider how to give further guidance to the College to assist the latter to be more reflective and analytical about its provision.
23 A Standing Executive Committee of the Validation Board, comprising six members, considers the annual report on behalf of the Board. Given the number of validated centres, the Standing Executive Committee has a considerable task in giving detailed attention to the reports of any one centre. The somewhat miscellaneous nature of the Portobello reports cannot have made this task any more tractable; nor can the fact that, to judge by the four sets of 'reports' of the Standing Executive Committee's meetings supplied to the audit team, the committee is rarely at full strength in its meetings. The team understood that the Standing Executive Committee did look at each full report and checked distilled information from annual reports from external examiners and moderators. These synopses typically take the form of a summary of action points prepared by officers. The quality of the synopses is obviously an important factor in the Committee's ability to perform its monitoring role. Validated institutions are also required to return a 'Response Form' commenting on action taken as a result of recommendations by external examiners and moderators, which is copied to the relevant external examiners and moderators. Whilst this is a commendable attempt to ensure that the recommendations of external examiners and moderators are considered, the utility of the forms seems to some extent impaired by the fact that they are dealt with typically 12-15 months after the submission of the reports to which they respond. The combination of volume of business of the Standing Executive Committee and the time-cycle over which it is handled seems to put an extremely demanding burden on its members and on the officers of the Validation Unit.
24 On successful validation of a programme of study, the University of Wales appoints a moderator for the programme. The duties of the moderator have changed over the past few years as the University's practice on collaborative arrangements has developed. At present the moderator is expected to act in both a quality enhancement and a development role, and also to report on standards annually to the Validation Board via the Standing Executive Committee. The moderator liaises with the validated institution over the preparation of examination papers and assists at the examination board to ensure that the requirements of the University of Wales are met. The moderator is also expected to assist the validated institution with certain other quality assurance requirements, such as preparation for reviews.
25 The Handbook states that in the developmental role the moderator is 'expected' to ensure generally that the academic integrity of the validated scheme of study will stand up to the scrutiny of the external examiners and that the University's 'expected standards of academic achievement are established and maintained'. For these purposes the moderator is required to pay 'particular attention' to assessment practice and procedures; teaching strategies and learning outcomes; physical resources; staffing and staff development; student feedback; and the whole management and operation of the validated scheme, including the review and implementation of the external examiners' and moderator's comments and recommendations. It is expected that staff development seminars will be conducted 'as necessary'. To carry out this very wide range of duties the moderator is expected to keep 'close and regular contact' with the validated institution, which in 'the nascent stages' means at least twice a year. The moderator for Portobello also acts as an adjudicator in the early stages of student appeals and, if necessary, staff complaints. The audit team understands that, although the moderator is not part of the College's formal management structure, the role is used to ensure that the University's requirements, including those concerning library and other resources, are fully met.
26 The Handbook makes no specification about the duration of a moderator's term of office. The present moderator for the Portobello programmes has been serving since its inception, thus more than six years. The moderator was a member of the original review panel; one of those called upon in the postal validation of the two-year programme; and a member of the review panel in 1997. The moderator's reports have assured the University every year, without equivocation, that the University's standards have been maintained at the College. The audit team read several annual reports and noted that in format they were closely aligned to those required from external examiners. In meetings, however, the team heard of the wide range of work undertaken by the moderator, the majority of which was not evident from his reports. The reports did not readily elicit full consideration of the aspects which a moderator might have been expected to monitor, nor did they allow the Validation Unit to be aware of the frequency or nature of contact between the partner and the moderator. It appeared to the team that the use of a common annual report form for both external examiners and moderators obscured the important differences between the roles and encouraged a potentially unhelpful overlap. The University may wish to consider the advisability of reconsidering the format of moderators' reports to aid more effective monitoring.
27 The audit team heard that the proprietor of the College regarded the collaboration with the University as a model for other institutions, and it was clear that a major reason for this rested on the relationship which this particular University moderator has developed with the College. The moderator at Portobello College has from the outset fulfilled a role going beyond even the University's present expectations. Being a uniquely qualified expert in Irish Law as well as one of the University's senior professors, his understanding of the course and its aims and objectives was from the outset considerable. Owing to his other academic and advisory commitments and his professional engagements in Dublin, the moderator has been able to maintain close contact with the College, enabling him to visit on virtually a weekly basis. He has been peculiarly well-qualified and positioned to advise the College generally and to bear witness to the University as to the standards being maintained. It has also been the practice of the current moderator to chair the LLB Examination Board throughout the life of the scheme, rather than attending the Board merely as an observer. By chairing the panels which have appointed the academic staff, he has further assisted in ensuring that the College's staff are fully qualified to teach and assess to the University's standards.
28 The audit team read in the validation submissions that the present moderator acted as a consultant in developing the programmes before they began, and the team was informed by the College, and the moderator himself, that he also acted as an academic consultant to the College, receiving a retainer fee from the College throughout his period of office. This was a fact that he had notified to the University and one known to them when he was appointed to the office, and the team was made aware by the University of the uncharacteristic nature of this partner/moderator relationship. The team was conscious of the potential for conflicts of interest, and therefore had some concerns about this model of operation. The moderator stated that he was very aware of the danger but believed he was successfully balancing the roles. In the team's view, given those involved, the benefits outweighed the disadvantages for this particular partnership. Certainly the moderator's close association seems to have had considerable benefits in terms of development for Portobello. An experienced external examining team has been appointed which is able to provide a balancing factor, (see below, paragraph 45) permitting the University to be confident, through additional sources of information and scrutiny, of the standards of student achievement. At the same time, the University benefits from an intensity of moderator influence and contact which it would not normally be able to secure through its usual voluntary system. To both staff and students, the moderator functions as the embodiment of the University of Wales, having a real presence within the College and acting, if need be, as a person to whose judgement staff and students can appeal if dissatisfied by the decisions of the management. However, the fact that the activities of the present moderator exceed by such a margin the requirements laid down in the Handbook, suggests that the benefits enjoyed by Portobello might not be continued in the event of a different appointment, and are untypical of the normal practice of the University's moderators.
29 The University has recently abandoned the concept of the 'link department' which it formerly employed to back up the role of the moderator. This move will place more responsibility with the moderator and the audit team had some doubts whether the University's high expectations of its moderators' time commitments were realistic. The team understood that the research missions of the University's constituent colleges might restrict the degree of co-operation that the University was able to obtain to support its collaborative arrangements. However, if this is indeed the case, the University may wish to reflect carefully on its present international policies and the advisability of examining the extent to which it has the necessary resources available to implement them.
30 The audit team learnt that in recent years the University has instituted
annual meetings of moderators to enable the exchange of good practice.
The team welcomed this development and would encourage the University to
look for additional means to benefit from the experiences of its best moderators
and to ensure that the valuable information to be gained from them is properly
exploited.
Arrangements for the admission of students
31 The formal agreement between the institutions binds the College to admit a viable minimum cohort of students from the outset and the College received clear guidance from the University on the entry standards it required. One of the points made in the original submission for programme validation was that it was desirable to increase access to the study of Law in Ireland, since the Irish universities were imposing high entry standards because of an excess of demand. The quinquennial review submission states that many of those disappointed students now come to the College, but the audit team noted that, while the College's students include many who have not attained sufficient points to enter the Republic's universities, in 1995-96 and 1996-97, 59 per cent and 76 per cent of entrants to the three-year course scored in excess of the minimum Leaving Certificate entry requirement. Entry to the two-year programme is confined to non-Law graduates. The team found no evidence that the College does not strictly apply the University's requirements as to admission. Indeed, it received evidence that the College exercises minimal discretion and that even where the applicant's 'points count' in the Leaving Certificate falls below the minimum 300 by only five points, cases are referred to the moderator for his decision. No doubts or issues have arisen about admissions policy or practice through the moderator, through any external examiner or in the findings of the quinquennial review panel.
32 Students are admitted and enrolled by the College, but they are registered not only as students of the College but also as students of the University of Wales. For this purpose the College sends to the University a composite list of students after the beginning of the academic year. Registration is handled by the University in liaison with the College; the University has no direct communication with the students other than through the moderator, not even formally to advise them that they are now students of the University and what that means. Indeed, the audit team learnt from the students that at no time do they receive communications of any kind from the University and they regard themselves as students of the College, not of the University. Students are aware of the University because of the documentation and an early meeting with the moderator, but generally the University has no formal significance for them until the day of graduation. The University may wish to consider whether it does all it should do to establish and maintain its relationship with this part of its student body.
33 Students who met the audit team gave assurances that they received good
advice about admission and what they might expect if admitted. All expressed
satisfaction with the quality and accuracy of the Prospectus, Student
Handbook and other information they are given about the College and
its programmes. They had seen the moderator on one or two occasions and
been helpfully advised about aspects of the programmes and legal practice.
They were aware of the moderator's presence and availability should they
want to take a matter further than the head of the College's Law department.
34 The audit team learnt that students receive about 18 hours of tuition (lectures and seminars) per week. Students have additionally the opportunity to take courses in study skills and information technology (basic word processing) and, if they are intending to practise Law in the UK, options are also available in some English legal subjects. In the syllabuses submitted for validation, no learning outcomes were set out beyond the very general objectives for each subject. Teaching styles remain traditional. In both lectures and seminars, students on both the two-year and three-year programmes are taught together. Noting that at the time of the original validation of the two-year programme some questions had been raised about this strategy, the team inquired of the students whether it had created any problems for them and were assured that it had not. The team also heard from an external examiner that this approach was unremarkable in English and Welsh legal education, with comparable teaching strategies being adopted for different cohorts in many programmes there. The team noted from the reports of the external examiners that, judging by results, the teaching was thought to be good. There were concerns about some quite high failure and drop-out rates, particularly in the first year, but the moderator and quinquennial reviewers had concluded this was a problem of attitude and motivation within the student body, particularly on the three-year course, rather than a consequence of any deficiency in the teaching. The students who met the team cheerfully acknowledged that this was the case, attributing it to the adjustment to the independence of student life in contrast to secondary school. They acknowledged the very active attention of the College in matters such as student attendance and course work submission.
35 The College's Prospectus claims that 'a student centred approach' is one of the four pillars of its operation. The College aspired in its original submissions to offer its students personal development and a full life. The audit team was not in a position fully to explore the meanings of these claims; nor did it receive a clear view as to the extent to which the University concerns itself with the quality of the higher educational student experience delivered by its partners, though clearly, through the annual reports and responses it receives from the moderator, external examiners and the College itself, the University does acquire relevant information. Moreover, a moderator who has such frequent and regular contact with the validated institution, its staff and students as the moderator of the link with Portobello College, is in a good position to make a judgement on this matter.
36 The audit team learnt that the moderator believes that the College's students are receiving a sound legal education. The team was disappointed to find that because the quinquennial review took place immediately following a summer examination board, the reviewers were unable to meet any students. According to the Handbook, this was a feature of the timing of all the University's quinquennial reviews. The team considered the lack of an opportunity for reviewers to speak with students left a gap in the current process which the University should address. The team noted, nevertheless, that the reviewers had inspected the College's facilities and found them to be more than satisfactory: the reviewers declared the law library was adequate and the computing facilities generous. The team also learnt from the students they met that there were apparently no significant areas of student dissatisfaction, either with the teaching or generally with arrangements made by the College. They indicated that the library was able to meet their needs, and books could also be obtained by the library from Trinity College, Dublin, if necessary. Students did however emphasise the importance of receiving timely feedback on their written work so that they could learn any lessons before tackling subsequent assignments.
37 The formal complaints procedures of the University apply within the
Law programmes, but problems are generally dealt with informally. The students
confirmed that the Head of Law and her staff, including part-time staff,
were all very accessible. A personal tutor system operates, with every
student assigned a tutor who posts set times of one or two hours a week
when he or she is available. Tutors may be changed by arrangement with
the head of department. The scale and size of the College mean that as
staff-student relations are good at an informal level there is less dependence
on the formal personal tutorial system. The department receives representations
from the student body through a system of class representatives, elected
by each class at the beginning of the academic year. The class representatives
meet regularly with the Head of Law and sometimes are called upon to make
presentations of the class's views to meetings of the department's staff,
who then discuss among themselves, in the absence of the class representatives,
what they have heard. The audit team also heard that, occasionally, issues
have been addressed by a meeting of staff with the whole year group; for
example to discuss poor attendance, but that this is not the usual way
in which decision-making is approached. The team, like the quinquennial
reviewers, concluded that the students were generally satisfied with their
experience at the College.
Student assessment and academic standards
38 In addition to its regulations in the University Calendar and the terms of its formal agreement with the College, the University has published in the Handbook a set of notes intended to provide general guidance to the conventions that shape the pattern of assessment and examination of the University's courses. In the early years of a partnership the predominant form of assessment is expected to be written unseen examinations. The notes set out guidance as to the form of examination papers, the conduct of examinations, examination boards, and the classification of student performance. There is a statement of the University's classification system which is expressed in terms of the characteristics of student performance. While the University accepts that it is legitimate for assessment practice between institutions to differ, variations from what the University regards as best practice is permitted only after full disclosure, discussion and agreement by the Validation Board. The scheme of assessment and the regulations implemented by the College are formally those of the College but they are either identical to or consistent with the regulations of the University. The audit team was satisfied that the written regulations and guidance provided by the University are sufficient, coupled with the advice of the moderator, to give College staff a full understanding of what is required of them by the University. Appropriate extracts are published in the College's Student Handbook. The College is bound by the terms of its formal agreement with the University to comply with the University's Standing Orders, and the moderator has stated that the University's procedures and regulations have been 'fully met'. Changes to the scheme of assessment and the formal regulations can only be made with the prior approval of the moderator and the external examiners, reported to the Validation Board in the moderator's annual report. The University's most recent regulations further require that changes in excess of 30 per cent of the regime must receive the formal prior approval of the Board. The team's concerns about the approval mechanisms for making changes to programmes of study have already been aired in paragraph 19 above.
39 The regimes of assessment for the two awards have no significant differences beyond those dictated by the different workloads and options which may be taken by the students on the three- and two-year programmes respectively. The ratio of unseen examination to coursework is 3:1. Marks awarded by internal examiners for coursework are not disclosed to students until the relevant external examiner has approved the marking. Following a recommendation of the external examiners, the College has recently introduced a regime of double marking by internal examiners, the better to ensure consistency and fairness in marking. Examinations are organised and overseen by College administrative staff. The College provides invigilators from outside its own staff. The moderator has been on hand to advise the College on all matters relating to assessment and the conduct of examinations and has indicated that these matters are all well carried out. By chairing the examination boards he has been in a position to ensure that the consideration of student performance is fully in accord with University regulations and practice. The quinquennial review panel found that the examination process had been operated in an 'exemplary manner', for which the University and the College are to be commended.
40 While also commending the practice that, certainly in the early years
of a validated scheme, the moderator chairs the examination boards, the
audit team found the documentation regarding the chairing of the boards
to be both confused and confusing. In the original submissions the examination
board was to be chaired by the course director; the consultant course director
and another member of the Cardiff Law School were also to be members. The
University's Standing Orders (which under the formal agreement bind the
College) provide that the examination board for any validated degree shall
comprise the Principal of the institution, ex officio; the external
and internal examiners; the moderator; and such other persons as the institution
may wish to appoint in an advisory capacity. The Handbook now provides
that the board will usually be chaired by the institution's head of school
or course director and that the moderator attends in an advisory capacity.
The quinquennial review submission stated that in fact the composition
had been as originally submitted 'except [the moderator] attends in his
capacity as representative of the Cardiff Law School, the linked department
in the University of Wales. Under the University of Wales regulations he
also acts as Chairperson'. The team believed that there should be greater
clarity on this matter. The University may wish to consider the advisability
of amending its documentation accordingly.
41 The College's only significant departure from University practice is
in the creation of an Academic Progress Committee, which it has established
to address its staff's long-standing concern over poor attendance at lectures
and seminars (see above, paragraph 37). The records of this committee inform
the meetings of the examination boards when they are exercising discretion
in considering borderline cases and claims regarding mitigating circumstances.
42 Appeals are dealt with under the University's Verification and Appeals
Procedure. This requires that the procedure be brought to the attention
of the College's candidates and the audit team found that this obligation
was met by setting out the procedures in the Student Handbook. However,
the Student Handbook for 1996-97 did not accurately represent the
University's regulations and it failed to mention the verification procedure.
However, the team was assured by students that the procedures were known
to them, and documentary evidence of an appeal case was provided to the
team which showed that the University's regulations had in fact been applied.
It was noted that the College had been approved to operate its own appeal
procedure for first- and second-year students which was very much based
on that of the University, since the team was informed 'this (ie the University's
procedure) applies to final-year students only'. The front cover preamble
to the 1995 version of the Procedure states, however, 'With the exception
of LLB candidates, undergraduate students may only invoke this procedure
once they have received the results of final-year University examinations',
which suggests that LLB students may be unable to invoke the University's
Procedure earlier, although the team was unclear as to why such an exception
should be made. The University may wish to consider the advisability of
ensuring these appeals matters are clear and are understood by students
at its collaborative institutions.
Establishment and maintenance of academic standards
43 The Handbook does not explicitly set out in detail the ways in which, through its procedures for collaborative partnerships, the University ensures the establishment and maintenance of academic standards in validated or franchised programmes. The audit team considered that it would be useful if it were to do so in a more explicit manner. Nevertheless, the team had no difficulty in establishing the key elements of the University's approach, which are to rely on the moderator and the external examiners, supported by the Validation Unit, to give it the assurance it requires.
44 The moderator is charged with the responsibility of reporting to the Validation Board 'as to whether academic standards at the validated institution (for the validated scheme of study) equate to those pertaining within the University of Wales itself'. In relation to examination and assessment the moderator is required to ensure that, prior to submission to the external examiners for approval, the examination papers prepared by the College's internal examiners are at the correct level. They are further required to attend, in an 'observer/advisory capacity', meetings of the Examination Board to ensure that the University's regulations and procedures are adhered to; that marking and classification occurs 'in line with the standard University of Wales practice and published criteria'; and that the assessment scheme applied is that which was agreed at validation. The annual report made to the University about these matters is essentially the same as those submitted by the external examiners (see above, paragraph 26).
45 The University's Code of Practice sets out the University's requirements of its external examiners. The audit team found that, in order to give assurance to the legal professional bodies, the College goes beyond the University's strict requirements by having external examiners for its year one examinations as well as those examinations which contribute to the University's final award. The Validation Unit handles the procedures for the appointment of external examiners. The team noted that the validated programmes are serviced by a large and experienced team of four external examiners, all of whom are of professorial rank. The team received evidence that external examiners were properly appointed and adequately briefed for their role; the administrative service they are given is good and timely; they approve draft examination papers (though not coursework or dissertation titles); they receive appropriate samples of student scripts and also of coursework; they generally attend examination boards at least once a year, and boards to consider resit marks in September also have at least one external present; the examination boards are well conducted; their views on the marks of individual scripts are discussed but their decisions are conclusive; they are required to sign off the pass lists; and they are very pleased with the way their general advice and recommendations are unfailingly responded to and usually agreed. The team noted that the only real concern of the external examiners was that an appropriate balance should be secured between vocational preparation and the academic study of underlying theory, since the emphasis in the early years has been on the former. This view is reinforced by the recommendation of the quinquennial reviewers that the focus of staff development should begin to move from updating scholarship towards original research and publication. The external examiners' reports have at all times explicitly assured the University that its academic standards are being maintained and that these are comparable with those being applied in other UK universities of which this experienced team of examiners has knowledge.
46 The audit team was satisfied that, in general, the University's Code
of Practice was being followed in its collaboration with the College.
The team concluded that the University's requirements as to schemes of
assessment and the duties of external examiners were appropriate and were
being fully implemented in its collaboration with the College and that,
therefore, stakeholders could have full confidence that academic standards
have been established and are being maintained. Despite some general concerns
about the University's approach in the longer term to the role of the moderator
(see above, paragraph 29), the team was satisfied that the University's
current support to Portobello College was sufficient, full and effective
to secure the standards of the University's awards.
47 The HEQC Code of Practice expects UK institutions to satisfy themselves that staff teaching on approved programmes are qualified and competent and that they remain so. The University of Wales, within its Handbook, describes ways in which it reassures itself about the staff who are teaching programmes leading to the University's awards. These include consideration of staffing matters at validations and quinquennial reviews, such as full-time/part-time balance; staff development policies and practices; non-teaching staff support; monitoring of staff experience and qualifications through scrutiny of curricula vitae.
48 At the time of the audit visit six permanent and 10 part-time and short-contract
staff were teaching on the LLB programmes out of a total complement of
staffing at Portobello College of 52 academic and 24 non-academic staff.
The majority of those permanent staff teaching on the course at the time
of the visit were part of the original teaching team approved within the
initial validations of the two- and three-year programmes. The agreement
between the University and the College requires that details of staff teaching
on the LLB, such as their qualifications and experience, be provided and
states that, without written permission of the University, the staff:student
ratio as approved at the commencement of the scheme may not be permitted
to fall.
49 Selection and appointment of staff is the formal responsibility of the
Head of Department in the College assisted by the Assistant Director Law
and Finance and the moderator. Part-time staff are appointed by the Head
of Department and the Assistant Director Law and Finance. In its submission
for the quinquennial review the College explained its practice in advertising
for posts and declared its selection criteria. These criteria included
a requirement for new lecturers, including part-time appointments, to have
at least one year's experience for teaching posts. The audit team heard
that all but one of the new lecturing staff appointed to full-time posts
had also previously taught part-time at the College. Although the moderator,
as the Chair of the LLB Staff Appointments Committee, has been involved
in new appointments (see above, paragraphs 24 to 30), information on new
staff is also formally lodged with the University through the annual monitoring
report, which contains a curriculum vitae for any new member of
staff appointed during the year.
Induction of staff and role in programme delivery
50 The Head of Law is responsible for the induction of new academic staff
and the supervision of all staff teaching on the programme. The College
does not operate an appraisal scheme but it does have a form of mentoring
whereby those responsible for particular modules work with and support
other, sometimes less experienced, staff within the module team. In this
way calibration of marking and co-ordination between lectures and tutorials
are achieved. The audit team was told that the tradition of peer appraisal
and external quality assurance at a national level was not a feature of
Irish third level education. The Head of Department had attended some new
teachers' lectures, with their consent, but the team was told that rather
than using more formal appraisal, the practice would be to trust staff
as competent professionals, identifying via student feedback where
problems were developing. The moderator told the team that he does not
observe teaching but is able to monitor it through sight of student questionnaires
and meetings with students. He took a greater role in supporting staff
in the earlier stages of the implementation of the programme (see above,
paragraphs 24 to 30), but with the increased experience of the staff this
support had reduced. In the context of concerns expressed about student
attendance, some strategies involving observation of teaching by fellow
teachers, discussion of pedagogic matters and mutual exchange of good practice
within the College might be worthy of some further consideration. The team
would wish to commend the close attention to, and involvement of, the moderator
in the selection and induction of the teaching staff on the programme.
The moderator's active support and advice had clearly been beneficial to
the University's further confidence in the standard of work delivered by
staff at the College.
51 The College states that it has the following policy objective with respect to staff development: 'to encourage staff to engage in research, publication, continuing professional development and curriculum development, thereby enhancing the quality content delivery and course material and ultimately improving the quality of the student learning experience'. The audit team was told that the moderator had conducted workshops with new lecturing staff and has given advice to staff on teaching and learning matters through the departmental meetings. However, it was understood that the College did not have any formal mechanisms for the development of staff as teachers. The Head of Department would play a key role in conveying constructively to the staff member concerned any shortcomings revealed by student feedback and assisting him or her to find a solution. On the advice of the moderator attention had been given to staff research, mainly in the context of pedagogical issues, for the first five years of the programme and there had been encouragement for staff to follow masters programmes and undertake continuing professional development. In the future, the team was told, it was intended that staff would begin to engage in higher level research. The University may wish to consider the advisability of increasing its support for the staff development at Portobello College, particularly with reference to more advanced pedagogical skills and techniques, communications and infrastructure.
52 In contrast with most of its other international collaborative partnerships, the University of Wales link with Portobello College is characterised by close similarities in culture, history and educational practice, a common language, together with physical proximity. The main conduit of communication between the University of Wales and Portobello College is the moderator. The ease of contact (facilitated by the moderator's circumstances (see above, paragraphs 24 to 30) was viewed by both the College and the University as an important feature in the programme's success, and of especial benefit in the early stages of programme development. The moderator is relied on to explain and interpret material which is issued by the University, primarily communicating through the Head of Law. The Head of Law conveys queries and the University's responses to colleagues and students, with academic staff participating little in the dialogue with Wales. Teaching staff with assessment responsibilities at the College have contact with external examiners but did not see the formal reports. Neither did staff contribute formally to the preparation of the Annual Course Report to the University, which some academics had not seen. This appeared to be viewed by staff as mere administrative communication within the purview of the Head of Law.
53 The audit team heard of one area of communications difficulty of which the University was aware but which it was not in a position easily to resolve. It was understood that the full-time teaching staff are not able to participate in the Association of Irish Law Teachers, since they are working outside the public sector of Irish legal education. The team considered that the staff development value of the opportunity to discuss pedagogical issues with peers is important in the context of higher education. It recognised that although some benefits would be gained for Portobello staff in sharing practice with staff teaching in the UK, it would be more expensive in time and resources and would not permit the exchange of views on the curriculum afforded by contact with an Irish peer group. The University may wish to consider whether it could provide support to the College in this area.
54 The main source of regulatory and other written material is the University's Validation Unit with key contact during the processes of annual monitoring, quinquennial review and the appointment of external examiners. More recently, lines of communication have also been established with other branches of the University Registry, from which advice on appeals matters has been received. The contact with the Validation Unit is fundamentally an administrative one, whereas with the moderator the interaction is both regulatory and subject based, relating to the curriculum and associated teaching and learning issues. It was also evident that staff at Portobello were able to contact counterparts in the University for advice; for example, when the College was setting up its careers service. The audit team noted that the communications system between the College and the University appeared responsive and effective, although it was heavily reliant on the good relationship between the Head of Law and the moderator. The team would wish to commend the University on the attention paid by the moderator to his communications role, which demonstrated good practice.
Publicity and promotional materials
55 The memorandum of understanding between the University and the College
has a section on promotional material, which states that 'the University
shall have absolute discretion as to the contents of any statements, advertisements
or other promotional material'. The College is required to obtain written
approval from the University before publishing promotional material and
indemnifies the University should problems arise as a result of unapproved
material. The Handbook shows this to be a standard term included
in the University's partnership agreements. The audit team noted that a
member of staff at the College carried responsibility for publicity and
he was in contact with the Validation Unit to obtain approval for material.
Examples were given describing contact over the production of a newsletter
and a video and it was understood there was contact over the production
of the prospectus. The students appeared satisfied that the image of the
College portrayed in the publicity material was accurate (see above, paragraph
33). The team heard that the moderator had participated in the promotional
activities of the College, addressing prospective students and their parents
at one of the College's open days. The effectiveness of this aspect of
the partnership did not appear to be systematically monitored, either as
part of the quinquennial review process or as part of the moderator's brief
in his annual report.
56 The audit team heard that the degree certificate parchments are prepared in Wales, once the documentation authorised by the external examiners has been submitted by Portobello College. Students normally graduate in absentia but there is a celebratory ceremony at the College attended by University of Wales representatives and dignitaries from the field of Irish Law. The team was provided with an example certificate, which in Welsh and English clearly states the name of the College and the location where the programme of study is delivered, together with the language of instruction. This arrangement for certification follows the HEQC Code of Practice for Overseas Collaborative Provision in Higher Education, 1996 (2nd edition). The certification does not, however, distinguish between those students who had followed the two-year and those who had followed the three-year programme. The team was informed that this was the University's practice, identical certification being provided for a similar arrangement of a two- and three-year LLB programme at the University of Wales, Cardiff. The University of Wales Academic Board determines the format and wording of the degree certification. Although the practice for the LLB award appeared to be consistent within the University (and might be common in other degree awarding bodies), the University, for the benefit of transparently showing the nature of a student's learning experience and in the context of the developing national qualifications framework, may wish to consider making a distinction in the certification between awards with separately validated programmes of study.
Conclusions
57 The University's relationship with Portobello College has developed
systematically and constructively since it began in 1991-92. During this
period the University has extended its own experience in the validation
of collaborative partnerships and refined its quality assurance mechanisms
accordingly. It has, thereby, increased its ability to have confidence
that the standards of awards made in its name through such validated and
collaborative partnerships can be secured. Through the work of the University
Board of Validation and its Validation Unit, the range of quality assurance
procedures which are now operated provides for a coherent and systematic
mechanism, underpinned by both formal and informal processes. In the case
of Portobello College the University's processes have worked effectively
and creatively, not only to provide quality assurance and fully to secure
standards on the LLB programme concerned, but also to promote the development
of the programmes and the development of the institution.
58 The role of the University moderator in the University of Wales quality assurance system is of great significance and particular comment was made on this matter in the previous HEQC Overseas Partnership Audit Reports on the validation work at two centres in Spain. The moderator practice in the case of Portobello College appeared to be exemplary. The particular relationship between the moderator and the Portobello College has been unusual in the frequency and the scale of the contact. It was valuable for the College that it had established a consultancy relationship with the academic concerned prior to his acting as moderator and that, holding both roles concurrently for a number of years, he had been able to be highly effective in supporting the development of the programme. When a replacement moderator is needed, the University is likely to find it hard to identify someone with the special mix of expertise which has benefited this link. It was clear to the audit team that many of the features of the partnership in the Republic of Ireland which made it successful would be difficult to replicate in other partnerships. Whilst recognising the strength of the model in this particular partnership, there are still concerns surrounding the over-reliance on one individual and the potential for conflicts of interest. For as long as it continues to draw them as volunteers from its constituent institutions, the University may also need to find a way of managing its significant reliance on moderators, given the increasing and competing demands made upon academic staff time. Since the University gives them a role in the maintenance of standards, that reliance also confirms the need for moderators to share good practice.
59 Although its focus was the relationship with Portobello College, the audit team was aware of the wide range and large number of collaborative partnerships being managed through the University's Validation Board and Validation Unit: over 47 centres with 7,500 students. Crucial to the University's system which ensures the quality of this work is the role of the Validation Unit in closely monitoring the partnerships and if necessary taking an active role to alert the University to matters of difficulty. Although the moderator role is fundamental to the system, the adverse pressure upon staff time is such that the Validation Unit becomes the principal locus where informal monitoring can alert the University to potential problems. Yet the Unit comprises a small team carrying a large administrative load. It was not clear to the audit team how the University would know when the scale of its validation work had reached a point when the present structure of support might cease to be sufficient or appropriate.
60 Consideration of the quality of teaching provided by Portobello College and the standards of achievement of its students were outside the formal remit of the audit team. By virtue, however, of the material provided by Portobello College, and the opportunity provided by the visit to discuss the student experience with both staff and students of the College, the team was able to form an impression, as a background to its work, of the calibre of the College as an academic environment. Good foundations have been laid in this relationship between the University and the College which give confidence that this collaboration will be maintained in the future, and that the standards of the University's awards are being fully and effectively secured and could be maintained through the quality assurance systems presently in place.
Annex
Commentary on the audit report supplied by the University of Wales
The Quality Assurance Agency report of the University of Wales' validated LLB degree at Portobello College provided a number of useful recommendations which, it is hoped, will further strengthen the University's quality assurance and monitoring procedures in this important area of activity.
With respect to specific issues highlighted within the report, the following response is offered:
Paragraph 1
The Validation Board has now established a Planning Advisory Group to help ensure
that appropriate management and resourcing of the University's validation activities
takes place.
Paragraph 14
The University's standard Agreement document has now been augmented to include
references to the residual responsibilities of both the University and the validated
Institution to students in the event of termination.
Paragraph 17
In the event of future validation exercises being conducted by correspondence,
the Validation Unit will ensure that comments submitted by individual assessors
are circulated to all other members of the panel of assessors.
Paragraph 19
Formal procedures for the approval of amendments to existing schemes of study
have been introduced by the Validation Board. The implementation of these procedures
will be monitored carefully over the next few months.
Paragraph 23
Specified sections of the Executive Committee agenda are now given particular
scrutiny by designated members of the Committee (acting in pairs) thus ensuring
that all reports submitted are carefully read and acted upon.
As noted in this section, all reports are also carefully scrutinised by Validation Unit officers.
Paragraph 26
A revised Moderator's Report Form has now been drafted for approval by the Validation
board.
Paragraph 32
The Validation Unit hopes to have the necessary resources in the near future
to enable it to issue registered students with a letter of registration, identity
cards (in cases where these are requested) and provide University of Wales merchandise
etc to such students.
Paragraph 42
A new version of the University's Appeals procedure has been issued (October
1998). The preamble to this version states that ‘candidates may only invoke
the procedure once they have received the results of final year examinations’.
Paragraph 55
Heads of validated institutions are required to formally sign an annual declaration
re: advertising and publicity. Further consideration will be given to this recommendation
by the Validation Unit.
Paragraph 56
This section has been referred to the University Registry for further consideration.
Paragraph 58
There has been progress in this report - centred on the annual meeting of Moderators.
Induction of new Moderators also takes place and there have been Moderators'
workshops centred on specific issues.
Top