Introduction
1 This is a report of an audit carried out by the Higher Education Quality Council
(HEQC) of the quality assurance arrangements for a partnership between Sheffield
Hallam University and N Avgerinopoulou Centre for Liberal Studies (NAC), Athens
to deliver programmes of studies leading to the University's Dip HE Psychology
and Education, and BA/BA (Hons) Psychology and Education. It forms part of a
series of audits of overseas collaborative partnerships undertaken in the UK
and Greece in May, June and July 1997.
2 This audit of the partnership arrangements between Sheffield Hallam University and NAC examined the policies and procedures used by the University to satisfy itself of the academic quality and standards of its awards being offered in Greece.
3 The Council is grateful to the University and to NAC for their assistance
and co-operation.
Abbreviations used in this report
4 In this report the following abbreviations are used:
AAB: Awards and Assessments Board;
AQR: Annual Quality Review;
AQSC: Academic Quality and Standards Committee;
AQSC Report: 'Development of SHU's procedures for approving and monitoring
collaborative provision' (sic) December 1996;
BPS: British Psychological Society;
HEQC Code: Code of Practice for Overseas Collaborative Provision
in Higher Education, HEQC, first edition 1995; second edition 1996;
HCS ('the School'): School of Health and Community Studies;
IEK: Institutes of Higher Professional Studies (Greece);
LMU:
Leeds Metropolitan University
NAC ('the Centre'): N Avgerinopoulou Centre for Liberal Studies;
the Office:
the Outreach Office
SME: Senior Management Executive [of the University];
TEL: Technical Education Lyceums (Greece);
1993 Report: Sheffield Hallam University. Quality Audit Report. HEQC,
1993
1994 Report: Sheffield Hallam University. Quality Audit Report. Collaborative Provision. HEQC, 1994.
The audit process
5 Following initial discussions, the University provided HEQC with documentation
on its relationship with NAC and details of its quality assurance processes.
At a briefing meeting to discuss this material, the audit team proposed a series
of visits to the partner institutions in the UK and in Greece, and sought additional
contextual materials to extend its understanding of the structure and processes
of the University's quality assurance arrangements for this aspect of its overseas
collaborative provision.
6 The audit team held discussions with members of the University before and after its visit to NAC and met the Chair of the Academic Quality and Standards Committee (AQSC), the Deputy Registrar, the Principal Officer (Outreach), the University Secretary, the Principal Officer (Examinations and Awards) and members of the School of Health and Community Studies (HCS). During its visit to NAC, the team met the President of NAC's parent Group, managers, teachers, administrators and students, and the Athens-based external examiner for the University's programme of studies.
7 The audit team comprised Dr R M Allen, Professor A Gale and Mr A Jones, auditors and Ms D Cerqua, audit secretary. Dr D W Cairns, Assistant Director, Quality Assurance Group, accompanied the team and co-ordinated the audit for HEQC.
The University context for collaborative provision
8 In its 1996 Strategic Plan the University states that '[its] primary
aim ... is to enable its students to develop intellectual, professional
and practical skills to maximum potential through vocationally relevant
programmes of study, developed in association with industry, commerce,
the arts and the professions'. In order 'to operate nationally and internationally'
it has worked to establish 'a range of collaborative links between [itself]
and partners overseas for both teaching and research purposes'.
9 The University's quality assurance arrangements for its academic provision and its collaborative provision were the subject of Reports by HEQC in 1993 and 1994. The publication of the 1994 Report pre-dated the establishment of the University's partnership with NAC.
10 The University describes its collaborative provision as 'outreach activity',
and divides it into three types. It uses the term 'franchise' where the
whole or part of a University programme of study is delivered in and by
another organisation, whilst 'validation' refers to the approval process
for a programme of study proposed by an institution or organisation, leading
to an award of the University. The term 'accreditation' is used by the
University to describe its scrutiny of a programme provided by another
organisation for the purpose of awarding it an educational credit value.
Accreditation is a significant part of the University's outreach activity
and at the time of the current audit some 7,000 students were following
programmes of study and courses which had been accredited in the sense
used by the University.
'Rich links'
11 The University's recognition of the scale of the resources required to support
successful collaborative provision in the UK and overseas has led it to set out
the concept of a 'rich link'. The University sees a 'rich link' as a partnership
which is capable of development over time from small beginnings to a range of
academic provision and contacts including postgraduate provision, capitalising
on the initial energy devoted to establishing the partnership.
Changes in quality assurance arrangements since
the 1994 Report
12 Wherever feasible, the University's arrangements for the quality assurance
of its outreach provision have been closely similar to those for its other programmes
of study. Its 12 schools bear much of the responsibility for annual and periodic
review of academic provision and for some aspects of programme approval. The
schools exercise their quality assurance responsibilities within a set of procedures
established by the AQSC, which is a sub-committee of the Academic Board and stands
in the same relationship to it as the former Academic Quality Committee (see
1994 Report, paragraph 22). The AQSC has central responsibility for approving
programmes to be delivered outwith the UK.
Since publication of the 1994 Report, the University has established a
number of 'University Standing Panels' (USPs) to scrutinise proposals for
new courses and programmes of study. Two of the USPs deal solely with collaborative
provision: one is responsible for provision within the UK, and one for
overseas provision. These arrangements came into effect at the beginning
of the 1996-97 session. All new programmes, major modifications, and franchises
continue to require the approval of the University's Academic Board, acting
on the recommendation of the AQSC.
The Outreach Office
13 Until recently, responsibility for the administration of outreach provision
rested with the University's Outreach Office under the leadership of the Deputy
Academic Registrar within the University's Registry. The Office was also responsible
for providing administrative support for validation and approval processes and,
from mid-1996, was responsible for co-ordinating institutional checks on potential
partners. In August 1997, the Office was incorporated, along with other central
quality assurance offices, into a single integrated Quality Office, under the
leadership of the Deputy Academic Registrar. The former Office's responsibility
of drafting an annual 'Outreach Overview Report' on behalf of the Academic Registrar,
as part of the University's Annual Quality Review process, continues under these
new arrangements. In late 1996, the former Outreach Office was one of the means
through which the AQSC enquired into aspects of the quality assurance of its
collaborative provision overseas, the outcomes of which were presented in a searching
report which formed part of the University's briefing materials (see below, paragraph
48ff).
Initiating a new collaborative proposal
14 Proposals for all new collaborative programmes must be notified to the University
at the earliest opportunity through submission of a request for planning approval
on a standard form. The University refers to its agreement that a proposal may
be developed with a potential partner as 'permission to negotiate', which its
Senior Management Executive (SME) aims to grant early in the development process.
In considering whether to give permission to negotiate, the University assesses
the compatibility of the proposal with the University's mission and the strategic
plans of the relevant school. New procedures introduced in 1996-97, taking note
of the recommendations of the first edition of the HEQC Code, clarified the strategic
nature of the decision required from the SME in order to allow negotiation to
proceed and set out new Registry and International Office procedures to establish
the suitability of a partner. Once planning approval is granted, schools are
required to appoint a specially trained member of staff to act as a 'negotiator'
to develop the proposal together with the partner institution.
The link tutor: roles and responsibilities
15 When full approval is obtained, the relevant school is required to appoint
a link tutor. In many cases the former negotiator will become the first link
tutor for a new outreach programme. Link tutors are required to undertake a range
of duties, including the maintenance of links with the partner institution; liaison
with the appointed course leader at the partner institution; visiting the partner
institution at least once a year; and producing an annual report to accompany
the annual report from the partner.
16 The University's use of members of its academic staff to act as the
day-to-day points of contact between itself and its partners is discussed
in paragraph 32ff of the 1994 Report, and in paragraphs 47-51 of this report.
The link tutor's annual report informs the Annual Quality Review report
(AQR) of the relevant school, which is prepared by its director, and the
accompanying action plan for the following year (see below, paragraph 52).
In 1996, the AQSC introduced a requirement that AQR reports for all outreach
provision, and reports from the link tutors, be copied to the Outreach
Office.
Formal agreement
17 The framework of relations between the University and NAC was initially set
out in a 'Validation Agreement' ('Agreement 1') relating to the Dip HE, completed
in January 1996 and valid for three years. 'Agreement 1' sets out the status
of the partners, noting that NAC is not to hold itself out to third parties as
an agent of the University, and covers such matters as 'Copyright and Confidentiality';
'Equality of Opportunity'; 'Assessment'; and 'Assignment and Sub-contracting',
the latter being forbidden without the express approval of the University. 'Agreement
1' also provides that in 'the event of termination [of the Agreement], arrangements
satisfactory to the University will be made to ensure the rights of any students
are safeguarded'. 'Agreement 1' provides for a review of the operation of the
partnership within the first three years of its term (see below, paragraph 45).
The University was actively preparing for this review at the time of the current
audit.
18 An Annex to 'Agreement 1' provides for 'Quality Assurance and Programme Administration'.
It confirms that the 'Academic Board of the University will have responsibility
for the oversight and maintenance of academic quality for the programme' and
covers such matters as academic quality; management, monitoring and review; resources;
admissions; assessment; and examinations and appeals.
19 Under the terms of the Annex the University has conferred responsibility for the day-to-day operation of the programme on its partner, including the advertisement and promotion of the programme, the admission of students and administrative responsibility for their assessment. The draft Validation Agreement for the BA/BA (Hons) ('Agreement 2') and its Annex dealing with quality assurance and programme administration are couched in identical terms. 'Agreement 2' was signed on 31 January 1997, to terminate on 30 September 1997.
20 The audit team noted that at several key points in each Annex, including the section dealing with assessment, it was difficult to determine whether the University or NAC was to be responsible for initiating and concluding specific actions (see below, paragraph 63). The University will wish to consider the advisability of specifying in the annexes to its validation agreements which party is to be responsible for initiating particular processes and actions.
N Avgerinopoulou Educational Organisation of
Liberal Studies (NAC)
21 NAC is a private organisation categorised as a Liberal Studies
Laboratory by the Government of Greece. The Educational Group of the Organisation
('the Centre') was established in 1975 and at the time of the audit was
reported to have over 2,000 students in three schools on five sites in
Athens. NAC provides Government-approved courses categorised in Greece
as being at the level of the Technical Education Lyceums (TEL), and other
courses described as at the level appropriate to the Institutes of Higher
Professional Studies (IEK) and Advanced Business Services (ABS). The University-validated
programme of studies which leads to the Dip HE and BA/BA (Hons) Psychology
and Education is delivered in the Centre's largest premises, in the middle
of Athens. A company profile of the Centre included in the University's
briefing materials identified collaborative links with two other named
UK universities and links with a University in the USA. The audit team
was told that NAC was designated as an approved BTEC/EdExcel centre.
Background to the partnership
22 Prior to establishing relations with Sheffield Hallam University,
NAC had entered into a partnership in the late 1980s with Leeds Polytechnic,
through which it had offered validated programmes of study leading to diplomas
of the Council for National Academic Awards (CNAA). Two of these programmes
of study, a Diploma in Psychology and a Diploma in Education, had been
approved by the Polytechnic in 1989, and later became awards of Leeds Metropolitan
University (LMU). One of the reasons cited by NAC for pursuing this initiative
was that it wished to provide students in Greece with access to a professional
qualification in psychology. There had been some suggestion that, following
successful completion of their studies in Greece, former NAC students would
be able to proceed to LMU to complete a 'top-up' course in psychology leading
to a qualification recognised by the British Psychological Society (BPS).
23 In 1993 the BPS declined to grant professional recognition to LMU's psychology provision. An incidental consequence of this decision was that NAC sought to bring its partnership with LMU to an end. Members of LMU made informal contact in late 1993 with members of Sheffield Hallam University's School of Health and Community Studies ('the School') to explore the possibility that NAC might enter into a partnership with Sheffield Hallam University, since the BPS had recently recognised the latter's BSc (Hons) Psychology programme. In January 1994 LMU made a formal approach along these lines to Sheffield Hallam University's International Office, which was passed to the School for discussion and development. At all times, the University made it clear to NAC that it considered it unlikely that the BPS would offer recognition to a diploma offered through a Greek institution, taught partly in Greek, which did not contain a full range of psychology modules. Sheffield Hallam University did, however, advise NAC that it was planning to develop a postgraduate conversion course which would satisfy the BPS accreditation requirements, which students who had completed a suitable programme of studies in Greece might follow in the UK.
24 At this stage the University's familiarisation with its potential partner was assisted by informal approaches by members of HCS to the LMU member of staff who was acting as the liaison contact between LMU and NAC, and by the fact that one of the external examiners for Sheffield Hallam University's BSc (Hons) Psychology was a member of staff of LMU, and in that capacity had visited NAC on several occasions. Further information was provided by a member of Sheffield Hallam University who had formerly acted as an external examiner for one of LMU's courses in Athens.
25 The Academic Director of NAC visited the University in early May 1994
to meet its Course Leader for Psychology and the School's Director of Academic
Development, and to discuss the practical possibilities of co-operation.
Later the same month, the Course Leader visited Athens to carry out institutional
checks on NAC on behalf of the University, at which time she met the Athens-based
external examiner for the LMU programmes of study. Additional discussions
also took place with LMU, which provided background information on its
programmes with NAC and their quality assurance. At the same time, the
views of the British Council in Athens were sought on the Centre's position
as a Greek higher education institution. On the basis of the information
gathered through each of these processes the University's SME granted the
School permission to negotiate and the Course Leader for Psychology, who
had previously been trained as a University negotiator, was formally appointed
as the Negotiator in this instance.
26 Prior to its discussions with NAC, virtually all the School's outreach partnerships
had been within the UK; not unreasonably, therefore, the School wished to proceed
with caution. The School did, however, have hopes that the partnership might
be a future 'rich link', and it appeared clear to the audit team that NAC wished
to undertake further developments in physiotherapy and sports science in partnership
with the University. Indeed, as the School's account of communications with its
partner makes clear, throughout their relations a substantial part of the communication
with NAC was concerned with possible future developments (see below, paragraphs
46 and 48).
The validation process
27 The University has clear guidelines on its quality assurance processes
and procedures and the guidance it offers its partners through documentation
was commended in the 1994 Report. At the time of its discussions with NAC,
the University's document on external validation procedures did not contain
information on approaches to be followed for collaboration with partners
overseas. However, a later version, revised in November 1996, includes
an annex on international proposals which the audit team judged to provide
commendably clear and helpful guidance for members of the University and
potential and existing partners developing such proposals.
28 The programme submitted for validation to lead to a Dip HE, was based on the two diplomas formerly validated by LMU. It was proposed to offer and assess about half of the first year courses of this new programme through the medium of Greek, and about a fifth of the second year courses similarly. It was envisaged that any third level (that is, honours level) top-up courses would be taught and assessed entirely through English. The design of the programme, and the decision to validate rather than franchise the University's provision, allowed the partners to build on the existing work being done by NAC. It also appears to have had the effect of limiting the number and frequency of contacts between the partners. The data on which the University could call in judging the health of the programme, and the readiness of students to progress to the Dip HE, was consequently limited to that from the 'access' and 'bridging' modules and information on developments within the Centre was limited to that provided by reports from visiting University staff and, with less frequency, from the reports of its Athens-based and UK-based external examiners (see below, paragraph 60).
29 In 1994 the greater part of the University's collaborative arrangements
was based in the UK and in its briefing materials the University itself
described the proposal for an overseas external validation as constituting
'uncharted territory'. In 1994 the Approvals Sub-committee of AQSC appointed
a special panel to consider the proposal, and in 1996, seeking to build
on this experience, substantially the same panel undertook the validation
of the BA/BA (Hons) Psychology and Education. The audit team noted that
in future, the Overseas USP would constitute the approval panel to consider
and recommend approval for such proposals.
30 In 1994, the University did not have formal separate procedures for the recognition
of partner institutions, the suitability of a partner being tested in the course
of a validation or franchise exercise. The formal process through which the University
familiarised itself with its potential partner and validated a newly constructed
Dip HE took in all about nine months. Whilst the availability of advice and information
from LMU no doubt helped the University in coming to terms with the cultural,
legal and educational context into which it was venturing, the relatively short
span of time which was devoted to developing a full understanding of these matters,
and familiarising itself with its partner (and vice-versa) may have been unduly
abbreviated.
31 The membership of the University's Validation Panel in 1994 included specialists on accreditation (that is, rating for educational credits), and quality assurance, in addition to subject specialists from within the School and a member from outwith the University. In addition, the University invited one of the LMU staff associated with the closing programmes at NAC to attend for its discussions. The proposal put to the Panel involved two separate but simultaneous processes. The first was the external validation of a programme of studies leading to a Dip HE award, based on the two diploma programmes validated by LMU. The second process proposed the accreditation of programmes of study already offered by NAC, leading to TEL and IEK certificates in Nursery Nursing and Special Education and approved by the Greek Ministry of Education.
32 The Validation Panel took the view that the arrangements put to it for its approval were 'unnecessarily complicated and a potential source of confusion'. It decided that it had insufficient information to carry out a general credit-rating of NAC's TEL and IEK courses, and agreed that 'a programme of study should be validated that offered TEL/IEK students the opportunity to choose, at various points in their study, to progress to the Dip HE or IEK' in addition to a 'two year full-time course leading to the Dip HE (both Years 1 and 2 to run from October 1994)'. This programme would include NAC's TEL and IEK courses in Nursery Nursing and Special Education; an access course for all students entering the Dip HE (covering, amongst other topics, basic principles of higher education in the UK), and bridging modules for students seeking to enter Year 2 of the Dip HE, which would include the access course. Success in Year 2 of the TEL course, or Year 1 of the IEK course, together with completion of the access and bridging modules, would offer access to Year 2 of the University's Dip HE.
33 The Panel congratulated NAC on the quality of its course team and recommended approval of the proposal subject to the fulfilment of a number of conditions. These included the submission of a development plan covering a range of matters, including resources and resource planning, student guidance and learning support, staffing and staff development and entry requirements. The Panel also recommended that the term of the initial approval of the proposal be limited to three years, and required the Centre not to promote the courses as leading to a BA/BA (Hons).
34 Although the 1994 Validation Panel approved two routes through which
students at NAC could progress to the University's Dip HE award, students
at NAC have not so far entered Year 1 of the Dip HE but have entered Year
2 through completing the Centre's TEL and IEK programmes, together with
the access and bridging modules. At the time of the audit, 25 students
were registered for the Dip HE and BA/BA (Hons) programmes of study, of
whom nine were studying for the latter. The remainder were students registered
for the Centre's IEK course and registered with the University to undertake
the additional modules of study specified by it (see above, paragraph 33).
Validation of the BA/BA (Hons) Psychology and Education
35 Before the 1994 validation, NAC had indicated to the University that it wished
to proceed with a degree programme in psychology in 1995. This would have been
achieved by developing and validating a suite of modules at honours level (level
3) to 'top up' the Dip HE. In 1994 the University indicated that it was willing
to consider such a proposal in principle, but that such a development would be
dependent on progress in other matters, including the successful implementation
of the academic development plan for NAC, required as a condition of the 1994
approval.
36 Although NAC met the University's conditions of approval by the due
date, the start of the Dip HE was delayed because of the flooding of the
buildings in Athens designated as the base for the programme. Consequently
the Dip HE ran for the first time in February 1995, rather than October
1994 as initially envisaged. A further consequence of the delay caused
by the flood was that the first annual review of the Dip HE was not required
until the summer of 1996 and hence covered the first 18 months of its operation
(see below, paragraph 52). The audit team heard that this had delayed by
a year the submission for validation of the proposed 'level 3 modules',
leading to the BA/BA (Hons), and had also delayed completion of the formal
Validation Agreement for the Dip HE until January 1996.
Preparations for the 1996 validation
37 When accepting that the conditions of approval had been met, the University
identified 'a number of additional points of advice and good practice' for the
attention of both NAC and the School. One point related to the development of
learning resources and teaching and learning styles, and another on the need
to monitor NAC's links with other higher education institutions in the UK and
elsewhere.
38 In January 1996, members of the University's teaching team for psychology
visited NAC to discuss the addition of level 3 modules to complete the
programme for a BA/BA (Hons) degree in Psychology and Education. In February,
the Link Tutor submitted a positive report on the basis of a recent visit,
which the School regarded as providing assurance that the validation of
the BA/BA (Hons) could proceed. The Link Tutor subsequently being unavailable
through illness, a member of the same Subject Group was appointed to act
as the Link Tutor and Negotiator in her absence, and the proposal documents
were submitted to the University in April 1996. To assist in maintaining
continuity, the University selected a Validation Panel with substantially
the same internal and external membership as that which had considered
the 1994 proposal. It held a preliminary meeting to discuss the proposal
in May 1996, which requested further documentation from NAC, and identified
a list of matters to be addressed at the validation event, which was held
in Athens in the following month.
Report of the 1996 Validation Panel
39 The report of the June 1996 validation event states that 'a cautious partial
approval of the proposal was the only positive possibility' open to the Panel.
Amongst the reasons given for this recommendation were the stance of the BPS
on awards gained from study overseas; what the Panel considered to be the likely
outcomes of the 1996 HEQC pilot overseas visits to Greek institutions and the
short time for which courses leading to the Dip HE had been running.
40 The Panel considered that the School needed to provide more support and closer monitoring of developments in NAC, and to ensure greater support for assessment and verification of standards. It recommended approval of the proposal for one year only, terminating in 1997 at the same time as the three year approval of the Dip HE expired, at which point a review of the entire programme would allow the University to decide whether or not to extend approval. A list of conditions was attached to the approval including the requirement that NAC would not be able to advertise the programme as leading to a degree award until the further review had been carried out. This review was to be carried out in the summer of 1997, shortly after the conclusion of the audit team's visit to Athens.
41 It was clear to the audit team that in 1994, and again in 1996, the
University had wished to be cautious in venturing into this unknown territory.
The team commends the University for the application of comprehensive approval
processes in approving this overseas collaborative arrangement and the
care taken by its validation panels to safeguard the University's interests.
University review of the procedures for approving and monitoring
collaborative provision
42 In September 1996, the AQSC considered the report of a review of outreach
validation, which in turn stimulated a more detailed review of the validation
and approval of international proposals. This second review was conducted by
the Outreach Office for the AQSC, which considered the report, entitled 'Development
of SHU's procedures for approving and monitoring collaborative provision' (henceforth
referred to as: 'AQSC Report'), in December 1996.
43 The AQSC Report is a frank analysis of the recent experience of the University in approving outreach provision overseas. Commenting on how proposals for collaboration had emerged in the past it noted the urgency with which some proposals had been put forward and, notwithstanding its acknowledgement of the 'creative tension between entrepreneurial development and rigorous checking', advised the University through the AQSC to incline towards the latter. The AQSC Report recognised the difficulties that its proposals would cause for schools by imposing delays at certain points in the academic development process but it saw these as the price that had to be paid to protect the University's good name and the quality of its provision. The conclusions and recommendations of the AQSC Report as they apply to other aspects of the University's quality assurance processes for its overseas collaborative provision are considered elsewhere in this report (see below, paragraphs 48 and 54).
Monitoring and review
44 The University monitors its outreach provision through its link
tutors, through the process of Annual Quality Review (AQR) which is mainly
a school-based process, and through periodic review, operated on a six
year cycle. Since the partnership in question had been in operation for
less than three years at the time of the audit visit, the last of these
procedures had yet to be brought into play, although the review called
for under the terms of the 1994 Agreement between the University and NAC
was about to take place shortly after the audit visit (see above, paragraph
18).
45 The School administers its outreach partnerships through its Flexible
Learning Development Centre (FLDC), headed by an administrator responsible
to the School's Head of Academic Development. All communications between
the NAC and the University go through the FLDC and a chronological tabular
analysis of the dates and nature of such communications was provided for
the audit team, in which communications were categorised under one or more
of four headings: development of the partnership and new proposals; validation
processes; monitoring and moderation, and other communications. The table
offered the team a helpful picture of the range, frequency, and nature
of communications between the University and its partner.
Link Tutors
46 The University regards its link tutors as the key means through which it can
monitor the health of individual outreach arrangements on a frequent, if not
day-to-day, basis. Link tutors are expected to be in regular contact with colleagues
in the partner institution and, on a more formal level, to visit the partner
institution at least once a year, and usually more frequently.
47 The role and duties of the University's link tutors were described in the 1994 Report (paragraphs 32-35), and for UK-based partnerships they have not changed significantly since then. In this particular instance, the University 'enhanced' the link tutor role, although it stated during the audit that responsibilities for reporting on quality assurance and quality control matters had not been formally been assigned to the link tutor. This view does not appear to coincide with that of the University's 1996 Validation Panel for the BA/BA (Hons) Psychology and Education, which had called for 'clarity about the role of the Link Tutor, paying particular attention to the effectiveness of combining the responsibility for simultaneously facilitating and monitoring the Centre's progress'. In similar terms, the AQSC Report also recommended in December 1996 that the University should produce clear written descriptions of the 'responsibilities of staff at each institution to minimise the scope for "system failure" and over-reliance on informal networks and committed individuals'. The University will wish to reflect on the difference of views outlined above.
48 The Link Tutor's reports on NAC indicated to the audit team that a wide range of the partner's activities was being monitored and that matters of concern were being fed back to the University for summary and inclusion in the School's AQR. However, the full effectiveness of this arrangement was to some extent called into question for the team firstly, by inaccuracies in a report submitted by the Link Tutor and secondly, by a report from the Link Tutor of a breakdown in the academic leadership of the programme of study at NAC during the 1995-96 session, at a time when the Link Tutor herself was ill, and had been replaced for a period of about six months by a colleague in the same School.
49 In one of the Link Tutor's reports, dated October 1996 a recent visit was described as having confirmed 'fears' that procedures prescribed by the University had not been followed by NAC, and that deficiencies in the administration of the Dip HE had occurred 'during the previous academic session'. These deficiencies appeared to the audit team to have included lapses in communication between the University and NAC, and administrative lapses within NAC. Having been informed of this matter by the Link Tutor, the Director of NAC had secured the immediate appointment of a new Course Leader at the Centre and the School later carried out a strategic review of the partnership.
50 Reviewing these matters, it appeared to the audit team that the University's
reporting arrangements had not been sufficiently robust for it to follow
developments in Athens, or for it to be able to supply the team with reliable
information. In the light of the recommendations of the AQSC Report, the
University will wish to consider the necessity of reviewing the adequacy
of reporting arrangements for its overseas provision and to clarify and
formalise the quality assurance and quality control responsibilities of
its Link Tutors (see also below, paragraphs 54 and 70).
Annual Quality Review
51 Since the original approval of the Dip HE in 1994, and as a consequence of
the floods which delayed the start of the Dip HE programme until February 1995
(see above, paragraph 37), the University has only received one Annual Report
from NAC for inclusion in the School's AQR. Dated January 1997, it relates to
the period from the approval of the Dip HE until June 1996 and, together with
reports from the Athens-based external examiner and the Link Tutor, was incorporated
into the AQR completed by the Director of the School. The AQR prepared by NAC
describes some of the developments taking place at the Centre but it contains
no information on student performance and progression, or student numbers, and
the reader is referred elsewhere for this information. The team also noted that
the report was titled 'Bachelor of Arts (Honours) in Psychology and Education',
although this programme only commenced in October 1996 and the Annual Report
therefore in fact referred to the Dip HE.
52 The Outreach Appendix to the School's AQR in 1996 identified the need to 'monitor changes to the learning culture at NAC' and stated that 'various problems relating to the moderation of examination papers were encountered at the Avgerinopoulou Centre and remedial action was taken'. In the absence of a more explicit statement of the nature of such 'problems', and of clearly stated progression and assessment data in the text of the AQR, the audit team found it difficult to envisage how members of the University outwith the School might be able satisfy themselves as to the well-being of the School's outreach provision. The team therefore welcomes the School's decision that future School AQRs dealing with outreach provision should include statistical information on student progress.
53 At the time of the audit the AQSC Report had concluded in general terms
that the 'University's AQR process [was] ... too infrequent and too remote
to ensure quality is being maintained by an organisation to which regular
visits by School staff are costly and whose own staff rely more naturally
on their own established cultural traditions and norms of practice'. For
the future the Report recommended closer quality control of overseas collaborative
provision by schools, supported by central mechanisms and a range of other
detailed measures, including 'an enhanced AQR process for overseas programmes'.
The audit team would endorse this view and believes that, taken together,
these and the other measures identified by the Report (see, for example,
paragraphs 48 and 51 above) would do much to address the shortcomings in
quality control and monitoring identified by the team in this particular
instance. The University is to be commended for its clear understanding
of the full range of measures required to improve the operational quality
systems needed to support its overseas collaborations, and to be encouraged
in their rapid and complete adoption.
Feedback information from students
54 Course team meetings at NAC are held monthly, and there is formal provision
for the election of two student representatives from each year to attend such
meetings, a procedure which was currently not being used. Nonetheless, students
are able to put matters forward for discussion at course team meetings. Students
are invited to complete evaluation forms on each unit of teaching but, while
staff were able to obtain aggregate feedback, this did not appear to be available
to students, nor did they receive minutes of course meetings.
55 The cohort of students which was registered for the BA/BA (Hons) Psychology and Education at the time of the audit had progressed together from Year 2 of the Dip HE. They had been studying at NAC for a number of years when they met the audit team and clearly enjoyed good working relationships with their tutors. The students and the Centre have developed a number of means of providing informal feedback on the experiences of the former to staff as well as means for students to seek and receive feedback on their attainment in return.
56 The students who met the audit team expressed themselves content with the written and oral feedback they received on course work and had also met the link tutor on her visits to NAC. The team was satisfied that students did have opportunities to provide feedback on the University's courses at NAC but the University may find it desirable, if and when there is a significant increase in the number of students, to encourage the development and use of more formal procedures for gathering and responding to feedback information from its students at NAC.
Assessment and academic standards
58 An Annex to each of the Validation Agreements which govern the
operation of the Dip HE and BA/BA (Hons) Psychology and Education, sets
out the broad quality assurance responsibilities of the parties, and assigns
responsibility for the administration of assessments to the Centre (see
above, paragraph 19). Each Annex states that the Centre's assessment regulations
must be approved by the University's Academic Registrar and that a 'Board
of Examiners will be established in accordance with University requirements
and regulations'. The audit team considered that the clarity of the Annex
might be improved if the responsibilities of each of the parties were to
be specified in greater detail (see above, paragraphs 21 and 54). NAC is
a University of London examination centre and the University is confident
that the administration and invigilation of examination sessions is scrupulously
effected.
59 The work of students following the Dip HE and the BA/BA (Hons) Psychology
and Education is assessed by course work and examination. Examinations
are set by staff from the Centre and moderated by staff from the School.
The assessment of course work and examination assignments is governed by
the University's standard Assessment Regulations. Students are issued with
a comprehensive handbook which includes detailed assessment regulations
and provides information on types of assessment, the weighting between
course work and examinations, and criteria for assessment and progression.
Appointment of external examiners
60 The University has appointed two external examiners to oversee its programmes
at NAC. The UK-based 'chief' external examiner, who was already acting as an
external examiner for the School's BSc (Hons) Psychology, was appointed in 1996
to join the Athens-based external examiner for the Dip HE and BA/BA (Hons) Psychology
and Education (see below, paragraph 61). The duties of the UK-based external
examiner include overseeing the progression of students to level three (that
is, from the Dip HE to the BA/BA (Hons)); moderating all degree-level work, and
maintaining the standards of the University's awards.
61 To monitor the performance of students in all the University's courses at
the Centre, including those in the first two years of the Dip HE delivered in
Greek, the University took steps from the beginning of its programme in Athens
to appoint an Athens-based, Greek-speaking external examiner. At the time of
the audit, a new appointment had recently been made to this position.
62 The Athens-based external examiner, who had formerly been an external
examiner for one of the predecessors to the Sheffield Hallam University
programmes at NAC, met the audit team and was able to confirm that she
had received an information pack from the University about her role and
responsibilities, and that arrangements had been made for her to moderate
scripts and to attend the Examination Board at NAC in July 1997, together
with the UK-based chief external examiner. The process for making this
appointment followed the University's prescribed procedures, including
scrutiny by the AQSC, where a framework for relations between the proposed
Athens-based external examiner and the UK-based chief external examiner
had been discussed.
Conduct of assessments and award boards
63 The University's briefing papers included a tabulated summary of communications
between NAC and members of the School which the audit team was able to use to
supplement its understanding of exchanges between the University and the Centre
during Spring and early Summer 1996 as students at the Centre prepared to sit
formal examinations for the Dip HE. The team noted a succession of unfortunate
misunderstandings between the partners at this point, which suggested that the
responsibilities of each for the detailed administration of the assessment process
had not been clearly understood by one or both parties (see above, paragraph
51).
64 At the beginning of August 1996, NAC conducted an Assessment and Award
Board (AAB) in Athens. Although the Annex to the 1994 Validation Agreement
confers responsibility for the administration of assessments on NAC, the
University's Assessment Regulations and Procedures state that the members
of an Awards Assessment Board shall include 'the Award External Assessor(s)'
and a 'Secretary appointed by the Academic Registrar'. Neither the Athens-based
external examiner for the Dip HE, nor any representative of the University
appeared to have attended the Board, which was chaired by the Academic
Director of NAC.
External examiners' reports
65 At the time of the audit, the University had received one formal report from
the first of its Athens-based external examiners at some time after the end of
the third semester of the programme. It indicated that staff at NAC might be
experiencing difficulties in grading the students' work in line with her understanding
of UK expectations, and recommended changes in the guidance given to students
on how to study and that they should be encouraged to improve the standard of
their writing in English.
66 In addition to this report from the first of the Athens-based external examiners, the FLDC record of communications between NAC and the University showed that at least two informal moderation reports had been received from the UK-based external examiner. The first of these had been recorded by the FLDC as a 'UK External Examiner's Report', in which the UK-based external examiner had commented adversely on the generosity of the marking he had seen and the levels of attainment achieved by the students. In internal correspondence, University staff suggested that the standard being adopted by the UK-based external examiner might be inappropriate for a programme of study not designed to lead to a specialist award in Psychology or to meet the requirements of the British Psychological Society. The University will recognise the importance of ensuring that all those concerned with the assessment of students are in no doubt about the levels and standards of attainment that are required for its awards.
67 Notwithstanding the reservations expressed by some members of the University regarding the comments of the UK-based external examiner, the audit team noted that action had been taken on a number of issues. For example, the University had provided staff development opportunities for members of NAC, concentrating on teaching, learning and assessment. Additional essays had been set for students at NAC and tutorials had also been provided by members of the University, dealing with the development of critical and thinking skills. A report from the UK-based external examiner some months later found that significant improvements were to be seen in students' work and their standards of attainment, and expressed greater optimism for their future performance.
68 As externally validated programmes, there are no direct equivalents of the Dip HE and the BA/BA (Hons) Psychology and Education on offer within the University. Comparisons for the purposes of setting and maintaining academic standards are therefore difficult, a matter raised in correspondence between the UK-based external examiner and the School. The levels of attainment of the students on the programmes in Greece appeared to the audit team to have been the subject of continuing scrutiny by the external examiners, but the University will nonetheless doubtless consider it advisable to keep these matters under careful review, as it seeks to discharge its responsibilities for maintaining the standards of its awards. When the reports of the two external examiners have been received at the end of the 1996-97 session, the University will be well placed to conduct a careful analysis of the level of attainment of its students at NAC and to offer informed guidance to the School, staff and students at the Centre.
Staffing and staff development
Appointment
69 Within NAC, the Academic Standards Director, the Course Leader, the Academic
Director and the staff responsible for the library and information technology
provision are full-time members of staff. In common with other Liberal Studies
Laboratories in Greece, a majority of the teaching staff are on part-time contracts.
The audit team was told that the University had required the Centre to review
the contractual conditions of such staff following the 1994 validation. The Centre
consults the Link Tutor in making all appointments relevant to the collaborative
arrangement, and the performance of teaching staff is monitored through observation
of teaching sessions conducted on behalf of the University by the Link Tutor.
It appeared that the University had been consulted by the Centre when it had
made part-time teaching appointments, but the team was less clear that the University
had been appropriately involved in full-time appointments to NAC.
Staff development
70 Staff development has been a continuing feature of the partnership. Following
the 1994 validation the School was encouraged to provide staff development in
teaching and learning styles for NAC, whilst in 1996 the Validation of the BA/BA
(Hons) had required the School to commit itself to a comprehensive staff development
programme for NAC staff. As a result, three staff development events were held
in 1996-97 and a two-day staff development course was provided in Athens by members
of the Schools of HCS and of Education, linking learning outcomes to assessments
and levels of attainment. The audit team was told that the Link Tutor had also
provided some staff development sessions for small groups of the Centre's administrative
staff and that they were encouraged to attend staff development sessions provided
for the teaching staff.
Promotional Material
71It is part of the University's agreement with NAC that the former
should approve all promotional material for its programmes in Greece. Advertising
copy originated in Greek is available in translation. The University also
retains copies of all promotional material used by NAC. As stated earlier,
it was a condition of the limited approval for the BA in Psychology and
Education that no promotional material be published until the major review
in 1997 had been carried out.
72 The School has been in contact with NAC over publicity associated with a possible collaboration in the area of physiotherapy and, according to the table of communications which the University provided, one advertisement at least had been approved. The audit team was also informed that promotional material had been prepared for the direct recruitment to the Dip HE in 1997 and that this material had been submitted to the University.
Conclusions and points for further consideration
731 The University's validation of the Dip HE Education and Psychology
in 1994, in partnership with N Avgerinopoulou Centre (NAC), represented
its first validation of an overseas partnership programme and, at the same
time, its first experience of delivering collaborative provision in Greece.
To prepare itself for this new venture, over a period of some nine months,
the University took advice from NAC's previous UK partner and consulted
a range of bodies including
the British Council in Athens and the HEQC.
74 Following exploratory discussions with its future partner a proposal for a Dip HE was developed by NAC in 1994 with the assistance of the University's School of Health and Community Studies (HCS). The University subjected this proposal to a thorough process of validation before approval was granted. The validation and approval process for the BA/BA (Hons) Education and Psychology in 1996 was similarly thorough, and made plain the University's intention to safeguard its reputation and to proceed cautiously. However, in choosing to validate this provision rather than, for example, to franchise an existing University programme of study, the University may have established a relationship with its partner which was insufficiently close to allow either partner to develop the provision as securely as each intended, as revealed by an analysis of the working of the University's arrangements for monitoring and reviewing its provision at NAC.
75 Arrangements for monitoring and reviewing the provision at NAC followed the University's general requirements for validated provision. There is some possibility that these were augmented by the enhancement of the role of the University's Link Tutor, the person in its School charged with day-to-day liaison between the University and NAC, although the evidence for such enhancement is inconclusive. Under its normal arrangements the University reviews validated provision within a six year cycle, although in this instance a condition of its agreement with NAC established the University's right to review its provision after three years. Since at the time of the audit these arrangements had yet to come into play, the full burden of assuring the maintenance of quality and academic standards had, until that point, fallen on the University's annual monitoring process and, in the period between annual reports, and to an undefined degree, on the Link Tutor. Tested by an unfortunate chain of events, these arrangements appeared to the audit team to have proved insufficiently robust for the University to be confident, until recently, that it had a full and accurate understanding of how its provision in Athens was faring.
76 At the end of 1996 a searching inquiry was undertaken by the University's Academic Quality and Standards Committee into arrangements to assure the quality of its overseas provision and maintain the standards of the resulting awards. The audit team congratulates the University on the thoroughness of this inquiry and the resulting AQSC Report, and supports the implementation of the recommendations of the latter which, when taken together with its own recommendations will allow the University to continue to maintain with confidence the quality of its provision overseas and the academic standards of the awards to which it leads.
Top