Introduction
1 This is the report of an audit, carried out by the Higher Education Quality
Council (HEQC), of the quality assurance arrangements for a collaborative partnership
between the Open University, and Euro-Contact Business School, Budapest, Hungary
(hereafter referred to as ECBS). The audit examined the policies and procedures
used by the University to satisfy itself of the academic quality and standards
of its awards being offered in Hungary. The collaborative programmes included
in the audit were: the University's Professional Certificate in Management, the
University's Professional Diploma in Management and the constituent courses of
these awards.
2 The audit of the Open University formed part of a series of audits of overseas
collaborative provision carried out in the summer of 1997 and included a visit
to ECBS.
3 A general description of the Open University's mechanisms for assuring the
quality of its collaborative provision activities is contained in HEQC's TheOpen
University Quality Audit Report:Collaborative Provision published in July 1996.
An audit of the University's overall quality assurance systems had previously
been carried out by the Committee of Vice-Chancellors and Principals' Academic
Audit Unit, leading to the publication of a report in June 1992.
4 The Council is grateful to the University and to its collaborating partner,
the Euro-Contact Business School, for their assistance and co-operation throughout
the audit process.
The audit process
5 Prior to the audit visit, the University provided briefing documentation
outlining its aims and implementation strategies in relation to the collaborative
relationship with ECBS. The briefing material was largely drawn from working
papers. Following its reading of the briefing documentation, the audit
team proposed a programme of meetings for the overseas visit, and sought
additional contextual material to help it to confirm its understanding
of the structure and processes of the University's collaborative quality
assurance arrangements with ECBS.
6 A preliminary visit to the University took place on 15 May 1997, during which
representatives of the audit team met the Pro Vice-Chancellor (Quality Assurance
and Research); the Secretary for Quality Assurance; the Dean of the Open University
Business School (OUBS); the Country Manager for Hungary and other members of
OUBS; members of course teams; members of the Examinations and Assessment Office
and other University administrative staff concerned with the partnership. The
team visited ECBS on 9 June 1997, and met the Director of ECBS; course tutors;
administrative and support staff; and students of the Certificate and Diploma
courses, past and present. A further visit was made to the University on 6 July
1997, during which representatives of the team met the Pro Vice-Chancellor (Quality
Assurance and Research); the Secretary for Quality Assurance; the Dean of OUBS;
a Course Manager; and one of the members of staff involved in the verification
of Hungarian language materials. A commentary, prepared by the University, is
appended to this report.
7 The audit team comprised Professor T J Kemp, Dr J Longmore and Dr D Timms,
auditors, and Mr D P Fearnley, audit secretary. The audit was co-ordinated for
HEQC by Ms V M Rivis, Assistant Director, Quality Assurance Group.
Glossary of terms and abbreviations
8 In this report the following abbreviations have been used:
The University - The Open University
ECBS - Euro-Contact Business School
OUBS - Open University Business School
MBA - Master of Business Administration
Certificate - Professional Certificate in Management
Diploma
Professional Diploma in Management
Collaborative Provision
- The Open University Quality Audit
Report
Report: Collaborative Provision
(HEQC, July 1996)
OUWW - Open University World-Wide
OUEE - Open University Educational Enterprises Ltd
EAB - Examination and Assessment Board
TMA - Tutor-Marked Assignment
The background to the partnership
The University's policy and strategic aims
9 The University claims that all its international activity springs from its
stated mission and its strategic aims and objectives, which are described in
its publication Plans for Change (OU 1995). Strategic Aims 1 and 8 define the
University's 'International Roles':
'To play a leading role in the expansion of higher and continuing education in
the UK and of English-language, distance teaching in Europe.'
'To stimulate and respond to a world-wide demand for open lifelong learning by
making the Open University's distinctive courses, awards, materials and expertise
widely available internationally, by developing these further to meet identified
needs, and by positioning the University as a respected, effective and accessible
provider of quality.'
10 As the language of instruction of a number of the University's courses
offered by ECBS is not English, the collaborative relationship with ECBS
goes beyond the strict boundaries of the aims stated above; however, the
University is to be commended on developing its international activity
within a clearly stated and publicly accessible framework of policy.
The collaborative relationship with ECBS
11 ECBS is one of a number of new ventures in business education which have grown
up in Hungary as a result of the development of free-market economics. Since
1989-90, there have been considerable changes, not only in the curriculum of
subjects in business-related higher education, but also in the whole approach
to learning, in which didactic formalism is beginning to give way to a more student-centred
approach. However, the content of public higher education is still, to some extent,
determined by state agencies, and a developing system for the accreditation of
subject-offerings in Hungarian higher education institutions is likely to increase
this control, particularly in terms of the curriculum. As a private institution
which does not seek to offer qualifications accredited within the Hungarian system,
ECBS operates within somewhat different constraints: namely, the need for financial
viability, on the one hand, and the quality assurance demands of the Open University,
on the other.
12 ECBS was established, at least in part, under the stimulus of the 'Know-How
Fund', which has provided resources to enable travel and the transfer of knowledge
between the UK and a range of Eastern European countries since 1989. The University's
legal relationship is with a Hungarian limited company, EC-NVT (in English, Open
Management, School and Business Consulting Ltd). Open University Educational
Enterprises (OUEE) is a 15 per cent shareholder in EC-NVT, and appoints one member
to its three-member 'curatorium', roughly equivalent to a board of directors.
The purpose of the initial relationship, which was established in 1989, was to
allow EC-NVT to make use of the University's course material for the preliminary
constituent course of its Professional Certificate in Management, 'The Effective
Manager'. A legal agreement defined EC-NVT's and the University's rights and
responsibilities in the use of the materials; permitted EC-NVT to translate the
materials into Hungarian and to present the course in that language; and specifically
excluded the University from any responsibility regarding assessment, effectively
defining any award available on successful completion of the course as EC-NVT's
alone. In 1992, the status of the arrangement changed and developed, after a
further injection of resources from the Know-How Fund and an assessment of EC-NVT's
capacities by the University. Under the new arrangements the responsibilities
of each party were redefined, especially those concerning assessment and quality
assurance and control, and the awards became those of the University. The portfolio
of provision was extended so that the complete range of courses needed for the
award of the Certificate could be offered. In 1994, the legal agreement was supplemented
to make available the University's Diploma courses. A range of courses was specified
as being available in Hungarian translation, under conditions prescribed by the
contract. EC-NVT was given special dispensation, enshrined in the legal agreements,
to continue to offer the Hungarian version of 'The Effective Manager', although
it had elsewhere been replaced by a new course within the University's standard
curriculum for the Certificate (see below, paragraphs 19 and 24).
13 In 1993, EC-NVT established a not-for-profit foundation, under the provisions
of the Hungarian Civil Code, named the 'Euro-Contact Business School British
Hungarian University - Distance Learning Fund'. The Fund's trustees were appointed
by EC-NVT. The Managing Director of EC-NVT is the President of the trustees of
the Fund. The objective of the Fund was the establishment of a 'Hungarian university-level,
internationally acknowledged business school... through implementing the international
distance learning university business school programme of the British Open University
Open Business School.' The Fund established ECBS, whose Director is in fact the
Managing Director of EC-NVT. 'Euro-Contact Business School' is the name that
now appears on all ECBS's learning materials, in addition to that of the Open
University, and it is the name by which its students know the organisation as
a whole. The school occupies a suite of offices in central Budapest, which comprise
administrative facilities, a small library of OU and ECBS course materials, computer
facilities, and a small number of seminar rooms. In November 1996, 958 students
were registered for courses contributing to the Certificate and the Diploma with
ECBS, including 235 registered for 'The Effective Manager'.
14 In the legal documents provided by the University (though not in a document
published by the University itself) EC-NVT is described as 'the exclusive representative
of the Open University Business School in Hungary'. The audit team learnt that
ECBS acts as a recruiting agent for the University's MBA in Hungary. The Director
of ECBS described the ECBS offices as akin to an Open University regional office
in the UK, in terms of its functions. The team formed the view that the collaborative
partnership between ECBS and the University was an unusually close one and was
interested to explore how the relationship was perceived by staff and students
of both institutions.
Systems and arrangements for quality assurance
15 The details of the arrangements adopted by the University for the
quality assurance of its collaborative activities are described in the
Collaborative Provision report. The audit team learnt that, since the last
audit visit, there had been some changes in the University's organisational
structures relating to the management of overseas collaborative relationships.
In particular, a new University body, Open University World-Wide (OUWW),
had been established, which incorporated the former Office for International
Collaboration, and Open University Educational Enterprises Ltd (OUEE).
16 Arrangements for the initial investigation and development of international
partnerships are now the responsibility of OUWW. On behalf of the University,
OUWW provides an initial filtering mechanism, judging potential partners against
the key criteria of academic probity, operational feasibility and financial viability.
OUWW also checks the local status and credibility of partners with education
departments or ministries. For any requests which pass this initial screening
process, OUWW will routinely seek the advice of the local British Council Office
or the Embassy. This paper research is backed up by visits, in both directions,
often organised via the British Council in the initial stages, before any collaboration
is put through the formal approval structures of the University. Given the date
of its inauguration, it is not possible to comment on the efficacy of these arrangements
in relation to this particular collaborative relationship. However, it appeared
that the University had considered the requirements of international partnerships
and their establishment carefully, and had put in place a structure that, in
the view of the audit team, seemed likely to provide safeguards against inappropriate
arrangements.
17 In academic matters directly relating to his or her particular course, the
management of collaborative relationships is the responsibility of the University
Course Team Chair, and, for more general matters, of the OUBS Country Manager,
who has responsibility for all presentations of University courses in a particular
territory. Course teams also have 'course managers', who are not academics, but
who attend meetings of course teams as full members, and who manage the day-to-day
affairs of courses, dealing with, for example, residential schools and possible
problems with tutors. The ECBS partnership also has strong links with the University
Registry and Examinations and Assessment Office. In this way the University offers
a matrix structure of individuals and specific expertise to cover all operational
aspects of partnerships, both academic and administrative, which may arise.
18 While the University made no stipulation as to the means of course management
at ECBS, it appeared to the audit team that the School had, in fact, adopted
some elements of University practice. In particular, each course has an ECBS
counterpart corresponding to the University course team chair, known by the same
title. The Hungarian Course Team Chair's responsibilities cover the initial development,
translation and adaptation of a particular course, its management while it is
in the period of presentation, and chairing assessment boards in Hungary or attending
'umbrella' boards in the UK (see below, paragraph 30). There is no 'course team'
in Hungary (despite there being a 'chair'), although the team learnt that informal
ECBS staff meetings take place at the beginning of each semester.
Validation and approval
19 Since Open University partnerships are established to run courses
already designed and approved for use by the University itself, they inherit
the validation and approval of the original programmes. As the University's
courses are offered through distance learning, there is no intrinsic difference
between presentations (the University's term), wherever they are mounted.
Thus the initial validation of the courses encompasses the ECBS versions.
There are three differences of substance between the ECBS presentations
of courses within the Certificate and the Diploma and the presentation
of the same course in the UK: the ECBS presentation is in Hungarian rather
than in English; ECBS has dispensation to vary the content of the course
by up to 30 per cent to respond to local needs; and ECBS has a special
arrangement to continue, for a period, to present the preliminary course
for the Certificate, 'The Effective Manager', although it has been superseded
(see above, paragraph 12, and below, paragraph 24). It was the view of
the University that these differences were not significant enough to warrant
separate validation. Approval for the variations and the processes by which
they would be monitored was granted directly by the University's Academic
Board.
20 The continuation of the original 'Effective Manager' course has had the consequence,
acknowledged by the University, that students are required to obtain credit transfer
(that is, recognition of this course as if it were studied entirely at another
institution) in order for it to count towards other University awards. The audit
team was surprised to find that students of the course, by far the largest single
group of ECBS students, had no knowledge of this fact. The team was informed
by the Director that the transfer was initiated by ECBS on behalf of the students,
and was automatic, under the terms of its relationship with the University. The
team was unable to find any reference to the automatic nature of the transfer
in the legal documentation. However, there was a reference, within other documents
provided by the University, to a discussion of the problems that would arise
from the need for students to pay a fee to the University for the process of
transfer, in circumstances where their studies were not covered specifically
by this 'automatic' arrangement. It appeared to the team that students would
benefit from being in possession of all the facts about the status of the programme
within the University and the details of the operation of its credit transfer
system. The University may wish to review the extent to which the information
given by ECBS to its students about the University's procedures and regulations
is in line with its own practice in relation to students elsewhere.
Programme monitoring and review
Translation
21 The requirement that no more than 30 per cent of the content of the ECBS courses
may be varied is monitored by a bilingual 'verifier' appointed by the University,
who reads and comments on all materials from the point of view of language, comprehension
and adaptation to the local cultural and business environment. This practice
was followed in the adaptation of all the original courses, and it has also applied
in the adaptation of the course replacing 'The Effective Manager', due to begin
in November 1997. The University is to be commended on its attention to the translation
and verification of learning materials.
Staff visits
22 At the outset of the relationship, initial training was provided by University
staff for ECBS tutors. Arrangements were also made for ECBS part-time tutors
and full-time administrators to visit the UK for training. In this way there
was direct monitoring of the development of the ECBS presentations. While the
training function now largely rests with ECBS, and the frequency of visits by
ECBS staff to the UK has diminished, there is nonetheless regular and frequent
visiting of ECBS by University personnel. The University Country Manager for
Hungary visits ECBS at least four times yearly, and reports on his visits. There
is a regular exchange of ECBS and University staff in connection with examination
and assessment boards (EABs). ECBS course team chairs visit the UK yearly for
'umbrella' boards, (see below, paragraph 30), the membership of which includes
their counterparts from other European presentations of OUBS courses. These arrangements
appeared to the audit team to provide useful opportunities for ECBS staff to
exchange general information with University staff about the progress of courses.
The team was told that other OUBS academic staff associated with the courses
visit on an ad hoc basis. Contacts between University staff and ECBS staff appear
to be regular and frequent: in the view of the team, this fact, and the commitment
it implies, has been an important factor contributing to the success of the partnership.
23 The Country Manager makes written reports of his visits to the Dean of OUBS
and reports have been produced as a result of ad hoc visits by other staff. In
particular, the audit team noted a report produced by members of the staff of
the Examinations and Assessment Office, which, although from staff based within
the University, represented a view external to OUBS. However, there appeared
to be no formal avenue of scrutiny for such reports. Members of ECBS staff who
met the team showed no knowledge of the content of the reports, nor of any others
made by teams or individuals visiting from the University. It appeared that no
formal reports resulted from the visits of members of University course teams.
The team considered that, while commendable care was taken by the University
to maintain contact with ECBS, such that the University's commitment to the Hungarian
presentations and their quality was beyond doubt, the lack of formal processes
for scrutinising such reports meant that there remained the possibility that
important observations contained in them might be overlooked. It also appeared
to the team that significant opportunities for enhancement were lost, by not
ensuring that a wider range of staff at ECBS were acquainted with the reports.
The University may wish to consider a review of its procedures for the consideration
of, and response to, reports of staff undertaking visits to partner institutions.
Course review
24 Since all University courses are time-limited (normally to six years) from
the date of their initial presentation, no course is ever the subject of 'review'
as conventionally understood. Instead, replacement courses go through the process
of initial validation. Although the ECBS version of 'The Effective Manager' course
was permitted to continue for two years beyond its UK 'shelf-life', the audit
team noted that arrangements were in hand to introduce, in November 1997, an
adapted version of the University course that replaced 'The Effective Manager'
(see above, paragraphs 12 and 19). The team learnt that this new course will
be subject to the same process of verification of the accuracy of the Hungarian
translation as before, and to the regulation that there should be no more than
30 per cent variation in content from the UK presentation.
Arrangements for assessment
Assessment policy
25 It is the policy of the University that the assessment process is common to
all presentations of courses leading to University awards. The assessment strategies
for all courses must be approved by the University's Examinations and Assessment
Committee, which is responsible for ensuring compliance with the University's
academic standards. Students are advised in a 'study calendar' about the assessment
strategy for their course; formative and summative assignments are identified,
and the contributions of continuous assessment and examination towards the course
result explained. The University ensures that the strategy is applied by a variety
of means, including the production of prescriptive guidelines and frameworks
for marking, for those who mark assessed work, and for examination and other
assessment boards.
26 Continuous assessment is a key feature of all University courses. The
methods of assessment used are mainly based on Tutor-Marked Assignments
(TMAs) marked by locally-appointed tutors. Tutors receive training for
their marking and assessment roles. Their marking is regularly monitored
within a formal three-tier framework. Tutors' marks are sampled by monitors
appointed by ECBS, and the monitors' comments are further sampled directly
by University staff in the UK, after translation. Tutors are monitored
differentially, by reference to their experience and the results of monitoring
itself. The audit team formed the view that the system used by the University
in the UK was also used by ECBS.
27 In common with most other University courses, the courses presented at ECBS
also involve terminal three-hour examinations. The University has a range of
mechanisms for assuring itself that marks are awarded in accordance with its
specified guidelines. Script markers are appointed and trained to mark and grade
examination papers to University specifications (see below, paragraphs 43-44).
While Script markers need not necessarily also be tutors, staff of ECBS reported
that, in practice, they were always also tutors in Hungarian presentations of
University courses. A system similar to the one described above for continuously
assessed work is also applied to examination scripts; in addition, work recommended
for failure and borderline marks, and work recommended for the highest levels
of awards, is sampled.
Examination processes
28 There are currently two main examination periods, in October and May. For
overseas collaborations, examinations are identical to those taken by UK-based
students on equivalent programmes and conducted, as far as possible, at exactly
the same time. ECBS provides the examination centre; it is a requirement that
partners must operate their examination centres just as in the UK. Examination
papers are set in English in the UK and translated by the University into Hungarian.
Question papers are previously unseen, but students receive a specimen paper
during their course, so that they are familiar with the appearance and structure
of the examination. The University provides an extensive programme of training
in running examinations, co-ordination (where marking guidelines are agreed),
and standardisation (where any necessary adjustments are made to the mean scores
given by particular markers or for particular questions). Quality control is
exercised through the provision of written procedures and marking guidelines,
cross membership of co-ordination meetings and EABs, as well as through the appointment
of external examiners (see below, paragraph 32).
29 For all courses leading to the Certificate and Diploma, TMA and examination
scores are collated and presented to an EAB at ECBS. The EAB recommends results
on the basis of conflating continuous assessment and examination results, weighted
in accordance with the course's assessment strategy. In reaching their recommendations,
EABs use the University's Senate Guidelines for the Award of Results. At its
meeting, the EAB has available all the examination scripts, information about
special circumstances affecting individuals, information about the Script markers,
any adverse conditions affecting examination centres, and supporting statistical
information on the distribution, means and standard deviations of the continuous
assessment and examination scores. These arrangements are intended to ensure
that the marking standards and pass/fail thresholds set are consistent with UK
practice, and that the administration of these arrangements is properly carried
out.
30 According to the University, the results of courses are compared with
each other and UK parent courses at 'umbrella' EABs held in the UK, consisting
of representatives from the various collaborative schemes in Eastern Europe,
including relevant ECBS course team 'chairs', and the members of UK Award
Boards. Results are also considered by the University's Results Ratification
and Awards Classification Panel, which is responsible for monitoring the
maintenance of standards of approved awards and the ratification of all
course results, on behalf of the Senate. The University suggested that
these arrangements operate for all courses.
31 During the visit, from discussions with the staff of ECBS, the audit team
heard that in fact a dual system was in operation. For some courses a member
of the University's staff chairs the EAB held in Hungary: where this is the case,
there is no 'umbrella' board. The team understood that this system was used for
courses leading to the Diploma. In other cases, a member of ECBS staff chairs
the EAB held in Hungary, and results are then recommended to the 'umbrella' board
which meets in the UK. The team understood that this system was used for most
courses leading to the Certificate. However, according to ECBS, 'The Effective
Manager' course lies outside this system, in that its EAB is chaired by the ECBS
Course Team Chair, includes no University representative, and has no 'umbrella'
board. The team was unable to reconcile these conflicting accounts of the system
or systems in operation. Nevertheless, it seemed to the team that there was regular,
if not invariable visiting by OUBS staff, and that regular training in assessment
matters was given by University staff to ECBS staff at all levels. There were
strong circumstantial grounds for believing that good practice in other courses
was extended to 'The Effective Manager'. Although this course will cease to be
offered in its present form after 1997, the University may wish to ensure that,
in future, no other course is treated separately in this way. The University
may further wish to ensure that there is a common understanding between itself
and its partner of the membership of, and arrangements for, all examination and
assessment boards relating to ECBS presentations of Open University courses.
External examiners
32 The University states that 'the maintenance of standards is assured by the
appointment of external examiners for each course'. The same UK external examiner
is appointed, with identical responsibilities, irrespective of where a course
is presented. In addition, ECBS is required to appoint local bilingual external
examiners, who are approved by the University. External examiners proposed by
ECBS submit full curricula vitae which are considered by the Dean of OUBS, and
checked by the Examinations and Assessment Office against CVCP guidelines for
the appointment of external examiners. If the proposal meets these requirements,
the Dean recommends appointment to the Pro Vice-Chancellor (Curriculum and Development),
who may approve it or reject it on behalf of Senate.
External examiners' reports
33 The UK external examiners are required to make written reports annually. These
are submitted to the University Examinations and Assessments Office, for distribution
to course teams at the University. Responses to the comments of external examiners
are made in the following year's examination and assessment board meetings, with
opportunities for discussion regularly set aside for this purpose. While ECBS
representatives are normally present at 'umbrella' assessment boards at the end
of the year (see above, paragraph 31), the boards are quorate without such representatives.
It appeared to the audit team that there was no absolutely reliable means whereby
the University could be assured that comments made by external examiners, having
a direct bearing on the operations of ECBS, would be considered and, if necessary,
acted upon. For instance, one external examiner's report seen by the team commented
that it appeared that the workload of individual ECBS tutors might be too heavy.
It appeared that this comment had not yet been brought to the attention of the
Director of ECBS.
34 The audit team was informed by members of staff at ECBS that the 'local' bilingual
external examiners, appointed in Hungary, make written reports on the conduct
of assessment at the relevant assessment boards, which are sent back to the University.
No mention of these reports was made by the University and no examples of them
were included in the documentation provided by the University prior to the audit
visit. The team concluded that the reports may not be fully utilised as a further
means of monitoring courses. The University may wish to consider whether it should
establish a more reliable means, whereby the contents of written external examiners'
reports, from both UK and Hungarian external examiners, might be communicated
to relevant staff, and the responses to the comments registered.
Appeals
35 In the University's programmes offered in the UK, procedures are in place
for appeals against assignment scores; against course results; against the cancellation
of registration; and against decisions taken about the progress of an individual
student. Students have a right of appeal if they think that a decision is unfair,
or that an application has not been properly considered. The appeal is initially
referred back to the committee which took the decision. If students are not satisfied
with the outcome, they may then appeal formally to the Academic Registrar, who
refers the matter to the Pro Vice-Chancellor (Student, Tutorial and Regional
Services), or to a panel established by the Senate to act as independent investigators.
The Pro Vice-Chancellor or the panel has the right to ask a committee to reconsider
a decision.
36 In documentation supplied to the audit team the University states that the
standard procedures 'are publicised in the Student Handbook and apply to students
in Hungary as in the UK'. The team learnt that students at ECBS are aware of
the mechanism for making appeals against decisions of examination boards, and
also against the marks awarded by individual tutors: these arrangements replicate
those which apply in the UK. Similarly, tutors appeared to the team to be fully
aware of the processes applicable when students complained about their marks
for TMAs.
Certification
37 All award certificates are produced at the Open University and are in English.
Certificates do not currently make reference to the fact that the language of
instruction and examination is not English. The University recognises that it
will need to reconsider this matter in the light of HEQC's Code of Practice for
Overseas Collaborative Provision in Higher Education 1996.
The student experience
Student feedback
38 The audit team was told that 'the University considers comments and complaints
to be a valuable source of information which helps it to maintain standards and
improve services'. In its meeting with ECBS students and tutors, the team was
informed that formal, albeit relatively limited, means existed to enable students
to make known their feelings to ECBS about the development of courses. Residential
weekends were arranged in connection with the courses, at the end of which students
were invited to return questionnaires detailing their views. The team learnt
that the services of some part-time staff had been dispensed with as a result
of unfavourable comments in these questionnaires. However, it appeared to the
team that there was no formal arrangement whereby information gained from these
exercises could be relayed to the University. It appeared to the team that the
University may have been inadvertently overlooking an important source of information
about its courses.
39 There is no mechanism for ECBS students to be represented in the University's
policy-making structure, as UK students are, for example, by the Open University
Students' Association. The legal agreements between ECBS and the University specify
that, in the matter of complaints, ECBS is the first point of contact for students,
and that only where no satisfaction is gained do the standard University procedures
come into play. The audit team learnt that the Country Manager received e-mail
communications from students and that where he was unable to answer queries,
they were referred to the Course Team in the UK. It was not clear to the team
how students were made aware of the mechanism for making complaints, and, in
particular, what route should be followed when they were dissatisfied with the
responses of ECBS. There appeared to be no monitoring by the University of complaints
made to ECBS. The University may wish to consider ways of ensuring that feedback
from students is obtained and used to monitor the effectiveness of its courses.
OU policy and the Student Charter
40 The policy of the University is to produce high quality courses to be taught
at a distance, and to establish the quality assurance procedures to ensure that
this is carried out to a comparable standard wherever they are delivered. Overseas
partners 'add value' to the courses by offering additional language and tutorial
support, and, in the case of ECBS, some variant course material to respond to
local needs and conditions. ECBS is responsible for offering student support,
such as tutorial support, academic counselling and careers guidance; the University
takes no formal responsibility for these matters. However, the audit team formed
the view that practice was not wholly in line with policy, in that the Country
Manager apparently provides some informal academic counselling to ECBS students.
Discussions with students of ECBS suggested that, while they understood that
ECBS and the Open University were formally distinct organisations, for practical
purposes, they were barely distinguishable. The team took the view that this
possible confusion about the respective functions of the two institutions was
understandable, given the way in which ECBS represented the partnership to students.
41 To assist the audit team in its inquiries, the University included an informative Student Charter amongst its briefing papers. Many of the provisions of the Charter appear to be at variance with the policy stated by the University for its ECBS students, for instance, with regard to representation and the provision for giving feedback. Moreover, it seems that the Charter is not translated into Hungarian, and is not distributed to ECBS students. The University may wish to consider where the boundaries of its responsibilities with respect to ECBS students lie, and how best to explain them to the students themselves.
Staffing and staff development
42 For the students who undertake ECBS courses, the most important staff encountered
are the part-time tutors employed by ECBS, who provide face-to-face tutorials
for students wishing to take advantage of the service, and who mark continuously-assessed
work. The involvement of the University in ECBS staffing has become more indirect
over time. For the very first presentation, University staff took part in the
selection of potential tutors. The first cohort of tutors all had to take, as
students, the first course in the programme before they began to teach students
themselves. This course was 'The Effective Manager', taught in English by University
staff. There was some ambiguity in the briefing papers supplied to the audit
team as to whether the requirement for tutors to take courses first as students
had, or had not, continued into subsequent presentations. The team was able to
establish that, in practice, this was true of most, but not all tutors. Some
tutors had followed the course as students in the most literal sense, having
paid fees. More recently, the University has provided a specification for tutors
which has been used by ECBS in recruiting new staff. The requirement to use this
specification is built into the legal agreements between the partners, as is
a requirement that ECBS should provide training to new staff in performing their
roles. The team would wish to commend the care of the University in extending
this arrangement to ECBS administrative staff, as well as to academic staff.
43 Script markers are appointed against a specification provided by the University,
and are given training in their work. The first group of Script markers was also
given training by the University. Subsequently, training has been implemented
by appointing a 'lead' Script marker, who attends a training and moderating session
in the UK provided for all markers for the course, and passes on the knowledge
and information gained to the other Script markers. Marking is also carried out
against a specification provided by the University. The audit team formed the
view that the training given to the first group of tutors as Script markers had
been thorough, and that the cascade of expertise to other Script markers was
working effectively.
44 The audit team was less sure about the effectiveness of the University in maintaining a full awareness of the conditions of service under which tutors operate. For instance, the team was surprised to note that financial rewards for marking were based on a variable scale related to quality of the markers' work, including the speed with which marks were returned to ECBS. It seemed to the team that this implied that marking of poorer quality might be tolerated, even though penalised by a lesser reward. While this information was available to the University's Course Team, since it had been mentioned in an evaluation of the operation of the course made by University staff, there was no indication in papers seen by the team, or in meetings with University staff, that the arrangement had been considered by the University. The University may wish to consider whether some more formal method might be introduced, to ensure that arrangements for the appointment and reward of ECBS part-time tutors and Script markers meet its own criteria for good practice.
Publicity and promotional materials
45 The doors of the Budapest ECBS offices bear the names of ECBS on one side,
and that of the University and its logo on the other. A coat of arms is prominently
displayed, combining the logos and names of ECBS and the University: the device
also appears on brochures included in the briefing papers supplied to the audit
team. These brochures appeared to the team to suggest that the relationship between
the ECBS Certificate and Diploma presentations and the University MBA was continuous.
For instance, every page on the ECBS primary marketing brochure supplied to the
team had a running 'header' consisting of a list: 'Certificate Diploma MBA'.
The University claimed that, while publicity materials are not formally scrutinised
page-by-page like course materials, it is 'satisfied that brochures are entirely
appropriate'. It is the responsibility of the Country Manager to ensure that
publicity and promotional materials for the course are accurate. To enable him
to do so, he receives copies of all ECBS materials: where there is any aspect
which he believes may be misleading, he uses one of the University verifiers
for course materials to check on his behalf. The University acknowledged that
this system is largely based on trust. It appeared to the team that the association
of the MBA with the courses which ECBS presents under its legal agreement might
confuse students, in suggesting that ECBS had a status that properly belongs
only to the University itself. The University may wish to consider how best to
make clear to students the nature of its arrangement with its partner, and to
consider whether its current arrangements for ensuring the accuracy of ECBS publicity
and promotional material meet its own requirements.
Conclusions and points for further consideration
46 The collaborative relationship between the Open University and
the Euro-Contact Business School in Budapest, Hungary, has been in existence
since 1989. Since that date it has been developed and extended so that
many Hungarian students have successfully followed Open University courses,
and courses very closely related to University originals delivered in Hungarian,
leading to the Professional Certificate and Diploma in Management of the
University. Such students also have the opportunity to progress to the
University's MBA, for which ECBS acts as recruiting agent.
47 The partnership has operated within a coherent and public framework of University
policy, consistent with its mission. The University has exerted effective controls
over the original translation and adaptation of course materials, and over the
introduction of the changes necessary to meet local conditions. The University
has also gone to considerable lengths to ensure that the first tutors for the
programme were provided with appropriate training in matters of tuition and assessment.
Training has been extended to those with administrative responsibilities for
managing the course and its assessment boards. Steps have been taken to ensure
that such training was ongoing, and cascaded from existing to newly-appointed
tutors. Assessment arrangements appear to be managed with thoroughness, and to
be such as to promote confidence in the maintenance of the standards of the awards
at a level equivalent to UK presentations. Frequent visits by various members
of University staff and reciprocal visits by members of ECBS staff ensure that
the relevant members of OUBS and those responsible for international relations
in the University are kept aware of progress and developments in Hungary.
48 Nonetheless, there are certain aspects of the partnership to which the University
may wish to pay further attention. ECBS students appeared to be less than fully
informed about the status of their studies with ECBS vis-a-vis entitlements from
the University, and the University appeared to be less than completely clear
about the status of students taught by its partner institution, particularly
in relation to pastoral matters. The substantial evidence, particularly written
material, generated by the University's monitoring processes could be used to
fuller effect for quality enhancement purposes. There appear to be some discrepancies
between the ECBS tutors' and the University's understanding of some procedures,
including the membership and organisation of examination and assessment boards
for all courses, which could readily be clarified. The publicity and promotional
material developed by ECBS to market its offerings might be monitored more closely
by the University. The planned replacement for the now out-of-date version of
'The Effective Manager' is to be welcomed, as it brings ECBS course offerings
more fully into line with UK presentations of Certificate and Diploma programmes.
The University's commitment to consider the issue of reference to the language
of instruction on its award certificates issued to ECBS students is also a positive
development.
49 In sum, the University appears to have managed effectively the difficult task
of reconciling the demands of local conditions and aspirations on the one hand,
and the preservation of standards and the reputation of UK awards on the other.
The University's claim that the quality assurance measures adopted to guarantee
that the quality and standards of the ECBS courses are either close, or identical,
to those applying to the presentation of University courses in the UK, is largely
substantiated. This achievement has been made possible by the considerable and
commendable commitment of University staff time and resources, at both an academic
and an administrative level, at the outset of the partnership, and in its continuing
operation.
Top