1 The Quality Assurance Agency for Higher Education (QAA) is a UK organisation which aims to promote public confidence that the quality of provision and standards of awards in higher education are being safeguarded and enhanced. It provides public information about quality and standards in higher education to meet the needs of students, employers and the funders of higher education. One of QAA's activities is to carry out quality audits of collaborative links between UK higher education institutions and some of their partner organisations in other countries. In the spring of 2001, QAA audited selected partnership links between UK higher education institutions and institutions in Hong Kong. The purpose of the audits was to provide information on the way in which the UK institutions are maintaining academic standards and quality of education in their partnerships with institutions in Hong Kong.
The process of audit of overseas partnership links
2 In planning these audits of overseas collaborative provision, QAA invited all UK institutions to provide a list of their collaborative links with partners in Hong Kong. On the basis of the information provided on the range and scale of the links, computing and information technology was selected as the subject focus for the audit. Each of the UK institutions whose collaborative link had been selected for the audit provided for QAA a Commentary describing the way the partnership operated, and commenting on the effectiveness of the means by which the UK institution assured quality and standards in the link. In addition, each institution was asked, as part of its Commentary, to make reference to the extent to which the link was representative of its procedures and practice in all its overseas collaborative activity or specific to this link, subject or country. QAA identified four UK institutions which had established arrangements where most or all of the educational provision was delivered through a partner in Hong Kong, and these institutions were visited by small teams of auditors to discuss the arrangements with appropriate staff and to look at relevant documentation.
3 Audit teams visited the partner in Hong Kong to gain further insight into the experience of students and staff, and to supplement the view formed by the team from the institution's Commentary and from the UK visit. During each of the visits in Hong Kong, further documentation about the link was made available to the team, and discussions were conducted with key members of staff, lecturers and students. In addition, members of the team, who included subject specialists in computing and information technology, were able to see facilities and resources available to the students. Examples of students' work were seen by members of the team, including the subject specialists either in Hong Kong or in the UK. QAA is grateful to the UK institutions and their partners in Hong Kong for the willing cooperation provided to the team.
4 Institutions were invited, in their Commentaries, to make reference
to the way in which their arrangements met the precepts of QAA's Code
of practice on the assurance of academic quality and standards in higher
education, Section 2: Collaborative provision (1999) (QAA's Code).
QAA's Code contains precepts and guidance about the assurance of
quality and standards in all collaborative provision, and it is expected
that institutions would be adhering to the precepts of the Code.
In the context of these audits, the Code was used as a reference
point by the audit team, and its contents are reflected in the observations
in
this report.
The context of collaborative provision with partners in Hong Kong
5 Under Hong Kong legislation introduced in 1997, all courses conducted
in Hong Kong leading to the award of non-local higher academic or professional
qualifications have to be registered or exempted. This means that all programmes
offered by UK institutions in partnership with institutions in Hong Kong
have to be registered or exempted. The criteria for the registration of
a course include the requirement that there are effective measures in place
to maintain the standard of the course at a comparable level with that
of a similar course conducted in the home country. Those courses that are
conducted in collaboration with a local institution of higher education
may be exempted from registration. Registration is approved by the Registrar
of non-local higher and professional education courses, and the decision
is based on advice from the Hong Kong Council for Academic Accreditation
(HKCAA). Registration or exemption does not mean that the course is also
recognised by HKCAA or employers in Hong Kong. Courses which are considered
to be 'purely distance learning' are excluded from the registration requirement.
The collaborative link between the University of Ulster and the Hong
Kong College of Technology
6 The collaborative provision considered in this report relates to the two-year part-time level 3 'top-up' course delivered by the Hong Kong College of Technology (HKCT, or the College) leading to the award of BSc (Honours) Computing Science of the University of Ulster. It also relates to the delivery of the same level 3 programme at the South China Agricultural University (SCAU) operating as an outcentre of the College. The College is based on sites in Mongkok and Homantin. It was established in 1957 as non-profit making organisation specialising in the delivery of professional programmes in computing, electronic and electrical engineering, financial services, management and languages. In addition to the link with the University of Ulster, the College has associations with other Universities in the UK and with universities in Australia and China. Hong Kong College of Technology International (HKCTI) is a separate legal entity set up by the College for the management and operation of overseas collaborative programmes. For the purposes of this report reference to the College, or to HKCT, will be taken to include HKCTI except where it is necessary to make a distinction between the College and HKCTI. The course is registered with HKCAA. The language of instruction and assessment is English.
7 An initial visit to the University of Ulster took place on 1-2 February 2001. The audit team comprised Mrs N J Channon, Dr D H Furneaux and Dr S Jackson. The team met the Vice-Chancellor together with the Pro-Vice-Chancellor (Quality Assurance and Enhancement). It held two formal meetings, one with University staff centrally responsible for the link, the other with Faculty staff responsible for the programme aspect of the link.
8 A visit to the College took place on 22 March 2001. The audit team comprised
Dr D H Buckingham, Professor R S Burgess, Mrs N J Channon, Dr S Jackson
and Professor B S Lee. The team met the Principal and senior staff of the
College, staff with responsibility for administration and management of
the programme, academic staff and a representative group of students. The
team also had the opportunity to review samples of students' work, including
projects.
Background
9 This collaborative link was established in 1997. It involved transferring a link started in 1991 with City University, Hong Kong, previously City Polytechnic of Hong Kong (CPHK), to HKCT when CPHK gained university status. The newly established City University sought changes to the existing agreement if it were to be continued; in particular it wanted students on the top-up course to be registered as students of the University of Ulster. This was not acceptable to the University, hence its decision to seek a new partner. An evaluation exercise, chaired by the Pro-Vice-Chancellor (Academic Affairs) took place in 1997, which resulted in the University giving approval to transfer the course to the College, the format and content of the programme remaining the same as in the collaboration with CPHK.
10 In September 1998 a proposal was made to the University to deliver the same level 3 course at SCAU. After a pre-validation visit had concluded that additional teaching staff would be needed at SCAU, a proposal describing the arrangement as delivery at 'an outcentre of HKCT' was submitted, with some part-time teaching staff of the College delivering the programme at SCAU. A University validation exercise in December 1999 recommended conditional approval for commencement in September 2000. The assessment of the competency of students at SCAU in English language was an issue which was not resolved until July 2000.
11 The initial approval at the College was for a two year part-time course at level 3, referred to as a 'top-up' or 'conversion' course designed to allow holders of the Higher Diploma in Computer Studies, or equivalent, to achieve a BSc (Hons) in Computing Science. The Computing Science programme is part of the portfolio of programmes offered by the University's Faculty of Informatics. The University's understanding of the status of the course was unclear to the audit team in the documentation. In various places it was referred to either as a franchise or as a validation. During its meetings with the University, the team was told that the course was a validation because what was offered was not the same as the final year of the University's degree programme although it resulted in the same award. In its discussions with the University's partner in Hong Kong, the team learnt that the College viewed the programme as belonging entirely to the University, and not a programme designed by HKCT. Although there is not a direct correlation with an existing programme at Ulster it is nonetheless a programme which has been developed at, and approved by, the University. The University may wish to clarify, both with its Faculty of Informatics and with HKCT, the ownership of the programme and the responsibilities of both partners.
12 In May 1999 the University received a proposal from the College to offer the existing University of Ulster MSc in Information Systems Management at HKCT from October 2000. Formal validation took place in December 1999 by the same panel that was considering the SCAU proposal, and conditional approval was given. This programme has not yet started and there are a number of outstanding issues yet to be resolved including the timing of delivery, the contributions from University staff and local approval.
13 General principles covering the selection of partners are described in the Guide to Collaboration in Course Provision, and the Commentary stated that the University 'may investigate the financial health of the (partner) institution'. When the need to find a new partner in Hong Kong arose, the audit team heard that HKCT came under consideration because of its close links with City University, that many of the City University staff could continue to teach the level 3 course part-time at the College, and the fact that the College had links in China. The transition from the arrangement with the former CPHK was managed with the assistance of City University Consultants Limited, an enterprise company of City University, which was contracted to secure the staff appointments and advise on the establishment of the programme at HKCT. The company has had a continuing role in the management of the transition process, but that role is now coming to an end.
14 According to the Guide to Collaboration in Course Provision, the University expects that 'all partner institutions should share its mission and values'. The procedures laid down for validation, approval, and monitoring are detailed and comprehensive, but make no specific provision for dealing with the additional challenges to quality and standards presented by overseas links. It appeared to the audit team that the University relied primarily on procedures and arrangements that have been successful in the UK, with overseas partners being expected to adhere to these procedures and arrangements. In discussion with staff at the University the team formed the view that some of the procedures appeared to have been developed in reaction to issues encountered rather than representing a fully developed policy. The University may wish to reflect upon its current procedures to ensure that they adequately cover the specific requirements and challenges of overseas collaborative provision.
15 The validation event was conducted in accordance with the normal University procedures and concentrated on programme issues. The approval of the institution and the approval of the programme were not considered separately, or recognised in separate agreements, as recommended in QAA's Code of practice. In view of the recommendations expressed in the precepts of the Code, the University may wish to reflect on the sufficiency of its current procedures for dealing with overseas partners.
16 The University's Commentary described an internal consultative process regarding QAA's Code of practice which took place in 1999-2000. The outcome of the consultation was that the University considered that it met the expectations of the Code, and in addition the Faculty of Informatics also 'confirmed that its arrangements met the expectations of the Code'. In the view of the audit team, the interpretation that the University had placed on aspects of the Code was not always consistent with the expectations of the Code (see above, paragraph 15 and below paragraphs 18, 21, 22, 38 and 39). In the light of matters discussed elsewhere in this report, the University may wish to further reassure itself that all aspects of the Code have been considered and implemented appropriately.
17 The Commentary made no specific statement about how representative
this link was of its overseas collaborative arrangements, but in view of
the small amount of the University's overseas collaborative provision,
the audit team concluded that the link with HKCT was probably representative.
The extension of the link to SCAU appeared to the team to be a new type
of arrangement, and that element may not be considered representative.
Formal arrangements
18 Two written agreements exist between the University and the College, the current versions having been signed in September and October 2000. The agreements are described as recognition agreements, and deal with the MSc in Information Systems Management and the BSc (Honours) in Computing Science respectively. Whilst the agreements require the College to conform to the official course document and associated quality assurance and administrative procedures of the University, in the view of the audit team there is insufficient detail in the agreements about the respective responsibilities for each partner. The written agreements do not contain a firm commitment to students in the event of early termination, but merely state that students 'shall, with the agreement of the College, have the opportunity to complete the said course for the award'. This is not in line with the expectations expressed in the precepts of the Code, and the University may wish to review its agreements to ensure that the interests of students engaged on one of its programmes are adequately safeguarded.
19 The College of Science at SCAU provides a base for the delivery of the programme in South China. Approval has been granted by the Bureau of Higher Education, Guangdong Province, for the programme to operate in collaboration with HKCTI and the University of Ulster. The formal agreement is between HKCTI and SCAU, and it makes clear that the responsibility for the teaching and administration programme rests with HKCTI. According to the Operation Manual, SCAU is responsible for the recruitment of students, the provision of teaching resources and for assistance in the management of students to 'ensure smooth operation of teaching'.
20 The Recognition Agreement between the University and the College does not address responsibility for ensuring accuracy of publicity materials. The Commentary states that 'the institution (in this case the College) must ensure that course publicity material and other information provided to prospective and current students are accurate', and goes on to say that 'statements regarding the University must be submitted to the University for approval'. An example of course publicity was provided to the audit team. From discussions with staff at the University, it appeared to the team that the University had no formal central approval procedure for such material, and that it relied mainly on the University Assessor (the Assessor) (see below, paragraph 23) to check publicity informally during visits. The team was told that the Assessor might refer material to other staff informally. The team confirmed in Hong Kong that the College sends copies of all publicity material, including web pages, to the Assessor for approval.
21 Award certificates are issued by the University and satisfy the expectations
of QAA's Code of practice. Transcripts are issued by the College
following meetings of the Board of Examiners, and follow the University's
standard format. The issuing of transcripts by the partner institution
does not adhere to the practice recommended in the Code.
Responsibility for quality and standards
22 There is no clear statement in the Recognition Agreement between the University and the College identifying their respective responsibilities for the quality of provision and the standards of awards. The Agreement states that 'at all times the students are students of the College and not the University'. The audit team inferred from this statement that the University considers the College to have primary responsibility for the learning opportunities and support available to students on the course. The extent of this responsibility carried by the College is unclear. The University's insistence in the Agreement that students on the course at HKCT and SCAU are not students of the University, an insistence that contributed to the decision to transfer the collaboration from City University to the College, has implications for the limitation of its responsibilities toward its students. The team formed the view that the clarity of responsibilities of the University and its partner for assuring quality and standards in this collaborative arrangement fall short of the expectations of QAA's Code of practice. In its position as the awarding body, the University will wish to review its responsibilities to students in partner institutions who are following programmes of study that lead to an award of the University.
Quality of learning opportunities and student support
Liaison and administration
23 The University's main liaison with HKCT is through a member of University staff nominated as the University Assessor for the link. The Commentary explained that 'the Assessor has a key role in liaising between the institution and the University, providing advice both in relation to academic aspects of the course and the University's procedures, and in monitoring standards'. The Assessor's terms of reference include: monitoring entry standards; quality of staff appointments; quality of delivery and physical resources; arrangements for coursework; written examinations; and marking and moderation. During visits to the College, the Assessor may sit-in on lectures to evaluate the effectiveness of teaching delivery and provide advice on the University's expectations. The Assessor is expected to attend meetings of the College's Course Committee and the Board of Examiners.
24 The University places considerable responsibility and reliance on the Assessor, who has delegated authority for many activities, such as approval of non-standard entry and publicity material. The Assessor makes regular visits to the College, and HKCT staff reported to the audit team that they expected the Assessor to keep them up-to-date with developments in the programme at the University. The Commentary explained that 'the Assessor acts as the link between the course and the Faculty's Learning and Teaching Committee, ensuring the two-way flow of relevant information'. The Assessor does more than act as a link between the partner and formal University-based monitoring processes, and the team considered that there was some risk that the level of responsibility and authority delegated to the Assessor could leave gaps in University-level knowledge of the quality of provision in the collaborative link. Overall, the team formed the view that the liaison in relation to the support provided by the Assessor to the College was effective and reliable.
25 The University has issued guidelines to the College on its requirements
for administrative arrangements such as the conduct of examinations, control
of student progression, awards, fees and maintenance of student records.
The University's approval has to be sought by the College for the appointment
of staff, admission of students, the setting of examination papers, confirmation
of student results and any modifications to the programme. During discussions
with University staff, the audit team was told that, as this was a partnership
with a reliable institution, the University could assume that regulations
would be followed by the College, and that the Assessor's role was to ensure
that administrative arrangements in the collaborative link were adequate.
The University may wish to consider whether this approach enables it to
be fully confident that all its administrative requirements are routinely
followed.
Monitoring of provision
26 The Assessor receives minutes from meetings of the College's Course Committee, the Staff/Student Consultative Committee, and produces an annual report on the link which goes to the Faculty of Informatics. These annual reports feed into the Faculty's annual review of courses, and the report of the Faculty review is considered by the University-level Quality Assurance and Enhancement Committee. The audit team considered that the Assessor's annual reports to the Faculty contained a limited amount of detail in their evaluation of the operation of the course, which could result in low visibility of overseas provision above the level of the Faculty. The team was informed that the University had reviewed its quality assurance procedures following the report of the HEQC 1995 audit of collaborative provision, and that revised procedures would be agreed during the 2000-01 session. These would include a re-consideration of the route followed by assessors' annual reports so that collaborative provision would receive specific consideration at the Quality Assurance and Enhancement Committee. The team would encourage the University to progress this issue in order to ensure that overseas collaboration is monitored effectively at University level.
27 Questionnaires are used at the end of each module to gather feedback
from students on the operation of the modules. The outcomes of these, together
with the outcomes of meetings of the Staff/Student Consultative Committee,
feed into the Course Committee at the College. Students on the course are
represented on the Course Committee, and the representatives who met the
audit team expressed the view that course managers were responsive to their
concerns.
Curriculum
28 The course is based on the final-year of the BSc (Honours) Computing Science programme at the University, but is not directly equivalent since it is designed for part-time study over two years. The normal requirement for entry to the course is successful completion of the Higher Diploma in Computer Studies in Hong Kong, or equivalent, and the Commentary stated that 'the University considered that the Higher Diploma corresponded to the initial stages of a degree in the subject'. The HKCT web-site sets out the nine modules, with no options, that, together with a project, make up the course. Year 1 of the course includes five taught modules. Year 2 has four taught modules plus an extended project. Each taught module consists of 30 contact hours, made up of 21 hours of lectures and nine hours of tutorials. All teaching at HKCT is at weekends.
29 From its study of the teaching materials for the course, the audit team formed the view that the content of the modules was appropriate although, in places, the specification was rather dated; some references had been updated in handouts but not in the course scheme. In the minutes of a recent meeting of the Staff/Student Consultative Committee, students had raised concerns about the basic level of some of the content and the incomplete coverage in some areas. These matters appeared not to have been addressed directly by the College, whose staff pointed out to the team that responsibility for updating the programme lay with the University. The team was advised that a new version of the course, based upon the current course at the University, would commence in 2001. The new programme will contain more relevant and topical issues, including a new module in Management Information Systems. The team acknowledged that the new syllabus may resolve the current concerns about the curriculum.
30 From its meetings with staff and students of the College, the audit team learnt that lecturers for the taught modules provided the students with lecture notes of generally good quality, which they had developed by drawing on the teaching materials available on the University's web site. Staff of the College are in the process of converting these materials into web-based materials to support the operation of the course at SCAU. The team saw evidence of careful planning of both lecture and tutorial sessions, and students reported that some of their lecturers have set up their own web sites of relevant, additional reading materials. Lecturers encourage students to read around the subject, and the team was interested to note in HKCTI publicity material about the course a quote from a second-year student advising fellow students to 'read some reference books apart from the textbook materials and notes'.
31 The Agreement between HKCTI and SCAU specifies that the course content
and syllabus, and the number of contact hours, will be identical for both
locations. In order to accommodate the delivery of the course by part-time
HKCT staff into South China, each semester has been extended to 20 weeks
and the modules have been delivered in alternating three-week blocks. At
HKCT the block comprises two weekends of lectures followed by a weekend
of tutorials. At SCAU, the lectures are delivered on a single visit with
tutorials two weeks later. Delivery of the course at SCAU is in the early
stages of development, with a small cohort of students. The University
will wish to reassure itself that the logistical arrangements put in place
by the College are sufficient to ensure that students have appropriate
teaching and learning support.
Learning resources
32 The adequacy of facilities and resources to support the course, in both Hong Kong and China, were checked by the University as part of the validation process. Although there were no academic journals available in the current library provision at the College, the students explained to the audit team that they could access appropriate materials from local universities if necessary. The team noted that the position in respect of journals would be improved shortly when the University made available access to its electronic library resources.
33 All the students who met the audit team at the College had their own
computers at home for coursework. The team noted that little use was made
of advanced software tools (such as CASE, DBMS or GKS) within the taught
modules. The University may wish to reflect on the software requirements
in relation to the new course being introduced in September 2001, and on
means of ensuring the availability of journals to support study of the
new syllabus.
Projects
34 The project in the second year is a major element of the programme.
Project proposals are developed in Hong Kong and a list of titles is sent
to the University. From the examples available to the audit team in Hong
Kong, projects appeared to be generally well-managed by HKCT, involved
well-qualified supervisors, and had topics appropriate to honours-level
study. The document provided to the team about the link with SCAU specified
that projects will be supervised by HKCT staff.
Staffing and staff development
35 The College is responsible for the selection and appointment of all teaching staff. Appointment criteria include research achievement, and the Assessor provides guidance on the staff requirements for this level 3 course. The audit team considered that appropriate procedures were in place for the recruitment, induction and observation of part-time teaching staff by the Course Director at the College. All staff CVs were approved by the University at initial validation, and the College confirmed to the team that it sends the CVs of proposed new appointments to the University for comment before appointments are confirmed.
36 Staff development has been provided by the University for both teaching and administrative staff, notably a week's programme on course-related procedures and student materials in March 2000 for five members of College staff at the University, and a similar programme for staff from SCAU. Since then further sessions have been provided in Hong Kong on an opportunity basis. In its Commentary, the University expressed its intention that 'colleagues at HKCT receive the same information as that available to staff at the University', and identified learning outcomes, assessment procedures, student support and quality enhancement as topics for further staff development activity. The University Assessor may also provide some additional staff development activity during visits to HKCT. Staff at the College expressed to the audit team a wish to see some exchange of academic staff with the University to strengthen the course and develop a better mutual understanding of approaches to teaching and learning.
37 The audit team was interested to learn about the staffing arrangements
for the delivery of the programme at SCAU. It was made clear to the team
that HKCT staff have full responsibility for the delivery of lectures and
tutorials, but SCAU support staff provide back-up for practical sessions
and may advise on assignments in-between the visits of staff from HKCT.
Lecturing staff at SCAU are experienced and well-qualified, and most are
holders of master's degrees in Computing Science. Information provided
by SCAU indicated that some of its lecturers would attend lectures and
tutorials at HKCT during the current academic year. The team learnt from
its discussions with senior College staff that, with the increased number
of students on the programme next year, there is an expectation that SCAU
staff will have an input into academic support for the course. The contractual
arrangements for the contribution of SCAU staff to the programme are entirely
a matter for the College, or more specifically with HKCTI since the Agreement
is with that legal entity of the College, with the University having no
involvement in the identification or approval of staff. The University
has approved the delivery at SCAU on the understanding that HKCT staff
will have sole responsibility for teaching the course. It may wish to reassure
itself that this arrangement is sustainable for the larger number of students
that will participate in the programme at SCAU in the near future.
Student information and support
38 The audit team gained the view that the quality of information available to students about all aspects of the course was high, and had largely been derived from the University web site. Nevertheless, the team was uncertain of the route by which students would be made aware of complaints and appeals procedures. A procedure described in the Approved Course document whereby the student could ask for a review of the decision of the exam board through the Chair of the Board of Examiners was discussed with staff at the University. In this instance the chair of the Board of Examiners is the Course Director who is a member of the College staff. It seemed that the University did not have a formal role in ensuring that the request for a review was heard at this stage, neither did it have any formal role in any further stages of the review. The view expressed to the team was that students were the responsibility of the College, but the University 'would seek to look after them'. From the discussions with staff at HKCT, the team learnt that no students had so far pursued a review with the University. Whilst the College accepts responsibility for the students, and would normally deal with any appeals on an informal basis in the first instance, seeking advice from the Assessor if appropriate. The right to appeal to the University, as the awarding body with the responsibility for the standards of the award, is not specified in the course documentation. This aspect of the collaborative arrangement does not meet the expectations of QAA's Code of practice. Student complaints are handled by the College, again usually on an informal basis.
Assurance of the standards of awards
39 The way in which the University satisfies itself that its expectations for standards are being met is not set out specifically in the formal Recognition Agreement. The University's Policy Statement on collaborative links states 'the University must ensure that the academic standards of a particular course and the resources available to deliver it meet the University's criteria for the award in question', but this expectation is not confirmed in the legal Agreement. QAA's Code of practice on collaborative provision recommends that the awarding body makes a clear statement as to the respective responsibilities for quality assurance and control in relation to the award. The recognition agreements do not make a clear statement concerning the University's responsibility for standards.
40 The Commentary did not expand upon the mechanisms used by the
University to check that its expectations for standards were being met
in practice, although it implied that the processes of course evaluation
and review, and external examining and moderation were expected to perform
this function. Reports on the activities of the external examiner and to
a limited extent the Assessor, appear to be the only way in which the University
can check its expectations for standards are being met. The audit team's
view was that by not involving itself more directly in the marking and
moderation of assignments, examinations and projects the University was
missing both a valuable opportunity to monitor and check on the maintenance
of standards and to develop the familiarity of College staff with the academic
standards expected at honours-level.
Entry requirements
41 The Approved Course document states that 'applicants must normally hold a Higher Diploma in Computing from Hong Kong or a Higher Diploma in Computing from People's Republic of China which is deemed to be equivalent'. In the HKCTI publicity material supplied to the audit team the entry requirement was stated as 'holders of recognised Higher Diploma in Computing, Electronic Engineering, or related discipline; or mature entrants who possess the following qualifications may also be admitted subject to final approval: Higher Certificate or Diploma holder in Computing related discipline, with 3-5 years relevant working experience'. Although Course Committee minutes indicated that over three-quarters of entrants did have a Higher Diploma in a Computer related subject, the University will wish to investigate why the entry requirements used by HKCTI are broader than those stated in the Approved Course document.
42 The College admits students on the basis of guidelines provided by the
University. Monitoring of entry qualifications is a responsibility of the
Assessor, and the cases of applicants who do not conform with the stated
entry requirements are referred to the Assessor for confirmation. In discussion
with staff at the University, the audit team learnt that transcripts of
admissions were sent to the University, and that the International Office
might check them. These procedures were confirmed by staff of the College.
Assessment of students
43 The University's Commentary was largely silent on this aspect of the collaboration, and the audit team explored the assessment process during its meetings at the University. Examinations are set by College staff. The Assessor has a role in the approval of examination papers, marking schemes and assignments, and acts as the link with the University's external examiner. Examinations, coursework, and projects are marked by the College staff who set the examinations, and samples of work are sent separately to the Assessor and the external examiner for moderation. The results of assessment are considered at a Board of Examiners. This Board is chaired by the Course Director, who is a member of staff of the College, and is attended by the College module leaders, the external examiner and the Assessor. The chairing of the Board of Examiners is a heavy responsibility for the University to put on a member of the College. The team considers that it would be more appropriate for a suitably qualified member of University staff, other than the Assessor, to chair a meeting that acts on behalf of the University in its role as the awarding body.
44 The University has taken steps to ensure that the administration of the examination process at the College is secure. HKCT staff control and invigilate their own examinations, including those delivered at SCAU. Administrative staff check the completeness and accuracy of marking. The document provided by SCAU on the arrangements for its link with HKCTI makes clear that the examinations taken at SCAU will be identical to those at the College, and will be marked by HKCT staff. However, the Operation Manual indicated that examinations at SCAU will take place one week after they have been sat by students at HKCT. In the interest of security of assessment, this arrangement would appear to require examination papers for students at SCAU to be different from those sat by students at HKCT. The University will wish to check that the arrangements proposed for examining at SCAU do not present problems of security of assessment.
45 The documentary information provided to the audit team on the link between HKCTI and SCAU indicated that the Board of Examiners would consolidate the results of assessment of students on the level 3 course at both campuses into one annual report. It was not clear to the team if this referred to the Board of Examiners at HKCT or the Board of Examiners at SCAU for the diploma stages. The latter does not include the Assessor as a member. The University will wish to clarify the position, and, in so doing, assure itself that it is properly represented at any Board of Examiners that is empowered to make recommendations about its awards.
46 A wide range of assignments is used, including group projects, analysis of journal papers and practical projects. In the view of the audit team, the examples that it saw were appropriate to an honours degree in Computing Science. Students reported that they received good feedback on assignments. Examination papers seen by the team, though in some cases containing fragmented and leading questions, were considered to be at an appropriate standard for level 3 and reflected the syllabus content.
47 Project reports are double-marked by HKCT staff. In the examples of
project reports seen by the audit team at the College, the team considered
that the reports demonstrated only limited attention to the analysis and
critical evaluation that would be expected at honours level. The team felt
that there would be merit in making the assessment criteria of an honours-level
project report more explicit to students in the project handbook, and supporting
this with more explicit comments by supervisors on expectations for a project
at this level. The Commentary acknowledged that the external examiner
has suggested that attention should be given to the project module, and
to the development of project-related student skills. The Assessor has
subsequently addressed these matters in a staff development workshop at
the College, and a document for staff and students on project supervision
has been developed. The external examiner is invited to moderate project
work, but project reports of HKCT students are not seen by University specialist
staff. For the purpose both of securing standards and of supporting the
development of College staff to deliver those standards, the University
might wish to consider the merit of involving University specialist staff
more directly in the appraisal of the projects of HKCT and, later, SCAU
students.
External examiners
48 The University appoints an external examiner from the UK for the course
at HKCT. The current external examiner was the external examiner for the
course when it was delivered by HKPU. The external examiner is not required
to be an external examiner of the University's own BSc (Honours) Computing
Science programme. The Commentary stated that 'external examiners
are appointed in accordance with normal University processes. Their reports
are circulated to the Assessor, Dean of Faculty and appropriate staff in
the institution for action and response. The reports are included in the
annual course review process'. Staff at the College confirmed to the audit
team that they saw the reports of the external examiner, and made a response
where appropriate. Administrative and reporting aspects of the external
examining process appeared to be effective and sound, although the team
considered that the University was asking a great deal of a single external
examiner to cover the breadth of content of the course. The team saw reports
of the external examiner and noted concerns expressed in successive reports
on speed of response to the examiner's comments on projects and course
content. The University, as the awarding body, will wish to ensure that
responses to the reports of the external examiner are acted upon in a timely
way and are checked at Faculty and University level for completeness and
effectiveness.
Language of instruction
49 The University's Policy Statement for collaborative links states that 'the University expects the language of instruction and assessment to be in English'. The Approved Course document does not specify this. The audit team noted that a recent addition to the Policy Statement stated that 'where necessary, overseas institutions are responsible for the provision of translations of documents into English and the cost of the verification of the accuracy of such translation by the University'. The team noted that the SCAU document on its arrangements with HKCTI referred to the need for increased English tuition for students in the diploma stage, and demonstration of appropriate proficiency in English before entering the level 3 course. The Commentary explained that discussion of the proposed link with SCAU within the University 'centred on the assessment of competency in English language of the students', and that resolution of the issue involved visits to HKCT and SCAU by a specialist in language teaching. The University is clearly aware of the challenge posed by the language dimension of the extension of the level 3 course into South China. It will, no doubt, continue to monitor the delivery of the course at SCAU to satisfy itself that English remains the language of instruction and assessment, and that in this respect students at SCAU receive a learning experience comparable with that at HKCT.
Conclusions
50 The University of Ulster has approved the award of a BSc (Honours) in Computing
Science for successful completion of a course delivered by the Hong Kong College
of Technology (the College) at its base in Hong Kong and at its outcentre in
South China Agricultural University (SCAU). The course is a two year, part-time
level 3 conversion programme for holders of the Higher Diploma in Computing or
related subjects.
51 The College has set up Hong Kong College of Technology International as a separate legal entity for the management and operation of overseas programmes. The formal agreement for delivery of the course in Hong Kong is between the University and the College. The formal agreement for delivery of the course in South China is between SCAU and HKCTI. HKCTI has responsibility for the teaching and administration of the course at SCAU with staff travelling regularly from Hong Kong. Oversight of the course at SCAU by the University is through its existing links with the College; there is no direct relationship between the University and SCAU.
52 The University prepared for this audit a Commentary which provided a limited view of the partnership link. The Commentary stated that the University had reviewed its practice with regard to the section of QAA's Code of practice on collaborative provision and had expressed itself generally satisfied that it is compliant with the Code's precepts. The report of this audit identifies areas where the University's approach to collaborative arrangements does not yet adhere to the recommendations of the Code. The Commentary made no statement of the extent to which this partnership link was representative of its arrangements for overseas collaborative provision.
53 The level 3 course at the College has been developed specifically to meet the requirements of students studying part-time. It is not a franchise of a programme currently operating at the University, although it contains material drawn from the final year of the University's BSc (Honours) Computing Science programme. The College takes the view that the programme belongs to the University of Ulster, and seeks approval from the University for any adaptations or amendments. The University defines the students registered for the award as students of the College and not of the University. This definition appears to limit the responsibility of the University toward these students. Consideration needs to be given to the consequent limitation of the right of these students to make direct representation to the University as the awarding body.
54 The evidence provided by the College, including examples of students' work, provided confidence that the academic standards of this level 3 course, delivered in Hong Kong, are broadly comparable with those of equivalent programmes in the UK. Project work is appropriately supervised, although there is scope for improvement in the advice given to staff and students at the College about the expectations of honours-level project work.
55 The University exercises its responsibility for the oversight of standards through a University Assessor, who has day-to-day contact with the College and makes regular visits to Hong Kong. The quality of the teaching and learning experience of students rests with the College, monitored by the Assessor. The curriculum is appropriate to the level of the award, and an updated and revised syllabus will be implemented for the academic year 2001-02. The findings of this audit suggest that there can be confidence in the quality of learning opportunities and support offered to students in Hong Kong by the College.
56 There can be less confidence that the University has effective oversight of the arrangements for students at SCAU. By extending the link with the College to an outcentre of the College in South China, the University has introduced an additional set of circumstances which may influence the success of the programme. The arrangement between SCAU and HKCTI means that the operation of the course at SCAU is one step further removed from the oversight of the University, giving cause to doubt that the University is in a position to monitor effectively the experience of those students who receive its award from their study in South China. Consideration should be given to strengthening the University's oversight of delivery at its partner's 'outcentre' at SCAU to ensure that the quality of provision in this extended relationship fully meets the University's expectations.
57 The indirect relationship with SCAU presents a challenge to the University in terms of the security of standards. It is too early to make any comment on the standards of the award, since the first students on the course at SCAU will not complete until 2002. The demands upon students for rapid progress in English language competence present an additional challenge which could influence the quality of the student experience and academic standards overall. The University needs to be able to assure itself that its responsibility as the awarding body for the standard of the award for the course delivered in South China is adequately safeguarded.
58 The operation in South China is only in its first year, and it is too early to assess whether the logistical arrangements for delivery by part-time staff of the College allow for an equivalent experience to that of students in Hong Kong. The current cohort at SCAU is small and manageable with the present arrangements. A significantly larger group of students at SCAU are currently following the Diploma programme, many of whom will wish to proceed to the degree programme next year. The findings of this audit support only limited confidence that the University had made adequate preparation to ensure that the programme can be 'scaled-up' in this fashion.
Comments*
Comments on the audit report supplied by the University of Ulster
From 2001-02 the University has put into place arrangements to annually monitor its collaborative provision centrally via its Quality Assurance and Enhancement Committee. It will undertake an overall review of the courses provided by institutions rather than on a course by course basis. In this way the quality and standards of franchised and validated programmes may be rigorously monitored at the appropriate level, and any institutional issues identified and acted upon. This arrangement will apply to overseas partnerships as well as those in local institutions.
The role of University assessors including the format and use of assessor's reports will be the subject of a themed audit in 2001-02 undertaken by QAEC.
The University has begun exploratory meetings with the senior management at SCAU with a view to entering into a direct relationship with the institution.
The Centre of English Language Teaching (CELT) at the University of Ulster has now established its provision directly at SCAU and the first cohort of students has successfully completed the English Language programme to the required standard.
The University intends to scrupulously follow its operating agreement with HKCT and until such times another arrangement with SCAU may be in place the operating agreement will be strictly enforced.
*As supplied by the University of Ulster
Top