University of Wales, Newport overseas collaborative audit
1 This report considers a collaborative arrangement between the University of Wales, Newport (Newport) and the Hainan Overseas Education Centre (HOEC) for the delivery of an MBA programme.
Introduction
2 The Quality Assurance Agency for Higher Education (QAA) is a United Kingdom (UK) organisation which seeks to promote public confidence that the quality of provision and standards of awards in higher education are being safeguarded. It provides public information about quality and standards in higher education to meet the needs of students, employers and funders of higher education. It does this mainly through a peer review process of audits and reviews. These are conducted by teams of auditors and reviewers comprising academic staff from higher or further education institutions, but with some members drawn, where appropriate, from industry and the professions. The most recent institutional review of Newport was conducted by QAA in November 2004.
3 One of QAA's activities is to carry out quality audits of collaborative links between UK higher education institutions (HEIs) and their partner organisations in other countries. In the spring and early summer of 2006, QAA conducted audits of selected partnership links between UK higher education institutions and institutions in the People's Republic of China (PRC). The purpose of these audits was to provide information on the way in which the UK institutions were maintaining academic standards and the quality of education in their partnerships. The reports on the individual audits will be used in the preparation of an overview report on the collaborative arrangements for the management of standards and quality of UK higher education provision in mainland China.
The audit process for overseas collaborative links
4 In July 2005, QAA invited all UK higher education institutions to provide information on their collaborative partnerships in PRC. On the basis of the information returned on the nature and scale of the links, QAA selected for audit visits 10 UK institutions with links in mainland China. Each of the selected institutions produced a commentary describing the way in which the link operated, and commenting on the effectiveness of the means by which it assured quality and standards. In addition, each institution was asked, as part of its commentary, to make reference to the extent to which the link was representative of its procedures and practice in all its overseas collaborative activity. Institutions were also invited in their commentaries to make reference to the ways in which their arrangements met the expectations of the Code of practice for the assurance of academic quality and standards in higher education (Code of practice), particularly the section on Collaborative provision and flexible and distributed learning (including e-learning), published by QAA in 2004.
5 In spring 2006, audit visits were made to each of the selected UK institutions to discuss its arrangements in the light of its commentary. In April-May 2006, one of three separate audit teams (based respectively in Beijing, Shanghai and Guangdong province) visited the partner institutions in PRC to gain further insight into the experience of students and staff, and to supplement the view formed by the team from the institutions' commentaries and from the UK visits. During the visits to institutions in PRC, discussions were conducted with key members of staff and with students. The full audit team conducting audits of institutions with collaborative links in the Guangdong province comprised Professor J Bailey, Dr D Furneaux, Dr R Haggerty, Ms M Heycock, and Professor G Roberts (auditors) and Ms C Smith (audit secretary). The audit was coordinated for QAA by Mrs S Patterson, Assistant Director, Reviews Group. QAA is particularly grateful to the UK institutions and their partners in PRC for the willing cooperation they provided to the team.
6 5 In PRC responsibility for higher education planning and policy resides with the Ministry of Education, which is the State Council's executive body for education. It also has direct responsibility for a number of universities and for the evaluation of bachelor and master's level programmes. However, responsibility for the majority of provision, including vocational education, is devolved to the relevant regional and municipal authorities. China has a comprehensive set of laws covering the provision of higher education, notably the current Higher Education law, 1998. Of specific relevance to collaborative arrangements, are the Regulations on Chinese-Foreign Cooperation in Running Schools, 2003, which apply to higher education institutions and cover matters relating to the management of overseas partnerships and the level of student fees. As part of the continuing process of modernisation of higher education, two state programmes have been introduced: Project 211, which aims to create 100 first-class Chinese universities and a number of key fields of study for the twenty-first century; and Project 985, which aims to develop world-famous research universities. Both these projects channel funding selectively to designated universities. China has been progressively developing its quality assurance system for higher education since 1985, when the evaluation of institutions was first implemented. Current arrangements entail the evaluation of tertiary colleges by provincial education authorities, according to a national plan introduced by the Ministry of Education in 2003; the evaluation of bachelor degree programmes by the Ministry's Higher Education Evaluation Centre, established in 2004; and the evaluation of graduate education (master's and doctoral programmes) by the Academic Degree Committee of the State Council. Further information on higher education in China is contained in the overview report.
The background to the collaborative link
General background to the link
7 The linkage between HOEC and Newport was established in 1996 on transfer of the provision from another UK HEI. All teaching is conducted at HOEC or its outreach centres (see paragraphs 24 and 25). At the time of the audit the most recent cohort at Hainan (2004-05) numbered 31 students and there were 39 students registered at each of the two existing outreach centres.
8 The programme is delivered in three parts, the first two being a Certificate and a Diploma in Business Administration with the final part leading to the award of the MBA. The certificate and diploma stages are taught and assessed in Chinese by HOEC staff, the MBA stage is delivered in English by Newport staff with simultaneous translation. The assessment for the final stage is set in English by Newport staff, is then translated into Chinese and is undertaken in Chinese (see paragraphs 63-68).
9 HOEC, located in Haikou in Hainan province, operates from premises on the campus of Hainan University. The organisation was established in 1994 as the Hainan University External Programme Centre (HUEPC), being a joint venture between Hainan University International Division and the current Executive Director of HOEC who also leads an education company registered in Malaysia.
10 In the Commentary Newport stated that its strategy for the development of overseas activities was 'firmly embedded' within its Strategic Plan (2005/06-2009/10). The Commentary went on to state that the current strategy was to concentrate on developing close links with major overseas partners and on developing articulation arrangements to 'encourage secure recruitment streams'. At the time of the audit Newport had an extensive range of collaborative partnerships involving about 1,300 students. The MBA programme delivered at HOEC was also offered through collaborative arrangements with two other partnerships in China but these were closing in line with Newport's revised strategic direction for collaborative provision.
11 The Commentary stated that the quality assurance arrangements for the HOEC link were representative of procedures and practices in all Newport's overseas collaborative partnerships, 'with the exception of the particular processes associated with delivery and assessment through the medium of Mandarin'.
The UK institution's approach to overseas collaborative provision
12 Newport's undergraduate and master's provision operates under the auspices of the University of Wales (the University) which is the Chartered Degree Awarding Body. Under the University's Academic Framework, Newport carries the major responsibility for the quality assurance of programmes delivered overseas. In accordance with the devolution of further powers from the University to its constituent institutions in 2005, Newport also has formal responsibility for academic standards in its provision.
13 The Commentary emphasised that Newport acknowledged its responsibilities for the academic standards of awards and for ensuring that the quality of learning opportunities offered through collaborative arrangements was adequate to enable a student to achieve the academic standard required for the award. The Commentary was clear that formal responsibility for the quality and standards of academic provision rested with Newport's Academic Board. In practice this responsibility is exercised through the Academic Standards Committee (ASC) and the activities of the schools.
14 Newport's policy and procedural requirements for the management of its overseas collaborative provision are set out in two main documents: Mandatory Procedure for Approval, Monitoring and Development of Schemes of Study, including Collaborative delivery and Flexible Distributed Learning and the Franchise Operational Guidelines. The Commentary described a 'carefully staged approach' to the development of overseas international activities, identifying key committees and sections of the institution involved in the management, approval and ongoing operation of collaborative arrangements overseas.
15 The Overseas Collaborative Group (OCG) is responsible for the management and development of all overseas activity and reports to the ASC and the Academic Development and Planning Committee (ADPC). The operational management of collaborative links is undertaken in schools with designated link staff providing day to day support to staff delivering the programme (see paragraph 40). The Quality Assurance Unit within the Academic Registry and the International Office provide central administrative support for the operation of collaborative linkages.
16 As noted above, the Commentary made reference to 'particular processes associated with delivery and assessment through the medium of Mandarin' and provided details of the operational aspects of the particular mode of delivery at HOEC. The audit team's scrutiny of documents and discussion with staff at Newport and at HOEC did not identify any formal policy statement or documented procedures for the delivery and assessment of programmes to cover the use of Chinese as the language of study and assessment. Further detail of the operational processes for delivery and assessment in Chinese may be found at paragraphs 65 to 68 and 80.
17 From its reading of documentation and discussion with staff at Newport and HOEC, the audit team concluded that the policies and procedures for the management of overseas collaborative arrangements were consonant with the relevant precepts of the Code of practice and were operating as intended in respect of the collaborative arrangement at HOEC. Newport may wish to consider whether codification and documentation of existing practice in relation to the delivery and assessment in Chinese might provide additional assurance of quality and standards and consistency of approach in the management of the collaborative provision.
Public information, publicity and promotional activity
18 Memoranda of agreement governing the operation of individual linkages prohibit the publication of any material relating to Newport programmes without prior approval. The Commentary stated that Newport was 'keenly aware' of the guidance of the Code of practice in this area and had 'taken steps to ensure that publicity relating to collaborative provision accurately represent[ed] that provision'. The Franchise Operational Guidelines require that all publicity material making reference to Newport be provided to the International Office for approval prior to publication. Where amendments are necessary Newport requires a revised draft of the material to be submitted for final approval.
19 The Commentary noted that 'constant vigilance [had to] be maintained regarding publication of materials that ha[d] not been given prior agreement'. In its review of documentation and discussion with staff at Newport and at HOEC, the audit team found clear evidence that procedures for the control of publicity material were well-understood and operating as intended. The team reviewed documentation relating to a breach of Newport's requirements by another partner institution and noted that prompt action was taken in redress. The team concluded that Newport was exercising effective control over the accuracy of publicity material in accordance with the relevant precepts of the Code of practice.
20 The Commentary stated that Newport maintained an 'up-to-date, authoritative and publicly available record' of partnerships and collaborative arrangements in accordance with the advice of the Code of practice. The audit team was provided with a list of overseas collaborative links but this did not provide any detail about the category of provision, contractual status or student numbers. The institution may wish to consider whether the inclusion of such data in its record of collaborative arrangements provision might provide for additional public confidence in the openness and completeness of information in its collaborative provision.
Formal arrangements for establishing the link
Selecting and approving the partner institution
21 The Commentary set out the procedures for the selection and approval of partner organisations, including the requirements for compatibility in educational objectives and for due diligence enquiries. The processes are summarised for potential partners in a 'Process for Setting Up Overseas Partnerships' document. The first step in the process is for the OCG to consider information from the school about the proposed partnership, including a financial analysis. The Commentary emphasised the importance of OCG in both this initial assessment and the ongoing monitoring of the financial viability of collaborative arrangements.
22 Following initial approval from the OCG, the Dean of International Affairs or nominee undertakes an investigatory visit to the potential partner and produces a 'Stage One Report' that appraises the proposal against Newport's requirements, taking into account human and physical resources, quality assurance and financial viability. In cases where Newport has limited in-country experience advice is also sought from the British Council. The OCG considers the Stage One Report and makes recommendations to the ADPC on whether the proposal should proceed to validation. In the event of a positive recommendation a Memorandum of Understanding, being an agreement in principle to cooperate, is signed by both institutions. Further detail of the validation process may be found at paragraphs 30 to 34.
23 The arrangement with HOEC was developed prior to the devolution of responsibilities from the University of Wales and therefore the approval process differed from Newport's current in-house procedures. The Commentary provided a detailed account of the approach followed, which included particular consideration of provision for students registered with HOEC's previous validating partner. The process followed the requirements of the University of Wales' Validation Board and included representation from the University of Wales on the validation panel.
24 In response to a wish to increase recruitment to the programmes, HOEC identified the possibility of delivering the final stage of the programme at outreach centres. At the time of the audit there were two such sites in operation, one in Nanjing and one in Beijing, and plans were in train for an additional centre at Changsha. In meetings with senior staff at HOEC the audit team was informed that Newport had stipulated a limit of four for the number of outreach centres.
25 The audit team discussed the process for approval of the outreach centres with senior staff at Newport and were informed that, as delivery was by 'the same staff of the same programme', a 'fairly light touch' was applied. The proposed outreach centres were subject to visits by a representative from the school or the International Office to confirm the adequacy of the resources. The reports on the findings of the visits were considered by the OCG and the ADCP. Staff at Newport and at HOEC were firm in the view that the arrangement at the outreach centres was analogous to 'hiring a building'; staff at Newport were clearly mindful of the need to avoid serial arrangements.
26 A previous QAA report on delivery of the MBA at another institution in China advocated caution in extending the number of delivery centres and number of students involved, a view which the present audit team would endorse. The team would encourage Newport to revisit its approach to the approval of outreach centres, including consideration of the merit of requiring some internal externality in the process and the establishment of a standard format for reporting on the site visits. Newport may also wish to consider the formalisation and documentation of the requirements for the approval of outreach centres in the interests of consistency of approach and clarity of purpose.
27 As the approval of HOEC as a collaborative partner predated Newport's current procedures, the audit team was unable to comment specifically on their application to this link. Nonetheless, from its reading of documentation and discussion with staff, the team came to the view that the claim in the Commentary that the staged approach to the development of partnerships was 'rigorously applied' was justified. The team concluded that the procedures for institutional approval of collaborative partners were in accordance with the guidance in the Code of practice.
28 In 1997 the Provincial Government gave approval to the partnership. Subsequently, the report of a QAA scoping exercise in China published in 2000 expressed some concern that some master's programmes delivered in China were operating without the requisite approval from the central Chinese government. Consequently Newport requested HOEC to inform the relevant central authorities about the link, and in September 2001 a letter of approval from the provincial government was copied to the central Ministry of Education in Beijing. The audit team discussed the question of central and local government approval with staff at HOEC and learned of a requirement that Hainan University apply for formal central government approval on behalf of HOEC. The outcome of the application was not known at the time of the audit visit. The team also noted that operation of the outreach centres had been subject to approval from the authorities of the city of location. The team concluded that Newport was making every effort to comply with requirements for local and national approvals and would encourage the institution to maintain its vigilance in this respect.
29 The Commentary stated that Newport's use of agents in recruitment was 'extremely limited' as hitherto the focus of international activity had been to establish collaborative programmes overseas. With the strategic move to seek articulation rather than franchise arrangements in future developments, the policy changed subsequent to the submission of the Commentary. In meetings at Newport, staff invited to provide information on developments since the submission of the Commentary informed the audit team that the institution had expanded its network of international agents for recruitment purposes. The team was provided with a copy of the agency agreement and letters of appointment which specified the protocols for the recruitment function. Agents are not used for recruitment to the programmes at HOEC.
Programme approval
30 The Commentary stated that validation activity 'focussed explicitly on the capacity of the partnership to deliver an appropriate experience for students'. The Commentary went on to identify the need to ensure that academic standards were appropriate as a purpose of the validation event. The validation of the arrangement at HOEC was undertaken in 1998 according to the procedures in place at the time which met the requirements of the University of Wales as the awarding body. There was external representation from a subject expert with experience of both the UK and Chinese education systems, in addition to a nominee from the University of Wales Validation Board. The validation panel did not attach any conditions to approval of the franchise other than the signing of a memorandum of agreement. The operation of the programmes was subject to formal approval from the University of Wales Validation Board. The audit team reviewed the documentation from the original validation and confirmed that the process was thorough and rigorous with appropriate external input.
31 The Commentary described the course as 'one programme severally delivered'; the module specifications are therefore essentially the same as for the UK programme. The Commentary also identified a need for some contextualisation to take account of the Chinese business environment and its particular characteristics, adding that such variation could 'usually be accommodated within the existing module specification'. In meetings at Newport the audit team was informed that changes to module titles, learning outcomes or assessment required formal approval and that a change to the module of more than 50 per cent would require revalidation. The team was assured that the Quality Assurance Unit maintained an overview of the whole programme and of any local modifications to ensure that the integrity of the franchise arrangement was maintained.
32 The Commentary did not include any detail about the procedures for consideration at Newport of validation reports on collaborative provision. The report of the QAA institutional review of Newport undertaken in 2004 confirmed that a central overview of validation reports was maintained through an annual review of validation by the Quality Assurance Unit resulting in a report to ASC.
33 The Commentary reported that the MBA programme at Newport was revalidated in 2002, using the Academic Infrastructure as a reference point, and was first offered at Newport in the academic year 2002-03. Normal practice at Newport is for a revised programme to operate at home for one year before operating it at collaborative partners to allow for the necessary staff development for partner staff. The SARS [Severe Acute Respiratory Syndrome] epidemic in 2003 delayed the planned implementation of the revised programme at collaborative institutions and, in the meantime, the course team reviewed the content and structure 'in response to the needs of home and overseas markets'. The Commentary indicated that the revised programme would be rolled out to collaborative partners starting in November 2005. The revised programme was scheduled to begin at HOEC in September 2006. Staff at HOEC meeting the audit team confirmed that they had received details of the revised scheme in Chinese and that staff from Newport had visited to conduct a seminar on the new programme.
34 From scrutiny of documentation and discussion with staff, the audit team would concur with the findings of the institutional review in 2004 that the validation process 'was well-structured and was generally reflective of the expectations of the Code of practice'. The team noted that the original validation panel had highlighted the significance of the role of interpreters in the classroom and in translation and formed the view that, in the case of programmes delivered and assessed in languages other than English, it would be beneficial for the additional requirements for consideration by the validation panel to be specified and documented, advice which the present team would endorse.
Written agreements with the partner institution
35 The Commentary confirmed that all partnership arrangements were subject to a written and legally binding contract. There is a standard framework for memoranda of agreement which was drafted with legal advice. The standard memorandum includes details of 'quality procedures and processes' and 'activities, costs and fees related to the project'; in addition programme specifications and the validation report form appendices to the document.
36 The audit team was provided with the latest generic version of the memorandum of agreement which had been modified to take account of the revised version of Section 2 of the Code of practice. The team noted that, while the agreement specified provisions for termination, it was not explicit about residual obligations to students and protection of their interests. Discussion with staff established Newport's commitment to the students and the team did not doubt that, in the event of termination, the institution would act to fulfil its obligations to students registered on its collaborative programmes. Nonetheless, Newport will wish to take steps to review the form of the standard agreement to provide for greater specification of how the interests of students, including those at the outreach centres, would be protected in the event of termination of a collaborative arrangement. Overall, the generic agreement seen by the team was found to be in alignment with the relevant precepts of the Code of practice.
37 The original agreement for the linkage with HOEC was signed in 1998 for a period of five years. Since 2003, there have been four short term extensions to the agreement, the latest of which was scheduled to expire at the end of August 2006. At the time of the audit a revised memorandum was under negotiation; the audit team was given to understand that the question of applicable law in the event of a dispute was a matter of some discussion between the parties to the agreement. The team noted that the extant memorandum of agreement had not been supplemented to take account of the introduction of the outreach centres. Newport may wish to take the opportunity in renewing the agreement to include details of the rights and responsibilities of the parties in respect of the operation of the outreach centres, including any limitations on the number of centres to be established (see paragraphs 24 and 26).
38 The position with regard to the involvement of Hainan University in securing central government approval to operate has been noted (paragraph 28). Assuming a successful outcome to the application, as it revises the memorandum of agreement, Newport will wish to give consideration to the implications for its contractual arrangements of the formal role of Hainan University in the negotiation of approval for the collaborative linkage.
39 The audit team noted a condition in the current Memorandum of Agreement which stated that the students were enrolled at Newport 'for the sole and exclusive purpose' of enabling Newport 'to award a qualification authorised by the validating body' and explicitly excluded any other contractual rights, reflecting the strict legal position. In meetings with the team, senior staff were firm that students understood their position in relation to this limitation; the institution will wish to monitor the information provided to students to ensure that it is explicit in relation to their rights in relation to Newport and the formal awarding body.
Management and quality assurance of the link
40 The Newport Head of Research and Income Generation has overall responsibility for collaborative partners in China. Newport appoints a Link Coordinator for each collaborative arrangement to provide support to staff in the partner and to monitor the management and quality assurance of the linkage. In the case of HOEC day-to-day management of the programme is the responsibility of the HOEC Programme Leader who liaises with the Newport Link Coordinator. There is a designated administrator within the Newport Business School who, in addition to providing administrative support to the programme, briefs staff visiting HOEC on any revisions to operational and procedural requirements so that they can advise the relevant staff accordingly.
41 Once established, collaborative provision is subject to Newport's standard annual monitoring and evaluation (AME) procedures. AME reports are written by the HOEC staff and the Programme Leader with 'guidance from the Newport Link Coordinator'. As Newport considers the MBA to be 'one programme severally delivered', reports from collaborative partners are presented with the related academic school reports.
42 The Memorandum of Agreement stipulates that HOEC be responsible for recruitment to the course in accordance with Newport's stipulated academic standards. Newport staff are required to examine and countersign the enrolment forms. The Commentary stated that admission and creation of the student record were administered by means of the standard Newport enrolment form. Student numbers are included in annual monitoring reports but in reports seen by the audit team the data were not presented in a consistent format, inhibiting effective comparison and analysis of data.
43 The Commentary noted that the approval process for the partnership had established that there was an appropriate range of services to support learning. Students have access to library and information services, welfare and advisory services, study skills support, academic guidance and personal tutoring. The effectiveness of local arrangements for student support is monitored by the Link Coordinator, and through annual monitoring and Periodic Partnership Review.
44 With regard to student representation, the Commentary reported that each cohort identified two students to represent them at course boards and student focus groups in accordance with practice at Newport. Although this system was not precisely mirrored in practice by the arrangements at HOEC, the audit team was convinced, following meetings with the students, that there was a strong informal student support and representation system which was accessible and pragmatic, given the nature of a student body comprising mostly mature students in full-time employment.
45 The Commentary identified AME as one of the mechanisms whereby the Academic Board sought to 'satisfy itself as to the effective discharge of responsibilities for the standards and quality of the education provided'. Primary responsibility for the conduct of AME is at the programme level. AME reports are expected to take account of feedback from students, external examiners, and the programme team. The reports should also include analysis of student progression and achievement. Action plans are produced in response to the AME reports.
46 AME reports from partners are received and monitored by the Quality Assurance Unit where they are checked against specific criteria. The report is also provided to the Link Coordinator for comment before being aggregated into the school AME report. The school reports are then presented to the ASC at the same time as an overview report produced by the Pro-Vice-Chancellor (Academic). Neither of these reports identifies collaborative partnerships as a separate category for discussion. It was not clear to the audit team how AME reporting allowed the Academic Board to 'satisfy itself as to the effective discharge of responsibilities for the standards and quality of the education provided' in respect of collaborative provision.
47 The audit team viewed a sample of AME reports for the delivery of the MBA at HOEC and found them to be largely descriptive with little evaluation. There was no specific comment on matters associated with delivery and assessment in Chinese. Newport may wish to consider providing additional guidance to programme staff at partner institutions on the requirements for AME reporting, including the need to draw on an appropriate evidence base and to include self-critical analysis of the operation of the programme.
48 The report of the institutional review in 2004 advised Newport to 'review and revise the AME process with the aim of enhancing its robustness and improving its effectiveness'. In particular, the report expressed concern that 'issues raised at one level could be lost in the aggregation of reports at the next level up', a view shared by the present audit team. In discussion with senior staff at Newport the team heard that, in response to the recommendations in the institutional review report, the approach to AME which placed heavy reliance and burden on the Pro Vice-Chancellor (Academic) was under review. In conducting the review Newport may wish to consider whether there might be merit in providing for specific consideration at programme, school and institutional level of matters relating to collaborative provision, including delivery and assessment in Chinese.
Formal review
49 Periodic review of the collaborative programme is undertaken within the periodic review of the Newport based programme which includes external input. The report of the institutional review in 2004 concluded that the process was 'sound, well-constructed and met the general expectations of the Code of practice in this area'.
50 Newport also operates a process of Periodic Partnership Review (PPR) which enables all parties in each partnership to review the extent to which their respective obligations have been met and whether the partnership is operating successfully. The Commentary stressed that the focus of PPR was the partnership not the partner or the programme of study. Initially, PPR panels did not include any externality which led to comment in the report of the institutional review in 2004 that Newport was losing an 'opportunity to benchmark Newport's partnership arrangements against others in the sector'. Accordingly, in response to the report, Newport now includes an external member with experience of collaborative provision on PPR panels. On completion of the reviews both periodic review and PPR reports are submitted to the ASC for consideration.
51 The PPR process is conducted in two stages: an initial meeting at Newport and a visit to the partner institution. At the time of the audit, Newport was in the process of conducting a routine PPR with HOEC, the report of which was available to the audit team prior to its visit to HOEC. The report provided the team with additional insight into the level of critical reflection at Newport on the operation of the partnership. In meetings at HOEC senior staff confirmed to the team that they had been involved in the preparation for the review which they considered to have been of 'real benefit'. Staff at HOEC had received a copy of the report and were engaged in responding to its findings.
52 The PPR panel recommended to the Academic Board, through the ASC, that the partnership continue subject to fulfilment of two conditions and 14 recommendations. The conditions were concerned with availability to HOEC staff and students of programme specifications for the new version of the programme and provision by HOEC of training for students on the use of electronic journal databases available in the Hainan University library.
53 The recommendations covered a range of matters, some of which were subject specific and some of which were concerned with the overall operation of the partnership. The audit team noted in particular the recommendation that Newport proceed to 'further establishment of a limited number of additional outreach centres of HOEC', with the rider that such expansion be carefully planned and monitored to ensure 'quality of standards'. There were a number of recommendations associated with staff development and liaison between staff at all levels of the partnership. There was also a recommendation suggesting joint authorship by Newport Business School and HOEC of the AME report.
54 Having reviewed the documentation associated with the PPR and discussed the process with staff at both institutions, the audit team came to the view that the PPR process had been conducted with rigour and had been effective in pointing to areas for further development. The team noted that the terms of reference for the review did not include identification of good practice; Newport may wish to consider whether the inclusion of such a requirement in future PPRs might provide an opportunity for enhancement of its management of its collaborative provision. The team concurred with the findings of the review, full implementation of which would provide a sound basis for future development of the partnership.
Staffing and staff development
55 As indicated in the Commentary, the approval process confirms the qualifications and suitability of staff teaching on the course and considers the measures in place at the partner institution to 'monitor and assure the proficiency' of staff. Senior staff at HOEC confirmed that staff were employed on the basis of 'qualifications, experience, background and understanding of the Western context'. HOEC conducts the interviews and checks the validity of qualifications. Newport maintains a record of staff teaching on the programme and appointments subsequent to validation are subject to approval from the Newport Link Coordinator. In practice, the teaching team has been largely stable and there have been few changes of staff since the collaborative linkage was established.
56 Communication between HOEC staff delivering the programme and their counterparts at Newport is generally channelled through the HOEC Programme Leader and the Link Coordinator with frequent contact through email and telephone. Staff at HOEC meeting the audit team were emphatic about the importance of these clear lines of communication to the successful delivery of the programme. The staff at HOEC also spoke of some delays in response at the more senior levels, especially in discussion of potential new developments. Staff at HOEC were appreciative of visits from staff at all levels from Newport and particularly value opportunities for face-to-face discussion with staff from the most senior level in the institution.
57 The initial validation of the programme and subsequent reviews identified the importance of staff development for staff at HOEC to support delivery of the programme. Newport does not have any formal involvement in the induction of staff but would ensure that its staff met newly appointed staff during the routine visits to HOEC. Staff from Newport visit HOEC prior to each occurrence of the programme to brief staff on the teaching materials and they also conduct annual seminars about the dissertation element of the course. The Head of Research and Income Generation had visited to provide information about the revised programme. Staff at HOEC also made reference to useful discussions between HOEC staff and visiting staff from Newport about differences in teaching and learning styles between the UK and China.
58 There are frequent visits by staff from all levels at HOEC to Newport for staff development and academic study which is encouraged by HOEC. The Director of HOEC visits Newport annually to discuss the operation of the partnership and possible future developments. HOEC administrative staff have benefited from visiting Newport to gain experience of systems and procedures relevant to the operation of the programme.
59 The audit team found that the staff appointment process was robust and that Newport maintained an overview of staffing on the programme. There was clear evidence in documentation and discussion with staff of an interchange of staff visits and of provision of staff development for HOEC staff. The PPR report identified the need to 'build in sufficient time during staff teaching visits to undertake staff development and research collaboration activities'. The report also recommended greater coordination and more contact between staff in the Newport Business School and staff at HOEC. A further recommendation suggested a more structured approach to the joint identification and fulfilment of staff development needs. The team concurs with the findings of the PPR report which provide a basis for a more methodical approach, building on established effective practice in the areas of staff development and research collaboration.
Student admissions
60 The Commentary stated that Newport undertook to consider prospective students for admission without discrimination. The validation document for delivery of the programme at HOEC stipulates the same entry requirements as those for the home-based MBA programme. All applicants are required to provide details of qualifications and experience and in some cases Newport asks staff at HOEC to interview candidates and make recommendations on admissions. According to the Commentary the National Academic Recognition Centre for the UK (NARIC) is consulted as necessary for verification of comparability of qualifications. A significant number of students are admitted through Accreditation of Prior Achievement (APA) in through Newport's standard procedures, with Newport staff approving and documenting all such cases. HOEC staff meeting the audit team confirmed that Newport made the final decisions on admissions to the programme.
61 With regard to the English language requirements for entry to the programme, the audit team noted a discrepancy between the Memorandum of Agreement and the Commentary. The Memorandum of Agreement stipulates a minimum IELTS [International English Language Testing System] score of 6.0 for entry but the Commentary was unequivocal that there was no English language requirement for entry to the programme. Given the delivery of the early stages of the programme in Chinese and the use of simultaneous translation of lectures from Newport staff, Newport will wish to clarify its requirements in this area.
62 From review of documentation and discussion with staff at Newport and HOEC the audit team concluded that procedures for the consideration of applications and for admissions were rigorous and operating as intended. The team confirmed that practice in this area met the expectations of the relevant precepts of the Code of practice.
Assurance of academic standards
Assessment of students
63 In its Commentary Newport stated that it operated 'clearly defined and well-documented regulations and procedures for assessment related matters'. The regulatory framework for the HOEC programme is essentially the same as for the home-based provision.
64 Examinations are conducted at HOEC in accordance with Newport's regulations. HOEC employs independent invigilators to oversee the conduct of the examinations and keeps scrupulous records, including invigilation reports compiled after each examination. Staff at HOEC were also alert to the potential for academic dishonesty. Examination boards are conducted at HOEC, or at Newport with a video link to HOEC, in accordance with Newport's procedures and are chaired by a nominee of the Chair of the ASC.
65 According to the Commentary students receive 'comprehensive feedback [on assessment] from the relevant tutor using cover sheets similar to those used at Newport'. The feedback is provided in Chinese and is translated for moderation purposes. The audit team viewed a sample of such feedback forms and found them to be essentially summaries; staff at Newport attributed this to them having been translated into English and summarised at the same time. In meetings at HOEC the team heard from the students that feedback from HOEC staff was good, although substantially informal; at the MBA stage this feedback draws on information provided by Newport staff. The team considered that a greater degree of formal sampling by Newport of assignment feedback would further support the quality assurance of this feedback in the context of the delivery of the course in Chinese.
66 At the certificate and diploma levels assignments and examination papers are prepared in Chinese by HOEC tutors and are translated into English by HOEC staff. The Newport Link Coordinator arranges for internal and external verification of the assessment tasks. Marking schemes and model answers are supplied to HOEC by Newport staff. At the MBA stage the assignments and examinations are prepared by Newport staff in English, are internally and externally verified and then translated into Chinese by staff at Newport who are fluent in the language.
67 The Commentary stated that a 25 per cent sample of all completed assessments was translated into English for second-marking and verification by external examiners according to a schedule decided by the Link Coordinator. The Commentary claimed that this approach ensured that every student's work was sampled at least once at each stage of the programme. External examiners see the translated sample of scripts and marks prior to meetings of examination boards and can discuss the work with HOEC and Newport tutors. Chinese-speaking external examiners also see samples of work in the original Chinese. At the MBA stage external examiners review a 50 per cent sample of the student dissertations. The audit team noted and would support the recommendation in the PRP report that, in future, external examiners see all MBA dissertations for further security of the assessment process.
68 The Commentary emphasised the importance of the quality of the translators and interpreters to the successful operation of the programme. There is a pool of translators and interpreters at HOEC some of whom are drawn from Hainan University to maintain objectivity. In the initial stages of the collaboration a 10 per cent sample of the translated scripts was sent to an external translation service for verification of its accuracy, a practice that attracted external commendation. Translations are now verified by Newport's own Mandarin-speaking staff. While not doubting the integrity of the translations provided in-house, the audit team suggests that consideration be given to the reinstatement of external verification of translations for increased assurance of the security of assessment. The protocols governing translation in assessment processes are not documented.
69 Having reviewed the relevant documentation and discussed the arrangements with staff at Newport and HOEC the audit team was confident that the assessment process was substantially effective and sound. Assessment procedures are consistent with those obtaining at Newport. The security of administration of the assessment, including the use of independent invigilators, is identified as a feature of good practice in the audit. The team considered that codification of the protocols governing translation in assessment would provide additional security in the assessment process.
External examining
70 The Commentary pointed to the effectiveness of the external examiner system in securing the standards of awards. External examiners are appointed by Newport and operate in accordance with its normal procedures which the report of the institutional review confirmed as effective, enabling Newport to 'derive considerable benefit' from its external examiners. Standard practice is for external examiners to be appointed to consider both UK and overseas versions of the same programme. It is a requirement that the UK external examiners appointed to the HOEC programme be familiar with 'the business and education context' in South East Asia and China in addition to having experience of examining MBA programmes in the UK. For the last six years Newport has appointed UK-based external examiners who speak Mandarin. China-based external examiners are required to have experience of master's programmes and be familiar with Western education systems. In addition they must be sufficiently competent in the English language to communicate with Newport staff. The audit team formed the view that formal documentation of the requirements for particular linguistic competence and experience in external examiners appointed to its overseas collaborative provision would strengthen the institution's approach in this area.
71 Newport provides extensive briefing documentation for newly appointed external examiners all of whom are invited to an annual induction day. As it is largely impractical for the China-based externals to attend induction at Newport, the Link Coordinator provides a briefing for them in China. The external examiners are required to attend the examination boards.
72 The Head of Quality Assurance receives and reads all external examiner reports to identify generic issues and features of good practice highlighted in the reports. Deans of school are responsible for responding to external examiners, setting out action to be taken in response to any points raised in the reports. External examiner reports and responses to them feed into the AME process.
73 The audit team discussed the external examiner system with staff at HOEC who confirmed that they saw the summary of the external examiners' comments as recorded in the minutes of the examination boards but did not see the full reports. Newport may wish to consider providing copies of the original external examiner reports to its colleagues at HOEC to strengthen the operation of the partnership.
74 The Commentary claimed that the work of the external examiners for the MBA provided ' a measure of equivalence with respect to the output standards of students, not just between Newport and its collaborative partnerships but also with MBA programmes elsewhere in the HE sector'. From its review of external examiner reports the audit team was able to confirm that the claim was justified. The team concluded that the approach to external examining applied to this partnership, particularly the academic experience and language skills required of the examiners, was robust and effective, providing for confidence in the academic standards of the programme. The team found that practice in this area was consonant with the relevant precepts of the Code of practice.
Certificates and transcripts
75 As the formal awarding body, the University of Wales requires that the language and location of study be stated on certificates. The certificates for the MBA delivered in partnership with HOEC state that '[t]his scheme was taught and assessed in English and Mandarin'. As noted above (paragraph 61), there are no English language requirements for admission to the programme and the language of instruction and assessment is Chinese. While it is open to students to undertake assessments in English, the audit team did not detect any elements of the assessment regime where English was stipulated as the language to be used. It is therefore possible for students to complete the programme without any understanding of the English language. Senior staff at Newport meeting the team proffered the argument that, as the final stage was delivered in English, the wording on the certificate was accurate. The team consider this reasoning to be disingenuous as delivery is mediated through simultaneous translation. The team was of the strong view that the wording on the certificate had the potential to mislead prospective employers and other interested parties as to the level of English language competence that might be assumed in award holders. The team noted that the wording on certificates was a matter of similar comment in the report of the overseas audit in 2001 of Newport's collaborative arrangement with the South SCI-Tech Training Institute, Dongguan. The present team recommends that Newport give further consideration to the veracity of the wording on the certificate in consultation with the University of Wales as the awarding body. Newport may find precept A24 and the associated explanation in the Code of practice, Section 2: Collaborative provision and flexible and distributed learning (including e-learning) a useful point of reference in this respect.
Quality of information and support for students
Student information and support
76 The audit team viewed the information provided to prospective students, both as paper documents and on the relevant websites, and talked to students about the information they received before applying for admission. Most students had initially heard about the course through word-of-mouth and personal recommendation. The students confirmed that the pre-application information provided by HOEC and Newport was accurate and provided the details that they needed to assess the suitability of the course to their requirements. Information about the programme is provided in Chinese on the HOEC website. Students were clear about the nature of the partnership between HOEC and Newport and also understood that the University of Wales was the formal awarding body.
77 At registration HOEC provides students with handbooks, supplied by Newport's Academic Registry, which detail the relevant regulations and procedures. The handbooks also include information about facilities and resources, student entitlements, programme aims and learning outcomes, and learning and teaching and assessment strategies. Students are provided with module handbooks in Chinese, with programme and module specifications, and with the requirements for the dissertation. Students meeting the audit team confirmed that the information they received about the course had been comprehensive.
78 In meetings with programme staff at Newport the audit team was informed that students received a University of Wales Newport handbook in English with the sections that were of relevance to them translated into Chinese. As a consequence of this approach, unless the students understand English they do not have access to the full range of information included in the handbook. The team considers that there would be advantage in Newport translating into Chinese all documentation provided for students at the point of registration to ensure that all students have equal access to the relevant information.
79 HOEC operates a formal induction process which students who met the audit team confirmed as effective in introducing them to the programme and to the staff. Newport staff are involved in the induction process to answer any questions about the programme. Students were particularly appreciative of the opportunity, managed by HOEC, to meet students from the previous cohort to discuss their experience of the programme.
80 After induction the students' main source of academic guidance and counselling is through HOEC's internal systems which were appraised and confirmed as satisfactory in the initial approval. HOEC staff providing the simultaneous translation of lectures delivered by Newport staff are subject experts who act as 'co-teachers' to provide contextual and other clarifications as necessary. Newport staff expend significant effort to ensure that the co-teachers are familiar with the lecture content and concepts prior to each occurrence of the programme. In meetings with the audit team, students emphasised the value to them of the joint approach to delivery of the final stage. At the start of the partnership, dissertation proposals were approved jointly by Newport and HOEC but as the partnership matured responsibility for approval and supervision passed to HOEC. The team heard from the students of a strong informal system of feedback, advice and support at HOEC which applied equally to students in the outreach centres. Past students from the course confirmed to the team that the qualification had improved their career and promotion prospects.
81 As HOEC is within the campus of Hainan University students have access to the library and computing services of the University. In addition there is a small library within HOEC which has copies of management texts in English and in Chinese and past dissertations. The audit team learned that visiting tutors provided packs of learning resource materials to students in the outreach centres who also have access to local libraries at the delivery sites. The Commentary also noted that students could access Newport's library online resources and that a number had done so in preparation for the dissertation.
82 The main formal and informal communication with teaching staff from Newport occurs during the final stage of the programme. The team was informed that English-speaking students could also email staff at Newport and that enquiries from non-English speaking students would be channelled to Mandarin speakers on the Newport staff. In meetings with the team students conveyed a real sense of 'belonging' to Newport and confirmed that they were confident in the equivalence of the course to the cognate home provision. Notwithstanding the formal limitations in the Memorandum of Agreement on the rights of students in relation to Newport (paragraph 39), there was clear evidence of commitment on the part of Newport staff to the students.
83 At the time of the audit and until September 2006, under the arrangements for devolution of authority from the University of Wales, appeals in relation to postgraduate provision were the responsibility of the federal University. Appellants have the right to a personal hearing or to send an alternate to speak on their behalf. The audit team questions the value and practicality of this provision for students studying overseas, especially those who do not have a command of English language. Complaints are handled through Newport's normal procedures. In meetings with the team students confirmed that they received information about complaints and appeals procedures but stated that they were unlikely to invoke them as HOEC staff had always acted promptly to resolve informally any difficulties raised by the students.
84 From review of documentation and discussion with staff and students formed the view that information provided to the students was comprehensive and readily accessible. As it assumes responsibility for appeals Newport may wish to consider establishing an approach that takes account of the practicality of students exercising the rights afforded to them. The involvement of current and past students in induction of new entrants is identified as a feature of good practice in the audit which could usefully be applied to collaborative provision more generally.
Feedback from students
85 Student feedback is gathered through both formal and informal methods. Formal feedback mechanisms include module questionnaires, representation at course boards and student focus groups. The PPR process also includes meetings with students. Feedback forms written in Chinese are returned to Newport but in meetings with staff at Newport the audit team heard that these were only translated if a student had provided a 'fairly extensive comment'. The Link Coordinator meets students informally without HOEC staff present to gather direct feedback which is reported through the standard visit reports.
86 In meetings with the audit team students expressed satisfaction with the formal and informal mechanisms for providing feedback on their experience of the programme. They confirmed that the same mechanisms, including meetings with Newport staff, applied at the outreach centres. There was clear evidence of action taken in response to student feedback. Newport may wish to consider whether translation of all written feedback would provide additional assurance in this area. The team concluded that mechanisms for student feedback were effective and that Newport was responsive to matters raised by the students.
Conclusion
87 In summarising the findings of the audit, the audit team concluded that the collaborative arrangement was characterised by a clear sense of partnership between HOEC and Newport. There was a visible and tangible Newport presence at HOEC in the learning materials and the use of the institution's crest and logo. In considering the partnership the team identified the following features of good practice:
- The approach to security of the assessment process, including the use of external invigilators (paragraph 69)
- The care taken to establish a Newport presence at HOEC, engendering in the students a clear sense of identity with Newport (paragraph 82).
88 The audit team identified the following points for consideration by Newport as it develops its partnership arrangements:
- codify and document existing practice with particular reference to the approaches to delivery and assessment in Chinese and the approval of outreach centres (paragraphs 17, 26, 34, 69)
- review the approach to the wording on the award certificate indicating the language of delivery and assessment (paragraph 75).
89 The Commentary provided a useful account of the operation of the collaborative provision, with particular clarity in establishing where the guidance in the Code of practice had been applied to the institution's approach to collaborative provision. The audit team found that Newport had given careful consideration to the Code of practice in its management of collaborative provision and that its policies and procedures were in general alignment with the precepts. The team was supplied with the report of a periodic review of the partnership provision conducted in the spring of 2006. Taken with the Commentary, the report of the review provided the team with an evaluative and accurate appraisal of the collaborative arrangement. The Commentary indicated that, with the exception of the processes associated with delivery and assessment through the medium of Chinese, the quality assurance arrangements at HOEC were representative of Newport's procedures and practices in all of its overseas collaborative partnerships.
90 The audit team found evidence that academic standards in the collaborative arrangement were secure and that the quality of the learning opportunities was suitable to the nature of the provision. The team concluded that full implementation of the conditions and recommendations arising from Newport's internal review of the partnership and collaborative provision with HOEC would support confidence in the Newport's stewardship of academic standards and the quality of learning opportunities in its overseas collaborative provision.
Appendix A: The University of Wales, Newport's response to the collaborative provision audit report
The University of Wales, Newport has a strong track record of collaborative links with partner institutions in the People's Republic of China, evidenced through a previous QAA audit (2000) on Newport's collaborative links with South Sci-Tech Institute, Dongguan.
The University is pleased with the latest audit outcome, in relation to Newport's collaborative links with the Hainan Overseas Education Centre, which confirms 'evidence that academic standards in the collaborative arrangement were generally secure and that the quality of the learning opportunities was suitable to the nature of the provision'. The University notes the audit team's conclusion that 'the collaborative arrangement was characterised by a clear sense of partnership between HOEC and Newport'.
The University is pleased that the audit team have commended the care taken to establish a Newport presence at HOEC, 'engendering in the students a clear sense of identity with Newport', and the University's approach to securing the assessment process.
The audit report identifies two specific points for consideration by Newport, the desirability of codifying and documenting existing practice with particular reference to the approaches to delivery and assessment in Mandarin and the approval of outreach centres, and the desirability of reviewing the approach to the wording on the award certificate indicating the language of delivery and assessment.
The University intends to develop an action plan based on the two recommendations above, and any other matters identified in the report considered by the University to be of significance.
A recent review of the Annual Monitoring and Evaluation process, completed since the audit was conducted, has resulted in a requirement that specific consideration of matters relating to collaborative provision be addressed at programme, School and institutional levels.
In summary, the University of Wales, Newport welcomes the audit report and its findings, and will continue to build on the strengths recognised, whilst addressing the few areas identified as needing further attention.
Appendix B: Student statistics
As of June 2006 there are 96 students registered on the programme.
ISBN 1 84482 630 9
Top