1 Since January 2005, the Quality Assurance Agency for Higher Education (QAA) has published a series of papers under the generic title Outcomes from institutional audit, with the intention of making known the good practice and the nature of the recommendations identified in its institutional audit reports for institutions in England. QAA is currently considering how the findings of its institutional review reports for Welsh institutions can be made known in a similar way.
2 In the meantime, QAA sought to provide an opportunity for institutions in Wales to discuss and disseminate their approaches to the management of academic standards and quality. In order to do this, providers of higher education in Wales were invited to send a mixed team of academics and administrators to a conference under the title ‘Effective approaches to the management of academic standards and quality in Wales’, held on 15-16 May at Llandrindod Wells.
3 All the higher education institutions in Wales were represented at this event, as were the Higher Education Academy (HEA) and Higher Education Wales. In addition, three colleges of further education were represented. An initial meeting was also held between representatives of relevant further education institutions and QAA officers regarding the implementation of the process for ‘Developmental review of higher education delivered in further education institutions: Wales’. Documentation for this method is now on QAA’s website (www.qaa.ac.uk/reviews/developmentalReview).
4 Once at the conference, delegates joined discussion groups that were broadly representative of the sector in Wales. All groups were asked to discuss ‘academic standards’ and ‘academic quality’, while delegates were given a choice between sessions on ‘supporting learning opportunities’ or ‘students’. As a stimulus to discussion, selected ‘discussants’ from most institutions gave a short presentation on their institutions’ arrangements in the area to be covered. In addition, teams were provided, in advance of the conference, with a selection of papers in the Outcomes from institutional audit series. These papers were intended to stimulate discussion and act as points of comparison, rather than to provide any ‘answers’.
5 This report draws on the notes from the event and represents views expressed by delegates at the conference. The report does not necessarily represent the full scope of the discussions, rather it is intended to provide a starting point for a dialogue on enhancement between QAA and the providers of higher education in Wales. QAA would like to thank all those who helped in the event, especially those facilitators and discussants drawn from the sector.
Academic standards
6 For the purposes of this event, the term ‘academic standards’ was used to describe the level of achievement that students have to reach to gain an award. Among the arrangements delegates were invited to consider were the use of external examiners, regulations for assessment and the classification of awards, and the monitoring of progression and completion statistics.
7 Discussants described their institutional arrangements for the use of external examiners in securing and maintaining academic standards. Delegates noted that external examiners’ reports feed into annual programme monitoring, with summaries being forwarded to senior committees or academic managers. In some institutions senior officers read all the external examiners’ reports in order to identify and disseminate the good practice contained therein. Delegates reported that the actions and responses to those reports are also monitored.
8 Until recently there was some difficulty in Wales regarding the appointment of external examiners, as most institutions are members of the federal University of Wales. Staff from member institutions of the federal University could not always be considered properly external in other member institutions. This had the potential to make the recruitment of Welsh-speaking external examiners problematic. However, recent changes to the structure of the University of Wales have alleviated this. Delegates noted that University of Wales’ staff are no longer debarred from acting as external examiners in member institutions. External examiners can sometimes be recruited from outside the United Kingdom (UK), but this has cost implications. Some difficulties were also identified in appointing experienced external examiners. The usefulness of the HEA’s work in this area was noted. A number of delegates indicated that their institutions have adopted, or were considering, longer terms of appointment for external examiners, partly in order to reduce the frequency with which they need to be recruited.
9 Delegates described a number of systems for the use of external examiners. There was some discussion on the use of individual external examiners for programmes versus their deployment in teams. It is sometimes perceived that some external examiners can be excessively critical, and it was suggested that the use of teams may help to mitigate this. Delegates also noted the appointment of external examiners who look across the faculty, often concentrating on borderline cases, in an effort to achieve consistency in degree classification.
10 Emphasis was placed on the need to train and induct new external examiners carefully into the mission, policies and practices of the employing institution. Delegates described a range of different approaches taken to such inductions, including meetings with registry, faculty and school staff and students, or compulsory or non-compulsory induction and training days. Such events, while sometimes difficult to organise, were thought to provide a valuable opportunity for external examiners in different subject areas to discuss issues related to assessment and classification and thought to be helpful in promoting consistency of approach. Many of the delegates present reported positive feedback from external examiners on such induction arrangements.
11 There was some discussion of the need to ensure that external examiners’ reports are submitted in a timely fashion, and of arrangements for the removal of weaker external examiners or those who persistently fail to report in a timely way. In this context, the training and development of external examiners was considered important. The publication of external examiners’ reports was also raised, with some delegates noting that requests to make reports public have been made under freedom of information legislation. Delegates reported that institutions are making arrangements for such eventualities. In a later discussion it was thought helpful to make external examiners’ reports available to students on programme committees.
12 There was some discussion on the use of assessment regulations, which were thought to be essential, although not easy to implement. One delegate described how an institution had researched assessment practices elsewhere and as a result has introduced more formative assessment at level one. Standard assessment regulations were thought to be particularly helpful to combined honours students, in order to prevent them having to work with differing sets of regulations.
13 There was also some discussion on the feedback offered to students on assessed work. Some delegates noted that their institutions require that assessed work is returned to students within a set number of weeks. It was noted that students’ perceptions of formative feedback can be problematic. Peer review of work by other students may be a useful teaching technique, but the value of such feedback is not always seen. Feedback on assessed work is of considerable importance given that it has been highlighted in the National Student Survey.
14 The need for institutions to ensure consistency of degree classification, while taking into account students’ special or mitigating circumstances, was discussed. The locus of responsibility for degree classification varied between institutions with examination boards being permitted varying levels of discretion.
15 A range of methods was outlined for the consideration of mitigating circumstances by examination boards. One delegate described a system in which there is considerable flexibility for departmental boards, but where advice and guidance is provided by the institution, especially regarding the evidence that could be considered appropriate for mitigating circumstances to be taken into account. Other delegates described systems of student appeals panels that make recommendations to examination boards. Practice was divided between those institutions which consider extenuating circumstances only when a student fails, and those where such circumstances are taken into account even when the student has passed.
16 Delegates described varying levels of engagement with employers. While such engagements were seen to have some positive benefits, delegates noted that it was important to identify and target appropriate employers, and to ensure their involvement at key discussions. There is clearly a range of different mechanisms for communicating with employers, including the use of focus groups, and the membership of employers and recent graduates on advisory boards. It was thought by some delegates, however, that clarity is required in defining the purpose of such groups, and that employers do not always have a consistent or clear view about what they require from higher education. Moreover, some employers’ perceptions of what constitutes high standards do not always coincide with those of academics. The difficulty of balancing the needs of employers with the activity of academic research and teaching was also noted.
Academic quality
17 For the purposes of the event ‘academic quality’ was taken to be a way of describing how the learning opportunities available to students help them achieve their award. Delegates were invited to consider arrangements for the approval and validation of new provision, its monitoring and periodic review, and also collaborative provision.
18 There was some discussion on the structures in place to support quality assurance. Some institutions have central quality assurance units, while others have post-holders distributed among functions/departments. The role of these structures of quality support in disseminating good practice for the purposes of enhancement was highlighted. Handbooks are often produced to guide academic and support staff through quality assurance procedures. Delegates outlined efforts to align quality assurance procedures and documentation with the advice provided by QAA, particularly in its Code of practice for the assurance of academic quality and standards in higher education (Code of practice) and with The framework for higher education qualifications in England, Wales and Northen Ireland. A number of delegates reported that there are procedures in place for mapping the Code of practice, Section 7: Programme design, approval, monitoring and review against institutional regulations. Differences between elements of the internal quality assurance documentation can cause confusion and the need for clear, consistent procedures was stressed.
19 A number of institutions in Wales have recently been through institutional review engagements; others are currently undergoing such reviews, or will be in the near future. Delegates noted the careful preparations that institutions make for these events. Preparations often provide a useful opportunity to review and make changes to the institution’s structure or to elements of its activity, for example websites and access to information. As such, preparations are often led by the senior management team, and academic and support staff are usually closely involved in, and consulted on, the production of the self-evaluation document. Delegates noted the importance of keeping staff updated on the changes being made and on the developments in the institutional review.
20 Delegates viewed positively the value of review engagements, while accepting the increased workload for staff. It was thought important by some that quality assurance should not simply be a response to external review, but rather it should be turned to the institution’s own purposes, especially as review activities are undertaken by a number of bodies, including QAA, Estyn and the professional, statutory and regulatory bodies, all of whom have differing processes and requirements. Institutions engage with such processes at departmental, school and institutional level. It was noted that implementing the recommendations of reviews conducted at programme and subject level can pose difficulties where elements of the programme in question are used elsewhere, for example in joint, combined or multidisciplinary degrees.
21 Employer engagement was again seen to be important. The quality assurance of such arrangements was thought to raise some challenges, in particular around the assessment of academic work undertaken in the workplace. A tension was identified between the time taken for the approval of new provision, and the need to respond to employers in a timely way. It was suggested by some that a risk-based approach might be a way of dealing with this. It was also reported that some employers appear to take more interest in individual modules rather than whole programmes. However, the award of credit for an assortment of modules can raise difficulties of progression and coherence of award for the institution. Some delegates noted that their institutions are tightening their regulations around the use of ‘combined studies’ awards to avoid these being used as a ‘catch-all’ in such cases. It was thought, however, that attempts to impose ‘rules of combination’ to allow modules from different programmes to be used to create a coherent award can pose considerable difficulties. Current methods for assuring that combinations of modules from different programmes do indeed amount to coherent awards require a level of externality that involves some time delay.
22 Delegates from further education institutions noted that arrangements in higher education for the approval and validation of new provision can be problematic, as could differences in the academic year. This meant that higher education is sometimes seen as slow to respond, a situation which can be exacerbated by the fact that some further education providers often have to deal with a number of awarding bodies simultaneously.
23 There was some discussion of institutions’ collaborative provision. Although there are many benefits to collaboration, the risks of such arrangements, especially those with overseas partners, need to be taken into account. Aside from the reputational and financial risks, there are also risks to students’ learning and to the standards of awards, posed by the provision of learning opportunities by bodies outside the immediate control of the awarding institution. Some delegates suggested that there can be considerable pressure from academic departments and schools to enter into collaborative links, but such arrangements could be vulnerable to staff changes. It was noted that, despite the academic merits of some arrangements, demand among students could be variable. In overseas collaborative arrangements, language proficiency among both undergraduate and postgraduate students is an important matter for attention. Delegates noted a range of different mechanisms to manage the risks associated with collaborative provision, especially in programme approval, and in safeguarding student learning opportunities and the standards of awards. Collaborative arrangements were reported to involve a considerable workload, not least to align with the relevant sections of QAA’s Code of practice.
24 Student exchange schemes were also discussed. Delegates noted a range of different schemes available, for example exchanges for a single module, for a semester or for an academic year. It was regarded as essential to agree in advance the contribution of marks gained on such schemes to the final degree classification, and the timescales for the return of marks. It was noted that there are considerable differences between countries in the way that marks are awarded.
Supporting learning opportunities and students
25 For the purposes of the event, ‘supporting learning opportunities’ was used to describe how institutions ensure that appropriate learning support resources are in place to allow students to meet their learning outcomes. Delegates were invited to consider the provision of libraries, information and computer technology, virtual learning environments (VLEs), academic advice, guidance and personal support, work-based learning arrangements and e-learning. Delegates were asked to consider how providers of higher education in Wales deal most effectively with the diverse needs of different categories of students, both in terms of their academic work, student representation and in terms of the pastoral support that they might need.
26 In the discussions of ‘supporting learning opportunities’, discussants outlined the use of strategic frameworks for learning, teaching and assessment, learning environments, Personal Development Planning (PDP), skills and personal support and enhancement. In the groups on ‘students’, discussants described arrangements for student support in admissions, induction, childcare, language support, careers information and guidance, personal support, student representation and targeted support for groups such as international and part-time students. In practice, there was considerable overlap between the two discussions, and so they are reported together here.
27 There was some discussion of the resources used to underpin the support provided for student learning opportunities. Delegates described different approaches to the setting of learning and teaching strategies and the identification of resources to meet these strategies. Alternative sources of funding to support such strategies, for example contributions from employers, were also noted.
28 There was some discussion of the management and quality assurance arrangements around learning support. Delegates described a range of different processes, some to ensure that there is an institutional overview taken of such activities. Several institutions have appointed institutional-level managers and committees to effect this. The representation of student views to the institution is an important element of such arrangements. Delegates noted that there is often room for improvement in the training of student representatives. Attendance at institutional committees has been variable and some institutions have found it necessary to offer incentives, either in financial terms or in terms of academic credit. In some cases student attendance has improved as a result. It was also noted that representatives are often drawn from more mature students, who are not necessarily representative of the wider student body. Difficulties were noted in the representation of minority groups such as part-time, international or research students. A number of techniques to aid in the representation of such groups were outlined by the delegates, including specialist committees for research students, focus groups, and online fora for distance learning students.
29 It was noted that there has been considerable change in the modes of teaching, learning and programme delivery. Under these circumstances, changes in learning support have been necessary to help students achieve the intended learning outcomes of their programmes. This includes the induction of students into higher education, sometimes differentiated according to student need, for example, students with families and international students. A tension was noted, however, between effective induction and the need for students to begin their academic work quickly.
30 The importance of identifying varying student needs, without making unwarranted assumptions, was noted. Some delegates described systems of diagnostic assessments that are conducted early in year one to help direct support to those most in need. A range of approaches to the identification of students with special needs, for example dyslexia, was outlined here, from screening the student population to leaving the initiative to individuals to seek support where required. One institution has identified a single route for students to gain academic and pastoral support, and is monitoring the extent to which this is successful. Another way to deal with such issues is through PDP, although delegates reported a mixed reaction among students to the development and implementation of such arrangements. The support for postgraduate research students has been identified as a matter for attention in Wales. The support of students on collaborative provision arrangements was also discussed, in particular the locus of responsibility for the support of such students. In some cases students on such arrangements are brought into the awarding body for induction prior to commencing their studies. Other arrangements described included link tutors working in partner colleges who provide support parallel to that provided within the awarding body.
31 There was some discussion of the use of electronic media to support students and their learning. It was noted that VLEs are expensive to implement, and require careful support and evaluation. Where lecturers put their lecture notes on the VLE, student attendance at lectures can sometimes diminish. Delegates also noted the advantages that VLEs can bring: they can be used in assessment and to support teaching and monitor attendance. In some institutions, feedback from students on modules can be made electronically, which allows the module tutor to respond quickly. It was thought by some that the increasing use of email brings students and tutors into a greater degree of personalised contact. VLEs were regarded as important for the support of distance-learning students, and could help integrate those students into the wider institutional community. It was reported that students are often positive about such arrangements, especially as they provide easy access to learning materials. While there is increasing development of programmes to be delivered mainly through electronic media, some delegates thought that meeting face-to-face occasionally is likely to remain important on such programmes.
32 In terms of personal support it was noted that there is increasing pressure on academic staff to deal with students’ personal problems, even where these are of a complicated nature. This was thought by some to be a product of the good relations often established between tutors and students. Student expectations were also thought to be rising in the wake of the introduction of student fees. Some delegates noted that while the ability to act as a personal tutor is rarely a factor in the academic appointments process, institutions often produce manuals and training materials to help staff in dealing with key issues. In addition, institutions have central advice and support facilities for students, or trained specialist advisers in faculties or schools.
33 Matters around support for teaching and assessment through the medium of Welsh were also discussed. The challenge of identifying Welsh-speaking experts was considered, as was the need for targeted financial support for such activities. A low take up of some modules in Welsh was also noted. It was thought that some students are reluctant to be assessed in Welsh as this is perceived as potentially prejudicing future employment opportunities.
Conclusion
34 In conclusion, discussions at this conference suggest that providers of higher education in Wales make concerted efforts to secure academic standards appropriate to the UK academic community, and to manage the quality of their provision. Furthermore, there was good evidence that such providers continue to take a lively and active interest in ways of enhancing the quality of learning opportunities available to their students.
35 It was apparent from the discussions that there are opportunities for further work both by institutions and other bodies, such as the HEA and QAA. QAA has now begun to consider a successor to this event, likely to be held in the summer of 2008.
Top