Introduction
In 2007 the Quality Assurance Agency for Higher Education (QAA) began to explore options for including student members on to its institutional audit and review teams. An initial consultation was conducted with sector agencies and representative bodies in England, Wales and Northern Ireland in order to inform decision making.
A number of questions were raised in response to this consultation, many focusing on the operational issues that would need to be overcome if students were to be included as members of institutional audit and review teams. The issues raised included:
- recruitment, selection and training
- the role of students as team members
- students’ ability to engage with and understand quality assurance information and documents
- will students be able and willing to take time to contribute to
institutional audit and review? - will students have enough experience and confidence to contribute to judgements and reports?
- will students respect the confidentiality required by the audit process?
- the impact on peer review
- how the activity would be financed.
In order to inform QAA’s decisions, it was considered useful to gather evidence from other audit and review methods that include student members in their teams.
A number of UK higher education institutions include students as members of internal periodic review teams. These institutions can share valuable experience of working with students in this way and this can,inform QAA’s proposals.
Through its liaison scheme, QAA identified a number of institutions that include students as members of internal review teams and asked them to share their experiences of working with students in this way. This analysis includes the experience of:
- Exeter University
- The University of Manchester
- Sheffield Hallam University
- Queen Mary, University of London
- Royal Holloway, University of London
- London School of Economics
- University of Liverpool
- De Montford University
- Aston University
Students have been involved in review processes at these institutions for varying lengths of time. One institution has included student members within its internal review teams for over 15 years. Another has been trialling student membership during this academic year. Other institutions’ experience varies within this range.
This report gives an overview of institutions’ experience working with student reviewers and draws some conclusions regarding the lessons QAA can learn when developing its own proposals.
Recruitment, selection and training
All institutions reported that ensuring student members of their review teams are capable, willing and able to participate in institutional processes for quality assurance and enhancement is important when considering which students to involve.
The majority of institutions (six) use officers from their students’ union (SU). This would normally be the officer with responsibility for academic matters, although two institutions ask their SU to occasionally use other officers.
Institutions gave a number of reasons for and benefits of using SU officers as members of review teams, mainly focused on the fact that SU officers will have a greater understanding and insight into the institution’s quality assurance policies and procedures:
- all institutions who worked with SU officers on review teams commented that SU officers also serve on senior institution decision-making bodies, such as senate, academic board or learning and teaching committees. As a result they develop an insight into the nature and strategic priorities of the institution, as well as into the diversity of teaching and learning and devolved management structures
- institutions reported that student officers receive training for their SU roles from union staff and their predecessors. This training covers their involvement in periodic review and ensures they learn about the quality assurance process from an officer who has participated in a review. Two institutions reported that incoming officers will occasionally shadow a review with their predecessor to get a feel for the process
- institutions reported that the quality office has regular contact with SU officers and forms a relationship with them that lasts throughout their year in office. This helps to maintain reliable dialogue throughout reviews
- institutions felt that as SU officers are in regular touch with students from across the institution at various stages of their studies, they have a broader and deeper understanding of the student perspective when contributing to meetings and team decisions
- according to one institution, its SU officers participate in various networks and regularly share information and good practice from their own institution with other officers, and vice versa. Student officers bring this knowledge and insight to review teams, which adds depth to their contribution.
Typical comments:
‘We have an excellent relationship with our students’ union and we always include their representatives in everything we do, so it seems quite logical that they should sit on internal review teams.’
‘It’s part of the sabbatical officer’s job role to sit on review teams so we don’t have to worry about recruiting and persuading them to take part. They get very good support and training from their predecessors, which means they are well prepared for taking part when the time comes.’
A typical challenge raised by three institutions was that very occasionally officers are elected to academic affairs posts and find that participation in institutional quality assurance processes is not a priority for them. The institution then finds it difficult to engage these officers meaningfully in the periodic review process. This was not a situation that occurred regularly, but each of the three institutions had experienced this difficulty at some stage and had worked with the SU to overcome the problem. In some instances the SU would co-opt another officer to take their place, and one institution was thinking of changing its periodic review policies in order to include non-sabbatical students on panels.
Typical comment:
‘On one occasion, the education and welfare officer was singularly focused on welfare issues and did not appear to want to engage with academic matters. This hadn’t happened before and hasn’t happened since but it did mean we missed out on the student contribution for a while.’
Three institutions reported that they do not use SU officers as members of internal review teams. Instead they work with the SU to recruit and train students interested in serving on review teams to act as representatives at departmental and faculty level.
These institutions explained that they are not opposed to the principle of using sabbatical officers on internal review teams, and occasionally do if a review takes place at a time that is inconvenient for students (for example out of term time or during peak assessment periods). In these instances both the SU and institution felt that there is greater value in using students who are not SU officers, as they bring a uniquely student-focused perspective which is not compromised by involvement in decisions about institutional-level policies and strategies.
In these instances, the student members always have experience of serving as school, department or faculty representatives. This gives them some grounding and understanding of decision making structures and matters relating to quality assurance and enhancement.
The process of recruiting students to participate is usually fairly informal. The SUs communicate with their network of representatives to inform them of upcoming reviews, and ask for expressions of interest. At one institution this happened early on in the year, providing an opportunity for student representatives to attend a training session. Reviewers were then selected based on their performance at that training session. Another SU recruits separately for each review, with the student representation coordinator requesting expressions of interest. The coordinator then meets each of the students separately for a discussion and selects appropriate representatives based on that discussion. On average the SUs reported that between eight and 15 students per review express an interest in participating.
All institutions ensure that students do not sit on review teams for schools or departments within which they are or have been a student. Two institutions always use students to review departments within their home faculty, arguing that this ensures they have empathy for and understanding of other, related subjects and the management structures and processes.
Typical comment:
‘We have no objection to sabbatical officers being members of review teams but since we started including students on teams the SU has always nominated student representatives from faculty committees. This has always worked very well and the calibre of students we have on teams is very high.’
The role of students as team members
All institutions have clearly set out policies and procedures for internal review which outline the role of review teams. Students are included as full and equal members of review teams with the same remit as other members.
One institution suggests in its policies that students are ‘not to have a detailed input on the curriculum’. Another institution offers non-prescriptive guidance on issues students might deal with: ‘The representative from the Guild of Students is primarily concerned with the student experience and should ideally be given the opportunity to explore this, as appropriate, during the visit.’ This does not appear to cause tension between team members, as the student member’s knowledge and experience is particularly relevant in this area. The remaining seven institutions do not make any special provisions for student members of review teams.
Typical comments:
‘If students are going to play a full and equal part in the process we don’t think it’s necessary or useful to single them out for specific instructions/remits.’
‘We’ve found that student reviewers are fully and equally capable of engaging with the review process, so we haven’t seen the need to cut them off from certain aspects of review.’
Arrangements for chairing review processes and meetings vary across institutions. All institutions appoint a senior member of academic staff to lead and chair the entire review. The chair leads the process and acts as principal liaison for the school/department under review, the review team and the institutional quality assurance office.
Some institutions’ procedures allow individual review team members to chair different sessions of the review day. At three institutions students regularly take responsibility for chairing/facilitating meetings with students. One institution’s guidance document for periodic review stipulates that the student team member will chair meetings with students. All three institutions reported that this works well and that having the student member chair these meetings appears to put the students at ease from
the outset.
Feedback from reviewers and students confirmed that a student presence on review teams helped put students who met the teams at ease. This ensured that students meeting the teams contributed fully to the process and helped break down communication barriers between academic reviewers and students.
Typical comments:
‘Sometimes situations arise where reviewers and students do not connect very well, this can be because of differences in language or perspective. There are some obvious instances where having students on review teams has helped to ease these difficulties.’
‘When I was a member of review team I chaired the meeting with students and I think it helped a lot, the students seemed to be reassured when I said I was the education officer at the students’ union. A bit like having ‘one of them’ on the team.’
Training
None of the institutions reported that they have formal training structures for reviewers. All indicated that the institution’s quality office provides direct support for reviewers to ensure they understand the institutional context and the documentation they are provided with for the review.
The institutions that use student representatives rather than SU officers reported that the SU provides support and training to complement or replace support provided by the quality assurance office.
In two instances the SU provided formal structured training, as opposed to reactive support and guidance. These training programmes usually consisted of group exercises looking at mock documents and working through typical scenarios and lines of enquiry. Feedback from student reviewers indicated that this form of informal, activity-based training was very useful.
The support needs of students throughout the review processes tended to be similar across the institutions, and were very similar to the support needs of new academic reviewers. Support required by students usually included:
- explaining the process and procedures involved in internal review
- the institutional context for quality assurance and enhancement
- assistance to understand the language and terminology used
in documentation - assistance to identify relevant sections of departmental or school policies to explore and test evidence presented in self-evaluation documents.
Students’ understanding of quality assurance information and documents
The type and volume of information prepared for review teams varies slightly across institutions, as does the level of detail contained in documents.
In all review procedures included here, the team receives a self-evaluation document (SED) from the school or department under review and has access to any documentation that relates to the SED and the review.
Two institutions have particular arrangements for gathering the views of students ahead of the review. This is so they can be read and digested alongside the SED and other documents, to inform the review team’s understanding of the student learning experience in that department.
At one of the two institutions this is done by a member of staff from the quality assurance office, who meets a group of students at a focus group to gather views on their learning experience and comments on the SED. Their feedback is analysed and reported back to the review team. The other institution asks the student member of the review team to carry out a similar exercise, meeting with student representatives at the particular school and gathering their views to present to the full review team.
Feedback from student members of review teams suggests they do not generally have difficulty understanding the documentation they receive. Their contributions throughout the review process also suggest they have little difficulty digesting information and forming lines of enquiry and judgements based on them. In some instances it was reported that students have often made a greater effort to digest and understand the documentation they receive, probably to compensate for the fact that they have less experience than other reviewers.
Typical comment:
Once they get into the swing of it, student reviewers have no difficulty analysing the information, coming up with questions or using the evidence in the paperwork to back up judgements.’
Three institutions reported that students occasionally, particularly if participating in their first review, are daunted by the documentation, citing its volume and the language used as particular problems. Where this happens, the quality assurance officer responsible for the review or another member of the review team usually provides support, as they would with all new reviewers.
Typical comments:
‘We’re always worried about first time students because they’re sometimes daunted by the amount of information they are bombarded with for a review, especially when the language is also quite dense! This often happens with new reviewers so we make sure we book in some time to take them through it (the documentation and review process). They soon come round.’
‘We always make sure we brief new reviewers, this is no different for students, but it is a bit repetitive because we’re always working with a different student.’
The majority of institutions reported that students regularly make constructive and useful comments on documentation in advance of the review. Students suggest and pursue their own lines of enquiry, requesting additional documentation and information where necessary.
Time commitment
The time commitment expected from reviewers - taking a number of factors into account such as reading documentation, preparatory meetings, training, the review meetings and reporting - ranged between two and two and a half days.
Whether working with SU officers or student representatives, all institutions reported that students could be relied upon to fulfil their entire commitment as reviewers and were willing to give up the time necessary to fulfil their duties.
All institutions reported that great care is taken to ensure sufficient notice is provided to review team members, and to ensure that reviews do not take place at times that are inconvenient for reviewers (academic and student) and for the schools/departments under review.
The majority of institutions reported that students are very conscientious about their duties and that they have not experienced any serious difficulties with attendance. One institution, which works with student representatives rather than SU officers, reported some occasional difficulties with students having to drop out at short (3-4 weeks) notice because the assessment timetable had changed.
Two institutions working with SU officers reported occasions when SU officers had been elected and did not want to participate in review activities. This created some initial difficulties, but the institutions’ quality assurance officers worked with the SUs to find other solutions or officers to take their place.
Student role in making judgements, commendations and recommendations
At each of the institutions interviewed, student members of review teams play a full and equal role in making judgements, commendations and recommendations.
Student role in reporting
Arrangements in seven of the nine institutions mean that the review secretariat (provided by the institution’s quality assurance office) takes notes throughout the review and prepares the report based on the lines of enquiry and judgements made by the review team. This report is circulated to all members of the review team for comments and to suggest amendments. Reviewers do not often make suggestions for changes, though one institution reported that student members of review teams were particularly keen at this stage and would often make several constructive contributions to drafts.
Arrangements for periodic review at another institution allow the school or department to write their own reflective report on the review, drawing on the judgements and comments of the review team to inform a reflective analysis and action plan for their school or department. Once this report is prepared, a draft is sent to the review team for comment. As with other institutions, reviewers rarely make substantive suggestions for improvement.
One institution asks individual reviewers to take responsibility for drafting sections of the report, based on their lines of enquiry and judgements. These sections are gathered by the secretariat and compiled into a full report, which is then circulated to the entire team for comment. Student members write the sections relating to the student learning experience and have thus far produced clear, concise and
timely reports.
Respect for confidentiality
It is rare that review teams deal with highly sensitive or confidential information, but occasionally they have access to information such as individual student and staff details or assessment records.
There have been no incidences where students have divulged confidential information.
Effectiveness
Across the board it was felt that student members of internal review teams play a full, professional and effective role when carrying out their duties. They interact well with other reviewers, engaging in discussion on various issues and do not always limit themselves to contributing on issues related to the student learning experience. Students often propose new and additional lines of enquiry based on documentation and meetings with staff and students.
Student members can usually be relied on to ask pertinent and penetrating questions which help the review team’s understanding of particular issues. Where students experience difficulty in pursuing a line of enquiry or getting to the crux of a matter when meeting with staff, other team members feel quite comfortable stepping in to assist. Conversely student team members will assist other reviewers when they are meeting with students.
Added value
Every institution reported that including students as members of internal review teams was an important feature of their approach to student participation. A number of institutions reported that student participation at this level reinforced their commitment to listening to students and dealing with them as partners in the development of the institution. The institutions’ views were that students are active participants in their own education and are part of the academic community, able to impact on the design and delivery of their own programmes. It was also widely commented that student membership of review teams added to the students’ perception and understanding of their role in the wider college community.
Some institutions also commented that including students on review teams raises the profile of internal reviews among students and, therefore, raises the level of interest in the process.
Typical comments:
‘The more we involve students, the more they will feel part of what we do.’
‘The centrality of students as partners in the strategic development and improvement of the university is very important to us, and so their participation on internal review panels is essential.’
‘Students are part of the wider academic community and having them on review teams reinforces that message. We want them to contribute to development of the whole university, not just their own course, and this is one way for them to do that.’
A number of institutions reported that internal review processes are an important element of effective quality assurance and enhancement, and should have a major impact on improving the learning experience of students across institutions. Including students on review panels adds to the perception of legitimacy regarding this element of quality assurance and enhancement. This was a key consideration when the institutions included student members on review teams.
Typical comment:
‘It’s important that students feel confident in the processes we put in place to listen to them and to improve their experience. A student presence on university bodies, in this instance periodic review teams, does reassure students that their views and priorities will be taken seriously.’
Issues of language and the appropriateness of quality assurance and enhancement terminology when communicating with students were raised by some institutions. They observed that students, who may themselves have had difficulty coming to terms with the language and terminology at first, are particularly effective at framing or rewording questions during meetings with students, who then understand what the review team is asking them.
Typical comment:
‘It is sometimes quite apparent that the students review teams meet are very nervous and struggle to understand the questions being asked of them. Student reviewers are very effective at diffusing these situations by either suggesting alternative wordings for questions in advance of the meeting, or by rewording questions during meetings.’
It was also noted that a number of student reviewers commented on the language of final review reports, suggesting changes and improvements to make them more accessible to interested students.
Many institutions noted that student members of review teams possess valuable expertise on and insights into the student experience. As current or recent students they are able to empathise with the various issues that impact on their learning experience.
Institutions also reported that student reviewers add an important perspective regarding quality assurance and standards, based on their learning experience.
Typical comment:
‘You could put it like this, students bring additional expertise on the learning experience and an additional perspective on matters related to quality and standards.’
Students who take part in reviews develop a much clearer understanding of institutional processes and priorities and their impact on individual schools and departments. This gives them a fuller picture of the diverse nature of their institution and, therefore, greatly improves their ability to contribute to wider discussions about institutional policies and practices.
What can QAA learn?
It is not possible to draw direct parallels between institutional periodic review arrangements and QAA’s institutional audit process, due to differences in scale and nature. Nonetheless, there are some lessons that QAA can learn from institutions’ experience of including student members in review teams if it is to proceed with plans to include students as members of QAA institutional audit and review teams.
Recruitment, selection and training:
The experience of institutions shows that current students and SU officers are capable and willing to participate and contribute to review activities. Those students who are recruited to participate have experience of engaging in discussions about quality, standards, and learning and teaching at school, faculty and/or institutional level.
QAA should consider recruiting to review teams (as it does in Scotland) students with experience of representing students on decision making bodies and/or those who have participated in periodic review activities. QAA should make a concerted effort to ensure that it recruits both current students and SU officers.
Institutions that used student representatives as members of review teams found that informal methods of communicating with students - such as advertising via email and using networking websites - proved to be the most successful recruitment methods.
QAA should explore and exploit non-traditional methods of reaching students to ensure the maximum audience is reached. Applications will need to be reinforced by references from the students’ institutions.
Where school and faculty student representatives were selected to sit on review teams, a selection stage was built in to the recruitment process to ensure suitability.
QAA should consider including a screening stage for applications when recruiting student members of audit and review teams in order to assess their experience and capability.
Formal training for internal reviewers was not widespread, and where it was provided to students it was on a small scale. Given that the scale of internal review is radically different to that of institutional audit, it is difficult to draw any lessons from institutions in this area. However, informal support for students grappling with the volume, language and terminology of review was a regular feature of institutions’ processes.
QAA should build on current training and support provisions to ensure that students on audit and review teams are aware of standard quality assurance terminology. Students should feel confident that informal support and guidance is available throughout the audit and review process.
The role of students as team members:
At all the institutions included in this analysis students were treated as full and equal members of the review team, with the same responsibility for the review and its judgements as other team members. In two instances some additional guidance was provided to students to indicate areas where they may or may not wish to pursue lines of enquiry. These instances were intended as guidance rather than restrictions and did not cause confusion for review teams.
As in QAA Scotland, students can be integrated as full and equal members of review teams and contribute effectively to the entire review process. Should QAA wish to provide further guidance on areas that students members may or may not wish to focus on during audit or review, the experience of two institutions suggests that this can be helpful and does not cause confusion or tension for teams.
Institutions reported that having student members of review teams helped to relax students attending meetings and break down communication barriers. This was particularly true when student members of review teams chaired meetings with students and led the questioning.
QAA may wish to consider giving guidance to review teams that student members of teams should lead meetings with students, as evidence suggests this improves the productivity of those meetings.
Students’ ability to engage with and understand quality assurance information and documents:
Institutions reported that student members of review teams make every effort to digest information presented to them in advance of a review, conscientiously contribute to discussions and suggest lines of enquiry. It is clear that a number of students are daunted by the volume of documentation, the language and terminology contained within them.
QAA should ensure informal support is available to students throughout the process and that students can access advice if and when necessary.
Will students be able and willing to take time to contribute to institutional audit?
Feedback from institutions was that both student representatives and SU officers have little difficulty participating in internal reviews. Some problems do arise when reviews take place during assessment periods or when SU officers do not prioritise their membership of review teams. The number of students expressing an interest in participating in internal reviews (between eight and 15 per review) suggests that there is sufficient interest among the student body.
QAA should consider the potential difficulties and risks of allocating current students to audits during peak assessment periods.
Will students have enough experience and confidence to contribute to judgements and reports?
In all internal review processes covered in this paper, students play a full and equal part in forming judgements, commendations and recommendations. There is little evidence of reporting to draw upon. At one institution where students do produce their own sections of reports, no challenges or difficulties have been experienced. Where reports are written by the secretariat to review teams, students regularly make constructive contributions, comments and suggest amendments.
Will students respect the confidentiality required by the audit process?
No institutions reported incidences where students had breached any confidentiality.
Derfel Owen
QAA Development Officer (students and enhancement)
July 2008
Top