Introduction
In 2007, the Quality Assurance Agency for Higher Education (QAA) began to explore options for including students on its institutional audit and review teams, and an initial consultation was conducted with sector agencies and representative bodies in England, Wales and Northern Ireland to inform decision making.
QAA also undertook to learn from the experiences of quality assurance agencies in other countries which have student members as part of review teams, and the following is an extract from a report evaluating student involvement in the
Nordic countries.
Student participation on the external panels1
This chapter first starts with a discussion on student participation on the external panels. Secondly the practices and experiences in the Nordic countries are given. Then the report turns to some details about the selection of students to the panels, and finally, seminars or training for the panel members are discussed.
Overview of central aspects concerning student participation on the external panels:
Added value |
|
Obstacles |
|
Examples |
|
Reasons for including students or not on the external panels
While student participation in self-evaluation groups or as informants during site visits will arouse no comments, student involvement on the external panels is rarer, and hence probably more controversial. Traditionally, external panels have consisted of experts who are the most highly qualified in the academic world, that is, professors. A consideration when including students on the panels, therefore, is whether they should be considered experts like the other panel members, or have another, special status, for example, as stakeholder representatives. A stakeholder has rights as a concerned party. Depending on the theme of the evaluation, stakeholders may include teachers, employers, regional authorities, politicians and students. The external panel is sometimes referred to as a peer-review group. If students are viewed as co-actors in the education process, they will, in a peer-review model, have a right to be evaluated by their peers.
Up until now, students have not been included on the Danish or Icelandic external panels, while in Finland, Norway and Sweden students are sometimes included. In most countries, external panels consist of experts in a narrow sense. Choosing experts who have well-acknowledged competencies and special expertise
within the field to be evaluated has been considered important for the legitimacy of the panel’s work. When included on the panels, students have been viewed as stakeholders and as participants. Within higher education they have (at least) one area of expertise: as participants in the educational process. Student views cannot be replaced by any other expertise.
The issue of student participation on external panels has recently been the subject of discussion at EVA2 and in EVA’s Board for its practical consequences, as well as for its fundamental implications. The advantages, in the form of adding student perspectives and increasing legitimacy have been pointed out, as have potential problems. There is a formal obstacle, however: according to the Act of the Danish Evaluation Institute, members of the external panels must have special professional expertise within the field to be evaluated. Also, all members must be independent of the programmes/institutions evaluated. A student member can hardly fulfil these legal requirements, and the student’s possible role must, therefore, necessarily be
defined otherwise.
EVA’s arguments for student participation on external panels are:
- students on external panels can contribute with first hand knowledge of student perspectives of the education/institution; this is useful for planning the evaluation, and helps to ensure that the student perspective is kept in mind throughout the process
- at the site visits, students on external panels can give the other students a feeling of being heard and of their views being seriously considered; they can put questions directed at the students’ real situation concerning the setting/atmosphere and the quality of the education or programme
- students on external panels offer them the opportunity for direct contact with ‘what’s going on’ in the evaluation project
- student influence on the report may improve the chance of it being used as a tool for educational development
- students on external panels can help to further ensure the general legitimacy of the evaluation process and of the results for the students of the evaluated education or institution.
However, EVA has also raised some questions: Can students assert themselves within a group of professional experts - or will they be ‘hostages’ of the other members’ views? EVA appoints members for the evaluation groups who have strong professional qualifications and who are not representatives of organisations. EVA must remain impartial and perform the evaluation work independently of the organisations. If students are appointed by students’ organisations, how can EVA ensure an independent and impartial evaluation, and how will the other interest groups react?
In a pilot project started in 2002, EVA has included a student in the panel evaluating German studies. So far, the experience is very good.
The Ministry of Education, Science and Culture in Iceland bases its evaluation system on the European Pilot Projects for evaluating quality in higher education from 1995. A regulation on the quality control of university instruction was passed in 1999 and the provision on the composition of the external panel did not allow involving students. Currently, the regulations are under revision and this provision will
probably be more open and may leave more room for including students on the external panels.
FINHEEC3 always includes students in subject, programme or thematic evaluations. In those Finnish projects where the evaluation is targeted simultaneously at the university and polytechnic sectors, there may be two student representatives, one from each sector on the external panel. In such cases, the student members alternate in the site visits according to the institution being visited.
The practice of the former Network Norway Council has varied from one evaluation to another according to the focus of each evaluation project. When the former Network Norway Council started its evaluation projects five years ago, evaluation by a traditional expert panel was generally considered to be the model both theoretically and methodologically. However, there has been a shift towards viewing students as stakeholders. Student participation on the external panels was stated as desirable in the Council’s strategy decided in 2000. For the future, NOKUT4 has decided in favour of student participation on the panels of institutional accreditation and on audit panels. In Sweden, students are always included in subject, programme or thematic evaluations as well as in audits.
Even when students are, on principle, included on external panels, they may not be so in all types of evaluation. In Finland students are not included on evaluation panels of institutional evaluations of universities or in accreditation of professional courses. NOKUT will not include students on the expert panels accrediting new programmes, and in Sweden students are frequently not included in accreditations when a university or a university college apply for the right to award a special degree.
The reasons for not including students in certain situations vary. Evaluation of higher education is multi-faceted and evolving with, historically, different procedures for different types of evaluation. This is the case with the Finnish institutional evaluations of universities which were launched by the predecessor of FINHEEC. These evaluations were planned on an international basis without student representation and FINHEEC followed this tradition.
There may also be practical and/or economic reasons for not including students. That is partly why FINHEEC does not include students on the external panels in the accreditation of professional courses. Also, in these cases the evaluation team is very small, consisting only of one pedagogical expert and one expert on the subject area. The character of the object of evaluation may also determine student participation. As the programmes being accredited are in the planning stage,
NOKUT takes the view that they ought to be assessed by a small expert committee.
Overview of student participation on external panels in different types of evaluations:
Country |
Student participation on external panels in different types of evaluations |
|---|---|
Denmark |
Programme/subject evaluation (tested in a pilot study). |
Finland |
Audits, institutional evaluations of polytechnics, programme/subject evaluations, thematic evaluations, selection of quality units. |
Iceland |
Students have not participated on external panels of any type of evaluation up to now. |
Norway |
NOKUT appoints students to the panels for accreditation of institutions and to audit panels, but not in subject - or The Network Norway Council appointed students to the panels |
Sweden |
Sometimes in accreditation when a university applies for a degree and in thematic evaluations, always in audits and programme/ |
When students are included they take part in the work as equal members of the external panel. The experiences are mainly positive. In most evaluations, the student representative has been active, motivated and has integrated well into the panel, although, in some cases, the student has been a silent member of the group. However, this applies to other panel members as well. Student participation on the panels tends to strengthen the relevance of the evaluation for the students. Students often have the most up-to-date information, for example, about matters concerning student financial support, or implementation of the Bologna process, including ECTS, and the other panel members may consult the students on these matters. Indeed, the panels’ student members are often given special responsibility for issues that they feel especially strongly about. They may also, with good results, ask most of the questions when students are interviewed during the site visit.
Normally, the rest of the group respects the students’ views and accepts them as positive contributions to the panels. The other members of the panel have in most cases been genuinely concerned with the students’ views. In most cases only one student has been included in the expert panel. In a survey of attitudes among Norwegian students who had served on panels, they confirmed that they had been accepted by the other panel members but almost unanimously said that there ought to be at least two students to make it easier to make their voices heard. However, this wish can be hard to fulfil, as there is a limited number of members on the panels.
Representation on the external panel ensures student influence on the writing of the external report. There is a tendency, however, for undergraduate students to contribute somewhat less than the others.
Selecting students to serve on the panels
Like all other panel members, students are selected on the basis of individual ability. In Finland and Norway, the decision is made by the evaluation authorities but the students are suggested by the student organisations.
In Sweden, on the other hand, the institutions may suggest evaluators and this right extends to the suggestion of students, but the Agency decides the composition of the expert panel. Whereas the professionals suggested cannot be from the institution’s own staff, institutions may (and usually do) suggest their own students. The institutions are instructed to check the names with the local student organisation, however. Obviously, a student will never take part in the evaluation of his/her own institution. Students are nominated by the national student organisation when they are part of audit panels.
Selection criteria may include good knowledge of the education system, good knowledge of evaluations, and experience from decision-making bodies or student organisations. Student panel members are often expected to have experience of the field or subject, at the same level of study as the one being evaluated. Also, they should have sufficient experience but preferably not be taking final exams because of the workload. Naturally, the criteria vary with the type of evaluation. In audits and institutional evaluations, experience from student organisations and decision-making bodies at the institutional level is essential. Frequently, the students chosen have considerable experience from boards, other decision-making bodies and student organisations and find that these experiences are a great advantage for their work on the panel.
Training or seminars arranged for the panels include the students
All the Nordic countries arrange an introduction meeting or seminar for the panels at the start of an evaluation. The seminar/meeting is used to discuss the theory, methods and practice of evaluation, plan the individual project and give other relevant information. This meeting is also used to ensure that there is a common understanding about the mandate and the role of the panel. In those Nordic countries having student representation, students are introduced to the work alongside the rest of the panel.
In addition, FINHEEC and the National Union of Finnish Polytechnic Students on one occasion organised a seminar, targeted at potential participants. Also, FINHEEC has organised an evaluation course, intended for QA personnel but open to students.
Feedback to the panel given by the evaluation officer throughout the evaluation is also important in this context. The beginning and end of each site visit are important, providing an opportunity to discuss the contribution of each panel member. Another opportunity for feedback is during the writing of the evaluation report. According to the response from the students, the information and training offered are considered sufficient by some but inadequate by others, especially when training meetings are held the day before a site visit.
The need to learn about, for example, the legislation regulating the activities of institutions of higher education will vary between panel members. Students may need additional and/or separate introduction. On the other hand, as was pointed out earlier, students frequently know more about some aspects of education. Perhaps the most important difference between students and other panel members is their respective status in the academic world. This difference is likely to affect the group dynamics of the panels. This problem cannot be fully prevented by the agencies’ introductions, and needs to be monitored throughout the evaluation.
Students have expressed the view that a clear mandate, sufficient time to reach consensus and defined individual projects are important features of a successful evaluation project. Rather than separate training programmes for students, there seems to be a need for a seminar discussing evaluation methods and experience. Also, experience from student panel members ought to be brought back to the student organisations. The national student unions should find ways to ensure that the experience of students is passed on to the next student generation. FINHEEC intends to produce an evaluator’s handbook that may also serve to pass on experience.
Ultimately, the ability to evaluate requires practice. Therefore, it might be a good idea for students to accompany and observe another project on one of their site visits. Another idea is to establish networks of students who have served on external panels and can share their experience with beginners.
What can QAA learn from this report?
Role and remit
Where students are included as members of review or evaluation teams they are always treated as full and equal members. QAA has already adopted this approach in Scotland and should consider a similar role for England, Northern Ireland and Wales.
Student membership of review and evaluation teams improves communication with students interviewed during review or evaluation processes. Audit and review teams should consider whether it is appropriate for students to lead these meetings.
Reporting
Students make a valuable contribution to reports, although undergraduate students tend to contribute less than others. QAA should consider options to allow a meaningful and manageable contribution from students at the reporting stage.
Selection
Students are selected by the evaluation agencies based on their individual abilities, and this nomination is made by student bodies and institutions. QAA should choose which students are selected to be members of audit and review teams, and should invite nominations for student members of audit and review teams from both higher education institutes and student representative bodies, as is the case in Scotland.
The selection criteria for student auditors should be set out clearly and
include the following:
- knowledge of higher education issues
- knowledge of quality management processes
- experience of participating on decision making bodies in higher
education institutes - experience of sitting on any other boards which enhance questioning and analytical skills, such as student representative bodies, is also advantageous.
Training
There are elements of training that students will know more or less about than other reviewers or evaluators. To ensure that teams are operating from a similar starting point, it is important to train student members alongside other auditors and reviewers.
Group dynamics can sometimes be unavoidably affected by the difference in academic status between students and other reviewers. QAA should use its standard evaluation and feedback arrangements to monitor this and ensure students are able to make an effective contribution to the audit and review process.
Derfel Owen
QAA Development Officer (students and enhancement)
July 2008
1 Froedstad, W and Bakken, B (June 2003) Student Involvement in Quality Assessments of Higher Education in the Nordic Countries Helsinki: Nordic Quality Assurance Network in Higher Education
www.kka.fi/pdf/nordic/student.pdf
2 Danish Evaluation Institute (EVA)
3 Finnish Higher Education Evaluation Council (FINHEEC)
4 Norwegian Agency for Quality Assurance in Education (NOKUT)
Top