Round table meetings on external examining;

jointly organised by Universities UK (UUK), the Standing Conference of Principals (SCOP) and the Quality Assurance Agency for Higher Education (the Agency)

Introduction

Following the publication of the final report of the Task Group for Information on Quality and Standards in Higher Education and the new Agency approach to audit and review, the Agency, in collaboration with UUK and SCOP, agreed that it was an opportune moment to organise a series of round-table discussion meetings on external examining. Meetings were held in Birmingham, Manchester, York and London between May and July 2002; 263 delegates attended representing HE, FE and professional, statutory and regulatory bodies and including academics, administrators and managers.

Each of the meetings started with a brief, overarching presentation that was followed by extensive detailed discussions in small break-out groups. These sessions were the focus of the meetings as the purpose was to provide a forum for informed discussion and feedback between the institutions and the Agency and the representative bodies. Detailed notes were taken at all of the breakout and plenary sessions. They have provided the basis for an interim report requested by Professor Sir Ron Cooke (Chair of the Task Group for Information on Quality and Standards in Higher Education), briefing notes to DfES and others, and papers to the Boards of the Agency, UUK and SCOP.

The meetings indicated that there is overall confidence in the external examining system, that it is still considered to be of significant value, but that in some discipline areas there is increasing difficulty in recruiting examiners. The discussions demonstrated a great diversity of practice (more than most contributors had realised existed) and concluded that greater clarity in defining specific purposes, roles and responsibilities could strengthen the system and also provide greater reassurance to external audiences that it was indeed providing ‘independent guarantee’of an institution’s standards.

Topics raised during discussions included:

the section of the Code of practice on external examining (generally well received);
the range of roles required of external examining and external examiners
- very wide and requiring knowledge and skills of a subject specialist and/or an assessment specialist;
the proposed (Cooke) proforma for public reporting of external examiners’summary comments
- considerable concern about the nature of the reports, their number and their likely value in providing useful information;
induction or training for external examiners
- finding a balance between the requirements for effective action and imposing too great a burden; this may impact on recruitment that is already difficult in some areas;
- the benefits, but mostly difficulties, of a register or accreditation of external examiners;
use of the standards infrastructure (programme specifications, subject benchmark statements and qualification descriptors) in external examining
- finding a balance between meeting responsibilities and additional work burdens.

The roles of external examining and external examiners

The discussions included the following topics, issues and questions:

there are various and valid reasons for reviewing the external examining system and the roles of external examiners;
external examining and examiners are not used consistently: does this matter? Will the new review process will have an impact on this?
some consider that the role of the external examiner as the apparent guarantor of standards is being weakened by an increasing use of multiple layers of assessment /examination boards. External examiners need to be more empowered –they need to know that institutions are taking on board their comments.
more guidance is required on the roles/powers of the external examiner at an examination board; eg under what circumstances (if any) can they change marks?
an ‘enhancement function’associated with external examining is widely recognised and highly valued, particularly since it has an annual input;
should there be a separation of the roles in confirming standards and enhancement?

Publication of summary reports from external examiners
(the proforma in annex C of the Cooke report)

This issue provoked considerable, and mostly negative comment, in the discussions. Few, if any, participants were in favour of the proposed template; at the final and largest meeting of the series of meetings, all nine breakout groups fed back their concerns to the plenary session.

Issues raised, of both a practical and a theoretical nature, included:

who is the summary for? What is the value of the public information provided?
is the summary fit for purpose? At the moment it appears to be is 50% certification and 50% summary (or additional information).
how will the reporting be done? Who will have editorial control? Who will be responsible for putting the reports onto the web? Is this ‘publication’? For how long will each summary stay on the web? Should the summary be agreed between the institution and the external examiner?
how will the institution show that it has responded to any points raised in the summary report? The institutional response to a summary is as important as the summary. Several proposed that the summary should be presented as an agreed dialogue between the institution and the examiner rather than the institution publishing a response afterwards;
there is a risk of legal action/unsolicited attention if the external examiner is named;
publication of summary reports may will be used as a marketing tool;
publication of summary report will almost certainly inhibit frankness;
how will the system work in terms of modular and inter-/multi-disciplinary provision? Examiners’reports are often more useful at the module level but this does not fit with the summary of reports at programme level;
the method implies an annual validation against the reference points –surely this is more a matter for periodic internal review? The recommendations are confusing the role of EE and external reviewer;
how are the public assisted to understand the language used in an external examiner’s summary report, and to ensure that the summary is read in context?
if the summaries are to go ahead, an evaluation by their users should be planned.

Several alternatives approaches were proposed including:

publication of a summary by the chief external examiner only, at the end of his/her period of tenure: this was widely thought to be more useful for both the institution and for wider public information;
publication of an institutional summary of issues/themes arising each year and any action planned.

It was generally agreed that whatever is published should reflect the discourse between the external examiner(s) and the institution.

Induction and/or training for external examiners

There was a reasonable level of support for a greater emphasis on induction/training for the following reasons:

training at the national level would bring increased recognition to the role and would give it more explicit value;
training might encourage a clarification of what the role(s) of examiners actually are –what is the ‘bottom line’and what can institutions add to that?
external examiners should be provided with guidance on what the institution expect of them;
institutions should be provided with guidance on how to make their expectations clear to examiners;
training would allow for a sharing of good practice;
there were several suggestions that some aspects of induction/training should be organised at national level in order to provide an overview of common contexts and principles. Induction should also be at institutional level. In this way examiners would understand both the national expectations of the role and the institutional practices within s/he will be working;
LTSN may have a role in training –there was seen to be merit in both generic and subject-level training;
one group favoured a continuing professional development-based approach and suggested a suite of workshops designed for external examiners (may be delivered through distance learning);
an annual conference for examiners was also discussed, either within or between institutions (including various detailed proposals).

Use of the academic infrastructure (programme specifications, subject benchmark statements, the frameworks for HE qualifications)

The academic infrastructure as a series of reference points for articulating standards has been developed in the last few years. The meetings included discussions on the following aspects in this area:

how far do external examiners have to refer to the reference points in their reports?
will an expectation/requirement for knowledge of the ‘reference points’prove an additional and unreasonable burden on external examiners and make recruitment even more difficult?
external examiners from a professional/practice-based background are very valuable; will they also need to be aware of the components of the infrastructure?
the use of the infrastructure within external examining should be allowed to evolve. This will happen naturally over a period of time.

Other issues

The issues that were raised on several occasions during the meetings included:

institutions need to consider what they can do to minimise the increasing burden on external examiners;
the section of the Code of practice on external examining was generally well-received although it was thought by some to be too vague in places. One group, however, suggested that precept 1, especially 1.i, should be revisited. Is this the most important part of the external examiner’s role?
the idea of a national register of external examiners was discussed at several meetings. It had a mixed response but was less opposed if the discussion was around the provision of a register for information, rather than the register-owner recommending/making appointments as well as providing the list. One alternative was proposed: that each HEI provides a cadre of external examiners –thus the professional development of the external examiner becomes the responsibility of each HEI;
another role for the LTSN was suggested –some participants found the LTSN assessment guides helpful and wondered if there was a role for that body in producing macro-reports which highlighted themes within each discipline? It could then also compare these themes/issues across disciplines;
the impact of Cooke proposals on modular schemes should be thought through further;
professional, statutory and regulatory bodies representatives indicated that they would be happy to see the entire external examiners’report published.

Conclusion

The meetings have been very useful for UUK, SCOP and the Agency as they consider the development of policies in this area. The section of the Code of practice on external examining will be reconsidered during the academic year 2002/03

Evaluation forms indicated that delegates also found the meetings and their format valuable and, importantly, an indication of a new approach to work between the Agency and the HE sector. There was encouragement to continue this approach, which is reflected in the planning of a further series of round table discussion meetings for the period December 2002 to March 2003. These will be jointly sponsored by UUK, SCOP and the Agency and consider in further detail issues to do with the development and application of programme specifications. Preliminary details are available from Susan Melvin, s.melvin@qaa.ac.uk