The last 10 months have seen a major change in the direction of the Agency's work. The proposals for a new approach to the external review of academic quality and standards in England, as well as the emerging and innovative ideas being developed in Scotland and Wales, offer us all the opportunity for a new beginning.
Implemented, these proposals will capitalise on the considerable progress made by higher education institutions over the past decade in securing their academic quality and standards. This progress has been verified, across the whole of the UK, by the various audit and subject assessment processes that have been operating since 1991.
Despite some claims to the contrary, our higher education system is among the most coherent and consistent, in terms of academic quality and standards, of any in the world. This is largely the result of the work that institutions have done, in association with the Agency, to create and embed the academic structures envisaged in the Dearing report on higher education (1997). These structures are beginning to demonstrate their value: now the investment has been made, it is time to reap the benefits.
What are the benefits? First, there is much greater awareness, at all levels,
of the importance of careful and systematic attention to quality and standards
in higher education. This includes recognition of the public duty to inform
those to whom it matters - particularly students, potential students, and
employers - about the standards of qualifications, and the academic facilities
and services provided to help students reach those standards. Universities
and colleges are now in a good position to provide that information. Secondly,
the audits and assessments over the past decade have confirmed that there
are no major systemic problems with quality and standards across higher
education in the UK. There is no need to subject the academic community
to repeated detailed external reviews at the micro level.
It is sufficient to know that verified information is being provided and that
quality assurance systems are working effectively. Thirdly, the move towards
reliance on institutions' own internal quality assurance mechanisms means that
the threat, perceived by some, of external control of higher education should
recede. The twin traditions of academic collegiality and autonomy, so important
for the intellectual health of a mature democracy, can once again be nurtured.
These benefits are considerable. But they do depend upon institutions' willingness to acknowledge their public responsibilities and duties, and to discharge them conscientiously and effectively. That is the compact implicit in the consultative document issued in July (HEFCE 01/45).
One of the Agency's principal tasks is to reinforce institutions' own capacity
and effectiveness as guardians and stewards of academic quality and standards.
Outside review bodies or inspectorates, visiting institutions or departments
every five or six years, can at best have a limited impact on the quality
and standards delivered
day-to-day (albeit a dramatic and decisive impact at times). The effective way
of assuring and improving quality on a continuing basis is to ensure that institutions
recognise their own responsibilities and actively meet the resulting challenge.
This principle is underpinning the development of the new quality assurance arrangements.
The standards infrastructure (see page 11) makes
the task easier.
So what is the role of external quality assurance when responsibility for quality and standards cannot but lie with those providing the programmes and awards? It is, first and foremost, to check that institutions are running their academic affairs in a way that can command public confidence. This is the accountability role, whose integrity cannot be jeopardised. External quality assurance also ensures that the information institutions provide for potential students and others about academic quality and standards is full, useful, reliable and up to date. This is the information verification role. And, further, it offers opinions and information to institutions about how they might consider improving their approaches to bring them up to the best of observed current practice. This is the enhancement role.
The development of the new process for England is inevitably at the forefront of our current work and thinking and is creating a lot of fevered media interest. But the Agency is continuing to do many things that don't attract quite so much attention. An example of this is our cooperation with the Scottish, Welsh and Northern Irish higher education funding bodies. We want to ensure that their review requirements are met in a way that reflects their distinct national characteristics, while at the same time allowing a maximum degree of congruency of outcome across the whole of the UK. We continue to advise relevant national ministers on applications from institutions seeking university title or powers to award degrees. And we are operating the Access Courses Recognition Scheme. This scheme is making an important contribution to the widening participation strategy, and it commands the support and respect of its participating organisations. Internationally, our programme of overseas audits, and participation in the Bologna and Prague processes, are enhancing the reputation of our higher education system here at home, defending its strengths and maximising its opportunities.
This issue of higher quality has articles about many of these activities. I hope the topics included will both inform and offer some new insights into the range of issues we are dealing with. I also hope that you will recognise the spirit of considerate interest, professionalism and helpfulness that we wish and intend to characterise our dealings with institutions and their staff, students and other interest groups. It goes without saying that we would be very pleased to hear from readers who agree or disagree with our views, and who want to contribute to the matters being discussed in this newsletter.
Peter Williams
Acting Chief Executive
The new review process and operational issues
In July, the Higher Education Funding Council for England (HEFCE), in association with Universities UK, the Standing Conference of Principals (SCoP) and the Quality Assurance Agency, published consultation proposals for a revised method for the quality assurance of teaching and learning in higher education in England. The consultation document (HEFCE 01/45) explains the rationale for the proposals, and also includes an outline of the review method that is envisaged. A first look at the responses to the consultation follows.
Since July we have received many requests, from higher education institutions and others, for more information about the process and procedures that would be used if the proposals were to be implemented as described. These requests have presented us with some difficulties. It would clearly not be right for us to pre-empt the results of the consultation by publishing procedural details that depended upon a particular outcome. And, until the Information Task Group has made its report, it is not possible to devise a scheme that takes full account of the information sets that institutions will be expected to provide. Finally, the consultation document states that 'the QAA will issue in due course, also for consultation and in the light of responses to this paper, a more detailed statement of the proposed approach to implementing the proposed framework' (paragraph 57).
Nevertheless, we do think it is important that the possible operational implications of the proposals should be understood. So we have decided to produce a 'preliminary operational description' in response to the requests we have received. It fleshes out the limited information provided in the consultation document, but takes into account the factors mentioned above. The description can be found on our web site at www.qaa.ac.uk.
Readers of the preliminary description need to be clear that it is no more than an illustration of what might be done if the proposals in the consultative document were to be implemented. It is not a statement of our policy or a 'QAA-preferred' model; it contains gaps that cannot be filled until other decisions have been taken. It also contains some features that would need further development before they could be implemented with full confidence.
The preliminary description does, however, represent a first shot at designing a workable method for institutional audit and selective subject reviews along the lines indicated in the consultative document. We hope this is helpful to readers.
The description will be modified in the light of further discussions between HEFCE, Universities UK, SCoP and the Agency. In the New Year, when the other related discussions on information are completed, a more detailed set of proposals will be published for formal consultation.
Preliminary analysis of Quality Assurance in Higher Education - consultation responses
This article draws on an interim report on the consultation responses that was prepared on 31 October. The report was written to inform a series of meetings between the Agency and its main partner organisations which took place in the first two weeks of November. It was intended to provide an early and preliminary indication of mood and messages from the consultation, and was based on a representative selection of the responses.
We are grateful to all the organisations and individuals who have responded to the consultation.
Analysis of the responses has continued throughout November and a final report, based on all responses received, will be available very soon after this edition of higher quality.
Higher education institutions (and further education colleges in England)
In general, higher education institutions support the model set out in the consultation document. There is broad support for:
- an external quality assurance method that recognises that institutions themselves have the primary responsibility for quality and standards;
- a method that is as 'light' as possible, proportionate to the purposes to be achieved;
- the provision of useful public information to inform the choices of potential students and other stakeholders;
- an end to universal, comprehensive subject review programmes;
- a greater focus on enhancement, alongside accountability and information;
- evolution towards an audit-based method, that places greater reliance on institutions' own quality assurance processes, to secure these purposes.
There is strong encouragement to reinforce the message that the elements of the academic infrastructure - the Code of practice, Subject benchmark statements, qualifications frameworks etc - should be seen as advice and guidance, rather than prescription and expectation.
English further education colleges and their representative bodies are particularly concerned that they were not fully involved in the development of the proposals; that the model treats them differently from higher education institutions; and that there will be continuation of a comprehensive, subject review regime for further education colleges. There was strong encouragement for looking at how OFSTED and Adult Learning Inspectorate inspections could link with Agency audits/reviews.
Notwithstanding their broad support for the approach set out in the consultation document, higher education institutions identify a number of issues for further consideration. The two main ones relate to the place and nature of subject level processes in a system based on institutional audit, and to the expectations on provision of public information.
Subject level processes
There are concerns about: apparent tensions and confusions of purpose in the model; contradictions between objectives and operational descriptions in the document; 'drilling down' to subjects; whether following up at subject level was an appropriate response to weaknesses at institutional level; and the proposed 2002-04 subject review programme in higher education institutions.
There is general unease about the apparent continuation of subject review by another name, and about the range of subject level processes within what is presented as a method based on institutional audit.
Information
The expectations about publication of information are seen as crucial to any reduction in burden for institutions. Some respondents see an increase in burden in the proposals in the consultation document. Many institutions have reservations about the suggestion that summaries of the results of internal processes might be published: they see this as potentially compromising the rigour and candour of those processes if these are to form the basis of public information. The point is made that consistency of content and/or presentation would have to accommodate diversity of mission and structure and the need to capture value added. There are also concerns that publication of quantitative data should not lend itself to conversion into league tables.
Most respondents would have liked greater clarity and detail in the information section, in order to offer a reasoned view. The need for consultation by the Information Task Group is stressed.
Enhancement
There is support for a method that would give greater prominence to enhancement, but disappointment with the way it is handled in the consultation document. There is a general view that the enhancement dimension of the proposed model needs more development.
Professional and statutory bodies (PSBs) and subject associations
Overall, there is broad support for the proposals including the move to a lighter touch. There is a general willingness to work with the Agency, and to cooperate, coordinate and share information where possible. At the same time, there are clear statements about the different purposes of accreditation and audit, and a general mood that the PSBs would need to continue running their own review processes to meet their professional and statutory obligations.
From subject associations or groups speaking on behalf of a subject there is, again, a general welcome for the approach and principles set out in the consultation document. There is some concern about possible information burden; the expiry date for current information; that 'drilling down' should not become subject review by another means; and that the role of subject associations should be recognised more explicitly with further development to better articulate the relationship.
Students
Students welcome the current review of quality assurance methods. They support the principles and objectives set out in the model, but have some concerns about the extent to which an institutional audit can assure quality at the subject level, and the potential reduction in public information at subject level compared with present arrangements. They argue for a greater role for individual students' unions in the audit process.
Martin Johnson
Subject level reviews across the UK
Completion of subject review, 1993 to 2001
The final round of subject reviews for the period 2000-01 is nearing completion. With the exception of a very small number of revisits, the schedule that started in 1993 in England and Northern Ireland will be finished by December 2001. During the period October to December 2001, around 180 reviews will be carried out in the 11 subjects currently under scrutiny. There are 33 reviews, included in the figure above, taking place in further education colleges. Many of the visits to further education colleges are to provision with less than 30 full-time equivalent students, and these are part of the sample of small provision requested by the Higher Education Funding Council for England.
A review of the cycle of assessment and review visits from 1993 to 2001 is being prepared and will be published early in 2002.
Progress with academic review
Scotland
In 2000-01 we undertook 25 academic reviews at the subject level in Scotland, in what was designed to be year one of a six-year cycle of reviews across the UK. This programme involved 12 of the higher education institutions in Scotland and included provision across 15 broad subject areas. The 2001-02 review programme involves 52 reviews across 17 higher education institutions and includes the remaining six subject areas from the first half of the cycle. We are very grateful for the hard work and professionalism of all involved in these reviews - institutional staff and reviewers. The experience and evidence of these two years is providing an important and secure basis from which future arrangements in Scotland can be considered.
The last of the 2000-01 reviews ended in July. The majority of the reports resulting from these visits will be published during December, with the remainder following in January 2002. The first year of the new review method has been monitored carefully with formal and informal feedback from academic departments, institutions, review facilitators, specialist reviewers and review coordinators. While much of the feedback has been very positive, critical feedback has led to significant operational developments for 2001-02. These developments have been summarised in the paper A Practical Guide to Review, which was refined through wide discussion and consultation involving all Scottish higher education institutions. This paper has been used in reviewer training and has been widely circulated to all involved in the 2001-02 programme. Changes to the 2001-02 reviews include improvements in communications, containing reviews within more strictly defined time periods, and a tighter monitoring and production framework for review reports.
We are identifying the substantive outcomes in relation to learning and teaching from the first year of academic review. These will be discussed fully with colleagues from the sector and elsewhere, and reported in the next edition of higher quality.
Wales
The Higher Education Funding Council for Wales (HEFCW) has agreed with Higher Education Wales that the current hiatus in externally generated reviews cannot be allowed to continue for another year. After discussions with us, HEFCW has indicated its wish to commission up to three subject level 'engagements' in each higher education institution, using an adaptation of the method described in the Handbook for academic review (the Handbook). HEFCW sent a circular to each Welsh institution, indicating that up to three subjects should be put forward for review. We have already received some indication of institutional intentions.
We have prepared a booklet, Academic Review in Wales, 2002, in response to a request from the HEFCW. The booklet outlines a method based on the proposals in the Handbook, but involving a threshold judgement of academic standards and graded judgements of student progression and learning resources. It is expected that a self-evaluation document will be prepared for each engagement. Most teams will have three subject specialist reviewers and a review coordinator. The teams will spend a maximum of two days in the institution, preceded by a preliminary visit by the review coordinator and a desk-based analysis of the self-evaluation. The reviews will be followed by a report, which will remain confidential to the institution, HEFCW and the Agency. The method should reduce significantly the burden of preparation for, and execution of, the reviews in comparison with reviews experienced to date. A copy of the booklet has been sent out with the circular, so that institutions may understand the method and comment if they wish to do so.
The Welsh engagements should provide us with useful experience in running 'limited' reviews and contribute to the current debate about subject level engagements for the future.
Further education colleges
A large number of further education colleges offer higher education programmes. They are, therefore, subject to review by the Agency. In 2000-01, we carried out a 'scope and preference' exercise for further education colleges (as well as higher education institutions). The exercise indicated a preference for less than 30 reviews in 2001-02 and more than 200 reviews in 2002-03. This distribution would place an unacceptable burden on the Agency and reviewers. Discussions have been taking place with the HEFCE about a more even distribution of review visits.
In July, the schedule of reviews for 2001-02 and the Service Level Agreement were finalised and signed with HEFCE. The repercussions of the delay (caused mainly by the debate about the review of quality and standards in higher education institutions) are particularly important for further education colleges. From January to July 2002, there will be around 75 subject level reviews at these colleges. During this period five colleges will receive their first reviews from us. All of the reviews will be full-scale reviews, as described in the Handbook. We do not carry out institutional reviews of further education colleges. Since 1999-2000 we have, for the first time, been carrying out reviews in more than 200 colleges that had been the responsibility of the former Further Education Funding Council. There are, in addition, some concerns about quality and standards in a minority of further education colleges; this is demonstrated by the results from the current cycle of subject reviews.
In order to achieve the desired number of academic reviews for the 2001-02 schedule, a number of subjects have been brought forward from 2002-03. This action, and the need for full-scale reviews, have combined to cause concern in further education colleges and their staff.
In an effort to overcome the problems associated with this, we have placed those who asked for a 2001-02 review at the front of the schedule, and have ensured that those brought forward from the 2002-03 schedule have been given as much notice as possible. The deadline for submission of self-evaluations for those being reviewed in the spring and summer terms has been postponed until 1 February 2002.
The detailed schedule for the January to July 2002 reviews has been completed. Letters were sent to colleges in October, informing them of the arrangements, and negotiations with further education colleges will continue.
We have also been running workshops and seminars for the staff of further education colleges to try to ensure that they are well informed about academic review. In addition, we have contributed to seminars for further education colleges organised regionally and nationally by HEFCE, as well as making a number of other presentations to a range of audiences. Despite these activities, considerable uncertainty about the differences between peer review and inspection, and a worrying lack of knowledge about the method, prevail among further education staff. One of our biggest problems has been to persuade colleges to nominate their staff to act as peer reviewers; a problem not helped by the contractual obligations that make the release of staff difficult or impossible.
Peter Milton
Norman Sharp
Continuation audit
The process of continuation audit was first used in early 1997 when the universities of Sheffield and Exeter agreed to take part in the trials of the new process. Since that time a further 82 audits have been completed, with another 12 audits planned for the next few months. A great deal has been learned over this period, both about the process of audit and about the way institutions have developed successful strategies for ensuring the quality of learning opportunities and maintaining the standards of their awards.
As the continuation audit process has developed, and auditors and institutions have provided feedback on their experiences, we have refined what we do and how we do it. This experience is being used to inform the development of the new process of institutional audit.
We are preparing to publish a 'Learning from Continuation Audit' document that will draw out the key messages and themes which emerge from continuation audit reports. A great deal of interesting material has emerged from the reports, but the key messages are fairly clear, and while they may be considered largely self-evident, they are nevertheless worth saying. The reports support the view that an institution that has the capacity to manage its activities successfully will understand and support its 'mission'; will have strong lines of communication throughout the institution; and will have methods for ensuring that the right information goes to the right people at the right time.
Nicola Channon
Review of health profession programmes
The Department of Health has contracted with the Agency for the development and organisation of aprocess to review the quality and standards of higher education health profession programmes in England 1. Both the Department and the Agency are working closely with the professional and statutory regulatory bodies, the workforce development confederations and higher education institutions. This collaboration should ensure that the new approach to review will streamline existing retrospective quality assurance processes and will deliver the necessary public protection and accountability requirements.
Following completion of initial development work, and the publication of benchmark statements for the healthcare professions, prototype reviews are underway. During the academic year 2001-02, six institutions (and their partner placement providers) have agreed to participate in prototype reviews of their health care provision:
| Kingston University and St George's Hospital Medical School | physiotherapy, radiography |
| University College Northampton | nursing, midwifery, occupational therapy, physiotherapy, podiatry |
| University of Plymouth | health visiting, midwifery, nursing |
| Sheffield Hallam University | health visiting, nursing, occupational therapy, physiotherapy, radiography |
|
University of Teesside |
occupational therapy, physiotherapy, radiography |
|
University College Worcester |
midwifery, nursing |
A steering group, comprised of stakeholder representatives, has been established and will meet throughout the prototype cycle. The group will act as an advisory forum and will disseminate information emerging from the prototypes to professional networks.
Once completed, the prototype reviews will be independently evaluated. Any necessary amendments to the review methodology will be made in light of experience gained in the prototypes. We hope a full cycle of review will start from October 2003.
Louise Holder
1Dietetics, health visiting, midwifery, nursing, occupational therapy, orthoptics, physiotherapy, podiatry (chiropody), prosthetics and orthotics, radiography, speech and language therapy.
Next: Part 2
Top