Circular letter CL09/02

Number	CL09/02
Subject	Handbook for institutional audit in England
Publication date	1 August 2002
Recipients	Heads of higher education institutions, related bodies
Of interest to	Those with a responsibility for quality assurance in higher education
Further information	Peter Williams
Attachments	Annex 1 - Results of the consultation and Annex 2 - Notes on the use of institutional information

Dear Colleague

I am writing with further information on the work we have been doing to develop the new institutional audit procedures (and associated transitional arrangements) which the Agency is introducing in England during the coming academic year. This letter is for information only and does not need a reply, although we are always pleased to receive the views of institutions on our work. The letter accompanies a copy of the final version of the Handbook for institutional Audit in England, and a note on the transitional arrangements for 2002-2004, which we have published on 31 July.

Result of consultation on the draft Handbook

In April we circulated the draft Handbook for Institutional Audit in England with a request for responses by mid-June. A total of 74 responses (including a combined submission from the research-intensive universities) were received, the majority (60) coming from higher education institutions. The remainder were received from the NUS, non-subscribers, partner bodies and professional and statutory bodies. A summary report on the responses is enclosed with this letter, together with an indication of how we have addressed the matters raised (Annex 1).

In outline, the general response to the draft Handbook was favourable and indicated clearly that the proposals commended themselves to respondents. There were, however, three main areas where expressions of doubt or concern recurred. These were

The discipline audit trails;
The roles of students in the process; and
The information and documentation requirements.

Other topics which gave rise to occasional comment were collaborative provision; thematic enquiries; the particular needs of specialist institutions; representations and complaints against the Agency's judgements and management of the audit process; and the role of the proposed institutional nominee in the developmental engagements.

In the light of the responses, which were very helpful, we have reviewed and revised the Handbook, taking care to ensure that, where necessary, the text is clarified and ambiguities are removed. In some instances we have changed our original proposals. In others we will bring the concerns to the attention of auditors during their training. There remain, though, some important matters of principle which need additional discussion, but which it would not be appropriate to include in an operational handbook. These are dealt with in the following paragraphs.

Discipline audit trails

A small number of respondents indicated that they considered the discipline audit trails to be akin to full subject reviews. This is not the intention and will not be the case. The scope of the trails and the resources that can be devoted to them are quite different to those of a subject review. A subject review will only take place if a prima facie case observed during an audit trail is subsequently confirmed by specialist advisers.

The role of students in the audit process

A number of institutions expressed concern about the role of students in the audit process. Their particular concerns were that students might use the opportunity to submit a confidential document to make unjustified negative assertions about their institution or programme, which the institution would have no opportunity to rebut. Some were also worried that if the Students' Union officers were the group normally invited to make the submission they would not be representative of the student body as a whole. For its part, the NUS strongly supported the proposals for the participation of students, believing that if confidentiality were not respected then institutions would move to stifle or neutralise criticism either by adroit use of their PR function or possibly by attempting to disadvantage individual students.

These contrary views suggest a rather disturbing lack of trust between some institutions and their students. The Agency is seeking students' views in order to get the perceptions of those people to whom the institutions are offering educational opportunities and to understand their experience better. We have indicated in revisions to the Handbook that we would always wish to see students sharing their submissions with their institutions, and that if that is not done then the value of them to the audit team will be limited, because they cannot be discussed directly or verified with institutional representatives. But we have also indicated that we will not refuse a submission which student representatives wish to be confidential to the team. It must further be emphasised here that these submissions are not intended to be alternatives to the institutional self-evaluation documents. Nor will audit teams accept them at face value. As with any piece of information which is available to auditors, its value as evidence will be carefully weighed and checked. We will do our best to brief students on their roles and to make sure that their contribution strengthens the audit process, rather than weakens it. To this end we are in discussion with NUS about the provision of formal briefing for students who are likely to be in involved with audits.

Information and documentation

Most of the responses that raised matters about information and documentation were concerned about the requirements contained in the Cooke Report on Information on Quality and Standards in HE (HEFCE report 02/15). As you will know, this report was not compiled by the Agency, but we will be using it as the basis for our auditing of information. A note (Annex 2) is attached to this letter which describes the expectations of HEFCE which we will be incorporating into our audits.

Collaborative provision

The establishment of collaborative links with other institutions is a common feature of higher education in the UK. It is important that all participants in these links should have confidence in the strength of the quality assurance arrangements that support their operation. So far as is possible, we would wish to include the audit of collaborative provision within the scope of the normal six-yearly institutional audit. Some institutions, however, have a large number of links and/or links which involve complex management schemes. These may not be appropriate for inclusion in this way and may be better served by a separate audit. There is no formal definition of what constitutes 'large' or 'complex' provision for this purpose. In order that our judgements on all collaborative provision should be reliable, we will need to take a view, on a case by case basis, on whether we can sensibly incorporate an audit of collaborative provision within a normal audit, or whether we should undertake a separate review. This will be a question we will discuss with institutions when we have our preliminary meeting with them.

Small and specialist institutions

The audit needs of small and/or specialist institutions differ from those that are large and multi-disciplinary. We recognise that for some institutions there will be no sense in trying to undertake four to six audit trails, and that a five-day audit might involve the audit team seeing the same people many times. During our preliminary meeting we will discuss with institutions of this sort how best to ensure that there is no 'overkill' and that the audit process reflects their size and character, while at the same time ensuring that the audit judgements are consistent, reliable and comparable with those of other higher education institutions.

Recruitment of auditors/reviewers

We are writing separately to seek nominations for auditors, audit secretaries and subject specialists to participate in audits and developmental engagements. I do hope that institutions will be willing to nominate colleagues of high calibre to act in these capacities, as the success of the new arrangements will, as always, depend critically on the quality of those undertaking them.

Institutional member in developmental engagements

One of the topics raised most frequently in the responses to the consultation on the Handbook was the role of the institutional member of the developmental engagement teams. Several respondents were unhappy about an institutional nominee being a member of a QAA team that would be making a judgement - albeit an unpublished judgement - on their immediate colleagues. Others saw this as an indication of the truly developmental focus of the activity.

The institutional member was introduced into the process in an attempt to maximise the direct developmental benefit to the institution itself and we hope this will prove to be the case. It will be entirely for the institution itself to decide who will act in this capacity. If an institution, however, does not want to nominate a member of the team, then it will not be required to do so. In such an event, QAA will not nominate another reviewer to take the place of the institutional member.

Timetable for audits, subject reviews and developmental engagements

By now all institutions should have received from us an indication of when we wish to undertake their first institutional audit. I am very grateful for the co-operation in agreeing these dates that most institutions have shown.

We are also well advanced on discussions about which disciplines should be the subject of subject reviews or developmental engagements. The creation of a schedule for these transitional activities is particularly complex and it may be a few days yet before we can write to you to indicate which disciplines will be involved and when the engagements will take place.

Equal opportunities

We have been asked on a number of occasions whether our review procedures will include the audit of equal opportunities policies and practices. The Agency does not have a general remit in respect of equal opportunities, so there will not normally be a comprehensive and systematic enquiry into the area. Nevertheless, there is a clear equal opportunities dimension to the Agency's specific remit and references to equal opportunities recur throughout the Code of Practice. It will therefore be appropriate for audit teams to ask questions on occasion about the way in which an institution is embedding equal opportunities into the assurance of its academic quality and standards. Overriding all considerations will be the expectation that institutions are fully aware of their legal responsibilities and are discharging these effectively.

Service Standards

We have now published on our website the service standards that we intend to apply to the programme of institutional audits.

I hope that this letter, together with the annexes and the final version of the Handbook, gives a clear idea of how we are taking forward the new quality assurance process in the light of the comments we have received on our proposals. If there is anything further that you would like clarified, please do not hesitate to ask. I will send you further progress reports prior to implementation of the new arrangements.

With good wishes

Yours sincerely

Peter Williams