The handbook for Integrated Quality and Enhancement Review

Ref: QAA 214 01/08

• Introduction
• Structure of the handbook
• Section one: Overview of IQER
• Section two: Role of awarding bodies
• Section three: Developmental engagement
• Section four: Summative review
• Section five: Adaptations to IQER
• Section six: Guidance to colleges on preparing a self-evaluation
• Section seven: A glossary of terms
• Section eight: Annexes
• Annex A: QAA's purposes, values and standards
• Annex B: An indicative programme for a Developmental engagement or Summative review preparatory meeting
• Annex C: An indicative programme for a Developmental engagement or Summative review visit
• Annex D: Public information about academic standards and quality of learning opportunities
• Annex E: IQER role descriptions and person specifications
• Annex F: QAA training and development policy for review team members and review facilitators

---

Introduction

1 The Quality Assurance Agency for Higher Education's (QAA) mission is to safeguard the public interest in sound standards of higher education qualifications and to inform and encourage continuous improvement in the management of the quality of higher education. In furtherance of this mission, QAA undertakes reviews of higher education provision delivered in further education colleges (colleges) on behalf of the Higher Education Funding Council for England (HEFCE), which has statutory responsibility for ensuring that provision is made for assessing the quality of education provided by institutions it funds. From 2007-08 the process of review used in colleges in England is called Integrated Quality and Enhancement Review (IQER). Annex A provides details of QAA's purposes, values and standards.

2 The purpose of this handbook is to:

• state the aims and objectives of IQER
• describe the programmes of study that IQER covers
• explain how IQER works
• provide guidance to colleges and their awarding bodies preparing for, and taking part in, IQER.

3 The handbook is intended primarily for teams conducting IQER and for college staff who are directly involved in IQER. It is also intended to provide information and guidance for other staff in colleges and for colleges' awarding bodies. It is not intended for students, for whom QAA is producing a separate guidance note. QAA has also developed other guidance notes to assist institutions in preparing for IQER and an operational manual for the purposes of administering and managing IQER, for use by QAA staff and coordinators. In addition to this handbook, QAA will provide support for the implementation of IQER through briefing and training events.

Structure of the handbook

4 The handbook is divided into eight sections:

Section one provides an overview of IQER, including its aims and objectives.
Section two provides guidance on the role of awarding bodies.
Section three describes the Developmental engagement.
Section four describes the Summative review.
Section five:

• describes the abridged method for colleges with fewer than 100 full-time equivalent students funded by HEFCE
• describes adaptations to the method for colleges in HEFCE-funded consortia
• describes adaptations to the method for colleges undergoing significant changes to their higher education provision
• explains how QAA determines the number of Developmental engagements for each college
• explains how QAA determines when each college's reviews will take place.

Section six provides further guidance to colleges in preparing a self-evaluation.
Section seven is a glossary of terms. This handbook uses some terms and phrases in ways that may differ from common usage. When one of these terms or phrases appears for the first time, it is highlighted in bold type and a definition is provided in the glossary.

Section eight provides a series of annexes.

Figure 1: The relationship between the handbook, guidance notes, the operations manual, briefing and training to support IQER

Section one: Overview of IQER

5 This section gives an overview of IQER, including its aims, objectives and scope. A more detailed description of how IQER works follows in sections two to five.

Aims

6 IQER is an evidence-based peer review of a college's management of the student learning experience and performance of its responsibilities for the academic standards and quality of its higher education provision.

7 Colleges do not currently have powers to award higher education qualifications¹. They work with awarding bodies, in particular Edexcel and/or one or more higher education institution. The awarding bodies retain responsibility for the academic standards of all awards granted in their names and for ensuring that the quality of learning opportunities offered through collaborative arrangements is at least adequate to enable students to achieve the academic standard required for their awards. IQER focuses on how colleges discharge their responsibilities within the context of their agreements with awarding bodies. QAA reviews the responsibilities of higher education institution awarding bodies within these relationships through Institutional audit. There is more information about the relationship between IQER and Institutional audit in section two.

8 Against this backdrop, the overarching aims of IQER are to:

• support colleges in evaluating and improving their management of their higher education, for the benefit of students, and within the context of their agreements with awarding bodies
• foster good working relationships between colleges and their awarding bodies, for the benefit of students
• enable HEFCE to discharge its statutory responsibility for ensuring that provision is made for assessing the quality of education provided by the institutions it funds
• provide public information.

Objectives

9 To realise these aims, IQER has three objectives:

• to engage colleges in a process of self-evaluation and peer review focused on reviewing, evaluating and improving the management of their higher education provision
• to produce reports of these review activities
• to contribute to public information about the academic standards and quality of higher education in colleges.

10 IQER usually takes place in two complementary stages called Developmental engagement and Summative review. The emphasis of the Developmental engagement is on supporting the college in developing its higher education provision and its management of the student learning experience in an open and collegial way; whereas the Summative review is primarily concerned with reviewing, and making judgements about, the effectiveness of the college's procedures for the management of the student learning experience and their implementation. While there are some methodological variations between the Developmental engagement and the Summative review which reflect these different emphases, both share a number of common elements. In particular, both stages of IQER:

• focus on a college's management of the student learning experience for its higher education provision
• acknowledge the shared responsibilities of awarding bodies and colleges, and seek to enhance these relationships
• are based on a self-evaluation prepared by the college
• share three core themes: academic standards, quality of learning opportunities and public information
• assume that the college and its awarding body(ies) are already managing the provision effectively according to the expectations of the Academic Infrastructure
• are led by teams of peers with current or recent experience of managing, developing, delivering and/or assessing higher education in colleges and/or higher education institutions
• prioritise the interests of students
• lead to the production of reports.

Other considerations

11 Other considerations which have informed the development of IQER are:

• the benefits to colleges of a review method which is comparable to QAA's Institutional audits of higher education institutions in terms of both method and judgements
• the need for a flexible method capable of accommodating and adapting to the diversity of arrangements for the provision of higher education in colleges, which is characterised by an array of different types of delivery, sizes of provision, and awarding and funding relationships. There is more information about adaptations to IQER in section five
• QAA's obligations as a signatory to the Higher Education Concordat on quality assurance arrangements and data collection
• the benefits of producing review reports which can inform other types of external scrutiny, such as Institutional audits by QAA and inspections by Ofsted. There are more details about this in sections two and four
• the experience of a pilot study of IQER in 2006-07 and, in particular, the results of two separate evaluations of that pilot.

Core themes

12 IQER teams will manage and focus their enquiries within this process according to the three core themes: academic standards, quality of learning opportunities and public information.

• Academic standards refers to the level of achievement a student has to reach in order to achieve a particular award or qualification.
• Quality of learning opportunities considers the effectiveness of everything that is done or provided by the college to ensure that its students have the best possible opportunity to meet the stated outcomes of their programmes and the academic standards of the awards they are seeking.
• Public information is information about academic programmes, academic standards and quality of learning opportunities which is in the public domain whether in print or electronic form.

More information about the three core themes is provided in the glossary in section seven.

Figure 2: The relationship between the core themes and all stages of IQER

Academic Infrastructure

13 In considering colleges' management of higher education provision against the three core themes, IQER teams will be guided by the expectations of the Academic Infrastructure. The Academic Infrastructure is a set of nationally agreed reference points which inform and support the effective management of academic standards and quality in higher education. It is published by QAA and applies to all UK higher education wherever this is delivered. It comprises the following:

• The framework for higher education qualifications in England, Wales and Northern Ireland (FHEQ) which includes descriptions of the five levels of higher education qualifications (there is a separate FHEQ for Scotland)
• the Code of practice for the assurance of academic quality and standards in higher education (Code of practice), which comprises 10 sections
• subject benchmark statements which describe the characteristics of degrees in different subjects
• award benchmark statements, such as the Foundation Degree qualification benchmark which provides a description of the characteristics of a Foundation Degree
• guidelines for preparing programme specifications which are descriptions of what is on offer to students in individual programmes of study.

Further reference to the Academic Infrastructure is made throughout this handbook. A more detailed description is provided in the glossary in section seven.

 

Figure 3: The relationship of the Academic Infrastructure to the continuous improvement of the management of academic standards and quality in higher education

Role of students

14 One of IQER's aims is to support colleges in reviewing and improving their management of their higher education provision for the benefit of students. Within this context IQER teams need to identify students' views about their experiences as learners and to draw on these views in considering colleges' higher education provision.

15 Teams will meet students during IQER visits as a matter of course. The arrangements for organising these meetings are covered in sections three and four of this handbook. Before a visit, teams will also invite students to produce a Student written submission which may take a variety of different forms. QAA will provide a separate guidance note for students on the preparation of a written submission. The principle of the student submission, irrespective of its form, is that it should reflect the students' own views of their experiences as learners. Colleges may have a valuable role to play in helping their students to prepare a submission, for example by sharing information with them. QAA will provide further guidance to colleges during preparations for IQER. QAA will also work with colleges and students' representative bodies throughout the cycle of IQER to encourage and support student involvement in the review process.

Scope

16 IQER is concerned with taught higher education programmes of study funded by HEFCE at levels C, I, H and M of the FHEQ (or levels 4 to 7 of the National Qualifications Framework). All colleges funded by HEFCE to provide higher education programmes at these levels are required by HEFCE to participate in IQER, whether this funding is provided directly to the college, indirectly through a different college or higher education institution, or through a HEFCE-funded consortium.

17 IQER does not apply to research degrees funded by HEFCE at master's or doctoral levels (or level 8 of the National Qualifications Framework), nor to higher education programmes funded by other organisations.

FHEQ	Level	National vocationa lqualification level	National qualifications framework
Doctorates	D Doctoral	5	8 (specialist awards)
Master's degrees, Postgraduate Certificates and Postgraduate Diplomas	M Master's	5	7
Bachelor's degrees with Honours, Graduate Certificates and Graduate Diplomas	H Honours	4	6
Foundation Degrees, ordinary (Bachelor's) degrees, Diplomas of Higher Education and other higher diplomas, BTEC Higher National Diplomas (HNDs) and Higher National Certificates (HNCs)	I Intermediate	4	5
Certificates of Higher Education	C Certificate	4	4

Figure 4: The FHEQ and its relationship to the National Qualifications Framework

Section two: Role of awarding bodies

18 The purpose of this section is to provide guidance on the role of awarding bodies in IQER, with the intention of helping awarding bodies and colleges to consider how awarding bodies should be involved. It should be read in conjunction with the descriptions of the Developmental engagement and Summative review in sections three and four, which are primarily aimed at IQER teams and colleges taking part in IQER and therefore do not consider the involvement of awarding bodies in detail.

19 Colleges currently do not have powers to award higher education qualifications. They work with awarding bodies, in particular Edexcel and/or one or more higher education institutions, which retain responsibility for the academic standards of all awards granted in their names and for ensuring that the quality of learning opportunities offered through collaborative arrangements are adequate to enable students to achieve the academic standards required for their awards. IQER assumes no preferred model for higher education provision, other than that it expects that any model must permit the awarding body to assure itself about the standards and quality of its collaborative provision with colleges. Further details of a higher education institution's responsibilities for its awards may be found in the Code of practice, Section 2: Collaborative provision and flexible and distributed learning (including e-learning).

20 IQER is concerned with the way in which colleges discharge their responsibilities within the context of their agreements with awarding bodies. It is not concerned with how awarding bodies manage their responsibilities for collaborative agreements. QAA reviews the responsibilities of higher education institutions within these agreements through the process of Institutional audit. Nevertheless, higher education institutions are important stakeholders in IQER for several reasons:

• they are identified in IQER reports in association with those programmes in colleges which lead to their awards
• IQER reports will be used as a source of evidence for the audit of an higher education institution's collaborative provision and may replace the need for partner link visits to colleges; (QAA will say more about this in the supplement to the handbook for Institutional audit which deals with the audit of collaborative provision)
• Institutional audit reports will inform the number of Developmental engagements that colleges have
• judgements, conclusions, recommendations and action plans arising from IQER are not addressed to the awarding body; however, they may have implications for their relationships with partner colleges.

21 To enable awarding bodies to manage their responsibilities for their collaborative arrangements and interests around IQER effectively, QAA will make sure that all IQER correspondence between QAA and colleges is copied to the heads of the relevant awarding bodies and/or their nominated contacts. Such correspondence will include confirmation of the dates of any meetings or visits, provisional outcomes of visits and draft reports. In addition, QAA will encourage colleges to copy all correspondence they send to QAA to their awarding bodies.

22 Awarding bodies may also wish to support their partner colleges through IQER, by assisting, for example, with the preparation of the self-evaluation and by attending various IQER events, including review visits. The extent of an awarding body's involvement with IQER should be decided in discussion between the partners taking account of the arrangement and the provisions of the partnership agreement and at the discretion of the organisations involved in the collaborative arrangements. The participation of the awarding body should be considered against the maturity of the relationship between the partners; the extent of the responsibilities which the awarding body has conferred on the college; and the accuracy and completeness of existing written evidence about these responsibilities. The following examples are provided to illustrate this point:

23 QAA encourages colleges and their higher education partners to consider jointly how the higher education institution will be involved in the IQER process and to do this, ideally as soon as possible after QAA has confirmed the dates of the visits. At this stage or just after attendance at the briefing event, the college and its higher education partners should write down the details of their agreement of the higher education institution's involvement in the process and send a copy of this agreement to QAA signed by representatives of each partner institution.

24 Because of the existing pattern of good working relationships between colleges and their higher education partners, QAA expects that each partnership will reach agreement readily. However, in the unlikely event that this is not the case, institutions are invited to contact QAA to arrange for a QAA officer to meet with them to facilitate this process. QAA will provide details of the arrangements for such visit support in a separate guidance note.

25 IQER teams will be pleased to meet awarding bodies' representatives at any stage of the process, and occasionally may encourage awarding body representatives to attend particular events should they regard it as likely to aid their understanding of the college's responsibilities. As a minimum, QAA will make sure that awarding bodies are invited to the briefing, which will provide further guidance on the role of awarding bodies. However, awarding bodies are not required to attend the briefing, or any subsequent events. QAA has no desire to make unreasonable requests for awarding body involvement in this method which focuses on the responsibilities of colleges.

Role of Edexcel

26 There are two distinct forms of Edexcel higher education provision in colleges:

• Higher National Diplomas and Certificates that are approved directly by Edexcel, to which Edexcel appoints its own external examiners and which Edexcel certificates. In the case of this provision, Edexcel is the awarding body
• Higher National Diplomas and Certificates that are offered under the Edexcel Licence Agreement. The licence allows higher education institutions to validate their own versions of HND/C programmes as institutional awards. Where such licensed Higher National provision is operated on a collaborative basis in a college, the higher education institution will appoint its own external examiners and provide the certificate (although the certificate will also carry the Edexcel logo). In the case of this provision, the higher education institution is the awarding body.

27 In 2006-07, there were over 30,000 HEFCE-funded students studying for an Edexcel award at 120 different colleges. The scale of this provision will inevitably limit the extent to which Edexcel can support individual colleges and attend IQER events. In order to assist colleges and IQER teams to understand the responsibilities which Edexcel confers on colleges, Edexcel has produced a short guidance note. QAA encourages colleges to draw on this guidance note in describing the policies and procedures they have adopted with respect to Edexcel awards in their self-evaluations. Edexcel will provide further information to colleges about the contribution Edexcel officers will be able to make to the IQER process.

Section three: Developmental engagement

28 The purpose of this section is to state the aims and objectives of the Developmental engagement and describe how it operates. A timeline for the Developmental engagement appears at the end.

29 As described in section one, IQER takes place in two complementary stages called Developmental engagement and Summative review. Most colleges will have one Developmental engagement and one Summative review within the five year programme, 2007-08 to 2011-12. There will normally be a minimum of one year between a Developmental engagement and the Summative review. Section five provides details of how the programme is scheduled.

 

Overview

30 The emphasis of the Developmental engagement is on supporting the college in developing its management of the student learning experience for its higher education students in an open and collegial way. The Developmental engagement takes full account of the varying roles of awarding bodies and is sensitive towards differences in those roles between different colleges. To facilitate this approach, the Developmental engagement has three important characteristics that distinguish it from the Summative review:

• it focuses on a specific area of the college's management of the students' learning experience for its higher education and the first Developmental engagement in each college will focus on assessment
• it takes place with the full participation of the college members of staff within the IQER team
• the report is confidential to the college, its awarding bodies, HEFCE and QAA.

31 All colleges with 100 or more full-time equivalent students on HEFCE-funded programmes will have one Developmental engagement. Some may have two or, exceptionally, three. Colleges with fewer than 100 full-time equivalent students may opt for a Developmental engagement but are not required to have one unless they meet one of the risk criteria described in section five.

32 The first Developmental engagement for each college focuses on student assessment. This is because of the significance of assessment for academic standards and because it was the area identified as most in need of improvement in the earlier methods of reviews, such as Academic review. Colleges with more than one Developmental engagement will be invited to suggest a theme for their second (and, where necessary, their third) Developmental engagement. Any other Developmental engagement should normally focus on a cross-college theme, such as learning resources, placement learning or staff development, although QAA will consider a subject-focused Developmental engagement if there is a strong case for this. Colleges with more than one Developmental engagement will also be able to negotiate the timing of additional engagements to fit in with other external and internal reviews.

Preparation

33 Around 18 weeks before the visit, QAA will hold a briefing to which the college and its awarding bodies will be invited along with other colleges which have a Developmental engagement at about the same time. The purpose of the briefing is to prepare institutions for the Developmental engagement, in particular by providing further guidance on preparing the self-evaluation, identifying lines of enquiry and helping students to prepare a written submission. The briefing also allows college staff to meet the coordinator who will be responsible for managing the Developmental engagement on behalf of QAA. QAA encourages colleges and their higher education partners to consider jointly how the higher education institution will be involved in the IQER process and to do this, ideally as soon as possible after QAA has confirmed the dates of the visits. At this stage, or just after attendance at the briefing event, the college and each partner higher education institution should write down the details of their agreement of the higher education institution's involvement in the process and send a copy of this agreement to QAA, signed by representatives of each partner institution.

34 IQER is concerned with reviewing the college's performance of the management of its responsibilities for the student learning experience according to three core themes. These core themes provide a focus for the team's enquiries during the Developmental engagement. Lines of enquiry provide routes that the team takes to consider evidence to enable it to make evaluations of the core themes. They provide a way of reviewing the evidence and formulating conclusions about the area under review. They can be regarded as lenses through which the team views the area in question. So, for example, where the Developmental engagement is concerned with student assessment, possible lines of enquiry could include how work placements contribute to final assessments; how the views of employers inform assessment criteria; how the college's processes for providing formative feedback to students on coursework or the match of information on assessment in the programme specification, the student handbook and the assessment set.

35 The college is invited to suggest lines of enquiry that it would like the team to pursue. Colleges may find the Academic Infrastructure a useful point of reference in identifying lines of enquiry. The Code of practice, Section six: Assessment of students, may be particularly useful in identifying lines of enquiry for the first Developmental engagement on student assessment.

Self-evaluation

36 After the briefing, the college is required to submit a self-evaluation. The purpose of the self-evaluation is to describe the responsibilities which the college has within the area under review, making reference to its agreements with awarding bodies, and the processes and procedures it has adopted for discharging these responsibilities. The self-evaluation also provides the opportunity for critical self-reflection on the effectiveness of these processes and procedures.

37 Section six of this handbook provides further guidance on the self-evaluation. The self-evaluation should provide sufficient information and evidence against each of the headings set out in this guidance. If it does not, QAA may return it to the college for revision. QAA will invite students to provide a written submission and to send this to QAA at the same time as the college submits the self-evaluation. More information about the role of students is given below.

Preparatory meeting

38 The next stage is a preparatory meeting at the college between the college and the coordinator. This will take place about nine weeks before the visit. The purposes of the preparatory meeting are to discuss the arrangements for the Developmental engagement, to develop the agenda for the visit in light of the college's self-evaluation (and the student submission if one is made) and to identify further evidence for the college to make available during the visit. It is also an opportunity for the coordinator to clarify the process and for college staff to ask questions. Annex B provides an indicative programme for the preparatory meeting. QAA will provide the coordinator with a compendium of publicly available information, the information digest, to help him/her to prepare for this meeting.

Developmental engagement teams

39 The Developmental engagement team usually comprises four people: the coordinator, one reviewer and two nominees from the college under review. Where a college has more than 1000 students funded by HEFCE and/or more than four awarding bodies associated with its higher education provision, QAA may consider the inclusion of an additional reviewer.

40 The college nominees are full members of the team. All team members are responsible for analysing and commenting on the self-evaluation, conducting the visit, drafting sections of the report and contributing to the team's findings. More information about the composition of the team is provided in the glossary. QAA will provide each team member with a copy of the information digest to assist with preparation for the visit.

The Developmental engagement visit

41 The visit normally takes place over two consecutive days. Colleges with fewer than 100 full-time equivalent students funded by HEFCE may have a shorter visit or no Developmental engagement. Section five provides more details of an abridged method.

42 The purpose of the visit is to allow the team to pursue the lines of enquiry by scrutinising evidence on site and by meeting college staff, students and other stakeholders, including awarding bodies' representatives and employers, if appropriate. Annex C provides an indicative programme for the visit.

43 The team is led by the coordinator, who is responsible for ensuring that the team's activities are consistent with the method and that the visit is completed on time. The team is responsible for gathering, verifying and sharing evidence as a basis for testing the college's self-evaluation and the views of students. It is also responsible for considering the extent of the college's engagement with the Academic Infrastructure. The team will discuss the evidence, check its understanding and interpretation, refine its views and triangulate different sources of evidence to arrive at conclusions. It will keep notes of all meetings with staff and students, and of any emerging examples of good practice and areas for improvement; these will be circulated among the team and collated by the coordinator. Where concerns emerge in the course of the visit, the college will have the opportunity to provide additional evidence to address them.

44 Evidence comes in a wide range of forms and will vary from college to college. It is likely to include formal agreements with awarding bodies, policies and procedures for the management of the student learning expereince of higher education programmes, external examiners' reports, validation documents, and review and inspection reports by other organisations such as Ofsted, and any information arising from meetings with staff and students. The coordinator will agree with the college what evidence is required before the visit.

45 The team may also carry out direct observation of some elements of provision, for example learning resources. When a team requires evidence of teaching quality, this will usually be obtained from a documentary study of procedures such as the college's observation of higher education teaching, the analysis of student evaluation questionnaires and other arrangements for gathering feedback. Exceptionally, the team may carry out class observations.

46 At the end of the visit the team identifies areas of good practice and recommendations. The coordinator, accompanied by the other team members will make a short oral report about the team's findings to the college.

Role of students

47 The students' written submission is strongly encouraged but voluntary. If students are not able to make a submission, for whatever reason, this will not prejudice the outcome of the Developmental engagement.

48 At the visit itself the team will meet with at least one group of students. The college will normally be responsible for identifying students for this meeting within parameters agreed with the coordinator. The coordinator will confirm these parameters to the college at the preparatory meeting or at least eight weeks before the visit. Meetings with students are confidential and their views will not be attributed to individuals.

Reports

49 The Developmental engagement culminates in a report of the team's findings. The report has two main elements: good practice and recommendations for action by the college. In addition, the team will make evaluative comments about each of the three core themes. The Developmental engagement report will not be published, however, the Summative review report will include a short summary of the findings of the Developmental engagement team and this report will comment on the extent to which the action identified in the Developmental engagement action plan has been effective.

Good practice

50 An important element of Developmental engagement reports is good practice. In IQER, good practice is defined as practice that the team regards as making a particularly positive contribution to the college's management of the student learning experience of higher education in the context of that college, and which is worthy of wider dissemination within and/or beyond the college.

51 QAA will disseminate good practice identified in Developmental engagement reports through periodic reports and through collaboration with the Higher Education Academy and, where appropriate, with Foundation Degree Forward.

Recommendations

52 Recommendations for improving the college's management of its higher education provision are categorised as desirable, advisable or essential according to priority.

• Essential recommendations refer to important matters which the team believes are currently putting quality and/or standards at risk and which require urgent corrective action.
• Advisable recommendations refer to matters which the team believes have the potential to put quality and/or standards at risk and require preventative corrective action.
• Desirable recommendations refer to matters which the team believes have the potential to enhance quality, build capacity and/or further secure standards.

53 Exceptionally, recommendations may lead to the college having another Developmental engagement because they indicate that quality and/or standards are at risk. More information about risk criteria is provided in section five.

Action plan

54 QAA will send a draft of the Developmental engagement report to the college for its comments about four weeks after the visit. This gives the college and its awarding body(ies) the opportunity to draw the team's attention to any areas which they regard as inaccurate or incomplete. The team is then responsible for considering these comments and any supporting evidence which the college may offer at this stage, and for agreeing whether or not any aspect of the report should be amended in response. Teams will only be able to consider supporting evidence that was available at the time of the review visit.

55 The college is also asked at this stage to produce an action plan in response to the conclusions of the report. The action plan is intended to support the college in the continuing development of its higher education provision by describing how it intends to take the findings of the Developmental engagement forward. The action plan will be incorporated into the Developmental engagement report and will also form part of the evidence base for future review activity, particularly the Summative review. Once the team has considered and responded to the college's comments, a final version of the report is sent to the college, its awarding body(ies) and to HEFCE.

Figure 5 Key stages of the Developmental engagement

Developmental engagement: timeline

(Please note - during 2007-08 the timing of the events in stage 1 will differ from this published timeline. QAA will notify the colleges and awarding bodies and supply alternative times)

IQER stage	Time +/- visit	IQER process	What the college does	What QAA or the coordinator does	What the team does	How the awarding body is involved
1	-26 weeks	Preparation		QAA notifies the college of the review and proposed dates for the visit. QAA asks the college to identify nominee(s). QAA invites the college to attend a briefing event.		Notified by QAA of the review of its partner college. Invited by QAA to attend a briefing event.
	- 24 weeks		Arranges a discussion with the awarding body(ies) to agree the extent of awarding body involvement in the IQER process. Sends copy of agreement to QAA (or after briefing event).			Receives an invitation from the college to pre-briefing event discussion.
	-23 weeks		Confirms visit dates to QAA and identifies its nominee(s).	QAA notifies the college of the coordinator responsible for the review. The coordinator contacts the college to discuss the agenda and participants for the preparatory meeting.
	- 22 weeks		College and its awarding body(ies) hold discussions.			Takes part in discussion with the college about its involvement in IQER process.
	-18 weeks	Briefing event	Attends a briefing event. Organises the preparatory meeting and identifies college attendees including staff and students. Receives details of the team.	QAA provides a briefing event. QAA notifies the college and the awarding body of the proposed Developmental engagement team and the QAA officer responsible for the review. QAA asks the college to check the proposed team for conflicts of interests.		Attends briefing event, if this has been agreed in discussions with the college. Supports the college in the preparation of the self-evaluation and attends the preparatory meeting if these have been agreed in discussions with the college. Receives details of the proposed team from QAA and sends any comments to the college.
	-14 weeks		Collates comments on the proposed team and reports potential conflicts of interest to QAA.
2	-12 weeks	Self-evaluation	Sends the self-evaluation to QAA. Students send QAA their written submission, where appropriate. The college copies both documents to awarding body(ies).	The QAA officer and the coordinator scrutinise the self-evaluation, the proposed lines of enquiry, the partnership agreement(s) with higher education institution(s) and the student written submission. If the self-evaluation is suitable, QAA informs the college. If it is not suitable, a QAA officer will ask the college to revise it.		Receives the self-evaluation and any student written submission from the college.
3	-11 weeks	Analysis of the self-evaluation	Sends the self-evaluation to each member of the team.	The coordinator analyses the self-evaluation.	Analyses the selfevaluation and the supporting evidence and posts a report to ARCS, (QAA's secure electronic folder for this review) and includes suggestions for further evidence and the programme for the visit.
	-10 weeks			The coordinator produces a summary of team's analysis and sends it to the college.
4	-9 weeks	Preparatory meeting	Takes part in the preparatory meeting. Receives a letter confirming the arrangements for the visit.	The coordinator chairs the preparatory meeting. The coordinator sends the college, its awarding body(ies), the team and QAA a letter confirming the arrangements for the visit. The coordinator allocates areas of responsibility to each team member.		Takes part in the preparatory meeting, if this has been agreed in discussions with the college. Receives a letter from the coordinator confirming the arrangements for the visit.
5	- 1 week		Assembles further evidence in accordance with the team's requirements.
	0 week +2 days	Visit	Takes part in the Developmental engagement.	The coordinator chairs the Developmental engagement. At the end of the visit the coordinator gives an oral report of the findings of the Developmental engagement to the college.	Conducts the visit.	Attends meetings in accordance with agreements made in advance with the college.
	+1 week		Receives confirmation of the findings of the Developmental engagement.	The coordinator confirms the findings of the Developmental engagement to the college and its awarding body(ies).		Receives confirmation of the team's findings of the Developmental engagement from the coordinator.
6	+1 week	Reporting		The coordinator collates and edits the text and sends draft one to the team for comment.	Drafts text and posts it to ARCS.
	+2 weeks			The coordinator prepares draft two and submits it to QAA Reports team for proofing.	Comments on draft one of the report.
7	+4 weeks	Draft report to the college	Receives the draft report and action plan template.	The coordinator sends the third draft of the report to the head of the college and the awarding body(ies) for comments and provision of further evidence along with the action plan template.		Receives a copy of the draft report and action plan template from the coordinator.
8	+5 weeks		Checks the draft report for factual accuracy. Liaises with relevant staff to discuss and complete the action plan.			Sends any comments on the factual accuracy of the report to the college. Helps the college to develop the action plan, if this has been agreed in advance.
	+6 weeks		Collates and returns the report to QAA with comments on the factual accuracy of the report and submits further evidence (if appropriate).	Receives the college's response to the draft report.		Receives from the college a copy of the response to the draft report. Sends to QAA.
	+7 weeks				Considers the comments and further evidence (if appropriate) and posts responses to ARCS.
	+8 weeks		Returns the completed action plan to QAA, signed by the head of the institution.	The coordinator incorporates the team's response to the college's comments into the final draft report.
	+9 weeks			QAA combines the draft report and the action plan
9	+10 weeks	Report completed	Receives the final report for internal use only.	QAA sends the final report to the college and awarding bodies for internal use only and to HEFCE.		Receives the final report from QAA for internal use only.

Section four: Summative review

56 The purpose of this section is to state the aims and objectives of the Summative review and describe how it operates. This is summarised in a timeline at the end of this section.

 

Overview

57 The emphasis of the Summative review is on reviewing the college's management of the student learning experience of its higher education provision and making judgements about the effectiveness of its procedures. In common with the Developmental engagement, the Summative review takes full account of the varying roles of awarding bodies and is sensitive towards differences in those roles between different colleges. The Summative review differs from the Developmental engagement in five key respects:

• it covers all aspects of a college's management of its higher education provision
• it does not employ lines of enquiry proposed by the college
• there are no college nominees on the review team (although the college is invited to nominate a facilitator to act as a single point of contact between the college and the team)
• it leads to judgements about core themes one and two, and a conclusion about core theme three
• the Summative review report is published.

58 All colleges with HEFCE-funded higher education provision falling within the scope of IQER, as described in section one, will have one Summative review between 2007-08 and 2011-12, regardless of student numbers.

Preparation

59 In order to promote consistency and continuity, the coordinator for the Summative review will normally be the same as that for the Developmental engagement. About 22 weeks before the Summative review visit, the coordinator will contact the college and offer to hold a meeting at the college to renew contact and to discuss preparations for the Summative review. This contact is not a formal part of the review; its purpose is to refresh the college's understanding of the method and to provide further guidance on preparing a self-evaluation and helping students make a written submission.

Figure 6: The information flow between the range of external scrutiny processes and Summative review

Self-evaluation

60 The purpose of the self-evaluation for the Summative review is similar to that of the self-evaluation for the Developmental engagement: to describe the responsibilities which the college has for its higher education provision, making reference to its agreements with awarding bodies, and the processes and procedures it has adopted for discharging these responsibilities; and to provide a critical self-reflection on its approach. The self-evaluation for the Summative review differs in that it covers all aspects of the college's higher education provision. Section six provides further guidance on the self-evaluation for Summative review.

Preparatory meeting

61 The next stage is a preparatory meeting at the college between college staff and students, representatives of awarding bodies, if agreed between the partners, and the coordinator. The purpose of the preparatory meeting is broadly the same as the preparatory meeting for the Developmental engagement described in the previous section. An indicative programme for a Summative review preparatory meeting is provided in Annex B.

Summative review teams

62 The number of members of the Summative review team is normally four, the same as that for the Developmental engagement but the composition is different. It usually comprises the coordinator and up to three reviewers. Summative review teams do not include the college's own nominees, although the college is invited to nominate a facilitator to act as a single point of contact between the college and the team during the visit and beforehand. QAA advises that one of the nominees becomes the facilitator to provide continuity between the IQER stages. More information about the role of the facilitator is provided in the glossary in section seven.

63 Where a college has more than 1000 students funded by HEFCE and/or more than four awarding bodies associated with its higher education provision, QAA may consider the inclusion of an additional reviewer in the Summative review team.

The Summative review visit

64 The visit normally takes place over two consecutive days. Colleges with fewer than 100 full-time equivalent students funded by HEFCE may have a shorter visit or no visit at all depending on the numbers of students. Details of abridged methods are set out in section five.

65 The purpose of the visit is broadly the same as that for the Developmental engagement: to allow reviewers to scrutinise evidence on site, meet college staff, students and other stakeholders and consider the extent of the college's engagement with the Academic Infrastructure. The responsibilities of the reviewers at the Summative review visit are very similar to those of the Developmental engagement team. The role of the coordinator, however, becomes one of leadership and facilitation and the coordinator does not make judgements. Annex C provides an indicative programme for the visit.

Role of students

66 The Summative review again seeks to identify students' views of their education both before and during the visit. The mechanisms for identifying these views are consistent with those at the Developmental engagement and the coordinator is again responsible for discussing with the college methods of securing a students' written submission, which remains voluntary. Teams will also expect the college's self-evaluation to explain how the college ensures that students' views inform the management of its higher education programmes.

67 During the visit the team will meet with at least one group of students. The arrangements for this meeting reflect those at the Developmental engagement, as described in section three.

Provisional judgement meeting

68 The Summative review team meets again normally one week after the visit to agree summaries of evidence and to make provisional judgements about the college's management of its responsibilities for academic standards and the quality of learning opportunities, and to evaluate the effectiveness of the college's procedures for ensuring the accuracy and completeness of the information it is responsible for publishing. The provisional judgements are confidence, limited confidence or no confidence.

• Where a college is found to be managing effectively its responsibilities for the delivery of the academic standards and the quality of its higher education provision; the prospects for the future continuation of this appear good, and it has rigorous mechanisms for the management of its HEFCE-funded programmes in accordance with awarding bodies' requirements, the provisional judgement will be confidence.
• Where significant concerns exist about aspects of a college's current or likely future management of quality and/or delivery of the standards of its HEFCE-funded higher education programmes, the provisional judgement will be limited confidence.
• Where major concerns exist about significant aspects of a college's current or likely future capacity to secure and maintain quality and/or deliver standards of its HEFCE-funded programmes, the provisional judgement will be no confidence.

69 Judgements and conclusions will always be made with due reference to the requirements of the college's awarding body(ies).

70 Further details of the criteria for making judgements are set out in full under 'Confidence', 'Limited confidence' and 'No confidence' in the glossary in section seven. Differentiated judgements can be made where a team regards a college's management of the standards and/or quality of the programmes of study of one awarding body to be substantially different from those of another.

71 For core theme three the team will reach a provisional conclusion. The provisional conclusion will be whether or not reliance can be placed on the accuracy and completeness of the information which the college publishes about itself.

• Where a college recognises all of the information it is responsible for publishing within the area under review; has rigorous mechanisms for the management of these responsibilities, which ensure that the information it publishes is both accurate and complete; and has evidence that this is the case, then the conclusion will be that reliance can be placed on the accuracy and completeness of all of the public information which the college is responsible for publishing.
• Where a college does not recognise all of the information it is responsible for publishing and/or where there is evidence that this information is inaccurate and/or incomplete, then the conclusion will be that reliance cannot be placed on the accuracy and completeness of all the public information which the college is responsible for publishing.

72 A differentiated conclusion may be reached where a team regards the information as accurate but not complete or vice-versa. Further information about how teams arrive at this conclusion is provided in the glossary in section seven and in annex D.

73 At the provisional judgement meeting the team will also identify good practice and provisional recommendations. These share common definitions with those for the Developmental engagement. When essential recommendations are made they are likely to be reflected in a provisional judgement of limited confidence or no confidence against core themes one and/or two, and/or a conclusion that reliance cannot be placed on the accuracy and completeness of all the public information which the college is responsible for publishing about itself.

74 The coordinator will inform the college and its awarding bodies in writing about the outcome of the provisional judgement meeting within one week of the meeting. All judgements, good practice and recommendations remain provisional until the report is finalised.

Reports

75 Following the provisional judgement meeting, the team is responsible for writing a report of its findings. The report sets out the provisional judgements, good practice and recommendations and actions by the college as described above, together with contextual information and supporting evidence.

76 The coordinator will send a draft version of the report to the college and its awarding body(ies) for comment around four weeks after the end of the visit. This gives the college the opportunity to draw the team's attention to any areas which it regards as inaccurate or incomplete and, if necessary, submit additional evidence. Teams will only be able to consider supporting evidence that was available at the time of the review visit. The team is then responsible for considering the college's comments and any supporting evidence which the college may offer at this stage, and agreeing whether or not any aspect of the report, including the provisional judgements, should be amended in response.

77 The college is also asked at this stage to produce an action plan to accompany the report. The action plan describes how the college intends to take forward the reviewers' findings and the effectiveness of the action taken will form part of the evidence base for any future review activity and constitute a published record of the college's commitment to developing its provision. QAA will provide a standard format for the action plan.

78 Normally, once the team has considered, and responded to, the college's comments, it will confirm the judgements, QAA will set out these judgements in writing to the college and the awarding body(ies) and QAA will send a final version of the report to the college, its awarding bodies and to HEFCE. The final report is subsequently published on the QAA website 15 weeks after the end of the visit.

Figure 7: Key stages of a Summative review with one visit

Second visits and follow-up action

79 After the college has commented on the draft report and had the opportunity to submit further evidence, if the team continues to have limited or no confidence in either or both of the first two core themes, or considers that reliance cannot be placed on the accuracy and/or completeness of the information the college published about itself, then the team will conduct a second visit to the college to review further evidence. This will take place as soon as possible after the first visit, at the discretion of QAA. The team for the second visit will normally be the same as for the first. A second visit will also take place should a team be unable to reach provisional or confirmed judgements after the first visit. QAA considers it highly likely that the awarding body(ies) will wish to take part in a second visit and will ensure that QAA staff copy to the awarding body(ies) all correspondence from QAA to the college about the second visit.

80 If the second visit results in judgements of confidence for core themes one and two plus the conclusion that reliance can be placed on the accuracy and completeness of the information the college is responsible for publishing about itself, the review is completed following the process set out in paragraphs 77 and 78. However, should the second visit result in a judgement of limited confidence in either or both core themes one and two and/or the conclusion that reliance cannot be placed in the accuracy and/or completeness of published information, the report is published and there will be a formal programme of follow-up action, which must be completed by the college within 18 months of the publication of the review report.

81 If the second visit results in a judgement of no confidence in either or both of core themes one and two, and if the college and its awarding body(ies) have cause to disagree with these outcomes, they may consider using the QAA procedure on representations. If one or both of the judgements are confirmed as no confidence, the report will be published and there will be a formal programme of follow-up action. If after this time QAA is still concerned about the effectiveness of the remedial action, QAA may conduct a further visit; and if satisfactory progress has still not been made, the matter will be referred to HEFCE, which reserves the right to withdraw some or all of its funding.

Figure 8: Key stages of a Summative review which results in a provisional and/or confirmed judgement of limited or no confidence in one or both of core themes one and two

Summative review timeline (where provisional judgements are 'confidence' for core themes one and two and the conclusion for core theme three is that reliance can be placed on the accuracy and completeness of all of the public information which the college is responsible for publishing).

IQER stage	Time +/- visit	IQER process	What the college does	What QAA or the coordinator does	What the team does	How the awarding body is involved
10		Preparation
	-26 weeks			QAA notifies the college of the proposed date for the visit. QAA asks the college to identify the facilitator.		Receives notification from QAA of the review of its partner college.
	-24 weeks		Confirms the facilitator. Responds to the coordinator's offer of an informal meeting.	The coordinator contacts the college to offer an informal meeting to discuss the preparation for the review. The coordinator copies this invitation to the awarding body.		Responds to the college about its possible involvement in an informal meeting with the coordinator.
	-23 to -14 weeks (optional activity)		Takes part in an informal meeting with the coordinator if the college requires this meeting.	The coordinator takes part in an informal meeting with the college if the college requires this meeting.		Takes part in an informal meeting with the college and the coordinator, if this has been agreed with the college.
	-22 weeks			The coordinator contacts the college to discuss the agenda and the participants of the preparatory meeting.
	-14 weeks		Organises the preparatory meeting and identifies college participants, including staff and students.	QAA notifies the college of the review team and the QAA officer responsible for the review. QAA asks the college to check the proposed team for conflicts of interests.		Receives details of the preparatory meeting from the college and the coordinator and arranges to attend if this has been agreed with the college. Receives details of the proposed review team and offers comments to the college on the proposed team membership.
	-13 weeks		Reports potential conflicts of interest to QAA.
11	-12 weeks	Self-evaluation	Submits the self-evaluation to QAA. Students submit written submission, if appropriate.	A QAA officer and the coordinator scrutinise the self-evaluation, the partnership agreement for HEI awards and the student written submission, if provided. If the self-evaluation is suitable, QAA informs the college of this. If it is not suitable, a QAA officer will ask the college to revise it.
12	-11 weeks	Analysis of the self- evaluation	Sends the self-evaluation and the optional student written submission to each member of the IQER team and to the awarding body(ies).	The coordinator analyses the self evaluation.	Analyses the selfevaluation and the supporting evidence and reports through ARCS, including suggestions for further evidence and the programme for the visit.	Receives a copy of the approved self evaluation and any optional student written submission from the college.
	-10 weeks		Receives a summary of the team's analysis of the self-evaluation.	The coordinator produces a summary of the team's analysis and sends it to the college.		Receives a copy of the summary of the team's comments on the selfevaluation from the coordinator.
13	-9 weeks	Preparatory meeting	Takes part in the preparatory meeting.	The coordinator chairs the preparatory meeting. The coordinator sends the college, its awarding body(ies), the team and QAA a letter confirming the arrangements for the visit.		Attends the preparatory meeting, if this has been agreed in advance with the college.
	-2 weeks			The coordinator allocates areas of responsibility to each team member.	Receives a briefing and the allocation of areas of responsibility for the review.
	-1 week		Assembles evidence in accordance with the team's requirements.
14	0 week	Visit	Takes part in the Summative review visit.	The coordinator chairs the Summative review visit. At the end of the visit the coordinator offers a verbal update on the progress of the review to the college facilitator.	Conducts the visit. Accompanies the coordinator to provide a verbal update on the progress of the review.	Takes part in the review visit in accordance with agreements with the college made in advance.
15	+1 week	Provisional judgement meeting	Receives a letter from the coordinator setting out the provisional judgements.	The coordinator chairs the provisional judgement meeting. The coordinator sends a letter to the college, copied to the awarding body (ies), setting out the provisional judgements.	Meets with the coordinator to agree summaries of evidence, provisional judgments, good practice and recommendations.	Receives from the coordinator a copy of the letter setting out the provisional judgements.
16	+1 week	Report writing		The coordinator collates and edits the text and sends the first draft report to the team for comment.	Drafts report text and posts it on ARCS.
	+2 weeks			The coordinator prepares draft two of the report and submits it to QAA Reports team.	Comments on draft one of the report.
17	+4 weeks	Draft report to the college	Receives the draft report.	The coordinator sends the third draft of the report to the head of the college and to the awarding bodies for comments and for the provision of further evidence. This report draft includes the action plan template.		Receives a copy of the draft report from the coordinator.
	+5 weeks		Checks the draft report for factual accuracy and identifies any additional evidence it needs to submit. Liaises with relevant staff to discuss and develop the action plan.			Provides any comment on the draft report to the college to collate into one response to the coordinator on the draft report.
	+8 weeks		Collates all comments on the draft report and sends one set of comments on factual accuracy to QAA and provides additional evidence, if appropriate.	Considers the college's (and its awarding body comments) and any further evidence submitted and posts responses to ARCS. Confirms judgements of confidence or agrees that a second review visit to the college is needed.
	+9 weeks		Receives a copy of the letter setting out either the final judgements of confidence or notification that a second visit is required. Returns the completed action plan to QAA.	QAA confirms either the final judgements of confidence to the college and its awarding bodies or confirms that a second visit is required.		Receives from QAA either a copy of the letter of final judgements of confidence or notification that a second visit to the college is required.
	+10 weeks
	+15 weeks		Report publication	QAA publishes the report.		Contributes to the development of the action plan, if this has been agreed in advance with the college.

Timeline for a Summative review second visit.

IQER stage	Time +/- visit	IQER process	What the college does	What QAA or the coordinator does	What the team does	How the awarding body is involved
14	-1 week min	Second visit	Receives copies of all correspondence from QAA and the coordinator and prepares for the second visit.	The coordinator confirms the arrangements for the second visit and agrees the agenda with the college and its awarding body(ies).	Receives copies of all correspondence from QAA and the coordinator and prepares for the second visit.	Receives copies of all correspondence from QAA and the coordinator and prepares for the second visit.
	0 weeks		Takes part in the second visit to the college.	The coordinator chairs the second visit to the college.	Visits the college to conduct the second visit.	Takes part in the second visit to the college, if the awarding body considers this necessary.
15	+2 days		Receives the oral report and the judgements.	The coordinator provides an oral report and the judgements to the head of college and his/her nominated staff before the team leaves at the end of the visit.	Accompanies the coordinator to provide the oral report and the judgements to the college staff.	Receives the oral report and the judgements, if in attendance at visit.
16	+1 week	Report writing	Receives a letter from QAA confirming the review judgements.	QAA sends a letter confirming the outcomes to the college and copies this to the awarding body(ies). The coordinator collates and edits the team's text, and sends draft one of the report to the review team for finalisation.	Drafts report text and post it on ARCS. Comments on draft one of the report.	Receives a copy of the letter from QAA confirming the review judgements.
	+2 weeks			The coordinator prepares draft two of the report and submits it to the QAA Reports team for proofing.
17	+4 weeks		Receives the draft report from the coordinator.	The coordinator sends the third draft of the report to the head of the college and copies it to the awarding body(ies) for comment.		Receives the draft report from the coordinator.
	+5 weeks	Draft report to the college	Checks the draft review report for factual accuracy. Liaises with the relevant staff to discuss and complete the action plan.			Provides any comments on the draft report to the college. Contributes to the college's development of the plan of action to address any limited or no confidence judgement(s) that relate to its awards.
	+8 weeks		Returns collated comments on the factual accuracy of the report.		Considers the college's and the awarding body(ies)' comments and posts responses to ARCS. Confirms the review evaluation and judgements.
	+10 weeks		Returns the completed action plan to QAA, signed by the head of college.	The coordinator completes draft four of the review report.		Receives a copy of the completed action plan from the college.
18	+15 weeks	Report published		QAA combines the draft report and the completed action plan and publishes it only if the judgements are confidence and/or limited confidence. If one or more judgements are of no confidence, the report remains draft until the end of any representation the college and its awarding body(ies) raise. At the end of this period the judgements are considered and QAA finalises and publishes the report.
19		Action plan	Implements the action plan with its awarding body(ies).			An awarding body may wish to help the college to implement its action plan, particularly if the limited or no confidence judgement(s) relate to its award(s).
20	Within 18 months of publication of the report	Follow up only if limited or no confidence judgements	If the judgements are limited or no confidence, the college takes part in formal follow-up action to demonstrate its progress in implementing the action plan and the impact of this action on the students' education.	If the judgements are limited or no confidence, QAA carries out a formal programme of follow-up action to check the college's progress in implementing the action plan and the impact of this action on the students' education.		An awarding body takes part in formal follow-up to any limited or no confidence judgement(s) related to its award(s).

Section five: Adaptations to IQER

82 QAA recognises that the provision of higher education in colleges is characterised by an array of different types of delivery, sizes of provision, and funding and awarding relationships. QAA also recognises that IQER is being introduced at a time when this provision is changing rapidly in response to a number of factors, including government policy and the needs of students. QAA has, therefore, designed IQER so that it engages with the higher education provision in each college on its own merits. IQER has no preferred model for higher education provision, other than that it expects that any model must permit the awarding body to assure itself about the standards and quality of its collaborative provision with colleges. QAA has made three adaptations to the application of IQER method to reflect: the size of the provision; collaborative modes of management and delivery; and situations where provision is undergoing significant changes. These adaptations are described in the following paragraphs. In all cases, QAA will involve awarding bodies in discussions alongside colleges.

Abridged method

83 The external evaluation of the IQER pilot confirmed QAA's view that the costs to colleges of engaging in IQER are proportionately higher for those with a small higher education provision. In response QAA has developed an abridged method for colleges with fewer than 100 full-time equivalent students funded by HEFCE to ensure that the cost and workload associated with their review remains in proportion to the costs and workload borne by colleges with larger numbers of students. As with larger colleges, IQER reviews will result in a published Summative review report: some colleges may also elect to take part in a Developmental engagement.

Abridged Developmental engagement

84 Colleges with fewer than 100 full-time equivalent students funded by HEFCE may opt for a Developmental engagement but will not be required to have one unless they meet one of the risk criteria set out in paragraph 97. Where such a request is agreed by QAA, the abridged Developmental engagement will apply. This will normally involve:

• a three person review team: the coordinator, a reviewer and the college's nominee. QAA will provide two reviewers if a college is unable to provide a nominee
• a shortened visit. QAA will determine the duration of the visit according to student numbers as follows:
  • where the number of full-time equivalent students funded by HEFCE is fewer than 100 but more than 49, the Developmental engagement visit will normally take place over one and a half days or two days depending on the complexity of the provision
  • where the number of full-time equivalent students funded by HEFCE is fewer than 50, the Developmental engagement visit will normally take place over one day or one and a half days depending on the complexity of the provision.

Abridged Summative review

85 All colleges with HEFCE-funded higher education provision falling within the scope of IQER described in section one will be required to take part in one Summative review between 2007-08 and 2011-12, regardless of student numbers. QAA has, however, made adaptations to the process of Summative review for colleges with fewer than 100 full-time equivalent students funded by HEFCE to ensure that the review process remains proportionate. However, colleges which receive essential recommendations in a Developmental engagement are not eligible for an abridged Summative review. This is because essential recommendations indicate that quality and/or standards of the higher education provision at the college are at risk.

86 Where the number of full-time equivalent students funded by HEFCE is fewer than 100 but more than 49, the Summative review visit will normally take place over one and a half days and it will be conducted by a team of three: a coordinator and two reviewers.

87 For colleges with fewer than 50 full-time equivalent students funded by HEFCE, QAA will not automatically arrange a review visit. Instead the Summative review team will carry out a desk-based analysis of the college's self-evaluation and will draft the Summative review report based on this analysis. The report will be sent to the college and its awarding body(ies) for comments on factual accuracy and for the submission of further evidence following the standard process described in section four. Should the team be unable to complete the report from the evidence available, or if the team reaches provisional judgements of limited or no confidence, QAA will arrange for the team to visit the college for one day or one and a half days to complete the review. In such cases, the IQER team will normally comprise three members: the coordinator and two reviewers.

88 Exceptionally, under either of these adaptations to the Summative review, if QAA and the team consider that there is insufficient evidence to support the formulation of provisional judgements or the production of a report, or if the team reaches provisional judgements of limited or no confidence, QAA will arrange for the team to return to the college for a second visit. The second visit will normally take place over one day.

Adaptations for colleges in HEFCE-funded consortia, in lifelong learning networks and for higher education centres

89 The primary unit of IQER reviews is normally the individual college. QAA recognises, however, that some colleges manage and deliver their higher education provision through HEFCE-funded consortia: collaborative relationships with other colleges and awarding bodies. QAA has designed IQER to reflect such relationships by allowing adaptations to both the Developmental engagement and Summative review where it is feasible and appropriate to do so.

90 Possible adaptations may include:

91 QAA will also be willing to consider allocating two or three coordinators to lead the reviews of all the colleges in a consortium and also to have some overlap in the teams of reviewers involved. Such adaptations may help a consortium to review and enhance its management arrangements in a manner which reduces the potential duplication of involvement for the management team and the awarding body(ies) in HEFCE-funded consortia. These examples are not intended to be exhaustive and QAA would be willing to discuss other adaptations suggested by colleges and their awarding bodies in HEFCE-funded consortia, provided they retain the core elements of the Developmental engagement and Summative review described in sections three and four respectively.

92 QAA officers will contact the management team of each HEFCE-funded consortium in 2007-08 to discuss the possibility of adapting IQER for its members. In order to provide sufficient time to do this, members of HEFCE-funded consortia will not be reviewed in 2007-08. The adaptations described above only apply to HEFCE-funded consortia and QAA would welcome approaches from colleges involved in other types of consortia, such as lifelong learning networks, to discuss possible adaptations for them.

93 HEFCE is also supporting the development of a number of higher education centres, most of which draw on the provision of local colleges. As the status and constitution of these higher education centres varies from centre to centre, QAA will contact each centre individually in 2007-08 to discuss how IQER will apply to them. Higher education centres will not be scheduled for a review visit in 2007-08 to provide sufficient time to tailor the application of IQER to each centre.

Arrangements for colleges undergoing significant changes to their higher education provision

94 IQER teams consider colleges' present and likely future management of higher education based on evidence about processes and procedures in place at the time of the review. However, there will inevitably be cases where existing processes and procedures provide limited evidence about the future management of the college's higher education provision, particularly where a college is undergoing significant growth and/or changes to its awarding bodies. In such cases, QAA believes that limiting IQER to a review of existing processes and procedures would also limit the value of the process to the college, its awarding bodies and its students.

95 Normally, any adaptations to the method made to reflect significant change will be confined to a review of existing evidence about the college's future plans. Exceptionally, and again only with the agreement of the college, its awarding bodies and HEFCE, QAA may adjust the timing of the visit so that more evidence about the college's plans is available. Requests for such adaptations must be made in writing by the college to QAA.

How QAA determines the number of Developmental engagements for each college

96 All colleges with 100 or more full-time equivalent students on HEFCE-funded programmes will take part in at least one Developmental engagement. A small number of colleges will be asked to take part in two Developmental engagements and exceptionally in a third, because they have recently received unsatisfactory reports from either QAA or Ofsted and/or their partner awarding body has a judgement of limited or no confidence in the audit of its collaborative provision and which relates to the partner awarding body's management of its collaborative provision in colleges. This reflects QAA's commitment as a signatory to the Higher Education Concordat on quality assurance arrangements and data collection to target our activities according to risk.

97 Where a college meets one or more of the following criteria it will be scheduled for two Developmental engagements between 2007-08 and 2011-12:

• no confidence and/or a failing judgement in a QAA Academic review since January 2002, not redressed through a satisfactory re-review of the relevant provision
• no confidence and or failing judgements in two or more QAA academic reviews since January 2002 irrespective of the outcomes of re-review
• no confidence judgement in a review of a Foundation Degree in the academic year 2004-05 including a Foundation Degree consortium in which the college delivers the Foundation Degree under review
• a judgement of no confidence or limited confidence from an audit of a college's partner higher education institution's collaborative provision, where it is clear that the evidence for the judgement relates in whole or in part to the management of higher education provision in colleges in receipt of HEFCE funding
• a grade of 4 in leadership and management in an Ofsted college inspection not redressed through a satisfactory re-inspection of the relevant provision
• a reported 'cause for concern' with regard to higher education provision in a college, that has been investigated by QAA and resulted in a requirement for action to address the issues raised.

98 In exceptional circumstances a college may be asked to participate in a third Developmental engagement. This will be rare and only if:

• a college has essential recommendations in two separate Developmental engagements or in one Developmental engagement and the Summative review, or
• a 'cause for concern' has been raised about a particular college in accordance with QAA's published policy on 'causes for concern' and has resulted in a required plan of action.

99 Where a college has two or more Developmental engagements, QAA will normally spread them across the cycle to give the college and its awarding body(ies) time to reflect and act upon the findings of the first before taking part in a second Developmental engagement or the Summative review. Colleges may, however, ask to have two Developmental engagements concurrently or close together.

100 Where a new college is formed by merger or acquisition, QAA will determine the number of Developmental engagements for that college in discussion with HEFCE, the college and its awarding bodies and by reference to the experiences of its predecessors in external scrutiny by QAA and Ofsted.

How QAA determines when each Developmental engagement and Summative review will take place

101 QAA is responsible for conducting Developmental engagements and Summative reviews in about 270 colleges from 2007-08 to 2011-12. To help determine when they will occur for each college, QAA has adopted the criteria set out below. It is not possible on the basis of these criteria for a college and its awarding body(ies) to identify exactly when a particular review will take place, however, they are included here to give some indication of the likely timing of the reviews. QAA will always try to give colleges at least 26 weeks notice in writing of a Developmental engagement or Summative review visit; and there will be a minimum of one year between a Developmental engagement and the Summative review visits. Colleges also have the opportunity to negotiate the timing of reviews to fit in with external reviews by other bodies.

102 In determining when particular visits should take place, QAA will consider:

• the total number of reviews required for each college. Colleges with two Developmental engagements should have the first one early in the review cycle to give time to reflect and act upon the findings of the first before taking part in the second Developmental engagement and the Summative review (unless they have chosen concurrent Developmental engagements)
• previous QAA reviews. Colleges with substantial experience of QAA reviews, such as Academic review, will tend to fall later in the cycle (notwithstanding the number of Developmental engagements required); colleges with less experience will tend to fall earlier. However, QAA will not schedule colleges without any experience of QAA in the first year, 2007-08
• burden on awarding bodies. Where an awarding body makes awards at a large number of colleges, QAA will try to distribute those colleges across the cycle so that the awarding body is not involved in too many reviews in any one year
• possible adaptations to the method for colleges in consortia. QAA will avoid review visits to any members of HEFCE-funded consortia, or colleges involved in higher education centres, in 2007-08 to provide sufficient time to discuss with these colleges and their awarding bodies about possible adaptations to the method
• resources. QAA will distribute the reviews evenly across and within the review cycle to reflect the resources QAA has available to carry out reviews.

Section six: Guidance to colleges on preparing self-evaluations

103 Both the Developmental engagement and the Summative review stages of IQER are based on a college's self-evaluation. The purpose of the self-evaluation is to describe the responsibilities that the college has for the management of the student learning experience of its higher education provision, making reference to its agreements with awarding bodies, and the processes and procedures it has adopted for discharging these responsibilities. The self-evaluation also provides the opportunity for the college to reflect critically on the effectiveness of these processes and procedures.

104 Guidance is provided below for colleges on the structure and content of self-evaluations. This should be regarded as guidance, rather than a template or prescription, since each college has different responsibilities reflecting individual agreements with awarding bodies.

105 An effective self-evaluation is key to a college gaining substantial benefit from IQER and to the smooth running of IQER. QAA, therefore, encourages colleges to give its preparation due time and attention. QAA will return the self-evaluation to the college for further work if it does not enable the IQER team to identify the college's responsibilities and understand how these responsibilities are discharged. The preparation of a self-evaluation is a major focus of the briefing that QAA will arrange for colleges and their awarding bodies.

Self-evaluation: Developmental engagement

106 The self-evaluation for the Developmental engagement should focus on the area under review for all three core themes. The following guidance is based on the area of student assessment because this is the focus of the first Developmental engagement for every college. For colleges with more than one Developmental engagement, the structure of the self-evaluation for subsequent Developmental engagements will be similar, although the content may be quite different. The coordinator will be able to provide further guidance on self-evaluations for colleges with more than one Developmental engagement.

Structure

107 The self-evaluation for the Developmental engagement should be structured according to the lines of enquiry, which the college agrees with the coordinator following the briefing and before the preparatory meeting. The self-evaluation should also include an introduction to the college, including details of agreements with awarding bodies. Within each line of enquiry the college should explain how the college manages its responsibilities with respect to the three core themes of academic standards, quality of learning opportunities and public information. For example, if the line of enquiry is the contribution of work placements to final assessments, the college might explain, under the quality of learning opportunities, how it supports students in work placements to enable them to generate information or evidence for final assessments; or, under public information, what it says about the contribution of work placements to final assessments in its programme handbooks. Some lines of enquiry may not apply to all core themes.

Content

108 In order to limit the burden of the IQER exercise, colleges should as far as possible discuss their responsibilities, processes and procedures in a portfolio of documents which form the self-evaluation. Any new material should be limited to that which is required to signpost and/or contextualise the material in the portfolio and other existing material for the team. New material should not normally exceed four pages of A4. Returning to the example of the contribution of work placements to final assessment, the college may regard its existing policies on work placements as providing a comprehensive description of the college's approach. If so, then it may simply need to include these policies in the self-evaluation portfolio and make reference to them in a short covering document together with evidence of their effectiveness. The following guidelines provide details of the evidence that the college might consider including or referencing in the self-evaluation.

109 The self-evaluation also provides colleges with an opportunity to reflect critically on the effectiveness of its processes and procedures for managing higher education. Any material provided should be limited as far as possible to that which already exists, for example, a report on a recent internal review of higher education provision or a recent periodic review carried out by an awarding body. The length of a college's Developmental engagement self-evaluation depends on the college's level of responsibility, the lines of enquiry and the quality and comprehensiveness of existing written evidence. Reviewers will value a self-evaluation in which they can find the material they need readily; it will therefore be important for the college to consider the reviewers' needs when compiling the self-evaluation.

Indicative structure of a self-evaluation for Developmental engagement in student assessment

Introduction to the college and its awarding body(ies)

Suggested content:

• mission statement
• brief description of the college, its location, size and higher education programmes listed with their awarding bodies
• prospectus
• data showing three year student enrolment and progression across each programme
• higher education annual monitoring reports.

Details of the college's responsibilities for higher education

Suggested content:

• formal partnership agreement(s) with higher education institution(s)
• Edexcel standard note.

Outline of recent changes affecting higher education provision in the college

Suggested content:

• brief description of recent changes to higher education programmes
• brief description of changes to higher education programmes planned
• higher education strategy (when this policy has been implemented by HEFCE).

Possible sources of evidence or references:

• college strategic plan
• higher education annual reports.

Outline of assessment policy and practice

Suggested content:

• Use of the Academic Infrastructure
• Effectiveness of procedures for:
• programme and assessment planning
• programme internal validation
• the regulation and operation of assessment
• marking
• assessment moderation and verification
• staff development and training in assessment
• assessment record keeping
• feedback to students
• the review of assessment regulations.

Possible sources of evidence or references could include some of the following, depending on the extent of the college's responsibilities for standards in the college's agreement with each awarding body:

• assessment policy
• assessment planning process
• assessment design
• assessment tasks
• assessment criteria
• assessment records
• assessment review and evaluation reports
• action taken on receipt of external review or inspection reports
• programme specifications
• information for students
• minutes of formal meetings to consider students' achievement and academic progression.

Lines of enquiry

The lines of enquiry in this case could be, for example:

• the match between intended learning outcomes, the assessment set and marking practice
• the quality of the feedback to students on formative and summative assessment
• the match between the information on student assessment in the programme specification, in publicity and promotional material and in assessments set.

Possible sources of evidence or references could include some of the following, depending on the extent of the college's responsibilities for standards in the college's agreement with each awarding body:

• quality assurance policy
• monitoring and review processes
• resource policy
• admissions policy
• accreditation of prior learning policy
• student support policy
• student assessment policy
• external examiner or verifier reports
• action taken on receipt of external review or inspection reports
• teaching and learning strategy
• management structure
• meeting structure
• staff development policy
• college and awarding bodies regulations for progression
• statistical records
• programme specifications
• national student survey results (if appropriate)
• information for staff managing and delivering higher education
• promotional material
• information for students
• action plans resulting from:
  • awarding body reports
  • periodic review reports
  • internal reports
  • external examiner or verifier reports
  • evidence from reviews at subject level, including professional statutory and regulatory body (PSRB) accreditation
  • LSC strategic area reviews
  • Ofsted inspections
• the use of the Academic Infrastructure.

Self-evaluation: Summative review

110 The self-evaluation for the Summative review should address all aspects of the college's management of its higher education provision.

Structure

111 The self-evaluation for the Summative review should be structured according to the three core themes: academic standards, quality of learning opportunities and public information. The self-evaluation should also include an introduction to the college, with details of any agreements with awarding bodies. This section may be repeated from the Developmental engagement, but it is important to cover these arrangements again to reflect any changes since the Developmental engagement and because the Summative review team usually has two reviewers who will not have been involved in the Developmental engagement. Also it will be important for colleges to bear in mind that the Summative review team will receive only the final Developmental engagement report from QAA and not the full set of evidence on which the Developmental engagement conclusions were based. Within each core theme the college should describe the responsibilities which its awarding bodies have conferred on it and explain the processes and procedures it has adopted for discharging these responsibilities. The self-evaluation also provides colleges with an opportunity to reflect critically on the effectiveness of these processes and procedures.

Content

112 As with the Developmental engagement self-evaluation, colleges should as far as possible describe their responsibilities, processes and procedures by compiling a portfolio of existing documents, with any new material strictly limited to what is required to signpost and/or contextualise existing material for the team. Further guidance on the composition of this portfolio is provided below. It is not possible to provide definitive guidance on the length of the self-evaluation for the Summative review. The length depends on the college's level of responsibility and the quality and comprehensiveness of existing written evidence. However, where a college feels confident in relying on a portfolio of existing evidence about the management of its higher education provision, it may be able to restrict any new material to four sides of A4.

Indicative structure of a self-evaluation for Summative review

Introduction to the college and its awarding bodies

Suggested content:

• mission statement
• brief description of college, its location, size and higher education programmes listed against their awarding bodies
• prospectus
• data showing three years of student enrolment and progression across each programme
• higher education annual monitoring reports
• higher education strategy (when HEFCE has implemented this policy).

Details of the college's responsibilities for higher education

Suggested content:

• formal partnership agreement(s) with higher education institution(s)
• Edexcel standard note.

Outline of recent changes affecting higher education provision in the college

Suggested content:

• brief description of recent changes to higher education programmes
• brief description of changes to higher education programmes planned.

Possible sources of evidence or references:

• college strategic plan
• higher education annual report
• higher education strategy (when HEFCE has implemented this policy).

Academic standards

Suggested content (this will vary in accordance with the extent of each college's commitments as set out in agreements with awarding bodies):

• responsibilities for managing standards
• reporting structure
• terms of reference for committees
• effectiveness of the arrangements for curriculum development
• effectiveness of the arrangements for internal validation
• effectiveness of the arrangements for student assessment
• analysis of student progression and achievement
• effectiveness of programme monitoring and evaluation
• use of the Academic Infrastructure.

Possible sources of evidence or references:

• quality assurance policy
• monitoring and review processes
• admissions policy
• accreditation of prior learning policy
• student assessment policy
• management structure
• meeting structure
• internal validation processes
• college and awarding bodies regulations for progression
• action taken on receipt of external review or inspection reports
• statistical records
• programme specifications
• PSRB accreditation reports
• student complaints and appeals processes
• college