Major Review Trends Fact Sheet (2004-05)

in partnership with

This fact sheet is designed for practitioners, students, managers and academics involved in healthcare education, to learn about the key findings of external reviews. The fact sheet is a concise summary from the main report which is available at www.qaa.ac.uk/health/majorreview/reviewTrends05/contents.asp

The Quality Assurance Agency for Higher Education (QAA) was awarded the contract by the Department for Health (England) (DH) and its partners, to develop, implement and manage a cycle of Major Review for 2003-06.

Major Review is the periodic review process of NHS-funded healthcare education in England that looks at academic and practitioner standards and the quality of learning opportunities for students. The review method was developed in partnership with the DH, Skills for Health (SfH)¹, the Nursing and Midwifery Council (NMC), the Health Professions Council (HPC), the strategic health authorities (SHAs)/workforce development confederations (WDCs) and, additionally, with input (through the Major Review working and steering groups) from higher education institutions (HEIs), NHS Trusts, the voluntary sector and the independent sector.

Each year QAA produces and publishes a report on the emerging trends that have come out of the most recent completed cycle of major reviews - the Annual Review Trends Report. The second report looks at 28 Major Reviews across 15 disciplines that took place between spring 2004 and spring 2005², about one-third of the total number of reviews taking place during the contracted cycle. It identifies the key findings of the review teams, highlights positive practice that occurs in academic and clinical settings, describes the developments that have taken place in the Major Review process itself and the context in which it operates, and suggests possible approaches for any future quality assurance activity in relation to healthcare education.

This fact sheet summarises the findings in the second Annual Review Trends Report.

Overall, so far

The Major Review process is an effective periodic peer review system built on existing internal quality assurance processes in HEIs and SHAs and other external quality assurance arrangements. It has successfully been a vehicle for the NMC's annual monitoring process.

What the statistics said

Major Review reports contain three tables providing data from the HEI/SHA(s):

Table 1 provides data on completion and achievement statistics for all award-bearing programmes
Table 2 provides employment statistics for pre-registration programmes and where available, post-qualifying programmes
Table 3 presents recruitment and attrition statistics for pre-registration and NMC recordable qualifications.

Due to the different ways data are gathered and presented by the HEIs/SHAs, it was not possible to do a direct statistical analysis across the tables, but it is possible to pick out some key messages.

There is no significant difference in the profiles of the data provide in each table, when it is analysed by discipline or programme level (diploma, degree or postgraduate) in relation to each of the tables.
Two thirds of students registered on programmes were on nursing programmes. 10 per cent were on midwifery programmes.
Around half of all students were enrolled on diploma courses and just over one third on degree programmes.
Two institutions dominate the number of students registered - having around one quarter of all students.
There does not appear to be any clear difference in attainment levels between the allied health professions and health visiting, midwifery and nursing programmes.
Forty-six per cent of students graduating with a degree achieved an upper second class honours, 35 per cent a lower second class honours, 10 per cent a first, five per cent a third and two per cent a fail. This pattern matched sector averages. The pattern of results across disciplines was broadly similar.
High pass rates were seen in midwifery and nursing (93 and 95%) respectively.
Fifty-three per cent of students qualifying on pre-registration programmes went to work with local employers, 23 per cent with employers elsewhere and 25 per cent go on to do something non-work related. In the majority of cases those with a degree were likely to move away from the locality of where they studied, particularly in the allied health professions. Those with diplomas in paramedic science, health visiting, midwifery and nursing were more likely to work with local employers.
Withdrawal rates from programmes vary between disciplines with an average of nine per cent across the whole. The highest rate of withdrawal was in orthoptics (16%, but small student numbers can skew the results), operating department practitioners (15%), paramedic science (14%) and midwifery. The lowest rate of withdrawal was in podiatry (1%, again small numbers could be skewing the data), physiotherapy (5%) and speech and language therapy (6%).

So, what developments might be recommended for any future quality assurance of healthcare education?

The intensity of scrutiny is proportionate to the size of the provision and the level of risk attached.
Review method to include a judgement on maintenance and enhancement of standards and quality.
An alternative term for the commendable judgement is used, one which is more in keeping with everyday language.
Fitness for purpose needs to be explicitly confirmed for all commissioned/contracted programmes.
A different way of including interprofessional programmes in the structure of the report is developed.

For more details about the Major Review process, the production and findings of the Annual Review Trends Report 2004-05 see the QAA website at www.qaa.ac.uk/health/majorreview/reviewTrends05/contents.asp

Table 1 Outcomes from Major Review

		Quality of learning opportunities
	Academic and practitioner standards	Learning and teaching	Student progression	Learning resources
Allied Health Professions	9 reviews given a judgement of confidence	9 reviews given a judgement of commendable; 2 programmes in diagnostic radiography and therapeutic radiography given a judgement of approved	9 reviews given a judgement of commendable	6 reviews given a judgement of commendable; 3 reviews given a judgement of approved
Mixed - Allied Health Professions and Nursing and/or Midwifery	8 reviews given a judgement of confidence; 1 programme in clinical psychology given a judgement of limited confidence	8 reviews given a judgement of commendable	8 reviews given a judgement of commendable	7 reviews given a judgement of commendable; 1 review given a judgement of approved; 2 programmes in clinical psychology given a judgement of approved
Nursing, Midwifery and Health Visiting	11 reviews given a judgement of confidence	11 reviews given a judgement of commendable	10 reviews given a judgement of commendable; 1 review given a judgement of approved; 4 programmes in community nursing, nursing (all branches) and midwifery given a judgement of approved	11 reviews given a judgement of commendable
Totals	28 reviews given a judgement of confidence	28 reviews given a judgement of commendable; 2 programmes given a judgement of approved	27 reviews given a judgement of commendable; 1 review given a judgement of approved; 4 programmes given a judgement of approved	24 reviews given a judgement of commendable; 4 reviews given a judgement of approved; 2 programmes given a judgement of approved

How well did the providers do?

The results of the reviews have been generally positive with only one limited confidence judgement in relation to academic and practitioner standards in one programme. The quality of learning opportunities has also been positive, with the majority receiving a judgement of commendable and a few disciplines and programmes receiving a judgement of approved (see Table 1).

All those involved in the reviews thought it was good that judgements could be differentiated by programme and/or level (undergraduate or postgraduate programme) or mode (part-time or full-time, face-to-face or distance learning). It means that not everybody is penalised when only one area is in difficulty.

The action plans are thought to be useful and allow the HEIs, SHAs and partner placement providers to respond to the review team's findings. Publishing them as part of the report gives the reader a better overview of the provision and considerable insight into the approach of the providers.

At first the action plans were a little too long, some teams were over generous in idenifying strengths and good practices - often they referred to activities that you would normally expect to see - for example, meeting statutory regulatory body requirements such as fitness to practice is a requirement, not a strength. Sometimes the points made by the review teams were difficult to respond to and action, but now, after a little extra guidance from QAA, the actions plans are more streamlined and meaningful. As a consequence the responses from the providers have become, on the whole, more detailed, with specific target dates for completion identified, and fewer constraints to delivering the actions cited.

So what did the annual trends report say about the provision reviewed so far?

Positive features

The great role the clinical assessor (including practice or clinical educators, clinical practice facilitators, practice placement facilitators) plays in enhancing student learning, supporting mentors/supervisors and assessors, and improving standards and quality.
The effective partnerships that exist between most HEIs, SHAs and practice placement providers, made all the stronger by the Major Review process.
The generally high academic and practitioner standards seen, supported by appropriate learning outcomes, curricula and assessment methods.
The resources and strong support mechanisms that students have access to, including libraries, computers, clinical skills labs, intranets, virtual learning environments.
The activities that help widen access and promote retention of students.
Generally strong placement learning opportunities and effective use of placements, despite the national shortage in some disciplines.
Effective arrangements for the monitoring and enhancing standards and quality by these providers.

Recurring weaknesses

A lack of consistency in the use and understanding of marking criteria, and poor quality of feedback provided to students.
The absence of standard processes for returning work to students, or failure to apply existing processes.
Minimal and/or variable planned and organised opportunities for interprofessional learning/ education particularly in some of the allied health professions.
Limited and/or variable service user and carer involvement in programme development or implementation.
The lack of opportunity for clinical assessor updates and continuing professional development, usually due to workload.
The failure to maintain fully the live mentor register (relevant to health visiting, midwifery and nursing only).
Not keeping formal records of student evaluations and a lack of systematic evaluation feedback from the HEI to the practice placements.

What did the Annual Review Trends Report say about the MajorReview process?

The positives

It is effective and does what it is supposed to.
The training events run by QAA were rated positively by 92 per cent of those who attended - helped by the quality, skills and approachability of those delivering the events.
The preparatory meeting is an effective starting point to confirm which programmes are in or out of the review, identify practice placements to visit for day two and, more latterly, day three, and to establish a positive working relationship between the providers and the Review Coordinator.
The significant work undertaken by practice review facilitators (PRFs) in engaging clinical staff in the reviews, as well as deftly organising often complex and geographically spread practice placements into a workable, two-day programme of visits for the reviewers.
The review teams supported by the small group of review coordinators. Both have undertaken a significant amount of work for the benefit of the reviews and personal development.
The integration of NMC annual monitoring with the Major Review process.
The responsiveness of QAA's Health Team in responding to queries, issues and concerns of participants.
The Major Review Facilitator (MRF) and PRF roles which have ensured that clinical and academic staff have a voice in the writing of the self-evaluation document and the action plan. They also make sure that feedback from the team (via the Review Coordinator) is passed onto all staff involved - so everybody knows where they are.
Providing the action plans in an electronic format to HEIs/SHAs earlier in the process to combat the difficulties experienced by the MRF and PRF in securing agreement and sign-off by suitable senior staff.

The challenges

Using the same model of review for everybody - HEIs/SHAs with small healthcare provision have received the same level of scrutiny as the larger, more complex provision.
The time gap between training the reviewers and their first Major Review. A little nerve wracking for those where it was a long time, in spite of regular update newsletters.
Recruiting reviewers from the smaller disciplines and certain larger disciplines - thank you to all those who helped out.
The lack of service-user involvement directly in the Major Review process, although assessment of service user and carer involvement in the provision is a fundamental part of the review.
The continually and rapidly changing context in which Major Review operates.
Self-evaluation documents not following the guidance in the Handbook for major review of healthcare programmes (Annex D).

So what was particularly eye-catching?

The use of an e-learning environment to provide feedback on assessments of practice to students.
Tripartite meetings between student, clinical assessor and link tutor (where a lecturer has a formal relationship with that clinical area/staff) in helping the student to integrate theory and practice and in supporting the clinician and student.
Mechanisms developed to promote consistency and reliability in the marking of all academic and practice assessments, such as the use of a panel involving practice staff, clinical managers and academic staff to scrutinise student assessments.
The effective dissemination and implementation of actions agreed as a result of student feedback.
The development and use of specific support systems to manage the transition of students into higher education.
The ways in which HEIs are working closely with SHAs to address attrition and recruitment difficulties.
The use of a book delivery service to students on placement, the development of interactive CD-ROMs and the development of a resources centre available for individual study outside teaching time.
The use of mechanisms that ensure that special needs are appropriately assessed and supported.

A point to mull over

Each pre-registration degree level programme could enable the assessments of students in practice to contribute to the final honours classification, eg first class or third class honours. It is variable as to whether practice assessments contribute at all, or through a pass/fail or through a graded percentage result. This could mean that students who are assessed as having a high level of skills in patient/client care do not have this recognised in their final award. This is an important area for debate when considering the value attached to learning in practice. A third and final Annual Review Trends Report is due to be published in December 2006 and will look back over all of the reviews that have taken place during the review cycle.

¹ Responsibility for the Partnership Quality Assurance Framework for Healthcare Education in England, of which Major Review is one part, transferred to SfH, a sector skills council, under a service-level agreement from the DH from 1 October 2004.
² The Annual Review Trends Report 2003-04 and 2004-05 are available at www.qaa.ac.uk/health/majorreview/