Report of the consultation on the Partnership Quality Assurance Framework for Healthcare Education in England - March to June 2004

Contents

• Executive summary
• Introduction
• The consultation
• Chapter 1: The consultation process, the Partnership Framework, glossary and additional terms
• Chapter 2: Approval
• Chapter 3: Ongoing quality monitoring and enhancement
• Chapter 4: The evidence base
• Chapter 5: The Approval and OQME Standards Template
• Chapter 6: Action-planning templates
• Conclusion
• Appendix 1: Profile of respondents

PDF document (690KB)

Executive summary

1. On behalf of the Department of Health (DH) and its key stakeholder partners, the Quality Assurance Agency for Higher Education (the Agency) published a consultation document both electronically and in paper form in March 2004. The document outlined:

• the background to the Partnership Quality Assurance Framework for Healthcare Education (the Partnership Framework);
• the process protocol for programme Approval;
• the process protocol for undertaking ongoing quality monitoring and enhancement (OQME);
• the evidence base for the Partnership Framework;
• the Approval and OQME Standards Template for healthcare programmes.

2. The consultation took place between March and June 2004. The document was disseminated to all key stakeholders and supported by four one-day national roadshows. Fifty-six completed questionnaires were received; written commentaries were received from 26 organisations; a total of 140 individuals attended the roadshows and consensus views were recorded on the 21 group questionnaires, together with a wealth of comments.

3. The report analyses and summarises the results of the consultation on elements of the Partnership Framework - the Approval process, the OQME process, the evidence base, the Approval and OQME Standards Template and the glossary of terms and acronyms that support these. Currently existing subject benchmark statements and Major Review are also elements of the Partnership Framework but have already been agreed and, therefore, were not part of this consultation report.

4. The consultation document was understood and welcomed by most of the respondents. Eighty-two per cent of respondents said that they had prior involvement in the development of the Partnership Framework processes. Most thought that the processes and standards adequately encompassed the full range of practice learning providers and covered the full range of monitoring and enhancement activities. The principles of the Partnership Framework were felt to be met and the attempt to streamline current quality processes and engage in a partnership of all concerned was lauded. The use of an agreed evidence base and a set of standards common to all parties were also positively received, as were the templates available for action planning. The commitment to prototype the processes was welcomed by many of the respondents.

5. The Approval process is designed to answer four key questions.

• Is there evidence of an accountable system to sustain the provision?
• Does education, training and assessment meet the requirements for the award, the requirements of regulators for professional practice, the requirements of commissioners and the needs of the student?
• Do the providers' systems and processes operate in a fair, just and open way?
• Is there evidence that the providers' quality assurance systems meet the requirements of the Partnership Framework for OQME and Major Review?

6. A majority of respondents felt that the Approval process enabled the four key questions to be answered. Forty-six per cent thought that there were other questions to be asked at approval, particularly regarding resources. A large minority thought that there ought to be additional organisations represented at the Programme Development Group and Approval Panel. Most respondents felt that it was crucial to involve service users, carers and students in decisions about education and programme Approval. The role of the allied health professional bodies in Approval required further elaboration. The relationship of the Approval process to the Standards Template was not well understood.

7. In relation to OQME, the majority thought that quality would be assured by the process. Respondents acknowledged that there had been a serious attempt to streamline current processes and engage all stakeholders equally in monitoring and enhancing quality. However, it was apparent that many respondents from higher education institutions (HEI) did not appreciate fully that OQME would form part of existing annual monitoring quality assurance process within their institutions. Eighty-four per cent of all respondents felt that the OQME process would meet all stakeholder needs, while the timing of the stages was felt to be appropriate and feasible. However, of most concern was the implementation, specifically in its first year of operation and particularly for clinical areas. Most logistical concerns centred on the Annual Review Meeting. The expectation that the process would be prototyped was welcomed so that perceived difficulties could be identified, modified and the real workload ascertained. The balance between monitoring and enhancement was thought to have been achieved.

8. In relation to the evidence base, 55 per cent believed there was significant evidence missing from that suggested for the Partnership Framework. A number of correspondents asked for the evidence to be cross-referenced to the Standards Template. This would help ensure data gathering was consistent and could reduce the workload. The list of evidence was criticised for failing to state explicitly that evidence should have been analysed and acted upon, rather than just gathered.

9. The response to the Standards Template was positive, the list of standards was considered to be comprehensive. Some additions were identified and concerns were expressed about the frequency that the standards are monitored. Respondents welcomed the attempt to streamline, and the identification of locus of responsibility for each standard was felt to have been successful. The action-planning templates were welcomed and praised for their simplicity.

10. In conclusion, respondents welcomed and appreciated that the proposed processes and templates represented a positive attempt to streamline quality assurance processes in healthcare education. They made many helpful suggestions for modification of the processes.

The next stage of the process in developing the Partnership Framework is to test operationally the OQME and Approval processes, the evidence base and the Standards Template through a series of prototypes. These will take place between September 2004 and May 2005. A number of the suggested changes have been incorporated into documentation for the prototypes. Others are awaiting the outcome of the prototypes. The evaluation and lessons learnt from the prototypes will feed into the preparation for the full implementation of the OQME and Approval processes in the academic year 2005-06.

Introduction

1. In order to develop one shared quality assurance framework for healthcare education that is robust and meaningful, the DH has worked closely with all key stakeholders. These include education commissioners, education providers (HEIs and placement providers), regulatory and professional bodies (primarily the Nursing and Midwifery Council (NMC) and the Health Professions Council (HPC)), and service users (including students). The intention is to reduce the administrative burden on education providers.

2. Two national working groups took responsibility for developing proposals on Approval/re-approval and OQME. Throughout the report, this is referred to as the consultation document and its content as the Partnership Framework. These proposals were tested out with reference groups drawn from across the country with a strong practice and HEI presence. Education commissioners, the Workforce Development Confederations/Strategic Health Authorities (WDC/SHAs), responded through a variety of means during both the development of the processes and the refinements proposed during the consultation through groups, meetings, the four roadshows and/or the questionnaire provided. A special event for students and service users was held to test their initial reactions and to seek their suggestions.

3. The Standards Template was developed from standards already established in the quality assurance of healthcare education. Initially, over 2,000 standards were gathered from a range of sources. These were analysed, duplicates were removed and the remainder discussed by the national working groups and reference groups in order to reduce the number further. Those remaining were grouped into 10 aspects:

• management and organisation;
• effective use of resources;
• curriculum;
• learning outcomes;
• student selection, progression and achievement;
• student support;
• learning and teaching;
• assessment;
• quality enhancement and maintenance; and
• values, equalities and diversity.

4. Within each aspect, the locus of responsibility for monitoring each standard was identified. A final 105 standards formed the Approval and OQME Standards Template (known throughout the document as the Standards Template).

5. The evidence base for OQME consists of the existing documentation and data used by providers to support their self-evaluation and, in the case of Approval, that used by the Approval Panel to verify the quality of provision. In the consultation document, some core data (mainly quantitative) are prescribed, while the remaining evidence listed consists of suggestions that are not exhaustive.

The consultation

6. On behalf of the DH, the Agency published a consultation document both electronically and in paper form in March 2004. The document outlined:

• the background to the Partnership Framework;
• the process protocol for programme Approval;
• the process protocol for undertaking OQME;
• the evidence base for the Partnership Framework; and
• the Approval and OQME Standards Template for healthcare programmes.
• It posed 38 questions concerning:
• the glossary of terms and acronyms;
• the Approval process;
• the OQME process;
• the evidence base; and
• the Standards Template and associated documents.

7. The questionnaire also asked respondents to rank how important the issue raised in each question was to them. This enabled a sense of priority to be established.

8. The consultation took place between March and June 2004. Responses to the document were gathered in three ways:

• the questionnaire (in Section 6 of the consultation document) could be completed and returned to the Agency (by email, post or fax). Fifty-six completed questionnaires were received;
• written commentaries, not in the questionnaire format, were received from 26 organisations. They wished to provide feedback on the consultation document but considered the questionnaire format to be too prescriptive;
• four one-day national roadshows in York, Birmingham, Bristol and Gatwick were organised in April 2004 to provide an opportunity for individuals to give feedback directly on the consultation document. A total of 140 individuals attended. They were allocated to groups who worked through the questionnaire. Consensus views were recorded on the questionnaires by each of the 21 groups, together with a wealth of additional comments.

9. Altogether, 104 responses were received. A profile of the respondents is found in Appendix1. In analysing the data, respondents were grouped into the following categories:

• HEIs;
• professional, statutory and regulatory bodies (PSRBs);
• SHAs and WDCs;
• clinical areas including NHS Trusts, Primary Care Trusts, the private and non-profit sectors;
• other, those that could not be categorised in any of the above sectors, including the cadet nurses, educational consultants, Universities UK, Rethink, the Council of Deans and Heads of UK University Faculties for Nursing and Health Professions, Joint Medical Advisory Committee for the UK HE Funding Councils, Standing Council of Principals and the Better Regulation Review Group; and
• mixture, a joint response from representatives from practice and HEIs.

10. This report outlines the analysis of the quantitative and qualitative data. The reported quantitative data are cross-referenced to the relevant question in the consultation questionnaire. The cross-reference is given in brackets commencing with the letter Q. Specific comments from respondents (in italics) have been used throughout to support the general points being made. The sector of the respondent(s) making the comment is noted in brackets, together with the number of times such a comment was made. Where an individual comment is made from a national committee, it is attributed.

11. The chapters of the report reflect the sections of the consultation document. Each reports the principal themes arising from the analysis and highlights the changes to the proposals suggested by the respondents.

The themes arising from the analysis were:

• additions/changes to the document;
• enhancement;
• implementation;
• involvement in the process;
• locus of responsibility;
• relationship with PSRBs;
• relationship with the rest of the Partnership Framework;
• resources;
• rigour and validity of the process;
• streamlining and workload;
• terminology;
• timeframes; and
• service user/carer and student involvement.

 

Chapter 1: The consultation process, the Partnership Framework, glossary and additional terms

1.1 The consultation process

12. In the responses, 82 per cent of respondents said that they had prior involvement in the formation of the Partnership Framework processes (Q3). Ninety-two per cent indicated that this was an important or very important issue to them (Q3b). When asked about their involvement, a number of respondents acknowledged that they had been given the opportunity to attend events or knew of people in their organisation who had been to national meetings or reference groups in particular.

13. Generally, their comments about the consultation were positive but a number of respondents said that the questions in the document did not allow them to focus on the overall approach but limited discussion to detail only:

we are pleased to have been involved in this consultation (HEI) (x5);

this consultation is almost exclusively concerned with the content and format of the document rather than the wider issues that it raises (HEI) (x8).

14. Forty per cent of respondents felt the document contained inconsistencies (Q33), detailed in the text below, while 78 per cent thought that the processes and standards adequately encompassed the full range of practice learning providers (Q34). Ninety-seven per cent considered this was an important issue (Q34b). However, the absence of mention of, for example, prisons, military establishments and a number of other potential placements worried some correspondents. Overall, the language used in the document was felt by 85 per cent of respondents to be appropriate for everyone, but that the language used in any published reports should also be accessible to service users (Q35). The document was perceived to cover the full range of monitoring and enhancement activities by 85 per cent of respondents (Q36). Ninety-five per cent considered this to be an important issue to them (Q36b). The principles of the Partnership Framework outlined in Table 1 were considered to be met by 88 per cent (Q37). This was considered to be an important issue by 98 per cent of the respondents (Q37b). Details of responses to the other questions are dealt with elsewhere in the report.

1.2 The Partnership Framework - positive feedback

15. The qualitative data, particularly from the roadshows, indicate that the document was understood and welcomed by most of the respondents. It was generally thought to be consistent in its messages and use of language, although this was not universally reported.

16. The attempt to streamline current quality processes and engage in a partnership of all concerned was lauded. Overall, the Approval/re-approval process was recognised as familiar, and the OQME process, while considered quite complex, was seen as an improvement on the many different processes currently prevailing (for further detail, see Chapter 3). The use of an agreed evidence base and a set of standards common to all parties were also positively received, as were the templates available for action planning. The commitment to prototype the processes reassured many of the respondents.

1.3 The Partnership Framework - areas of concern

17. Respondents were most concerned about the apparent workload for placement providers undertaking OQME, especially in the first year of implementation. The Annual Review Meeting was seen as ambitious and potentially difficult to organise. To run effectively, all participants must have an input with the key messages not being trivialised.

18. Respondents felt that there would need to be a cultural change in some placement providers if acceptance of the level of responsibility for quality required in this process was to be achieved. There was also a concern that the processes were not sufficiently evaluative and that, therefore, it would be easy for them to be treated mechanistically.

1.4 Glossary of terms and acronyms

19. Much of the terminology used in the consultation document is common to all health disciplines. Nevertheless, it was necessary to develop some specific terms for the Partnership Framework in an attempt to establish a language common to all professions.

20. Generally, the glossary was welcomed, found to be useful and most terms were felt to be clearly defined and easily understood. Indeed, 96 per cent of respondents agreed that the definitions in the glossary were easily understood (Q1.) The qualitative data supported this:

glossary understood easily;

glossary at front logical (Practice) (x12).

1.5 Additional terms

21. Fifty-six per cent of the respondents thought that there were other definitions to be added (Q2). It was suggested that the following terms used in the consultation document should be included in the glossary of terms:

• exception reporting (x2);
• learned society (x1);
• Learning Development Agreement (x4);
• major change (x2);
• Major Review (x2);
• partnership agreement (x1);
• practice placement/practice placement supervisors (x12);
• programme (x4);
• re-approval (x7);
• stakeholder (x5); and
• Standard Model Contract (x1).

It was noted that not all acronyms used in the document had been listed.

22. Although defined in the glossary terms, seven respondents suggested further explanation of the term 'lead education person' was required.

23. The qualitative data suggested the importance of consistency in the use of language in the document. A frequently cited example was the use of the terms: 'local unit level', 'locality' and 'local area'. Five respondents also highlighted the need for:

consistency of terms between all organisations (e.g. QAA, HPC, NHS, contracting bodies) (HEI) (x5).

1.6 Standard Model Contract and model agreements - key partnership relationships

24. The relationship between the Partnership Framework and the Standard Model Contract was not clearly understood. Although it is intended that the Partnership Framework is an annex to the Standard Model Contract, this seemed not to be explained well enough in the consultation document.

25. Figure 2 represents in triangular form the partnership relationships between education commissioners, HEIs and placement providers. While 100 per cent of respondents indicated that they understood the diagram (Q4), it was criticised heavily on two counts. Firstly, the triangular shape was felt to be hierarchical, which is at odds with the partnership culture being cultivated. Secondly, the nature of the Standard Model Contract, and its associated agreements, is not well known and so the diagram lost some of its clarity:

the diagrammatic representation indicates a hierarchical structure which is not the case, as all partners are equal. The use of a circular or non-hierarchical approach would be useful (HEI) (x6);

there is also a significant unknown element as regards the interface with the Standard Model Contract. This is an essential issue if contract review is to be acceptably integrated into the process (SHA/WDC) (x7).

26. It was suggested by a significant number of the respondents that it would be more helpful to represent the relationships as a circle.

27. Ten respondents also commented that all those involved in healthcare education might not understand the language:

terminology could be too complex to understand for the average mentor/assessor in placement area (SHA/WDC) (x10).

1.7 Suggested changes

• Add the above list of terms to the glossary and explain more clearly what 'lead education person' means.
• Check that all acronyms are included.
• If Figure 2 is to be used in future documentation, consider reconfiguring it to avoid a hierarchical interpretation.

 

Chapter 2: Approval

28. Approval is the process of validation/accreditation that leads to a decision about whether or not a programme can be launched or, in the case of re-approval, continued. Those central to this process are the education commissioners and providers, academics, professional and regulatory bodies and, where possible, students. In the past, Approval/re-approval has normally been sought from the statutory or professional body relevant to the programme being planned.

29. A national working group developed this process with input from the NMC and HPC. As it is relatively new and evolving, the HPC's relationship with the professional bodies in relation to Approval was explained in Figure 3 in the consultation document.

2.1 Terminology and comprehensibility

30. Comments received emphasised that there were few surprises. Ninety-seven per cent understood the process of Approval as outlined in the document (Q11), while 99 per cent said that the terminology 'Approval' and 're-approval' was workable in all environments (Q5) and 76 per cent suggested that there is a difference between the two (Q6). Seventy six per cent of respondents thought that the workability of the terminology was an important issue.

31. Generally speaking, respondents were familiar with the Approval process. The qualitative data indicated that the term 'Approval' was deemed to be less confrontational than the older terms 'validation' and 'accreditation':

better than validation, less confrontational more contemporary (Practice) (x8).

32. The term 're-approval', however, was thought not to be explained well in the consultation document and respondents queried whether this was because it might be replaced by OQME:

clarity between Approval/re-approval and modification. Relationship between OQME and re-approval/modification' (SHA/PSRB/HEI);

ongoing monitoring is substantial and should remove the need for 'regular' re-approvals (HEI) (x5).

33. A small number of respondents felt that the description of the process lacked detail and might be aided by a flow chart:

more explicit regarding the details. Re-approval process and difference between the two. Mapping process would be useful (Practice/WDC/SHA) (x9);

the process as described is incomplete: provisions for failure to meet approval, processes following Approval, for example how are required, actions followed through and period of validity of Approval (SHA/WDC) (x3);

may need a bit of guidance on major change - what does this mean? (SHA/PSRB/HEI) (x2).

2.2 Timeframes

34. A number of respondents suggested that a set term for Approval should be given, usually five years. As a number of the standards were monitored every three years, some believed that this was now the implicit timeframe for re-approval:

clarification re length of approval needed (confusion re: three years in tables but previous expectation is five years) include length in definition (HEI) (x5).

2.3 Questions for the Approval event

35. Although 89 per cent felt that the Approval process enabled the four key questions to be answered (Q12), 46 per cent thought that there were other key questions to be asked at Approval (Q7). Ninety-four per cent thought that this was an important issue (Q7b). Consideration by the Approval Panel of the resources available for the planned programme appeared to be missing. This was a particular concern for placement providers who argued at several points that placements are becoming stretched. They suggested that this should be the subject of a fifth question to be asked at the Approval event. Some respondents wanted the process to be more strongly linked to workforce planning:

big concerns about the capacity of service placements (Practice);

not clear where in the process/at what stages the question of adequate resources to support the programme will be addressed (HEI);

resourcing? Including e.g. placement availability? If not then this should be asked a as key question (SHA/WDC) (x7);

evidence on which need for programme is based - market research. When small professions are involved, some evidence of fit to local regional strategy (Practice);

is there evidence that the provision is able to anticipate future needs and deliver current requirements? (PSRB);

it would be helpful if the links between commissioning/Approval/re-approval/ tendering/re-tendering are more explicit (HEI) (x5).

2.4 Involvement in the process

36. Thirty-four per cent and 48 per cent respectively thought that there ought to be additional bodies/organisations attending the Programme Development Group and Approval Panel (Q8 and Q9). A continuing theme through the feedback generally was the lack of service user and carer involvement in decisions about education. While most respondents acknowledged that it was not easy to engage them, they also felt that it was nonetheless crucial to do so. Students were also felt to be important contributors to the process and should be included in programme Approval.

if the programme has to meet the needs of service users then they should be on the Approval Panel (HEI) (x12);

student involvement not specific enough (HEI) (x3).

37. Respondents deemed that the other main representatives missing from the approval process were felt to be those from service and from the internal HEI quality assurance systems.

I do feel service providers need an input (Practice) (x9);

there should be representation from the HEI in line with HEI quality assurance processes as well as a representative from the 'appropriate office' (HEI) (x4).

2.5 Relationship with PSRBs

38. The relationship between the HPC and the allied health professional bodies was outlined in the text and diagrammatically represented in Figure 3. While the quantitative data indicate that the diagram was understood by 92 per cent of respondents (Q10), it caused considerable debate at the roadshows and high levels of concern about the role of the professional bodies in Approval. The role of the professional bodies in developing curriculum guidelines and acting as expert/professional advisers to the Programme Development Group (PDG) was not well understood or emphasised sufficiently:

external professional expertise - this is unclear (HEI);

post-registration education for role development and specialisation within the profession is an issue for professional bodies which hold the knowledge and expertise regarding the requirements for advanced/specialist practice (PSRB);

can the professional bodies have a more formal role in the programme approval? (HEI) (x25).

39. The relationship was felt by many to be too informal and several wanted this tightened. It is acknowledged here, however, that this is outside the scope of the consultation process as it is subject to consultation by the HPC.

40. The clarity of Figure 3 was considered by 92 per cent of respondents to be an important issue (Q10b).

2.6 Relationship with the rest of the Partnership Framework

41. It became apparent during the roadshows and subsequently, that the relationship of the Approval process, particularly with the Standards Template, was not well understood. It was not clear how the standards might be used in Approval or how the evidence base related to either of them.

there is no reference in this section to self-assessment of standards or indeed when standards designated 'Approval' will be judged (HEI);

there is no mapping between the requirements of the Partnership Framework and the listed evidence (SHA/WDC) (x3).

2.7 Suggested changes

• Continue to use the term 'Approval', but distinguish more carefully between Approval and re-approval (and between re-approval and OQME, if this is felt to be required).
• Consider setting a maximum term for which approval is given.
• Describe the process in more detail and add a flowchart showing the links to workforce planning.
• Add a further question, to the four listed in the consultation document, to explore more fully the resources available for the programme, particularly if there are clinical placements required.
• Strengthen the explanation of the role of the professional bodies, particularly in the PDG. Require that students, where possible, and service users and carers are members of both the PDG and the Approval Panel. Strengthen service involvement in both and consider adding internal HEI representation.
• Consider mapping the evidence base to the Standards Template and describing in more detail how the standards are to be used in Approval.

 

Chapter 3: Ongoing quality monitoring and enhancement

42. The OQME process was developed by a national stakeholder group, local reference groups and modified by a range of other inputs from relevant parties. It is a revised process developed from activities currently undertaken by, among others, SHAs/WDCs, statutory and regulatory bodies. It is expected to incorporate all other monitoring activities (see below), except Approval/re-approval and Major Review. It encapsulates a peer review, self-evaluation approach where exception reporting is the norm. It relies on programme providers looking at evidence against a number of standards and showing where these might not have been achieved or where there is good practice to report. The process is continuous but is consolidated every year at an Annual Review Meeting where all stakeholders debate and agree a final action plan to enhance quality in the coming period.

43. Current annual processes include:

• educational audit of clinical placements;
• contract monitoring by SHAs/WDCs;
• regular, routine visits from some PSRBs;
• regular written reports to PSRBs; and
• internal HEI annual quality assurance mechanisms at school and departmental/faculty level.

The OQME process will incorporate all of these.

3.1 Implementation

44. The qualitative data confirmed respondents' acknowledgement that there had been a serious attempt to streamline current processes and engage all stakeholders equally in monitoring and enhancing quality. Exception reporting was acknowledged to be timesaving and generally positive. In particular:

user-friendly document that will reduce paperwork and number of different reviews. Collaborative document (Practice) (x6);

like the idea of exception reporting. Helps with sharing good practices (Other) (x9);

the principle of exception reporting incorporated into the numerous proformas is to be welcomed. We also appreciate that the structure of identified responsibilities relating to the confirmation of standards across the different aspects is meant to ensure the ownership of responsibilities across the partnership. The clarity and formality is seductive and the formal requirement for a signature appears to secure ownership and responsibility. However, there is a danger that with the proliferation of such forms, responses become quickly routinised and automatic and that they add little to effectiveness rather than to bureaucracy. However, we do value the considered attempts to judge the frequency with which some aspects should be subject to reconfirmation (HEI)(x4).

45. Eighty-six, 89 and 96 per cent respectively of respondents thought that stages 1, 2 and 3 were clearly explained (Qs 13-15). At least 98 per cent of respondents thought that these were important issues (Qs13b-15b). Eighty-one per cent felt that the purpose of the Annual Review Meeting met the needs of all attending stakeholders (Q16) and 87 per cent felt that the final action report enabled all the purposes to be met (Q17). Eighty-four per cent felt that the OQME process would meet all stakeholder needs (Q18) and that the timing of the stages was felt to be appropriate and feasible by 74 per cent of all respondents (Q19). All the above issues were deemed to be important by at least 98 per cent of the respondents (Qs16b-19b)

46. However, the qualitative data indicated that the implementation, specifically in its first year of operation and particularly for service, it was the overwhelming concern. In particular, it was felt that smaller placement areas, for example, in the private/non-profit sector and overseas placements, might struggle to resource the data gathering and exception reporting workload. An agreed definition of the term 'placement' might help to clarify whether all placement areas needed to participate.

workload, especially in year 1 (HEI);

a particular concern for capacity and capability of smaller organisations and widespread organisations (Practice);

quantity of information required. Keeping it manageable/controllable (Practice) (x25).

47. The expectation that the process would be prototyped was welcomed so that perceived difficulties could be identified, modified and the real workload ascertained.

3.2 Terminology

48. Further clarity was sought about the role of the 'named person' as the lead in each placement provider. It was not clear who this person might be and if they would have the capacity and breadth of experience accurately to summarise exception reports received:

definition and level of key role (PSRB);

some clarification needed particularly lead education person. Should be an infrastructure to support this person (SHA/PSRB/HEI);

not certain who the lead education people will be in practice, especially how they might represent the smaller professions (PSRB) (x7).

49. Respondents also noted that the language used must be comprehensible to service users and carers if action plans are to be published. Additionally, the development of a common language for all healthcare professionals in all settings was commented upon:

the language must also be meaningful to clients especially if we are to report under Freedom of Information (Other) (x4);

some realigning of some terminology in use will be required for all healthcare professions to conform to a single set of terms (PSRB) (x5);

we need to be sure the private sector are happy with the language used (Practice) (x4).

3.3 Monitoring and enhancement

50. The balance between monitoring and enhancement was thought by 77 per cent of the respondents to have been achieved (Q21). All respondents considered this to be an important issue (Q21b). However, the qualitative data raised the concern that it would be easy for self-evaluation to degenerate into a 'tick box' exercise:

moves in right direction but worries it may turn into box ticking process (HEI) (x4);

not balanced but a much bigger emphasis on enhancement and opportunities to do this especially after year 1 (HEI) (x12);

the focus is upon monitoring (HEI) (x11).

51. Another balance-related theme looked at how the processes dealt with the full range of programmes. The consensus was that it was focused excessively on pre-registration and did not easily lend itself to being used for interprofessional learning and education:

the Partnership Framework on page 3 refers to programmes of learning beyond registration but the document seems to focus on pre-registration' (PSRB) (x3);

the processes...do not appear to achieve an appropriate balance between cross-professional overview and profession-specific scrutiny (PSRB) (x2).

3.4 Rigour and validity of the process

52. The quantitative data indicated that 83 per cent felt that quality would be assured by the OQME process (Q20), while 79 per cent agreed that Table 2 accurately summarised the responsibilities of all engaged in it (Q22). However, there was concern raised in qualitative responses that the OQME process would not be rigorous enough. There were four reasons for this concern:

• respondents felt that a self-reporting exercise lacked objectivity;
• they expressed concern that the detail contained in the individual report from each unit would be diluted by the process of summarising all of them;
• some felt that, because the process was based on trust at the self-evaluation stage, where there was a culture that did not embrace this, individuals might not feel comfortable enough to admit to faults; and
• the size of the Annual Review Meeting (see section 4.5 below) might cause important quality issues to be overlooked:

lack of objectivity in reporting mechanism (PSRB);

some concerns expressed as to the quality and standardisation of self-assessment - will this be monitored? (Practice);

when would the scrutiny of evidence occur and who would request the evidence? (HEI) (x16);

constant summarising may lead to dilution (Practice) (x8);

it has to be accepted that the light touch on HEIs...requires all parties to trust each other (HEI) (x5);

not likely to be robust enough - depends on management culture. It has to be a safe place to speak up about exceptions (PSRB/Practice) (x2).

53. Most respondents felt, however, that the OQME process would meet the needs of all stakeholders provided it was well managed and properly resourced. The main concern is whether or not professional bodies' needs are met where applicable, especially their public protection role:

we hope so - need to ensure all aspects previously covered are still covered (HEI) (x2);

the absence of any reference to professional bodies means that the process, as described, could not meet their monitoring needs (PSRB) (x5).

3.5 Streamlining and workload

54. Most logistical concerns centred on the Annual Review Meeting. This takes place when the initial action report is debated and the final action report agreed. Relevant stakeholders undertake to implement the agreed action plan, outlined in the final action report. All stakeholders who wish to be present are invited.

55. It was felt to be an ambitious meeting, potentially requiring a long agenda and a high level of attendance. Some respondents said that this might lead to less rigour in the proceedings, as attention to detail would necessarily be minimal, as is mentioned above in section 4.4. Prototyping will be important:

the organisation of the meeting needs to be considered in partnership (HEI);

needs to be structured. Huge agenda - how do you handle this, how would it work? What issues don't apply to you as a stakeholder? Challenge of meeting all stakeholder needs at the annual review meeting. How does the lead WDC coordinate this? (SHA/PSRB/HEI);

how will rigorous verification of self-evaluation be managed in a potentially large forum? It will require a high level of skill and organisation to make this stage work in a constructive way (HEI) (x10);

looking forward to testing and pilots (SHA/PSRB/HEI) (x8).

56. HEIs noted their need to recognise that OQME necessitated some adjustment of their own internal quality assurance systems if streamlining is to occur. It was apparent that many respondents did not appreciate fully that OQME replaced a number of existing quality assurance processes:

it would have to be accepted by each individual HEI as replacing current internal quality assurance arrangements for health related provision (HEI) (x3);

duplication with other processes, for example, Institutional Review (HEI) (x5).

3.6 Involvement in the process

57. Most respondents felt that the whole process could be jointly owned between HEIs and placement providers and that this would be an improvement on the current situation where the burden of responsibility for the quality of healthcare education appears to lie with HEIs. However, it was acknowledged that full participation might be dependent on there being an incentive to undertake the work (for example, financial support, inclusion in star-ratings or Healthcare Commission (formerly Commission for Health Improvement/ Commission for Audit and Inspection) reports):

Trusts, preparation, engagement and support (PSRB);

what are the incentives for 'buy-in' particularly for practice? (HEI) (x5);

clearly, it will be essential to ensure that all stakeholders remain committed to the concept of a single process, since otherwise any benefits of integration and streamlining will be lost if individual stakeholders introduce separate requirements (Other-Universities UK).

3.7 Timeframes

58. Timeframes for gathering all the evidence, particularly if some units proved recalcitrant, were felt to be very tight and that this would delay the annual process. Where HEIs and placement providers had commissions from more than one SHA/WDC, it was felt that it was very important to coordinate the timing of the Annual Review Meetings so that the fullest possible attention could be given to each event that needed to be attended by the SHA/WDC. Suggestions were made that the annual cycle might run over a two-year time period:

why annual process? Prefer bi-annual for proper job (Practice) (x2);

it doesn't fit within the academic year. September is a very bad time for many programmes (HEI) (x5);

consideration needs to be given to tying up cycles of HEIs and NHS to enable a timely completion of the initial action report prior to the annual review to take into account financial and academic planning years. Also there will need to be flexibility in the suggested deadline of completion of the document three months prior (SHA/WDC) (x7).

59. Some respondents felt, however, that the time lag between self-evaluation reporting and the Annual Review Meeting was too long, such as to render the reports invalid because they would no longer be applicable:

the time-lag between initial exception reporting and the actual action plan means that the action plan should be redundant if any ongoing enhancement is happening (SHA/WDC) (x2).

3.8 Suggested changes

• Consider having an agreed definition of the term 'placement' so that it is clear which units are required to participate in the gathering of evidence.
• Strengthen the enhancement aspect of OQME so that it becomes more explicit.
• Consider whether a two-year cycle is preferable.
• Publish guidelines about managing the Annual Review Meeting with, possibly, sample agendae. Clarify which issues are brought to this meeting and which should be dealt with as an ongoing process.
• Strengthen the case for full participation by all stakeholders.
• Consider representing the process diagrammatically.

 

Chapter 4: The evidence base

60. The list of evidence was derived from current data requirements from the DH, HEIs, SHAs/WDCs and placement providers. This was streamlined to reduce duplication and then divided into core and additional evidence.

4.1 Additions to the evidence base

61. Although 97 per cent of the respondents agreed that the statement about the purpose of the evidence was comprehensive (Q23), 55 per cent believed there were significant pieces of evidence missing from the list (Q24). Most importantly, perhaps, is the need to collect evidence about user and carer satisfaction:

there needs to be a mention of user or patient satisfaction improvements (Practice);

evidence is needed of the involvement of service users and carers in the design (of) programmes (Other);

patient satisfaction survey needed (Practice) (x7).

62. Other items felt to be missing were details of how the evidence was to be collected and issues related to this:

another significant area missing from the list is the detail on how the data are to be collected (by whom/when/where/how stored) and how their timeliness and relevance will be assured (SHA/WDC) (x2).

Other additions include:

professional body accreditation reports (PSRB) (x2);

for post-registration the number of courses commissioned from the number of courses available. Indication of confidence in the provision (Practice) (x1).

63. Nevertheless, 51 per cent of respondents thought that the amount of evidence resembled an annual Major Review too closely and could be reduced (Q25).

list detailed and reminiscent of the old 'base room' (HEI) (x19).

Indeed, 85 per cent thought it was important to reduce the amount of evidence (Q25b).

64. At least 95 per cent of the respondents considered having a comprehensive statement outlining the purpose of the evidence base and identifying any missing pieces of evidence to be important (Qs 23b and 24b).

4.2 Enhancement

65. A number of respondents criticised the list for failing to include evidence that data had been analysed and acted upon. In other words, evidence should contribute to outcomes. An example of exit interview reports was used to illustrate this. It was felt that these would not contribute to quality enhancement unless there was a summary of findings, an action plan and follow up:

would not want to increase the list but am concerned that action is taken on information (for example, exit interview results) (HEI);

not just collecting, it's analysing (SHA/PSRB/HEI) (x5);

potential for tick box responses (PSRB/Practice) (x4).

4.3 Relationship with the rest of the Partnership Framework

66. A number of correspondents asked for the evidence to be cross-referenced to the Standards Template. This would help ensure data gathering was consistent and could reduce the workload. The same point was made in response to questions about the Standards Template:

evidence should be cross-referenced so that one piece of evidence can be used for several purposes. Although there are many types of evidence (they) may not all be needed. At the moment it looks like a lot of it is mandatory and this could be reviewed (WDC/SHA) (x4).

4.4 Suggested changes

• Evidence of patient/user/carer involvement needs to be added to the overall list.
• Consideration should be given to adding the other items suggested but also streamlining it further if possible.
• The evidence could be cross-referenced to the Standards Template.
• Consider amending the language in the evidence base to show that information needs to be analysed and acted upon.

 

Chapter 5: The Approval and OQME Standards Template

5.1 Overall

67. The response to the Standards Template was positive, with 99 per cent of respondents considering that the list of standards was comprehensive (Q26), although some additions were identified. Respondents liked the attempt to streamline and 90 per cent felt that the language was universal and easily understood (Q27). Ninety-eight and 95 per cent respectively thought that these issues were important (Qs 26b-27b).

68. Some respondents were not clear about the process for reporting that standards had been met and demonstrating the level of enhancement that had taken place:

the process of recording meeting of standards generally remains unclear. What forms will be required. Will these be consistent across all placement areas? (SHA/WDC) (x3);

it is not clear how improvement and enhancement of quality standards will be measured (Practice) (x2).

69. All respondents welcomed the identification of who is responsible for monitoring each standard. The partnership relationships being engendered by sharing responsibility for quality assuring, particularly, placements is felt to be a strength:

strengthens the partnership between practice and education (Practice) (x3).

70. The main criticism was that there were too many standards. However, when the standards are listed by responsibility and timeframes, the list becomes less contentious (see Table 1 below to reflect the number of standards following consultation).

Table 1 Summary of the number of standards to be monitored by each type of provider and for each timeframe

	Overall	OQME only annually	Approval only	Approval and OQME annually
HEIs with institutional audit	55	7	18	30
Placement providers	21	4	1	16
HEIs and placement providers	15	2	2	11

71. It is important to note that, subsequent to consultation, there has been an agreement that 13 of the standards are normally monitored during institutional audit in England for relevant HEIs and, therefore, will be removed from the list of standards for OQME. Within Approval, 2.4, 2.5 and 5.4 are now for Approval only. All OQME standards will now be assessed annually. Those institutions that do not have institutional audit will still be required to monitor the 13 standards through this process.

72. The attempt to colour code the standards by locus of responsibility was felt to have been successful, with 100 per cent of the respondents answering positively to question 29, while its shortcomings, such as disadvantaging those with sight problems, and the difficulty of reproducing it without a colour printer, were acknowledged. Ninety-one per cent of respondents considered this an important issue (Q29b).

5.2 Terminology

73. Respondents commented that although the language in the Standards Template was understandable, it was inconsistent with the rest of the document. For example, the term 'education provider' had been applied to both campus and service settings but, in the Standards Template, it referred only to campus.

74. A small number felt that the language was biased towards education. However, this was not strongly expressed and many more were positive about the universality of the language. The term that seemed to create the most confusion was the 'practice placement supervisor' as its meaning is different to different professions:

practice placement supervisor? Consistency needed (HEI);

practice placement supervisor should be adopted as standard terminology (HEI);

term supervisor not used in nursing - use mentor (Other) (x9).

5.3 Timeframes

75. Although the quantitative data show that 83 per cent agreed that the indications of how often the standards needed to be monitored were appropriate (Q28), two main concerns arose about timing. The first related to the rationale for having some standards monitored annually and some triennially. This was not explained sufficiently well in the document. Secondly, respondents asked who would be responsible for triggering monitoring of the three-yearly standards. This, they considered, had the potential for confusion:

mixed timeframe (Practice);

doesn't appear to be a clear strategy of why some standards are annual or three-yearly (Practice) (x7);

who will ensure the three-yearly standards are monitored? (SHA/PSRB/HEI);

having three different timescales is onerous without someone triggering the process (Practice);

there may need to be more joining up between the OQME and Major Review processes, in that the OQME seems to hinge on a three-yearly cycle, as against the five years for Major Review (SHA/WDC) (x3).

5.4 Additions/changes to the document

76. Two main additions were suggested. Respondents felt that standards specifically related to the SHA/WDC ought to be included. Secondly, an additional standard ought to be added to ensure that the commissioned number of places on each programme was specified as early as possible and not to exceed the resources available:

some of the standards should relate to WDCs or commissioners (Practice);

standard 8.8 is not a placement provider responsibility. It is for the commissioning authority and the education institution to ensure effective placement learning (PSRB);

we are also concerned that the role, responsibilities and accountabilities of commissioners are not obvious within the overall process (HEI) (x6).

5.5 Streamlining and workload

77. Again, there are concerns about workload, especially for placement providers. However, standardised reporting may reduce the burden for practice placements. Equally, the standardised self-evaluation should aid practice placements that link to more than one HEI. It is hoped by some respondents that the standards will become common to all reviewing bodies. Several respondents suggested using sampling to reduce the workload:

the sooner we have a standard placement evaluation the better. If Trusts feed into more than one HEI need standardisation of information they will provide so that they are not providing several reports (Practice) (x2);

making it work, particularly at practice level (HEI);

potential to be bureaucratic dream, practitioner's nightmare (Practice) (x7);

will there be universal standards? Will the NMC have their own? HPC have not yet adopted these standards (HEI) (x2);

there is an argument for sampling on a rotational basis (HEI) (x4).

5.6 Format of Standards Template

78. There was an almost equal split between those preferring the portrait and those choosing the landscape version with 57 per cent preferring a portrait format (Q30). However, only 35 per cent of the respondents considered this to be an important issue (Q30b).

79. The coding of the document to show locus of responsibility was felt to be an excellent idea but the recognition that this would disadvantage some people was noted.

80. It was suggested that an extra column to show relevant evidence that might be collected in support of each standard might be helpful. One respondent requested the document be split into three sections, separating standards by locus of responsibility and publishing these separately. Others asked for it to be available electronically in a format that would allow typing directly on to the document:

good concept but colour not accessible - suggestion to use different fonts (Other);

suggestions - C for campus J for joint (Practice/SHA/WDC) (x10);

useful to have separate pull-out sections (HEI) (x1);

no strong feelings either way. Try to get this completed electronically (HEI) (x6);

landscape so that a 'source of evidence' column could be included to aid self-assessment and audit trails (PSRB) (x3).

5.7 Suggested changes

• Change the term 'education institution' throughout to keep the language consistent.
• Confirm that the term 'practice placement supervisor' is acceptable to all relevant professions.
• Either rationalise the sorting of standards so that it is clearly understood why each one is monitored at that particular frequency or agree to monitor all of them at the same time.
• Clarify any responsibilities SHAs/WDCs might have and possibly add a standard related to commissioning timeframes.
• Ensure the Standards Template is available electronically with a choice of portrait or landscape formats. Differentiate between standards without using colour.

 

Chapter 6: Action-planning templates

81. The action-planning templates are the suggested formats for presenting the initial action report to the Annual Review Meeting, with one for the final action report signed off by all stakeholders. The templates were welcomed and considered important, again for the standardisation they achieved and the potential reduction in workload (99 per cent and 97 per cent considered the templates to be helpful Q31 and Q32; 88 per cent and 93 per cent that the issues are important Q31b and Q32b). However, it was felt by a number of respondents that all stakeholders should sign the final action report:

may need more than one signature. One signature not representative (Practice) (x2).

They were praised for their simplicity. However, a worked example might prove useful.

82. The final two templates give suggested formats for the initial action report and the final action report respectively. This is an attempt to ensure that the annual reports of monitoring are standardised for use in a range of settings.

Respondents felt that there was the potential for this document to be used in 'naming and shaming' individual programmes and/or placement providers, thus creating a climate that limited opportunities to improve:

potential naming/shaming issue report - market sensitiveness (Practice) (x2).

6.1 Suggested changes

• Provide a worked example of each report.
• Allow all stakeholders to sign the final action report.

 

Conclusion

83. Perceiving that it covered the full range of monitoring and enhancement activities in healthcare education, the Partnership Framework was understood and welcomed by most of the respondents. The attempt to streamline current quality processes and engage in a partnership of all concerned was praised, while the commitment to prototype the proposed processes reassured many. Most respondents felt that it was crucial to involve service users, carers and students in decisions about education and programme approval. More than four-fifths thought that quality would be assured by the OQME process. The response to the Standards Template was positive, respondents being favourably disposed towards the attempt to streamline quality assurance protocols.

84. Respondents made many helpful suggestions for modification of the processes outlined in the consultation document. These are recorded at the end of each chapter. In addition, a number of concerns were expressed, including the apparent workload for placement providers undertaking OQME. Many HEI respondents did not appreciate fully that OQME would fulfil existing annual monitoring quality assurance process within their institutions. Also, Figure 3 in the consultation document caused high levels of concern about the role of the professional bodies in approval. Some respondents noted significant evidence missing from the evidence base required for the Partnership Framework.

85. With the exception of the format of the Standards Template, at least 88 per cent of the respondents considered the issues raised by the questionnaire to be important to them.

86. The next stage of the process in developing the Partnership Framework is to test operationally the OQME and Approval processes, the evidence base and the Standards Template through a series of prototypes. These will take place between September 2004 and May 2005. To this end a number of the suggested changes, highlighted in this report, have been incorporated into documentation for the prototypes. Others are awaiting the outcome of the prototypes. The evaluation and lessons learnt from the prototypes will feed into the preparation for the full implementation of the OQME and Approval processes in the academic year 2005-06.

 

Appendix 1: Profile of respondents

Of the 104 returns made, 22 were written responses not using the questionnaire and, therefore, quantitative data are not available. The following table represents the overall profile. 'Mixture' refers to roadshow returns where mixed groups of participants submitted joint responses.

Returns as at 9.00am Friday 11 June 2004

Total: 104 returns

Roadshows = 21

Return directly = 83

Organisation	Number	Percentage of total responses
HEI	42	40.4
Practice	33	31.7
PSRB	7	6.7
SHA/WDC	10	9.6
Other	9	8.7
Mixture	3	2.9

PSRB
British Association of Arts Therapists
The British Dietetic Association
The British Psychological Society
Chartered Society of Physiotherapists
College of Occupational Therapists
The Society of Radiographers

WDC/SHA
Avon, Gloucestershire and Wiltshire
Cumbria and Lancashire, with Greater Manchester
Hampshire and Isle of Wight
North East London
South West London

Other
Council of Deans
Independents
Joint Medical Advisory Committee for the UK HE funding councils
Pan London Dietetic Managers
Standing Council of Principals
Universities UK
Better Regulation Review Group
The roadshows were attended by 140 people. The profile of organisations represented at the roadshows follows.

The quantitative returns show the following profile:

Respondent	Frequency	Percentage
WDC/SHA	12	14.8
Trust	26	32.1
PSRB	6	7.4
HEI	29	35.8
Other	3	3.7
Mixture	5	6.2
Total	81	100