QAA report on the evaluation of Prototypes 2004-05
QAA 085 09/05
Contents
- Executive Summary
- Chapter One
- Chapter Two
- Chapter Three
- Chapter Four
- Chapter Five
- Ongoing quality monitoring and enhancement
- Stage 1: Self-evaluation in the local learning environment
- Stage 2: Authorisation and the Initial Action Report
- Stage 3: The Annual Review Meeting
- Stage 4: The Final Action Report
- Partnership working
- Partners and named leads
- Roles and communication
- The QAA facilitators
- Practice placement facilitators/clinical practice facilitators
- The statutory regulatory bodies
- Student and service-user involvement
- Timescales
- Perceived impact/influence of OQME
- Outcomes of the overall process
- Links with other QA processes
- Chapter Six
- Appendices
Chapter four
The approval process
47 Approval is the recognition of a programme by the regulator(s), where appropriate, the commissioner and the programme providers (Prototype Document section 2.1). The reasons for approval may be:
- the development of new provision
- a significant change in existing provision which partners agree requires approval
- a result of OQME and/or major review processes.
48 The approval process enables a decision to be made about the ability of the proposed programme to meet the requirements of a variety of stakeholders:
- regulatory bodies
- programme providers (practice and campus based)
- health and social care providers
- service users
- education commissioners.
All judgements must be supported by appropriate evidence (Prototype Document, section 2.6).
49 Within the approval process there are four key questions that must be answered.
- Does the education, training and assessment meet the requirements for the award, the requirements of regulators, the requirements of commissioners and the needs of the student?
- Is there evidence that the providers' systems and processes meet the requirements of the Partnership Framework for OQME and for major review?
- Do the providers' systems and processes operate in a fair, just and open way?
- Is there evidence that the provision is sustainable?
50 A central part of the approval process is the self-evaluation against approval standards within the 10 aspects. The standards relevant to approval are designed to address the four key questions identified above and represent: academic award requirements, the requirements of education commissioners through contractual arrangements, and relevant professional, statutory and regulatory bodies' requirements.
51 The stages of approval identified in the Prototype Document (2.4) are as follows:
- Stage 1 Commissioning
- Stage 2 Programme development
- Stage 3 Approval
Commissioning
52 Commissioning involves both service and programme providers and results in a decision to commission a programme of healthcare education and training. This decision should be informed by clear evidence that the programme reflects service-user and carer needs, wants and expectations. The approval prototypes were set up after the originating commissioning process was complete. However, there was evidence from the approval documentation provided in two of the prototypes that consideration of local workforce needs had been taken into account, and it was not always clear whether the developments reflected service-user and carer needs, wants and expectations (Prototype Document page 12).
53 As each of the prototypes had provision that included qualifications leading to eligibility for entry to a professional register, the appropriate regulatory body had been notified. Both the NMC and HPC agreed to participate in the prototypes in line with their current procedures. In relation to AHP provision, the HEIs/SHA also invited relevant professional bodies to attend the approval events. The decision to invite allied health professional bodies is at the discretion of the HEI and their partners. Within the approval process, the professional bodies have a specific role in curriculum development. The prototypes were taking place at a time of HPC implementation of their new processes, and some of the uncertainties experienced by prototype sites reflected this period of change. Further discussion is needed to clarify the role of the professional bodies in relation to the standards and the integration of the standards to approval events. A practical issue for providers was the number of panel members, which appeared disproportionate to the activity. This was exacerbated by the HEI/SHA decision to invite professional body representatives.
Programme development
54 The second stage of the approval process is programme development. Here the commissioner and the programme provider should jointly convene a programme development group (Prototype Document, page 12) to produce and sign-off documentation for submission to the approval panel. The Prototype Document (section 2.5) recommends that the membership of the programme development group be:
- HEI 'host' faculty, department or school
- expert/professional advice, eg the relevant professional body(s)
- service users
- commissioner representatives
- service management representative(s)
- practice learning providers
- representation from students.
55 Although not formally set up, equivalent programme development processes were undertaken prior to the beginning of the prototypes. It was evident that all the prototypes had gained some input from the groups above, and a considerable amount of development activity had already taken place. There was little evidence of service users and carers having a significant input to all of the provision in two of the prototypes. However, this was not the case in the third prototype, where there was clear evidence of active and effective involvement by service users and carers in the development phase.
56 It was apparent from the documentation submitted that the development group (or equivalent) had given due consideration to appropriate professional body curriculum guidelines (for AHP provision), HPC Standards of Proficiency and Standards of Education and Training (for AHP provision), NMC Standards of Proficiency (for Nursing, Midwifery and Specialist Community Public Health Nursing) and the elements of QAA's academic infrastructure: subject benchmark statements, the Code of practice for the assurance of academic quality and standards in higher education, and the Framework for Higher Education Qualifications (FHEQ).
Self-evaluation against the standards
57 The Prototype Document (page 12) stated that the programme development group should also use the approval standards as a point of reference. This requires both the programme providers on campus and the associated practice placements to undertake self-evaluation against the standards, which results in exception reporting of issues, standards 'at risk' and good practice or innovations. The self-evaluation and the documentary evidence to support the identified 'exceptions' should form the basis of discussions during approval, along with other documentation. It was the expectation that the placement self-evaluations should be then incorporated (with a direct link to the proposals, rather than as an adjunct) into the documentation to be considered at the approval event.
58 Two out of the three prototypes undertook programme development in the manner required by the process. However, one HEI/SHA did not undertake exception reporting, although a sample was undertaken of self-evaluation by practice providers and by the central quality assurance department in the HEI. This prototype noted that the Prototype Document gave no indication that exception reporting was required in the approval process. Therefore, a recommendation of the evaluation is to make more explicit the need for exception reporting in approval. Where self-evaluation against the standards was undertaken (prototypes 1 and 2), there were some very positive comments about their use. The completion of the self-evaluation 'had been a very helpful exercise for both the University and the Trusts concerned'. One practice manager reported that they found the standards difficult to work with at first but they were easier to understand with increasing familiarity. The collection of evidence through areas working together was reported as 'inspiring' by two Visitors. The process was useful for identifying gaps and creating action plans for all levels within partnerships. It was noted that one prototype had envisaged particular difficulty in engaging and undertaking the self-evaluation of placement providers in the independent sector and community placement areas and therefore these had not been included in their sample for self-evaluation.
59 The questionnaire data (both qualitative and quantitative) indicated that respondents considered that the self-evaluation process generated a significant amount of paperwork, far more than in other approval processes, and required additional resources. It was noted by HEIs, SHAs and Trusts that there were too many standards in the template. Four Visitors commented negatively about the amount of paperwork generated and the time available to read it, and for two Visitors there were concerns about the cost of the process. One Trust undertook a cost analysis of the exercise. Although useful, it did not undertake a comparative cost analysis with previous QA activity. The labour intensiveness of the self-evaluation process was highlighted in the questionnaire responses and the focus groups as a negative aspect of the process. However, this was tempered by the perceived benefit of enhanced partnership relations and more interprofessional collaboration facilitated through completing the standards template and enabling a broader range of those involved in practice to have the opportunity to be involved in and take responsibility for the approval process.
60 There was some concern amongst the respondents that the process of concatenating the self-evaluations into a single document to be considered at the approval event could result in the dilution of issues for verification at the approval event and in small disciplines being lost, particularly where a large number of programmes are approved at one event or where one discipline is dominant. Ensuring sufficient specific detail in the original standards templates could prevent this.
61 It is evident that those involved in the prototypes recognised the importance of undertaking significant development work with practice in preparation for the self-evaluation. Leadership of the process, including developmental work and implementation, was found to be crucial. In Trusts, the leadership was provided by the PPFs and CPFs, or equivalent roles in Trusts. There is no apparent consistency in titles; however, PPFs, CPFs or their equivalent take responsibility for supporting students and assessors/mentors, and for promoting a positive learning environment in practice.
62 An issue that arose from the data analysis was responsibility for the signing off of standards. Two prototypes noted that it was not always clear who should be signing off the standards and at what level of seniority, and thus confirming that the evidence is available to support the self-evaluation.
The approval event
63 Membership of the approval panel followed the Prototype Document recommendations in the main. It was noted that there were many participants at meetings and that meetings tended to be long. One Visitor noted that this could be 'very tedious', although the inclusion of course teams, when this occurred, was felt to be a very positive feature. One prototype noted that setting up the panels took a great deal of organisation and administrative time, particularly where the panel was large. One respondent to the questionnaire and one observer noted that the dual role of SHA staff meant that they could be a member of both the approval panel and the programme development group, and considered that this could lead to a conflict of interests.
64 User involvement was evident in some meetings. Two of the prototypes (prototypes 1 and 3) had users and/or user representatives on the programme development group equivalent, and one prototype had a former student on this group. On other programmes, user and student input did not seem to have occurred or it was not clear how input had been achieved. In one case, student meetings were held before the main approval event but were not particularly productive, mainly because the number of students was too small and the meetings were dominated by professional body concerns. In the remaining events, meetings with students took place during the approval event and these had been very informative.
65 In most cases, panel members had not received, or only received shortly before, the standards template and exception reporting documents prior to the approval event, and this was identified as a problem, particularly by the Visitors.
66 None of the approval events followed the prototype methodology in full. In some instances, the methodology had not been sufficiently adopted at earlier stages in the process to allow the approval meeting to use the exception reporting outcomes. In other cases, the initial work had been completed but at the meeting the prototype methodology was put on one side and the HEI's normal validation processes utilised. With one exception following a QAA prompt, it would seem that the validation Chairs did not explicitly direct the panels to discuss standards or the exception reporting. More guidance is requested on the logistics of running an approval event.
67 Even though discussion in the meetings did not specifically refer to the standards, all approval events followed the principles of the approval methodology. The four key questions were discussed briefly and it was recorded that they had been met, although detailed discussion did not take place. In one case, an account of how the provision met the standards was appended to the approval documentation after the approval event.
68 Although the prototype methodology was not used to its full extent, most approval events were observed to be efficient, comprehensive and thorough, with a good level of debate. An NMC Visitor reported that the approval process was rigorous and there was an opportunity to see the completed standards template and the evidence used to make the judgement. Another Visitor reported that it was an enjoyable experience and a learning experience for all involved. However, three approval events were reported as being 'bogged down in minutiae' or too focused on technical detail that could have been dealt with elsewhere.
69 The HEI undertook the responsibility to write up the minutes of the approval event and produce the subsequent action plan. These were circulated, with the exception of one prototype, to all stakeholders for agreement. In one prototype, there was some initial confusion about the role of the NMC in accessing and agreeing the minutes, the HEI determining that the document was for internal use only. The prototype documentation did not comment on the reports produced from an approval event. During the prototypes, it became clear that such advice was needed. QAA provided that advice, identifying that there were two sets of documentation that stakeholders need to agree. The minutes/notes of the approval event and the outcomes of the approval event in terms of actions agreed. Future guidance is needed to clarify the responsibilities of members of the approval panel following the event.
Fit with HEI/SHA quality assurance processes
70 Two respondents to the questionnaire felt that the approval process fitted completely with the existing university/SHA quality assurance processes. The remainder, including the Visitors, felt that there was a partial fit. However, all the observers noted that the prototypes still used the HEIs' 'normal' approval/validation process to a greater or lesser extent. One observer considered that the prototype approval process was not consonant with HEIs' 'normal' validation procedures and that this might be part of the reason for it not being wholeheartedly adopted. Generally, all parties represented at the approval meetings felt safer with their tried and tested methods. The culture of approval from an HEI's perspective may be to adopt a defensive position; for example, one HEI produced a written response to the exception reporting which was very defensive in tone. The notion of a standard being 'at risk' runs counter to the notion of a secure base from which to launch a new programme. This would suggest that adoption of the new process would require more development effort to initiate a culture change and alter the terminology/categories to make them more user-friendly.
71 It was observed that, although meetings did not make explicit reference to the PQAF standards, the standards template could have covered all issues that were discussed. At one meeting, an intervention by a QAA observer demonstrated how exception reporting had already identified an issue and described steps that would be taken to remedy it, which the approval panel were considering to be the basis for a condition/recommendation.
Verification
72 One purpose of the approval event is to verify the quality of the provision as demonstrated through the self-evaluation and approval documentation provided. While there were no explicit requests to see the supporting evidence by the approval panel, verification was undertaken orally. However, 75 per cent of the respondents (including the Visitors) considered that the evidence was not explored fully in the approval event, and 66 per cent considered that it had not been verified. However, 66 per cent of the Visitors thought the evidence base sufficient to enable them to undertake their role as a Visitor, but with some instances of Visitors requesting additional documentation prior to or at the approval event.
73 There was some confusion regarding the NMC Visitor's role in verification and how this could be achieved. Two Visitors reported that they would want to visit practice areas before the event to meet students and staff to help ensure that regulatory requirements were being met. Others reported that taking part in the verification visit was helpful and necessary. In one prototype, it was not clear that the verification visit was a single-day event, with Trust staff anxious that there might be further visits. NMC officers helpfully allocated the same Visitors to both OQME and approval processes for prototype 3, the aim being to see if this helped in the dialogue and verification process. Where possible, NMC also arranged the allocation of the Visitor who had previously undertaken annual monitoring of the provision. The benefits of continuity clearly outweighed any potential disadvantages. Regulators need to consider how this continuity may best be achieved.
Statutory regulatory bodies
74 Both the NMC and HPC agreed to participate in the prototypes in a manner appropriate to their current approval procedures. For HPC, this meant that HPC Visitors were present at the approval event for an AHP where new provision was being approved or where there was a significant change to an existing curriculum. In prototype 3, the HPC deemed that, in accordance with their procedures, there had not been sufficient change in the curriculum of two programmes to warrant the presence of the HPC; note would be taken of the final approval documentation. Therefore, five HPC Visitors were involved in one prototype (prototype 2). The HPC reviewers who commented found the documentation to be too lengthy and complex and so used HPC documentation and procedures. An explicit cross-referencing of PQAF standards to HPC standards had been carried out, which should have facilitated more explicit involvement of the HPC in the approval process. However, this cross-referencing needs to be widely shared and made visible to participants in the PQAF.
75 NMC Visitors were involved with all three of the approval prototypes, with Visitors from the appropriate Parts of the Register in attendance. The majority of the Visitors who completed the questionnaire felt that their role was clear and as expected (83.3 and 71.4 per cent); 66.7 per cent noted that the QAA/NMC/HPC briefing day was clear and helpful, and 83.3 per cent that the preparation received prior to the prototypes was clear and helpful. However, one Visitor felt that their role in the approval event was unclear and as a result they could not wholeheartedly report that regulatory requirements were being fully adhered to. The lack of understanding about the role of the Visitor from the perspective of the HEIs, SHAs and Trusts was also apparent from both the questionnaire data and focus groups. One Visitor noted that some HEIs had little understanding of the importance of the Visitor role and lacked appreciation of the expertise of the Visitor. More clarity about what is required from Visitors, and what evidence regulatory and professional bodies require, is needed.
76 Two of the six Visitors who responded to the questionnaire felt that their role worked well within the approval process, and the rest felt that it worked in part. There was a mixed response as to whether the evidence base for the approval event was verified and whether it would allow them to fulfil their regulatory role. However, the observers commented that the Visitors tended to revert to their known NMC mode of working during the approval event. This might be a result of the HEI's traditional approval processes being utilised rather than the nature of the prototype approval process itself. In two cases, it was observed that the Visitors dominated the approval meetings. Overall, none of the Visitors who responded to the questionnaire felt that they were able to fulfil their role as a Visitor in full.
Student and service-user involvement
77 The approval process includes specific roles/involvement for the students and service users in the commissioning and programme development (see paragraphs 48 and 52). While there was evidence in the prototypes that there had been student and service-user involvement in two prototypes, the degree of their involvement was not always clear. It was also noted from the focus groups that the timeframes for the prototypes did not enable full engagement with students or service users.
78 In three of the prototypes, service users or service-user representatives participated in the programme development, making a positive contribution to the approval (see paragraph 64). All the prototypes included, more usually as part of the standard HEI approval process, meetings with students (see paragraph 64). One prototype approval event included a former student on the subject panel.
79 The quantitative data arising from the evaluation questionnaire noted that 62.5 per cent of the Visitors who responded to the questionnaire agreed that stakeholder involvement was facilitated by the approval process, compared to 55.6 per cent of the respondents from HEIs, SHAs and Trusts.
Partnership
80 A positive outcome of the self-evaluation against the standards, noted by a significant number of respondents to the questionnaires and by the focus groups, was the strengthening of partnership between the Trusts, the SHAs and the HEI. There was a very strong message that the approval process facilitated the stakeholder involvement and that it enabled practice placements within Trusts to take responsibility for the education provision. There was some difficulty in ensuring that the independent sector had an input to the process.
Leadership
81 It is recognised in all the evaluation activity that strong leadership in the Trust or placement organisation encourages the appropriate completion of the self-evaluations and, as one observer suggested, would ensure that the evaluations hold a central place in the approval process. Whilst not all of the prototypes had this strong leadership, the role of the CPFs/PPFs, where they were in place, appears to be crucial for the process to work well and to ensure full participation from the practice areas.
82 Strong leadership in the HEI both at the faculty/school or departmental level, and close links with any central quality assurance department, are essential for the process to be carried out appropriately. Where these were in place, concomitant with a thorough grasp of the process by those with leadership responsibilities and those chairing the approval event, the approval process followed the prototype approval process more closely.
Resources
83 The approval process appears to demand a significant resource input and a suitably planned lead-in time. Almost all Visitors agreed that the large amount of paperwork, numbers of participants and potentially the number of programmes approved meant that a significant amount of time was required. There was concern that this might be costly and place the supporting infrastructures under strain. This was demonstrated through the perceived late production of documents. However, this might be predominantly because of the shortened timescales available for the prototypes rather than as a result of the process itself. The process was reported as being very labour intensive by the HEIs/SHAs/Trusts and by participants in the focus groups. However, the quantitative data from the questionnaire would indicate that the respondents considered the workload to be either about the same or less than previous methods. Almost all felt that the processes could not be undertaken within existing resources.
Facilitation and communication
84 There was a mixed response as to whether the prototype documentation was helpful in preparing for the prototype and completing the self-evaluation. The NMC visitors found the Prototype Document helpful, while the HEIs/SHAs/Trusts thought it unhelpful in explaining exception reporting and running the approval event. It was felt that the process was explained but interpretation was vague, leading to difficulty and misunderstanding. Further guidance was published by QAA in March 2005 on the use of the self-evaluation against the standards and exception reporting.
85 Evidence indicates that the prototype sites did not have a clear understanding initially of the role of the QAA facilitators. In time, the facilitators were used predominantly for briefings and as sounding boards and, while welcomed, were not always used to full advantage. This would indicate that a rigorous programme of briefing prior to any future rollout is required.
86 The prototype sites recognised that clear communication was crucial for implementing the approval process; where clear channels of communication were in place there was better involvement and engagement with the process. Communication between the Visitors and the prototype sites was variable; however, where a CPF/PPF was in place, communication was strengthened.
Chapter five
Ongoing quality monitoring and enhancement
87 OQME is a process by which education commissioners and regulators satisfy themselves that the quality of education programmes provided by HEIs and placement providers is maintained and improved (Prototype Document, section 3.1).
88 The four stages of the OQME process are as follows:
Stage 1 Self-evaluation in the local learning environment
Stage 2a Authorisation at organisation/institution level
Stage 2b Production of Initial Action Report (IAR)
Stage 3 Annual Review Meeting (ARM)
Stage 4 Final Action Report (FAR)
Stage 1: Self-evaluation in the local learning environment
89 Evidence serves three purposes (Prototype Document page 21):
- it gives validity to the claims being made, supporting local judgements
- it enables comparisons to be made between different placements or programmes
- it provides a means of measuring quality.
90 A key decision to make early in Stage 1 is how to choose what to sample in the local learning environment and where evidence should be sought. Different approaches were taken by prototypes within the framework. For placement areas, a sampling approach, for example 50 per cent, was taken by some and a clustering approach by others. In the latter, rather than looking at a very large number of individual placement areas, including some that because of the short time-scale were unable to accommodate a new process, placements were clustered into larger groupings. This raises the questions: what does a placement unit consist of, and at what level? It became clear that, because of the time constraints, placements deemed too difficult to embrace in the sampling were avoided. These included those spread widely on a national scale and also most of those in the independent sector. It will be important to include these in the future. Some participants pointed to the need to determine a standard sampling procedure that precludes the need for self-evaluation of all programmes and all sites every year. Others, however, emphasised the need to meet contractual obligations on an annual basis at the same time.
91 Sampling in the campus environment poses similar problems. It is important to give appropriate weight to post-registration as well as to pre-registration provision. It is also important to link campus and practice evaluation. The prototypes showed a lack of explicit evidence of this link taking place.
Exception reporting of evidence
92 The Prototype Document (page 17) gives guidance about exception reporting. This is fundamental to the process and involves reporting only for those standards judged at the level of the local unit to be:
- at risk
- demonstrating innovation and good practice
- showing change since the last review.
93 The definition of exception reporting was often not clearly understood. This points to the need for developmental work as part of standard practice. Participants suggested that it would be helpful in future to distinguish good practice from at-risk reporting by using an appropriate form of coding.
94 The Prototype Document emphasises that it is not necessary for evidence to be produced, but that it should be available for scrutiny if requested. However, the OQME prototypes have thrown up a challenge for clinical placement staff. Participants commented that many clinicians do not routinely generate evaluative evidence in relation to their own practice, nor the learning of students on placement. This means that there is a significant challenge in ensuring the effective completion of the standards template by clinical staff. The initial and effective briefing of clinical staff is critical to the successful implementation of the process.
95 Self-evaluations showed some indication that clinical participants found it easier to identify shortcomings and more difficult to see strengths and good practice. To give equal weight to both positive and negative requires a shift in culture. Variations in reporting meant that some Trusts identified good practice that others consider a routine part of their work. This points to the need for a decision-making process to determine what is good practice, new ways of identifying it, and a process for disseminating it.
96 Overall, Trusts were enthusiastic about the self-evaluation process. They saw it as allowing them to gather evidence in relation to their responsibilities that they would otherwise be unable to do. Evaluation events showed that most staff approve of the idea of self-evaluation. It promotes explicit recording and Trust staff felt they could be more open and give explicit detail.
Completed templates
97 There was wide variation in the quality of the locally completed templates for OQME. Clinical areas made a very positive effort to abide by the intended spirit of the process. Several participants noted that it gave them a feeling of actual significant involvement, of having 'a say in the provision for the first time'. HEI involvement at the programme level, on the other hand, was generally less clearly visible. The lack of reference to external examiners' reports and to the academic infrastructure in exception reporting underlined this. On both sides, there was often a lack of sufficient detail or specificity to allow the particular location, programme or sub-discipline to be identified, other than at local level. The logistics of completing a template for each placement were found not always to be straightforward. Some were completed peremptorily, suggesting the need to scrutinise the evidence upon which the judgements were based. This might be the kind of trigger for requests to see the evidence in order to overcome a perceived lack of rigour. However, many were filled in precisely the way intended and needed.
98 A point of significant concern is the progressive generalisation of the self-evaluation exceptions in moving through the IAR stage to the FAR stage. This 'filtering' process inevitably resulted in a loss of specificity of the exceptions reported to programme and unit, and precise responsibility for taking subsequent action. For example, in one prototype, there was a complaint made at an interim meeting that the detailed self-evaluation work that had been undertaken had not been utilised fully. In another, good practice from Trusts was criticised as being under-represented in the amalgamation process. This suggested a conflict about the need to keep documentation tight and economical while not running significant risk of conveying the impression of a lack of rigour. One provider noted the rigorous way in which those locally responsible completed their templates.
99 The emerging evidence base is proving very useful to Trusts. It has been found to be directly related to standards and would not have been assembled through existing procedures. In one subject area, for example, after producing their local self-evaluations, staff were reported as having already shared their exception reporting and were in the process of implementing action plans. However, through the process of reducing the commentary to manageable proportions, this excellent and reliable work had become invisible. This reflected the common experience of core evidence not generally being identified at IAR level. One participant commented that the templates simply did not reflect the reality and depth of what had been achieved. Therefore, visibility of the evidence is an issue that needs to be addressed. Ways of mapping and signposting evidence could be developed in line with increasing engagement with electronic means of collation.
100 A simple device to increase the visibility of the evidence is to reinstate the self-evaluation template column that was present in the draft Prototype Document. This required a list of evidence sources for each standard. However, the judicious use of this column is important. It is not sufficient here to point only to policies and strategies in place. Rather, the ways in which such policies are enacted provide the quality of evidence required. It will be important to give further clear guidance on what is needed in this additional column. The key outcome here is to ensure that robust verification is facilitated.
101 It will continue to be important that one particular area of evidence is not over-summarised. This is the quantitative evidence in statistical form of students' progression and achievement. These statistics are essential to the Teaching Quality Information needs of the HEIs as well as the contract monitoring needs of the SHAs.
102 Experience of the prototypes also points to a clear need for significant staff preparation and support for self-evaluation and appropriate use of reference points. For example, consistency of practice in relation to adequacy of clinical placements at Trust level is an important issue but does not mean much to the individual practitioner who does not have wide experience and specific preparation for the exercise.
103 Student views of standards are captured, generally by their HEIs, through a number of established mechanisms, such as module, programme, placement and exit questionnaires. However, it is acknowledged by participants that the input from service users on the standards of healthcare education is both unclear, unsystematic and in its infancy.
Stage 2: Authorisation and the Initial Action Report (IAR)
'self-evaluation should result in an initial action report which the Annual Review Meeting should then consider in depth' (Prototype Document 3.2)
104 Local self-evaluation is confirmed by a signature from the named person in each local area. The named person within the HEI might be a programme leader for a specific programme. For placement providers, the named person could be a ward or department manager or an individual healthcare practitioner. The signature indicates that there is evidence that the standards are self-evaluated accurately in the local environment. If standards have not been met, a supportive commentary is provided, on the reporting sheet for each cluster of standards, on the basis of exception reporting. The self-evaluations across the whole standards template then need to be signed at an organisation-wide level by the HEI and Trusts to indicate that the appropriate standards have been achieved with reporting on an exception basis. In the case of some Trusts, good practice was demonstrated in an iterative process of dialogue between directorate and lead level to produce the IAR content.
105 The timescale for the completion of the self-evaluation process was set for early December. Providers sought various ways of ensuring that the project was on track to meet the various deadlines agreed. In one case, this involved a networking day, half-way through the project, for all Trust Leads and Chief Executives. Another prototype instituted a critical reading day with key stakeholders, including the facilitator, in preparing the IAR. By mid-December, sharing of information and consolidation of evidence with the assistance of NMC Visitors was well advanced. The IAR needed to be completed ready for distribution to stakeholders four weeks before the ARM (this timing was a concession for the prototypes; in real terms, external stakeholders need the reports earlier).
106 With the production of the IAR, emerging themes became apparent from the exception reporting. Where providers produced a thematic analysis based on exceptions, this was perceived to be very helpful. The following eight themes were articulated by one prototype but were also common to many of the others:
- mentor preparation, mentor updates and live registers
- practice learning capacity
- monitoring and managing underperformance of students
- Trust involvement in curriculum development and setting learning outcomes
- partnership working in recruitment and selection
- sharing and using student feedback on practice placements
- practice assessment and inter-assessor reliability
- the role and value of PPFs/CPFs.
107 The emphasis on the practice part of the provision is obvious from these themes. However, named responsibilities do also include exclusively HEI ones, such as learning resources in the academic environment. 'At risk' exception reporting was balanced by commendations of good or innovative practice. However, the need for clarity around these definitions was apparent. For example, in one case, some strengths were identified also as good and innovative practice. Later, in this instance, it was telling that the FAR indicated 'no action' against all of these.
108 The way in which the IARs were pulled together from the self-evaluation evidence varied significantly. One example succeeded in retaining the detailed and comprehensive evidence at programme and placement level. However, in the majority, the core evidence was not apparent, making it difficult to sign off the IAR reliably. The key message here is that clear and consistent verification of the standards is paramount. Some providers developed very efficient electronic means of grouping the self-evaluations of Trusts together, compiling, cutting and pasting exception reports on screen to provide a summary of the evidence. One SHA (not a prototype site) is currently piloting an electronic shared-practice database that may in the future provide an effective and timesaving tool. This links with the perceived need by many placement providers for a standard placement educational audit tool that could be used with economy of effort throughout the country.
109 An important point emerged in relation to timing. Is the standard met once the actions are specified? A number of actions resulting from self-evaluation may well have been clearly dealt with by the time of the ARM. These should be signalled in the version of the IAR that goes to stakeholders attending the ARM.
Stage 3: The Annual Review Meeting
'self-evaluation should be subject to rigorous verification in the Annual Review Meeting...and supporting evidence be made available if needed' Prototype Document
110 The aim of the ARM is to focus on quality enhancement and to meet the need for quality monitoring by the stakeholders. It therefore needs to accommodate the requirements for quality monitoring of each stakeholder. Overall management of the prototype ARMs was in accordance with the protocol. They were coordinated and chaired by the lead education commissioner or their representative in agreement with the NMC and in collaboration with the HEI. Representation by individuals from placement providers was at an appropriately senior level although, in one case, was inadequately small. It is important that representation is equitable to help enhance shared ownership and genuine partnership.
111 It is clear from feedback and observation that the kind of robust debate that is perceived as essential to the rigour of the OQME process was difficult to achieve in the ARM. It is important to clarify the expected ethos of the event. The five that were held varied across a wide spectrum from a kind of formal contract review to participative problem-solving. All formats had strengths and weaknesses. There was a lack of testing of the evidence base in all. In two cases, the lack of disagreement in the ARM was identified as a consequence of it appearing too risky, with debate therefore being kept to a minimum. In most cases there was a filtering of issues thought to be significant enough for the ARM. Therefore, not all those that were important to those stakeholders present were considered. Moreover, practice learning featured strongly in all the ARMs, sometimes to the detriment of consideration and verification of claims made about campus learning.
Table 3: A summary of the ARM process across five prototypes
| Provider | Process | Strengths noted | Weaknesses noted |
|---|---|---|---|
| Prototype 4 | 32 attendees. Chair: SHA nominee, supported by four SHA staff who acted as recorders and facilitators. Plenary, followed by six workshops, followed by plenary. No students or service users present. Each workshop group discussed the relevant standards and exception reports from the IAR and then fed back recommended actions. NMC not consulted about agenda. They were unable to confirm at the end of the meeting whether or not their needs had been met. |
|
|
| Prototype 5 | 33 attendees. Chaired by partner Trust Chief Executive. Each agenda item addressed, with opportunities for partners to contribute. No students present. NMC Visitors actively involved by Chair and confirmed their needs had been met. |
|
|
| Prototype 6 | 25 participants, including observers; 2Þhr meeting chaired by SHA nominee. Agenda carefully set in relation to the comprehensive IAR. All stakeholders represented except students and users. NMC Visitors did not shape agenda but confirmed their needs had been met. Discussion led by HEI Lead. |
|
|
| Prototype 7 | Workshop format, with about one hour devoted to one, two or three aspects together; introduced by Ambassador; followed by discussion of risks, action plan, time scales and responsibilities. Ambassadors were appointed per aspect prior to the event and, before each discussion, they were asked to brief the group regarding the aspect and what they would like them to consider. All feedback from each table taken away and taken into consideration in the production of the FAR. |
|
|
| Prototype 8 | 43 participants. Very minimal attendance by partner Trusts. Lecture theatre layout. Chaired by WDC Senior Officer. NMC participants in general audience. Therefore, question and answer format: panel, chaired by SHA and including observers facing ranked attendees. Well-managed meeting, with high standards of discussion and scrutiny. |
|
|
112 Prototype 6 illustrates one possible model of good practice in enhancement. The stakeholders felt strongly that the ARM should be used as a developmental activity, in addition to its function as a rigorous monitoring event. They considered that it should be used to promote cross-organisational and interprofessional discussion, the sharing of good practice and the search for solutions to issues. To ensure that each item in the IAR was given due consideration and debate, and that stakeholders were able to shape the FAR and gain ownership of it, they were asked the following.
- Do you recognise the exception in the IAR?
- What are the three key risks associated with this exception?
- Do you believe the suggested action in the IAR is appropriate?
- What other three suggestions do you have for action?
113 The lack of a student voice at any of the ARMs gave cause for concern, as did the absence of service-user representation. Representatives need to be specifically invited and involved in agenda setting. The absence of clear student feedback to support the achievement of standards is a significant problem for some clinical areas.
114 In closing the ARMs, the timing and route for the FAR were not always clear and agreed in the way that they need to be.
Stage 4: The Final Action Report
'...discussion at the Annual Review Meetings should result in a final action report, which is shared and agreed by all stakeholders' (Prototype Document 3.2)
115 An integral function of the FAR is to identify and disseminate good practice. All partners need to sign off the document. There is a need for action plans to be formalised at Trust and HEI departmental level if the process is to be effective. Shared ownership of the FAR by all the stakeholders is an important principle.
116 A number of significant shortcomings were identified in four of the five prototypes at the interface between the ARM and the FAR. In one, hardly any of the exception reporting and planned actions focused on the HEI or the standards for which it was responsible. In this case, it was also noted that the FAR was identical in content to the IAR. This would indicate that the ARM served no significant function here. Also, the lack of evidence base and verification in the FAR meant that it would not serve adequately as a SED for major review. It is important to recognise precisely who takes responsibility for action on what.
117 In two other prototypes, the material for the FAR was derived directly from the IAR. There was 'no action' indicated in relation to anything labelled as good practice. In one of these, a number of exceptions (issues) are listed for development during 2005-06 and the roles of those responsible for actioning these are named. In two others, there was a lack of specificity to programme, location and responsibility. This made it difficult to know whether the high level of generalisation in the FAR will influence or impact on the necessary specific action at the local level.
118 The quality of the FAR in one prototype was exemplary. Exceptions were specific down to programme level, criteria for evaluation were given, responsibility for actions was identified and review dates were stated.
119 Following the ARMs, an emerging need became apparent for meetings to be held through the following year in order to verify that issues within the FAR were being addressed appropriately. This is particularly important where issues are considered 'high risk' in terms of student achievement, public safety, and fitness to practise.
Partnership working
120 Before Stage 1 could commence, key decisions needed to be made and protocols clarified to ensure a well-integrated and effective process to achieve the objectives of the OQME process.
121 Key guidance in the Prototype Document (section 3.2) about self-evaluation includes the following:
'self-evaluation should be subject to rigorous verification in the Annual Review Meeting...and supporting evidence be made available if needed;'
'self-evaluation should take place against agreed standards from the shared Approval and OQME Standards Template.'
'self-evaluation at local level should be authorised by each organisation;'
122 About three-quarters of those returning questionnaires considered that stakeholder involvement and partnership were facilitated and promoted by the OQME process.
Partners and named leads
123 There was a significant effort made to engage all Trusts and stakeholders in the independent sector. Initially, for logistics and engagement reasons, the numbers of Trusts and SHAs involved needed clarification. In most cases, the sound practice was adopted of using named leads in each placement provider for receiving and actioning documents. By the time of the planning meetings, those responsible for leading the prototype, whether from the HEI or SHA, were clearly identified. Where this was bipartisan, it clearly emphasised the key principle of partnership between HEI and practice components of the education. It will always be important that named leads are clearly identified in advance of self-evaluation commencing.
124 The importance of engaging Trust Chief Executives to help with the initial engagement with the process within the SHAs was noted. In one case, this was true from the start with, for example, an assistant director of nursing holding a joint appointment with the HEI. Good practice was evident in the production of two information sheets, one for clinical staff and one for Trust educational leads. These are two and three sides long respectively and provide succinct, explanatory summaries of OQME and the staff roles within it. This HEI/SHA partnership has developed further good practice by providing individual briefings for every Trust, so that clinical staff know exactly at what level they should be working. Link lecturers became involved with clinical staff as part of their ongoing partnership process as they, the link lecturers, have a wider experience of what contributes to a quality practice learning environment.
125 The evaluation events also identified another key partnership link that was enhanced by the OQME process. This was between health visiting, midwifery and nursing on the one hand and AHPs on the other. They discovered that working more closely together resulted in greater accountability.
Roles and communication
126 There were a significant number of partners and stakeholders involved in the OQME and approval prototypes. The communication protocol stressed the importance of effective, focused and clear communication and outlined the communication principles and channels agreed by the partners and other organisations. The principles were stated as follows:
- communication channels should be as simple and clear as possible
- wherever possible, the communication between partners and stakeholders should be through identified lead person(s)
- prototype organisations should experience as little disruption as possible
- prototype coordinators (QAA) and the external evaluators will agree specific processes.
127 A lack of clear and effective communication between key stakeholders was apparent in some prototypes at the start. Strong leadership is required to ensure that adequate communication is maintained between the key players in the HEIs and SHAs. Evaluations showed that the OQME process brought the positive benefit of putting education at the forefront in Trusts, particularly at board level. Participants welcomed websites to facilitate communication where these were developed by prototypes. However, a worthwhile attempt by one provider to use electronic communication to elicit self-evaluations from distant sites was unsuccessful. In this case, none of the five providers of one AHP submitted a self-evaluation.
128 The quality of communications remained a key ongoing theme throughout the prototypes. In one case, relating to academic development, discussion showed that differences of opinion between Trusts and the HEI were still evident at the ARM. In another case, a senior academic clearly felt isolated in the task of preparing the IAR. It had been difficult to translate the plethora of detailed and location-specific exceptions in the templates into a coherent set of messages for the IAR. Evaluation events showed that, where channels of communication and support were in place, there was widespread involvement and returns of self-evaluations.
129 However, a clear message also emerged that the OQME process led to improved communication between partners at Trust level. In one prototype, many partnerships were formed, building on strong collaborative working within the educational provision since major review had taken place. In one HEI, subject staff had not previously had the level of contact with supervisors that the OQME process now required. They stated that, 'Once started, this is a very positive dialogue that will help staff relationships and students.'
The QAA facilitators
130 The QAA facilitators were QAA contract reviewers with significant experience in the review of healthcare programmes. The facilitator role was to work with the HEI, SHAs/WDCs and Trusts to help coordinate the implementation of the prototypes. Each was allotted 10 days for this role. However, without exception, providers and partners found it difficult initially to engage effectively with this role. One facilitator commented early in the process that the partners had 'not sought to involve me as QAA facilitator except when they thought it was a specific requirement,' and found himself party to a very limited amount of communication. Later, facilitators were used increasingly effectively in providing briefings for clinical staff and for chairs of panels, and became active in early planning and discussion. In several cases, the facilitator was used effectively in helping to clarify aspects of the process and to support the lead persons with specific issues arising or at times of uncertainty.
131 Perhaps particularly valuable was the use of the facilitator in strategic meetings and, more informally, as a critical independent friend. In at least one case, working relationships between partners were so effective that the facilitator role was largely superfluous. Nevertheless, one of the facilitator's functions was to receive copies and monitor all the documentation generated by the prototypes. This did not take place systematically, with the provision of documentation being minimal in some areas. Self-evaluations were received in very few instances. This point is important, not for the prototypes or QAA, but because a constant theme in the prototypes was the level of reassurance external stakeholders can receive. If the rigorous and robust forms of verification are not visible to external stakeholders, integration and streamlining become difficult.
132 Overall, a majority of those returning questionnaires found the facilitator role helpful. There may be value in each provider being supported by a person in the role of 'external critical friend' during the first year of the new process, should it be rolled out, but it should not be essential thereafter. This role might be restricted to correspondence in support of the lead HEI and SHA and/or could involve attendance at early-stage planning meetings.
Practice placement facilitators/clinical practice facilitators
133 The successful completion of the self-evaluation templates at local level was the crucial underpinning of the whole OQME process. The effectiveness of this part of the process in practice areas was most notable in those where PPFs/CPFs provided the support and monitoring needed by the clinical staff. PPFs/CPFs were involved from the outset in giving support to clinical staff and in helping to ensure an even quality of self-evaluation. They may be funded by the SHA, but are Trust-based. In one prototype, the CPF delivered a 90-100 per cent return of self-evaluations from clinical staff. In another case, the CPF carried the heavy workload in successfully producing the collated composite Trust report. However, some SHAs and Trusts have not established these crucial posts, particularly in PCTs. Many that are in place do not have an SHA commitment to continuation of funding. A leadership role is essential for pulling staff together in the maintenance and enhancement of quality. For some placement providers, this role was undertaken by staff in education lead posts; these tended to be separated between allied health and nursing.
The statutory regulatory bodies
134 As the HPC does not have a separate annual monitoring process, preferring to link with existing HEI internal quality assurance procedures, there were no HPC Visitors present at the ARMs. The NMC role is as regulator and stakeholder. The NMC Visitor could ask for any additional evidence they needed to enable them to verify the standards of programmes. Ideally this request would be in advance of the ARM, however this required them to be informed of the ARM and the proposed agenda. The NMC has the right to request further visits after the ARM if it is not satisfied that it can sign off the FAR. It was made clear at an early stage that there could be further NMC visits if concerns about the self-evaluation arose during the NMC Visitor verification visits scheduled for all the prototypes during the December period. This option was not taken up in any prototype.
135 The role of NMC Visitors in the OQME process was a new one, limited to the verification of the effectiveness of self-evaluation at the local level. The role was not perceived to be clear by about half of those returning questionnaires. Visitors were keen to remind subject staff of their need to be assured that signing-off of standards was verified. Indeed, at one site, while the process was deemed to have proceeded effectively, NMC Visitors asked for evidence to support the self-evaluation. It was acknowledged that this level of scrutiny needed to be robust while, at the same time, a shift in the culture of the NMC and Visitor from gathering their own primary evidence and talking directly to individual students needed to be achieved, in line with the intentions of the OQME process. However, visiting practice areas is a key part of their involvement and allows verification of the summary provided by the IAR. Some Visitors reported finding that what happened in practice was much better than the impression they had gained from the documentary evidence.
136 There was some confusion and concern about the expectations of the NMC in relation to the ARM. It is important to ensure that SHAs actively involve the NMC, as well as other stakeholders, in the ARM agenda setting. It was noted that some of the NMC Visitors who had attended the briefing day had still retained the culture of previous NMC processes, despite being briefed. Briefing of Visitors was felt to be a positive feature by the Visitors themselves.
137 While the NMC had a clearly-defined approach to and involvement in the OQME process, the HPC only visits the education providers annually in exceptional circumstances. If the HPC needed to visit, it would have indicated its intention to do so. This option was not taken up in any of the OQME prototypes.
Student and service-user involvement
138 Students and services users, as one of the stakeholders in the OQME (and approval) process (Prototype Document, glossary of terms), should be involved either directly or indirectly (through other sources of evidence) in the self-evaluation process and the ARM. As highlighted in paragraph 103, students' views are often captured through established HEI mechanisms. However, one focus group noted that greater guidance in the prototype documentation on how student input should be gained in OQME would be welcomed, for example, a separate section as in the approval process. This might be particularly helpful in addressing the issue raised in paragraph 113 about the lack of student voice and service-user representation in any of the ARMs.
139 The quantitative data from the evaluation questionnaire noted that 62.5 per cent of the Visitors who responded to the questionnaire agreed that stakeholder involvement was facilitated by the approval process, compared to 72.7 per cent of the respondents from HEIs, SHAs and Trusts.
Timescales
140 The sites achieved the overall goals of the prototypes within the timescales. This is a positive feature and deserves congratulation. The timescales for the prototypes were tight, effectively a 7-8-month year. There was less time for stakeholders to prepare than there will be if an agreed version of the PQAF is implemented nationally. This meant that the time needed for gaining the necessary understanding of the process was short. It also meant that most prototypes made a slow start while deadlines for the completion of Stage 1 were looming. The challenging time-lines meant that many placement providers were not able to participate in the OQME process. This underlined the need to ensure that placement providers are given sufficient forewarning for any future collaboration. Planning and progress meetings during the September-October 2004 period formed an important part of the clarifying process for staff centrally involved in it. They brought partners together and began to address decisions that were required.
141 It will be important for any future roll-out that the timing of the process meets the needs of all stakeholders in an equal partnership. All need to be fully involved in order for the production of 'joined-up' evidence. It was suggested by some participants that reconsideration should be given to the frequency of the ARM but that self-evaluation should be annual. However, others considered that there was a continuing need for an annual OQME process, a biennial one would not meet the needs of all stakeholders. In practice areas, the face-to-face audit process that already exists could be enhanced through workshops set up to define exceptions, rather than being the result of individual working. This would help satisfy the need both for enhancement and annual monitoring. There was clear understanding of the time costs of developing and becoming familiar with a new process, although respondents to the questionnaire noted that the process could not be implemented using existing levels of resource.
142 The timescale was tight for all partners, involving much extra work for all. In particular, the timescale of the prototypes did not fit the annual programme review processes of the HEIs. It therefore proved difficult to test the extent to which the OQME process could replace the standard institutional annual monitoring process for the healthcare provision. One institution was clearly committed in principle to replacement. Others have shown an intention so far of running OQME in parallel with their own established quality assurance procedures. This is against part of the spirit of the process which is to reduce the overall burden of quality assurance.
143 If a full year is adopted, it will be necessary to plan the OQME process fully before the start of the annual cycle. Planning should:
- commence at the end of the cycle with publication of the FAR
- build in time to confirm drafts and sign them off
- set the date for the ARM
- plan and identify time to view evidence
- identify the last date for completion locally of self-evaluations and reports by exception
- plan for the receipt of initial action plans by each organisation's formal committee structure and timetable of meetings
- set the date for the receipt of the IAR
- agree the ARM agenda with all stakeholders
- afford the time necessary to update the IAR in preparation for the ARM
- plan the dates for the distribution of the notes of the IAR and the draft FAR
- agree the date for publication of the FAR.
Perceived impact/influence of OQME
144 Providers and stakeholders identified a number of significant positive outcomes within the prototypes in which they were involved. These included:
- a need identified for educational audit to cover all professions in placements. OQME provided a lever that helped to take this forward
- the opportunity to have contact with and learn from different parties participating in the prototypes
- participation in the OQME process enabled better communication between the HEI and placement providers and highlighted the possibility of further improvements in collaboration
- the process points to the need for a standard placement educational audit document used by all providers. It would appear sensible for the standard template to become that document so that: a) there is one document across England, with each placement area completing one self-evaluation of all the HEIs who place students in that environment; and b) that the standards should be evaluated for that practice placement environment, not by the use of separate audit tools for each profession.
Outcomes of the overall process
145 One provider noted that the contributing stakeholders took the process seriously and worked hard to use and test the OQME arrangements effectively. They were very involved in the emerging IAR and the format of the ARM and offered many constructive ideas. Also noted was the complex and, possibly, conflicting role of SHAs. They are an integral partner in OQME, yet they hold the contract. They are responsible for compiling the overall OQME FAR, but have limited control over its content. While this could be difficult, this programme provider did not regard it as problematical because they recorded having 'fantastic support from the SHA and Trusts', not least in coordinating the NMC Visitor verification day.
146 The Prototype Document (page 21) stresses that balance must be sought between ensuring the rigour of the OQME process on the one hand and excessive monitoring on the other that is likely to detract from opportunities to enhance quality. Opinions varied about the comparative weight of work involved, as the following table of questionnaire responses shows.
Table 3: Questionnaire responses about the OQME workload (20 returns)
| How would you rate the workload for the following against previous quality assurance methods? | More than previous methods (%) | About the same (%) | Less than previous methods (%) | Not applicable (%) |
|---|---|---|---|---|
| 1 self-evaluation against the standards | 11 | 11 | 78 | 0 |
| 2 preparing the documentation | 12 | 23 | 65 | 0 |
| 3 preparing for the ARM | 27 | 39 | 27 | 7 |
| 4 preparing for the post-event documentation (eg Final Action Report) | 14 | 43 | 43 | 0 |
| 5 getting documentation signed off | 14 | 33 | 53 | 0 |
| Yes (%) | No (%) | N/A (%) | ||
| 6 was it possible to undertake the prototypes with existing resources? | 56 | 38 | 6 |
147 The workload needs careful monitoring. Providers agreed that the new processes must be sustainable. Some participants regarded it as unmanageable and unsustainable, while others found little to complain about in terms of any excessive burden. Figures in Table 3 suggest broad overall support for the latter view, however the level of representativeness of the small number of respondents needs to acknowledged. The HEI lead and the PPF/CPF workloads were noted as being particularly onerous.
148 One prototype commented on its experience of the way in which the partnership process facilitated productive partnership working: 'We thought we were engaged, but now we're married!' It was felt that Trust placement providers, having found that voice, would take up the outcomes and act on them. Free and frank discussion, they concluded, will influence the sense of ownership of the education and action in relation to it. This has much to do with the development of a culture that embraces organisation for the development and management of change.
149 A clear message emerges that the success of OQME is dependent on clear and established lines of communication within and from the Trusts. There is a need for named contacts to provide clear guidance, support, monitoring and progress chasing.
150 Another prototype site intends to establish an integrated and comprehensive yearly review cycle to include six-monthly interim updates, informed by reports from relevant subgroups as appropriate, and that will meet the requirements of all stakeholders. They regard it as an important step in working towards the Government's agenda of providing a rigorous but more cost-effective approach to quality assurance and enhancement within the HE sector and DH-funded provision. This institution clearly built on the momentum of a 'much richer feel of the partnership' generated by major review, with the action plan items carried forward. Their major review helped them to establish effective partnership relationships and processes and to identify where further developments were needed. An element of their good practice is that they have devolved the work to those staff in placement areas who are at the cutting edge of the provision. The alternative, of an HEI small team going into practice, would have been contrary to the already established principles of partnership working.
Links with other QA processes
151 There was general agreement that OQME must use the established protocols of NHS clinical governance as an operating mechanism if it is to be a sustainable process. However, to shift emphasis in the way needed by OQME was seen as being a big challenge. The stakeholders were broadly of the view that clinical governance systems should be the formal means by which OQME is ensured, but that the hands-on, practical means of prosecuting the self-evaluation processes would be better delegated to the Trust Development Groups (or their equivalent). Some concern was expressed among Trusts that PQAF documentation might become public and affect future Healthcare Commission reviews. It will be important for Skills for Health to test out these issues in its discussions with the Healthcare Commission.
152 Those responding to questionnaires were asked how well the OQME process fitted with existing HEI and SHA quality assurance processes. Thirty-five per cent recorded 'completely', 41 per cent 'partly' and 24 per cent 'not at all'. Clearly there is much ground still to be gained in integrating quality assurance and thereby reducing the overall burden.
153 Stakeholders encountered much suspicion about the process within some HEIs which regard themselves as already having well-established quality assurance and enhancement procedures. In one case, the main exception to the commitment, cooperation and response of partners was seen to be the HEI, which was demanding full compliance with the institutional annual monitoring and refusing to accept OQME in lieu of any part of its established process requirements. It is essential that OQME be integrated with the existing QA systems of all the partners. One senior HEI administrator participant suggested that they 'can live quite happily with their School of Health using a somewhat different annual monitoring process.' They discovered through the prototype that 'the roof had not fallen in'.
ISBN 1 84482 373 3
Top