Outcomes from institutional audit published papers

Institutions' work with employers and professional, statutory and regulatory bodies
Second series

QAA 192 03/08

• Summary
• Preface
• Introduction and general overview
• Appendix

PDF version

Summary

Consideration of the 59 institutional audit reports published between December 2004 and August 2006 shows that institutions are engaging constructively with professional, statutory and regulatory bodies (PSRBs) and working with employers in a wide range of ways.

Many of the audit reports describe mechanisms for obtaining feedback from employers. This may occur formally through industrial liaison groups or employer surveys, or less formally through engagement with PSRBs or with individual employers supporting placement students or work-based learning programmes such as Foundation Degrees. Careers services often provide a central focus for employer liaison. Particular difficulties are identified in seeking feedback from small businesses.

Programme approval, monitoring and review processes often involve employers, particularly - but not solely - in vocational areas of provision. Such involvement may be through consultation about the curriculum and market prior to development, or approval/review panels may explicitly seek employers' views. In some cases, provision is made for employers or PSRB representatives to be panel members, or for approval/review events to be combined with PSRB accreditation. There is less evidence of direct employer involvement in annual monitoring, although some examples of this exist and PSRB reports and associated action plans may be included in annual monitoring documentation.

In many institutions, PSRB requirements and associated professional benchmarks are noted in audit reports as being key to programme design. Employers may also be consulted during programme development, and may be invited to become members of the development team. In the case of Foundation Degrees, there is an expectation of employer engagement and the audit reports confirm that, with a few exceptions, this expectation is generally met.

Meeting PSRB requirements to achieve accreditation may lead to changes in academic regulations or quality management processes and procedures. Some audit reports explain how such exceptions are formally agreed, and describe changes made to align processes such as annual monitoring more closely with PSRB activity. A number of institutions offer accreditation by a PSRB as evidence of the maintenance of appropriate academic standards.

Employers can be engaged in programme delivery in various ways. For example, practitioners are frequently employed as visiting lecturers or integrated into the teaching team in vocationally oriented programmes. In a small number of cases, employers or practitioners are involved in assessment, including being appointed as external examiners.

The HERO Teaching Quality Information website which was superseded by the Unistats website in 2007 (www.tqi.ac.uk/sites/tqi/home/index.cfm) made provision for information about employer links. A number of audit reports note that this information has been made available by the institution. Some also indicate that information about relevant PSRB and employer links have been published in programme specifications.

Some institutions require the results of PSRB accreditation visits to be reported and discussed by the deliberative committee structure, either directly or as part of the annual monitoring process. In other cases, liaison with the PSRB and response to accreditation reports remains at departmental, school or faculty level. In such cases there may not be opportunities for the institution to learn more widely from issues or good practice identified in a PSRB report.

Preface

An objective of institutional audit is 'to contribute, in conjunction with other mechanisms, to the promotion and enhancement of high quality in teaching and learning'. To provide institutions and other stakeholders with access to timely information on the findings of its institutional audits, the Quality Assurance Agency for Higher Education (QAA) produces short working papers that describe features of good practice and summarise recommendations from the audit reports. Since 2005 these have been published under the generic title Outcomes from institutional audit (hereafter, Outcomes...). The first series of these papers drew on the findings of audit reports published between 2003 and November 2004. This paper is based on the findings of institutional audit reports published between December 2004 and August 2006.

A feature of good practice in institutional audit is considered to be a process, a practice, or a way of handling matters which, in the context of the particular institution, is improving, or leading to the improvement of, the management of quality and/or academic standards, and learning and teaching. Outcomes... papers are intended to provide readers with pointers to where features of good practice relating to particular topics can be located in the published audit reports. Each Outcomes... paper therefore identifies the features of good practice in individual reports associated with the particular topic and their location in the Main report. Although all features of good practice are listed, in the interests of brevity not all are discussed in this paper. In the initial listing in paragraph 6, the first reference is to the numbered or bulleted lists of features of good practice at the end of each institutional audit report, the second to the relevant paragraphs in Section 2 of the Main report.

Throughout the body of this paper, references to features of good practice in the institutional audit reports give the institution's name and the paragraph number from Section 2 of the Main report. So that readers can readily refer to the relevant audit report, the name of the institution used when identifying references is the name that appears on the relevant audit report on QAA's website. For those institutions where a change of name has subsequently taken place, this is noted in Appendix 1 (page 14), and is the correct name at the time of publication of this paper.

It should be emphasised that the features of good practice mentioned in this paper should be considered in their proper institutional context, and that each is perhaps best viewed as a stimulus to reflection and further development rather than as a model for emulation. A note on the topics identified for this second series of Outcomes... papers can be found at Appendix 3 (page 18).

As noted above, this second series of Outcomes... papers is based on the 59 institutional audit reports published by August 2006, and the titles of papers are in most cases the same as their counterparts in the first series of Outcomes... papers. Like the first series of Outcomes... papers, those in the second series are perhaps best seen as 'work in progress'. Although QAA retains copyright in the contents of the Outcomes... papers, they can be freely downloaded from the QAA website and cited, with acknowledgement for educational and research purposes.

Introduction and general overview

1 This paper is based on a review of the outcomes of the 59 institutional audit reports published between December 2004 and August 2006 (see Appendix 1, page 14). A note on the methodology used to produce this and other papers in this second Outcomes... series can be found in Appendix 4 (page 20).

2 As part of the institutional audit process that ended in 2006, QAA asked its audit teams to discuss with institutions how they worked with external bodies including, where relevant, employers and PSRBs. As part of such discussions, audit teams were also asked to explore with institutions how they obtained and analysed feedback from graduates and their employers, and employers more generally.

3 Such discussions allowed the audit reports to comment on how institutions informed themselves about shifts in the external environment that might require changes to the curriculum and/or learning environment. An increase in the emphasis given by institutions to work-based learning in higher education through work placements during vocational degrees, or in the development of Foundation Degrees, also provided opportunities for audit reports to consider this aspect of institutions' relationships with employers.

4 Likewise, higher education institutions offering a significant number of vocationally oriented awards were likely to engage with a wide range of PSRBs. In some cases, such bodies might offer the only means of obtaining a licence to practise a particular profession. In others, admission to membership of a PSRB is taken to demonstrate the attainment of a particular range of occupational and professional skills.

5 As seen in the audit reports covered by this paper, the accreditation of a programme of studies by a PSRB usually required an institution's staff to prepare some form of self-assessment. This was then scrutinised (often by a panel appointed by the PSRB) to check, among other matters, curricular content, the skills developed by students and the facilities and resources available to them, and the extent to which the programme met the requirements of the PSRB. Reports of such scrutiny (and any associated visit by the panel on behalf of the PSRB) were then usually sent to the institution, where frequently they were received at departmental or faculty level. In other cases, however, PSRB reports were received centrally (see below, paragraph 30). If a bid for accreditation was successful, accreditation might be awarded in full for a number of years, or conditions might be set. It should be noted that some PSRBs require at least one cohort of students to have graduated before they will consider a request for accreditation.

Features of good practice

6 The 59 institutional audit reports published between December 2004 and August 2006 noted the following features of good practice:

• the active engagement of Industrial Advisory Panels in quality management and course development [Cranfield University, paragraph 196 i; paragraphs 47, 55, 76, 125, 137 and 145]
• the development of Employer Liaison Fellows [University of Luton, paragraph 251 i; paragraph 92]
• the University's regional agenda as evidenced in particular both by student recruitment and by the links with local employers, agencies and practitioners [University of Derby, paragraph 294 ii; paragraphs 114 and 118]
• the role and use of professional advisers, and links with employers and professional bodies [Harper Adams University College, paragraph 189 iii; paragraphs 36, 56 and 61]
• the use of a broad range of external peers, including industrialists and academic staff from institutions in Europe, in periodic review of undergraduate programmes [Imperial College of Science, Technology and Medicine, paragraph 302 i; paragraph 67]
• the strong engagement of the College with industry, as evidenced through the involvement of employers in curriculum design, delivery and review [Birmingham College of Food, Tourism and Creative Studies, paragraph 229 v; paragraphs 70-1 and 101-2]
• the effective integration of professional expertise into all appropriate aspects of the curriculum [University of the West of England, Bristol, paragraph 257 ii; paragraph 94]
• the receipt and consideration at institutional level of the reports of professional, statutory and regulatory bodies [Sheffield Hallam University, paragraph 202 iv; paragraph 63].

7 The reports also contained a number of formal recommendations concerning institutions' work with employers and PSRBs. These related to: establishing systematic policies and procedures for gathering feedback from employers; making engagement with employers a more visible part of quality management arrangements; increasing and making more visible employers' input to subject development; and improving institutional oversight of PSRB reports.

8 Consideration of the sections of the audit reports addressing programme review and accreditation by external agencies, together with sections examining engagement with employers, suggests the following broad themes:

• gathering and using feedback from employers
• the involvement of employers and representatives of PSRBs in approval, monitoring and review
• the involvement of employers and PSRBs and consideration of their views in curriculum design
• changes to regulations and processes in connection with engagement with employers and PSRBs
• PSRB involvement with quality assurance and standards
• the involvement of employers in delivery and assessment
• provision of public information
• institutions' oversight of PSRB reports on their provision.

Feedback to institutions from employers

9 The largest cluster of references to employers in the audit reports concerned the ways in which institutions sought feedback from this group of stakeholders.

10 Formal approaches most frequently involved the establishment of Industrial Advisory Panels or similar groups either at institutional level (mentioned in four of the 59 reports) or more locally (noted in 22 reports). Locally organised panels might be located in all discipline areas or might be restricted to certain subjects, but were generally to be found where courses were strongly vocational in nature. Such panels might advise on curricular content, employability skills or the marketability of course proposals. In this context, one audit report identified good practice in respect of 'the active engagement of Industrial Advisory Panels in quality management and course development' [Cranfield University, paragraphs 47, 55, 76, 125 and 137]. In this instance, the report noted that panels met students and staff independently and that the panels 'can make a significant contribution...to ensuring the relevance of courses to industrial needs' [Cranfield University, paragraph 76]. In an example of good practice in another institution, employer advisory boards had been replaced by Employer Liaison Fellows who acted as champions within their departments for contact with employers [University of Luton, paragraph 92]. Recommendations to institutions related to this matter included that consideration be given to establishing a single forum to provide a focus for discussion with employers, to underpin commitment to providing high rates of employment for graduates.

11 Contact with PSRBs regarding accredited professional courses or employers' involvement in awards, including Foundation Degrees and awards tailored to specific clients, also provided a valuable source of employer feedback, although some audit reports expressed reservations about the level of employer contributions to Foundation Degrees. In some institutions, attendance at meetings of employers or other voluntary organisations provided another route for obtaining information, while in a few institutions employers were members of university committees in that capacity. Surveys to establish employers' views, or meetings specifically to seek feedback from employers, were noted in a number of reports. Meetings could be at the most senior level in the institution, focused for example on its regional agenda, or more locally with individual departments seeking links with particular employer groups.

12 Acquiring feedback from small and medium-sized enterprises had proved challenging for a number of institutions. The audit reports cited examples where invitations to join in discussions had been aimed specifically at such employers. They also described instances where effective relationships had been developed between small and medium-sized enterprises and institutions, in particular through institutions' careers services.

13 Less formal approaches to seeking feedback from employers included, in particular, interaction between institutional staff and placement or work experience providers. This occurred through support for volunteering, NHS contract monitoring meetings, contact with alumni, and information provided by part-time staff who were also employers. However, audit teams were not always able to find specific evidence to support claims for the effectiveness of informal feedback mechanisms made by institutions in their self-evaluation documents. In several institutions, careers services formed or were being encouraged to form a central focus for employer liaison, and central committees concerned with careers might have employer members.

14 A number of audit reports noted that in their self-evaluation documents, institutions had identified difficulties with procedures for obtaining and evaluating feedback from employers, or commented to the effect that they had been seeking ways to formalise and extend such engagements. In this connection, recommendations in several audit reports were concerned with establishing systematic, consistent policies and procedures for liaison with employers, and for gathering and evaluating feedback from students, graduates and employers. The approach adopted by one institution was identified as good practice. This was to engage with employers, including voluntary agencies, through the institution's regional agenda. As evidence of the success of this approach, the report cited features of student recruitment and links with local employers, agencies and practitioners [University of Derby, paragraphs 114 and 118].

Involvement of employers and PSRBs in approval, monitoring and review

15 A second significant theme was the involvement of employers and PSRBs in the processes of programme approval, annual monitoring and review. Some institutions required external feedback to be sought from these bodies in the early stages of approval and review, either by making use of existing employer liaison groups or by approaching relevant employers and PSRBs. Others simply encouraged such contacts to be made. One audit report noted the routine involvement of panel members from the professions and employers in the validation of new provision [Harper Adams University College, paragraph 61].

16 At the approval stage, and in periodic reviews of existing programmes, the inclusion of an appropriate employer or PSRB representative as an external member of, or adviser to, the approval committee might be allowed (nine audit reports), encouraged (three reports) or required (five reports). Sometimes, though, such expectations applied only in the case of explicitly vocational programmes. In this connection, one institution had established the practice of including a lay member of council to represent the perspective of an employer or parent on its review panels. Employers might also be invited to meet panel members or be involved in other ways rather than on the panel itself (11 audit reports). In other cases, written comments to inform the review might be sought from a major employer or PSRB (two reports), and feedback from employers might form part of the documentation required by a review or approval panel (five reports). There were also examples of institutions seeking to combine approval and/or periodic review with accreditation by the relevant PSRB, to reduce the burden for staff and the institution (eight reports). One audit report, however, expressed concern about instances when the requirements of the PSRB were allowed to override or replace the academic review process itself.

17 Reports from accreditation visits by PSRBs, together with any action plans resulting from such visits, featured in the annual monitoring processes of many institutions, thereby providing them with the means to monitor their overall activities with PSRBs (see also paragraph 30 below). In several cases, audit reports cited examples of employers' views as set out in annual monitoring reports. Some reports, however, recommended that this aspect of institutional arrangements could be further strengthened, for example by including an identifiable section on employer feedback in the annual report template.

18 One institution's routine use of a broad range of external peers in its periodic review of undergraduate programmes, including industrialists and academic staff from institutions in Europe, was noted as an example of good practice [Imperial College of Science, Technology and Medicine, paragraph 67]. In particular, the involvement of industrial assessors was seen as assisting in maintaining the relevance of programmes to the needs of employers. Another institution's 'strong engagement...with industry, as evidenced through the involvement of employers in...review' was likewise seen as a feature of good practice [Birmingham College of Food, Tourism and Creative Studies, paragraphs 70-1 and 101-02].

19 In terms of recommendations in this area, one audit report suggested that engagement with the employers of an institution's students should be made more visibly a part of quality management arrangements. Other reports expressed concern about a lack of evidence that employers had been involved in the approval of Foundation Degrees. One report endorsed the institution's own view, expressed through its self-evaluation document, of the need to improve the participation of employers and PSRBs in review. Another suggested that the relevant institution considered strengthening the contribution of feedback from employers to annual monitoring, in order 'to enhance its contribution to subject development'.

Involvement of employers and PSRBs in curriculum design

20 Employers were often involved in programme design, the most common mechanism being the use of feedback from industrial liaison panels to develop the curriculum, making what was described in one report as a 'rich contribution' to such development. They were also included as members of programme advisory or development committees. Employers might provide formal input to programme design, or staff who supported the development of provision might be encouraged, advised or expected to make contact with relevant employers through a variety of mechanisms, including reference to providers of student placements. Several reports stated that feedback from recent graduates also played a part in institutions' programme development procedures.

21 PSRB requirements were also frequently mentioned as being key to programme design (10 reports), and there were examples of programme specifications requiring reference to relevant PSRB benchmark statements. The Foundation Degree qualification benchmark states that 'Foundation Degrees integrate academic and work-based learning through close collaboration between employers and programme providers'; the audit reports generally confirmed such collaboration. However, a lack of employer involvement was noted in a few cases.

22 As noted in paragraph 18, one audit report found good practice in 'the strong engagement of [the institution] with industry, as evidenced through the involvement of employers in curriculum design...' [Birmingham College of Food, Tourism and Creative Studies, paragraphs 70-1 and 101-2]. In another institution, good practice was found 'in the effective integration of professional expertise into all appropriate aspects of the curriculum' [University of the West of England, Bristol, paragraph 94]. However, some audit reports noted limited evidence of input into course development, or suggested that the employer contribution to subject development be promoted, maximised and rendered more visible.

Changes to regulations and processes

23 While a number of institutions sought to impose a single regulatory framework on their taught provision, some audit reports noted that exceptions to this might be made to meet the requirements of PSRBs. The relevant reports explained how such exceptions were formally agreed: for example, in one instance by an institution-level group reporting to the deliberative committee structure, in another case to one of the major institutional quality committees, and in a third institution at faculty level. One report noted that the institution maintained a register of approved exceptions to its regulations.

24 The need to meet PSRB requirements might also lead to changes in process, for example in timescales for programme approval or the maximum period before revalidation was required. One report noted that the institution's annual monitoring process had been extended in one area of provision to incorporate evidence required annually by the relevant professional body.

PSRB involvement with quality assurance and standards

25 Several audit reports included extracts from the relevant institutions' self-evaluation documents, which referred to the involvement and requirements of PSRBs in certain of their programmes when explaining how they determined their standards and assured quality. In this connection, some audit reports discussed the relationship between professional standards and academic standards, and in several instances they indicated points of tension. For example, one report queried the appropriateness of adopting the position 'that as a professional vocational qualification the [PSRB's] provision falls outside the FHEQ [Framework for Higher Education Qualifications]'. It suggested that since the programme in question led to a qualification of the institution, it needed to be located within the FHEQ. In other instances, however, institutions had been negotiating the harmonisation of professional and academic standards more successfully. In several cases this was through the appointment of external examiners with professional qualifications and experience [University of the West of England, Bristol, paragraph 94].

26 Several reports noted the view of institutions that successful accreditation by PSRBs provided an indicator that they were maintaining appropriate standards and comparability with programmes in other institutions. One report noted that 'professional advisers' and employers' representatives sat on award boards of the institution. These professional advisers were chosen for their 'industrial qualifications or experience appropriate to the programmes and award outcomes and appropriate professional standing, expertise and experience to offer advice in support of maintaining standards in higher education'. Together with the external examiner, professional advisers were said to 'act as external reference points on quality, standards and student achievement', and were expected to 'have particular responsibilities with respect to student achievement of professional skills' [Harper Adams University College, paragraph 36].

Employers' involvement in delivery and assessment

27 In vocationally oriented programmes, practitioners were frequently involved in delivery (noted in 14 audit reports). They might have roles as visiting lecturers making one or two presentations, or as a more integrated part of the teaching team (for example via joint appointments of teacher-practitioners with NHS Trusts), or as staff who continued to practise their profession while employed by the institution. Employers might also set project briefs. Such involvement was particularly noted in connection with Foundation Degrees and with provision in the area of health care, where close engagement with employers was expected (see above). In one report, a lack of arrangements to ensure that employers were engaged in the delivery and assessment of Foundation Degrees gave rise to critical comment.

28 A smaller number of reports noted the involvement of employers and practitioners in assessment. There was also more limited reference to the appointment of external examiners with professional rather than purely academic experience. The good practice of one institution, which was manifested in its 'strong...engagement' with industry, 'as evidenced through the involvement of employers in curriculum... delivery' has already been noted (see paragraph 18).

Provision of public information

29 Eleven audit reports commented on the information that institutions published in their programme specifications about links with employers and accreditation by PSRBs, or exemptions from the requirements of PSRBs. Some reports noted the efforts being made by institutions to make such documents more accessible to employers. However, not all programme specifications were found to include information relating to benchmarks published by PSRBs. In 2007, of the 59 published reports 19 noted the existence on the HERO Teaching Quality Information website of detail about how institutions measured and responded to employers' needs and trends.

Institutions' oversight of the findings of PSRB reports on their provision

30 Institutions took a range of approaches to PSRB accreditation visits, and many described in their self-evaluation documents how the results of such visits were reported at institutional level to their Academic Board or quality committee. This was to demonstrate how they oversaw cross-institutional matters and provided for the identification and spread of good practice and the approval and monitoring of any action plans resulting from a PSRB visit. Alternatively, central oversight might be maintained by a senior member of the institution such as a pro-vice chancellor or, less frequently, the academic registrar. Several audit reports noted instances where central review of PSRB activity was planned or being actively considered or extended. In such cases, the reports welcomed such projected developments.

31 One audit report noted the institution's receipt of PSRB reports at institutional level, and their consideration at that level, as an instance of good practice. In this institution, reports received from PSRBs (and the associated faculty-level action plans) were a regular item on the agenda of one of the institution-level committees. An annual report was produced covering all such activity, highlighting cross-institutional themes [Sheffield Hallam University, paragraph 63].

32 In a number of institutions, contact and discussion of outcomes from PSRB engagement appeared to remain at a more local level, whether that was departmental, school, faculty or even course level (16 audit reports). In some of these cases there was provision for upward reporting through the institution's annual monitoring system. One institution monitored reports from PSRBs via an institution-level steering group, with reporting by exception to the academic board.

33 Preparation for visits by PSRB panels might be supported centrally by an institution, or more locally by faculties or departments. Such support might include a protocol to inform relationships with the professional body. The level of intervention might depend on the outcome of risk assessments. In some cases, a register or schedule of accreditation visits was held centrally. Several audit reports contained recommendations to institutions to consider improving monitoring at institutional level of reports on their provision from PSRBs.

Conclusions

34 Taken together, the information from the 59 institutional audit reports published between December 2004 and August 2006 demonstrates a range of ways in which institutions work with employers. In particular, they contain many references to the various mechanisms used for obtaining feedback from employers, and to the use of employers' personnel and the views of employers in curriculum design and delivery and in programme approval, monitoring and review. Information about employer links is widely available on the HERO Teaching Quality Information website (www.tqi.ac.uk/sites/tqi/home/index.cfm). However, a number of institutions recognise the need for more formal, systematic and consistent approaches to the liaison they undertake with employers.

35 The audit reports further demonstrate that institutions engage constructively with PSRBs relevant to their provision. Increasingly, this engagement is not purely at subject level, but involves an institution more generally. Processes are in place to maximise the benefit of such engagement and encourage the spread of good practice between subject areas. However, in a number of cases further development is needed to ensure appropriate institutional oversight of PSRB reports.

36 The corresponding Outcomes... paper in the first series identified themes concerning:

• registering the needs and requirements of PSRBs and/or employers in programme development, approval and review
• institutional oversight of the receipt of reports from PSRBs
• members of staff who represent the views of PSRBs and/or employers to their institutions
• supporting work-based learning
• feedback to institutions from employers
• the provision of public information about institutions' links with employers and PSRBs.

37 From the present paper, it can be seen that there is considerable similarity in the issues identified by audit teams. However, it is observable in the second set of reports that institutions are placing less reliance on members of staff to represent the views of employers and PSRBs than appeared to be the case in the material reviewed for the corresponding paper in the first series of Outcomes.... It also appears to be the case that institutions are now giving greater attention to employers' contribution to curriculum delivery and assessment, and to PSRB accreditation as a measure of the maintenance of standards.

Appendix >>