Foreword
Academic review of subjects in UK higher education
The Quality Assurance Agency for Higher Education (the Agency) carries out reviews of individual subjects through service level agreements with the main higher education (HE) funding bodies. The reviews help to provide: public assurance that the quality and standards of HE are being safeguarded; enhancement of the provision; and accountability for public funds.
In developing its method for academic review, the Agency has published a wide range of materials designed to provide a background against which the reviews can take place. These are:
- The framework for qualifications of higher education institutions in Scotland;
- The framework for higher education qualifications in England, Wales and Northern Ireland (FHEQ);
- Subject benchmark statements;
- Code of practice for the assurance of academic quality and standards in higher education (Code of practice);
- Guidelines for preparing programme specifications.
Together, the FHEQ, the Subject benchmark statements and the Code of practice are referred to as the academic infrastructure.
This report covers reviews undertaken in general further education colleges (FECs) in England between January 2002 and June 2003 through an agreement with the Higher Education Funding Council for England (HEFCE). Only provision which is directly funded by HEFCE has been reviewed.
Academic review of subjects
Academic review of subjects is a peer review process. It starts when institutions evaluate their provision in a subject in a self-evaluation document. This document is submitted to the Agency for use by a team of reviewers who gather evidence to enable them to report their judgements on the academic standards and the quality of learning opportunities. Review activities include meeting staff and students, scrutinising students' assessed work, reading relevant documents and considering learning resources.
Full details of the process of subject review can be found in the Handbook for academic review, QAA, 2000.
Judgements
The range of judgements that the reviewers may utilise when they have completed a subject review is summarised below. In joint reviews, each subject has a separate judgement in academic standards but judgements in the quality of learning opportunities are generally combined.
Academic standards
Reviewers make one of the following judgements
- confidence, which may be expressed as limited confidence; or
- no confidence
To reach this judgement, the reviewers look at: on standards:
- learning outcomes;
- the curriculum;
- student assessment; and
- student achievement.
Quality of learning opportunities
Reviewers make one of the following judgements for each of three aspects of learning opportunities:
- commendable, (which may include exemplary features); or
- approved; or
- failing
The three aspects of quality of learning opportunities are:
- teaching and learning;
- student progression; and
- learning resources.
Maintenance and enhancement of quality and standards
Reviewers also report the degree of confidence they have in the institution's ability to maintain and enhance quality and standards in the subject under review.
Summary
1. There is much to celebrate in the achievements of FECs as they continue to develop their HE provision. There is also much good practice of HE in FECs from which all current and potential providers can benefit.
2. This report is based on the findings of 153 academic reviews carried out between January 2002 and June 2003 in FECs in England. The majority of reviews were in one or more of three subjects: engineering, computing or social policy and administration and social work. Overall, the reviewers had confidence in the academic standards of over 90 per cent of the programmes reviewed. In the quality of learning opportunities, the strongest areas were teaching and learning and student progression. Around 60 per cent of the reviews resulted in commendable judgements in these aspects.
3. These judgements show that students generally experience education of high quality and of the appropriate standard. Typically this is provided by dedicated and enthusiastic staff and supported by appropriate learning resources. College staff usually build on their links with employers to support their HE provision, promoting relevance to employment and currency of the curriculum.
4. The colleges make an important and growing contribution to the achievement of the Government's policy for increasing participation in HE. The colleges are effective in creating flexible educational opportunities for a range of students who have not traditionally been able to benefit from HE. The programmes are successful in attracting students from widely differing backgrounds, with diverse qualifications, a broad age profile and varied educational and work experience.
5. A crucial element in the success of the colleges' HE provision is the quality of student support. In most cases, student groups are small, providing very good opportunities for close working relationships between staff and students. Staff know their students well and play a key part in helping them through the challenges of balancing learning with work and domestic demands. Staff are generally well qualified in their subjects and in the practice of teaching.
6. Almost all providers successfully match their aims with the intended learning outcomes (ILOs) at the programme level. The majority of programmes have up-to-date and relevant curricula. In general, student achievement matches the ILOs and level of the award. The assessment process, together with the way it is secured by quality management, is the element with most scope for further development.
7. Colleges that have devised a specific teaching and learning strategy for HE, provide a supportive framework for their staff and students. Reviewers found the best resourced HE programmes in colleges where the needs of HE students have been fully and strategically addressed. A number of colleges have recently refurbished teaching accommodation to meet the growth in HE, particularly in engineering. Colleges that have achieved a Centre of Vocational Excellence (COVE) status for their FE-level provision are able to use their specialist facilities to benefit HE students. Many colleges have a clear information and communications technology (ICT) strategy and possess modern and up-to-date learning resource centres to which students have ready access.
8. Overall, academic review of HE in FECs has identified a wide range of good practice from which all providers can benefit. The reviewers found examples of good practice in almost every college reviewed. HE in FECs can be enhanced if providers increase opportunities to share good practice to assist with improving the maintenance and enhancement of quality and standards.
Introduction: the findings of academic review
9. This report provides a summary of the findings of the academic reviews of subjects completed by the Agency between January 2002 and June 2003. The chief purposes are to highlight elements of good practice identified by the reviewers, to assist current and potential providers in further developing and enhancing their provision and to disseminate good practice to subjects yet to be reviewed. The report will also assist professional, statutory and regulatory bodies (PSRBs) and those undertaking scholarly work in the field.
10. This report is based upon the findings of 153 academic reviews of subjects taught at HE level and funded either directly by HEFCE or through HEFCE consortium funding in FECs in England. Around 5,800 full-time and 11,400 part-time students were registered for the programmes reviewed; a total of about 11,500 full-time equivalents.
11. The Agency carried out these reviews in 107 of the 162 colleges directly funded by HEFCE (at September 2003). This included 75 colleges reviewed in one subject only and 32 reviewed in two or more subjects. The largest number of reviews in any one college during this period was four, and 11 colleges had three reviews. Some 20 of these colleges have substantial experience of providing HE programmes over many years. Most, however, are in the early stages of developing their HE provision and at the time of writing this report, the Agency has yet to carry out reviews in 55 colleges which provide HE in FECs. The Agency reviews HE programmes which are indirectly funded by a higher education institution (HEI) through the process of institutional audit.
12. The 153 reviews included programmes within the following subjects scheduled for review by the Agency in 2002-03 (Handbook for academic review, 2000, annex K):
- Accountancy;
- Computing;
- Earth, environmental sciences and environmental studies;
- Engineering;
- English;
- Geography;
- History;
- Law;
- Social policy and administration and social work; and
- Sociology and anthropology.
13. The full list of subjects reviewed at each FEC is given in Annex 1. In 69 cases the reviews were aggregate; that is, they covered more than one of these subjects. Fifty-eight were aggregate reviews of engineering and computing.
14. About 60 per cent of the programmes reviewed during this period are at HNC/D level, while about 35 per cent are bachelors degree awards and 5 per cent are taught masters degree programmes. HNC/D programmes predominate in engineering and computing, while bachelors degrees are most numerous in arts and humanities and in social studies. The immense richness of the education on offer is indicated by the list of awards in Annex 2.
15. Overall, at the time of the reviews, about 5,800 full-time and 11,400 part-time students were registered for the programmes, in total 11,500 full-time equivalents.
16. The distribution of the reviews across the subjects is shown in Figures 1 and 2.
Academic review HE in FE (January 2002 - July 2003)
Distribution of subjects reviewed
Figure 1
Academic review HE in FE (January 2002 - July 2003)
Number of reviews in each of the subjects reviewed
Figure 2
| Subject | No. | % |
|---|---|---|
| Computing (single and aggregate reviews) | 89 | 42 |
| Engineering (single and aggregate reviews) | 88 | 41 |
| Social Policy (single and aggregate reviews) | 25 | 12 |
| Others (single and aggregate reviews) | 10 | 5 |
| Total reviews | 153 |
|
17. The Agency will carry out academic reviews of other HEFCE directly or consortium funded subjects provided at HE level in FECs between 2003 and 2006. These subjects include:
- Medicine;
- Dentistry;
- Veterinary Medicine;
- Biosciences;
- Subjects Allied to Medicine;
- Nursing and Midwifery;
- Physics and astronomy;
- Chemistry;
- Psychology;
- Agriculture, Forestry, Agricultural and Food Sciences;
- Materials;
- Building and Surveying;
- Town and Country Planning;
- Mathematics, Statistics and Operational Research;
- Linguistics;
- Area Studies;
- Languages and Related Studies;
- Communications, Media, Film and Television Studies;
- Art and Design;
- Dance, Drama and Performance Arts;
- Music.
18. Overall, the reviewers have confidence in the standards of over 90 per cent of the programmes reviewed. In some 92 per cent, the reviewers had confidence in the standard of all the programmes. In around another 3 per cent, they reached differentiated judgements; that is they had confidence in the standards of only some of the programmes they reviewed. Figure 3 provides more detail of the standards judgements.
Academic review HE in FE (January 2002 - July 2003)
Academic standards judgements
Figure 3
19. Within the three aspects of the quality of learning opportunities, 99 per cent of the reviews resulted in commendable or approved judgements; teaching and learning achieved 59 per cent commendable judgements, student progression 58 per cent and learning resources 39 per cent. The reviewers reached differentiated judgements in each of these aspects. Figure 4 provides details of these.
Academic review HE in FE (January 2002 - July 2003)
Quality of learning opportunities judgements
Figure 4
20. Within this report, the identification of good practice is based on the Code of practice. The Code of practice is intended to help institutions by providing a framework within which they can consider the effectiveness of their individual approaches to a range of HE activities. The reviewers have paid close attention to the precepts in the Code of practice and have identified as good practice some of the ways in which colleges have addressed the intentions of the precepts and introduced changes. Pinpointing this good practice is key to enabling colleges to build on their strengths and to identify areas for further development.
Overview of outcomes from academic review
Aims
21. Nearly all FECs offering HNC/Ds in the subjects under review include the five core aims for the subject laid down by the awarding body, BTEC Edexcel. These aims are generic and concern preparation for employment, further study and the acquisition of subject and transferable skills. Few colleges include significant subject-specific aims. Where the college has followed established good practice, the aims are set within the framework of the college mission and are also supplemented with others that underpin the student experience in important ways.
22. The aims of the 35 per cent of the programmes reviewed lead to bachelors-level awards and these have some distinguishing features appropriate to the level of achievement expected. Strengths include distinctive features such as developing enthusiasm for, and competence in, independent research.
23. The provision reviewed at masters level, is confined to two colleges. In both cases, the programmes build on the knowledge and skills students acquire during first degrees, while providing comprehensive and challenging opportunities for them to specialise in line with individual interests and career pathways.
24. Good practice in setting aims for HE programmes includes:
- setting the aims within the framework of the college mission;
- providing aims to embrace features that underpin the student experience;
- setting subject-specific aims as well as generic aims;
- giving indications of how the students' needs for support and guidance will be identified and addressed;
- matching the level of the programme in relation to the level descriptor set out in the FHEQ;
- defining an aim relating to the quality assurance and enhancement of the subject;
- developing a programme of study informed by staff scholarship and professional expertise.
25. For degree level work, appropriate aims also address:
- allowing students to enhance their existing transferable skills, extend and enhance their developing abilities, reason logically and engage in informed debate;
- developing a programme of study informed by staff research;
- developing enthusiasm for and competence in independent research.
26. The most effective statements are those that make links between institutional and subject-related aims and make reference to the academic infrastructure. Figure 5 sets out a compilation of examples of good practice in the statement of the aims of the provision of HE in FECs.
Examples of good practice in the statement of aims for HE in FECs
Figure 5
The overall aims of the provision are to:
- provide a nationally recognised higher education foundation for a range of technical and managerial careers in land based occupations, with specialist studies in relevant pathways with accreditation where relevant by professional bodies (for example, of specialist subject elements);
- in line with the college's mission, develop partnerships with key local organisations; to enable graduates and diplomates to make an immediate contribution to employment; and, in so doing, support sustained economic development in the region (for example, of mission context);
- provide flexibly-structured coherent programmes, consistent with national guidelines outlined in the FHEQ, preparing students for a wide range of careers (for example, of reference to the academic infrastructure);
- encourage students' personal and professional development through providing a learning environment within which students experience positive achievement, develop a range of transferable skills, and are able to achieve high levels of relevant lifelong skills for a successful working life (for example, of generic skills);
- arrange practical placements for students that promote the integration of theory with practice, productive competence at work, and opportunities for later employment (for example, of work-based learning);
- in line with the College's student assessment strategy, which itself uses the Code of practice, Section 6: Assessment of students as a benchmark, provide a rigorous and fair system of assessment while continuing to meet the requirements of Edexcel (for example, of reference to academic infrastructure and student assessment);
- adopt recruitment strategies that encourage widening participation from the local community and from non-traditional educational backgrounds and providing FE to HE progression routes for local people (for example, of student progression);
- through students and staff working closely together, provide a caring and supportive learning environment that pays attention to the circumstances and differing academic and pastoral needs of students (for example, of student support);
- allow students to succeed in their programmes through the provision of learning resources appropriate to an HE level of study (for example, of learning resources);
- continue to operate robust quality assurance and course management procedures through working closely with staff, students and employers; and to systematically review, evaluate and update the provision to ensure its continued quality (for example, quality assurance and enhancement).
Strengths and areas for development: academic standards
Intended learning outcomes
27. Almost all providers successfully match their aims and ILOs at the programme level. The majority of providers have used the appropriate Subject benchmark statement when setting their ILOs. In general, ILOs are effectively communicated to staff and students through programme specifications. They are included in student handbooks and discussed during recruitment and induction, although they are not always clearly understood by students.
28. Learning outcomes of degree programmes are clearly differentiated and reflect the academic infrastructure for HE. This consists of FHEQ; the Code of practice, Subject benchmark statements and programme specifications. On law and social work programmes these also comply with PSRB requirements and, for engineering programmes, are closely linked to the needs of local employers. The use of ILOs for social work programmes is variable, with some inconsistencies in describing outcomes, consultation with key stakeholders and some use of outcomes at an inappropriate level.
29. BTEC Edexcel sets guidelines for ILOs at the unit level for HNC/D programmes. The majority of providers have understood the benefit of generating the required programme-level subject outcomes to underpin the programme. Although the HND is intended to widen the breadth of study from the HNC, not all providers make this clear. HNC/D provision which aims to enhance progression to a degree, would generally benefit from more explicit ILOs at both unit and programme level.
30. In computing and engineering, local employers are increasingly involved in the specification of ILOs and this increases the currency and relevance of the provision. In both accounting and computing, the matching of ILOs to PSRB requirements is a valuable feature which enables students to study for additional qualifications alongside the HNC/D, or degree, and so gain simultaneous accreditation.
31. ILOs concerning the acquisition of common and transferable skills are an important feature of all programmes. Common and transferable skills play an essential role in developing students' personal effectiveness in working life and provide a foundation for personal development. They include, managing roles and responsibilities; undertaking personal and career development; treating others' values, beliefs and opinions with respect; working effectively as a team member and communicating effectively in writing.
32. The way in which ILOs are communicated to students is important for giving their learning a clear focus. Key features in the specification of ILOs include:
- matching ILOs to the aims of the provision;
- reflecting on the relevant Subject benchmark statements at an appropriate level;
- differentiating the level of the award, in line with the FHEQ
- differentiating subject-specific learning outcomes from generic learning outcomes;
- matching with PSRB requirements in applied subjects;
- linking to the needs of local employers, where appropriate;
- recording clearly in the programme specifications;
- communicating effectively to staff and students through programme and student handbooks with clear links between programme and module/unit aims and ILOs;
- discussing with students during recruitment and induction.
Curricula
33. Curricula are generally designed to meet the requirements of external bodies, including BTEC Edexcel, validating universities and PSRBs. The majority of programmes reviewed have up-to-date and relevant curricula. At their best, curricula explicitly related to the achievement of ILOs and relevant Subject benchmark statements. However, there is scope for colleges to place further emphasis on the achievement of the higher order skills of analysis and criticism as set out in the FHEQ.
34. In many colleges, choice within the curriculum promotes student interest and specialisms. Reviewers praise many programmes for their curricula, which are comprehensive and well integrated. In a minority of cases, student choice within the curriculum is limited. This is usually because low student enrolments make it uneconomical for colleges to offer more options.
35. Curriculum development, effectively linked to teaching, learning and assessment strategies and the availability of the essential learning resources, contributes significantly to the achievement of the ILOs. Where colleges have successfully integrated these elements, student achievement is usually commendable.
36. The content of the curriculum of most programmes is influenced by local circumstances and is relevant to employers' needs. Generally, programmes prepare students well for future employment. A key strength of most programmes is the opportunity for students to combine study for an HE qualification with relevant employment, particularly through part-time HNCs, and work-based projects and placements.
37. The development of vocationally relevant skills, which can lead directly into employment, is emphasised in many reviews and is particularly evident in HNC engineering programmes. Achievement is enhanced through the involvement of students in work-related projects that utilise the resources of local employers and allow students to study real work topics. Achievement levels are particularly high on some centre-devised units which attract students from a wide area, and which are well supported by the staff. Part-time engineering students also often have enhanced career progression prospects because of the close links between colleges and local employers.
38. Linking programmes with additional qualifications assists student achievement and progression. For example, students on some computing programmes are able to take sector-specific qualifications in association with HNC/Ds and achieve a dual qualification within the same timescale. This facilitates entry to the job market for full-time students and career development opportunities for those in employment. Employers particularly value the additional development of skills such as programming.
39. The curricula that are generally most effective in providing good opportunities for students to achieve the ILOs are those that:
- meet both the needs of students and employers;
- have been developed by programme staff;
- have strong formal and informal links between college staff and employers, for example, employers' forum;
- plan, deliver and assess students' achievement of common and/or transferable skills;
- have some centre-devised units for HNC/Ds;
- give advanced standing for additional specialist or generic qualifications;
- provide opportunities for students to interrelate material from different parts of their studies, for example, theoretical and practical studies and/or work-based learning;
- where key programme staff are using their own scholarship and expertise to inform the structure, content and delivery of individual modules and the programme as a whole;
- where currency is maintained and enhanced through staff development activities, such as work shadowing, and contact with external agencies and employers.
Assessment
40. The assessment process, together with the way it is secured by quality management, is an area with potential for further development of HE in FECs. This was also the picture that emerged from subject reviews in both HE institutions and FECs, carried out between 1993-2001. Areas for development that reviewers frequently identified are provided in Figure 6.
Areas of student assessment with scope for development
Figure 6
| Problem identified | Likely result |
| No assessment strategy. | Unreliable assessment practice. |
| Assessment is not designed to test students' achievement of ILOs | Difficult for the FEC to be sure students have achieved appropriate standard. |
| Uneven assessment loads for students. | Difficult for students to plan their work; overload for students and staff at certain times, typically at the end of a unit or module. |
| Unplanned changes in the assessment schedule. | Difficult for students to plan their work. |
| Insufficient rigour and consistency in the assessment of common skills. | Common skills may not be addressed effectively. |
| Lack of clarity in grading criteria. | Difficult for students to know what is expected of them and for staff to mark fairly. |
| Inconsistent or ineffective internal verification. | Assessments set may not be well matched to ILOs, may not be expressed clearly and unambiguously, marking criteria may be applied inconsistently. |
| Marking criteria not applied consistently. | Marking maybe inconsistent and/or inappropriate to the standard of the award; students may perceive assessment process as unfair. |
| Variable quality of feedback to students on assessed work. | Students may not receive sufficient guidance to help them to improve their work. |
| Unclear criteria for student progression from stage to stage and for successful completion of the award. | Students unclear about standard to achieve in order to progress, examiners may not have sound basis for making decisions about student progression. |
| Method for the award of final grades is unclear and lacks rigour. | Final grades may be more subjective than objective. |
41. Good practice in assessment is based on a strategy that ensures a close link between ILOs, assessment methods used and assessments set. This promotes a shared understanding of the rationale supporting assessment and allows students to demonstrate achievement of the ILOs. In most subjects this is a strength. However, not all assessment is securely linked to the ILOs. This results in uneven assessment loads for students and lack of clarity in grading, as well as variable standards of formative feedback. When assessment is not securely linked to the ILOs, this results in either over or under-assessment and reviewers found examples of this in all subjects reviewed, and especially in computing. This difficulty was usually also associated with the lack of an assessment strategy or with ineffective internal verification processes. Internal verification, used in HNC/D programmes and in some degree programmes, is largely effective in identifying problems if internal verification occurs before assessments are set.
42. HE student groups in FECs are often small and this facilitates the timely return of marked and graded work. This is much appreciated by students, as is the comprehensive nature of written feedback where it occurs. Tutors almost always supplement written feedback with oral feedback.
43. The effective use of formative assessment is demonstrated across all subjects. It is particularly effective in many engineering and computing programmes. In these, the use of project modules promotes independent learning. Projects undertaken by HNC/D students are generally focused on work-based assignments.
44. Staff generally mark work with care, although a common lack of clarity in grading criteria inhibits consistency across all subjects. Moderation, such as double-marking, is largely effective in ensuring security of grading. The use of double and anonymous marking is commonplace, particularly in social policy and administration and social work and in computing, and helps to ensure fairness. However, the method for the award of final grades in some programmes is unclear and lacks rigour.
45. In computing, one college has adopted the good practice of assignment self-evaluation, allowing students to reflect on their achievements and to identify their own strengths and weaknesses.
46. The effectiveness of the assessment of common skills varies across the provision. The most effective practice was found in computing, where about 25 per cent of providers have successfully integrated the development and assessment of HNC/D students' common skills into their specialist studies.
47. Good practice in assessment invariably involves:
- a carefully planned and coherent strategy, with due reference to the Code of practice, Section 6: Assessment of students; Section 4: External examining; and where appropriate, Section 9: Placement learning;
- appropriate assessment instruments linked to ILOs;
- clarity of assessment and grading criteria;
- transparent, consistent, careful and fair marking and grading practice;
- appropriate moderation and rigorous internal verification procedures;
- consistent written feedback that helps students understand the grade awarded and promotes improvement in future work.
48. Good practice in securing robust application and award of final grades is achieved in HNC/D programmes by:
- establishing formal arrangements for determining final grades, and keeping a formal record of decisions;
- regulations specific to HE programmes that, when implemented at programme level, ensure that student achievement is effectively measured;
- staff development activities focusing on establishing and maintaining a high level of consistency in the grading of work;
- effective internal verification.
Student achievement
49. Most students achieve the ILOs and demonstrate this in their assessed work which matches the level of the award. Reviewers noted students' strong achievement in project work. Achievement levels in all programmes reflect the wide range of entrants' ability and prior experience. 'Value added' is a notable feature and is frequently endorsed by external verifiers and examiners. This is especially true of computing and law programmes where students have improved self-confidence and a strong commitment to continue their studies.
50. Students' achievement of the intended award varies between subjects and institutions. In social policy and administration and social work nearly 100 per cent of students who complete their programmes gain the award. Success and achievement levels for part-time sponsored engineering students are also high. The reviewers commend the students' high levels of achievement in about half the reviews of computing.
51. There are significant variations in the proportion of Pass, Merit and Distinction grades awarded in HNC/D programmes with similar intakes and ILOs. In some programmes, this is because of unclear grading criteria, and a lack of consistency in the method for the award of final grades. In general, students' achievement of common skills is a successful feature. Students progressing to degree studies and those on postgraduate courses would benefit from increased emphasis on higher order skills such as critical analysis.
52. Because a high proportion of students are already in employment, colleges do not always have a good set of first-destination statistics and verifiable information on career progression. However, informal feedback from students who have recently completed, confirms that part-time students generally benefit from their education and secure promotion at work.
53. Student achievement is generally commendable where colleges have:
- successfully linked curriculum development, teaching and learning strategies and the availability of the essential learning resources across to the achievement of the ILOs;
- enabled students to develop relevant skills that can lead directly to employment, particularly for vocational programmes;
- involved students of vocational programmes in work-related projects that utilise the resources of local employers and allow students to study real work topics;
- devised units or modules that attract students from a wide area, and are well supported by the staff;
- enhanced the career progression of part-time students through close links between colleges and some local employers and contribute to the employer's staff development;
- linked programmes with additional qualifications to increase students' employability.
Top