Summary
The 59 institutional audit reports published between December 2004 and August 2006 pay close attention to topics related to the assessment of students. Features of good practice in assessment are cited in 15 of the 59 reports. They identify strengths in the provision of information and guidance to staff and students, the development of assessment practices which assist students in their learning, the use of information technology in assessment, and the implementation of assessment strategies which support student retention. There are recommendations linked to aspects of assessment practice in almost 90 per cent of the 59 reports.
The audit reports indicate that students' views on the clarity and sufficiency of the information provided to them about assessment were largely favourable. In many cases, students were provided with clear assessment criteria and could therefore have a better understanding of what was required of their work. Some reports recommend the use of subject-specific assessment criteria with links to identified learning outcomes. Others call for the use of generic assessment criteria to ensure greater transparency in maintaining consistent institutional academic standards.
Almost all of the audit reports contain several references to the quality and timeliness of feedback, often drawing specific attention to its importance in supporting student learning. There is evidence of much good practice in the provision of feedback, but also much variation within and among institutions. Some poor practice is also reported. Institutional guidelines setting out minimum expectations can be helpful in avoiding weaknesses that disadvantage students, but the reports indicate that there are difficulties in ensuring adherence to such guidelines.
The operation of assessment boards is a significant determinant in maintaining academic standards and, in general, these boards appear to be operating so as to enable them to discharge their responsibilities in a proper manner. Several audit reports make reference, however, to respects in which academic standards might be placed at risk as a consequence of weaknesses in the practices of assessment boards. The use of discretion by boards is discussed in several reports, and concern is expressed when guidelines for exercising discretion are absent. In a few cases, there are concerns that academic standards can be weakened without the consistent exercise of discretion, and that students can be treated inequitably.
Variability in the methods used to determine the classification of honours degrees for undergraduate students is a prominent theme in these reports. In several cases, the reports note that institutions have acted to reduce variability. It seems likely that with the further development of institutional assessment policies this trend will continue.
A comparison of assessment arrangements in the audit reports published after December 2004 with those analysed in the reports published from 2003 to November 2004 shows improvements in the consistency of institutions' assessment practices. In many cases this has arisen from attention to the relevant section of the Quality Assurance Agency for Higher Education's (QAA's) Code of practice for the assurance of academic quality and standards in higher education (Code of practice), and to the Framework for higher education qualifications in England, Wales and Northern Ireland (FHEQ). Many of the recommendations in the reports refer to inconsistencies within institutions in their assessment arrangements. Against this, however, there is evidence of growing use by institutions of guidelines or regulations to control variability and/or set minimum standards, though some institutions are reported to have had difficulty in ensuring that their guidelines and/or regulations are implemented.
Preface
An objective of institutional audit is 'to contribute, in conjunction with other mechanisms, to the promotion and enhancement of high quality in teaching and learning'. To provide institutions and other stakeholders with access to timely information on the findings of its institutional audits, QAA produces short working papers that describe features of good practice and summarise recommendations from the audit reports. Since 2005 these have been published under the generic title Outcomes from institutional audit (hereafter, Outcomes...). The first series of these papers drew on the findings of audit reports published between 2003 and November 2004. This paper is based on the findings of institutional audit reports published between December 2004 and August 2006.
A feature of good practice in institutional audit is considered to be a process, a practice, or a way of handling matters which, in the context of the particular institution, is improving, or leading to the improvement of, the management of quality and/or academic standards, and learning and teaching. Outcomes... papers are intended to provide readers with pointers to where features of good practice relating to particular topics can be located in the published audit reports. Each Outcomes... paper therefore identifies the features of good practice in individual reports associated with the particular topic and their location in the Main report. Although all features of good practice are listed, in the interests of brevity not all are discussed in this paper. In the initial listing in paragraph 3, the first reference is to the numbered or bulleted lists of features of good practice at the end of each institutional audit report, the second to the relevant paragraphs in Section 2 of the Main report. Throughout the body of this paper, references to features of good practice in the institutional audit reports give the institution's name and the paragraph number from Section 2 of the Main report.
It should be emphasised that the features of good practice mentioned in this paper should be considered in their proper institutional context, and that each is perhaps best viewed as a stimulus to reflection and further development rather than as a model for emulation. A note on the topics identified for this second series of Outcomes... papers can be found at Appendix 3 (page 22).
As noted above, this second series of Outcomes... papers is based on the 59 institutional audit reports published by August 2006, and the titles of papers are in most cases the same as their counterparts in the first series of Outcomes.... Like the first series of Outcomes... papers, those in the second series are perhaps best seen as 'work in progress'. Although QAA retains copyright in the contents of the Outcomes... papers, they can be freely downloaded from the QAA website and cited, with acknowledgement for educational and research purposes.
Introduction and general overview
1 This paper is based on a review of the outcomes of the 59 institutional audit reports published between December 2004 and August 2006 (see Appendix 1, page 18). A note on the methodology used to produce this and other papers in this second Outcomes... series can be found at Appendix 4 (page 24).
2 Although the audit reports had no specific section in which comments on the principles and practices used by institutions to assess students could always be found, almost all of them referred to matters relating to assessment. In many cases, these references were located in the section of the report describing and evaluating the framework used by the institution to manage academic quality and standards. But references to assessment principles and practices were also included in the sections dealing with the use of external examiners and their reports, the use of progression and completion statistics, academic support and guidance, and discipline audit trails. Accordingly, some of the material used in this Outcomes... paper is also used in other papers in the series.
Features of good practice
3 Features of good practice in relation to assessment were cited in the Findings sections of 15 of the 59 institutional audit reports published between December 2004 and August 2006. They reflected strengths in the provision of information and guidance to staff and students, the development of assessment practices to assist students in their learning, the use of information technology in assessment, and the implementation of assessment to support student retention. They comprised:
- the design, development and utilisation of the University's managed learning environment StudyNet, which offers outstanding potential for the University's planned move towards a pedagogy of blended learning [University of Hertfordshire, paragraph 204 v; paragraphs 157 to 165]
- the institutional framework for the assurance of quality and standards which defines central control and the devolution of authority and responsibility and includes appropriate checks and balances [City University, paragraph 320 i; paragraphs 29 to 46]
- the ways in which links between formative and summative assessments have been made in some areas [University of Huddersfield, paragraph 317 iv; paragraphs 144, 179, 194 and 240]
- the fully articulated linkage of module-level and award learning outcomes and assessment strategies mapping to external reference points [Staffordshire University, paragraph 251 iii; paragraphs 46, 77 and 84]
- the student retention project, in particular, its multifaceted action lines and use of performance indicators [Buckinghamshire Chilterns University College, paragraph 238 ii; paragraphs 84, 110, 142, 165 and 213]
- innovations in student-centred assessment [University of Northumbria at Newcastle, paragraph 254 i; paragraphs 32 and185]
- the introduction of Assessment Standing Panels...across the institution [University of Gloucestershire, paragraph 316 i; paragraphs 51 and 210]
- the effective use of automated testing and [the Continuous Assessment Tracking Engine - CATE] to provide rapid feedback to students in the Department of Computing [Imperial College of Science, Technology and Medicine, paragraph 302 v; paragraph 222]
- the work of the Assessment Regimes Working Party in the development and documentation of assessment practice [Henley Management College, paragraph 213 i; paragraph 33]
- the provisions for constructive feedback to students on assessed work [Henley Management College, paragraph 213 vii; paragraphs 122 and 156]
- the careful use of assignment briefs, moderation and marking criteria to assist in measuring the attainment of standards and providing effective feedback to students [Harper Adams University College, paragraph 189 ii; paragraphs 34, 112 and 156]
- the implementation of the University's Assessment Process Handbook [University of Bolton, paragraph 196; paragraph 32]
- the use of post-examination board module reviews to enhance the link between learning outcomes, assessment and student achievement [University College Worcester, paragraph 250 vi; paragraphs 36, 137, 165 and 211]
- the development of templates for the formative assessment of dissertations in the Department of Fashion [American InterContinental University - London, paragraph 217 iii; paragraph 167]
- the composite report of issues raised by external examiners, considered by Academic Board, as an effective vehicle for developing opportunities for enhancement [Heythrop College, paragraph 200 i; paragraph 66].
The Academic Infrastructure
4 Section 6 of QAA's Code of practice, first issued in May 2000, is concerned with the assessment of students. A large majority of the audit reports published between December 2004 and August 2006 referred specifically to this section of the Code of practice, often drawing attention to the way it had been used to inform the development of assessment policies and practices in institutions. Many reports also cited evidence that assessment procedures were consistent with all or most of its precepts. In some cases where progress in implementing policy developments arising from consideration of this Section of the Code of practice had been slow or delayed, reports called for accelerated action. In one instance, the lack of transparent and coherent policies and regulations had led to significant variability and inconsistency in assessment practice. The institution concerned was advised to develop and implement such policies and regulation so that procedures for assessment, progression and award would align with this section of the Code of practice. The reports also contained some comments drawing attention to the need for further consideration of assessment practices so that intended adherence to, or consistency with, this section of the Code of practice could be more clearly demonstrated.
5 In addition, the audit reports commented on the role of the FHEQ in the assessment process. Several reports indicated that it had been taken into account in determining assessment strategies and regulations. In some cases, institutions had developed their own qualifications frameworks as part of an assessment policy, and aligned them with the FHEQ. Other reports suggested that further attention to the implications of the FHEQ would be appropriate. One, for example, drew attention to a 'lack of consistency in the use of appropriate terminology in the setting of assessments' which could be remedied by 'a closer appreciation of the FHEQ'. Elsewhere it was noted that an assessment scheme did not clearly differentiate between the learning outcomes appropriate to distinct FHEQ level descriptors. Another report observed that progression rules permitted students who had not met the requirements of relevant level descriptors to proceed to a higher level of study, and recommended that the institution should 'continue to monitor student performance in this context'.
Themes
6 The principal themes emerging from these audit reports are closely related to the topics covered in QAA's Code of practice, Section 6: Assessment of students. They are:
- the development and use of consistent assessment policies
- the classification of undergraduate degrees
- the operation of assessment boards
- the security and reliability of assessments
- the use of assessment criteria
- the provision of feedback on students' work.
The development and use of consistent assessment policies
7 Following publication of the first edition of QAA's Code of practice, Section 6: Assessment of students in May 2000, almost all institutions took steps to improve the consistency of their assessment arrangements. The audit reports considered in this paper show that many institutions had provided regulations or guidelines for matters such as second-marking or moderation, anonymised examination scripts, the permitted extent of compensation and condonement, and the quality and timeliness of feedback on assessed work. They also show that such guidelines were widely used [City University, paragraphs 38-9; University of Huddersfield, paragraphs 35-6; University of Northumbria at Newcastle, paragraph 32; Henley Management College, paragraph 33; University of Bolton, paragraphs 32-3 and 196]. A few reports identified matters where further consideration was needed, though sometimes it was clear that the institution was intending to take action as a consequence of an internal evaluation or review of the effects of the frameworks it was using.
8 In a small number of cases, audit reports noted the absence of a comprehensive institution-wide policy for the quality assurance of assessment processes, or inconsistencies in implementing assessment requirements, and recommended taking steps to develop policies that would be applied consistently across the institution. In one institution, for example, it was found that procedures for anonymity in assessment were not being followed; the audit report recommended the provision and consistent application of transparent, coherent policies in this and other aspects of assessment. In another institution, the regulations on plagiarism, extenuating circumstances and the assessment of students with disabilities were found to be capable of varying interpretation. The report suggested that as such variations could lead to inconsistency, there would be benefits in developing guidelines to structure and confine the discretion used by examination boards when they considered such matters.
9 The audit reports also show that there could, however, be difficulties in reconciling consistency with the needs of individual academic disciplines. Thus, a number of reports noted that institutions had permitted exceptions to guidelines and rules so that the requirements of relevant validating or accrediting bodies could be met. One report, however, commented on the wider question of how a balance should be struck between 'institution-wide consistency...on the one hand, and the acceptance of some diversity within faculties and schools on the other'. The principles adopted by the institution were that 'consistency is necessary in matters relating to quality and standards, to the [institution's] capacity to discharge its formal obligations and to situations where variation would be unjust or unethical'. 'Elsewhere,' it was reported, 'such diversity as is compatible with the demands of the overall institutional framework is permitted, both to respect discipline traditions and to enhance staff commitment.' In the latter case, the relevant report judged that 'these principles are clear and appropriate, and appear to be generally observed.' Another report which raised the same topic noted 'the effectiveness of the procedures which permit appropriate diversity of disciplinary practice at faculty level, but within a robust overall structure which is sensitive to the external framework of guidance for maintaining academic standards'.
10 In many cases, the audit reports noted that assessment practice within disciplines was in accordance with institutional requirements. However, the scope of the guidance provided by assessment frameworks in different institutions varied widely. Some reports commented on finding evidence in discipline audit trails of continuing inconsistency and differences in practice.
11 In several cases, this was because certain topics had not been addressed at institutional level. For example, one audit report noted considerable variability of understanding and practice relating to locally devised grading criteria and recommended the publication of generic assessment criteria, to justify the institution's expectations of student performance at different academic levels. The institution could thereby provide guidance on framing discipline-specific assessment criteria and strengthen its confidence that standards were being maintained in a consistent manner. Several reports noted the use of a variety of methods for penalising late submission of coursework, and questioned whether this could lead to inequalities and unfairness which might not be justified - especially if the variation were no more than the product of inherited custom and practice. In other cases, inconsistencies had arisen because institutional-level guidelines were not being fully implemented. For example, several reports noted that guidelines concerning the timeliness of feedback to students on their work were not always observed. Moderation of marking was another example of variable implementation of institutional policy.
12 Where audit reports indicated that variation in assessment practice was permitted, in some cases no rationale for the variation had been provided. For example, there might have been variations in the weighting given to marks obtained at different levels in a programme of study without it being clear why weighting was greater for some programmes of study than for others. In one case there was significant variability in the pass mark for individual units of both undergraduate and postgraduate programmes; the institution concerned was advised to review this matter in order to achieve consistency of approach and demonstrate equity of treatment of students.
The classification of undergraduate degrees
13 Of the 59 institutional audit reports published between December 2004 and August 2006, 50 contained explicit references to aspects of degree classification, and 12 included recommendations concerning degree classification. None of the audit reports identified a feature of good practice regarding degree classification. The general impression given is that many institutions had had, and a number continued to have, significant difficulties in resolving matters relating to degree classification.
14 Some institutions had reduced or eliminated differences in the rules they used to classify honours degrees. Several audit reports which noted that differences remained made suggestions or recommendations that the institution should give consideration (or further consideration) to the need for greater consistency. However, as some reports made clear, the use of different methods within an institution did not necessarily lead audit teams to take the view that students were being treated inconsistently or unfairly. This reflected the advice of Section 6 of the Code of practice, which, while drawing attention to the need for consistent operation of assessment rules in general, acknowledges that in the interests of promoting fairness, 'flexibility at subject level may be necessary to reflect different disciplinary needs and marking conventions' [Precept 8]. Accordingly, some reports which noted an institution's use of a variety of rules made no recommendation that consideration should be given to reducing this variety. In these cases, it was accepted that degree classification criteria were academic matters most appropriately dealt with at a local rather than central level. More than one audit report expressed the view that comparability of standards should be seen 'primarily in terms of external intra-subject comparisons, and that internal comparisons across disciplines have less prominence'.
15 However, a number of audit reports saw variation in methods of classifying degrees as problematic. One such noted that, although a previous continuation audit report indicated that consideration should be given to variations in the assessment regulations among schools, it remained the case that 'regulations allow variability in calculating first degree classifications (in terms of weighting between levels, the algorithm for determining degree classification, and in the weighting of placement training outcomes)'.
16 In the case of another audit report, the institution had acted on the advice of a previous report that it should 'review the wide range of methods used for the classification of undergraduate degrees, with a view to promoting greater consistency and equity'. As a consequence, a new system permitting the use of only two different methods had been introduced. In addition, consideration had been given to the method by which joint honours students were classified, 'to ensure that students on joint honours programmes are considered as a cohort in their own right'. However, the report judged that although significant progress had been made, the newly introduced procedures had not been consistently applied and there was therefore a need for further attention.
17 Where institutions had taken steps to secure greater uniformity in the methods they used to classify honours degrees, and had reduced the approved number to two or three, audit reports noted that there could still be difficulties. In one case, for example, although two distinct methods had been sanctioned to take account of different marking practices, some boards of examiners were changing the method they used 'so as to optimise student success'. The same audit report noted that the two sanctioned methods had 'given rise to a number of sub-schemes, which...might go beyond the "slight variations" envisaged'. It concluded that there was 'potential for unfairness to students'. In responding to the audit report, the institution subsequently indicated that it intended to 'review the mechanism for determining the assessment scheme which departments follow for classifying awards, pending any national recommendation on degree classification'.
18 Another audit report noted that the institution was aware that its use of three distinct methods, which were 'intended to reflect the marking traditions of different subjects', raised questions about whether students were being treated fairly. The report noted the 'lack of formal procedures for ensuring equity of treatment of students in the classification of degrees' and the 'limited quality control above school level of recommendations for the award of degrees', and encouraged the institution to continue its careful monitoring in this area.
19 A number of audit reports commented on how the adoption of different methods for classifying honours degrees could create difficulties for students following joint honours programmes. This is illustrated by one report which noted 'concerns raised by external examiners about lack of clarity in approaches to the classification of joint honours degrees'. Despite steps taken to address these concerns, the report nevertheless advised the institution to establish and maintain 'effective monitoring of assessment schemes for joint honours students, with particular reference to consistency in the regulations for borderline classifications, and the coordination of the scheduling of examination boards where joint honours students are considered'.
20 From the audit reports, even institutions which used a single method for classifying honours degrees could have significant variations in matters which had a bearing on degree classification. For example, different departments, faculties or schools could differ in the effect of local rules or conventions on compensation for fail marks in component elements of a degree programme. It was also not uncommon for there to be differences within an institution in the extent to which the weightings of marks obtained varied in different years of study. In some cases, there did not appear to be any justification for such differences.
21 There were several comments in the audit reports on variations in the extent to which boards of examiners used discretion when judging degree classifications. For example, one report noted that discretion 'may include discounting the lowest module mark, provided that the mark indicates that a pass standard has been achieved'. The same report also noted that although boards of examiners were required to provide explicit justifications for the discretion they used when making classification decisions, there was no evidence that the use of discretion was systematically monitored at institutional level. The report concluded that 'potential inconsistencies could arise from the practices of boards of examiners', and recommended the institution 'to further develop the assessment framework with a view to securing greater commonality of judgement and practice...with regard to...degree classifications'.
22 Another audit report noted that there was no institution-wide guidance on the use of discretion, and no monitoring of 'how, and in what circumstances, schools use the discretion allowed to them to set the class of an award'. In this case the institution was advised 'to define more closely for its schools and award boards the criteria they are to follow when deciding how to use the discretion it allows them in its degree classification arrangements'.
23 A number of audit reports commented on the use of statistical data concerning the outcomes of the method(s) used to classify honours degrees. In general, although such data was often made available, there was little evidence that it was analysed effectively to provide assurances that academic standards were comparable within an institution, or with other institutions. One report, for example, recommended that the institution consider 'means at...institutional level for monitoring data regarding...degree classification, and benchmarking these data against internal and external comparators'. Another report found no evidence that the institution was taking 'an...overview of aggregated...award data or [making] comparisons with similar data from comparable institutions'. It, too, concluded by recommending the desirability of reflecting 'on ways of enhancing the use of...achievement data to allow cross-institutional and inter-institutional comparisons of student performance'.
24 Consideration of the 59 audit reports published between December 2004 and August 2006 shows that a number of institutions had introduced, or were seeking to introduce, greater uniformity and consistency in the rules they used for classifying honours degrees. Others, however, appeared to be taking the view that the use of uniform rules would distort judgements and make it more difficult to make comparisons at subject level. As with the 70 audit reports published between 2003 and November 2004, many institutions in these 59 reports appeared to share a general concern that the use of methods to secure fair results across different subjects within an institution might well deliver results that were unfair across different institutions offering a particular subject. There was, however, little robust evidence to show whether such concerns were well founded. In these matters the conclusions suggested by the first 70 reports and the 59 reports published between December 2004 and August 2006 were broadly similar.
25 When assuring the academic standards of their awards, higher education institutions rely on evidence from external examiners together with progression and completion and other data to demonstrate that their method (or methods) for classifying degrees maintains standards, and is fair to students. In general, the audit reports show that such evidence is limited and of poor quality. For all institutions, external examiners' views on the comparability of academic standards are an important source of such evidence. It is difficult to see, however, how external examiners' views might be better informed than those of individual institutions, given that the views of institutions and their external examiners are both likely to be based on partial and limited data. It is difficult not to conclude, therefore, that more could be done to provide institutions, and indeed external examiners, with reliable information on the effects of their assessment practices, particularly insofar as they have a bearing on how honours degrees are classified. Institutional arrangements for gathering, analysing and making use of progression and completion data are considered in a separate Outcomes... paper in this series.
The operation of assessment boards
26 Several audit reports mentioned the manner in which assessment boards operated. Many institutions used a two-tier system of boards, with marks being determined by subject boards, and progression and achievement decisions being made by programme boards or sometimes by institutional-level boards. In general, this system appeared to work in a satisfactory manner. A small number of reports drew attention to the formal position of assessment boards, and noted that they derived their authority from the body with overall responsibility for maintaining academic standards. One report commented that in practice their accountability was indirect.
27 In those cases where an institutional-level board considered the outcomes of assessments, there was an opportunity to address questions about the consistency of academic standards. The audit report on one institution which had decided to introduce such a board, with a remit to receive confirmed marks from subject boards, confirm awards and produce an overview of external examiners' reports, judged that the initiative had the potential to enhance practice. Another report, however, found that the information provided to the institutional-level board did not demonstrate how the board was able to maintain, as claimed, the comparability of the awards it considered.
28 Some audit reports mentioned the operational instructions that assessment boards were expected to follow to safeguard consistency. These specified the constitution and procedures of assessment boards so that the institution could be assured that in such matters as the treatment of non-standard cases and mitigating circumstances students were treated equitably. The means which institutions used to ensure parity of procedure included: mandatory training for chairs of assessment boards; attendance of academic registry representatives at meetings of boards; and the appointment of internal moderators to assessment boards to observe examination discussions. Where evidence showed that the scope of the procedures for boards was insufficient, occasioning frequent recourse to the chair's action to authorise students to progress (whether to an award or to the next stage of their studies), reports advised giving consideration to improvements, particularly in cases where weaknesses were relevant to the maintenance of academic standards.
29 Comments in the audit reports indicated that the accountability of assessment boards could be assisted by clear, detailed records of decisions taken. A few reports commented on the arrangements made by institutions in this respect. Thus, one noted that assessment boards provided detailed records of their decisions and actions, though it was also pointed out that these records were not subject to scrutiny at institutional level.
30 As indicated earlier, the audit reports identified difficulties where institutions provided limited or no guidance on how to employ the discretion granted them to those making decisions on degree classifications. Likewise, a number of reports noted more generally an absence of institutional policy on assessment boards' use of discretion. In some cases, faculties or schools might provide guidance, but the parameters for boards' use of discretion might vary within an institution, thus leading to inconsistent treatment of students - and not just at the point of classification. In such circumstances, one report recommended that an institution should define more closely the criteria for assessment boards' use of discretion, and monitor the effects of the action taken. Another noted that though assessment boards were required to provide an explicit justification for any discretion they exercised, there was no evidence of systematic monitoring of the explanations provided. In the interests not only of consistency but also of maintaining academic standards, it was suggested that the safeguards provided by the monitoring of discretion should be strengthened (see also above, paragraphs 20-22).
The security and reliability of assessments
31 Almost all the institutional audit reports published between December 2004 and August 2006 referred to aspects of the security and reliability of assessments. For the most part, they indicated that requirements concerning this matter were met in a satisfactory manner. Thus, there were a number of favourable comments on arrangements for moderation and double-marking of examination papers and coursework, anonymity of students in the assessment process, plagiarism, and penalties for late submission of summatively assessed work.
32 Most institutions had institution-wide arrangements for moderating the marking of examination papers and coursework, and often used double-marking for important programme elements such as projects and dissertations. In one instance where this did not appear to be the case, the audit report recommended the institution to take steps to ensure that it had an institutional policy on this matter. Another report, for an institution which required its faculties to develop means to undertake the internal moderation of marks, noted that because practical arrangements for meeting the requirement varied, the institution could not be fully confident about the consistency or appropriateness of those arrangements in securing the rigour of assessment practice. It recommended that clear and explicit minimum expectations for internal moderation be provided.
33 There was also increasing recognition that internal examiners needed to provide clear evidence to external examiners that the marking of summative work had been moderated or double-marked. Where this evidence was lacking, audit reports recommended that it be provided.
34 Some audit reports noted weaknesses in implementing institutional policies regarding moderation and/or double-marking. In two instances this appeared to be a consequence of uncertainties about the distinctions between moderation and double-marking. In one case this uncertainty had led to 'changes in the moderated marks of some students within the moderation sample without an overview of the effect of the moderation process on the marks of the whole cohort'. The report recommended that the institution 'assure itself that [its] assessment moderation policy is universally understood and adopted'.
35 It would appear from the audit reports published between December 2004 and August 2006 that for individual students' assessed work (mostly written work), almost all institutions required the student's identity to be concealed from the examiner. A common view was that student anonymity helped to demonstrate the reliability and security of assessments, as well as promoting fair treatment. In those few cases where it appeared that student anonymity was not uniformly used, the reports noted that its wider use was encouraged and drew attention to its potential advantages. One report noted that though there was a clear, directive institutional policy regarding anonymity in assessment, it was not always followed.
36 Several audit reports mentioned safeguards against plagiarism in assessed work. One commented that the provision for students within a department of 'strong, detailed statements...about what constitutes academic malpractice and what penalties would be incurred as a result of it' was of particular note and ensured that students were in no doubt about penalties for plagiarism. Another report commented on a strategy to encourage the use of assessment methods that deterred or prevented plagiarism. Elsewhere, however, there were indications that regulations regarding plagiarism were difficult to interpret, might lead to inconsistent treatment of students, and might not be sufficient to maintain the integrity of student authorship. In one instance, the report recommended that recognised good practice in identifying plagiarism should be further embedded in the institution.
37 The audit reports indicated that several institutions had developed, or were developing, policies regarding extensions of deadlines for the submission of coursework, penalties for submission after the deadline, and extenuating circumstances. However, in some institutions the authority for adopting particular practices in such matters was lacking, or varied from department to department. In several such cases, the audit reports recommended that further steps be taken to ensure institution-wide consistency.
The use of assessment criteria
38 To assist in maintaining academic standards, Precept 7 of Section 6 of the Code of practice (first edition, May 2000) indicates that 'institutions should publish and implement consistently, clear criteria for the marking and grading of assessments'. The audit reports showed that many institutions had responded to this suggestion by developing and publishing assessment criteria. The sections of the reports dealing with discipline audit trails contained many references to the use of assessment criteria, and to their usefulness for students. In the few cases where assessment criteria did not appear to be being used systematically and consistently, the reports encouraged or recommended their further development.
39 A number of reports noted that generic assessment criteria had been, or were being, developed at institutional level. Where this was not the case, and departments or faculties were expected to devise subject-specific assessment criteria, inconsistencies and confusions had sometimes arisen. In general, where useful assessment criteria had not been provided, this was often because of a lack of institutional guidance. One report, for an institution which had introduced generic assessment criteria, called for the development of subject-specific assessment criteria. Another, for an institution which required departments to provide subject-specific criteria, recommended the use of generic assessment criteria.
40 One reason for audit reports encouraging the use of subject-specific assessment criteria was the need for explicit and secure links between them and learning outcomes. Some reports noted that such links had been or were being provided. In institutions where links were not strong, the audit reports concluded that steps should be taken to ensure that assessment criteria were well matched with learning outcomes, and that the connections between them were clearly articulated. For example, one such report invited consideration of 'ways in which assessment criteria might be developed which reflect more explicitly the level of modules, their learning outcomes and their modes of assessment'. Another drew attention to the value of embedding practice which ensured that all students were made aware of linkages between assessment criteria and learning outcomes.
41 Students were often informed, and sometimes well informed, about assessment criteria by handbooks [Harper Adams University College, paragraph 34]. There were, though, instances where audit reports noted that information provided to students was not always as helpful as it might be. Several reports mentioned that student awareness of the links between assessment criteria and learning outcomes helped to enhance their understanding of what was expected of them.
Feedback to students on assessed work
42 Almost all the audit reports mentioned the quality and timeliness of feedback to students on their assessed work, usually in the context of discipline audit trails. In many cases, the reports observed that there was good practice in using this feedback to enhance students' learning [Imperial College of Science, Technology and Medicine, paragraph 222; Henley Management College, paragraph 122; Harper Adams University College, paragraphs 34 and 112]. For example, one report noted that feedback in one discipline 'is generally clear and balances appropriate use of the criteria associated with the assessment task with more individual and discursive comments on the student's performance'. Meanwhile, in another discipline, feedback was 'full, detailed and constructive' with prominence given to 'praiseworthy elements in student work and to areas in which there was scope for improvement'. Another report observed that 'marking criteria are used effectively to enable students to understand how they are performing and in the work reviewed there was evidence of valuable feedback to students about their work'. A further report identified the 'careful use of assignment briefs, moderation and marking criteria to assist in measuring the attainment of standards and providing effective feedback to students' as a feature of good practice [Harper Adams University College, paragraph 34]. Several reports mentioned that a standard form, when used systematically and with suitable reference to assessment criteria and learning outcomes, facilitated constructive feedback.
43 There was though (and perhaps inevitably) variation in practice both within and among institutions. A number of audit reports drew attention to instances where variation had led to weaknesses, and in some cases recommended improved arrangements. Such variations, as indicated in one report, could be confusing to students and result in inequity. In some cases, guidelines had been produced at institutional or local level as an indication of minimum requirements. Quite often, though, there were signs that these were not always followed. Hence, institutions were recommended to reconsider and strengthen the implementation of their guidelines.
44 Where there were no guidelines, or where local guidelines permitted a wide range of practice, audit reports encouraged institutions to review the diversity of practice and consider whether such guidelines would be helpful. In one report, for example, the institution was advised 'to develop appropriate procedures to ensure a central overview of faculty practices with respect to the provision of timely feedback on assessed work in order to ensure both that written commitments are adhered to and also the comparability of the student experience across the institution'. Another report noted that the students' written submission highlighted that a significant number of students perceived the quality and timeliness of some feedback to be poor (a point later confirmed in the discipline audit trails). The institution was encouraged to respond firmly and ensure that its guidelines were implemented.
45 Many audit reports included students' comments on the quality and timeliness of feedback on assessed work. In a number of cases, the students' written submission was a source of information, but more often the views of students were obtained from discipline audit trails. There were complimentary comments, as well as some criticisms. One report noted that students had expressed the view that 'feedback of their assessed work was helpful, consistent and related to the stated marking criteria'. Another indicated that students had commented favourably on the timeliness with which assignments were returned and the helpfulness of the markers' comments [Henley Management College, paragraph 156]. One typical criticism of feedback by students who met audit teams in the course of discipline audit trails was that 'when it is good it is very good, but at times it is too short and uninformative'. Another was that some staff took too long to return marked work. Comments in a number of audit reports drew attention to the value of, need for, or availability of feedback to students on their examination performance.
46 Many audit reports contained comments to the effect that the purpose of feedback was not just to evaluate the quality of students' work; feedback should also assist them in showing how to improve that quality. Comments in several audit reports suggested that this latter purpose was not always fulfilled. Some reports indicated that feedback was absent or less constructive in this respect than it might be. For example, one report recommended consideration of ways to ensure that the institution's requirements were fulfilled so that students on all programmes could 'receive sufficient feedback to allow them to recognise and enhance their achievements and the areas in which they may be deficient'.
47 Overall, however, in the audit reports published between December 2004 and August 2006 most negative comments associated with feedback related to delays in its provision so that students did not receive it in sufficient time for it to be useful, or to variability of practice in returning marked work to students. One report observed that a school's introduction of a suitable policy on returning work to students had, in the team's view, improved the timeliness of feedback. As noted in some reports, the early provision of feedback on students' work could make a significant contribution to a student retention strategy. In addition, as one report indicated, the provision of prompt and constructive feedback was an important indicator for international students of what was expected of work at a UK higher education institution.
Conclusions
48 The institutional audit reports published between December 2004 and August 2006 indicate that institutions have given sustained and careful attention to the security, reliability and consistency of their assessment arrangements. From the audit reports it appears that the 2000 edition of QAA's Code of practice, Section 6: Assessment of students has played a prominent role in guiding and promoting the good practice that has developed. Discipline audit trails have enabled the auditors to observe assessment practices and identify a number of features of good practice. Given the important role that assessment plays in any student's experience of higher education, it is not surprising that assessment practices are closely scrutinised in the audit reports, and that a number of recommendations for improvement in those practices are made. Consistency of practice is an important theme, though there are indications in some reports that discipline integrity might not always sit comfortably with the uniformity that consistency might seem to require.
49 There are a large number of recommendations - significantly more than the number of features of good practice. The recommendations call for improvements in practices relating to the classification of undergraduate honours degrees, the effective and open use of assessment criteria, and the use of feedback on students' work to promote their learning. In a number of cases, it is clear that some of the recommended improvements were already being introduced and implemented. The brief responses to audit reports, included as appendices to the reports, show that in some cases steps have already been taken to address recommendations. In other cases, it is clear that they will receive careful consideration.
Top