Assessment and Qualifications Alliance (AQA)
August 2001

Foreword

1 The Quality Assurance Agency for Higher Education (QAA) is responsible to the Department for Education and Skills for the recognition of Access to Higher Education courses. QAA exercises this responsibility through a national network of authorised validating agencies (AVAs), which are responsible for the processes of individual course recognition and the award of Access to HE certificates to students on QAA's behalf. QAA has developed a scheme for the licensing and review of the AVAs, the principles and processes of which are described in the QAA Recognition Scheme for Access to Higher Education in England, Wales and Northern Ireland. The Recognition Scheme is regulated and administered by the Access Recognition and Licensing Committee (ARLC), a sub-committee of the QAA Board of Directors.

2 This is a report of a major review of the AVA function of the Assessment and Qualifications Alliance (AQA) undertaken by QAA. The Agency is grateful to AQA and to those who participated in the review for the willing cooperation provided to the review team.

Aims and objectives

3 The aims of the system of AVA review are:

i to provide the basis for an informed judgement by the ARLC about the fitness of the AVA to continue as a licensed agency;
ii to promote public confidence in Access to HE as a properly regulated and respected route into higher education by assuring:

the quality and adequacy of AVAs' systems and procedures;
the quality,comparability and range of AVAs' operations;
the adequacy and comparability of AVAs' standards for approval, moderation and monitoring of courses;
consistency across AVAs in the operation of criteria for the granting of the Access to HE award;

iii to stimulate reflective and self-critical perspectives within AVAs, as an instrument to promote quality enhancement;
iv to provide an opportunity to identify and disseminate good practice of AVA operations;
v to provide a mechanism for ensuring necessary, and encouraging desirable, improvements and developments in AVAs.

4 The objectives of each AVA review are:

i to examine, assess and report on:

the development of, and changes in, the AVA since its last review or initial licence, and its plans and targets for the future;
the organisation's continuing viability and robustness and the ways in which the AVA demonstrates sound governance;
the efficiency and effectiveness of the AVA's operational and quality assurance systems;
the range and scope of the AVA's activities, and the appropriateness and value of these activities;
the ways in which the AVA approves and monitors courses and the ways in which these processes take account of the need for consistency and comparability;
the ways in which the AVA satisfies itself of the adequacy and comparability of standards achieved by students gaining the Access to HE award;
the evidence available to indicate the AVA's success in achieving its aims and targets;

ii to identify and report on:

strengths and good practice in procedures and operations;
areas which would benefit from further development;
areas requiring attention.

Outcomes

5 Following the review of an AVA, the reviewers' report is presented to the ARLC by a member of the review team. The Committee then makes one of six decisions:

i unconditional renewal of licence for a specified period;

ii conditional renewal of licence with conditions to be met by specified date;

iii provisional renewal of licence with conditions to be met and further review visit by specified date;

iv suspension of licence until specified conditions are met;

v withdrawal of licence for operation as an AVA;

vi temporary renewal of licence with request for further information by specified date (decision suspended).

The review process

6 The review was conducted in accordance with the process detailed in the QAA Recognition Scheme for Access to Higher Education in England, Wales and Northern Ireland. The preparation for the review included an initial meeting between AQA officers and the QAA Assistant Director with responsibility for Access to HE matters to discuss the requirements for the Analytical Account (the Account) and the process of the review; the preparation and submission by AQA of its Account, together with a selection of supporting documentation; a meeting of the review team to discuss the Account and supporting documentation and to establish a draft programme for the review visit; and negotiations between QAA and AQA to finalise the programme and other arrangements for the review visit.

7 The review visit took place on 27 and 28 February 2001. The visit to AQA consisted principally of meetings with members of the AQA senior management team and Quality Audit report team; AQA officers; members of the Access Management Group and Quality Committee; moderators and external examiners for Access to HE programmes; Access to HE course coordinators; senior managers from further education; and higher education admissions staff. 8 The review team consisted of Dr Peter Easy, Assistant Director, Cheltenham and Gloucester College of Higher Education; and Mr Alan Smith, Quality Assurance Officer, North Wales Open College Network. The review was coordinated for QAA by Ms Kath Dentith, Assistant Director (Access), Institutional Review Directorate.

AVA statistics 1999-2000* 9

Providers offering Access to HE programmes 10
Access programmes available 25
Access programmes running 25
Access learner registrations 590
Access to HE certificates awarded 576

*as provided by the AVA in its annual report to QAA in December 2000

The AVA context

General background

10 The Assessment and Qualifications Alliance (AQA) was formed in 1997 as a joint venture company bringing together the Associated Examining Board (AEB) and the Northern Examinations and Assessment Board (NEAB), with the participation of City and Guilds. In 1999, AEB and NEAB announced a formal merger which came into effect in April 2000, creating AQA as the largest English unitary awarding body. The original AVA licence was granted in 1991 to the Joint Matriculation Board (JMB), one of the four constituent bodies which were amalgamated to form NEAB in 1992. At this point, the licence was transferred to NEAB and, in 1995, was confirmed by the Higher Education Quality Council (HEQC) following a periodic review. In 1998, the Quality Assurance Agency agreed that, since AQA had maintained the structures and procedures operated by NEAB in relation to the AVA, this amounted to a change of name, rather than a transfer of licence, and the newly formed AQA was confirmed as the AVA licence-holder.

11 AQA has offices in Bristol, Guildford, Harrogate, Manchester and Newcastle. It employs around 900 full-time staff and a large number of part-time staff who are responsible for the administration, setting and marking of a range of AQA examinations covering GCSE, GCE, AVCE, GNVQ and other vocational qualifications for some 3.5 million candidates. Its Access to HE activities are based at the Manchester office and are located within its Education and Research Division.

Membership and size

12 As an AVA, AQA currently validates 25 Access to HE courses across 10 providers, all of which are colleges of further education. AQA's position as a national body is reflected in the geographical spread of the Access providers which use its services, six of which are in Yorkshire, one in the Midlands, and three which are further afield in Trowbridge, Taunton, and Portsmouth. The total number of learners awarded Access to HE certificates has varied in the past from over 1,200 in 1994 to 349 in 1998, this decline being largely due to the departure in 1996 of 12 former providers as a result of the creation of the Hampshire Authorised Validating Agency. There has been some slight growth in the last two years, and 576 certificates were awarded in 2000.

13 Those institutions which use the services of the AVA for course recognition and certification of students are regarded as its 'members' in the further education sector. The broader membership of what was described in the Account as the 'Access Partnership' includes the five universities formerly associated with NEAB (Birmingham, Leeds, Liverpool, Manchester and Sheffield) together with a number of other higher education institutions in the Midlands, Lancashire and Yorkshire, most of which have longstanding links with the five 'NEAB' Universities. However, this 'membership' is somewhat notional: whereas the NEAB had a membership structure which included these universities, AQA is an independent company. As the licence is held by AQA itself, it is difficult to identify the sense in which these institutions are 'members' of the AVA. There is no AVA statement which indicates the nature of members' rights and responsibilities, and the institutional commitment of the 'members' of the AVA is not assured through a formal membership agreement or any financial commitment.

14 All former reviews of the predecessors of AQA have remarked on the distinctive nature of its role as an AVA. In common with those bodies from which it inherited its licence, and unlike the majority of locally based AVAs, AQA considers that its catchment area is national rather than regional. As a result of this particular characteristic, while fulfilling those aims of the QAA Recognition Scheme for Access to HE which promote diversity, AQA faces particular challenges in meeting satisfactorily other requirements which are inherent in the holding of an AVA licence. The review team heard, for example, that providers valued the sense of impartiality which was offered by membership of an AVA which was not based in their geographical region; however, this might be balanced with the evident difficulties faced by the AVA, for example in the organisation of development events for its members.

Financial position

15 On 1999-2000 figures, AQA's total annual income is some £93 million. Separate figures related only to AQA's activities as an AVA were not available at the time of the review, although it appeared likely that income directly attributable to Access to HE courses was less than 0.1 per cent of the overall total. The review team noted that fees for providers had been raised for 2000-01 but that a 'shortfall' (in terms of the AVA activity covering its costs as a part of AQA) was still expected.

16 The review team also noted that further discussions on the level of fees had recently taken place with the aim of meeting all costs, making a contribution to AQA's overall finances, and creating a surplus for use by the AVA for publicity and development purposes. In the view of the team, this did not appear to be a realistic ambition without a substantial increase in fee levels. The financial contribution made by the AVA stands as a measure of the relative scale of its activities within AQA's overall operation. As in many other areas of the AVA, it appeared to the team that a more pertinent discussion might focus on whether the future of the AVA should be dependent on its support from the general AQA resource base, or whether it should be conceived as a separate and relatively free-standing activity. In the present context, the AVA, as a distinct area of activity within AQA, has no real responsibility for the management of budgets, as it has little or no control over the resources which it is allocated, beyond providing advice which may influence the setting of fees (see paragraph 34, below). Drawing from this example, part of the general context for this review can be seen as the nature of the difference between AQA as an overall organisation, and the operation of the AVA which is housed within that larger structure.

Mission, aims and objectives

17 An apposite expression of the particular issue of the 'independence' of the AVA within AQA can be found in a consideration of mission, aims, and objectives. The review team noted AQA's clearly articulated mission and heard that, although a minor part of AQA, Access to HE fitted well with the organisation's wish to contribute to the lifelong learning agenda. However, there appeared to be no established or formal statement of objectives for the AVA per se. Although the Account did contain a statement of aims and an action plan for the future, these appeared to have been constructed for the particular purpose of the review, and there was little supporting evidence that the AVA's committee structures operated against any formally articulated or continuing strategic plan for action, development or enhancement. The responsibilities involved in holding an AVA licence would suggest that such regular planning, and the periodic monitoring of targets created by it, is a necessity, and should be a priority for the AVA.

Targeting policies and strategies

18 While the aims of the AVA identify a commitment to support providers, and to encourage them in their work of widening participation and lifelong learning, it was unclear to the review team what specific actions by the AVA might contribute to this. That part of the AVA's Account which addressed the issue of target groups and related policies, placed emphasis on the role of individual providers to respond according to their own analyses of local needs. In this context, the review team noted examples of good practice in the strategies of some colleges. There was little evidence, however, that the AVA itself took an active role in the promotion of targeting or the encouragement of participation from traditionally under-represented groups. The Account described the responsibility of initial validation panels and, subsequently, of moderators to monitor and report on issues such as equal opportunities and widening participation. However, these matters did not figure prominently in the AVA's consideration of moderators' reports seen by the team, and there were few indications that the committee structure of the AVA had addressed these issues in any detail. Given that the promotion and enhancement of Access to HE, on the principles described in the Recognition Scheme, is a clear licence responsibility, a more proactive and purposeful approach should be adopted by the AVA in the context of targeting policies and strategies.

Developments since the last review

19 The review of the AVA conducted by HEQC in 1995 made a number of recommendations including the establishment of a new committee structure; the introduction of revised quality assurance arrangements; the provision of appropriate staffing levels to support the AVA; and a clarification of the moderating/external examining processes. The Account supplied for the current review discussed the progress made in respect of these recommendations, and the review team noted the development of the AVA's governance structure and the work of the Quality Assurance Committee which had resulted from it.

20 In some areas, however, there appeared to have been little real progress made. These include some continuing issues raised by the relationship between the AQA-appointed moderators (previously known as Assessors) and the provider-appointed external examiners (see paragraph 65 ff). In addition, the adequacy of the staffing resource dedicated to the support of the AVA remains a matter of concern although, in the view of the review team, this is a matter of its nature rather than its volume (see paragraphs 42-44, below). As regards targeting strategies, which were also criticised in the previous review and in the initial approval of the AVA in 1991, the fact that some parts of the Account bore a strong similarity to the AVA's critical appraisal compiled for the HEQC review in 1995, did not inspire confidence in the nature and development of the AVA's thinking in the intervening period.

Current and planned AVA developments

21 Members of AQA's senior management acknowledged that Access to HE had not been a priority for the organisation in recent years, and this is reflected in a lack of major policy developments for the AVA over this period. The recent establishment of AQA has brought about a number of organisational changes, and AQA intends to transfer the location of its AVA function from Manchester to its offices in Harrogate from September 2001. This will result in its co-location with AQA's Unit Award and Centre Accreditation Schemes, operations which AQA believes have more in common with the nature of the work of the AVA. With the retirement of the current Senior Officer who has held responsibility for the management of the AVA, a new Principal Subject Manager will be appointed to take over this function.

22 The title of the supplementary paper given to the review team which described these changes, Future administration of Access within AQA, encapsulated some of the concerns of the review team since it confirmed an emphasis which was evident elsewhere during the review on the administrative duties associated with an AVA rather than on broader management and development functions. The team was told that, as the transfer to Harrogate was part of a restructuring within the organisation, the new Principal Subject Manager was likely to be an internal appointment, and unlikely therefore to have any significant experience relating to Access to HE provision. The current plans caused the team to have concerns about whether there is sufficient appreciation within AQA of the full range of functions - beyond the simply administrative - which are required in the management of an AVA licence. In particular, it is a reasonable expectation that the appointment of a lead officer for an AVA should involve some measure of confidence in his or her expertise in Access to HE matters. In reaching these conclusions, the team does not consider that there is any wilful neglect within AQA but that there is a lack of proper awareness of the full role and responsibilities of a licensed AVA.

23 The action plan provided for the review described a programme of future developments between the summers of 2001 and 2002 which included constitutional changes to the AVA's Access Management Group, a review of the AVA as a whole by that reformed Group, developments within the areas of unit-based courses and part-certification, and a move towards greater standardisation in a number of aspects of Access to HE courses validated by the AVA. At various points throughout the Account, there are references to other procedural developments, either recently introduced or intended to be introduced.

24 In view of the considerable operational and other changes about to be implemented, and the number of recently introduced changes which have yet to be fully embedded, the review team considered the dates given in the Account to be ambitious. Although the current Senior Officer will be present to support the transfer for a limited period, the team would query whether the action plan described in the Account can be fully accomplished in the announced timescale and would wish to see a more carefully considered and detailed action plan developed to take forward the issues identified.

Governance and committee structures

Legal and constitutional status

25 AQA is an independent company limited by guarantee and has charitable status. The governing body is its Council, which is an independent body with membership drawn from the following categories: school/college teachers, Higher Education, Local Education Authorities, and Workplace/others.

AQA Council

26 The AQA Council was described to the review team as the ultimate locus of responsibility for the AVA licence, in keeping with its responsibilities for all AQA awards. The links between the Council and the AVA's specific committee structures and management are thus a matter of some importance. Since the merger in April 2000, the Access Management Group (see paragraphs 33-35, below) has been made a direct sub-committee of the Council although no minutes of the Access Management Group have as yet been forwarded to the Council. The team also noted a wish within AQA to simplify the current AVA structure, perhaps by greater incorporation into AQA's own committee system, in order to improve integration. Such thinking was influential in the changes to the membership of the Access Management Group which have recently been adopted.

27 Despite this, the review team encountered some difficulties in understanding how the AQA Council might currently, or in the future, exercise its responsibilities for Access to HE. Given the proposal for the possible inclusion of a member of the Council on the Access Management Group (see paragraph 35 below), there appeared to be no plans for the concerns of the AVA to be directly represented on the Council other than by receipt of minutes from the Access Management Group (although a sub-committee, it did not appear that the Access Management Group's Chair would automatically be a member of Council). Elsewhere, the team noted that no providing institution within the AVA was represented on the Council. The Account accentuated this apparent division since it made only passing reference to the role of the Council, or any other AQA committee, in relation to the work of the AVA.

28 For these reasons, the review team was not able to confirm that the Council represented a robust and responsible location for the AVA licence. As the new committee structure of AQA begins to become fully operational, QAA would expect to see evidence that the Council is fulfilling its role in relation to Access to HE.

29 The AQA Council also maintains a Quality Assurance Committee which has a remit to 'monitor quality assurance procedures and outcomes within AQA' and specifically to 'consider, where appropriate, matters concerning quality assurance raised by outside agencies, including the Regulating Authorities' (this committee is not to be confused with the Quality Assurance Committee of the AVA itself, see paragraphs 36-37, below). The review team understood that the work of this committee was to scrutinise all of AQA's operations, and not to include any detailed responsibilities for the routine quality assurance of any one particular area unless it were to prove necessary. In this sense, the Quality Assurance Committee has no direct or specific role in the AVA's operations except by default in the case of QAA, as the regulating body, raising matters of concern.

30 One recent development associated with the work of the Quality Assurance Committee has been the establishment of a Quality Audit Unit. The brief of this Unit is to undertake self-assessment and risk management studies of the various component operations of AQA. Among the first pilot studies undertaken by the Unit is a scrutiny of the validation processes for Access to HE courses, and assessment standards in Access to HE providers. The Account records that the findings of these studies will be reported to the AVA's major committees as well as to AQA's overall Quality Assurance Committee and 'an appropriate Action Plan will then be drawn up'. Given that these studies were still being undertaken at the time of the review, no specific plans for action have yet been drawn up, and the team is not able to comment on how the AVA intends to address any of the particular outcomes of this work, or more generally how they would be coordinated with existing quality mechanisms. However, the team was able to scrutinise draft reports of the pilot studies and had an opportunity to discuss them with members of the Quality Audit Unit. It appeared to the team that the self-assessments were likely to provide important material for consideration, and would encourage some early reflection of how such studies could be integrated with the current quality assurance systems of the AVA.

Annual AQA 'Partnership' meeting

31 The particular nature of an AVA housed within a large and diverse organisation gave rise in the 1995 review to detailed discussions about the ways in which the membership of the AVA might become more fully involved in its governance and operations. Perhaps unlike a more locally based 'partnership' AVA, there existed no forum where all members could meet to contribute to management and development issues. In partial response, an annual 'Partnership Meeting' was established in 1995. The usefulness of this meeting as a method of communication was demonstrated by the minutes made available to the review team: the AVA's annual report to QAA was discussed; reports on the year's activities within the AVA including validation exercises, membership changes, and staff development events were received; and members were kept abreast of changes both nationally and within AQA.

32 Notwithstanding these advantages, this annual meeting has no position in the governance structure of the AVAand, therefore, cannot fulfil the purpose of allowing the broader membership to become involved, on a constitutional basis, in the operation of the AVA. Such opportunities continue to be limited to those members who have staff nominated or elected to the Access Management Group. The review team would recommend that the AVAcontinues to seek ways in which it might grant all of its members the right to become involved in the AVA's activities and future planning.

Access Management Group

33 The Access Management Group (AMG) is responsible for the overall management of the AVA function within AQA. At the time of the review, it had 10 members divided equally between representatives from the five 'NEAB' Universities and representatives elected by the providing further education colleges. Its terms of reference include responsibility for all aspects of validation, review, and monitoring, as well as for staff development and publications.

34 In its scrutiny of the minutes of the AMG, the review team noted the Group's systematic receipt and discussion of issues presented to it by its two sub-committees (the Quality Assurance Committee and the Staff Development Committee). In particular, there was evidence of a methodical approach to quality assurance matters, with the Group requesting further reports in cases where it felt that existing information was not adequate for it to fulfil its proper remit. The Group had also been engaged in a variety of other discussions including the setting of fee levels and the consideration of issues arising from the future management of Access to HE within AQA.

35 These latter discussions have involved the possible reconstitution of the membership of the AMG in response to a growing sense that the existing committee structures were no longer fully reflecting changes in the AVA. In May 2000, the AMG proposed that its university representatives, currently drawn wholly from the five 'NEAB' universities (which accept a very small minority of learners from the AVA's providers) should become more representative of the main receiving higher education institutions. These were characterised as post-1992 universities mostly located in Yorkshire, and the Midlands and south coast regions. Following a series of iterations between the AMG and AQA's Management Executive Team (MET), a final compromise was reached in approving the five higher education members as one representative from the AQA Council (or its Education and Training Committee) who 'is in regular contact with Access students', one representative from a former NEAB constituent University, and 'three HE representatives from other users of the scheme'. The review team would welcome this revised membership as fulfilling the aim of broadening the constituency of the AMG .

Quality Assurance Committee

36 Membership of the Quality Assurance Committee (QAC) is determined by the AMG. Currently, its composition calls for six members, one of whom must be from a higher education institution. The remit of the QAC requires it to make recommendations to the AMG on quality assurance matters including the validation and revalidation of Access to HE courses, and all issues related to their moderation.

37 While the operation of the AVA's quality assurance systems through the QAC raised a number of more detailed issues, the review team found that the work of the QAC was generally sound. Recommendations following validation events were given appropriate consideration and the committee's scrutiny of moderators' reports was detailed and systematic. In this context, the AVA appears to have responded positively to the recommendations made at the time of the last review. The review team noted that the QAC intended to discuss the outcomes of the pilot studies of the Quality Audit Unit (see paragraph 30, above) at its next meeting. The team would hope that the QAC would be able to reflect on the detailed findings presented, and consider whether its overall monitoring of the AVA's quality systems has yet attained the proper level of detail.

Staff Development Committee

38 As with the QAC, and using precisely the same terms for its composition, membership of the Staff Development Committee (SDC) is decided by the AMG. The basic remit of the SDC is to identify and respond to the development needs of Access staff within the AVA. Drawing a further comparison with the QAC, the Committee was also established as a response to the recommendations made by the last review, although it appeared to the review team that it had been less successful in fulfilling its remit.

39 The review team encountered some difficulty in understanding the current status of the SDC. Although the Committee's remit calls for it to meet three times a year, it has met only annually since 1997 and has not met at all since October 1999. This issue of status was highlighted by the Account which appeared to announce the SDC's demise in noting that its function 'was no longer crucial' since 'this aspect of the AVA's work is now embedded in its procedures'. During the course of the review visit, however, the team heard that members of the SDC had met informally when in the office for other meetings, and that a final decision on the future of the Committee had not yet been taken. As a minor, but nevertheless indicative, comment on the SDC's role, the team noted that there was no record of these informal meetings, and that its past formal meetings had not been minuted with the same care as those of other committees in the AVA's structure with, for example, no record of the receipt of previous minutes, nor of any discussion of matters arising.

40 The business of the SDC has been largely dominated by the arrangements for the AVA's annual conference which is a part, but certainly not the major component, of its remit. While the review team would acknowledge the value of the annual conference as an event for the dissemination of good practice or the opportunity to update members on recent developments in Access to HE, it cannot alone take the place of a more systematic and thorough approach to the development responsibilities of the AVA. Other workshops and development events have been organised by the AVA but, on a number of occasions, these have been cancelled or otherwise thwarted, often through poor attendance.

41 It might be argued that one of the disadvantages of an AVA which operates on a national basis is that the concept of membership is weaker than that which may be found in more locally based organisations. The geographical distance between providers, and therefore the greater time required for staff release to attend AVA events, as well as their relative lack of involvement in the formal structures of the AVA, may well be a significant contributor to the rather sporadic participation in staff development activities. The AVA's future plans are to allow such activities to be generated through the work of the AMG and QAC. It may be a matter of debate as to whether a body such as the SDC is needed in order to promote development activities for the AVA's members. However, it is not a matter of dispute that these functions are currently an underdeveloped aspect of the AVA's activities and the AVA should move quickly to resolve this situation.

Management, organisational structures, and administrative arrangements

Staffing roles and remits

42 In essence, the AVA has no dedicated full-time staff. An AQA Senior Officer is responsible for providing administrative and technical guidance to the AVA's committees, and first-line advice and support to its providers. However, the review team was told that the majority of the current Officer's time is spent on other duties within AQA. The Officer is supported by a 0.5 part-time clerical assistant. The Account points out that other AQA staff are available to assist if additional help is required. Even for a relatively small AVA, and with no intention of undervaluing the work which is undertaken by the current Officer, such levels of staffing can only be expected to provide limited support beyond the basic administrative duties which appear to dominate AQA's approach to its AVA responsibilities. There is little time for the more wideranging support and development functions which are characteristic of other AVAs.

43 Those who support the AVA through its committees and as moderators or external examiners provide some developmental support, and this can serve a useful purpose in responding to particular issues as they arise for providers in relation to validation and moderation, for example. The role of the moderator, in particular, includes some elements of development in support of the AQA Senior Officer for updating colleges on aspects of AVA procedures and guidance received from QAA and in the constant development, overview and revision of the Access to HE courses for which they are responsible. However, the AVA, as a distinct area of activity within AQA, does not currently have either the staff or established mechanisms through which, as a routine aspect of its activities, it can provide systematic support for the development and enhancement of the AVA's Access provision as a whole.

44 The Senior Officer is, by default, a key figure in the quality assurance functions of the AVA. The Officer is normally involved in preliminary visits to prospective member institutions; gives a range of informal and formal advice related to the validation of courses; supports the AMG and QAC and prepares a papers for their discussion; takes a lead in the receipt and response by QAC to moderators' reports; and generally acts as a source of advice for providing institutions. Such a list of responsibilities is indicative of the reliance which is placed on this sole officer and, while the review team heard that other members of AQA staff could provide some cover as a contingency, the AVA should consider as a matter of urgency a greater spread and depth of officer support to ensure its effective operation.

Data management, collection and analysis

45 The AVA benefits from the extensive data management systems employed by AQA. Providers are required to register students in the early part of each calendar year, and a data record is then created and checked by both the AVA and the colleges. All claims for Access to HE awards are verified against this record before certificates are issued. While this system has operated effectively to ensure accuracy in the issue of certificates to successful students, it has not provided an appropriate base for other purposes. The AVA has encountered some difficulties in collecting the broader range of statistics on students on recognised Access courses required to make a full and accurate annual return to QAA. The review team was told that frailties in the management information systems of some providers were a contributory cause, but the team also noted that no particular sanctions were in place to encourage the colleges to make timely returns; furthermore, in spite of its obligations to QAA, the AVA appeared reluctant to consider such sanctions. The team was informed that some guidance would shortly be given to colleges for statistical returns: the team considered that Access providers should also be reminded of their obligation as participating institutions within the QAA Recognition Scheme for Access to HE to make data returns to the licensed AVA which will allow it to fulfil its own reporting obligations to QAA.

46 The lack of consistency in data returns from providers makes it difficult for the AVA to track learners during their Access to HE courses, and scrutiny of the AVA's statistics supplied to QAA showed a number of significant anomalies in certain categories of basic information, such as the numbers of students registered on Access courses and the number who have left the course as opposed to those who are continuing on to the second year of a two-year course. The incompleteness and lack of reliability of the data collected will inevitably make meaningful analysis difficult and hinder the AVA's ability to formulate policy for the AVA and guidance for Access providers on the basis of well-evidenced information.

47 The AVA has been more successful in its attempts to track students as they move into higher education (see paragraph 82, below). The significant time and resources which have been devoted to this exercise is indicative of the emphasis which the AVA places on such 'destination' data as a measure of the quality of its validated courses. The review team noted that the AVA's action plan included a stated intention to 'develop strategies for improving tracking student progress in Higher Education'. The AVA may wish to consider whether this focus on the success of learners in higher education should be matched by an equal effort to collect and analyse statistics on the recruitment and progress of students on the Access to HE programmes themselves.

Communications with providers and promotional materials

48 Standard communication from the AVA to its providers is achieved by a system of nominated contacts who are normally the Access coordinators for each validated course. Moderators are also expected by the AVA to communicate information to the institutions in which they undertake their duties. In discussion with representatives from the providing institutions, the team heard that communications were generally timely and informative and course coordinators spoke with satisfaction of the availability and nature of the support provided by the AVA Senior Officer.

49 Other aspects of communications did not appear to the review team to be as well developed, particularly in the area of publicity materials. In scrutinising a sample of promotional leaflets from providing institutions, the team noted that they contained inaccurate versions of the QAA Access to HE logo, did not acknowledge AQA as the authorised validating body and, in one case, appeared to attribute authorisation to the National Open College Network. In addition, the most recent version of AQA's own Access to Higher Education Courses: A Directory included no acknowledgement of its own authorisation from QAA, and contained details of courses which had subsequently not been approved and were therefore unavailable. Such inaccuracies are not helpful to learners in arriving at informed decisions about the choice of courses, and the AVA will wish to establish a monitoring mechanism through which it can ensure that accurate information on the availability and status of courses is offered by its providers.

50 While the AVA annual conferences over the last eight years have provided opportunities for consideration of a range of subjects and current issues in Access, there has been an underlying trend of poor attendance, cancelled events and general acceptance that it is difficult for staff to attend such activities. The Account identifies that the dispersed nature of the AVA is a major contributor to these problems, a view supported by a number of participants in review meetings.

Development, validation and evaluation of Access courses

Development and validation of courses

51 The development process involves a visit from the AVA's senior officer and may also involve a visit or visits from a member of the QAC or AMG. In this way, committee members are directly involved in, and well informed about courses under development. New providers may also be put in touch with an experienced Access tutor who can act as a mentor during the development phase.
52 The AVA provides written guidance on the process involved in preparing and submitting Access to HE courses for validation, and the validation process itself, in its Validation Procedures document. This has been updated recently and the revised version (retitled the Handbook on Validation Procedures) provides a fuller description of procedures and criteria than its predecessor. Validation panels are appointed by the QAC, and consist of two members of that committee and 'a representative from an HE institution not within the AVA'. A validation panel will receive written comments from subject specialists about individual units within the course, and may also co-opt one or more external subject specialists to assist it in its deliberations. The validation process involves the scrutiny of documentation by the validation panel, followed by a visit to the provider to discuss the course with staff, according to an agreed agenda. Following any necessary further iterations with the provider, the validation panel reports to the QAC and a recommendation is made to the AMG on the approval of the course. The QAC is thus directly involved in the process of both the development and validation of courses, and this may assist in providing a mechanism to enhance consistency in decision making and standard setting. However, the AVA will wish to assure itself that there is no potential for any conflict of interest in the direct involvement of members of the QAC in both of these processes which lead to the recommendation to the AMG for the formal recognition of a course.

53 Detailed notes of the validation meeting are sent to the provider to assist with the resubmission of the amended documents to the AVA by agreed dates. Following the resubmission of revised material, a letter is sent to the providing college confirming that all the requirements have been met and that the course has been approved for a specific period of time, usually five years.

Monitoring, evaluation and review

54 After an initial validation period of five years, courses must be revalidated. The revalidation process is largely a repeat of the validation process, with an updating of course units and procedures. It does not include any review of the recognised course, informed, for example, by the reflections of previous students, moderators, and receiving HEIs. The review team recommends that a more discursive process be considered for revalidations, which includes a full evaluation of the successes and any shortcomings of the course, and a consideration of relevant statistical indicators.

55 Changes can be made to courses between validation and revalidation with the agreement of the course moderator. Although the QAC is advised of any changes made in the moderators' reports, there do not appear to be any guidelines which set parameters for the extent of revisions which moderators may authorise, and the QAC's consideration of moderators' reports, and the committee's subsequent response to providers, takes place after the next run of the course has started. External examiners may also act as subject experts for the provider when new units are being developed, and advise on the structure and design of Access to HE courses. While external examiners may be able to provide valuable advice based on subject expertise, their role in advising colleges on the structure and design of Access to HE courses, having received no training or guidance from the AVA, and without clearly specified common criteria, gave the review team occasion for concern. The team considers that this is an area that the QAC should review to ensure that the AVA is in full control of programme development and is able to maintain consistency across programmes.

56 While most colleges produce an annual review of their Access to HE courses for internal purposes, there is no current requirement for providers to supply such a review to the AVA, and only some provide them to their moderators. The Account explains, however, that 'In future, the Quality Assurance Committee intends to ask for these written reports to be provided for the AQA moderators so that details can be fed back to the Committee'. The review team supports this development, although timing for the production of these reports to meet college schedules, and their supply to moderators to meet moderators' reporting schedules to the AVA will need further consideration if course reports are to be fully integrated into the AVA's quality assurance systems. In the absence of course reports, the only review mechanism available to the AVA to date has been the annual moderator's report to the QAC.

57 Following a decision by the QAC to review comparability of its recognised Access courses, a survey of all course documents was undertaken at the beginning of 2000. The report which presented the findings of the survey, Comparative Review of AQA Validated Access Courses, identified a considerable number of variations in the structure and operation of courses and made a series of recommendations for action by the AVA. The review team commends the AVA for undertaking this worthwhile initiative, but was concerned about the extent of inconsistency revealed by its outcomes, and, moreover, was concerned to note that the nature of recommendations made seemed to emphasise improving consistency of course documentation without addressing some of the more fundamental inconsistencies revealed. In the view of the team, a number of the differences described question the effectiveness of the AVA's procedures to assure the consistency of outcome standards on Access courses. The team therefore concluded that this work should be taken forward as a major area of developmental activity, and that AQA should consider as a matter of urgency how it can address consistency in the award of the Access to HE certificate more broadly.

Unitisation and credit-based programmes

58 A major area of recent change for the AVA in relation to course development has been the unitisation of courses, and the use of credit as the basis for the Access to HE award has gradually become more common across the AVA. The Account states that this development was initiated by providers who 'wished to move to such an arrangement' which was 'accepted by the AVA', rather than being the result of a policy decision by the AVA itself. Perhaps because of the reactive nature of this development, the move towards all colleges adopting a unitised and credit-based system for Access to HE awards has been slow, and the AVA has only recently developed a policy requiring a consistent approach by Access providers in relation to credit. However, the recent validation of a non-credit based course led the AVA to consider its position, and AQA has now decided that all new courses, and courses presented for revalidation, should be presented in a unitised, credit-based format.

59 The AVA was at one stage an associate member of the National Open College Network (NOCN), and ascribes credit value according to the same formula as that used by NOCN, and makes use of the NOCN level descriptors. These are published in its Validation Procedures, and this document includes a section which provides specific guidance for unit-based courses. However, following NOCN's withdrawal of 'associate member' as a category of membership, this status was no longer available to AQA. The award of credit by AQA is therefore not recognised by NOCN, and credits awarded by AQA have no formal equivalence with those awarded by Open College Networks.

60 The AVA has a standard requirement of 18 credits at level 3 for the award of an Access to HE certificate, but there appears to be no maximum number of credits that may be awarded, with one course offering up to 30. Although there appeared to be a common understanding that the Access certificate should be awarded on the basis of the achievement of 18 credits, the availability of additional credits was offered as evidence of the opportunity for students to demonstrate a higher level of achievement and, indeed, as evidence of a higher standard of course demands. Some practitioners whom the review team met referred to the requirement for additional credits for admission to some higher education courses which are in high demand. However, the AVA's Comparative Review of AQA Validated Access Courses (see paragraph 57, above), notes that 'some HE institutions ask for more credits [for] particularly popular courses…but the variation did not seem to be related to that issue'. The team considered that closer liaison with higher education admissions tutors would assist in establishing the currency of the Access to HE certificate as the basis for appropriate offers to students, and may serve to increase the understanding that volume of credit did not equate with standard of achievement. It appeared that there was some misunderstanding about this matter among Access practitioners, and the review team would recommend that this be addressed by the AVA as a staff development issue.

61 The review team was concerned to note some inconsistency in the way the AVA's guidelines about credit were applied, and indications that they were not firmly established throughout the AVA. The AVA's Comparative Review of AQA Validated Access Courses twice identifies the anomaly of the same unit carrying level 2 credits on one pathway and level 3 credits on another in the same college. The team considers that the QAC's recommendation that guidance should be provided as to what should be considered to be level 3 should be actioned as a matter of urgency. More generally, the team considered that it was important that the AVA should establish a system whereby units relating to the same or similar subject areas could be reviewed by subject specialists to ensure the consistency of level, credit value and precision of assessment criteria. The team strongly recommends that formal arrangements be established on a subject by subject basis to enable standard-setting policies and practices to be developed.

Assessment, moderation and award of certificates Policies/guidance on student assessment methods

62 In accordance with the published procedures, all colleges use a mixture of course-work assessment and work assessed in timed conditions. The Validation Procedures stipulate that formal written examinations or time-constrained tests on an Access course 'should normally carry approximately 40 per cent of the total marks for the scheme of assessment'. It was not clear to the review team how this reference to a percentage of total marks would be applied within a credit-based scheme of assessment. Although the Account notes that 'guidance will obviously need to be reviewed in the light of the decision that all new programmes be written in a unitised and credit-based format', the team noted that the wording quoted above was unchanged in the revised Handbook on Validation Procedures. The Account's statement that 'assessment is usually endweighted within courses' also did not appear to sit comfortably with the usual principles of modularised, credit-based assessment.

63 Although no grades appear on students' certificates, differentiated student performance is recorded internally by providers according to a diverse range of methods, with the award of 'merits' and 'distinctions', grades, percentages, credit transcripts and combinations of these being used by different providers. The information resulting from these practices is commonly made available to higher education institutions considering the admission of Access students from AQA-recognised courses. The AVA's agreed minimum requirement of 18 credits at level 3 for the award of the Access to HE certificate does not appear to be widely used as the basis of offers made to students, and admissions tutors who regularly admit students from AQA-recognised Access courses whom the review team met all referred to particular grades being required for admission to higher education courses. Where there were long-standing links with colleges, and admissions tutors were familiar with the course, they were able to make offers to students based on their familiarity with the standards of individual courses. However, the AVA has no policy or guidance in relation to methods of recording differentiated achievement, or the provision of this information externally, and the review team concluded that it would be difficult for receiving institutions using this information to be confident of the equivalence of the levels of student achievement.

64 In addition, discussions with course coordinators suggested that practitioners held different views about the status of the Access certificate itself and what it represented. The team considers that the AVA should develop explicit guidance for providers to enable them to address these issues and to ensure that those who are responsible for the admission of Access students to higher education are provided with information which enables them to make confident and well-founded judgements about the suitability of applicants for particular courses.

Moderators and external examiners: appointment and training

65 Moderators are appointed by the AVA to hold overall responsibility for the assurance of quality and standards on Access courses. One moderator is appointed to each Access provider, irrespective of the number of separate programmes, pathways or learners involved. In view of the difference in the scale of Access provision delivered by different colleges, the review team would suggest that the AVA considers whether its current practices ensure that all programmes can be given the same level of attention, and would support the recommendation made in the Comparative Review of AQA Validated Access Courses that 'AQA should consider whether more than one AQA Moderator would be appropriate for large courses'.

66 The AVA has recently considered how it can best safeguard the quality of moderation when a new moderator appointment is made, and the review team commends the decision to allocate an experienced mentor to each newly appointed moderator, and for that mentor to accompany the new moderator on his or her early visits. The team also considered that the revised specification for moderator appointments, which allows the AVA to review an appointment after the first year, provides an improved model for moderators' period of office. The team looks forward to the early implementation of the AMG's decision of May 2000 regularly to review moderator appointments to ensure that no moderator stays with the same provider for more than four years.

67 External examiners also make an important contribution to the AVA's quality assurance process through the verification of standards of student achievement. In contrast to moderators, they are selected and appointed by providing institutions. The Validation Procedures stipulates that an external examiner 'may be responsible either for a whole Access course or programme or for one course within a wideranging Access programme'. The general practice seems to be that an external examiner is appointed to each main subject area or pathway, although this is not consistently the case. Following the receipt of external examiners' CVs, the AVA does confirm appointments, but formally it is the providing institution that is the external examiner's employer, and it is therefore to the providing institution that the external examiner is finally accountable. Some providers have developed very long-established arrangements with external examiners, and whether there is a formal contract or not, and the particular details of the contract or letter of understanding provided to the external examiner, varies according to providers' individual practices.

68 There is also a distinction between moderators and external examiners in terms of training requirements. While new moderators are required to attend a briefing meeting, the AVA does not provide any training explicitly for external examiners. Expectations and experience varied among the external examiners whom the team met: one examiner had attended a moderator training day which she found useful, for which the AVA had been responsible; another external examiner had received no training at all, but had considered that it should be the responsibility of the college who appointed and paid her.

69 The role of the external examiner is regarded as central to the AVA's quality assurance process, and practitioners met by the team particularly valued the opportunity provided by the external examiner system for close and expert subject-specific attention to be given to standards of academic achievement. In view of the centrality of this role, the team had concerns about the fact that, as external examiners were appointments of the provider, albeit 'approved by the AVA', they were not ultimately accountable to the body which carried responsibility for the award of the Access to HE certificate. The team concluded that, if external examiners were to continue to be the means by which standards of student achievement were to be assured by the AVA, it would be necessary for external examiners to be accountable to the AVA, rather than to the provider, and for this to be achieved it would be necessary for external examiners to be appointed, trained and paid by the AVA.

Moderators and external examiners: roles and responsibilities

70 The moderator has a comprehensive role in relation to monitoring the quality of the course and ensuring that it conforms to the description of the course as it was validated. The duties of the moderator are clearly identified in the Validation Procedures, and the new Handbook on Validation Procedures also provides a checklist of specific matters to be covered at each visit.

71 In addition to those duties relating to monitoring the structure and progress of the course, the checklist makes clear that moderators also have a duty to 'inspect samples of course material and students' course work or assignments' and to monitor 'written and course work from as many modules as possible'. The Account explains, however, that 'Moderators do not look in detail at student work. Their role is to check that the colleges' procedures are being applied correctly and that the work complies with the specification'. This role is explained in more detail in the Handbook on Validation Procedures, which explains that moderators have responsibility for 'ensuring that the procedures for assessing individual students' performance on the course and the standards achieved are in accord with those agreed when the course was validated'. The review team sought to clarify the distinction between this role and that of external examiners, who have a responsibility for monitoring 'the standards of any course work undertaken by students during the course which contributes to the final assessment'. It was not clear to the review team how the external examiner was able to perform this role without reference to 'the standards…agreed when the course was validated'. These separate statements suggest that, unless external examiners perform their role with reference to standards other than those agreed when the course was validated, there is some duplication of activity. However, in practice, it appeared that in most cases the moderator's specific role in this regard was delegated to the external examiner, and the team heard that, in some instances, moderators did not actually see any examples of student work.

72 The review team considered that the current model of moderation and external examination left room for some uncertainty or ambiguity as to where final authority for the verification of standards of student achievement lay. The AVA's documentation suggests this responsibility lies with the moderator, who is responsible for the award of Access certificates and who has a responsibility to moderate the external examination on the course. However, given that it is common for external examiners to make only one visit to view student work, and that this occurs at the end of the course immediately prior to the examination board meeting, the opportunity for the moderator to identify any weaknesses in external examining is extremely limited. In effect, the responsibility for verifying standards of student achievement lies with the external examiner. The ultimate authority of the external examiners is also suggested in their power to 'arbitrate over the final decision on students' overall Access course results, and qualifications for the award of the Access course certificates'. In this context, the lack of external examiners' accountability to the AVA was of particular concern.

73 While a dual moderation/external examining system can provide a sound basis for the assurance of quality and standards on Access courses, its effective operation demands clarity about the division of responsibilities and articulation between the two parts of the system. The review team concluded that, in this instance, roles and responsibilities in relation to standards of student achievement were not sufficiently clearly differentiated, nor the nature of the articulation between the two systems sufficiently clearly specified, to provide that assurance. The team therefore concluded that the AVA should review the system for moderating and external examining and establish with absolute clarity where responsibility for verifying standards of student achievement lay, how that verification should take place, and consider modifications to the roles of moderators and/or external examiners accordingly.

74 External examiners are also asked to ensure that 'performance of students is assessed appropriately, fairly and consistently (both within a course, from year to year and in comparison with Access courses elsewhere)'. Although the documentation acknowledges that 'this may require contact and consultations with other External Examiners', the mechanisms through which this might happen, or whose responsibility it might be to ensure that it does happen, are not identified. Discussions with those currently acting in this role did not reveal any systematic approach to ensure that such comparison would take place, and the team would recommend that the AVA considers how it might best be achieved.

Moderators and external examiners: reporting

75 Moderators receive information at various times and through a variety of means. They are required to report comprehensively about a range of matters, including college resources, teaching and assessment, appropriateness of college quality assurance systems, changes to course content and the scheme of assessment, and the review team was satisfied that the procedures followed allowed them to report in detail on the continuing fitness for purpose of Access courses.

76 According to the Validation Procedures, moderators receive reports from external examiners prior to submitting their own reports to the AVA, accompanied by the external examiner(s) reports. However, this has not happened consistently, and the QAC has had to remind colleges to forward examiners' reports to moderators. Copies of moderators' reports should also be supplied to the provider and external examiners. The review team was concerned to learn that none of the external examiners whom they met had ever received a copy of a moderator's report, although the AVA's procedures clearly indicated that this was one part of the monitoring process.

77 Moderators' reports are considered at the autumn meeting of the QAC, and minutes of the Committee seen by the review team confirmed that this exercise was undertaken thoroughly. A letter is then sent to each providing institution, summarising the report, and asking the institution to respond to any issues raised and to discuss them with the moderator at his or her next visit. This letter is addressed to the Access course coordinator. The review team recommends that the AVA adopts the system expected by college senior managers and used by other awarding bodies, and addresses all reports about quality matters directly to the head of the providing institution from whom it can be appropriately delegated.

78 The AVA has recently introduced a requirement that an action plan should be drawn up by the provider in response to the moderator's report, although it was not clear how well-embedded this system had become, and the review team heard that, although moderators would expect to see that matters had been addressed, they would not necessarily expect to see a written action plan. There appears to be no documented system to deal with any shortcomings not addressed by providers and the team would recommend that written procedures be developed by the AVA to address this contingency.

Award of the Access to HE certificate

79 The award of Access to HE certificates is confirmed at the final course examination boards, attended by the moderator and external examiners. Oral reports are provided by external examiners, and achievement of all the component units is confirmed at the examination board, and the related documentation is forwarded to the AVA where it is checked as an administrative procedure before the issue of certificates. Although the Comparative Review observes that 'practice varied from the very formal Exam Boards to more informal arrangements' the review team noted that this had been addressed by the AVA for 2001 through the preparation of a sample agenda for use as a model.

80 It is the moderator who, according to the current Validation Procedures, 'must countersign the college's Pass/Results List before it is submitted to AQA after the college meeting at which the awards were made'. However, this does not appear as one of the moderator's responsibilities in the recently revised version of this document, and it is unclear whether the AVA intends a change of procedure for the future. The review team would advise the AVA to consider this document again to ensure that this responsibility is clearly identified. [Subsequent to the review, AQA has informed QAA that this omission was an error which has now been rectified, and that the AVA does not intend a change to this procedure.]

Learner experience, standards and progression

81 Programme validation documents seen by the review team indicated that Access students were represented on formal college committees and that regular evaluation and feedback opportunities existed for them. Although the absence of annual reports means that the AVA does not currently receive information directly from the provider about student feedback, moderators are required to include 'feedback from discussions with students' in their reports. Moderators also report on a number of other matters impacting on the learner experience, such as college facilities and student support.

82 Those committee members, moderators, external examiners and higher education admissions tutors whom the review team met confirmed that, in their experience, Access students were well prepared for higher education. As mentioned previously (see paragraph 47, above), the AVA collects data about former Access students' progress in higher education, including tracking cohorts through to their final degree or diploma awards. Although the data cannot be complete because it is collected from a wide range of higher education institutions, not all of whom may be able to provide this information, the AVA reports in its Account that 'the data collected over many years indicates that the Access route is at least as useful a preparation for higher education as other routes'. The review team commends the thoroughness and high prominence the AVA gives to the monitoring of achievement of its successful Access to HE learners throughout their HE programmes.

Conclusions

83 As an AVA, AQA is distinctive, and in many ways unique. Many of its distinctive qualities derive from its status as a major, national awarding body, and not least among these qualities is its operation on a nationwide basis, rather than the usual model of an AVA which provides services within a specific region or locale. While it offers its services in all parts of the country, the number of Access courses currently recognised is not large, and they are located within a small number of widely dispersed geographical areas. This presents the AVA with some practical difficulties, for example in ensuring attendance at development events for its participating providers, and the problems faced by all AVAs relating to the difficulty of securing staff release for such purposes are exacerbated where the distances and time involved in attending such events are greater. Apart from these practical difficulties, the national perspective of the AVA presents a more fundamental difficulty for the AVA in its role in relation to the enhancement of Access to HE provision and in facilitating effective cross-sector collaboration between further and higher education. Without a clearly identified region as the focus for its work, it is unable to develop a strategic approach to develop the work of the AVA in relation to regional issues of targeting and widening participation in higher education. The absence of a regional focus also makes it more difficult for the AVA to develop and build on relationships between local course providers and receivers of students in higher education to identify needs, demands and opportunities for the further development of the AVA's work and to build understanding about the nature and value of the Access to HE award.

84 The AVA is also distinctive in terms of its structure, in which there is an unusual separation of most of the operational and governance arrangements which relate to the exercising of its responsibilities as a licensed AVA, from the larger body which makes decisions relating to budgeting and which formally holds the licence. The concept of membership used by AQA refers not to membership of the licensed body itself, but to the 'Access Partnership' which exists within it. This partnership, however, has no constitutional place or powers within AQA, and the AVA is not owned by or accountable to its 'members'. This raises questions about the ability of those who provide Access courses and those who receive Access students in higher education to influence and direct the decision-making of the body which holds the AVA licence, or to be involved in the development of the AVA through the identification and implementation of shared aims and objectives for Access to HE which are embedded within the structures of the licensed body. The licensed body does not itself currently have aims and objectives which relate specifically to Access to HE, nor is there a formally articulated or continuing strategic plan for action, development or enhancement in relation to its responsibilities as a licensed AVA. This situation places AQA's Access work in a potentially vulnerable position. If the AQA Council is to continue to be the ultimate locus of responsibility for the AVA licence, there will need to be clear links between the Council and the specific committee structures and management of the AVA. Although there has been some consideration about the composition and reporting lines for the Access Management Group, plans for the future development of the AVA are associated essentially with operational matters.

85 However, these structures bring some benefits to the AVA in terms of AQA's experience and resources, and enable the AVA to provide an efficient administrative service to provider colleges. As Access work represents a very small proportion of AQA's work and does not currently cover its own costs, the AVA's administration is able to draw on the support of the larger resources of AQA, without which the AVA, as presently funded, would not be viable. If it were to be self-financing, fees would need to be arranged on a different basis, and the possibility of autonomy in decision-making, including budgeting, would have to be considered.

86 Although some modifications are required in parts of the moderation and external examining system, the AVA's quality assurance arrangements are broadly adequate. Both the Access Management Group and the Quality Assurance Committee conduct their business relating to quality assurance properly, and with appropriate attention to detail. There is, however, an absence of equivalent emphasis on the AVA's quality enhancement functions. Until very recently, there had been little systematic reflection or evaluation of the AVA's activity as a whole. While the focus on data collection in the AVA is strong on HE progression, there are marked weaknesses in relation to the evaluation of the Access courses themselves. In particular, there is very little evidence of analysis of data in relation to matters such as success in targeting, retention and achievement which might lead to the identification of areas of good practice for development and dissemination, or particular areas of weakness which the AVA would need to address. Inevitably, the AVA's ability to engage in meaningful analysis of statistical data and take forward recommendations on the basis of conclusions reached depends on the completeness and robustness of the data which it receives. Providers have apparently not yet provided the AVA with the quality of data it would require to undertake this exercise effectively, and the AVA has yet to establish clearly that the provision of such data is an absolute requirement. Paradoxically, for a major awarding body, the AVA seems somewhat uncomfortable with exercising its authority as a licensed AVA in relation to Access course providers, and uncertain of its obligations to meet the requirements of the Access Recognition Scheme.

87 There is no doubt about the hard work and commitment of the individuals who serve on the Access Management Group and Quality Assurance Committee; whether they would be able to give the time and to take on a wider remit to include the necessary development of the AVA's work is more doubtful. While the AVA's committees should clearly have oversight of such activities, it is likely that the related work involved would need to be designated to an officer. Current staffing allocations for Access within the organisation do not offer this capacity, however, and staffing for the AVA is currently insufficient to do more than provide an administrative service. While plans to give responsibility for the administration of the AVA's work by a team rather than a single individual will, no doubt, produce benefits, the likely appointment of a lead officer with little or no expertise in Access to HE matters does not reflect an organisational commitment to this area of work, and does not augur well for any development of the AVA's role.

88 In order fully to exercise its responsibilities as a licensed AVA, AQA will need first to review the basis on which it undertakes this activity, and this will require more than a consideration of the operation and administration of the work. The organisation will need to consider whether its current structures and procedures allow it to fulfil its AVA role, and meet its obligations in relation to Access to HE in as rigorous a manner as its activities in relation to other national awards.

Commendations

89 The review team commends AQA for:

i the potential value to its quality assurance arrangements of the work of the Quality Audit Unit;
ii the value of the annual AQA 'Partnership' Meeting as a method of communication with its providers;
iii its undertaking of a comparative study of Access to HE courses;
iv its tracking of students in higher education.

Recommendationto the ARLC

90 The review team recommends that AQA be granted a provisional renewal of its AVA licence, with conditions to be met by the dates specified below, and revisit in summer 2002.

Conditions

91 Subject to the approval of the ARLC, AQA's AVA licence is renewed on condition that AQA:

i clarifies the role of the AQA Council as the ultimate locus of authority for the AVA and ensures that AQA's governance structures and constitutional arrangements enable it to exercise its responsibilities in this regard, with particular reference to:

the adoption of clear aims and objectives, with specific targets relating to Access to HE;
the introduction of a regular planning process to monitor success in the achievement of AVA targets and the development of new objectives and targets;
the development of a more proactive and purposeful approach in the context of targeting policies and strategies;

a consideration of staffing to ensure that:

the appointment of a lead officer for the AVA involves some measure of confidence in his or her expertise in Access to HE matters;
levels of staffing are amended to allow the provision of support and development functions beyond the basic administrative;
a greater spread and depth of officer support is provided to ensure the effective operation of the AVA;
the provision and promotion of development activities for the AVA's members, specifically those relating to consistency of the Access to HE award;
the appointment, training and payment of external examiners;

ii undertakes a review of all Access to HE programmes recognised by the AVA, with particular reference to credit levels and values, to ensure consistency and adequacy of academic rigour and outcome standards;

iii establishes a monitoring mechanism through which the AVA can ensure that accurate information on the availability and status of courses is offered by its providers;

iv develops clear guidance for providers in relation to methods of recording and providing information about differentiated levels of student achievement, to assure that any information relating to internal marking schemes which is provided to HEIs takes account of the award of the Access certificate itself and has consistent reference points across the AVA;

v clarifies for providers their obligations to provide statistics to the AVA, as required by QAA, and ensures compliance with the requirements;

vi implements the decision to review moderator appointments to ensure that no moderator stays with the same provider for more than four years;

vii develops the process for the approval of changes to courses to ensure that the AVA is in full control of course development, and provides clear guidelines specifying, in particular, the nature and extent of authority granted to moderators and/or external examiners;

viii reviews the system for moderating and external examining, and establishes with absolute clarity where responsibility for verifying standards of student achievement lies, and how that verification should take place, and modifies the roles of moderators and external examiners as necessary;

ix reviews and develops the current Action Plan, to include and prioritise all of the above in addition to the current specified activities, and to identify detailed specific actions, named responsibilities, and carefully considered timescales. (Condition ix to be met by 1 November 2001; evidence that all other conditions have been addressed by 1 April 2002, with full implementation by the time of the revisit in summer 2002)

92 Provisional confirmation of licence allows the AVA to continue to operate under licence but makes provision for the immediate withdrawal of the licence if the conditions are not met in the time allowed and to the satisfaction of QAA. Were it to be necessary for the Agency to withdraw the licence, the AVA would need to put in place procedures for the transfer of registered students to an alternative licensed AVA.

Recommendations

93 The review team recommends that AQA:

i seeks ways in which it might grant all of its members the right to become involved in the AVA's activities and future planning;

ii reflects on how the work of the Quality Audit Unit could be integrated with the current quality assurance systems of the AVA;

iii analyses statistics received from AVAs on the recruitment and progress of students on Access to HE courses, and introduces the consideration of statistical data as a part of the AVA's planning processes;

iv includes in the revalidation process a greater level of review of the strengths and weaknesses of the recognised course;

v develops closer liaison with admissions tutors in higher education to increase understanding of the Access to HE certificate and how it might be used as the basis of appropriate offers to students;

vi ensures that there are rigorous systems for assuring that final submissions have met all conditions set by the panel, prior to the AVA advising the college of the approval of the course;

vii directs all communications from the AVA to providers about moderation outcomes and regulatory matters to the heads of providing institutions;

viii develops written procedures describing the AVA's response should a provider fail to respond adequately to a moderator's report;

ix provides staff development for Access practitioners to ensure that the principles and practice of a credit-based system of awards are thoroughly understood and implemented throughout the AVA;

x establishes formal arrangements on a subject by subject basis to enable standard-setting policies and practices to be developed;

xi develops mechanisms to encourage contact and collaboration between external examiners for the purpose of ensuring consistency in the assessment of students' performance.

Assessment and Qualifications Alliance (AQA) August 2001

Contents