Open College Network (South Yorkshire and Humber Region)
November 2003

Foreword

1 The Quality Assurance Agency for Higher Education (QAA) is responsible to the Department for Education and Skills for the recognition of Access to Higher Education courses. QAA exercises this responsibility through a national network of authorised validating agencies (AVAs), which are licensed by QAA to recognise individual Access to HE programmes, and to award Access to HE certificates to students. The AVAs are responsible for implementing quality assurance arrangements in relation to the quality of Access to HE provision and the standards of student achievement.

2 QAA has developed a scheme for the licensing and review of the AVAs, the principles and processes of which are described in the QAA Recognition Scheme for Access to Higher Education in England, Wales and Northern Ireland. The Recognition Scheme is regulated and administered by the Access Recognition and Licensing Committee (ARLC), a committee of the QAA Board of Directors. The ARLC is responsible for overseeing the process of AVA review through which AVAs are periodically relicensed. The criteria applied by the ARLC and by review teams operating on the Committee's behalf, in reaching judgements about whether and under what terms an AVA's licence should be renewed, are provided within the Recognition Scheme documentation. These criteria are grouped under the seven principles that provide the main section headings of this report.

3 Following the review of an AVA, a member of the review team presents the team's report to the ARLC. The Committee then makes one of five decisions:

i unconditional renewal of licence for a specified period;

ii conditional renewal of licence with conditions to be met by specified date;

iii provisional renewal of licence with conditions to be met and further review visit by specified date;

iv withdrawal of licence for operation as an AVA;

v temporary renewal of licence with request for further information by specified date (decision suspended).

4 This is a report of a review of the AVA function of the Open College Network (South Yorkshire and Humber Region) (OCN (SYHR)) undertaken by QAA. The Agency is grateful to OCN (SYHR) and to those who participated in the review for the willing cooperation provided to the review team.

The review process

5 The review was conducted in accordance with the process detailed in the QAA Recognition Scheme for Access to Higher Education in England, Wales and Northern Ireland. The preparation for the review included an initial meeting between OCN (SYHR) representatives and the QAA Assistant Director to discuss the requirements for the Analytical Account (the Account) and the process of the review; the preparation and submission by OCN (SYHR) of its Account, together with a selection of supporting documentation; a meeting of the review team to discuss the Account and supporting documentation and to establish a draft programme for the review visit; and negotiations between QAA and OCN (SYHR) to finalise the programme and other arrangements for the review visit.

6 The review visit took place on 12 and 13 November 2003. The visit to OCN (SYHR) consisted principally of meetings with representatives of OCN (SYHR), including; members of the Executive Committee, Accreditation Quality Committee, and Access Advisory Group; senior managers from providing institutions; moderators for Access to HE programmes; representatives from HE; and former Access students now studying in HE.

7 The review team consisted of Mr Steve Babbidge, Director, South of England Open College Network and Ms Jill Ward, Principal, Wedgwood Memorial College. The review was coordinated for QAA by Ms Kath Dentith, Assistant Director (Access).

The AVA context

8 Established as South Yorkshire Open College Federation in 1987 and granted an AVA licence in 1989, the organisation changed its name to Open College Network (South Yorkshire and Humberside) in 1997. This change reflected its increased operating area, following a merger with Humberside Access Validating Agency, with consequent responsibility for programmes recognised by Humberside Open College Federation.

9 The AVA currently operates in South Yorkshire, Humberside, North Nottinghamshire and North Derbyshire. The organisation has 257 members, 17 of which are providers of Access programmes, most recent new members of the AVA coming from voluntary, community and work-based organisations. The AVA views this membership base as providing 'a significant strength in relation to the development of its Access activities'. The Account acknowledges that Access to HE 'has historically provided a relatively small proportion of the OCN's total number of learners', but states that 'it has nevertheless formed a very significant element of the totality of the overall service provided', and notes the 'emblematic nature' and 'strategic importance' of Access in its overall portfolio.

10 The report on the review undertaken by the Higher Education Quality Council (HEQC) in 1996 reported that 43 Access courses were running in 14 organisations: this compares with 42 programmes running (53 available) in the 17 Access-providing organisations recorded in the AVA's statistical return to QAA for 2001-02. While this may suggest a static picture, the HEQC report also recorded 1,166 students registered on Access courses and 594 students awarded Access certificates in 1994-95, approximately half the number recorded in 2001-02 (see paragraph 12, below). The figures also mask the inclusion of new providers and mergers of others, with a general trend over this period towards larger modular frameworks, rather than discrete courses, for those providers that are large enough to sustain them.

11 The Account reports that numbers of Access learners and levels of activity have, in general, 'remained stable during the last three years' and that, in view of the context of change and uncertainty in which providers are working, this stability is 'a substantial achievement'.

AVA statistics 2001-02 (as provided by the AVA in its annual report to QAA for 2001-02)

12 Providers offering Access to HE programmes 20

Programmes available 53

Programmes running 42

Access learner registrations 2,361

Access to HE certificates awarded 1,094

Principle 1

The organisation has a structure which is based on a partnership of members, including institutions which provide Access to HE programmes and institutions of higher education

13 The AVA is a large partnership of members, with a wide range of organisational types in membership across the statutory and non-statutory sectors. Members include providers of Access programmes, most of which are FE colleges, and three higher education institutions (HEIs) (Sheffield Hallam University, the University of Hull and the University of Sheffield). Providers also include an adult community service and a primary care trust. Membership is sufficiently broad and cross-sectoral to ensure, in principle, the AVA's operational viability, and to secure informed decision-making about Access.

14 The AVA admits organisations into membership through well-established procedures that do not discriminate between Access and other providers, and membership is coterminous with the AVA's role as an OCN, which accounts for the vast majority of the organisation's activity. The principles of membership are set out in the Constitution and restated in the AVA's Procedures Manual. Central services staff assess each application on behalf of the Director, who presents a summary to the Executive Committee in order for formal approval to be granted. Decisions are reached on the basis of an organisation's willingness to endorse the AVA's mission, aims, rules and regulations. Procedures are also in place for the suspension and withdrawal of membership, largely on the basis of a contravention of the principles of membership.

15 The AVA operates as a full cost business unit within Sheffield Hallam University, and there are well-established understandings with regard to the services provided to the AVA and its financial contribution to the University. There is a draft Memorandum of Agreement, under discussion at the time of the review, which is seeking to codify a number of these well-established understandings. The Memorandum provides for the following services from the University: human resources services, including employment of the AVA's core staff and moderators; financial services (including the separate identification of the AVA's accounts); office accommodation and services; meeting rooms and services; communications; and insurance and liability cover for all staff. The Memorandum scrutinised by the review team, which had been endorsed by the AVA's Executive Committee was, at the time of writing, with the University for further consideration pending the outcome of the review.

16 The review team concluded that the AVA did not meet the licensing criteria expressed under Principle 1 because the formal agreement between the AVA and Sheffield Hallam University raised two particular concerns in relation to the appropriate 'separation of responsibilities, liabilities and authority' which is required under criterion 3 of Principle 1. As a resolution to this issue will require consideration of the AVA's governance structures, this matter is dealt with in detail in the next section.

Principle 2

The organisation has governance structures which allow it to discharge its AVA responsibilities securely

17 The AVA has a formal constitution, which sets out its mission, ethos, definition of membership, staffing, committees, standing orders and licensed status from the National Open College Network. The AVA's legal identity is not defined within its constitution, although the description within the constitution of its relationship with Sheffield Hallam University (see below, paragraphs 27 and 30) indicates that the organisation is not incorporated. It was clear to the review team that there was an intention by AVA staff, committee members and the University to ensure that the position taken in the Account, which noted that 'the AVA's ability to operate independently has been custom and practice for over a decade' is reflected in its governance structures and relationship with the University.

18 The AVA is accountable through its Executive to its members, whose representatives comprise a Council. The Council has responsibility for the AVA's mission, constitution, annual report and for electing the Executive. The relationship between the Executive and Council is one where the latter notes the achievements of the OCN over the year, as manifested in its annual report, and elects nominees to the Executive in response to vacancies. There is no mechanism for the retirement or rotation of Executive Committee members, and replacements are sought in response to a constitutional obligation to maintain 13 places on the following basis: six from the FE sector; four from other organisations not FE or HE; one HEI; Sheffield Hallam University; and one staff member. The Constitution allows for three co-options and the attendance of the Director in an ex-officio capacity. The Director is accountable to the Executive, with the rest of the AVA staff accountable to him.

19 The constitutional arrangements which define the composition of the Executive allow for a broad sense of representation and accountability to be achieved across the AVA's membership. Most of the places on the Executive are allocated through a process of nomination and election, although the review team noted that vacancies had been uncontested in recent years. Two places are reserved for named organisations: the OCN, through a 'staff member' place and Sheffield Hallam University. This reserved place for Sheffield Hallam University is a product of the AVA's status as a part of the University, although it is not defined as such in the Constitution. While the team saw no evidence to suggest that the University sought at any time to exert undue influence over the decisions made by the Executive, and, indeed, the recently drafted Memorandum of Agreement (see paragraph 15, above) confirmed its desire not to, the team considered that its special status as a named organisation on the Executive conferred upon it a right which was not given to any of the organisation's other members. In particular, this position gives Sheffield Hallam University preferred status among the AVA's HE members: whereas other HEIs have a single place through which to represent their, perhaps different, views and interests, Sheffield Hallam University has guaranteed representation in a way which is not consistent with the requirement to ensure that the AVA is not exposed to undue influence from any one member. The combined role of employer, insurer and landlord of the AVA is one which could be held by any of the AVA's members and the current arrangement clearly works satisfactorily for both parties. The team noted that there were clearly defined formal agreements, allowing the host organisation proper mechanisms to protect itself in relation to the AVA's business activities, and that the AVA paid a commercial fee for services provided by the University. Given this context, the team considered that a guaranteed seat for the University on the AVA's Executive was not necessary, and confused the responsibilities, liabilities and authority of the University as the host institution with those relating to its role as a member of the AVA.

20 The Executive appoints the AVA's one other decision-making committee, its Accreditation Quality Committee (AQC), which has delegated responsibilities for maintaining an oversight of the development, monitoring and review of policy, regulations and procedures related to accreditation. The Constitution allows for between eight and 16 individuals to serve on the AQC, selected on the basis of their sectoral backgrounds and experience in managing quality systems. Two of its members are drawn from the Executive (one of whom assumes the Chair), two are elected by the AVA's Moderation Advisory Group, and one is elected by its Access Advisory Group (AAG). Both of the latter groups are constitutionally defined as advisory and as conduits for communication. Specific responsibilities for these advisory groups are determined by the AQC on an annual basis. There is a specific requirement that membership of the AQC will take account of 'the OCN's obligations as an Authorised Validating Agency licensed by the QAA' and that 'membership will reflect the breadth of the OCN Membership in terms of sectors served' but nothing specifically requiring the representation of Access providers or HE receivers, beyond the inclusion of the AAG member.

21 The AAG is the main arena in which matters specific to Access are discussed, providing a focus for debate about current practice and wider developments in Access. Membership of the group is diverse and, although the majority of members do not attend on a regular basis, it is sufficiently cross-sectoral to achieve a balance of views. The AVA's Director also maintains an active role in the AAG and is able to inform the group of relevant local, regional and national issues, and take forward issues raised by the group. Meetings of the group cover a wide range of items, including changes to Access provision and regulation, developmental issues, and HE admissions issues. The role of the AAG is advisory, and the review team saw evidence that issues it raised were fed into the work of the AQC and Executive, as appropriate. One example of this is the circulation to the AAG of the AVA's draft annual report to the QAA and an invitation to send suggestions to the AVA's Director prior to its despatch.

22 Oversight and effective control of financial matters is the specific responsibility of the Executive, subject to the procedures and responsibilities of Sheffield Hallam University. The AVA has access to legal and financial advice through the University. The AVA produces a 'resources report' for each meeting of the Executive and this covers budget monitoring and financial health. There is evidence that, where expenditure and income projections are not met, careful evaluation of the causes by AVA staff and subsequent monitoring by the Executive take place.

23 The strategic direction and policy development of the AVA is the responsibility of the Executive, with the AVA's Director having responsibility for making proposals to it about broad strategic direction and for policy development. The AVA no longer maintains a separate plan for Access, having opted instead for an integrated planning process across its AVA and OCN operations. The Director's regular reports to the Executive do, however, comment specifically on matters relating to Access.

24 Organisational structures and management are the responsibility of the Director with monitoring by the Executive as appropriate. Staffing matters are reported to the Executive on a regular basis and it also approves the overall staffing complement. Significant effort has been put into ensuring that the AVA's staff base can respond to organisational need and there is a considered approach to staff development.

25 According to the Account, the locus of authority for AVA matters resides with the Executive, which maintains an oversight through the Director and its AQC. The authority of the Executive to oversee AVA responsibilities is not, however, explicitly articulated in the Constitution. Responsibility for the quality assurance of Access provision might be inferred from the criteria for membership of the AQC (see paragraph 20, above), but it is not specifically stated as a committee responsibility in the AVA's Constitution. While the committee's primary function in relation to procedures for programme approval, moderation, standardisation, and certification is detailed in a series of more specific responsibilities, none of these requires the committee to be involved in consideration of the outcomes of these procedures: it has no specified responsibility for assuring the quality and standards of the AVA's provision. Although the AQC's agendas ranged across a variety of policy and regulatory issues and matters for report, including receiving the minutes from the AAG, the review team did not see any evidence to confirm that there was a specific constitutional focus on, or actual engagement with, the quality assurance of Access provision.

26 The review team concluded that the AVA did not meet all of the criteria expressed under Principle 2 and that, although the AVA's governance structures were well established and allowed the Executive to maintain an overview of Access matters, they did not give sufficient focus through constitutional obligation to guarantee that the organisation had governance structures which allowed it to discharge its AVA responsibilities securely. A review of the AVA's Constitution and associated documents and processes is therefore required to enable it to: clarify and confirm its legal identity; ensure it is constitutionally protected from the undue influence of any one of its members; specify a clear locus of authority for its AVA responsibilities as distinct from those relating to its role as an OCN; and ensure that responsibilities are clearly specified within the governance structures for the quality assurance of QAA-recognised Access to HE provision.

Principle 3

The organisation is aware of, and in a position to meet, its legal and public obligations

27 The Constitution (revised in 2000) does not state the AVA's legal identity per se, although the Account refers to the AVA as being 'non-incorporated'. The Constitution notes that the AVA is 'a full cost centre of the Sheffield Hallam University'; that staff are employees of the University; and that its finances are managed according to the University's procedures, a position which is reinforced in the statement in the Account that 'The AVA (OCN) is formally part of the Student Services Centre of Sheffield Hallam University (SHU) and operates currently as a business unit within the university.'

28 In the months preceding the review visit, the AVA had been exploring changes to its legal status and constitutional structure through a variety of means. The Account noted that, notwithstanding the 'track record of de facto independence' of the AVA, a decision had been taken by the Executive in May 2003 to 'proceed with a more rigorous exploration of options for acquiring a new or re-confirmed legal status', an exploration which was to involve a 'systematic risk analysis of the advantages and disadvantages of changing its legal status'.

29 The draft Memorandum of Agreement (see paragraph 15, above) with Sheffield Hallam University confirms that 'the University will in no way seek to influence or constrain the power of the OCN to discharge its stated functions outlined in its Constitution, strategic and business plans, beyond its entitlement as an ordinary member'. It also states that the AVA's Director is 'jointly appraised by the Chair of the OCN's Executive Committee and a University Senior Manager' and that 'The University will hold on behalf of the OCN any surpluses accrued, which may be used for the management of financial risk and investment purposes'. The AVA's Executive had not proposed any changes to the Constitution as part of this review of its legal status. The responsibilities, liabilities and authority of each party are set out in the Memorandum of Agreement and also in the Constitution. The review team noted that there was no separation of the authority of the AVA and its status as part of the Student Services Centre at the University with regard to the appointment and formal appraisal of the AVA's Director.

30 While it sets out a range of respective authorities and confirms a variety of services and accountabilities, the draft Memorandum was judged by the review team to offer a codification of the status quo. The AVA continues to be described as 'legally part of the University' with its liabilities covered through the provision of insurance and public liability cover by Sheffield Hallam University. The team noted that implicit in the University's provision of liabilities insurance was a recognition that all assets and liabilities would, in the event of the OCN winding up, fall to it to administer and take responsibility for, as there is no provision in the Constitution for dispersal of assets and liabilities at the point of dissolution. This arrangement is judged to be satisfactory by both parties, as the AVA has undertaken extensive consultation and sought advice about its status from a range of sources.

31 While it is clear that the intentions of the AVA and the University are to ensure the independence of the AVA, the review team could find no constitutional instrument that would provide for equality of representation by members on the AVA's Executive Committee and assure the AVA's essential independence. The team concluded that the AVA's current constitutional basis and relationship with Sheffield Hallam University did not guard against restrictions on the independent decision-making or operation of the AVA. Accordingly, the team concluded that, although the AVA met most of the criteria expressed under Principle 3, there were insufficient constitutional safeguards to ensure the AVA's essential independence, and a review of the AVA's Constitution and associated documents and processes is required to remedy this. In particular, the AVA should amend its Memorandum of Agreement with Sheffield Hallam University to ensure that its independent operation is not restricted with regard to the appointment and appraisal of the Director.

Principle 4

The organisation is able to manage effectively its AVA responsibilities and the structure which supports them

Planning and development

32 As noted above, the AVA has, as a combined OCN and AVA, a well-developed process for strategic and business planning and the subsequent reporting and monitoring of achievements. There is an astute evaluation of the political and policy framework within which the organisation as a whole operates and a robust mechanism to engage stakeholders in setting and monitoring activities undertaken in pursuance of its strategic intentions. Robust risk assessment procedures are built into the process.

33 While the Strategic Priorities Statement 2002-05 (2003 review) does not make any specific reference to Access, it does feature in the AVA's Annual Business Plan 2003-04, through a discrete section within one of the six business objectives: 'To maintain the volume and quality of Access to HE provision, whilst developing new access opportunities'. This objective is supported by four 'actions', two of which relate to expanding provision, although it is not clear that these will necessarily concern QAA-recognised Access provision.

34 The development of a regional approach to Access and of 'new access opportunities' is a significant feature of the AVA's work, and for its involvement in the development of Access. The principal way in which this development has taken place has been through the AVA's engagement with the HEFCE-funded 'Building Pathways Project' (BPP). Access is an important strand in the BPP, through which HEIs and colleges in the region, together with the AVA, have formed a working partnership to 'develop clear and coherent routes into higher education'. The project has been particularly successful in the development of Access pathways in the areas of social care, health and nursing, and it has improved consistency in these areas through the use of common units in Access programmes among participating colleges. The project has provided a model that could be applied more widely to the AVA's recognised Access provision, particularly in relation to the further development of flexible and work-based routes relating to vocational pathways. The original project itself is now at an end, and, as noted in the executive summary of The Building Pathways Project Evaluation Report (October 2003) 'Without the activities of Building Pathways much that has been already achieved may dissipate due to lack of concerted co-ordination and drive'. There are therefore questions for the AVA about how it can take forward these initiatives and ensure that the productive working practices established are maintained under the auspices of Aimhigher: P4P.

35 While the AVA is clearly engaged in development and enhancement activities which relate to Access to HE through its links with BPP, the review team was unable to locate specific objectives in the AVA's planning documentation which indicated a coherent and distinct strategy for the development and enhancement of QAA-recognised Access provision as such. The team concluded that the developmental agenda for Access was being addressed more obviously through the agenda for BPP, than the AVA's own strategic intentions and commitments.

36 The AVA operates a systematic and rigorous approach to the management of its financial affairs. With the backup and support of the University, the AVA is able to project income and expenditure targets realistically and monitor them reliably. The Executive is involved formally in this process and receives clear information at each meeting on the financial health of the AVA. In using the University's finance mechanisms, the AVA is able to demonstrate that it operates standard accounting procedures and is able to produce annual accounts in its annual report to QAA. Although the importance of Access as a source of revenue generation is acknowledged by the AVA, and its business planning model is congruent with a resource allocation model, the review team was unable to identify any specific planned expenditure on Access, no doubt related to the AVA's 'combined' planning model for its OCN and Access portfolio.

37 The levels of staffing and the physical resources available to the organisation are adequate for its function as an AVA. It is situated in a dedicated building which is part of the University's estate and it has access to a range of good quality meeting rooms on campus and within member organisations. There are 21 staff employed by the University on behalf of the AVA, who have access to a wide range of University and AVA-led training opportunities. There are no dedicated 'Access' roles and the entire team is involved in one way or another with the work of the AVA. The AVA encourages a 'cross team' approach to working which facilitates an understanding of the organisation's AVA work. The AVA conducted a major review of its organisational structures during 2000 and has since implemented a three-person management team: the Director and two assistant directors, one with responsibility for accreditation and the other for operations. The management team is involved across a range of AVA work, with the Director taking a lead role in terms of engagement with external agencies and the AVA's own consultative group, the AAG. All other staff are directly accountable to one of the two assistant directors, who in turn are accountable to the Director. As noted earlier, the Director is accountable to the Executive, but is appraised jointly by the Executive and the University.

38 The AVA provides good quality services to its members as judged by a 'stakeholder survey' independently produced by the Responsive College Unit. This survey notes that 'The reputation of OCN was said to be reliable, responsive, flexible, friendly, local, professional, sensitive to the needs of fragile learners. OCN programmes were said to be accepted as a valuable progression route to higher education'. The AVA employs a variety of means to communicate its AVA activities to its members and has, in the past, produced an HE admissions guide setting out detailed course-specific information of use to admissions tutors. This is currently under review as part of the AVA's emergent ICT-driven communications strategy.

39 There are well-established and documented operational procedures relating to the specific obligations of the organisation's AVA work. While some are integrated in the interest of administrative efficiency, others are quite properly separated, as in the development of Access programmes. The AVA has a comprehensive procedures manual (under review at the time of the review visit) which provides detailed operational guidance for AVA staff, including, for example, procedures relating to applications for membership; disaster recovery; creation and distribution of circulars; as well as procedures to be followed in relation to approvals, moderation and certification. There are clear procedures in place to enable complaints, grievances and appeals to be received, considered and equitably resolved. Documented procedures and operational processes are consistent with the AVA's mission and ethos, and there is a comprehensive and clear equal opportunities policy in place.

40 The AVA has invested considerable sums in the development and implementation of a bespoke database. Data concerning Access provision is collected in composite form from centres' management information systems and is used by the AVA to produce 'centre profiles' to enable it to monitor programme activity levels throughout the year. The AVA has taken action to increase the completeness and validity of its AVA data collection through charging college MIS staff with the responsibility for providing learner profile and achievement data and requiring Access coordinators to provide progression data. This has enabled the AVA to provide QAA with more complete data sets and allowed it to construct a useful datum point for subsequent comparative and trend analysis. Although the review team was not able to identify any specific examples of the data being used strategically at this stage, it noted the achievements of the AVA in this often problematic area.

41 The criteria expressed under Principle 4 were considered to have been met, with the exception of the requirement under criterion 6 that staff should be fully accountable to the AVA as it relates to the position of the Director's appraisal, discussed under the previous section.

Principle 5

The organisation is able to assure the quality and fitness for purpose of Access to HE programmes at the point at which they are granted formal recognition

Programme development

42 Guidance notes are issued to providers who wish to develop programmes, including Access-specific guidance. Standard systems and procedures ensure that there is consistency of process leading to Access programme submissions. Providers are familiar and confident with the processes and procedures leading to validation, and follow-up advice and direction is deemed to be appropriate and valued by providers. The AVA was still exploring the means by which it might most successfully incorporate its Access guidance for the development of programmes into its national OCN submission processes but, at the time of the review visit, it had not been able to resolve the difficulties presented by this.

43 The AVA has established clear criteria and documentation defining the membership and composition of validation panels to ensure appropriate curriculum expertise and representation from providers, moderators, and receiving institutions. The accreditation team has oversight of this and ensures that criteria are adhered to. HEIs are involved in the programme development process and as members of recognition panels.

Final approval of recognised Access to HE programmes

44 While the AQC has responsibility for monitoring quality assurance procedures, and the Executive Committee oversees the AQC's activity, it was clear from evidence provided for the review and confirmed in review meetings that responsibility for the approval of Access programme submissions rested with the AVA's recognition panels, and that compliance with conditions was monitored by officers.

45 While the review team was encouraged to note the statement in the Account that the AVA is 'currently reviewing processes for the final approval of programmes', and that it has identified 'clearer lines of responsibility and processes for the final approval of programmes' as an area for further development, the team would wish to emphasise that the current position does not meet the licensing criterion that the recognition of Access programmes should be made with the full authority and approval of the AVA.

Requirements for the award of the QAA-recognised Access to HE certificate

46 The AVA's Access to HE Certificate and Planned Programmes of Study Guidance Notes explains that 'the OCN will award an Access to HE Certificate to any learner...who successfully completes a balanced and coherent programme of study and achieves a minimum of 16 credits, at least 12 of which are at Level Three. The OCN believes this will effectively indicate a learner's general preparedness for HE study'. The policy to award the Access to HE certificate on the achievement of 16 credits is recognised across the AVA, and evidence available for the review indicated that the policy was implemented consistently by officers, in accordance with the AVA's procedures.

47 The AVA's approach to the award of the Access certificate is confirmed in the introduction to the AVA's guide for admissions tutors (see paragraph 38, above), published in 2001. However, entries in the guide providing course-specific information (prepared by individual providers and collated and published by the OCN) indicate some uncertainty about the status and purpose of the minimum national credit requirement and its relationship to the award of the Access certificate. Although entries in the guide commonly stated that 16 credits was the minimum requirement for the award of the Access certificate, a number stated that more credits would be expected, with one, for example, explaining that 'successful completion of the course, ie where students can be recommended to admissions tutors at relevant HE institutions, has been set at 20 credits at Level 3 and four credits at Level 2', and another specifying that 'Learners are normally expected to achieve more than 16 credits (on average around 20) to reflect the acquisition of the broad range of skills commonly seen as necessary in Higher Education'. Variations of this kind are explained in the guide's introductory section 'Access and Entry to Higher Education', in which the AVA states that 'a negotiated planned programme...may require the student to achieve in excess of the minimum sixteen credits to enable them to benefit from their programme of study in Higher Education'. The review team also saw course documentation that indicated, for example, that students would not be considered to be prepared for progression to HE until they had achieved 24 credits, despite the Access certificate being awarded at 16 credits. Individual programmes may thus have credit targets at which 'preparedness for HE' is defined, which may exceed the 16 credits at which the QAA-recognised Access certificate is awarded. Such a position calls into question what is represented by the Access certificate which is awarded at 16 credits on these programmes. The team sought to explore this matter with those involved in the AVA's work and concluded that the uncertain position indicated in the 2001 guide had not been clarified by the AVA and that it remained an issue in 2003.

48 The situation described above results, essentially, from the AVA's response to what was described, both by members of the AAG and AVA officers, as an 'on-going issue', this being the requirement by some HE programmes for a higher credit achievement than the 16 credits which is the AVA's current formal target for the award of the Access certificate. This has been a matter of debate within the AVA, as providers have sought ways to address the differing entry requirements of HE receivers, and there seems to have been some confusion about the need, on the one hand, to identify a single, consistent threshold level of achievement (represented by the award of the Access certificate) and, on the other, to provide students with the means to meet particular entry requirements through the achievement of additional credits. Because those additional available credits may be regarded as a part of the 'programme', the point of their award is regarded in some places as 'programme completion', thereby separating the concept of programme completion from the award of the Access certificate. As members of the AAG noted in discussion with the review team, credit inflation is also a potential consequence of this approach, as providers continue to try to respond in this way to HEIs' demands for increased credit achievements.

49 Practitioners' confirmation that students who had achieved sufficient credits for the AVA's award of the Access certificate had, on occasion, withdrawn from the programme before achieving the credits that were required for 'programme completion' highlights another area of potential difficulty if programme completion and achievement of the required credits for the award of the Access certificate are separately defined and the latter does not represent 'preparedness for higher education'.

50 The review team considered the AVA's current way of dealing with the difficulties in this area to be unacceptable. By permitting variation between providers' published credit expectations, as they relate to 'preparedness for HE', without an established rationale for this variation, inconsistencies in the achievement required on different programmes leading to the same award can occur, and the achievement required for the award of a national certificate and that required for the successful completion on a local programme are separately defined where the two should be coterminous. The team judged that this position potentially undermined the sufficiency and consistency of standards for the QAA-recognised award which the AVA has a responsibility to uphold.

51 In conclusion, the review team agreed that the AVA did not meet criteria expressed under this principle in two respects: first, with reference to criterion 3, the recognition process does not ensure consistency of outcome in relation to the fitness for purpose of Access programmes, because recognition does not ensure that the credit requirements for the award of the Access certificate are coterminous with successful completion of a planned programme of study, where successful completion is defined by preparedness for HE; secondly, with reference to criterion 4, the responsibility for final approval of QAA-recognised Access programmes is not made with the full authority and approval of the AVA. Both points must be addressed as conditions of licence.

Principle 6

The organisation is able to safeguard the continuing quality of Access to HE programmes, and to secure the standards of achievement

Moderation

52 The AVA conducted a 'fundamental review' of its processes and procedures for moderation and standardisation between December 2002 and August 2003. This was undertaken in recognition of identified concerns about the recruitment and selection of moderators and the effectiveness of standardisation arrangements. The AVA employs external moderators in a number of different categories with different roles: 'principal moderators', 'lead moderators' and 'subject (or "subject specialist") moderators'. The precise model of moderation and deployment of different categories of moderator is determined by the size of provider and organisation of the provision. A new category of 'organisation moderator' has been introduced as a consequence of the review, to bring the benefits of the principal moderator to smaller organisations. The embedding of this scheme is currently underway but at the time of the review visit, during this period of transition, there was still some uncertainty among moderators about their revised roles and position in the new structure, and associated terminology, particularly for those who took on multiple roles.

53 The review team heard that providers and moderators welcomed this scheme as it is perceived as professionalising the moderator's role, and moderation documentation has been designed to dovetail with providers' requirements in relation to the Common Inspection Framework. The scheme has established criteria for performance review, clearer service specifications and criteria for the selection and recruitment of moderators. All newly appointed moderators are required to attend an induction and initial training programme. Following this they are strongly encouraged to undertake the OCN External Moderators' Award. There is also an ongoing programme of staff development that, although not specifically designed for Access moderators, does address generic issues that are of relevance.

Selection and recruitment of moderators

54 New moderator roles and the corresponding person specifications for selection and recruitment have been developed. However, the documentation is used generically across the OCN and does not identify any specific requirements for the academic knowledge base needed to make reliable judgements about achievements and standards on Access programmes. Guidance documentation for assessment, internal and external moderation, and award of credit has also been developed in draft form but, as with recruitment documentation, there are no specific guidelines for Access programme moderators.

55 The AVA identified concerns about the recruitment of HE-based moderators, and it was recognised that the newly implemented scheme had not addressed the AVA's difficulties in this area. The AVA emphasised the importance of the involvement of HE moderators and the Account states that the AVA aims for all Access programmes to 'have at least one member who has current or recent higher education experience within their moderation teams'. The AVA is also concerned about the unwillingness of some HE-based moderators to commit to the OCN External Moderators' Award. The review team heard that, in response to these problems, a further role is under consideration for HE moderators of Access programmes, which will focus less on verifying procedures and more on assuring appropriate standards for HE progression, and will not include the broader responsibilities currently identified in moderator roles. This could bring about a valuable addition to the AVA's procedures for assuring standards on Access programmes.

Moderation outcomes

56 Currently, principal moderators produce annual reports based on the outcomes of both internal and external moderation for each provider. These principal moderators' reports provide the 'moderator's report to the AVA about the effectiveness of the provision' in providing institutions and constitute the primary means through which issues for concern and recommendations are monitored. Procedures for the receipt, evaluation and monitoring of the output of the moderation process in the AVA office are well documented. The review team heard that the AQC receives and notes principal moderators' reports, and the AQC noted at its meeting in October 2002 that it 'wished to receive the Moderation Annual Review Report and monitor any action plans that emerged'. However, the team was unable to locate any record of such monitoring by the AQC in the documentation provided for the review. The team noted with concern that there appeared to be no specific summative or sampling mechanism in place to ensure that moderation outcomes were analysed and the AVA's Access portfolio as a whole systematically reviewed.

Award and issue of certificates

57 There are formal procedures for the award of credit as a component of Access certificates, and moderators are clear about the role in verifying achievement within this context so that learners can achieve credit. Appropriate documentation is in place to support this process for the award of credit. In relation to any specific additional procedures required for the award of the Access certificate, however, the process is less clear. The review team heard that in some cases there was a final assessment meeting, comprising moderators and tutors, where decisions about achievement and the award of an Access certificate were made. In other instances, decisions were made on the basis of achievement of credit at the point of final moderation, and a final assessment meeting did not occur. Guidance from the AVA concerning the function, process and imperative to conduct a final assessment meeting was absent. The resulting outcome was varying practices which do not assure consistency in the decision-making process for the award of the Access certificate.

58 Certificates are issued according to a standard format and comply with QAA requirements. Administrative procedures for the issue of certificates are secure and providers are able to supply them to their learners, with a transcript of achievement. Providers and the AVA value the status and currency of QAA-recognised Access certificates.

Standardisation

59 There is a developing coherence between internal and external moderators based on good communication and liaison. The AVA has made progress with some areas of standardisation though the mechanisms of the Building Pathways Project (see paragraph 34, above) which, in developing common regional units, has introduced standardisation in some areas. There are some concerns, however, in the AVA's overall mechanisms for standardisation of assessment across providers.

60 In the past, the AVA has conducted standardisation meetings at the end of the academic year in selected programme areas. The AVA has recognised that these partial, retrospective activities, which have not always been well attended, have not constituted an effective way to assure consistency of standards. It 'sought to address these concerns within its comprehensive review of moderation and standardisation arrangements' and the Review of Moderation and Standardisation Consultation Paper states the AVA's position that 'Standardisation is not a separate process from moderation - standardisation should be fully embedded within moderation'. The consultation paper provides a number of examples of how this might happen through, for example, the use of a single moderator across a number of institutions. The consultation proposed that 'Standardisation Meetings are discontinued and alternative ways of bringing subject specialists together are considered', and the AVA did not hold standardisation meetings for Access to HE in 2002-03. Some concern about the cessation of these meetings for Access was identified in the minutes of the AQC meeting of October 2003, and the Account reports that the AVA is 'exploring options for alternative approaches for promoting standardisation'. While cross-institutional moderation by principal moderators provides a mechanism for an element of standardisation to take place, it is not sufficiently formalised to ensure that all Access providers are included. In the absence of any 'alternative approaches' having been established, at the time of the review visit no mechanisms were in place through which the AVA could assure the consistency of standards across all Access programmes.

61 The review team agreed that the AVA did not meet all of the criteria for this principle: the outcomes of moderation procedures need to be reported to and considered within the AVA's governance structures and used to improve the quality and standards of the provision; and the AVA should develop clear guidelines for providers to ensure consistency of final assessment decisions in the process of the award of Access to HE certificates. In addition, the team would suggest that the roles and responsibilities of the different categories of moderators be further clarified, including the place and role of representatives from HE.

Principle 7

The organisation is underpinned by structures and processes which enable it to review, evaluate and develop the Access to HE provision for which it has responsibility

Quality and comparability

62 The AVA notes in the Account that it does not require providers to submit their internal programme annual review reports to the AVA for scrutiny, nor does it require any other evaluative report from providers. Rather, it depends on moderators to 'monitor the internal quality assurance and review processes of providers through access to internal quality files and programme review reports'. There is therefore no requirement made by the AVA that providers should engage in a process of regular evaluation of their provision, and the AVA's awareness of the provision is based on information provided in moderators' reports. The Account explains that 'Any significant issues are reported to the AVA through standard moderation reporting mechanisms', indicating that it is by exception ('significant issues'), rather than by regular scrutiny, that information about the provision is brought to the AVA's attention. As indicated above (see paragraph 56), these mechanisms do not, in any case, provide an overview of provision, so the AVA does not have the opportunity to review and monitor provision systematically through this source. The monitoring of quality of provision is dependent on principal moderators' reports. These reports are considered by officers but are not subject to scrutiny and report by committees of the AVA. Responsibility for monitoring comparability and consistency of provision is not clearly located.

Re-approval processes

63 In 2002, OCN(SYHR) conducted a review of its re-approval processes, although this review of re-approval processes did not include Access to HE. As with the initial approval process, the re-approval of programmes is through peer panels facilitated by officers of the AVA and not through the AVA's committee structure.

Statistical data

64 The AVA acknowledges that there is currently incomplete longitudinal data on ethnicity, employment and disability. Nevertheless, the review team was assured that recent improvements in data collection would improve this situation and the AVA's capacity to analyse its provision through its consideration of statistical data in the future. This data is considered by the AVA and is also shared with providers.

65 The review team considered that the AVA had not met the criteria expressed under this principle in full, because it does not have a rigorous system for the monitoring, review and evaluation of the Access to HE provision for which it is responsible.

Conclusions

66 Open College Network (South Yorkshire and Humber Region) is a large and well-established AVA, which is efficient and professional in its internal organisation and in its dealings with its providers as customers. It is generally well regarded and operates within a framework of clear service standards and according to thorough internal procedures for regulating its own activity. These standards and procedures are supported by detailed and comprehensive documentation which provides appropriate guidance for the AVA's staff.

67 The organisation has a strong relationship with its host institution, Sheffield Hallam University, which has leant the AVA considerable support and credibility. This relationship has also assisted the AVA in maintaining a close HE link for the AVA; has helped the AVA to build strong and effective networks and partnerships; and provided a structural framework from which to develop its Access provision. The organisation has clearly benefited substantially from this relationship and, with no overt interference by the University in the AVA's day-to-day decision-making, allowing operational independence for the AVA in most significant respects, it is, perhaps, understandable that this relationship has remained essentially unexamined until recently. Unfortunately, the AVA's recent assessment of the position has not taken sufficient account of the need for an AVA to demonstrate and be able to assure its independence.

68 Notwithstanding the strength of this relationship, the AVA has also established and developed a strong identity of its own. That identity, which is very largely defined by its role as an OCN, has been effectively promoted within its area of operation, and the organisation has been involved proactively in a wide range of activities which, while not necessarily directly relating to its Access work, nonetheless bring recognition to the organisation as a whole and, in some instances, serve to underpin QAA-recognised Access provision.

69 The organisation's role as a licensed AVA, operating under specific obligations as an awarding body for the QAA-recognised Access to HE certificate, is not separately or explicitly defined, either constitutionally or operationally. In such circumstances there is always a possibility that the focus on the needs of Access itself is diminished, both in relation to quality assurance procedures and development priorities. In order to avoid this risk, an AVA operating an 'integrated model' needs to put in place structures and procedures which will maintain sufficient emphasis on its Access work to meet its AVA responsibilities in full. In this instance, the interest in Access work and its development is evident in the attention and involvement of the OCN's Director, but the functions and remits of the AVA's committees are underdeveloped with respect to the specific needs of Access and the responsibilities of the organisation's AVA licence. While there is a clear appreciation by the AVA that Access has some particular characteristics which mark it out from other provision, and some procedures indicate specific requirements for Access, there is a lack of a focused critical overview of Access provision through the AVA's governance structures.

70 The AVA's committee structure is slim, with an Executive Committee, which manages the essential governance functions of the business, and an Accreditation Quality Committee which is responsible for 'the development, monitoring and review of policy, regulations and procedures' relating to accreditation, but not, specifically, for the assurance of the quality or standards of provision. As a result, while there is considerable focus on developing detailed procedures, less evident is any equivalent focus on the evaluation of their outcomes, in relation to Access provision as a whole.

71 As the committees do not have direct responsibility for monitoring standards or the quality of provision itself, the AVA must meet its responsibilities in this regard through other means. Its approach to monitoring its major quality assurance processes demonstrates an attention to detail and diligence in the AVA office which is characteristic of the AVA's practices, with information received being subject to careful checking and processing by officers. While undoubtedly thorough at this point of the process, this approach effectively locates responsibility for assuring quality either with those who are involved in the delivery of programmes, or with those who do not have the authority to challenge the judgements and decisions being recorded, while those who might have the authority to intervene on the AVA's behalf are not provided with the effective means or systematic information to do so. There is a tendency within the AVA to rely on attenuated quality assurance processes of this kind which place actual decision making too far from those who hold ultimate responsibility for the decisions made. This tendency is evident, for example, in the AVA's systems for programme recognition, where final approval decisions are made by individual recognition panels; in the AVA's manner of dealing with moderation outcomes, which does not provide an evaluation of the quality of Access provision as a whole; in its approach to programme consistency, through proposing that standardisation should occur as a function of moderation; and at the point of decision-making for the recommendation for the award of the Access certificate, for which the AVA does not require providers to follow standard procedures determined by the AVA to assure consistency of process and outcomes.

72 While the diversity of programmes is a necessary characteristic of Access provision, so also is the common outcome of a nationally recognised qualification. Given that, for some programmes, the AVA awards the Access certificate at a level which the providers concerned apparently do not consider sufficient to constitute 'readiness for higher education', there is a threat to the integrity of the QAA-recognised Access award in the AVA's approach.

73 The organisation is forward-looking and, with effective leadership, has an acute awareness of the importance of developing a strategic position that will allow it to take the business forward and address current political agendas in adult, further and higher education. This can have obvious benefits for Access, especially with support from HE and a strongly committed practitioner base such as exists in this area. However, in order to maintain and advance such benefits, it is essential for the AVA to provide a lead in developing and promoting a strategic developmental agenda specifically for QAA-recognised Access to HE provision.

The AVA licence

Review outcome

74 The Open College Network (South Yorkshire and Humber Region) is awarded a provisional renewal of its AVA licence, with conditions stipulated below to be met by the times specified. The AVA will be revisited in winter 2004-05.

Conditions

75 The licence is provisionally renewed on condition that the AVA

i amends its constitution to

provide a clear and unambiguous description of its legal identity (paragraph 17);
ensure that it is protected from the undue influence of any one of its members which may be exercised through a constitutionally conferred right (paragraph 19);
specify a clear locus of authority for its AVA responsibilities (paragraph 25);
ensure that responsibilities are clearly specified within the governance structures for the quality assurance of QAA-recognised Access to HE provision (paragraph 25); (paragraph 26)

ii amends its Memorandum of Agreement with Sheffield Hallam University to ensure that its independent operation is not restricted with regard to the appointment and appraisal of the Director (paragraphs 29 and 31);

iii amends its operational procedures and committees' terms of reference to ensure that the responsibility for the final approval of QAA-recognised Access to HE programmes is specifically located within the AVA's governance structures (paragraphs 45 and 49);

iv ensures that the award of the QAA-recognised Access to HE certificate is co-terminous with successful completion of a planned programme of study, where successful completion is defined by preparedness for higher education (paragraphs 46-51);

v develops and implements procedures to ensure that moderation outcomes for QAA-recognised Access to HE programmes are reported to, and considered within, the AVA's governance structures; and that outcomes are used to evaluate and improve the quality and standards of the provision (paragraphs 56 and 61);

vi develops and implements clear guidance to ensure consistency of final assessment decisions in the process of the award of the QAA-recognised Access to HE certificate (paragraphs 57 and 61);

vii develops and implements a rigorous system for the monitoring, review and evaluation of the Access to HE provision for which the AVA holds responsibility (paragraphs 62 and 65).

Conditions to be met by 1 October 2004 (evidence of implementation to be considered at revisit in winter 2004-05)

76 Provisional confirmation of licence allows the AVA to continue to operate under licence but makes provision for the immediate withdrawal of the licence if the conditions are not met in the time allowed and to the satisfaction of QAA. Were it to be necessary for QAA to withdraw the licence, the AVA would need to put in place procedures for the transfer of registered students to an alternative licensed AVA.

Recommendations

77 The review team recommends that the AVA:

i clarifies procedural documents to ensure that moderators for Access to HE programme have appropriate qualifications/experience;

ii ensures that operational procedures and documentation clearly identify the responsibilities of the different categories of moderators involved in the moderation of QAA-recognised Access to HE programmes, including the place and role of representatives from higher education; (paragraphs 48, 53 and 59).

Appendix

Aims and objectives of AVA review

The aims of the system of AVA review are:

i to provide the basis for an informed judgement by the ARLC about the fitness of the AVA to continue as a licensed agency;

ii to promote public confidence in Access to HE as a properly regulated and respected route into higher education by assuring:

the quality and adequacy of AVAs' systems and procedures;
the quality, comparability and range of AVAs' operations;
the adequacy and comparability of AVAs' standards for approval, moderation and monitoring of programmes;
consistency across AVAs in the operation of criteria for the granting of the Access to HE award;

iii to stimulate reflective and self-critical perspectives within AVAs, as an instrument to promote quality enhancement;

iv to provide an opportunity to identify and disseminate good practice of AVA operations;

v to provide a mechanism for ensuring necessary, and encouraging desirable, improvements and developments in AVAs.

The objectives of each AVA review are:

i to examine, assess and report on:

the development of, and changes in, the AVA since its last review or initial licence, and its plans and targets for the future;
the organisation's continuing viability and robustness and the ways in which the AVA demonstrates sound governance;
the efficiency and effectiveness of the AVA's operational and quality assurance systems;
the range and scope of the AVA's activities, and the appropriateness and value of these activities;
the ways in which the AVA approves and monitors programmes and the ways in which these processes take account of the need for consistency and comparability;
the ways in which the AVA satisfies itself of the adequacy and comparability of standards achieved by students gaining the Access to HE certificate;
the evidence available to indicate the AVA's success in achieving its aims and targets;

ii to identify and report on:

strengths and good practice in procedures and operations;
areas which would benefit from further development;
areas requiring attention.

Open College Network (South Yorkshire and Humber Region) November 2003

Contents