Oxfordshire Open College Network
May 2001

Foreword

1 The Quality Assurance Agency for Higher Education (QAA) is responsible to the Department for Education and Employment for the recognition of Access to Higher Education courses. QAA exercises this responsibility through a national network of authorised validating agencies (AVAs), which are responsible for the processes of individual course recognition and the award of Access to HE certificates. QAA has developed a scheme for the licensing and review of the AVAs, the principles and processes of which are described in the QAA Recognition Scheme for Access to Higher Education in England, Wales and Northern Ireland. The Recognition Scheme is regulated and administered by the Access Recognition and Licensing Committee (ARLC), a sub-committee of the QAA Board of Directors.

2 This is a report of a major review of the AVA function of the Oxfordshire Open College Network (OxOCN) undertaken by QAA. The Agency is grateful to OxOCN and to those who participated in the review for the willing cooperation provided to the review team.

Aims and objectives

3 The aims of the system of AVA review are:

i to provide the basis for an informed judgement by the ARLC about the fitness of the AVA to continue as a licensed agency;

ii to promote public confidence in Access to HE as a properly regulated and respected route into higher education by assuring:

the quality and adequacy of AVAs' systems and procedures;
the quality, comparability and range of AVAs' operations;
the adequacy and comparability of AVAs' standards for approval, moderation and monitoring of programmes;
consistency across AVAs in the operation of criteria for the granting of the Access to HE award;

iii to stimulate reflective and self-critical perspectives within AVAs, as an instrument to promote quality enhancement;

iv to provide an opportunity to identify and disseminate good practice of AVA operations;

v to provide a mechanism for ensuring necessary, and encouraging desirable, improvements and developments in AVAs.

4 The objectives of each AVA review are:

i to examine, assess and report on:

the development of, and changes in, the AVA since its last review or initial licence, and its plans and targets for the future;
the organisation's continuing viability and robustness and the ways in which the AVA demonstrates sound governance;
the efficiency and effectiveness of the AVA's operational and quality assurance systems;
the range and scope of the AVA's activities, and the appropriateness and value of these activities;
the ways in which the AVA approves and monitors programmes and the ways in which these processes take account of the need for consistency and comparability;
the ways in which the AVA satisfies itself of the adequacy and comparability of standards achieved by students gaining the Access to HE award;
the evidence available to indicate the AVA's success in achieving its aims and targets;

ii to identify and report on:

strengths and good practice in procedures and operations;
areas which would benefit from further development;
areas requiring attention.

Outcomes

5 Following the review of an AVA, the reviewers' report is presented to the ARLC by a member of the review team. The Committee then makes one of six decisions:

i unconditional renewal of licence for a specified period;

ii conditional renewal of licence with conditions to be met by specified date;

iii provisional renewal of licence with conditions to be met and further review visit by specified date;

iv suspension of licence until specified conditions are met;

v withdrawal of licence for operation as an AVA;

vi temporary renewal of licence with request for further information by specified date
(decision suspended).

The review process

6 The review was conducted in accordance with the process detailed in the QAA Recognition Scheme for Access to Higher Education in England, Wales and Northern Ireland. The preparation for the review included an initial meeting between OxOCN officers and the QAA Assistant Director with responsibility for Access to HE matters to discuss the requirements for the Analytical Account (the Account) and the process of the review; the preparation and submission by OxOCN of its Account, together with a selection of supporting documentation; a meeting of the review team to discuss the Account and supporting documentation and to establish a draft programme for the review visit; and negotiations between QAA and OxOCN to finalise the programme and other arrangements for the review visit.

7 The review visit took place on 22 and 23 November 2000. The visit to OxOCN consisted principally of meetings with representatives of OxOCN, including OCN officers; members of the Executive Committee, Quality and Audit Committee and Access Programmes Committee; moderators for Access programmes; Access programme course tutors; HE admissions staff; and former Access students now studying in HE.

8 The review team consisted of Mr Steve Babbidge, Co-ordinator of South of England Open College Network, and Ms Priscilla McGuire, Development Officer at West and North Yorkshire Open College Network. The review was coordinated for QAA by Ms Kath Dentith, Assistant Director (Access), Institutional Review Directorate.

The AVA context

AVA statistics 1999-2000

9

Providers offering Access to HE programmes: 9

Access programmes available: 23

Access programmes running: 16

Access learner registrations: 447

Access to HE certificates awarded: 229

Background

10 OxOCN was awarded its AVA licence in 1994, the AVA licence having been held by Oxford Brookes University AVA prior to that date. The AVA has maintained a close relationship with the University, and has recently moved to office premises leased from the University from premises which were provided rent-free by Oxfordshire County Council.

11 Since 1994, the number of Access programmes validated, the number of students enrolled on Access programmes, and the number of students receiving Access Certificates have all increased, from 10 to 18, from 312 to 447, and from 136 to 229 respectively. Although these figures identify OxOCN as one of the smallest AVAs, they also indicate steady growth of the organisation's AVA activity. The OCN's Access work continues to represent a significant proportion of its total business and the Account records that approximately one third of all credits awarded on OxOCN programmes are awarded to students from Access programmes.

Membership

12 The AVA serves the population of Oxfordshire and, through the membership of Swindon College, parts of Wiltshire. The AVA membership includes seven FE providers; two adult residential colleges; Oxfordshire Local Education Authority (which provides Access programmes through two Community Education Centres); the WEA for Thames and Solent; and two HE institutions (Oxford Brookes University (the University) and the University of Oxford Department of Continuing Education). The Open University Southern Region is listed as a member of the OCN, but is not mentioned in the Account as a member of the AVA. The Account also records that 'Harris-Manchester College, an adult college within the [Oxford] University, took the initiative to explore closer links with OxOCN...OxOCN would like to continue this dialogue, particularly with those colleges that concentrate on admitting adult students'. It was not clear to the review team, however, whether it was intended that this aspiration would be pursued at a strategic level. The team would encourage the AVA to develop any such relationships which may enhance local HE progression opportunities for students, and provide increased HE involvement in the affairs of the AVA.

13 The AVA makes a distinction between members and users, with the user category being available to encourage smaller organisations to purchase services from the AVA before making a full commitment to AVA membership and its wider obligations. The AVA hopes to be able to encourage employers who may wish to offer Access programmes to become involved with the AVA on this basis, thereby expanding the range of Access providers.

Finance

14 The (draft) Memorandum of Agreement between OxOCN and Oxford Brookes University (see paragraph 27, below) refers to the Executive Committee's responsibility for determining the budget and staffing levels, but also states that 'The University requires the OCN normally not to operate a deficit budget and to set staffing levels so that they can be met within the agreed budget'. As matters stand currently, however, the OCN has complete operational and management control over its financial affairs: it has its own bank account, employs its own auditors and is responsible for administering appropriate financial records and systems. These responsibilities are new to the OCN as, prior to August 1999, OxOCN accounts were kept by Oxfordshire County Council and the OCN did not have a detailed set of accounts available to it. Following a period several years ago when it found itself with a significant deficit, the OCN was able to recover its position, and accumulated reserves of £80,000 which it was able to transfer when it established its own account. The OCN is conscious of the need to maintain sufficient reserves to provide a contingency fund, although it also sees its reserves as a possible source for funding development work.

15 As a way of reducing costs to members, OxOCN plans to reduce programme registration charges, with greater emphasis placed on charges for learner registration and award of credits. On programmes with a large number of available credits, such as Access to HE programmes, there is a fixed fee so that there is no financial penalty when students gain a greater number of credits than the required minimum. In addition, the charging policy has been adjusted so that differential membership fees are charged according to organisations' gross incomes.

Mission and aims

16 The OCN's mission, which encompasses the values of the widening participation agenda, is supported by seven aims which 'guide the setting of objectives and targets in the Strategic Plan and Annual Operating Statement'. The role of the AVA is embedded in the OCN with the aim 'to apply the principles, standards and procedures of the Quality Assurance Agency's Access to Higher Education scheme [sic] to all aspects of OxOCN's work as an Authorised Validating Agency'.

17 The Strategic Plan for 2000-03 has six objectives relating to this aim, each of which has one or more linked targets for both 2000-01 and 2000-02. Apart from the objective to complete the current review successfully, these objectives specify development goals including increasing recruitment among men; increasing the use of Access programmes by employers; developing moderation systems; improving data gathering; and securing funding for Access provision.

Current client groups and targeting

18 As well as an overall increase in the numbers of students enrolling on Access courses, there has also been an increase in the number and proportion of enrolments of students from ethnic minorities. In 1999-2000, the figure was 16.6 per cent, compared to figures of 13 per cent and 11 per cent for the previous two years. These percentages compare favourably to the most recent census figures (1991), which give 3.3 per cent and 9.2 per cent as the ethnic minority proportion of the population for Oxfordshire and the city of Oxford respectively. However, increases and decreases between different ethnic groups vary, and the actual numbers involved are too small to be able to draw any clear conclusions.

19 The Access to HE Annual Review of Programmes for 1999-2000 states that 'The main target groups are local people, the unemployed, and women, alongside social groups traditionally under represented in Higher Education'. If this statement accurately represents the target groups of Access programmes currently available, it would appear that the objectives specified in the Strategic Plan (see paragraph 17, above) represent an attempt by the AVA to bring about some significant changes in emphasis in AVA targeting policy, or additions to target groups for Access programmes, to emphasise the targeting of men (rather than women) and the employed (rather than the unemployed).

20 The AVA's own statistics show that there has been a steady increase in the proportion of women enrolling on Access courses over the last six years, from 70 to 80.5 per cent, although this change can be accounted for largely by increasing numbers of registrations by women, rather than any significant decrease in the number of registrations by men. The AVA has responded to this increasing disproportion of registrations by men and women on Access programmes by adopting the strategic objective to 'Encourage providers to increase recruitment from under represented groups, particularly adult males, on to Access programmes'. Discussions with the AVA's committees and officers confirmed that this was considered a key objective, with an associated intention to monitor the recruitment of men on to Access programmes. The review team considered that, given the context described above, this was a worthwhile aim, and sought to clarify the means by which OxOCN intended that it should be achieved. The specific targets associated with this objective comprise the monitoring of recruitment of men on to Access programmes; devising 'flexible delivery' for Access programmes; and constructing 'appropriate Access programmes for under represented groups'.

21 Both the documentation presented to the review team and discussions during the review indicated that greater flexibility in course delivery patterns was a matter of current interest and development among course providers, and the greater availability of day release and part-time provision were options for delivery that were being explored within the AVA. Irrespective of its relationship to the particular objective, the extension of Access provision through a wider range of modes of delivery was regarded by the team as a positive development. While the team recognised that more flexible modes of delivery might appeal to men in certain circumstances, it was not clear that this measure would, of itself, necessarily address the under-recruitment of men.

22 The second strand of the AVA's strategies relating to this objective, concerned with developing 'appropriate Access programmes for under represented groups', is less well developed at this stage, and there was no evidence available to indicate that any programmes resulting from this target had been developed. Although there has been some work in relation to work-based programmes (see paragraph 24, below), the two particular examples provided had both been initiated by HE partners, and were both in vocational areas which tended to recruit women rather than men.

23 The strategic objective to encourage more employers to be involved with Access to HE courses is consistent with the AVA's view of its role in meeting the needs of both the local and national skills agenda by developing Access programmes to meet the needs of employers, and providing work-based Access to HE routes. Given that the county has a very low unemployment rate (1.2 per cent), this strategy would appear to be an appropriate way for the AVA to consider how it might best extend the reach of its Access provision and, at the same time, contribute to the Government's skills agenda. The Oxford Strategic Partnership Widening Participation Project, based at Oxfordshire LEA, has provided data on skills shortages to the AVA and this has been used in the development of the Strategic Plan.

24 Strategies to attract the low-skilled employed also include plans to develop the provision of specific vocational programmes and pathways, and healthcare is seen as a particular area for potential growth and development. Successful work has already been undertaken in collaboration with the Radcliffe Health Trust and the School of Health Care at Oxford Brookes University, leading to the validation of an Access to Health Care programme which builds on a larger framework of OCN-accredited programmes at other levels for healthcare workers. The AVA has also validated an Access to Work Based Initial Teacher Training programme (currently delivered in a college in Birmingham). The AVA indicates that it regards both of these as possible models to develop work-based provision.

25 In discussion with the review team, higher education admissions tutors expressed some concern about the potential dangers of over-specialisation in the curriculum in vocationally defined courses. While no criticisms of specific currently validated OxOCN courses were made, the team noted admissions tutors' observations that, in their experience, vocational Access courses sometimes included material which would form part of an undergraduate programme, leading to a repetition of curriculum study for students, when it was felt that students needed to develop a more general skills and knowledge base, from which they would be able to progress to the study of these subjects at a more advanced level. There was also some concern expressed that Access programmes should not be so narrowly defined that students' progression opportunities became restricted. This view was supported by practitioners, who expressed some concerns about those students who started an Access programme with a strong idea about a particular vocational progression route and then changed their minds: programmes leaders emphasised the need to build into programmes sufficient breadth and flexibility to enable such students to move between courses with different progression routes.

26 While the value of labelling courses to indicate a particular progression route is recognised, both in terms of the provision of clear information for potential students and as a key element in attracting new groups of students, the AVA will wish to be mindful of the expressed concerns of admissions representatives of the HE institutions to which Access students may wish to progress. The review team would recommend that the AVA considers carefully the way in which it develops this major policy initiative in order to take full advice from HE representatives, including admissions tutors, about the development of the policy as a whole, as well as continuing to take appropriate advice about the development of individual programmes.

27 The Short briefing paper on a strategic approach to Access, prepared by the Director, also observes that '...if you take a post code analysis across Oxfordshire then, overwhelmingly, students are not coming from those post codes with a widening participation factor'. The paper goes on to propose a solution which supports plans to strengthen the vocational emphasis of Access programmes, but does not specifically address targeting of the socially excluded, which its data has identified as a gap. Although this document does not have the status of a policy statement, many of the ideas in it are reflected in the OCN's Strategic Plan.

Governance and committee structures

Legal status and constitutional position

28 The AVA describes itself in its Account as '...an unincorporated association of members owned and run by the membership through its committee structure and appointed staff'. The draft Memorandum of Agreement between the OCN and Oxford Brookes University sets out the recognition of respective authority; services to be provided; accountability; decision making for finance and staffing; and the 'key functions' of the AVA. The Memorandum also explains that staff are '...employed on joint contracts of employment with the University and OxOCN'. The Memorandum has not yet been approved and the University has identified a number of areas requiring further development before it can become a legally binding agreement, to include duration and termination of the agreement; more specificity about services; liabilities and indemnities; ownership of intellectual property; confidentiality and data protection responsibilities.

29 The review team sought clarification from senior representatives of the University and the AVA that the AVA was indeed able to discharge all its duties and responsibilities in an independent and autonomous manner, and about the nature of 'joint employment', particularly with regard to personnel and management functions. The team was assured that the relationship between the University and the AVA was one in which the University 'held' the employment contracts for the AVA; that there was no confusion or conflict over ownership or control of staff; and that the relationship could be described as one of 'landlord and tenant', with 'expert services' provided as required. The team was also assured that responsibility for the Director's staff appraisal would lie with the Chair of the OCN. The team noted that this degree of clarity was not yet present in the Memorandum of Agreement (nor in diagrammatic representations of OxOCN's management structure), and would expect that the areas noted by the University would be incorporated as a matter of urgency and would be augmented by clarification of roles and responsibilities for staff development and appraisal.

Summary of structures, roles and remits

30 The AVA operates a governance and committee structure which currently has four tiers: a Council of Members; an Executive Committee; a Quality and Audit Committee (QAC); and an Access Programmes Committee (APC), Credit Exemption Committee and FE/HE Forum. Each group has a role and remit described in a new constitution which was approved in February 2000. Governance is constitutionally located with the Council of Members; decision-making for Access is constitutionally located in the Executive Committee, advised by the QAC, which itself 'receives, considers and acts upon reports from the Access Programmes Committee'.

31 This structure is relatively new, with the most recent change being the creation of the Executive Committee, which met for the first time in autumn 2000. The implications of the insertion of this new tier in the governance structure on decision-making at other points within the AVA would not yet appear to be understood in the same way by all AVA officers and committee members. The review team noted, in particular, some uncertainty about the extent to which the Executive Committee was responsible for governing, managing and/or functioning at an operational level with regard to key functions, and it would appear that the model described through the constitution has been subject to some reconsideration by Executive Committee members. It is recognised that, although the Account alludes to the Executive Committee's 'managerial role', some further refinements may be required to clarify its governance rather than managerial or operational functions, and to draw a distinction between the Executive Committee's and Director's responsibilities in relation to management.

32 From discussions with the Chair and members of the Committee, and from scrutiny of the minutes of meetings, it was apparent to the review team that work was still underway in clarifying roles and responsibilities since the creation of the Executive Committee. The review team noted that, for example, the Executive Committee is constitutionally required to 'Approve nominations to the Quality Assurance Committee', yet the review team could find no evidence, either through its scrutiny of the minutes or in discussions with Committee members, of this function being carried out.

33 In considering the effectiveness and efficiency of the new committee structure as a whole, the review team noted that there appeared to be considerable overlap between the responsibilities and functions of the Council of Members and the Executive Committee. If the low attendance which has characterised meetings of the Council of Members over the last two years continues, there is the potential for decisions of the Executive Committee to be simply approved by the same group of people acting as the Council of Members.

34 The review team therefore considered in some detail the production and approval of the AVA's strategic plan and operating targets (which is located within the strategic plan for the OCN as a whole) as an example of decision-making involving a number of different layers of this new governance structure. The Executive Committee's remit includes a responsibility to 'produce' a three yearly strategic plan and annual operating plan and to 'review and evaluate targets set' in these documents, while the Council of Members carries responsibility for the approval of both. The new Executive Committee received and discussed the Strategic Plan 2000-2003 at its first meeting in October 2000, prior to its approval by the Council of Members, the Plan having been produced on this occasion by the Director and officers. The team noted that the Committee had fulfilled its remit in relation to the review and evaluation of targets of the Strategic Plan and, having expressed some initial doubts about the achievability of some elements of the Plan, it had properly required the Director to produce regular reports of progress towards the meeting of targets. The team noted that the responsibility to 'produce' the Strategic Plan was an unusual one for the main governing body of an AVA, and it did not appear that, in practice, the Executive Committee had actually had this level of direct involvement, the process being described to the team as being one of 'continuous development' in which the Director advised the Committee about proposals presented. It was not clear whether it was intended that the Executive Committee would have a fuller role in the production of the Plan in future years, but the Committee appeared to be satisfied that the process which had been followed for the current Plan had allowed it sufficient opportunity for debate about the detail of the Plan and associated targets.

35 It appeared to the review team from discussions with committees' representatives at the review visit, that the relationship between committees was currently operating in what might be termed an 'organic' way, with a good measure of communication and reporting between committees. However, there was some uncertainty about the extent to which a committee acted under its own authority, with devolved authority, or existed to 'refer up' the chain of decision-making. In seeking to clarify chains of communication and decision-making in order to distinguish governance, management and operations, the team noted some dissonance between what was stated in formal documentation and what was understood by those making decisions. The examples given above led the review team to form the view that, in certain respects, the process of, and responsibilities for, decision-making needed to be more clearly delineated.

36 The AVA is clearly at a stage of refining its organisational model and clarifying its committees' remits. It is essential to the organisational health and viability of the AVA that the way in which committees act is reflected consistently in all documentation, and that both officers and committee members share a common understanding of their roles and remits. The review team concluded that this stage had not yet been reached by OxOCN.

Quality assurance functions

37 Quality assurance functions are discharged through the QAC which has the overall responsibility for monitoring and reviewing the quality of the accreditation process, including 'monitoring the process for the award of kitemarked access certificates'. The QAC is constitutionally responsible for a combination of apparently operational functions (including the endorsement of minor modifications to programmes) and extensive quality control, including an annual review of Access programmes every autumn. The latter process is based on a composite report prepared by the Access Development Officer which includes all individual Access programme and moderation reports, the Development Officer's attendance at a selected number of final moderation meetings, and data produced from the AVA's database. The report allows a broad but focused overview to be taken of developments and issues in the delivery and quality assurance of Access, a process assisted by the use of dedicated Development Officer support for Access.

38 Although the report is comprehensive in its coverage, it was not clear to the review team how the analysis which can be made of trends observed feeds into the AVA's planning process or into its annual reporting to QAA. The QAC noted at its November 1999 meeting that 'possibly in time it [the annual report to QAA] could be amalgamated with the Access report...' but, to date, the AVA does not appear to have taken forward this suggestion. In the view of the review team, the detailed document which is produced as a result of the process of annual review, with its inclusion of full moderators' reports, and clear summary and analysis, merits further consideration for its potential contribution to the AVA annual report to QAA.

39 The AVA has historically dedicated a full meeting of the QAC to the process of annual review at which the QAC could give detailed consideration to the output from Access provision. However, the nature and extent of analysis recorded in the minutes since 1999 (when the review was incorporated into a general QAC meeting) would appear to indicate that the QAC's overview may not be as detailed as it had been previously. The review team noted with regret that where much useful work had been undertaken in previous years in identifying general issues of concern as well as good practice in Access provision, this did not appear to have been continued over the last 12-18 months. The team would urge the AVA to maintain the level and nature of scrutiny of Access that the QAC previously exercised.

Access Programmes Committee

40 The APC monitors developments in relation to Access, and reports from the APC are taken as a standing agenda item at QAC meetings. The APC is well-established and well-attended, and the commitment to Access of the committee was evident from discussions and also from a review of both attendance figures at meeting and the business of its meetings as recorded in its minutes. The APC's membership includes a representative of Access Programme leaders from each provider institution, as well as two Access moderators, a representative from each member HE institution, and the Development Officer (Access). With this membership of practitioners who are actively involved in delivering Access programmes and taking part in AVA processes such as validation and moderation, it is appropriately constituted to fulfil its responsibility to maintain an oversight of Access to HE provision. It is also evident that it provides a worthwhile forum for discussion and development work.

41 The APC is not, however, a decision-making body, although this was not entirely clear to all members of the Committee, nor had it apparently been clear at other levels in the AVA. For example, in response to a directive from NOCN in 1998, the AVA initiated a policy for the inclusion of Key Skills which incorporated all three core skills on a mandatory basis. By 2000, the APC had determined that a change in this policy was necessary to allow course teams to determine which Key Skills were necessary for entry to different progression routes in higher education. The chain of decision-making which brought about the change does not appear to be that which would be expected from the AVA's constitution. Similarly, the decision to incorporate 'The experience of preparing for and sitting an exam...' and ensure that 'Every newly validated [Access] programme should therefore include this experience and list it as an essential completion criteria', was 'agreed' by the APC. The minutes record that this decision should be submitted to QAC for 'approval', although the associated action point uses the word 'ratification'.

Organisational structures

Premises

42 During the review visit, AVA officers expressed their satisfaction with the move to Oxford Brookes University, citing the increased accommodation and access to facilities and services of the University as particular advantages. The Director and Chair of OxOCN also believe that its location on the University's premises will help the AVA assert its independence from the local authority and any one FE provider. The AVA has only recently completed the transfer of its staff and office base to the University and some arrangements are still to be finalised following the transition.

Management, administrative arrangements, and officer roles

43 The AVA is managed by a full-time Director who is responsible for a team of two part-time Development Officers (0.75 each, one dedicated to Access work for a designated third of her time), and two full-time administrative staff. The AVA would appear to have an adequate relationship between its FTE establishment staffing and its size and operational scope. The Director and his staff are directly accountable to the Executive Committee, producing regular monitoring reports across the range of the AVA's activities. The current Director was appointed in summer 1999. He is the sixth director of the AVA since 1994.

44 Committee meetings are serviced by the AVA's officers on an ad hoc basis according to availability and interest. The review team noted that the reporting of the AVA's activities, discussions and decisions was not to a uniform standard, and the quality of minuting and reporting had varied across committees and time, sometimes to the detriment of clarity. The team also noted that while some of the conventions of committee minuting and agenda setting had been used over the years, there did not seem to be a consistent approach currently. While some recently introduced changes to the format for minutes may improve the usefulness of minutes as an internal reminder of outcomes to be addressed, in terms of public accountability (to its members and regulators alike) the AVA should be mindful of the need to ensure its formal records are produced to a standard expected of an awarding body.

45 As the AVA has recently taken on responsibilities it had previously secured through its host, in particular financial management, the need for staff development in this area has been identified by the Executive Committee. Appropriate accounting software has been purchased and the Administrator, who is responsible for keeping the accounts, has received training in its use. OxOCN has committed approximately £3,500 to this area of work.

46 The Director reports to the Executive Committee on financial matters and the Executive Committee's remit includes the responsibility to 'cause proper accounting records to be kept in the office and have records open to inspection by Committee members'. The Committee has requested that quarterly financial reviews are available to it to enable it to fulfil its remit in this respect. The Account describes these reviews as providing a review of 'all financial transactions', although those seen by the review team appeared less detailed than this, giving only a summarised account of income and expenditure.

47 Following discussions with AVA staff, and having scrutinised the AVA's quarterly financial review documents, and having noted that the Executive Committee and OxOCN 'did not have any financial experts within its membership or staff', the review team concluded that the transition to financial independence would require a more robust approach within the governance structures and processes of the AVA than that proposed by the Committee at its first meeting ('inviting persons with such expertise, or for members to suggest or enlist others with a view to co-opting them onto the Executive Committee to help draw implications from any such financial issues and proposals'). The team also noted that the Director's job description incorporated the requirement to 'Develop, implement and refine comprehensive management information systems and reporting procedures with particular reference to financial management and reporting', and would expect this to be reflected in any strategy aimed at raising skill levels.

Summary of structures and processes

48 The AVA's organisational structure is based around its committees and operational activities. The structures exist to facilitate the delivery of the AVA's mission as an OCN and an AVA and support for, and quality assurance of, Access provision is clearly established within them. The operational processes which underpin these structures would appear, on the whole, to be functioning efficiently. The review team was informed, however, that as a result, in part, of recent office relocation and organisational change, the written documentation to support these processes was in need of substantial updating. The AVA intends to replace this documentation with a new OCN operations manual, describing the revised procedures, although this work had not, at the time of the review visit, been completed. The team was concerned that, while the current staff were apparently familiar with the AVA's procedures, key operational processes, and recent revisions to them, depended on this familiarity being lodged with particular individuals rather than being securely documented. With a small staff team, in which individual staff often have single responsibility for key elements of particular processes, an organisation's internal quality assurance mechanisms are especially vulnerable if staff are absent or leave. To obviate such risks, it is particularly important that all of the AVA's key procedures are clearly recorded and that records are kept up-to-date. The team would therefore expect the AVA to progress its plans for the production of an operations manual as a matter of urgency.

49 The team also noted that the AVA does not yet operate a committee cycle with clear scheduling of review, monitoring, reporting and planning activities, thus raising concerns about organisational efficiency and accountability. The review team noted that at its October 2000 meeting the Executive Committee requested that the Director produce '...a Committee Programme document', and would urge the AVA to ensure that the programme adopted provides an appropriate relationship between the AVA's committees, operational procedures and the strategic planning process.

Data management, collection and analysis

50 The AVA has access to extensive data relating to its Access programmes from its own records and database. This is augmented by comprehensive progression and achievement data at university for those students who progress to Oxford Brookes University (the single largest destination for OxOCN Access students), which is provided by the University. The AVA has been systematic in its recording of recruitment, retention and progression data since its inception in 1994. There has been some thoughtful use of data analysis which led to a survey of retention rates in 1997, when the previous year's data had shown a 31 per cent discontinuation rate. The survey argued for more flexible Access provision (by mode of study) and the AVA adjusted its procedures to accommodate this.

51 Since the survey in 1997, the AVA has continued to collect data (including frequently untrapped data on qualifications held by learners on entry to Access programmes) but it is not clear that much use has been made of it for planning purposes. While the review team noted that effective use of the AVA's data had been made in the past, it is concerned to encourage the use of data analysis to inform the strategic planning process.

Communications with providers

52 The AVA has a clear commitment to supporting its Access providers, both through the termly meetings of the APC and in the dedicated function of an Access Development Officer. There are also clear arrangements for points of contact within the AVA's member organisations, usually a programme co-ordinator or manager. Lines of communication for the administration of the AVA's Access provision are clear, usually with examinations officers. There is evidence of extensive and effective networking across the AVA and those involved with delivering Access provision would appear to be well supported and were complimentary of the AVA's staff. The AVA also produces a leaflet to promote Access generally and its members' courses specifically.

Quality development

53 As well as the quality assurance functions discharged through the QAC commented on above, the AVA operates a detailed quality development plan which has been introduced recently. The overall purpose of the plan would appear to be the demonstration of compliance and continuous improvement as driven by licensing requirements from NOCN. OxOCN asserts in its Account that its approach to quality development embraces its Access work, but the review team noted that it was not yet clear whether anything specific to Access would be incorporated that is not already covered through the AVA's annual review of Access. Given the AVA's assertion that quality development in relation to Access is accommodated in this broader plan, the team anticipates that the model itself will develop once implemented for a full year and expects that the position of Access and the AVA's relationship with QAA will be made clearer within OxOCN's Quality Development Plan.

The development, validation and evaluation of Access programmes

Development and validation

54 Although strategic development of Access is the responsibility of the Executive Committee, the actual proposition for the development for a new Access programme ordinarily starts with a provider. As the APC's remit includes statements relating to identifying and promoting 'opportunities for curriculum development', maintaining an 'overview of the delivery and assessment of Access programmes' and 'noting trends and issues which need to be taken up locally and nationally', the AVA may wish to consider how the APC could assist it more fully to develop and meet its strategic objectives in relation to new areas for programme development.

55 New providers receive comprehensive documentation to support their preparation and writing of Access submission documents, including the AVA's Access Award Specification. With its requirements about assessment, key skills, credit targets, credit transfer and exemption and accreditation of prior learning, this provides a measure of consistency across programmes. The Development Officer with responsibility for Access supports providers in writing a programme and subsequently sets up the validation panel. Providers can draw on help from what the AVA terms 'HE consultants', although this term was largely unfamiliar to practitioners whom the review team met, and the team determined that, in practice, these individuals are more an informal network of contacts working in HE than 'consultants'. The validation panel procedure operated by the AVA mirrors standard procedures operated by other OCNs around the country, and HE representation is required at all Access to HE panels.

56 Validation panels for Access programmes are always chaired by the OCN's Director. While the review team acknowledged the potential benefits, in terms of consistency, of the same person chairing all panels, it was not convinced that it was either necessary or desirable for the OCN's Director to be the sole individual carrying this responsibility. The team considered that the essential principle of the impartiality of the Chair might be open to question if the individual concerned was the Director of the organisation which would itself benefit from the recognition of the programme, and that, further, any such potential risks would be exacerbated if the position of Chair were reserved for a single individual. If the AVA wishes to retain this as an officer responsibility, rather than appointing panel Chairs from among appropriately experienced personnel within the AVA's membership, the team would advise that the AVA considers the validation process as a whole, and introduces checks and balances that will ensure that are there can be no possibility of undue influence from any one individual.

57 Examination of panel reports suggested that there was not always a clear understanding by panel members of the difference in status between 'conditions' and 'amendments'. This lack of clarity was identified by the QAC at its meeting in February 1999, when it noted that it was 'often a matter of interpretation as to what was reported as a condition and what as an agreed amendment'. Although, in practice, programmes are not approved until all changes suggested by a panel have been met, the review team considered that the distinction between the two types of outcome should be explicit to panel members and to the officer compiling the notes of the panel meeting. The team was concerned that the nature of any required follow up action should be absolutely clear to the provider and that those responsible for monitoring those changes should be certain as to whether they should be seeking evidence of a substantive change to the proposed programme or a textual amendment to the documentation.

58 The precise location of responsibility for the formal recognition of a new Access programme by the AVA is not clear. The Account states that 'overall responsibility for the accreditation of Access programmes rests with the OxOCN Executive Committee', although it was unclear to the review team how the Executive Committee exercised its responsibility in this respect. The terms of reference of the QAC include the remit to 'monitor the process for the award of kitemarked access certificates' but do not include any responsibilities in relation to programme approval. The team noted that the QAC received panel reports, and that QAC representatives whom the team met considered that the QAC held responsibility for the approval of programmes on the recommendation of officers. The officers and practitioners interviewed, however, asserted that the final decision lay with the panel, with the QAC receiving a list of programmes only in order that they might select a sample to check that the process of validation was being properly conducted. The team noted that minutes of meetings in 1998 provided examples of occasions when the QAC had formally approved programmes by ratification of panel decisions, but that panel reports had not apparently been presented to the QAC for this purpose more recently.

59 The team considered that this matter should be unambiguously resolved to ensure that the responsibility for the final approval of an Access programme lay with the AVA itself, rather than with a series of panels or with the AVA's officers, and to ensure that all those involved in the process were clear as to their responsibilities in this respect.

Monitoring and evaluation

60 The system of monitoring and evaluating programmes occurs at different levels within the AVA and it was clear from documentation and discussion that practitioners, students, moderators, course tutors and AVA officers all provided input into the process of monitoring and evaluation.

61 Monitoring of Access courses is addressed through moderators' reports and annual course reports, which require tutors to action plan for the coming year. Their reports require them to comment on delivery and management of courses, quality assurance, internal moderation, and consistency of standards between different centres. Course tutors report back on the moderation procedure in their reports. They also comment on recruitment, retention, course management, and organisation. Moderation and course reports feed into the APC and QAC via the process of annual review.

62 In addition, the APC maintains an overview of Access through a standing agenda item when Access programme coordinators report on individual course progress, including commenting on matters such as recruitment, learner profiles, retention and moderation. While this discussion undoubtedly provides an important forum for sharing experience, and can sometimes serve to answer particular enquiries from practitioners, this activity is not a part of the AVA's formal quality assurance procedures.

63 Although the annual review of Access programmes provides an opportunity for the QAC to consider matters raised by moderators and/or providers, the committee's remit in relation to this exercise is not entirely clear. The minutes of the QAC's meetings record the committee's expression of 'concern' about some matters which relate to the quality of courses, but the committee's only recorded action points relate to improvements in the monitoring process.

64 In its Account, the AVA notes the introduction of a quality development plan for the OCN as a whole and asserts that 'This more systematic approach to monitoring and review will improve quality across the whole operation, as well as indicating specific areas for improvement'. The review team would suggest that, in its review of self-assessment, the AVA considers particularly carefully how its monitoring processes can actually lead to the assurance and enhancement of the quality and standards of the Access provision for which it has responsibility.

Assessment, moderation and award of certificates

Policies and guidance on student assessment methods

65 The AVA produces comprehensive guidance documents for general programme validation which offer a series of prompts on assessment methods as well as a new Guide to Assessment Practice. These are augmented by an appendix specific to Access which offers a clear policy on examinations in Access programmes. In combination, these documents demonstrate a robust approach to ensuring that assessment methods are appropriate and valid. The adequacy and fitness for purpose of assessment methods and instruments is judged at the recognition panel and monitored by moderators.

Selection, appointment and training of moderators

66 The AVA has a dedicated officer for moderation who is the same person providing support to Access, with this function consuming the remaining two-thirds of her fractional post. Moderators for Access programmes are identified by the AVA, required to complete application forms and to provide references as part of the approval process. Although the QAC is formally responsible for the appointment of moderators and the AVA notes in the Account that the work 'is delegated to the Access Development Officer', the review team could not establish where and how the QAC discharged its formal responsibility.

67 There is a clear commitment in the AVA to using staff predominantly from HEIs to moderate Access provision and to encourage the moderation of more than one centre where possible. The AVA sees this as a useful mechanism to ensuring the continuing fitness for purpose of its Access provision and to provide a structured means by which standardisation can be facilitated. The review team noted that 36 per cent of moderation was by staff from institutions outside the AVA and commends this approach to securing externality in the process. The team also noted that 41 per cent of moderation was by staff from Oxford Brookes University, which is also the largest single recipient of the AVA's students. The team was persuaded that, on balance, there was no undue influence on the judgement of academic standards by one receiving institution, but would encourage the AVA to diversify its moderation pool further.

68 Following appointment by the AVA, moderators are offered an initial one-to-one training session and provided with a contract and guidance documentation outlining their role and responsibilities. Although the guidance documentation is titled Moderating Access to Higher Education Programmes, the guidance is largely generic and does not cross refer to the separate assessment regulations for Access incorporated in the Access Specification document (Programme Submission Guidelines, Appendix: Access to Higher Education Submissions), in particular, the function of an assessment board. An annual meeting for Access moderators is held to discuss areas of concern and to raise issues for discussion by the QAC.

69 The Annual Review of Programmes states that there are 'broadly two models for final moderation meetings' but although it identifies one of these as providing greater objectivity, there does not appear to have been any action taken to bring a single consistent approach to this process.

Receipt of and response to moderators' reports

70 Moderator reports are received in the AVA office and 'approved' before being sent on to the moderated centre. They are subsequently used as the substance of the annual review of Access. The commentary of the annual review report for 1999-2000 identified generic process issues requiring further development, and asserted that academic standards were appropriate and consistent across centres (although the team also noted that the moderation report format does not lend itself to clear statements from moderators that the requisite academic standards have been achieved by all learners). What was less clear, and confirmed in discussions with moderators, was the process for alerting the AVA to issues of concern prior to the final moderation meeting and mechanisms deployed by the AVA for monitoring concerns once reported. There did not appear to be a systematic approach to providing early warnings to the AVA: interim reports are noted as being forwarded to internal moderators but it is not clear how they feature in the AVA's quality control procedures.

Procedures for the award of the Access certificate

71 The review team noted that although there was an apparently clear line of decision-making and quality control process in the award of the Access certificate, there was potential for confusion, contradiction and error. The guidelines for Access moderation referred to above state that 'The EM has the final responsibility in authorising the award of credit to learners' but also that 'OxOCN will accept the decision of the EM as final and issue credits accordingly unless the tutor or IM exercise their right to appeal.' Scrutiny of documentation and discussions with moderators did not identify clear guidelines from the AVA on the nature of appeals which would be allowed, and whether appeals could be made against a moderator's academic judgement, as well as on procedural grounds. Although those moderators spoken to were sure of their ultimate authority and the facility to seek guidance from the Access Development Officer if necessary, this view was not entirely consistent with that expressed by Access practitioners with whom the team met, some of whom took the view that their 'right to appeal' included a right to appeal against a moderator's academic judgement.

72 The AVA operates a model where all evidence contributing to the award of an Access certificate undergoes a series of checks prior to certification. Initial judgements made by tutors are verified by an internal moderator; a sample of work is scrutinised by the external moderator to ensure academic standards are being met; systems used to ensure all required completion criteria are being met are checked; an 'award of credit' form is signed off by the moderator and forwarded to the AVA; and AVA staff check for accuracy and certificates are produced accordingly.

73 The AVA's common specification for the award of the Access to HE certificate aids the process of checking by moderators and AVA staff. AVA staff check that the correct number of credits have been achieved at the right level but assume that any requirements for mandatory units have been checked by the moderator, as there is no automated procedure in the AVA's database. The review team noted that the Director had ultimate responsibility for the accuracy of this process and that he received every certificate for checking before it was despatched. While the Director's involvement in this aspect of the process may signify the AVA's recognition of the vital importance of accurate certification, the team would query whether an essentially mechanical process of this kind need be located at this level in the organisation. The team would suggest the AVA considers how it might most effectively minimise the potential for human error, and implement a rigorous and economical procedure for quality controlling the award and issue of Access certificates.

Learner experience, standards and progression

74 Students met by the review team were positive about the Access courses they had completed. They believed that the Access courses that they had undertaken had met their objectives in preparing them for HE, and were anxious to commend the dedication and support of Access tutors.

75 Monitoring of the learner experience by the AVA is achieved through a number of mechanisms. Although there is some inconsistency in the way written feedback is given, all tutors and moderators provide feedback. The Access to HE Course Report Guidelines asks tutors to comment on how they obtained feedback from students and how they responded to comments. Under the heading 'Action Plan', tutors are also asked to list 'criticisms or suggestions arising from student evaluation', and to indicate timescales and responsibilities for action. Course reports for 1999-2000 considered by the review team suggested that students were, on the whole, satisfied with their courses. In addition, the students met by the review team were unanimous in stating that they had been given the opportunity to provide input into the development of their respective courses and that their concerns had been addressed by course tutors whenever possible. The moderation report is structured to accommodate moderators' suggestions on 'improving the learner experience', and moderators use a variety of methods to gather feedback from students, including formal evaluations, group discussions and video recordings.

Progression and retention

76 Statistics show that the number of students progressing to HE from OxOCN Access to HE programmes has increased since 1994. Out of 447 registrations during the 1999-2000 period, 229 students were awarded Access certificates, 160 students progressed to HE and, of those, 86 went on to study at Oxford Brookes University.

77 The University's effective tracking and monitoring system of Access students, shows that there was a retention rate of 90 per cent among first year students who completed Access to HE courses in 1999. The team was informed that the statistical tracking of students undertaken by Oxford Brookes University over the last 10 or 12 years also demonstrated that Access students had done very well at the University, and that they had proved to be excellent students with high levels of achievement and retention. HE admissions tutors supported this conclusion, and expressed the view that, in their experience, Access students performed as well as students who had progressed to HE from other routes.

Conclusions

78 In recent years, Oxfordshire Open College Network has been required to make considerable adjustments to its structures and systems because of changes in the context within which it operates, and demands from the regulatory bodies to which it is accountable. At this stage, parts of the structure of the organisation, and a number of its procedures, are somewhat fluid as the AVA responds to these demands, identifies its own priorities for development, and attempts to maintain some of its distinctive strengths.

79 Among these strengths has been the OCN's focus on its AVA activities, and the commitment and involvement of Access practitioners and those receiving Access students in higher education. Both Access practitioners and representatives from HE are appropriately involved in the work of the AVA at various levels, and there are clear lines of communication with providers. The AVA supports practitioners through comprehensive and well-produced documentation in relation to Access which provides helpful guidance on a range of matters including assessment and a clear approach to standardisation.

80 There are some unresolved tensions between the previous model of full practitioner participation in policy and decision-making, which was largely self-regulating, and the current model of governance through a committee structure supported by a team of full-time paid officers. As a result, while the move to a more professional and publicly-accountable organisation has undoubtedly brought about improvements, there is a certain degree of uncertainty about the locus of decision-making within the AVA, not only by those who are affected by the decisions but by those who are responsible for making them. In particular, the locus of decision-making for Access programme approval is not well defined and, with some lack of clarity about what constitutes an allowable appeal on a judgement about the award of an Access certificate, the final locus of authority for Access is somewhat unclear. In addition, changes have also brought about greater responsibility for the AVA in terms of financial planning and monitoring, and the AVA's mechanisms for these are not yet certain or secure.

81 The current governance and committee structure is recognised as still evolving, and there is some recognition that chains of decision-making and accountability are not always clear. In a number of areas, details of organisational and committee structures need further clarification to ensure proper distinction between operations, management, governance, decision-making and accountability. Further work is required, therefore, to ensure that the viability of the AVA as an organisation which understands and is in control of its own decision-making structures, is not compromised by conflicts between operational, management and governance procedures and its constitutional obligations, nor for the committee structure to produce duplication of effort or distractions from a committee's core functions.

82 There is a clear commitment to improving the quality of the organisation's functions and processes as a whole, and some of the detail of the AVA's quality assurance procedures in relation to monitoring and review are well developed. However, there is a need to clarify the position of Access in the AVA's approach to quality development to ensure that in adopting strategies for the whole organisation, OxOCN does not overlook the role of Access and the separate regulatory requirements which pertain to it. At a superstructural level the inclusion of Access is quite clear: there is a separate strategic aim that relates exclusively to it. What is less clear, is how the AVA will ensure that Access is embedded in its new quality processes in a manner that will not detract from the focus which it has traditionally received.

83 While there are extensive monitoring processes in place, which make use of a variety of mechanisms, there is some lack of clarity about the particular responsibility and mechanisms for ensuring that action follows as a result of monitoring processes, including the relative responsibilities of the APC and QAC. Some weaknesses at this point lead to a weakness in the link between quality assurance and quality enhancement and the AVA has yet to take full account of the outcomes of its monitoring and review processes in the process of strategic planning in relation to Access.

84 There are other areas in which the process of strategic planning is currently under-developed, such as the use of the data available to it. The AVA benefits from extensive data gathering, both that which it has conducted itself and that which is supplied by Oxford Brookes University, which has kept and provided to the AVA regular data about Access students progressing to the University. In view of the richness and extent of this data, it is therefore disappointing to note that little use is made of any detailed analysis of the data in the AVA's forward planning.

85 The AVA has made good use of input from other stakeholders to inform its thinking in the strategic planning process, and the AVA has considered a number of broad influences from recent changes in government policy and the employment market, which may influence the future for Access students and potential Access students. These potential influences include both matters of fact and matters of conjecture which together have given a particular steer to the development of the OCN's strategy for Access. With the passing of time, as the actual force of some of these influences becomes clearer, the AVA may benefit from giving this further consideration and debate within the AVA to ensure that there is a clear and shared sense of the AVA's direction and the full rationale for this direction, and to ensure that policy fully reflects the fundamental purpose of Access to HE courses, and the joint interests of Access students and member institutions.

86 The review team commends the AVA for:

i the comprehensiveness of the annual review of Access programmes;

ii the steer provided to ensure that students have a voice in the processes for the review of Access programmes.

Recommendation to the ARLC

87 The review team recommends that OxOCN be granted a provisional renewal of licence with conditions to be met by the dates specified below and a revisit in spring 2002.

Conditions

88 Subject to the approval of the ARLC, OxOCN's licence is provisionally renewed on condition that the AVA:

i confirms the Memorandum of Agreement with Oxford Brookes University to establish the essential independence of the AVA, and to clarify responsibility for the appraisal and staff development of the Director;

ii reviews the Constitution, and committee terms of reference, names, functions and relationships and current practice; and revises documentation or practice as necessary to ensure consistency between constitutional obligations and operational practice;

iii produces and makes available the document in which all of AVA's current operational procedures are described;

iv demonstrates how the strategic planning process relates to its committee cycle to ensure synergy between review and planning and effective use of committee time, and the provision of advice to those responsible for producing and approving the plan;

v reconsiders the allocation of responsibility for chairing validation panels, in order to ensure and demonstrate appropriate independence and impartiality of judgements;

vi clarifies the locus of authority for the formal recognition of Access to HE programmes;

vii provides clarification on the mechanisms used to monitor and respond to issues of concern before and after the production of the final moderation report;

viii reviews procedures for quality controlling the award and issue of Access certificates to ensure that they are robust and reliable;

ix implements the structures and procedures resulting from meeting conditions i - vii, to ensure that they are fully operative and effective by the time of the revisit.

Conditions i - vii to be met by 30 June 2001.
Condition viii to be met by the date of the revisit in spring 2002.

89 Provisional confirmation of licence allows the AVA to continue to operate under licence but makes provision for the immediate withdrawal of the licence if the conditions are not met in the time allowed and to the satisfaction of QAA. Were it to be necessary for QAA to withdraw the licence, the AVA would need to put in place procedures for the transfer of registered students to an alternative licensed AVA.

Recommendations

90 The review team recommends that OxOCN:

i reviews its approach to data analysis as part of the strategic planning process;

ii reviews its quality development plan to ensure its regulatory relationship with QAA is incorporated;

iii considers its strategy to raise the skill base in the OCN's financial management in relation to officers' agreed job descriptions and roles;

iv reviews the approach used for the servicing of committees, and implements a strategy to ensure that all minutes, agendas and formal documentation are produced to a consistent and professional standard;

v evaluates the composition of its moderation pool and confirms through the QAC that there is no undue reliance on the host organisation, or proposes a strategy for diversification;

vi considers its reporting cycle to ensure that the results of the Access to HE annual review of programmes can be fed into the annual report prepared for QAA;

vii clarifies its policy in relation to appeals against moderators' decisions;

viii considers whether there is a role for the APC in the AVA's strategic planning processes in relation to Access programme development.

Oxfordshire Open College Network May 2001

Contents