Foreword
1 The Quality Assurance Agency for Higher Education (QAA) is responsible to the Department for Innovation, Universities and Skills for the recognition of Access to Higher Education courses. QAA exercises this responsibility through a national network of Access validating agencies (AVAs), which are licensed by QAA to recognise individual Access to HE courses, and to award Access to HE qualifications to students. The AVAs are responsible for implementing quality assurance arrangements in relation to the quality of Access to HE provision and the standards of student achievement. QAA has developed a scheme for the licensing and review of AVAs, the principles and processes of which are described in the QAA Recognition Scheme for Access to Higher Education in England, Wales and Northern Ireland (Recognition Scheme). The Recognition Scheme is regulated and administered by the Access Recognition and Licensing Committee (ARLC), a committee of the QAA Board of Directors.
2 The ARLC is responsible for overseeing the processes of AVA licensing and periodic review and relicensing. The criteria applied by the ARLC and by review teams operating on the Committee's behalf, in reaching judgements about whether and under what terms an AVA licence should be confirmed or renewed, are provided within the Recognition Scheme documentation. These criteria are grouped under the four principles that provide the main section headings of this report.
3 Following an AVA review, a member of the review team presents the team's report to the ARLC. The Committee then makes one of four decisions:
- unconditional confirmation of renewal of licence for a specified period
- conditional confirmation of licence with conditions to be met by a specified date
- provisional confirmation of licence with conditions to be met and a further review visit by a specified date
- withdrawal of licence for operation as an AVA.
4 This is a report on the review for Open College Network North East Region (OCNNER). QAA is grateful to OCNNER and to those who participated in the review for the willing cooperation provided to the review team.
The review process
5 The review was conducted in accordance with the process detailed in the Recognition Scheme. The preparation for the event included an initial meeting between OCNNER representatives and QAA's Assistant Director to discuss the requirements for the Overview Document (the Overview) and the process of the event; the preparation and submission by OCNNER of its Overview, together with a selection of supporting documentation; a meeting of the review team to discuss the Overview and supporting documentation and to establish the main themes and confirm the programme for the review; and negotiations between QAA and OCNNER to finalise other arrangements for the review.
6 The review visit took place on 8 to 11 May 2007. The visit to OCNNER consisted principally of meetings with representatives of OCNNER, including AVA officers; members of the Board of Trustees, the Quality Committee, the Access to HE Committee, the Strategy Group, moderators for Access to HE programmes; providers of Access to HE programmes and receivers of Access to HE students.
7 The review team consisted of Professor Reginald Davis, Emeritus Professor of Chemistry (Kingston University), Sara Jennett, Director of Quality Support (University of Plymouth), and Alan Smith, Access Co-ordinator (Open College Network Wales) and lead reviewer. The review was coordinated for QAA by Matthew Cott, Assistant Director.
The AVA context
Background and major developments since the last AVA Review
8 Access to HE in the north-east of England developed during the latter half of the 1980s at a number of further education colleges. By 1989, the polytechnics of Sunderland and Teesside established the Tees-Wear Access Federation which had representatives from many of the region's Access to HE providers. Following the development of a number of open college networks in the region during the 1990s, the Tees-Wear Access Federation transferred its authority to the Teesside Region Open College Network (TROCN) in 1995. With the national network of open college networks (OCNs) moving towards a regional structure, the Open College Network North East Region (OCNNER) was formed in August 2005 by the merger of TROCN and the North East Open College Network (NEOCN). The latter covered the region from Tyneside to the Borders, but did not hold an AVA licence. Therefore, the AVA licence held by TROCN now operates within OCNNER, covering the north-east of England (Tees Valley to the Borders), Cumbria and some parts of southern Scotland.
9 The previous review of the AVA, conducted by QAA in spring 2002, resulted in a conditional renewal of the licence with a number of conditions which were met, and the licence was confirmed in July 2004.
10 The major development since the previous review has been the merger of TROCN and NEOCN. The merger necessitated a series of other major changes. There was a need to restructure the organisation to accommodate the wider range of members and programmes offered by OCNNER. A new staffing structure was put in place in April 2007 and new premises acquired and occupied as of March 2007. The local management information system (MIS) has now been replaced by the National Open College Network's (NOCN) MIS (Opus), although this has not been achieved without difficulties, some of which are ongoing. The constitution of a new Board of Trustees was accompanied by the establishment of a new committee structure.
Regional Access to HE frameworks
11 Recognition of the distinctive nature of Access to HE students and providers in OCNNER's two regions (north-east England and Cumbria) led to the establishment by the AVA of two separate regional Access to HE frameworks. The OCN uses the terms 'Regional programme in Access to HE' to describe its framework in the north-east and 'Cumbria Access to HE Framework' for its framework in Cumbria. In this report the term 'regional frameworks' refers to both frameworks, unless it is necessary to differentiate them.
12 The north-east region has a well defined series of Access to HE awards delivered entirely through further education colleges (FECs). Each award prior to the introduction of the regional framework had its own distinctive structure and set of constituent units. A rationalisation process has accompanied the creation of the regional framework, whereby it has been possible to reduce the total number of units considerably and to permit only this subset of units to be used in existing and new awards, now termed 'pathways'. This approach has been embraced by providers who now deliver pathways made up of units considered to be the best of their type available within the AVA. A further advantage has been that higher education institutions (HEIs), who were involved in the development and approval process, now have a clearer picture of the content of both units and pathways. This gives the HEIs greater clarity over the subject knowledge and standards achieved by Access to HE students, regardless of the location where the student has achieved the award.
13 The nature of the region, with its broad geographic spread of small communities, provides a challenge to both those wishing to progress to HE and to the providers of Access to HE provision. In Cumbria, only the Cumbria Institute of the Arts and in the south of Scotland, Dumfries and Galloway College, currently deliver QAA-recognised Access to HE awards. The units used by both providers have been included in the pathways constituting the Cumbria Access to HE Framework. Therefore, the focus of the Framework in Cumbria was the development of a scheme which would encourage students to consolidate their existing achievements, gained from other OCN programmes, and turn them into a full Access to HE award. Parts of the curriculum that are included in the Framework are delivered by community-based organisations such as Cumbria Learning Links and Cumbria Youth Alliance. Nevertheless, encouraging progression into HE remains a challenge in the region and OCNNER continues to work strategically to raise awareness of the potential of the Cumbria Access to HE Framework, particularly in the context of the development of the University of Cumbria. It is relevant to the present review to note that there is a national imperative to align OCNs with regional development agencies (RDAs) and local Learning and Skills Councils (LSCs). In this context, the Cumbria region aligns with the north-west of England. At the time of the review, discussions were well advanced between OCNNER and OCN North West Region (OCNNWR) regarding the transfer of the Cumbria provision to OCNNWR. Providing details could be agreed, this transfer was expected to be effective from September 2007.
14 With the exception of Dumfries and Galloway College, all Scottish colleges engaged in Access to HE provision, managed by OCNNER, deliver the Trades Union Congress (TUC) National Framework for Access Programme (for which OCNNER is also the national lead organisation). These awards are not part of either regional framework. Although historically this programme has been popular with students, most wish only to follow specific units needed in their professional lives and few progress to HE. It has therefore been agreed with the TUC that its Access to HE programme will not be offered to new students from September 2007. The award will be moved into the Qualifications and Curriculum Framework to create a Level 3 diploma, but this will not be an Access to HE award.
AVA Statistics 2005-06
15 The AVA reported the following statistics in its annual report to QAA for 2005-06:
| Providers offering Access to HE awards | 22 |
| Access to HE awards available | 74 |
| Access to HE awards running | 59 |
| Access to HE student registrations | 1,493 |
| Access to HE certificates awarded | 671 |
Principle 1
The organisation has governance structures which enable it to meet its legal and public obligations, to render it appropriately accountable, and to allow it to discharge its AVA responsibilities securely
Constitution and legal identity
16 OCNNER is a fully independent, legally constituted company, responsible to its members. Its Memorandum and Articles of Association amount to the formal constitution of the AVA. These contain details of the AVA's legal identity, function, aims, structures, membership and liabilities. The AVA's aims are congruent with the overall purposes and aims of the Recognition Scheme. Members are entitled to send a representative to the annual general meeting and to elect a Board of Trustees.
17 The governance structure ensures an appropriate level of formal accountability. The NOCN and QAA licences set the framework within which OCNNER operates. Apart from these, there is no restriction on its decision-making or on the operation of the AVA.
18 The AVA is a company limited by guarantee, operates in accordance with the Companies Act and has charitable status. A fully audited set of accounts is prepared each year and submitted to Companies House and the Charities Commission. The Memorandum and Articles of Association and the status as a limited company limit the legal liability of members and their representatives.
19 A Register of Interests is maintained and trustees are required to declare any conflicts of interest as part of all meetings. All trustees have a signed job description, which requires them to operate within the Nolan Principles.
Membership
20 The AVA is a membership organisation and all members have voting rights. Members include HEIs, FECs, community and voluntary organisations, and private training providers. At the time of the review, 17 members were offering Access to HE programmes within the regional frameworks. Of these, 14 were located in the north-east of England, two in Cumbria and one in Scotland. One of these, together with a further four colleges, offers the TUC National Framework for Access Programmes. The review team met representatives of a range of these stakeholders and noted their commitment to the aims of Access to HE and their support for the work of the AVA.
21 Since the merger of the two OCNs, membership of the AVA has remained fairly stable. Beyond what is set out in the Articles of Association, the review team was not made aware of the actual procedures for organisations seeking to gain membership of the AVA or, indeed, procedures for the removal of members. The team recommends that the AVA publishes its formal membership application and removal procedures.
Governance
22 The Board of Trustees is the locus of authority for the operation of the AVA and the maintenance of its AVA licence. Nine trustees are elected at the annual general meeting and these elected members may co-opt up to six further members. The normal term of office of a trustee is not clearly specified in the Articles of Association, although the review team was informed that it is three years. One-third of the elected trustees are required to retire annually. However, retiring members can be re-elected without limit. The team recognises that retaining trustees over a longer period can bring benefits in terms of experience and continuity and that, on occasion, it may prove difficult to identify suitable replacements. However, in the interests of effective governance, the team recommends that the AVA sets a limit on the number of terms of office that a trustee may serve.
23 Although the composition of the Board at the time of the review, was broadly representative of the OCN's membership, with HE and FE and other stakeholders represented, there do not appear to be criteria within the Memorandum or Articles of Association or any other guidance covering the categories of representation required on the Board. The review team formed the view that without this, the constitution incorporated no guarantee that appropriate levels of stakeholder representation would be maintained and offered no protection from the undue influence of any one of, or a minority group of, its stakeholders. It is therefore a condition of licence that the AVA should, in its Memorandum and Articles of Association specify the categories of stakeholder representation and the criteria for their appointment or election to the Board.
24 The merger of the two OCNs was reported to the review team as bringing stronger membership to the new Board of Trustees. The team noted that in the period prior to the review, there was only one Board member in current employment by a receiver HEI and no trustees were drawn from either Cumbria or Scotland. However, the Board had recently recruited a further member from a receiver HEI and an employee of a Scottish college. The latter represents both the Cumbria and the south-west Scotland regions. In respect of further representation from Cumbria, the team acknowledged the impending transfer of the Cumbria provision to OCNNWR. The Board had recently conducted a skills audit of its present members, and, while no specific skills deficits were identified, members with financial and marketing skills were being sought. The team recommends that the AVA continues in its efforts to ensure that the Board of Trustees' membership is geographically representative and includes sufficient current HE and Access to HE experience.
25 New trustees undertake a thorough induction. There is some ongoing training for Board members, but the Board members who met with the review team were not aware of any structured training plan. The team recommends that the AVA strengthens its arrangements for providing ongoing training and development for Board members. The trustees see themselves as proactive supporters of the AVA and its activities and as critical friends to the officers. The team formed the view that the trustees make a valuable contribution to the operation of the AVA.
Committee structure
26 The Board delegates many of its responsibilities to its four subcommittees: the Access to HE Committee, the Quality Committee, the Finance and Business Development Committee and the Personnel Committee. Each committee has a clearly defined remit and membership and each is chaired by a Board member. The Access to HE Committee and the Quality Committee members broaden the range of educational experience brought to the AVA's governance. Committee members are drawn from FE providers of Access to HE programmes and HEI receivers of Access to HE students, together with representatives of private training, and adult and community learning organisations.
27 The Access to HE Committee's purpose is to ensure, through a systematic process of monitoring and review, that all licence and quality assurance requirements relating to Access to HE provision are met. It is also required to oversee the AVA's self-assessment process, ensuring that there are established procedures for monitoring and assessing the continuing quality and effectiveness of its management and operations. Trustees confirmed to the review team that these functions have been delegated by the Board to the Access to HE Committee. The Committee meets regularly and attendance by members is satisfactory. Its agendas are wide ranging and ensure that both strategic and operational matters are fully considered. It also receives and approves the AVA's annual report for onward transmission to the Board. However, consideration by the Access to HE Committee of the annual AVA report does not in itself systematically ensure adherence to the requirements of the Recognition Scheme.
28 From an examination of the Access to HE Committee minutes across the full annual cycle, the review team was unable to find evidence of the operation of a process of monitoring and review against the AVA licensing criteria or any other means of ensuring continuing adherence. Neither were such processes evident from the minutes of the Board or the Quality Committee. The team recognised that the merger of the two OCNs and the implementation of the major changes (see above paragraph 10), with all the immediate strategic and operational consequences that flow from these, had been the recent prime focus of the AVA officers, the Board and its subcommittees. Notwithstanding this, the team found that a number of the licensing criteria (across all four Principles but particularly Principles 3 and 4) were unmet, and yet the Access to HE Committee had not itself identified and acted upon this, as required by the remit delegated to it by the Board. It is therefore a condition of licence that the AVA revises its monitoring and reporting procedures so that the Board of Trustees can be assured that all of the licensing criteria continue to be met.
29 The Quality Committee is responsible for monitoring and reviewing quality assurance arrangements associated with the OCNNER's external licences. Monitoring the quality of the Access to HE student experience and the standard of the award is within the remit of the Access to HE Committee. The outcomes of this Committee's deliberations are further considered as part of the remit of the Quality Committee. Although there are inevitably some elements of duplication in such an approach, the review team concluded that it provides a robust mechanism for the maintenance of academic standards and the enhancement of quality.
30 The first joint annual meeting of the Access to HE Committee and the Quality Committee was held in November 2006. The purpose of the meeting was to provide a joint forum for the consideration of three key annual reports: the AVA's report to QAA, the Quality Assurance Manager's report and the Self-Assessment report. The self-assessment process was still in development and so, rather than consider a report; the 2006 meeting endorsed the intended process. The objective of this joint meeting is to keep under review and advise upon the content of these reports in order to meet the requirements of both QAA and NOCN and to report the joint meeting's recommendations to the Board of Trustees. Members of both Committees reported positively on the joint meeting to the review team. However, the team concurred with the views of committee members and AVA officers that the process was in need of further development (see also below paragraph 43).
31 The review team formed the view that both the Finance and Business Development Committee and the Personnel Committee are effective. The period during which the two OCNs were brought together has required the resolution of delicate staffing issues, the acquisition and occupation of new premises and the creation of a new unified business plan, as well as dealing with the more routine management issues. The two Committees have given AVA officers strong support throughout this period. The team considered that the effort that has gone into effecting these changes should now place the AVA on a strong footing for the future.
32 The Board and the subcommittees have not yet developed fully robust procedures for monitoring their own effectiveness. A system, based on a monitoring grid, has been developed for monitoring the fulfilment of the remits of the Board and subcommittees, as well as representation and attendance at each meeting. The subcommittees monitor against each item of their remits, whereas the Board only monitors those items of its remit which are additional to the responsibilities outlined in the Articles of Association. When comparing each of the monitoring grids against the minutes of the Board or the relevant subcommittee, the review team experienced some difficulty in reconciling the two: it was not always obvious which items of the remit were being met by which agenda items. The team recommends that, since this has been the first year of operating the system, the AVA should review the fitness for purpose of this approach at the earliest stage possible, so that adjustments can be made before the commencement of the next cycle of meetings. Quality Committee members informed the team that they were aware of the need for improvements and described their intention to map their remit against likely agenda items during their annual forward look at meeting agendas. The team considers that the other committees and the Board would also benefit from this approach.
33 The review team identified that the locus of authority for confirming awards to Access to HE students by the AVA is implicitly located with the Board of Trustees, since the Memorandum of Association requires it to maintain the AVA licence. However, Board members told the team that this authority was delegated to the Access to HE Committee and the Quality Committee. However, this delegated authority is not listed within the remit of either committee. Based upon this, and their discussions with Board and committee members, the team formed the view that the designated locus of authority for confirming the Access to HE award was unclear. It is therefore a condition of licence that the AVA makes explicit, within the terms of reference of the Board of Trustees, and the most appropriate committee, where the locus of formal authority for the Access to HE award is located.
Principle 2
The organisation is able to manage its AVA responsibilities effectively, and to maintain an appropriate structure to support them
Strategic planning
34 The review team was informed that the AVA was, at the time of the review, developing its strategic plan for the period 2007-10. A date in June had been set aside for staff to work on the new strategic plan. This was to be followed with a date in July when members of the Board, Access to HE Committee and Quality Committee would meet to finalise the OCN and AVA strategic plan for the next three years. At the time of the review therefore, the organisation was between strategic plans; with the 2003-06 strategic plan developed by TROCN having expired and the new strategic plan not yet created. The TROCN Strategic Plan integrated AVA and OCN strategy into one plan. While the team accepted the policy of integrating AVA and OCN strategy, it is a requirement of the licensing criteria that there is an identified strategy for Access to HE provision and that its implementation and effectiveness is appropriately monitored.
35 The OCN does have a Strategic Business Plan 2005-08 containing eight strategic business objectives. However, only objective four relates directly to Access to HE provision: 'increasing AVA provision in Tyne and Wear by 200 learners'. The Plan does not indicate how this target will be achieved, or the timescale, but it does indicate that progress will be monitored through 2005-06. The review team was unable to find evidence that the results of any monitoring against this objective had been reported to the Access to HE Committee at any of its three meetings between November 2005 and April 2006.
36 The AVA's strategy since the merger has been to concentrate on the development and implementation of the two regional frameworks for the North East and Cumbria regions; the need for this having been identified following the merger in 2005. The process leading to the development of the regional frameworks has been led by a 'Strategic Group', but it was agreed by all AVA officers and other representatives that the group had more of a 'steering' remit than strategic.
37 The AVA has been working to two Access to HE action plans (2005-06 and 2006-07) since the merger with the latter action plan consisting principally of actions carried over from the previous year. These action plans have dealt largely with operational issues and were clearly laid out in respect of responsibility, target dates and completion status. Progress with these plans has been monitored by the Access to HE Committee.
38 The pressures of work faced by the AVA since the merger were recognised by the review team and it is hoped that the new staffing structure, in place since April 2007, will provide a period of stability for the organisation during which long term strategic planning systems can be put in place rather than the current process of addressing pressing operational issues. Representatives from the Board who met with the team acknowledged the need for the strategic planning process to be strengthened. The Overview identified that the responsibility for strategic planning related to Access to HE has been delegated by the Board to the Access to HE Committee.
39 The review team accepted the AVA's intention to integrate its Access to HE strategy within the overall strategy of the OCN, but was concerned that the strategy should already have been identified and the associated operational plans developed. The team considered that a strategic plan for the next three years was a priority for the AVA in view of the considerable changes and challenges that are imminent, and in view of the areas of risk related to Access to HE provision (see below paragraph 45) that were articulated by different groups who met with the team. It is therefore a condition of licence that the AVA develops a strategic plan which includes its AVA responsibilities.
40 It was not clear to the review team exactly how the arrangements with regard to strategic planning makes effective use of the views of Access to HE providers or receivers other than those involved with the Access to HE Committee. While the team heard that AVA staff meet with providers to discuss their annual reviews, the team was concerned that these meetings would not happen until the autumn, by which time the strategic plan would have been developed.
41 It was clear from discussions with officers, members of the Strategic Group and providers, that the AVA had used its analysis of statistical data to advise providers in relation to the use of units when they were being considered for inclusion in the two regional frameworks. Providers informed the review team, however, that rather than the AVA advising providers with regard to the development and targeting of their Access to HE programmes, they (the providers) were in a better position to advise the AVA of developments necessary.
Monitoring and assessing management, operation and risk
42 As already noted, the Board and each subcommittee have clear remits and terms of reference. The AVA maintains a tracking grid on which is recorded the link between the minutes and the remit of the committees. The review team, however, found it difficult to correlate the confirmation of committee remits with the committee minutes. The Board has the remit of ensuring that all the external licensing criteria are met but this was not tracked to any level of detail and several gaps were identified by the review team.
43 While the AVA carries out its self-assessment review for QAA each year, and the OCN produces its own review for NOCN, the self-assessment process that provides information for these reviews and to enhance organisational quality and leadership had only recently been approved. Although the joint annual meeting between the Access to HE Committee and the Quality Committee is designed to ensure that the report from this process is considered alongside the Quality Manager's report and the annual AVA report to QAA, the team considered that further thought should be given to the self-assessment process. In particular, this should take account of the timing and inputs (see also below paragraph 44) required in each of the three reports so as to ensure a comprehensive, evidence-based consideration of all relevant issues. The team therefore recommends that the AVA implements its intended self-assessment process and coordinates the sequence of reports so that the Quality Assurance Manager's report and Self-Assessment report can usefully inform the development of the annual report to QAA.
44 The AVA does not require providers to produce an annual evaluative report of the previous year's Access to HE provision, relying instead on an annual review meeting between the provider representatives and an AVA officer. The review team considered that this annual course review process should be revised (see below paragraph 75). In revising the process, the AVA should consider the timing of the outcomes from providers' own annual course review processes so that this information can usefully inform the AVA's own planning processes, as reflected in the annual AVA report to QAA.
45 Various groups who met with the review team identified several areas of risk that are likely to affect Access to HE provision in the near future: these areas included financial considerations, competition from other AVAs and from Foundation Degrees and the effect of these factors on Access to HE numbers, funding decisions, and possible problems keeping providers 'on board' with the changes being made to Access to HE provision. A member of the Board explained to the team that the areas of risk were not of equal weight, and that some risks also presented opportunities. In the 2005-06 annual AVA report to QAA, the OCN Business Plan for 2006 identified risks related to funding issues and competition from other awarding bodies, but did not articulate these risks in relation to Access to HE. The response related to these stated risks also gave no indication of its relevance to the Access provision which accounts for 14 per cent of the business of the OCN. While the OCN conducts a risk assessment, the team found that it was insufficiently specific in relation to Access to HE and the measures proposed to address the risks identified were not precise enough to be effective or capable of being measured or monitored. It is therefore a condition of licence that the AVA develops appropriate procedures for recognising and responding to risks associated with Access to HE.
Committee administration
46 The Board, which reports to the members' annual general meeting, has four subcommittees (see above paragraph 26). Each has a clear statement of membership, terms of reference and mode of operation. The minutes from the Board and each committee are clear and well documented. The style of numbering each minute enables speedy cross referencing and action tracking. Each set of minutes concludes with an action plan; progress against the actions is confirmed and signed off by the Chair at subsequent meetings.
Financial management
47 The Finance and Business Development Committee reports to the Board on the financial management and status of the OCN. There are clear processes involved in the financial management of the AVA with external auditors appointed and final accounts, which are agreed by the Board and annual general meeting, being forwarded to QAA, Companies House and the Charities Commission.
48 The review team noted that there are no separate accounts for the AVA, the only identifiable financial information being the income from student certification charges; however, the Overview identified that certain costs, such as Access to HE moderation, can be separated out. The team considered that the AVA could benefit from knowing how its Access to HE related expenditure compared with its income and that a more systematic recording of AVA expenses would enable a more thorough analysis to be made of risks or opportunities associated with changing Access to HE numbers. The team therefore recommends that the AVA develops an Access to HE specific budget.
Staffing and physical resources
49 The AVA has recently undergone a change to its staffing structure and has moved to spacious new office premises. The revised staffing structure aims to make the organisation more efficient and effective in its business operations. All members of staff have clear reporting lines and job descriptions. Staff members holding AVA-specific responsibilities are recognisable both within the organisational structure and their individual job descriptions.
50 All members of staff take part in an annual appraisal and training and development needs are identified as part of this. Staff commented favourably on staff development opportunities available to them. The OCN operates an effective process for dealing with requests for staff development opportunities.
51 The physical resource identified by the AVA representatives as being least satisfactory was the Opus database and the associated Business Objects software. AVA representatives considered these to be ineffective and often inaccurate in the data requested in reports. The review team recommends that the AVA continues to develop its student data systems, including its capacity for cohort analysis and maintains its discussions with NOCN to seek improvements to the Opus database.
Published operational procedures
52 In most instances the review team found that there were clear procedures and guidelines available to staff and providers for the registration of students, final award boards, checking and award of certificates, moderation, accreditation of prior learning (APL), complaints, disciplinary and mitigation processes.
53 Details of the APL and mitigation processes had been submitted to the Access to HE Committee for adoption by the AVA, but it was not entirely apparent when other processes, or their associated documentation, had been confirmed by any of the OCN's committees. The team heard from the Board representatives that approval of documents by committees depended on the significance of the document, and that 'the Board had an understanding with the officers'. The team considered that the process by which procedures and guidelines are approved by the AVA lacked rigour and transparency. This should be addressed by ensuring that all definitive AVA documents are approved by the relevant committee.
54 The Regional Programme in Access to HE Centre Handbook is a useful, succinct document. It covers in outline the areas that a provider needs to be aware of when running an Access to HE programme: however it does not address the development or approval of new pathways. The absence of clear written guidance or direction relating to framework/pathway validation, reporting and revalidation was a concern for the review team (see also below Principle 3). There are brief documents which outline the composition of validation panels and the roles of those attending, and guidelines for providers attending panels, but no document which clearly details the remit and responsibilities of a validation or revalidation panel or which identifies the differences between the two. The Regional Programme in Access to HE Centre Handbook does, however, give brief details of the procedures for reviewing the programme. The guidance provided to validation panels indicates that programme approval could be confirmed by the panel; however, the review team noted that the panel's remit is to make a recommendation for approval to the Access to HE Committee.
Data collection, recording and holding
55 The NOCN Opus system allows for the registration of students and the recording of their programme of study on a unit by unit basis, with levels and values of units recorded. The programme has been set up to contain the credit tariff and rules of combination for each pathway so that once the student has accrued the correct number of credits in the appropriate combinations the award of the Access to HE qualification is confirmed. Unit details are recorded against specific pathways for each provider and it is possible to track frequency of use of a unit and which providers have the unit in any of their pathways. Unfortunately, as the reporting element of the Opus system does not operate reliably, when accurate statistics are needed, the AVA officers have to check all outputs from the database against hard copies of student registrations and achievement of credits to date.
Communication with stakeholders
56 In its meetings, the review team heard that the AVA ensures that stakeholders are well informed and up-to-date with AVA-related news and regulatory matters. In addition to face-to-face meetings, the AVA uses newsletters, email and its website for this purpose. Copies of policies, if not sent directly to providers and other stakeholders, are made available on the website. The website also has password protected pages for its members to access. One such page includes the staff development calendar available for providers. The Regional Programme in Access to HE Centre Handbook is a good example of an informative document that the AVA has developed to keep stakeholders informed.
Monitoring providers' information
57 External moderators are required by the AVA to view examples of course documentation such as schemes of work and guidance to students. The moderators who met with the review team reported that the standard of provider documentation had improved in recent years. The moderators told the team that they were expected to meet with students, however, the team noted that this requirement was not explicit within the moderator job description or in the moderator report form. The team was also told that AVA officers are responsible for confirming that the appropriate QAA Access to HE logo appeared on provider publicity material and that this was achieved through receipt of providers' prospectuses and other promotional materials. In reviewing the websites of the Access to HE providers selected for audit trails, the team noted an absence of the QAA Access to HE logo. The team recommends that the responsibilities for the scrutiny of course documentation and publicity material are identified in the relevant AVA documentation; the outcomes of this scrutiny should be reported and followed up through the relevant quality assurance procedure(s).
Equal opportunities and complaints policies
58 The equal opportunities policy went to the Board for approval in November 2006. In all of its work, it is clear that the AVA is aware of the equal opportunity issues that surround age, gender, ethnicity, academic ability, as well as geographical constraints for those students based in rural areas. The AVA gives due consideration to all of these matters when planning programmes and developing strategic plans.
59 The AVA has a complaints policy and set of procedures which are available to providers and students. Over the period reviewed, there had been few complaints, but those that had been received were fully documented and had been dealt with according to the procedures established by the Board. Individual complaints are reviewed by the Access to HE Committee prior to being forwarded to the Board.
Annual AVA report
60 The quality of annual reports to QAA has been variable over the last three years. The most recent was accepted by the ARLC as submitted but the previous one required a meeting between AVA and QAA representatives and then considerable rewriting before being accepted. The new staff team and committee structure, which includes a joint meeting of the Access to HE Committee and Quality Committee should now provide sufficient rigour of review to ensure that subsequent reports meet all the necessary criteria prior to submission.
Principle 3
The organisation is able to assure the quality and fitness for purpose of Access to HE programmes at the point at which they are granted formal approval, and to have effective means to develop, evaluate and review the Access to HE provision for which it has responsibility
61 Since the establishment of OCNNER in 2005, programme development has been focused upon the rationalisation of the units held in the OCNNER database, and the clarification of pathways within two regional frameworks. A decision had been taken early on to develop two separate frameworks, as the Cumbria provision was differently structured from that in the north-east, involving community and adult education providers rather than FE colleges, with fewer HEIs.
Regional framework development
62 The rationalisation of the provision and its development into two regional frameworks was achieved through an extensive consultation process involving FE providers, HE receivers and employers. The Strategic Group looked at the combination of units, the pathway titles, APL and other over-arching issues, while subject groups addressed the selection of units to be retained within particular subject areas. The review team's meetings with providers, HE representatives and moderators confirmed that the developmental process had been very open and inclusive, including all providers and a number of HEI representatives. The AVA saw the development process as effective; achieving a reduction from 500-plus validated units to 150, thus improving efficiency and reducing numbers of units with low student registrations. The review team commends the AVA for the development of the regional Access to HE frameworks and the collegiality this has fostered amongst providers and receivers. The team heard from AVA senior staff, providers and stakeholders that there had been some enhancement of the provision as those units which were retained had been chosen because of their high enrolments, and a further review was under way this year to see if the numbers of units could be further reduced. A website was being set up which logged the units on a database, and was accessible to providers and HEI receivers alike.
63 The requirements concerning award titles, units and credit outlined in The Access to HE Diploma and credit specifications document, published by QAA, were considered by the AVA officers to be implicit in the way the regional frameworks had been developed. The review team saw the Regional Programme for Access to HE Handbook, the guidance used to support this developmental process. The guidance included selected and modified sections of text from The Access to HE Diploma and credit specifications document, credit and credit values, and forms to be used for centre approval, as well as the Regional Programme Information document which centres had to complete when they 'signed up' to deliver the regional framework.
64 The review team also saw examples of Programme Award Regulations, submitted along with the Regional Programme Information to show the choice of units, including those that are mandatory which are available at each centre, necessary to achieve the award of the Access to HE certificate at a specified college. The Regional Programme Handbook indicated the quantity and level of credits and the broad rules of combination needed for the achievement of an Access to HE award. From the examples seen, it was clear that the curriculum design was based upon a model where the mandatory units were primarily introducing essential skills for study in HE, with additional credits to be gained from subject-specific units chosen from a named pathway. The award titles were chosen to reflect the intended progression routes of a particular programme of study.
65 Of the set of forms to be used in preparing for programme validation, those seen by the review team addressed most of the requirements of the licence criteria for programme development, with a few exceptions. It was not clear from the paperwork that HE representatives would have been consulted regarding the intended progression route, nor was there a statement that QAA-recognition could be given only to programmes delivered in the UK. Guidance for centres wishing to offer an Access to HE programme was provided, which included advice on unit specifications, but there was no outline of the stages of the validation process itself in this guidance for providers. The review team recommends that the AVA adjusts its pathway submission form so that it demonstrates agreement by HE representatives of the suitability of the provision to HE progression routes.
Regional framework validation process
66 The review team was provided with the OCN's approval panel guidance which established separate criteria for the membership and composition of validation panels for Access to HE programmes and stipulated that the Chair must be a member of the Access to HE Committee. Evidence seen in the audit trails confirmed that this process had been consistently used under earlier TROCN administration.
67 The review team queried whether the process the AVA was using in the development of the regional frameworks was validation or revalidation. AVA senior staff indicated that it was a form of rationalisation of existing units, following the merger of the OCNs, rather than either a validation or revalidation, as no new units were approved. The team saw a panel report which showed that the revised regional programme submission was considered at a large meeting which included representatives from all providers as well as relevant HE receiver representatives. The reports from both the regional framework events were submitted to the Access to HE Committee where they were both deemed to be 'approved'.
68 The review team noted that it was not clear from the panel report who was on the panel, and who were those submitting the programme for approval in this process. While it was clear that both HEI receivers and FE providers were involved, there were no panel members external to those providing the programme, that is, outside of the region. The report did not confirm that the standard requirements relating to the specifications of the common credit framework for Access to HE and the qualification specification for the Access to HE Diploma had been met. The report did not set any conditions, it only made recommendations, with the result that some quite important points which had been raised had not yet been followed through.
69 The review team noted other irregularities in the approval process: the Chair was a member of the Access to HE Committee, but was also a senior manager of a college which was delivering the regional programme. AVA senior staff confirmed to the team that the AVA was, in effect, the submitting team, presenting its rationalised programme to providers and HEIs for their approval. This seemed to the team to be a reversal of roles, as the providers should be making the programme submission, and the AVA, as the awarding body, should be managing the process for approval by recruiting panels with appropriate externality and subject expertise.
70 To establish consistency of practice and guidance in the future, the review team considered that the AVA needed to establish, approve and publish full descriptions of the various stages of its processes for development and validation, and the various roles involved, for the guidance of all concerned, including providers. A statement that QAA-recognition could only be given to programmes being delivered in the UK should be included in the relevant documentation. The validation report should also confirm to the Access to HE Committee that the programme meets the standard QAA requirements and includes an account of the key issues discussed, as well as the conditions and recommendations made by the panel.
71 The approval panel for the regional frameworks did not approve individual providers to deliver the programme; this was done through a separate 'signing up' process conducted by the AVA's officers. The review team was informed that the officers had visited each centre to agree which units each provider wanted to offer from the regional framework. In this centre approval process, the team heard that AVA officers took responsibility for making the recommendation to the Access to HE Committee about the suitability of a centre's provision for the delivery of particular subject pathways, and there was no requirement in the process for endorsement by an external peer (or external moderators) with appropriate subject expertise to inform their decisions. In establishing its procedures for the approval of individual centres to offer subject pathways, the AVA should incorporate an appropriate level of externality and relevant subject expertise.
Modifications to existing units, programmes and award titles
72 The review team was informed that full approval panels are not always required and that the AVA employs other methods of approval, for example, written comments from readers with relevant subject expertise or internal quality panels (see also below, Audit trails). These alternative processes and the principles underpinning their application were not explained in the panel guidance document, or elsewhere. The team considered that a written process for the modification of units and pathways was needed to establish consistency of practice; explaining how and when (and in what timescale) modifications to units could be made; who should be involved to ensure continuing sustainability of pathways, and when changes to various units were sufficient to require a revalidation of a particular pathway. The team considered that this is particularly important now that the units and named pathways are shared across providers.
73 From discussions with AVA senior officers and providers, the review team heard that some amendments had been made to award titles as a result of the recent revisions. However, there was no clear process for deciding how a new award title might be developed and approved. AVA staff pointed out that the rules of combination for the regional frameworks as a whole had been left deliberately broad, and there were no regulations to indicate what rules of combination should apply to a particular title. Similarly, where students were taking a mixture of units it was possible to have a 'Combined Studies' award, but there were no established rules about how this was to be done. AVA officers agreed that this was a process that needed further development so that the AVA could ensure consistency of content (and outcome) in line with the title of the award. It is a condition of licence that the AVA, as part of its Access to HE Diploma implementation plan, addresses the requirement for approved award titles to be linked to approved rules of combination.
Annual review
74 With regard to the Annual Quality Review process, the review team saw a selection of reports from AVA officers following an annual visit. Providers who met with the team confirmed that a regular annual process was undertaken by the AVA officers, and built upon reports written by external moderators. However, the team noted that there was no description of the process written down by the AVA for its stakeholders and providers which set out its data requirements, and that reports from different AVA officers could vary in what they commented on depending on moderators' reports and discussions with providers, so that there was no real comparability possible across programmes and therefore no overall commentary on the consistency of student outcomes. Although the reports identified 'issues' there was no clear action plan or timescale for addressing these issues.
75 It became clear from discussions with AVA officers and providers that the process for annual course review no longer involved self-assessment by providers, although it had done in the past, nor did it necessarily involve consideration of feedback from students. The review team noted the difficulties the AVA was experiencing with regard to the gathering of accurate student data for registration, retention and completion, and considered that the AVA needed to revise its annual course review process in the light of its development of the regional frameworks, to establish a greater degree of comparability across providers by improving consistency of data input to the annual course reviews. The data underpinning the annual course review should include self-assessment by providers and consideration of feedback from students. It is therefore a condition of licence that the AVA revises its annual course review process so that it requires self-assessment by providers which, itself, includes consideration of feedback from students.
76 The review team welcomed the comments from AVA senior staff that they intended to continue to rationalise the units within the regional frameworks as part of an annual review process. This process was understood to be already under way. In planning the next stages, the team recommends that the AVA also establishes a clear schedule for the revision of unit content within the remaining units of the regional frameworks.
Periodic revalidation
77 Regarding periodic revalidation, the review team noted that the Overview did not make a convincing distinction between annual review and periodic revalidation. When asked how periodic revalidation involved critical review, the AVA senior staff acknowledged that they had no established process as yet for developing a critical review but were intending to ask the Access to HE Committee to consider the development of a process soon. It is a requirement of the licensing criteria that AVAs should have a system for the periodic revalidation of programmes which assures their continuing quality and fitness for purpose. It is therefore a condition of licence that the AVA develops and approves its periodic revalidation process and its forward schedule for periodic revalidation, together with supporting documentation. The AVA will wish to consider carefully how this is to be achieved in the context of a regional programme.
78 Overall, the review team found significant gaps in AVA's quality assurance processes. It is therefore a condition of licence that the AVA establishes, approves and publishes full descriptions of its processes for development, validation, pathway and unit modification, annual review and periodic revalidation.
Principle 4
The organisation is able to secure the standards of achievement of students awarded the Access to HE Diploma
External monitoring of programmes and moderation
79 For quality assurance purposes, the AVA has aggregated the regional framework providers into four clusters containing three or four providers. Each cluster has a lead moderator and an appropriate number of pathway moderators depending on the curriculum being offered.
80 Pathway moderators cover the range of pathways offered by the providers in each cluster, and have the appropriate knowledge and expertise necessary to moderate the subject pathways involved. The role of the lead moderator is new to the AVA, having been developed to meet the needs of the new regional frameworks. It is as much a support role for pathway moderators as it is a moderation role in itself. Lead moderators are available to their teams of pathway moderators for advice and guidance as necessary. The review team met with several moderators and heard that the lead moderator role offers a valued source of help to pathway moderators, and that moderators now felt to be a part of a team.
81 The AVA provides an External Moderator Handbook which is updated annually. This comprehensive document clearly defines the function of moderation by detailing the criteria for selection and appointment of moderators, rates of pay, details of support available from the OCN, the quality model that applies to the regional framework, guidelines for writing reports, and copies of moderator report forms. The Handbook also contains guidelines for the reporting from final award boards. Background information on internal moderation systems and assessment methodologies are also included, together with 'top tips' for moderators and a troubleshooting list with possible courses of action to be taken for each instance identified. The review team commends the AVA for the thoroughness of its moderator appointment and training procedures, the External Moderator Handbook, the quality of moderation reports, and the feedback provided to moderators by AVA staff.
82 Moderators are appointed by the AVA and receive a contract which specifies the number of visits they are required to make (two per year); the duration of the contract; the mandatory induction training, and the requirement that they attend an annual update session with the AVA. Moderators are required to declare any conflict of interests to the AVA. The review team, however, could find no statement in the moderator contract, that prohibited any subcontracting of their moderation duties. It is therefore a condition of licence that the AVA makes explicit within its moderator contract that the subcontracting of duties is prohibited.
83 The External Moderator Handbook contains optional checklists for moderators to use to monitor the completion of their duties. The checklists and report forms are different for lead and pathway moderators but both are comprehensive and cover all elements of review as required by the licensing criteria. The remit of moderators also involves recommending the review and updating of units as necessary. Moderators are selected according to the requirements of the different pathways: appointments follow from an application and interview process. Attendance at an induction session is a condition of appointment. Most lead moderators will have subject sector experience and may moderate a pathway within their sector. In the AVA's appointment of moderators, subject expertise is a critical requirement.
84 Standardisation meetings are held where students' work is reviewed to confirm that standards are common across providers. The deployment of subject moderators to clusters of providers also contributes to a more standard level of student achievement being recognised. The process enables good practice in assessment to be shared and swift remedial action for issues to be identified and disseminated.
85 Moderators spoke highly of the support received from the AVA staff and told the review team that they found the AVA's update sessions invaluable. All Access to HE moderators are required to attend one update session each year. Where moderators find it impossible to attend on any of the range of dates offered, AVA officers will provide one-to-one update sessions if necessary. Failure to attend update sessions, or other deviations from the moderator employment guidelines could initiate the disciplinary process which can lead, eventually, to dismissal from the moderator position.
86 Moderators are required to submit their standard visit reports electronically to the AVA. The introduction of this requirement has improved the speed of communication between the moderators and the AVA, and between the AVA and the providers. Copies of all pathway moderators' reports are forwarded to the lead moderators together with any request for action plans and the responses from providers to these requests. All report forms conclude with an action plan which is completed if required. Providers are expected to respond to this action plan and pathway and lead moderators receive copies of the providers' responses. Where there is any delay in responding to action plans, an AVA officer will visit the provider to ensure that they are aware of their obligations. The review team noted, however, that there was no clearly documented procedure for the actions that the AVA would take in the event of providers failing to address issues raised in moderation or other reports. It is a condition of licence that the AVA develops and publishes clear procedures for action to be taken in the event of providers failing to address issues raised in moderation or other reports.
87 The AVA monitors moderator performance via the quality of their written reports, through moderator self-assessment reports, by accompanied visits and by feedback requested from providers following moderation visits. Reports are available for each type of monitoring and feedback is given to moderators. Where necessary, additional training is arranged.
88 The review team was told that summaries of moderation reports and analyses derived from those reports are available to the Access to HE Committee. In the documentary evidence provided, the team was unable to identify the involvement of any AVA committees in the receipt of, and response to, moderation reports. It is a condition of licence that the AVA revises its processes to ensure that there is involvement of relevant committees at appropriate points within the framework for moderation including the receipt of, and response to, moderators' reports.
Internal moderation
89 Providers' internal quality assurance processes are identified in the Moderator Handbook. Expectations of provider systems are clearly documented and substantial sections of moderators' reports relate to this process. The areas that external moderators are expected to comment on include the application of the process; the clarity of documentation; advice and guidance for assessors; confirmation of assessment decisions and the implementation of standardisation events. One of the standard staff development activities for providers relates to setting up and implementing internal moderation processes.
Final assessment boards
90 Lead moderators play a key role in the final assessment boards, signing off the recommendations for the award of credit (RACs), thus confirming the award of the Access to HE certificate on behalf of the AVA. The final assessment boards are held following completion of the moderation processes: these confirm that the award of the Access to HE qualification is conducted in a uniformly fair and equitable manner. Explicit guidance on final assessment boards is available to moderators and providers. Guidance includes the composition of the board, the documentation that must be available to confirm the award of credit (including internal moderation documentation), arrangements for signing off RACs, and details of the mitigating circumstances procedures, and deferral arrangements. Lead moderators are required to submit a written report on the final assessment board to the AVA. The review team formed the view that the AVA has robust arrangements in place for final assessment boards.
Certification
91 The certificates issued by OCNNER meet the necessary criteria which include the appropriate QAA Access to HE logo and the award name and pathway. All students, whether achieving the award or not, receive credit transcripts which list the units achieved together with their levels and values. All certificates and credit transcripts are checked independently of the person inputting the data. The review team was told that the number of certificates returned to the AVA due to errors was negligible, although the AVA sets no specific target for staff to benchmark their performance against. Appropriate records of the issue and re-issue of certificates are maintained by the AVA.
Audit trails
92 In the course of the visit, the review team conducted audit trails on six of the AVA's providers. The purpose of conducting these trails was to establish the consistency and effectiveness of the AVA's processes at the centre level.
93 The providers selected for the trails comprised an appropriate sample, delivering a range of Access to HE programmes validated by the AVA. The group consisted of one HEI and five colleges of varying sizes spread across the regions covered by the AVA.
94 The review team was presented with audit trail files which included programme and moderation files, completed learner registration and RAC forms, and the AVA's activity file which covered various events and meetings organised by the AVA. Programme documentation covered the academic years 2003-04, 2004-05, 2005-06 and 2006-07. The team also had access to all of the AVA's committee files.
95 The examples of the programme approval process all predated the formation of OCNNER as a new AVA region, and included a selection from Cumbria and the region previously known as TROCN. These programmes therefore also predated the rationalisation of the provision into two regional frameworks. The review team formed the view that the approval processes in use at that time were mostly thorough and well documented, and included evidence of validation panel reports, conditions which were followed through by the AVA, and final approval letters confirming committee approval and the date when the programme could commence. The TROCN programme documents included the criteria for the award, and appropriate use of credit and rules of combination. Meeting reports made at the time of the validation confirmed the presence of HE and FE representatives on the panel. Formal panel reports to the Access to HE Committee were rather brief, but distinguished between conditions and recommendations and noted any changes to documentation needed. A fuller 'meeting' report (usually handwritten) was also made at the time of the approval event which had rather more detailed information.
96 The review team found three instances where a programme validation (or revalidation) had been formally approved by the Access to HE Committee after the start date approved for the programme. Except for these late approvals by the Access to HE Committee, the TROCN approval process appeared to have been generally sound and consistent in practice.
97 The review team also saw evidence of amendments to programmes being made, with a consistent procedure of written amendment forms and confirmation of revised definitive documentation from the AVA in use. For programme amendments, the team noted that there were various approval processes which did not require the use of a full panel, such as sending the revised documentation to an internal quality panel and external reader, or to the internal quality panel only. Senior AVA staff who met with the team confirmed that the decision as to which procedure should be used was made by the relevant AVA officer. The team formed the view that, while the process of amendments and re-issue of definitive documentation was sound, some written guidance for both AVA officers and providers on the amendments procedure was important to ensure consistency and an appropriate level of externality.
98 The review team was informed that the annual review process was based on a meeting between an AVA officer and the provider's Access to HE representatives. The AVA officer produces the annual review report. The team found that the reports summarised the strengths, weaknesses, good practice and issues arising, but could find no consistent pattern of the documentation seen and discussed within the process, nor any evidence of providers' own evaluations nor feedback from students informing the report. The reports confirmed some issues, but there was no accompanying action plan agreed with the provider detailing responsibilities and timescales. Senior AVA staff informed the team that they had dispensed with an earlier process of using colleges' own annual course reviews, and acknowledged that as a result, their annual review reports lacked an appropriate underpinning of provider data and comment, including feedback from students. The team formed the view that more use could be made of providers' own annual reviews where these existed, and that the AVA should publish a checklist of its requirements to ensure that it receives consistent information from all providers which can then inform its own self-assessment process.
99 Moderation reports were also included in the audit trails. In the External Moderator Handbook, the review team saw guidance and a form stating the requirements for moderators' reports. Completed forms were based clearly on these requirements. Forms were, in general, fully completed by both pathway and lead moderators with lead moderators providing an overview. From this evidence, it was clear that, in coming to their conclusions, moderators had checked that the correct units were in use; seen student work and assessment strategies; considered APL and appeals strategies; seen evidence of internal moderation and checked the accuracy of RACs.
100 The review team saw evidence of both internal and external moderation in operation, including final award meeting reports which confirmed that the previous year's recommendations had been acted upon. At the conclusion of each visit an action plan was agreed with the provider where this proved to be necessary. Where issues were identified by external moderators, the designated AVA officer follows these up with the provider so that the action plan agreed with the external moderator is actioned. External moderators are kept informed of all stages of development during this correspondence between the provider and the AVA.
101 The review team heard that moderation reports were valued by providers and used by the AVA to inform annual review. The team heard from moderators that the 'cluster model' of moderation introduced by OCNNER to support the regional frameworks was working well and was helping to promote consistency of standards across providers. Moderators confirmed that they were expected to either speak to students or to look at student evaluations on their visits, although the team noted that the moderator form did not require confirmation of this. The AVA will wish to include this as a stated requirement so that it may assure itself that the student experience is being monitored by moderators. Overall, the team found the moderation process to be thorough and effective and commends the AVA for its robust moderation system and the continual improvements to the quality of the feedback from moderator reports.
102 The audit trails confirmed that the AVA has effective processes in place for checking that the correct credits were being included in the certification process.
Conclusions
103 Following the regionalisation of OCNs, OCNNER was formed in August 2005 by the merger of TROCN and NEOCN. This necessitated a complete restructuring of the organisation and this has required major inputs from the Board of Trustees and its committees, AVA officers, Access to HE providers and local HEIs as receivers of Access to HE students. The effort entailed in creating the new organisation should place the AVA on a strong footing for the future. During the review visit, the review team heard a good deal of praise for the AVA officers from providers, receivers and moderators, and the team commends AVA staff for their commitment, availability and responsiveness.
104 The AVA is well established with provision being managed through two separate regional frameworks covering the north-east of England and Cumbria. In the north-east, provision is delivered entirely through FECs, whereas provision in Cumbria is delivered through a variety of providers including FE, HE and community-based organisations. The creation of the regional frameworks was accompanied by the rationalisation of units. The total number of available units has now been considerably reduced and these are shared across the pathways offered by the various providers. The shift to the regional frameworks has been embraced by providers and receivers, with the latter now having greater clarity regarding the subject knowledge and the academic standards achieved by Access to HE students, wherever they are studying. The review team commends the development of the regional Access to HE frameworks and the collegiality this has fostered amongst providers and receivers.
105 Membership of the AVA has remained stable since the merger and the AVA has not, therefore, had to confront the need for procedures for other organisations to gain membership or, indeed, for the AVA to remove members. Members elect a Board of Trustees but neither the period of office nor the number of terms a trustee may serve is clearly specified. In the interests of effective governance, the review team recommends that the AVA addresses this. The trustees see themselves as proactive supporters of the AVA and critical friends to the officers. The review team formed the view that the trustees make a valuable contribution to the operation of the AVA. The team recommends however that the AVA continues in its efforts to ensure that the Board's profile is geographically representative and includes sufficient current HE and Access to HE experience amongst its membership. The categories and levels of representation required on the Board should also be formalised in the AVA's constitution.
106 The Board has delegated responsibility to its Access to HE Committee for ensuring that the AVA meets in full the requirements of the Recognition Scheme. However, the review team did not find evidence of a process through which the Access to HE Committee meets this responsibility. Given that the team identified several areas where the licensing criteria were not met, the Board will need to give greater attention to the effectiveness of its monitoring and reporting procedures to ensure that delegated activities are properly undertaken.
107 For the early part of the period covered by this review, the AVA was managed by TROCN. The review team conducted audit trails on six AVA providers operating under TROCN management. In all cases, procedures for approval, review and external moderation were found to be sound. This led the team to the view that the standards of the awards were secure and that the quality of the student experience was high. Both these aspects were appropriately monitored by TROCN using well developed procedures. However, since the formation of OCNNER, a number of the procedures appeared to be in the process of change and had not been codified, approved and published.
108 In some cases work which is crucial to the proper functioning of the AVA has not been completed. For example, the AVA has not developed a strategic plan, although the review team recognised the intention to do so in summer 2007. The team formed the view that this work should be completed to time and that the strategic plan needs to be informed by identified risks to Access to HE provision. The team also considered that the absence of a specific budget for Access to HE work is a hindrance to monitoring the cost effectiveness of the AVA.
109 The AVA intends to use two reports to inform the content of its annual report to QAA. These are the Quality Assurance Manager's report and the Self-Assessment report. At the time of the review, the self-assessment process was not fully operational. The nature of the self-assessment process had been agreed but it had not yet been used. Furthermore, the current process has concurrent submission dates for the above reports and the first draft of the report to QAA, all of which are considered at a joint annual meeting of the Access to HE Committee and Quality Committee. This means that the Quality Assurance Manager's report and the Self-Assessment report cannot fully inform the content of the report to QAA. The review team recommends that a more appropriate schedule is developed for consideration of these reports.
110 The centre approval and pathway approval processes are facilitated by a helpful overarching framework which relies on both completion of forms and AVA officer consultation. However, there is a lack of clarity in written guidance about the different stages of the processes of development, approval, amendment and revalidation; how the forms are to be used in particular processes; when the full validation process is needed for programme modifications, or when alternatives such as external readers can be used for endorsing proposed amendments to particular units. In the approval process for the revised regional frameworks, externality became lost as the distinction between panel and providers became blurred. The team considered that it is especially important for the AVA to formalise and publish the various stages of its process for validation, in particular distinguishing the developmental and formal validation stages, and ensuring an appropriate level of externality and relevant subject expertise in the validation panel throughout the process and a distinction of roles between panels and providers.
111 The review team also considers that clearer written guidance is needed to explain the kind of minor modifications which were permitted to validated units, giving clear directions about timescales and how such minor changes could be approved by those with appropriate subject expertise. It was also necessary to distinguish how this process differed from that which approved more major changes, for example, involving pathway or award title changes.
112 The review team found the external moderation systems to be rigorous and fit for purpose. The use of a series of lead and pathway moderators operating in clusters, each covering the work of three or four providers is particularly effective. Moderators are strongly supported by the AVA both in terms of a comprehensive handbook and interaction with AVA officers. There is a mandatory programme of ongoing training for moderators. The AVA also holds standardisation meetings where student work is reviewed in order to confirm that standards are common across providers. This allows good practice in assessment to be shared and swift remedial action relating to the identification of issues. The AVA monitors moderator performance by a number of means. However, it was not clear to the team which committee took responsibility for this.
113 Overall, the review team concluded that the AVA has a number of key strengths, not least of which are its staff. Considerable work has gone into creating the new regional frameworks and this has further strengthened the links with providers and HEI receivers. The AVA, however, needs to address a number of key areas in order to meet the licensing criteria in full. Having dealt with the major issues relating to the merger, restructuring and relocation, the AVA is well placed to address these areas. In arriving at their licensing recommendation, the team, while sympathetic to the AVA's circumstances, were of the view that the AVA should refocus its attention and move to address the conditions specified in this report without delay.
Commendations
114 The AVA is commended for the:
- development of the regional Access to HE frameworks and the collegiality this has fostered amongst providers and receivers (paragraph 62)
- thoroughness of its moderator appointment and training procedures, the External Moderator Handbook, the quality of moderation reports, and the feedback provided to moderators by AVA staff (paragraphs 81 and 101)
- commitment, availability and responsiveness of AVA staff in support of its providers, receivers and moderators (paragraph 103).
The AVA licence
Review outcome
115 Open College Network North East Region is awarded a provisional confirmation of its AVA licence, with conditions to be met by 28 March 2008 and a further review visit by summer 2008.
Conditions
116 The licence is renewed on condition that the AVA:
i in its Memorandum and Articles of Association, specifies the categories of stakeholder representation and the criteria for their appointment or election to the Board (paragraph 23)
ii revises its monitoring and reporting procedures so that the Board of Trustees can assure itself that all of the AVA licensing criteria continue to be met (paragraph 28)
iii makes explicit, within the terms of reference of the Board of Trustees, and the most appropriate committee, where the locus of formal authority for the Access to HE award is located (paragraph 33)
iv develops a strategic plan which includes its AVA responsibilities together with appropriate procedures for recognising and responding to risks associated with Access to HE (paragraphs 39 and 45)
v establishes, approves and publishes full descriptions of its processes for development, validation, pathway and unit modification, annual review and periodic revalidation. A forward schedule for periodic revalidation should also be approved. A statement that QAA recognition may be given only to programmes which are delivered in the UK should be included in the relevant documentation (paragraphs 53, 54, 65, 70-72, 74, 75, 77, 78)
vi revises its annual course review process so that it requires self-assessment by providers which, itself, includes consideration of feedback from students (paragraphs 43, 74, 75)
vii requires in its validation and revalidation reports clearly identified conditions and recommendations (paragraphs 54, 68, 70)
viii as part of their Access to HE Diploma implementation plan, addresses the requirement for approved award titles to be linked to approved rules of combination (paragraphs 64 and 73)
ix makes explicit within its moderator contract that the subcontracting of duties is prohibited (paragraph 82)
x develops clear procedures for action to be taken in the event of providers failing to address issues raised in moderation or other reports (paragraph 86)
xi revises its processes to ensure that there is involvement of relevant committees at appropriate points within the framework for moderation including the receipt of, and response to, moderators' reports (paragraph 88).
Recommendations to the AVA
117 The review team recommends that the AVA:
i publishes its formal membership application and removal procedures (paragraph 21)
ii sets a limit on the number of terms of office that a trustee may serve (paragraph 22)
iii continues in its efforts to ensure that the Board of Trustees' membership is geographically representative and includes sufficient current higher education and Access to HE experience (paragraph 24)
iv strengthens its arrangements for providing ongoing training and development for Board members (paragraph 25)
v implements its intended self-assessment process and coordinates the sequence of reports so that both the Quality Assurance Manager's report and the Self-Assessment report can usefully inform the development of the annual AVA report to QAA (paragraphs 30, 43, 44)
vi reviews the fitness for purpose of its system for monitoring the Board's and subcommittees' effectiveness (paragraph 32)
vii develops an Access to HE specific budget (paragraph 48)
viii continues to develop its student data systems, including its capacity for cohort analysis (paragraphs 51 and 55)
ix identifies the responsibilities for the scrutiny of course documentation and publicity in the relevant AVA documentation; the outcomes of this scrutiny should be reported and followed up through the relevant quality assurance procedure(s) (paragraph 57)
x adjusts its pathway submission form so that it demonstrates agreement by HE representatives of the suitability of the provision to HE progression routes (paragraph 65)
xi continues to rationalise the units within the regional frameworks and establishes a clear schedule for the revision of unit content (paragraph 76).
Appendix
Aims and objectives of AVA review
The aims of the system of AVA review are:
i to provide the basis for an informed judgement by the ARLC about the fitness of the AVA to continue as a licensed agency
ii to promote public confidence in Access to HE as a properly regulated and respected route into higher education by assuring:
- the quality and adequacy of AVAs' systems and procedures
- the quality, comparability and range of AVAs' operations
- the adequacy and comparability of AVAs' standards for approval, moderation and monitoring of programmes
- consistency across AVAs in the operation of criteria for the granting of the Access to HE award
iii to stimulate reflective and self-critical perspectives within AVAs, as an instrument to promote quality enhancement
iv to provide an opportunity to identify and disseminate good practice of AVA operations
v to provide a mechanism for ensuring necessary, and encouraging desirable, improvements and developments in AVAs.
The objectives of each AVA review are:
i to examine, assess and report on:
- the development of, and changes in, the AVA since its last review or initial licence, and its plans and targets for the future
- the organisation's continuing viability and robustness and the ways in which the AVA demonstrates sound governance
- the efficiency and effectiveness of the AVA's operational and quality assurance systems
- the range and scope of the AVA's activities, and the appropriateness and value of these activities
- the ways in which the AVA approves and monitors programmes and the ways in which these processes take account of the need for consistency and comparability
- the ways in which the AVA satisfies itself of the adequacy and comparability of standards achieved by students gaining the Access to HE certificate
- the evidence available to indicate the AVA's success in achieving its aims and targets
ii to identify and report on:
- strengths and good practice in procedures and operations
- areas which would benefit from further development
- areas requiring attention.
ISBN 978 1 84482 750 3
Top