Anglia South Open College Network
January 2004

Foreword

1 The Quality Assurance Agency for Higher Education (QAA) is responsible to the Department for Education and Skills for the recognition of Access to Higher Education courses. QAA exercises this responsibility through a national network of authorised validating agencies (AVAs), which are licensed by QAA to recognise individual Access to HE programmes, and to award Access to HE certificates to students. The AVAs are responsible for implementing quality assurance arrangements in relation to the quality of Access to HE provision and the standards of student achievement.

2 QAA has developed a scheme for the licensing and review of the AVAs, the principles and processes of which are described in the QAA Recognition Scheme for Access to Higher Education in England, Wales and Northern Ireland. The Recognition Scheme is regulated and administered by the Access Recognition and Licensing Committee (ARLC), a committee of the QAA Board of Directors. The ARLC is responsible for overseeing the process of AVA review through which AVAs are periodically relicensed. The criteria applied by the ARLC and by review teams operating on the Committee's behalf, in reaching judgements about whether and under what terms an AVA's licence should be renewed, are provided within the Recognition Scheme documentation. These criteria are grouped under the seven principles that provide the main section headings of this report.

3 Following the review of an AVA, a member of the review team presents the team's report to the ARLC. The Committee then makes one of five decisions:

i unconditional renewal of licence for a specified period;

ii conditional renewal of licence with conditions to be met by specified date;

iii provisional renewal of licence with conditions to be met and further review visit by specified date;

iv withdrawal of licence for operation as an AVA;

v temporary renewal of licence with request for further information by specified date (decision suspended).

4 This is a report of a review of the AVA function of Anglia South Open College Network (ASOCN), undertaken by QAA. The Agency is grateful to ASOCN and to those who participated in the review for the willing cooperation provided to the review team.

The review process

5 The review was conducted in accordance with the process detailed in the QAA Recognition Scheme for Access to Higher Education in England, Wales and Northern Ireland. The preparation for the review included an initial meeting between ASOCN representatives and the QAA Assistant Director to discuss the requirements for the Analytical Account (the Account) and the process of the review; the preparation and submission by ASOCN of its Account, together with a selection of supporting documentation; a meeting of the review team to discuss the Account and supporting documentation and to establish a draft programme for the review visit; and negotiations between QAA and ASOCN to finalise the programme and other arrangements for the review visit.

6 The review visit took place on 19 and 20 January 2004. The visit to ASOCN consisted principally of meetings with representatives of ASOCN, including members of the Board of Trustees, Quality Committee and Access Tutors' Forum; moderators for Access to HE programmes; representatives from higher education; and former Access students now studying in HE.

7 The review team consisted of Dr Peter Easy, Deputy Vice-Chancellor, University of Gloucestershire and Mr Keith Fletcher, Director, Open College Network of the South West. The review was coordinated for QAA by Ms Kath Dentith, Assistant Director (Access), Reviews Group.

The AVA context

8 The organisation that is now Anglia South Open College Network (ASOCN) was established in September 1996, under the name Accreditation Consortium for South Anglia. The establishment of the new organisation brought together a former AVA (the Essex Access Consortium, originally set up in 1986) and the Essex Open College Federation. The present organisation's AVA functions are now fully integrated in the structures and operations through which it functions as an OCN.

9 ASOCN's office base is located on the outskirts of Colchester in Wivenhoe House Conference Centre, which is owned by the University of Essex. The AVA describes its operational area as 'Essex, plus the Unitary Authorities of Southend and Thurrock, and Cambridge, Havering, Norfolk and Suffolk', with Access provision available across the area in urban, suburban and rural areas. Its operation as an AVA in parts of Norfolk is subject to a contractual arrangement with North Anglia Open College Network (NAOCN), the OCN that provides non-Access accreditation services in that region.

10 The last review of the AVA was conducted by the Higher Education Quality Council (HEQC) in 1995, when merger between the two predecessor bodies was under consideration. The outcome of that review was a recommendation that the AVA licence should be continued, with a number of recommendations. The review report required a response to HEQC on matters including strategic planning; the development of standard paperwork; details of quality assurance arrangements, including moderation; implications of OCN developments (including the effect of the introduction of a credit-based system to Access programmes); and the development of the AVA role in relation to targeting and widening participation.

11 AVA statistics 2002-03 (as provided by the AVA in its annual report to QAA for 2002-03).

Providers offering Access to HE programmes

Access programmes available

Access programmes running

Access learner registrations

Access to HE certificates awarded

683

252

12 In 2002-03, the AVA had 56 members, of which 11 were providers of Access programmes. This represents growth in membership of 120 per cent since 1996, including three new Access provider members, and an additional two members will offer Access in 2003-04. Access learner numbers have also grown, with some variation, from 369 in 1996-97 to 683 in 2002-03, with a particularly large increase in the last year. Over the same period, however, the number of Access certificates awarded has fallen from 307 to 252. While Access to HE provision is a small part of the OCN's business, with learner registrations accounting for 2.25 per cent of total registrations during 2002-03, this proportion represents a gradual increase since 1998-99, when Access represented only 1.45 per cent of all learner registrations. The importance of Access to the organisation is greater than is suggested by these figures, however, with Access learners being awarded 16.25 per cent of all credits awarded by ASOCN.

13 The ethnicity profile of learners reflects that of the region as a whole, with approximately 87 per cent of registered learners who declared their ethnicity registering as white. 75 per cent of registered learners were female.

Principle 1

The organisation has a structure which is based on a partnership of members, including institutions which provide Access to HE programmes and institutions of higher education

14 ASOCN is a company limited by guarantee and a registered charity. The 56 organisations in membership include voluntary and community groups, an NHS Trust, sixth-form colleges, adult community colleges, further education institutions, and four higher education institutions: Anglia Polytechnic University, University of East London, University of Essex, and Writtle College.

15 Membership of ASOCN is by application and comprises three categories with differential subscription rates. Full membership is available to institutions delivering adult, further or higher education, or who are generally in receipt of public funding. Full members must formally agree to support the mission statement of ASOCN. Once in full membership, they are entitled to vote at general meetings of the company. Voluntary and Community Group membership is available for charities and other 'not for profit' organisations which are also able to support the aims and mission of ASOCN. Associate membership is available for all other organisations which do not meet the criteria for the other categories. The review team heard that, in practice, the boundaries between these categories were permeable and that the final decision on all membership issues was exercised by the ASOCN Board of Trustees.

16 Applications for membership are made on a pro forma which requires details of the applicant's organisational structure, aims, legal status and financial viability. In all cases, the aims of the organisation must be in accordance with the ASOCN mission which is reproduced as part of the pro forma. Applications are considered in the first instance by the AVA's Finance and General Purposes Committee prior to formal approval by the Board of Trustees. From the evidence of its minutes, it was clear that the Finance and General Purposes Committee discussed membership applications with appropriate thoroughness. The Committee has also been active in the further development of the application process which now includes an enquiry procedure designed to assist in cases where an application might not progress beyond an initial enquiry. In addition, the AVA has been discussing the development of a particular membership package designed to attract more small organisations into membership.

17 In summary, the review team considers that ASOCN meets in full the criteria expressed under Principle 1.

Principle 2

The organisation has governance structures which allow it to discharge its AVA responsibilities securely

18 Each member organisation is entitled to attend the annual general meeting of the company at which the full members nominate and elect the Board of Trustees. The Board may have between five and 12 members, and must include at least one member from a higher education institution. The annual general meeting also receives nominations and elects members to the AVA's Quality Committee.

19 The Board of Trustees is the key committee in the governance structure of the AVA. Its duties and responsibilities are set out in the company's Memorandum and Articles of Association in the form of its comprehensive Objects and their furtherance. It is thus empowered to have oversight of all legal and financial matters, to decide on strategic direction and policy development, and to have ultimate responsibility for matters of quality. Three committees are described by the AVA as being advisory to the Board of Trustees. The Quality Committee has responsibility for all quality assurance procedures within the AVA including the development, validation, moderation and certification of Access programmes. The Finance and General Purposes Committee, which has recently absorbed the work of a separate Personnel and Staffing Committee, has responsibility for financial, accommodation, staffing, and general operational matters. A Remuneration Committee deals with all matters related to the employment of the Executive Director of the AVA.

20 In considering the remits of the AVA's committees, the review team noted that the terms of reference of the Quality Committee listed responsibility for 'maintaining the licence to operate as an Authorised Validating Agency awarded by the Quality Assurance Agency for Higher Education'. Although it is appropriate for a committee at a lower level in the governance structure to have delegated responsibility for some aspects of the maintenance of the licence (in this case, for quality assurance matters), the QAA's Principles and criteria for the licensing of Authorised Validating Agencies require that the final responsibility for the holding and maintenance of the licence be located with the organisation's principal governing body. In the case of ASOCN, the principal body is the Board of Trustees and it is thus a condition of the renewal of the licence that the Board's ultimate responsibility in this respect is made clear and that the terms of reference of the Quality Committee are appropriately amended.

21 The review team noted that, on the adoption of company status, the membership of ASOCN had agreed that the Executive Director should become a Trustee of the company, and thus a full member of the Board of Trustees and eligible for membership of other committees within the AVA. This decision challenges a common principle of corporate governance which makes a clear separation between the executive and deliberative structures of an organisation. The team noted that the AVA had considered this matter and had put certain arrangements in place to minimise the potential for conflicts of interest, for example, by not permitting the Executive Director to be a member of the Quality Committee. The team considered that such arrangements were useful in relation to known potential conflicts of interest, but that, in order to avoid conflicts of interest in both predictable and unpredictable circumstances, it was essential that the AVA should maintain actual separation of governance and management, and that this should apply in all areas, including the Executive Director's role. While the team accepted the statement in the Account that the AVA benefits from the 'technical, operational and specific expertise' of the Executive Director as a Trustee, it considered that this could be equally well provided through the post-holder's attendance at meetings of the Board in an ex officio capacity.

22 In respect of the Quality Committee, the review team noted that it employed a range of working methods. In some areas, it maintained an immediate link with processes, for example, receiving directly the reports of Access validation events. In other areas of its responsibilities, the committee is more reliant on the work of the officers of the AVA, and the internal audit process, through which aspects of the AVA's activities are subject to direct audit by members of the Quality Committee on a periodic basis. In the case of moderation, for example, the committee does not receive moderators' reports directly but relies on a summary of issues (the Access to HE Moderation Summary Report) arising from those reports (see paragraph 69, below), supplemented by the internal audit system. In the view of the team, such a process of internal audit is crucial if the Quality Committee is to be able to ensure the security of quality assurance arrangements with which it has no direct contact. In relation to its oversight of moderation, for example, insufficient detail is provided within the summary report itself for the committee to be able to exercise its responsibilities without some further opportunity to sample moderators' reports directly, such as can be provided through internal audit. As the nature of the committee's working methods places a particular emphasis on the effective operation of the internal audit system, it was disappointing that a planned internal audit of 'Access to HE' which was planned for October and November of 2003 had not been completed by the time of this review.

23 On a more detailed point, the review team noted that the terms of reference of the Quality Committee stated that if the committee 'is not quorate the chair will decide on whether the meeting should proceed'. It is not clear if this indicates that the committee may proceed with its normal powers (which would be unusual and undesirable in terms of good practice in governance), or whether it may proceed but without the power to make decisions. The team recommends that the committee's terms of reference be amended appropriately.

24 Prior to incorporation, ASOCN maintained a separate Access Programmes Committee. The work of this committee has since been absorbed into the Quality Committee although, as the Account noted and the review team confirmed, Access matters are separately identified within the agenda of the Quality Committee. ASOCN has continued to support a separate Access Tutor Forum (ATF) which it considers to be 'the main networking and staff development activity for Access tutors and the link between Access in practice and the Quality Committee'. However, the position and status of the ATF within ASOCN was unclear to the team. The Account noted that the ATF was not part of the formal governance structure of ASOCN although its notes were received at the Quality Committee and a representative from the ATF sits on the committee. The Account also stated that the ATF 'in conjunction with ASOCN...agrees an action plan for the year', although there was no evidence of this activity in the notes of the ATF other than the planning of its own agendas from meeting to meeting.

25 The Account also noted that the purpose of the ATF was to be reviewed in 2003-04 with a report to the Quality Committee in November 2003. The minutes of that meeting record that the committee received a 'verbal proposal' from the Executive Director 'to review the purpose' of the ATF, and 'to establish two groups which could address strategy and quality'. The minutes record that the committee agreed that the Executive Director should 'progress the proposal and present a paper to the next meeting' but, at the time of the review, there was no further evidence available to the review team that these plans had been advanced. In its discussions with members of the ATF, the team heard that the forum was valued as a staff development activity and an opportunity to share practice. It appeared to the team that there was unexploited potential in the ATF. At recent meetings, for example, it has been discussing issues which are of significance to Access and from which the AVA might benefit if they were fed more formally into its decision-making structures. These include standardisation of assessment, programme evaluation, HE progression links, and credit targets for Access programmes. The AVA may also wish to consider broadening the membership of the ATF to include representatives from higher education, which would add a further valuable dimension to its work.

26 While the Account prepared by the AVA for this review described the structure and operation of the governance arrangements, it did not offer any substantial evaluation of their effectiveness. From discussions during the review, and from the evidence of documentation including the minutes of all recent committees, the review team found that the AVA's committee structure was generally well established and operated effectively. The changes which have occurred to the governance structure since incorporation appear to have been managed efficiently. However, it is a condition of the renewal of the licence that the responsibility for the operation of the AVA licence is clearly located with the Board of Trustees; and that the position of the Executive Director as a trustee of the company is reviewed, in order that the AVA can ensure full separation of governance and management.

Principle 3

The organisation is aware of, and in a position to meet, its legal and public obligations

27 ASOCN's status as a company limited by guarantee furnishes it with a formal legal identity, and provides it with a basis for entering legal agreements and for ensuring the proper conduct of its financial affairs.

28 Prior to incorporation, the AVA was hosted by the University of Essex which provided many key services, including the employment of ASOCN staff. As a company, ASOCN has now become the employer of its own staff. The University, however, continues to provide a number of services including personnel, payroll and pensions, IT and network services, and accommodation. In each case, service-level agreements are in place which regulate the relationship between the two bodies, and ensure the essential independence of the AVA. The review team was able to examine these agreements and would commend the AVA for their impressive range, nature and detail.

29 The review team considers that ASOCN meets the criteria expressed under Principle 3.

Principle 4

The organisation is able to manage effectively its AVA responsibilities and the structure which supports them

Aims

30 A general statement of the aims of ASOCN is contained in the company's Objects. These include 'promoting and widening of participation in education and training, particularly by those who have benefited least from existing provision' and, more generally, promoting access and flexibility in educational opportunities. The Objects have also been recast to form the aims of ASOCN as expressed in its Strategic Plan (2002-2005). A separate mission statement, which is included as part of the application form for prospective members, is also built around enhancing the quality and flexibility of learning opportunities. The aims of ASOCN include a specific reference to its status as an AVA. In all respects, the aims of the AVA are consonant with those expressed in the Principles of the QAA Recognition Scheme for Access to Higher Education.

Strategic planning

31 The AVA produces a three-year strategic plan, the most recent of which covers the period 2002-05. Other than a statement of the AVA's mission, context and purpose, the plan consists of six very broad themes: curriculum, partnerships, marketing, quality, finance, and modernisation. It is the practice of the AVA to take these themes and, on an annual basis, translate them into an operational plan which has more specific objectives. Responsibility for monitoring the achievement of the targets within the operational plan is held by the Board of Trustees. The Account noted that, following a review of the process, strategic planning now involved the full staff team of ASOCN as well as the Board of Trustees.

32 In its Account, the AVA stated that the annual operational plan included 'agreed objectives, targets, responsibilities and resource implications', although this level of detail was not apparent in the Operational Plan 2002-03 contained in the papers made available for the review. The review team noted that ASOCN had supplied a separate AVA Specific Action Plan (2003-2004) in its most recent annual report to QAA, although this plan was not included in the review documentation. The provenance and status of this plan were unclear. The team heard that it was produced on an annual basis and monitored by the Quality Committee, but there was no evidence of any systematic monitoring in the committee's minutes. The AVA may wish to consider whether it might be more efficient for this action plan to be incorporated into its overall operational plan.

33 The AVA's Account provided a useful evaluation of some parts of the strategic planning process; in particular, the AVA intends to respond to a recommendation contained in its 2003 NOCN Quality Audit report which suggests that the outcomes of annual self-assessment should inform strategic and operational planning. The review team also heard that the AVA has aspirations to move to a more flexible rolling operational plan which would not be fixed annually but into which issues could be fed as they arose.

34 The review team concluded that some further development of the AVA's strategic planning processes would be required as it considers:

methods by which self-assessment outcomes and strategic planning can be coordinated;
the level of detail contained in its strategic and operational plans;
the nature and status of the AVA Specific Action Plan and its possible incorporation into the operational plan;
the possible adoption of a 'rolling' operational plan.

35 It is thus a condition of the renewal of the AVA licence that further details of a revised strategic and operational planning process are provided.

Financial management

36 With delegated authority from the Board of Trustees, the Finance and General Purposes Committee (FGP) plays an important role in determining ASOCN's financial policies and strategies, as well as undertaking the detailed monitoring of financial performance. The minutes and papers of the FGP demonstrated that the committee exercised an appropriate level of scrutiny and control of the financial affairs of the AVA. A particularly comprehensive and clear set of Financial Rules & Regulations is also in use.

37 The AVA maintains a contingency fund of a minimum of three months' operating costs and a reasonable level of reserves. These may come under some pressure in the future as the AVA has recently set a deficit budget in order to invest in further staffing as a means of stimulating additional business. Although the review team was not wholly convinced that the AVA had undertaken sufficiently precise targeting of sources of extra income, it had no general doubts about the financial management of the AVA.

Operational procedures

38 The key administrative systems and processes within the AVA, including those covering registration, programme submission, and moderation, appear to be well embedded and managed. The operational procedures which support these processes are detailed in a number of separate documents. The AVA may wish to consolidate these documents into a single manual. In so doing, it should take this opportunity to update those documents which contain outdated references (including, for example, the use of 'kitemarked' in the ASOCN Submission Document).

39 In its Account, the AVA notes that the introduction of self-assessment procedures in 1998-99 has led gradually to a well-established culture of continuous improvement and a greater staff understanding of the systems and policies which underpin the work of the organisation. The annual self-assessment report is detailed and well structured. As noted above (paragraph 32), further coordination between this process and overall strategic planning would be beneficial for the AVA.

40 The AVA's approach to risk assessment is less well developed. A separate paper, Risk Analysis, was among the documents provided for the review, but there was little evidence that the systematic assessment of risk was part of more general strategic and financial planning. The Risk Analysis itself, especially in those sections devoted to Access, appeared to base its judgements on an assessment of the current security of activities, rather than an analysis of potential future risks. The review team would recommend that the AVA reconsider its approach to risk assessment, with a view to embedding it more firmly in all of its planning activities.

Transparency of procedures

41 The AVA maintains a comprehensive set of mandatory and non-mandatory employment and other policies. These include a statement on equal opportunities, which underpins the organisation's approach to membership, promotional materials, staff selection, and curriculum. The review team noted that a statement on equal opportunities was also a requirement in submission documents, and that the principles which governed the AVA's approach to moderation required moderators to comment generally on the issue of inclusivity, and specifically to examine methods of assessment for 'fairness, equality of opportunity and clarity'.

42 ASOCN's arrangements for the receipt of appeals from providers cover moderation decisions, the procedures or outcomes of recognition panels, and the verification of the award of credit. The procedures for hearing such appeals are systematic and thorough, and the review team would suggest that they should be extended to cover complaints or appeals from providers on all appropriate aspects of the operation of the AVA. The internal grievance procedure which applies to staff of ASOCN requires some revision since, in some cases, the Chair of the Board of Trustees, who may be involved in earlier stages of a grievance, is also given the power to make a final decision.

43 The review team also noted that ASOCN 'does not deal directly with appeals or grievances from individual learners. Any grievance from a learner should be addressed by the provider's own procedure'. While not querying this position, the team would suggest that ASOCN, as the validating body, should retain responsibility for monitoring that the provider's own appeals procedures are robust and have been exercised properly. In this sense, it should be open for an individual learner who has not achieved a satisfactory resolution with a provider, to request that the AVA verifies that the provider's procedures have been correctly implemented. The team would suggest that the AVA also reviews the information requirements for programme recognition, to allow it to confirm that providers have appeals procedures in place.

44 It is thus a condition of the renewal of licence that ASOCN makes appropriate amendments to its arrangements for appeals and complaints to ensure that:

the procedures for complaints or appeals from providers are extended to cover all appropriate aspects of the operation of the AVA;
the internal grievance procedure is amended to remove any potential conflict of interests;
it develops procedures to respond to individual learners who have not achieved a satisfactory resolution with a provider, to verify that the provider's procedures have been properly implemented.

Staffing and accommodation

45 ASOCN's premises in the Wivenhoe House Conference Centre provide the AVA with a suite of offices, as well as access to meeting rooms within the Centre. The AVA employs 13 staff (10.32 full-time equivalent) led by the Executive Director and including an Operations Manager, a Quality and Curriculum Manager, development officers (one of whom has a particular responsibility for Access), and an administrative team. All staff have clear job descriptions and are offered appropriate training opportunities. The AVA has an established system of full staff meetings and specific team meetings to assist in the achievement of its targets. An annual staff appraisal process informs both the development of individual staff and the planning of the future staffing needs of the organisation as a whole.

Data collection and annual reporting to QAA

46 Until 2003, ASOCN maintained its own customised database which, in its Account, it described as 'efficient', although concerns had been expressed over its future capacity. In common with other OCNs, ASOCN now uses the OPUS database. The Account was equally helpful in describing the problems which were being encountered by staff in the development phase of this new system, although the review team heard that these were not affecting the operational efficiency of the AVA.

47 The AVA continues to develop the electronic transfer of information from its providers, although the review team learned that, for the purposes of the annual report to QAA, there had been little success to date in securing the learner statistics from providers which they submit to the Learning and Skills Council, usually via their management information systems. The team would recommend that the AVA continues to pursue this route for the provision of the required data. The team noted the AVA's acknowledgment of the importance of this data and its recognition that further work on collecting and using these and other key statistics was necessary.

48 The review team noted the evidence that the AVA was making use of statistics and other data to inform its evaluation and planning processes. In particular, local and national data from 2001-02 have been compared in respect of gender, ethnicity, disability, and learner achievement. The AVA has plans to extend the use of comparative data to inform its development of Access provision.

Support for providers and communications

49 The Account notes the well-established annual development event for examination officers and administrative staff from member organisations, but the AVA considers the ATF to be the main networking and development activity for Access practitioners. As noted in paragraph 25, above, practitioners value the opportunities provided by the ATF which has, in the recent past, begun to address some of the key issues in Access provision.

50 The AVA uses a variety of methods to maintain communications with members, which include its web site, direct contact with designated staff in provider institutions, staff development events, and the work of the ATF. The Access practitioners and members of the ATF met by the review team confirmed the view expressed in the AVA's Account that its communications were efficient and effective.

51 The review team noted that the AVA does not currently request examples of current promotional materials from its providers. It is a responsibility of a validating body to ensure that the programmes which it has approved are presented accurately and fairly to potential learners in such promotional materials. The team recommends that the AVA establishes a procedure which allows it to request and scrutinise promotional materials issued by providers.

52 In summary, the review team concluded that, while the AVA's organisation and operational structures and processes meet most of the requirements expressed through the criteria for Principle 4, weaknesses in certain aspects of strategic planning and some omissions in its appeals procedures need to be addressed in order for the AVA to be able to meet those criteria in full.

Principle 5

The organisation is able to assure the quality and fitness for purpose of Access to HE programmes at the point at which they are granted formal recognition

Programme development

53 One development officer has specific responsibility for Access to HE. The officer manages the programme development process as a whole, and works with providers in their preparation of programme documentation. This task involves ensuring that ASOCN's requirements are met, including those which are specific to the recognition of Access programmes.

54 The Account makes reference to difficulties experienced in relation to some providers' ability to meet scheduled deadlines for the development and validation processes. The AVA recognises that there is a need for further work in this area and AVA officers outlined to the review team measures being taken. The team was satisfied that these were appropriate and, when carried through, would address the problem identified.

55 The procedure for submitting a programme for approval is set out in the AVA's Submission Document guidelines. These general guidelines outline the information that must be provided on a standard AVA template. Though not designed specifically for Access programmes, there are some particular requirements for additional information that must be provided in Access programme submissions, including information about HE progression arrangements. HE staff are commonly involved in the programme development process, for example through providing advice on curriculum content or the requirements of particular progression routes, although this is not an AVA requirement, and no mention is made of the AVA's expectations in this regard in the Submission Document guidelines. Currently, the draft guidance for programme recognition panel participants (see paragraph 56, below) indicates that recognition panels will require evidence of 'appropriate liaison between providers...and receiving HE institutions', but it is not clear that liaison should occur during the development phase, nor what constitutes 'appropriate' liaison. The team would therefore recommend that the AVA clarifies its expectations about HE involvement in the development of Access programmes for validation.

56 The Submission Document guidelines also make particular reference to the special requirements for Access programmes in relation to credit targets. The requirement at this point is that potential Access providers should 'state the proposed credit target and specification for successful completion and award of the kitemarked [sic] Access Certificate', although this document does not include the AVA's requirements in this regard. Detailed information about the AVA's requirements is, however, provided in the Guidelines for Approval Participants. The guidelines include a recently updated set of specific requirements for Access programme recognition (Recognition of Access Programmes), to be considered for approval by the Quality Committee in March 2004. This document states that evidence must be provided that 'the overall programme credit and curriculum structure relates to QAAHE requirements for the award of an approved Access to HE Certificate'. These requirements are given as being 'that a minimum of 500 total notional hours is provided' (a requirement not made by QAA) and also that 'a minimum of 16 credits may be gained, 12 at level 3 and 4 at level 2 or 3'.

57 The review team noted, however, that beyond the statement of the national credit minimum, the AVA did not have a developed policy in relation to credit targets on Access programmes. The actual minimum specification for the achievement of an Access certificate on different programmes varies across the AVA, and the team was unable to discern any clear pattern or rationale to explain those differences. The team considered that, while the AVA's current policy provides some security in relation to the national minimum requirement for achievement on QAA-recognised Access programmes, it could, nonetheless, allow some inequity of demands on learners for the award of the Access certificates on different programmes recognised by the AVA. It could also lead, in effect, to demands for differing kinds of achievement for entry on to the same higher education course. The team concluded that there should be greater standardisation of the number of credits necessary to achieve the QAA-recognised Access certificate, and a clear rationale for any differences in credit targets.

58 The review team also recommends that the AVA amends its formal documentation to ensure that its statement about national requirements is consistent with statements in QAA's current documentation.

Programme approval

59 The draft statement, Recognition of Access Programmes, specifies that 'the approval of Access to Higher Education programmes will involve representatives with appropriate subject experience or expertise and/or experience of Access or validation procedures drawn from further, adult and higher education...', and evidence presented to the review team indicated that higher education representation on Access recognition panels was customary. In addition, the team heard from development officers, Access tutors and representatives from higher education that programme recognition was a rigorous process in their experience. Submitting tutors also confirmed that discussion at recognition panels extends to a detailed examination of the proposed curriculum and the programme's fitness for purpose and that, while panel decisions are reached by consensus, they do not themselves participate in this part of the process. The Executive Director chairs all Access recognition panels. On the basis of their examination of recognition panel reports and discussions with tutors, AVA staff and committee members, the team concluded that recognition panels are comprehensive in their examination of the programme submissions and rigorous in their methods and operation.

60 After the recognition panel, the programme is referred to the ASOCN Quality Committee for formal validation as a QAA-recognised Access to HE programme. A standard letter informs the provider that the programme has been approved, and the provider is sent a copy of the final programme document and is informed that this countersigned document will thenceforward be regarded as the definitive version of the programme document. It was evident to the review team, from its examination of the minutes of the committee and from discussions with committee members and AVA officers, that this process worked effectively.

61 It is also the Quality Committee's task to ensure that any conditions set by the recognition panel are met before final approval is given. The AVA will not register learners on a new programme until this approval has been granted, although it is possible for a provider to choose to enrol learners on a programme, and for the first run of that programme to start, before final approval takes place. AVA officers were clear that providers were advised that a programme should not run until it had been approved, and that, if a provider chose to run it nonetheless, it would be advised to inform any learners enrolled that the programme might be subject to change. The review team concluded that, while the AVA was providing appropriate advice in these circumstances, there were some inherent risks for all involved in such a situation. The team would therefore recommend that the AVA review this aspect of the recognition process, and consider whether the strength of advice provided is sufficient for its role as an awarding body.

62 In summary, the review team concluded that while the recognition process was essentially sound, in order to meet the licensing criteria in full, the AVA should develop its policy in relation to the number of credits necessary to achieve the Access certificate, in order to clarify its requirements and promote consistency within and across the AVA's recognised Access provision.

Principle 6

The organisation is able to safeguard the continuing quality of Access to HE programmes, and to secure the standards of achievement

Moderation processes

Moderator selection, appointment and training

63 Individuals wishing to become Access moderators are asked to apply formally to the AVA. The Executive Director approves appointments on behalf of the Quality Committee, and these are subsequently endorsed by the committee. The criteria used to establish whether an applicant has the necessary skills and experience to act as a moderator are set out in the AVA's Allocation Checklist. There is a formal agreement for the appointment of moderators which, together with the Allocation Checklist and the Moderation Handbook, sets out the duties and responsibilities of each moderator. Initial approval as an ASOCN moderator is made without reference to any particular programme. Formal appointment to a programme is made subsequently.

64 Moderator contracts run for one year and moderators are re-appointed annually, although five of the eight current moderators for Access have moderated the same programme for at least five years. While the Quality Committee is able to monitor the length of moderator appointments through the annual re-appointment process, the AVA has no regulations governing the maximum length of time that a moderator can be associated with a particular programme, or a systematic mechanism for checking the length of service of any particular moderator. The review team considered that this situation did not represent good practice and that the AVA should introduce a standard maximum term for the length of service of any moderator with one particular programme.

65 The Quality and Curriculum Manager is responsible for the provision of training and support for all moderators and works with the Access development officer and other administrative and operations staff to ensure systems are in place for all Access to HE programmes. Training is offered to all moderators, who are required to attend at least one training session every two years. Discussions with moderators and AVA officers indicated, however, that this requirement had not been met in a number of cases. It was also clear that the AVA did not have in place structured ways of monitoring or ensuring moderators' attendance, nor remedial mechanisms to address the situation should they fail to attend.

66 The AVA aims to ensure that each moderator works with more than one programme and at more than one provider and, during 2002-03, eight moderators were appointed to cover 11 programmes. Moderators are appointed to moderate whole programmes, which may include a variety of curriculum areas, but they are not required to have particular expertise in relation to those various curriculum areas. Rather, the AVA meets the need for curriculum expertise within moderation through requiring moderators to monitor and report on the soundness and operation of providers' internal moderation systems and to comment on whether they have received internal moderation reports at the time of final moderation. However, from the evidence supplied to the review team, including the Moderation Handbook, the AVA's pro formas for moderators' reports and discussions with moderators and providers, it was apparent that, while the AVA guided moderators to confirm information about moderation arrangements, there was no formal or structured requirement for providers to furnish moderators with internal moderation reports, or for moderators to use these in making moderation judgements. The team concluded that, should the AVA choose to continue to use a moderation system that relies on internal moderation to assure the standards of achievement within different curriculum areas, routine reference to internal moderation reports is essential for sound judgements to be made. The AVA should therefore develop its moderation procedures to ensure that students' achievement of adequate academic standards in different curriculum areas is subject to a rigorous system of moderation.

Monitoring and responding to the outcomes of moderation

67 The AVA monitors the receipt of reports from its moderators and provides a summary of outcomes to the Quality Committee. The Account states that for 2002-03, of the 21 reports received, 48 per cent were received within the target of 14 days, with the rest being an average of 17 days late. The AVA is aware of the need for improvement in these response times and is reviewing and revising its requirements. The review team would suggest that an improvement in attendance at moderator training, discussed above (paragraph 65), may also help to improve reporting response times.

68 The review team noted that there is some confusion within the AVA's documentation in its use of the terms 'verifier' and 'moderator' and associated terms. The AVA is aware of the potential for misunderstanding, and explained to the team that it was currently changing its use of these terms. The team recommends that the AVA clarifies its preferred usage and adopts the preferred terminology consistently in all its documentation.

69 The receipt of moderators' reports is logged by the AVA's operations staff and this triggers payment to the moderator. Reports are then read by a development officer, who takes action in response to particular issues or concerns raised about individual programmes. These actions can take a number of forms and can involve dialogue with providers at senior level or with course teams or further work with moderators. Following discussions with the AVA's representatives, and the examination of reports upon which the AVA had initiated and completed action, the review team was satisfied that appropriate action was taken within an acceptable timescale. A summary report of actions taken by the development officer is submitted to the Quality Committee. Although satisfied that the system currently operates in line with the criteria expressed within Principle 6, the team concluded that the security of the system is only assured by the use of internal audit, and the team would wish to repeat the concerns expressed in paragraph 22, above.

70 The review team noted, however, that the AVA did not draw upon the information contained within moderators' reports for any purpose other than the addressing of specific points and issues raised in connection with individual programmes. The team considered that these reports were a valuable resource, providing a means of identifying common areas of concern or examples of good practice, which had the potential to contribute more fully to the AVA's continued quality enhancement. The team concluded that the AVA should make further use of these reports to inform strategic planning for the AVA itself and as a source of information for advice to support the development and planning of provision by its providers.

The award of the Access to HE certificate

71 The AVA has adopted the agreed national minimum specification for the award of an Access certificate, for those AVAs which operate within the NOCN credit framework, with a general requirement of 16 credits, with at least 12 at level 3. However, as noted above (paragraph 57), there is a range of minimum credit specifications in operation on different programmes across the AVA, and it is the responsibility of the moderator to judge whether these have been met on a particular programme before recommending certification to the AVA. It is the additional task of the AVA staff to check whether the learner has achieved the general AVA minimum specification (but not to re-check the achievement of the minimum specification particular to an individual programme or pathway).

72 Apart from the concerns expressed above in connection with the insufficient use of internal moderation reports, the team considered other elements of the process for the award of certificates to be sound.

The issue of Access to HE certificates

73 The procedures for issuing certificates are clear and strictly managed, and certificates are issued promptly, following the receipt of the correct documentation by the AVA. The team was satisfied that certificates were issued on a proper basis, in the correct format and within an acceptable timescale.

74 In order to meet the licensing criteria in full, it will be necessary for the AVA to review the requirements of its moderation system. The AVA should develop and implement a policy that sets out the maximum length of time that a moderator can serve on any one particular Access programme. It should also specify its requirements for moderators to attend moderator training, together with details of the action the AVA will take should these requirements not be met by a moderator. The AVA should also consider the means by which it is assured of standards of academic achievement at the subject level and, should it wish to continue with its current model, it must ensure that providers make available to moderators the reports generated by internal moderation of Access programmes, and that moderators take account of the evidence from these reports when making moderation decisions. The AVA should also put in place a mechanism that ensures that moderators have access to appropriate subject expertise in situations in which internal moderation reports are not available.

Principle 7

The organisation is underpinned by structures and processes which enable it to review, evaluate and develop the Access to HE provision for which it has responsibility

75 Prior to 2001-02, providers were not required by the AVA to submit annual programme reviews and few chose to do so. The Quality Committee decided that this would become a formal AVA requirement from that date and a format was agreed and supplied to all Access providers as a template. Since that date, there have been difficulties, however, in getting a full response to the AVA's request and, at the time of review, a significant number of reports were still outstanding from 2002-03. Both the Account and discussions with members of the Quality Committee indicated that the AVA was aware of these problems and had discussed ways of ensuring that this requirement would be met in the future. It is the team's view that annual programme reviews could provide information essential to the AVA in monitoring programmes and in providing information for its own planning processes. In order to achieve these ends, the AVA will need to be firmer in insisting that its requirements in this regard are met by all providers for current and future years.

Standardisation activities

76 The AVA's Account and discussions with AVA staff and Access tutors indicated that standardisation across Access programmes was undeveloped. The AVA recognises the importance of this activity and plans that the ATF and the team of moderators should both play an increasingly important part in this developing process. The review team welcomes this view and concludes that the ATF is a valuable resource and that the AVA needs to build on its enthusiasm and knowledge.

Learner experience and progression

77 The review team heard from former Access students that, generally, they had been well prepared for entry to higher education. Higher education staff interviewed by the team supported this observation, identifying particular strengths of Access learners in relation to study skills and preparation for the higher education experience. While the reviewers did not doubt that the majority of students do, indeed, progress successfully and display these particular strengths, they would nonetheless welcome a proposal from the AVA to research this in more detail to evaluate the effectiveness of provision, for example through a survey of past learners.

Revalidation of programmes

78 Programmes are recognised for five years from the date of formal approval. The AVA reminds providers of the need for revalidation one year before the end of that five-year period and provides advice and guidance on the revalidation process. This process is essentially the same as that used for initial recognition and the team would wish to reiterate the earlier observations and conclusions it has made about this process under Principle 5.

79 The revalidation process takes into account issues and actions arising from previous moderators' reports, and annual reviews (where these are available). The moderator is involved in the revalidation process from the development to the panel stage. The team considered that the AVA's periodic revalidation process was able to assure the continuing quality of recognised Access programmes and their fitness for purpose.

80 The review team concluded that, in order to meet the licensing criteria expressed under this principle, the AVA should develop its system for the regular review of Access provision in relation to the quality and comparability of programmes and learner outcomes and achievements. In particular, the AVA should expand its standardisation activities across programmes, levels and curriculum areas in a structured a systematic way, and reinforce its requirement for annual programme reviews from providers.

Conclusions

81 Anglia South Open College Network (ASOCN) is a firmly established organisation, with roots in both Access and separate OCN work, and an integrated approach to those two aspects of its current work. The organisation's operation as a company is securely underpinned by fully developed and comprehensive documentation relating to its legal and constitutional arrangements. Governance structures are appropriate to the organisation's functions and responsibilities, including those relating to its status as a licensed AVA, although some few amendments to committees' remits are necessary for absolute clarity. On the whole, committees operate effectively: while clearly supportive of, and committed to, the organisation and its work, they have the necessary expertise to provide appropriate advice to the AVA and sufficient external scrutiny in their oversight of the AVA's affairs. The direct involvement of members through the work of the AVA's committees underlines a healthy balance in the relationship between the governance, management and wider membership interests of ASOCN.

82 The operational procedures and structures through which the AVA manages its business are broadly sound. While its financial affairs are regularly and appropriately monitored, the AVA may wish to develop further certain aspects of its approach to risk management and financial planning to protect this currently stable, but small, organisation against changes which could, should circumstances become less favourable for its operation, destabilise the AVA quite quickly. With regard to risk management, this involves attempting to anticipate and guard against the effect of possible future risks to its current position, rather than relying principally on an assessment of immediate and present risk. In its financial planning, the AVA would be well advised to ensure that it has explored the detailed evidence of potential additional income related to opportunities for growth, and the risks attached to investment in these opportunities, before making a long-term commitment to funding new areas.

83 Staffing and accommodation are adequate to its current purposes, and the AVA is considered approachable and responsive with regard to programme recognition and student certification by those involved in programme delivery. While it is appreciated by its members as a supportive and effective service provider, it will need to develop a more robust approach to other aspects of its role, if it is to meet its full responsibilities as an awarding body. An absence of necessary authority is evident in a number of areas, for example in relation to the AVA's approach to credit targets. As a result, there are inconsistencies in practices and some limitations in the AVA's ability to assure the quality and standards of the Access provision for which it is responsible. There is much work to be done to achieve reliable and effective mechanisms for standardisation, and the AVA will need to be more proactive if it is to be able to demonstrate that equivalence of outcomes across the AVA's Access provision is assured.

84 There are particular causes for concern relating to moderation. As external moderators are not required to have curriculum expertise which is specific to the programme that they moderate, the AVA's moderation model relies heavily on the effective scrutiny of internal moderation for the assurance of academic standards at the subject level. However, its requirements in relation to internal moderation and the use of internal moderators' reports are under-developed, and the procedures which would make this process reliable are not clear. This poses a threat to academic standards and, ultimately, to the integrity of Access certificates issued in QAA's name.

85 The emphasis on the AVA's primary role as service provider also under-plays its role in instigating and facilitating development and innovation. ASOCN is a relatively small AVA in terms of providers and student numbers, and although there has been a recent increase in Access learner registrations, they still account for a small proportion of the total number of learners registered with ASOCN. Like the organisation itself, ASOCN's Access provision is well established, with most programmes having operated through the AVA for a number of years. The relatively small percentage of Access learners, together with an absence of significant development of Access provision, suggests that there may be opportunities to extend Access provision in the region, which have not yet been explored. The AVA has a positive relationship with member higher education institutions, particularly in terms of facilitating the entry of students to local higher education institutions. There are also opportunities for the AVA to build on this relationship in developing Access opportunities in the region.

86 There is something of a disconnection between the organisation's strategic planning processes as a whole and the formulation of particular targets for Access, and this needs to be addressed to ensure that developmental activities relate both to officers' assessment of areas which require attention and to the organisation's larger strategic framework. The AVA does not presently make full use of the information available to it to evaluate and review current provision. This can be seen in relation to the limited use made of moderators' reports, annual review reports, the AVA's self-assessment reports, and the activity and potential of the Access Tutors' Forum. Completing that cycle of review and evaluation would enable the AVA to achieve a detailed overview of the current situation from which it would be better placed to develop a more strategic approach to increasing the range and type of Access provision.

Commendations

87 The AVA is commended for:

i the range, nature, and detail of its service-level agreements with the University of Essex (paragraph 28).

The AVA licence

Review outcome

88 Anglia South Open College Network is awarded a conditional renewal of its AVA licence, with conditions to be met by the dates specified below.

Conditions

89 ASOCN's AVA licence is renewed on condition that the AVA:

i clarifies that the Board of Trustees, as the principal governing body, has ultimate responsibility in respect of the holding and maintenance of the AVA licence, and that the terms of reference of the Quality Committee are appropriately amended (paragraphs 20 and 26);

ii reviews the position of the Executive Director as a trustee of the company and makes necessary revisions to ensure that there can be no conflict of interest for the post-holder in discharging the responsibilities of the AVA's chief officer (paragraphs 21 and 26);

iii provides further details of a revised strategic and operational planning process which considers:

methods by which self-assessment outcomes and strategic planning can be co-ordinated;
the level of detail contained in its strategic and operational plans;
the nature and status of the AVA Specific Action Plan and its possible incorporation into the Operational Plan;
the possible adoption of a 'rolling' operational plan (paragraphs 34, 35 and 52);

iv makes appropriate amendments to its arrangements for appeals and complaints to ensure that:

the procedures for complaints or appeals from providers are extended to cover all appropriate aspects of the operation of the AVA;
the internal grievance procedure is amended to remove any potential conflict of interests;
the AVA develops procedures to respond to individual learners who have not achieved a satisfactory resolution with a provider, to verify that the provider's procedures have been properly implemented (paragraphs 42-44 and 52);

v develops its policy in relation to the number of credits necessary to achieve the Access certificate, in order to clarify its requirements and promote consistency within and across the AVA's recognised Access provision (paragraphs 57 and 62);

vi develops and implements a policy that sets out:

the maximum length of time that a moderator can serve on any one particular Access programme (paragraph 64 and 74); and
its requirements for moderators to attend moderator training over a specified period together with details of the remedial action the AVA will take should this requirement not be met by a moderator (paragraph 65 and 74);

vii revises its moderation system and, should it decide to continue with the current model, develop and implement regulations to:

require providers to make available to moderators the appropriate reports generated by internal moderation of Access programmes;
require moderators to consider the evidence from these reports when making moderation decisions; and
put in place a mechanism that ensures that moderators have access to appropriate subject expertise in situations in which internal moderation reports are not available (paragraphs 66 and 74);

viii develops its system for the regular review of Access provision in relation to the quality and comparability of programmes and learner outcomes and achievements. In particular the AVA should:

reinforce its requirement for annual programme reviews from providers (paragraphs 75 and 80); and
extend its standardisation activities across programmes, levels and curriculum areas in a structured and systematic way (paragraphs 76 and 80).

Conditions i, iv and vi to be met by 1 October 2004

Conditions ii, iii, v, vii and viii to be met by 1 December 2004

Recommendations

90 The review team recommends that the AVA:

i amends the terms of reference of its Quality Committee to clarify its quorum regulations (paragraph 23);

ii considers the role, status and membership of the Access Tutors' Forum with a view to exploiting its full potential to contribute to Access policy and practice (paragraph 24);

iii re-considers its approach to risk assessment with a view to embedding it more firmly in its activities (paragraph 40);

iv reviews the information requirements for programme recognition, to allow it to confirm that providers have appeals procedures in place (paragraph 43);

v continues to develop the electronic transfer of information from its providers especially in order to secure for the annual QAA report, the learner statistics which were submitted by providers to the LSC (paragraph 47);

vi establishes a procedure which allows it to request and scrutinise promotional materials issued by providers (paragraph 51);

vii clarifies its expectations in its documentation, with regard to requirements for the involvement of higher education in the development process (paragraph 55);

viii amends its formal documentation to ensure that it is consistent with current external requirements for the award of the Access certificate (paragraph 58);

ix reviews the advice given to providers in relation to the first permitted run of a new programme, and considers whether the strength of advice is sufficient for its role as an awarding body (paragraph 61);

x clarifies its preferred usage of terms relating to the moderation process and adopts the preferred terminology consistently in all its documentation (paragraph 68).

Appendix

Aims and objectives of AVA review

The aims of the system of AVA review are:

i to provide the basis for an informed judgement by the ARLC about the fitness of the AVA to continue as a licensed agency;

ii to promote public confidence in Access to HE as a properly regulated and respected route into higher education by assuring:

the quality and adequacy of AVAs' systems and procedures;
the quality, comparability and range of AVAs' operations;
the adequacy and comparability of AVAs' standards for approval, moderation and monitoring of programmes;
consistency across AVAs in the operation of criteria for the granting of the Access to HE award;

iii to stimulate reflective and self-critical perspectives within AVAs, as an instrument to promote quality enhancement;

iv to provide an opportunity to identify and disseminate good practice of AVA operations;

v to provide a mechanism for ensuring necessary, and encouraging desirable, improvements and developments in AVAs.

The objectives of each AVA review are:

i to examine, assess and report on:

the development of, and changes in, the AVA since its last review or initial licence, and its plans and targets for the future;
the organisation's continuing viability and robustness and the ways in which the AVA demonstrates sound governance;
the efficiency and effectiveness of the AVA's operational and quality assurance systems;
the range and scope of the AVA's activities, and the appropriateness and value of these activities;
the ways in which the AVA approves and monitors programmes and the ways in which these processes take account of the need for consistency and comparability;
the ways in which the AVA satisfies itself of the adequacy and comparability of standards achieved by students gaining the Access to HE certificate;
the evidence available to indicate the AVA's success in achieving its aims and targets;

ii to identify and report on:

strengths and good practice in procedures and operations;
areas which would benefit from further development;
areas requiring attention

Anglia South Open College Network January 2004

Contents