London Open College Network
August 2000

Foreword

1 The Quality Assurance Agency for Higher Education (QAA) is responsible to the Department for Education and Employment (DfEE) for the recognition of Access to Higher Education courses. QAA exercises this responsibility through a national network of authorised validating agencies (AVAs), which are responsible for the processes of individual course recognition and the award of 'kitemarked' Access to HE certificates to students on QAA's behalf. QAA has developed a scheme for the licensing and review of the AVAs, the principles and processes of which are described in the QAA Recognition Scheme for Access to Higher Education in England, Wales and Northern Ireland. The Recognition Scheme is regulated and administered by the Access Recognition and Licensing Committee (ARLC), a sub-committee of the QAA Board of Directors.

2 This is a report of a major review of the AVA function of the London Open College Network (LOCN) undertaken by QAA. The Agency is grateful to LOCN and to those who participated in the review for the willing cooperation provided to the review team.

Aims and objectives

3 The aims of the system of AVA review are:

i to provide the basis for an informed judgement by the ARLC about the fitness of the AVA to continue as a licensed agency; ii to promote public confidence in Access to HE as a properly regulated and respected route into higher education by assuring:

the quality and adequacy of AVAs' systems and procedures;
the quality, comparability and range of AVAs' operations;
the adequacy and comparability of AVAs' standards for approval, moderation and monitoring of programmes;
consistency across AVAs in the operation of criteria for the granting of the Access to HE award;

iii to stimulate reflective and self-critical perspectives within AVAs, as an instrument to promote quality enhancement; iv to provide an opportunity to identify and disseminate good practice of AVA operations; v to provide a mechanism for ensuring necessary, and encouraging desirable, improvements and developments in AVAs.

4 The objectives of each AVA review are:

i to examine, assess and report on:

the development of, and changes in, the AVA since its last review or initial licence, and its plans and targets for the future;
the organisation's continuing viability and robustness and the ways in which the AVA demonstrates sound governance;
the efficiency and effectiveness of the AVA's operational and quality assurance systems;
the range and scope of the AVA's activities, and the appropriateness and value of these activities;
the ways in which the AVA approves and monitors programmes and the ways in which these processes take account of the need for consistency and comparability;
the ways in which the AVA satisfies itself of the adequacy and comparability of standards achieved by students gaining the 'kitemarked' Access award;
the evidence available to indicate the AVA's success in achieving its aims and targets;

ii to identify and report on:

strengths and good practice in procedures and operations;
areas which would benefit from further development;
areas requiring attention.

Outcomes

5 Following the review of an AVA, the reviewers' report is presented to the ARLC by a member of the review team. The Committee then makes one of six decisions:/p>

i unconditional renewal of licence for a specified period;

ii conditional renewal of licence with conditions to be met by specified date;

iii provisional renewal of licence with conditions to be met and further review visit by specified date;

iv suspension of licence until specified conditions are met;

v withdrawal of licence for operation as an AVA;

vi temporary renewal of licence with request for further information by specified date (decision suspended).

The review process

6 The review was conducted in accordance with the process detailed in the QAA Recognition Scheme for Access to Higher Education in England, Wales and Northern Ireland. The preparation for the review included an initial meeting between LOCN officers and the QAA Assistant Director with responsibility for Access to HE matters to discuss the requirements for the Analytical Account (the Account) and the process of the review; the preparation and submission by LOCN of its Account, together with a selection of supporting documentation; a meeting of the review team to discuss the Account and supporting documentation and to establish a draft programme for the review visit; and negotiations between QAA and LOCN to finalise the programme and other arrangements for the review visit.

7 The review visit took place on 1 and 2 June 2000. The visit to LOCN consisted principally of meetings with representatives of the OCN, including OCN officers; members of the Board of Trustees, Quality Assurance Committee and North and South London Access Validating Committees; moderators for Access programmes; Access programme co-ordinators; HE admissions staff; and former Access students now studying in HE.

8 The review team consisted of Professor Beverly Sand, Dean of Lifelong Learning, University of Derby, and Mr Mike Farmer, Head of Access and Continuing Education, Cheltenham and Gloucester College of Higher Education. The review was coordinated for QAA by Ms Kath Dentith, Assistant Director (Access), Institutional Review Directorate.

The AVA context

AVA statistics 1998-99

Providers offering Access to HE programmes - 35

Access programmes running - 169

Access learner registrations - 5,917

Access to HE certificates awarded - 2,848

Background

10 The London Open College Network (LOCN) was formerly known as the London Open College Federation (LOCF). LOCF became an Open College Network (OCN) in 1989. In the same year, four AVAs were licensed in London by the former Access Courses Recognition Group (ACRG). In 1991, ACRG agreed to transfer these four licences to LOCF, which became a single large AVA, with four Access Validating Committees (AVCs). In 1993, the two AVCs in South London merged to form the South London AVC, and in 1996 the two other AVCs merged to form the North London AVC. LOCF was established as a company in 1994, and in 1998 changed its name to the London Open College Network (LOCN). A new Chief Executive was appointed in autumn 1999.

11 LOCN is the largest AVA in the country. It has 201 members, among which there are nine higher education institutions (HEIs), 31 further education colleges (FECs), 23 adult and community centres, and 113 voluntary/community organisations. HEI and FEC membership has remained broadly constant since the last review in 1994. Thirty five providers currently offer Access to HE programmes, among which there are two HEIs, 26 FECs and five adult and community centres. There has been a slight fall in the number of providers in recent years as a result of the closure of some adult and community centres and mergers of FECs. One hundred and sixty nine Access to HE programmes are registered to run in the current year. Although this represents a substantial fall from the previous year's figure of 251, the change is largely explained by the development of large Modular Access Programmes (MAPs) through which providers have merged or combined previously separate programmes. The number of learners registering on Access programmes has remained fairly constant in recent years at just below 6,000, representing about 20 per cent of LOCN's 30,000 learners.

Financial position, structures and income sources

12 LOCN's main sources of regular income are membership subscriptions; annual programme registration fees; programme validation and re-validation fees; and learner registration and certification fees. Funded projects also make a significant contribution. AVA income and expenditure are not separately identified in the published accounts, although the review team was informed that this would be possible. The latest published accounts for the OCN show a deficit of £31,000 on operating activities, although there is an accumulated surplus of £403,000. The Board of Trustees has instituted a plan to eliminate the deficit in three years. There is a separate charging structure for Access to HE programmes and the main impact of this plan on Access to HE providers will be an increase in charges for 2000-01, the first for eight years.

Mission, aims, objectives in relation to Access to HE provision

13 LOCN's mission is 'to accredit the informal and formal learning primarily of adults, to enable them to take their deserved place in education, training and employment'; its aim is 'the advancement of the education of the public'. The ways in which this aim is to be achieved are specified in five objectives which include: 'improving access to higher education by establishing, carrying on and conducting the work of the National Open College Network (NOCN) accreditation system in London and elsewhere and conducting the work of the London Open College Network as an Authorised Validating Agency under licence from the responsible body'; and 'improving recognition of education qualifications by issuing nationally recognised Access Certificates and credits for learner achievement to be awarded independently of the providing organisations'.

14 The review team considered that the mission and aims were appropriate for LOCN's role as a licensed AVA, and were sufficient to secure the place of the AVA within its operation.

Development policies and strategies within the AVA

15 Policies and strategies in relation to the development of Access provision derive from LOCN objectives and from the aims, objectives and operating statements identified in the operational plan. The plan for 1999-2000 identifies for the first time a specific aim concerned with Access to HE, supported by six objectives and associated operating statements. The review team sought clarification on the process for the development and formal approval of these targets, and was informed that the current plan had been developed between the former Chief Executive and the Board of Trustees. There are no specific provisions to ensure that persons with experience or knowledge of Access to HE are elected to the Board, although the review team's attention was drawn to the provision for up to two co-options.

16 While the AVCs report indirectly to the Board of Trustees, and can comment on LOCN's work as an AVA, and their views can be sought on proposals for policy development, the AVCs have not, as a matter of course, had a formal input into the annual planning process. While the team recognised that the Board of Trustees was the ultimate authority for AVA policy and strategy, it considered that the route to it through the committee structures, and the current absence of any specific reference to a contribution to the AVA's development in the AVCs' remit, may inhibit the effectiveness of the process in relation to Access. Insofar as LOCN's work as an AVA is concerned, the review team concluded that the expressed intention to involve the AVCs in the processes of planning and development for Access was a positive indicator and was one which it would wish to support.

17 Some valuable initial work has been done on establishing benchmark data in relation to Access, but this has yet to be used in any systematic way to inform the planning process. The value of the statistical data that accompanies the annual AVA report to QAA was widely recognised, but the review team found that the use of the report as a whole, and its potential as a tool in evaluating the work of LOCN as an AVA, and in informing the annual planning process, had been less thoroughly considered.

18 The specified objectives themselves reflect current developmental priorities such as the promotion of a more strategic approach to Access provision in colleges, through support for modular frameworks; a requirement for key skills in all Access programmes; and the use of statistical data for benchmarking and other purposes. Although there is a general awareness of the currency of these issues, the objectives did not appear to be widely known or consistently owned across the AVA. At the same time, although moderation was identified by AVC members and others as a key development issue in relation to Access programmes, and LOCN had found it necessary to establish an ad hoc group to examine the issues, the team noted that moderation was not explicitly mentioned in the current Operational Plan. In addition, some members were uncertain about the relationship between the ad hoc group and the main decision-making processes. This example further suggests the importance of reviewing the role of the existing committees in the planning process.

19 In considering mechanisms for monitoring progress towards the achievement of the objectives set the previous year, the review team noted that no systematic monitoring had been undertaken to date, and the extent of progress made with the different objectives was unclear, although an interim report was planned which would fit with the new planning and committee cycle for 2000-01.

Target groups, targeting policies and strategies

20 LOCN does not itself identify any specific target groups for Access programmes, nor does its current Operational Plan include any explicit intentions to do so. However, it does require all Access programmes submitted for validation to state the target groups and the recruitment strategy for reaching those target groups. Moderators are then required to comment in their reports on the extent to which programmes are recruiting the target groups specified in the approved course document, and this enables LOCN to monitor recruitment practices.

21 The review team noted that members of minority ethnic groups are well represented, constituting 43 per cent of learners on Access programmes. Learners identified as 'white' constitute 40 per cent, with 'other' and 'not known' making up the balance. Women constitute 65 per cent of learners, and are in a majority in all ethnic groups except Bangladeshi and Pakistani. Initiatives by colleges to attract Bangladeshi women were reported in the AVA's 1998-99 Annual Report to QAA.

22 The review team considered that the work on benchmark data might usefully be developed to inform the OCN on an AVA-wide basis of its success in targeting different groups, and this, in turn, might lead the AVA to reflect on ways in which it might work with providers and receivers in HE to enhance its widening participation function.

Developments since the last review

23 The report of the last review contained a number of recommendations and LOCN's Account reports progress on a number of these: all Access programmes are now unitised and assessment is based on units of assessment; attention has been paid to the representation of under-represented groups on panels and committees (for example, 17 per cent of panel members and 16 per cent of moderators are from minority ethnic groups); the London Access Courses Directory has continued to be published annually, part-sponsored by member HEIs; and data collection has developed, reflecting LOCN's view that this is one of its core functions.

24 The review team found that some progress had been made in the sharing of practice between AVCs, through the reduction in their number from four to two, and by cross representation between them. Systems and procedures are fully documented, including those for moderation reports. However, the team found that less progress had been made on the development of links between LOCN's quality assurance systems and those of the member organisations. This is commented on more fully with specific reference to moderation in paragraph 59 of this report.

Governance and committee structures

Legal status and constitutional position

25 LOCN is an independent company limited by guarantee, and registered as a charity. It is managed by a Board of Trustees, elected by the members at the Annual General Meeting. There are provisions to ensure that the Board reflects the cross-sector diversity of the membership. The Chief Executive is appointed by the Board and attends its meetings. LOCN has adopted a set of principles of membership of all its committees based on those established by the Nolan Committee in 1996, and committee members are accountable to the Board of Trustees. The review team was satisfied that LOCN was an independent and appropriately constituted organisation for its functions as an AVA.

Summary of structures and processes

26 The Board of Trustees has ultimate authority for LOCN as an AVA. It approves the Strategic Plan and Operational Plan, the budget and annual accounts. The Board is supported by a Finance and General Purposes Committee, a Staffing Committee and a Quality Assurance Committee (QAC). The two Access Validating Committees (AVCs) are sub-committees of the QAC. The Account made reference to a possible merger of the two AVCs, but, on exploring the status of this statement with members of the OCN, the review team was informed that, for the time being, the Board remained to be convinced of the need to do this, and members of the AVCs also supported the status quo.

27 The Board of Trustees, the committees and sub-committees all have published terms of reference which are appropriate to their current roles within the structure. The review team noted, however, that where other committees' terms of reference refer to members as 'representatives', the terms of reference of the AVCs refer to 'delegates'. The team suggests that the nature of AVC members' representation and this term be reconsidered when the AVC terms of reference are next reviewed.

The Quality Assurance Committee

28 The QAC advises the Chief Executive and reports to the Board of Trustees on all matters relating to the quality assurance of LOCN's accreditation service. Its specific responsibilities include advising the Board of Trustees on policy relating to quality assurance; confirming recognition of programmes; confirming the appointment of moderators; confirming withdrawal of recognition or conditions on continued recognition as a result of moderation reports; confirming decisions of the AVCs; and advising the AVCs on quality assurance in relation to Access courses.

Access Validating Committees

29 The two AVCs report to the QAC on all matters relating to the quality assurance of LOCN's work as an AVA. Their specific responsibilities include confirming validation of Access programmes; confirming withdrawal of recognition or conditions on continued recognition as a result of moderation reports; and advising the QAC and LOCN officers of issues relating to LOCN's work as an AVA. Their regular duties include the consideration of validation panel reports for Access to HE programmes and the sampling of moderators' reports. They also consider matters of local and national significance in relation to Access.

30 Minutes of the AVCs are received and considered by the QAC, and its minutes are in turn received and considered by the Board of Trustees. It was confirmed that when the minutes of committees are minuted as 'received', this is taken to mean that their decisions are approved, as well as being noted. Indeed, LOCN uses this terminological convention in relation to all committee papers. The review team, while accepting this assurance, nevertheless would wish to draw attention to the possibility of misunderstanding, which has a particular impact on the question of authority for approval of Access programmes discussed later in this report.

31 The review team noted that the AVCs, as well as reporting through the committee structures, also have roles 'to advise LOCN officers', and the team was informed that officers transmit the AVCs' views and concerns directly to the Board. This provides the possibility of dual reporting lines, where the views of the two AVCs might be reported to the QAC or the Board via an officer's report as well as through the normal receipt of minutes. However, the team noted that the job descriptions of Development Officers included responsibility to support and advise panels and committees, and there appeared to be some potential ambiguity in the relative roles of committees and officers. The team acknowledged the critical importance to LOCN of its officers to be able to develop proposals for policy and strategy, but considered that, in order to ensure the effective operation of the committees, LOCN should give further consideration to the relationship between committees and officers to resolve this ambiguity, and to clarify, in particular, the AVCs' formal reporting line to the Board of Trustees.

Operational management and administrative arrangements

Summary of staffing structures

32 LOCN has a Chief Executive and Deputy Chief Executive. The latter also acts as Company Secretary and has overall responsibility for ensuring that LOCN maintains a comprehensive quality assurance system. These two form the Senior Management Team and report to the Board of Trustees. There are 13 other full-time and part-time staff, roughly half of whom undertake development and moderation duties, and half of whom have administrative and finance duties. Temporary staff are sometimes used, and some work (for example, the production of the London Access Courses Directory) is contracted out. The review team noted that the elimination of the budget deficit might have an impact on staffing levels over the next three years. Currently, however, staffing structures, the numbers of staff, and their disposition in respect of AVA work, appeared to be adequate and appropriate.

Roles and remits of staff

33 All posts have a job description and person specification, providing a clear description of the role and remit of the post in question. The Development Officers form a team which meets regularly and is consulted by the Senior Management Team. Members of the development team support providers, LOCN committees, validation panels, moderators and specialist forums and workshops. The Development Officer (Access) has the primary quality assurance function in relation to Access, having particular responsibility to 'support the development of policy...in respect of Access' and 'to service and support the work of the AVCs'. This includes reading all Access moderator reports and providing a report on any issues identified, using a standard report format. Although only one DO works exclusively on Access work, all DOs and the Moderation Officer undertake some duties related to Access, and all appear to understand the particular requirements of this area of work.

34 Administrative staff undertake the first line checking on responses to panel conditions, moderator reports, recommendations for the award of credit and Access certificates, and payments to moderators. All LOCN processes as set out in the Service Standards are underpinned by rigorous and detailed office procedures which are well-understood and owned by administrative staff, and which are recorded in a comprehensive procedures manual.

35 The review team was satisfied that LOCN's AVA functions were well supported by operating procedures, and organisational arrangements and structures which included appropriate experience and expertise within its staff team.

Premises

36 LOCN primarily operates in London postcode areas and in London boroughs. It has a five-year lease on accessible premises in Islington. They are modern and self-contained and provide office space, staff facilities, a seminar room, and a conference room where recognition panels take place.

Data management, collection and analysis

37 LOCN uses the AREV database system, in common with 15 other OCNs. Although it is adequate for most reporting functions, it needs a certain level of operating expertise and considerable technical support, and LOCN is looking at alternatives. It is awaiting the result of the NOCN consultancy on Information and Communication Technology, and will base any future decisions on those outcomes.

38 Data is collected directly from providers and input by administrative staff, who are generally satisfied with the way in which the data collection processes operate. Outputs provide management information which allows the management team to review the operating position on a regular basis. The statistical data for the AVA annual report to QAA is derived from the database. Two of the six Access objectives in the Operational Plan relate to data. One is the development of benchmark data, enabling comparisons between LOCN and national output measures and this has been welcomed by providers. This data is used internally to monitor achievement of providers against their own targets and in comparison with moderator reports. However, there are no current plans to set targets against these benchmarks. The other objective is to encourage HE members to provide data on Access student performance. It is not clear what progress has been made on this objective. Some HE admissions tutors acknowledged it would be difficult since their institutions did not specifically monitor Access student performance (see below paragraph 63).

Communication with providers

39 LOCN provides an extensive range of annually updated written documentation to support its members. This includes standard letters generated though the database, information about LOCN's processes and procedures, guides and handbooks, termly newsletters and calendars of events. LOCN's annual Report and Accounts is presented to the AGM in February and this contains a section on Access to HE. In addition, the London Access Courses Directory continues to be published annually, with sponsorship from nine HEIs. There has been significant investment in communication systems and LOCN has its own web site. Members of the development and administrative teams give advice and assistance by phone, and Access tutors met by the review team praised the support provided by LOCN officers.

40 The review team was satisfied that LOCN communicated effectively with providers through a range of channels. However, LOCN might also consider making use of its annual report to QAA to provide a fuller and more evaluative account to its members of its work as an AVA.

The development, validation, and evaluation of Access programmes

Development and design of new provision

41 While the AVA does not undertake a planning role in relation to the development of new Access provision, the cross-sector membership of the AVCs is intended to facilitate collaboration between providers of Access programmes and receivers of Access students in the development of new provision. Historically, close links exist between groups of FE colleges and their partner HEIs that support both curriculum development and progression opportunities. LOCN's role in the development and design of new provision is detailed in its Service Standards, which cover both its OCN and AVA operations. The Service Standards set out in detail the requirements and responsibilities of both the AVA and its users, and guarantee a minimum level of service in five areas: programme development, programme recognition, moderation, certification and organisation. Users are invited to comment on delivery of the Standards, and the team was informed that a more systematic review of these would be built into the self-assessment process due to commence within the next planning cycle.

42 New provision is normally initiated by the provider, on the basis of which LOCN will provide support from either the Access Development Officer or the lead Development Officer for the institution. Providers commented favourably on the nature and level of support offered by officers, which covers all LOCN validation requirements and may also include providing staff development sessions in relation to key skills, unit writing, assessment strategies and the implications for managers and Access tutors of developing modular frameworks. Comprehensive standard supporting documentation is provided for submitting teams and all draft Access submissions are read by two development officers. No programme is presented to a recognition panel until officers are satisfied with its quality. LOCN reports buoyant demand for new provision, with over 20 panels planned for the next panel cycle.

Validation of new and revised provision

43 The Service Standards, together with documentation relating specifically to Access validation, ensure that the process is well-supported, rigorous and transparent. Guidance is provided on the duties of panel members, the role and responsibilities of the scribe and the Chair, and a supporting leaflet includes specific matters in relation to the validation of Access programmes. Officers are responsible for ensuring LOCN's requirements for Access panel composition are adhered to, including matters relating to quoracy, HE and other membership. All conditions set by the panel are systematically checked and followed up by officers, and validation is not considered to be complete until all conditions have been met.

44 An appendix to the submission documentation specifically designed for Access provision requires additional details for the award of the Access Certificate, including information about the number and level of credits required, any rules of combination or specific forms of assessment, together with a note on LOCN's requirement for the identification of key skills. LOCN has established a minimum credit requirement of 16 credits, 12 at Level 3, for all Access programmes, and providers are strongly encouraged to set individual programme minimum requirements at this level.

45 In principle, some of this information is summarised on a cover sheet to the panel report, with the specification for the award of the 'kitemarked' Certificate identified on a separate sheet for inclusion with the report. In practice, however, the team noted considerable inconsistency in including this essential information in panel reports. Of the sample reports provided to the team, some had only partial details and others had none at all. Specification for the award of the 'kitemark' was not systematically available. In addition, it was not always clear from the documentation whether the 'kitemark' was attached to specified learner achievement (eg achievement of the minimum credit requirement or that agreed for a particular programme) or to successful completion of the course (eg some programme documentation contained a minimum attendance requirement). The wording of the title of Section 2 of Appendix 1 to the submission document may compound this lack of clarity in that it refers to 'Successful Completion for the Award'.

Authority for approval of 'kitemarked' Access programmes

46 The Account made reference to validation panels operating under 'delegated authority' from the AVCs, and the AVCs' terms of reference include 'confirming' validation of Access programmes after panel outcomes have been reported to them. This position is also stated in literature published by LOCN. Descriptions of the procedures, confirmed by representatives of LOCN, indicate that action is taken on validation as soon as it has been confirmed by the AVC. AVCs may also confirm withdrawal of recognition or place conditions upon continuing recognition. Taking these statements together, it may be assumed that it is the AVCs that have final authority for the approval of Access programmes. However, the QAC's terms of reference state that it 'confirms recognition of programmes' and that it 'confirms the decisions of AVCs'. The review team sought clarification and it was confirmed that the QAC had ultimate authority in respect of Access programmes, including withdrawal of recognition, and that decisions of AVCs were, in effect, recommendations to the QAC. It was reported that although AVCs may decline to confirm the recommendations of a validation panel, and that the QAC may decline to accept a validation recommendation of an AVC, in practice this had never occurred. The precise position has, therefore, not been tested.

47 The OCN's published documentation includes a description of a clear process for appeals against validation decisions. However, this process is somewhat undermined by the lack of clarity described above about where responsibility for validation decisions lies. The team was informed that the QAC held ultimate responsibility for appeals against validation decisions, although this role is not mentioned in the published documentation, nor is it included in the terms of reference of that committee.

48 The review team concluded that there was insufficient certainty about the precise locus of authority for the final approval of Access programmes, as well as that for withdrawal of approval and deciding on the outcome of appeals against non-approval, and considered that LOCN should clarify these responsibilities.

Monitoring, evaluation and review of programmes

49 Access programmes are validated for a period of five years, at which point they are presented for re-validation. During this five-year period, the OCN does not require the providers to produce an evaluation of or report on their programmes, annual monitoring of individual programmes being achieved through the the receipt and evaluation of moderators' reports. The moderator's report and its subsequent action plan provide a yearly overview of any issues, and, on the basis of this, moderators may recommend a change of programme or the addition of modules which may need to be presented to a validation panel. The review team recognised that moderators' reports played a valuable role in the monitoring process, and noted that the Moderation Guidelines required detailed information about a range of matters including recruitment, retention and progression. The procedures for the consideration of moderators' reports are not, however, explicitly directed to the review of provision as a whole. The OCN may wish to consider whether adjustments to the current requirements and procedures for the evaluation of reports provide a wholly adequate mechanism for the review of provision, particularly in the context of QAA's requirements for annual AVA reports. In addition, the OCN may wish to consider whether the introduction of a requirement on providers to reflect on and evaluate the the year's activities, changes and achievements would contribute to the process of quality enhancement of programmes. Undoubtedly, FE providers will already be required to engage in some such self-evaluative activity at the programme level, and, should the OCN decide to develop requirements of this kind for all providers, it may wish to consider how its own requirements would complement those made by providers.

50 Moderators also ensure that any issues of concern are systematically addressed and followed up, and meet students in order to ensure that their views are included in moderation reports. Moderators new to existing programmes are provided with copies of past moderation reports to ensure they are aware of any outstanding issues. Internal institutional self-assessment and quality procedures support this process and providers are invited to respond to moderation reports.

Evaluation of procedures

51 Systematic qualitative and quantitative evaluations of validation panels are produced annually and presented to the QAC for comment and action. The sample evaluation available to the team showed a high level of satisfaction with the process, the main concerns being a shortage of time and perceived lack of subject specialists for large modular Access panels, and the increasing difficulty experienced by FE staff in attending. Both these issues are being addressed at various levels within the AVA, using a range of strategies.

Assessment, moderation and award of certificates

Policies and guidance on student assessment methods

52 Section C of the Programme Submission Guidelines has for some considerable time included a requirement for providers to indicate both assessment methods and evidence of learner achievement. More recently, and in response to NOCN licensing requirements, LOCN has developed an Assessment Code of Practice and an Assessment Policy. While neither of these documents makes direct reference to Access, the review team was informed that their implementation is likely to have considerable impact on assessment within Access provision. Providers will be required to describe in more detail in their submission documentation the proposed assessment methods and the related achievement evidence which will be available for moderators. Panels will be expected to give closer attention to assessment matters, and moderators will be expected to pay more attention to the verification of evidence of achievement.

53 While the Assessment Policy had been formally approved in January and distributed to providers and moderators for implementation in September 2000, its implications for Access provision were not yet widely recognised or understood by Access tutors or moderators. Although it was planned to include assessment requirements in the current training offered to moderators, and to hold some staff development workshops during the summer term, the review team had some doubts as to whether this would enable the policy to be implemented effectively within the proposed timescale.

Selection, appointment and training of moderators

54 Criteria for the selection and appointment of moderators are clear and well documented, as are the recently agreed contractual arrangements. Attendance at a training session prior to or soon after appointment is an LOCN requirement and the sessions are organised and timetabled to maximise attendance. Experienced moderators are required to undergo updating at least once every three years, but this is currently under review. The scale of moderation activity is substantial and the Finance and General Purposes Commitee has asked the QAC to 'consider more efficient models of moderation'.

55 The moderation process is the key mechanism through which the AVA assures itself of the quality of Access programmes and the standards of learner achievement. A heavy reliance is therefore placed on moderators, who appear to be conscientious and committed in carrying out this crucial role and consider themselves to be well supported by LOCN. The moderation guidelines are effective in underpinning the moderation process and include useful descriptions of roles and duties and standard guidance for completing moderation reports. Moderators commented favourably on increased rigour in a number of areas, including training, feedback and contractual arrangements. The LOCN Moderation Officer is responsible for all matters related to the process of moderation including the allocation of moderators to programmes when the validation process is complete, subject to formal approval by the QAC; support for moderator training; and dealing with moderation reports.

56 As an example of good practice, an accredited training course for moderators has recently been offered to LOCN moderators free of charge, with the intention of supporting and enhancing the quality of the moderation process. Several moderators have taken up this offer.

Recent developments in the moderation process

57 The development by providers of their Access provision from discrete courses into large modular programmes has necessitated for LOCN the 'ad hoc development of a new model to deal with the emerging situation' (Models of Moderation, 20 October 1999). The new model of moderation includes two key elements: a new set of practical arrangements for the moderation of Access provision; and a shift of emphasis in the role of moderator from 'critical friend' to quality auditor, focusing on standardisation and verification.

58 In terms of the first of these elements, the process was piloted and reviewed in 1998-99, and has been implemented during the current year. All modular Access programmes (MAPs) now have appointed to them a Lead Moderator and Pathway Moderators whose duties are described in the moderation guidelines. Lead Moderators are responsible for oversight of the whole programme, for liaison with Pathway Moderators, for arranging the final moderation meeting, and for the submission of a final report covering the programme, which incorporates information from all Pathway Moderators. Lead Moderators are also likely to be Pathway Moderators for one or more pathways within the overall programme for which they are responsible. Pathway Moderators are often those who previously moderated a discrete programme, and their current responsibility is to report to the Lead Moderator in relation to the Pathway that they oversee. While there had been some initial lack of clarity about these new roles, discussions with moderators and tutors confirmed that these are now better understood and beginning to work effectively.

59 However, other aspects of this new system are not yet working as well or as consistently as envisaged. The flow of information between Lead and Pathway Moderators is causing some difficulty, despite the best attempts of moderators, for example by making use of email. Communication can be sporadic and meetings infrequent, resulting in difficulties in the coordination of the process. One moderator reported a concern that Pathway Moderators did not see a copy of the Lead Moderator's report prior to its submission to LOCN. Most importantly, the significance in the new system of the final moderation meeting is not yet well-understood and the review team was informed that due to pressures of time and other commitments it was not always possible for Pathway Moderators to attend. In the light of some of the other operational issues in relation to contact between Lead and Pathway Moderators, this poses a particular risk to the quality process and to the way in which LOCN is able to assure itself of the standards of learner achievement.

60 A further set of issues concerns the second key element of the new process relating to the increasing emphasis placed on standardisation and verification by the moderator. This involves not only a change of role for moderators, but also relies heavily on internal verification and moderation processes. Some providers had internal systems in place and some did not. Where these did exist, there were considerable differences and inconsistencies between them. Moderators had little guidance as to how they were expected to interact with internal systems, and practice was inconsistent. While a model of internal verification had been identified in one member institution as excellent practice which might be promoted and adopted, there was currently no clear process by which this might take place nor was there any certainty that it would be acceptable to all member institutions. The review team was unable to identify any minimum requirements made by LOCN, either in relation to the quality standards for systems and processes of internal moderation/verification, or in relation to their articulation with LOCN's own external moderation processes. LOCN is aware that there continue to be issues of concern around the new model, some of which were identified in 1999, and an ad hoc group has been convened to look at some of the problems. However, the remit of the group, its reporting procedures and timescale were unclear to the review team. The team considered that this matter needed urgent, focused and systematic attention.

Receipt of and response to moderators' reports

61 The Moderation Officer is responsible for receiving and logging moderation reports, which are written to a prescribed format, facilitating comparison and consistency. The Moderation Officer, normally in conjunction with the Development Officer responsible for the institution to which the report relates, reads and evaluates all moderation reports. Any matters of concern are signalled to the institution and are reported to the relevant AVC, although the Moderation Officer will take early action where appropriate. The Moderation Officer is also responsible for compiling an annual evaluation report for consideration by AVCs and the QAC. The report will highlight a range of issues, make recommendations for improvement, and identify any issues of concern. Moderation reports and evaluations are available to the autumn meeting of the AVC for sampling. All reports which raise issues in respect of the quality of programme delivery or the validity of the award of credit are separately identified to the AVC for action. All actions are followed up by the Moderation Officer and reported back to the AVC on a regular basis. The AVCs report any issues of on-going concern to the QAC. Evidence presented to the reviewers confirmed that this system was working well and that difficult issues had been picked up and followed through to appropriate outcomes.

Procedures for the award of the 'kitemarked' Access certificate

62 Recommendations for the award of the 'kitemarked' Access certificate are confirmed and signed by the Lead Moderator at the final moderation meeting. The administrative system in the LOCN office prioritises these awards, and an effective system is in place to ensure accuracy of learner details and security in the issuing of Access certificates. There is currently no administrative check to ensure that credit combinations and levels meet the validated requirements for achievement of the award/successful completion of the programme. This check is assumed to be the responsibility of the Lead Moderator at the moderation meeting. While the review team had no reason to believe that Access Certificates had been wrongly issued, in the light of the new moderation system and other issues identified above in relation to successful completion criteria and attendance of all moderators at the final moderation meeting, the review team considered that LOCN should assure itself that these procedures are completely secure.

Learner experience, standards and progression

Nature of learner experience and preparedness for HE

63 Discussions with HE admissions staff and students indicated that Access students were well prepared for entry into and success within higher education. Students considered that the Access course had provided them with an appropriate and clear route into university, giving them the opportunity to develop essential skills and subject expertise, as well as the opportunity for personal development. In general, students reported that they had been given opportunities for their views to be considered either formally or informally, or both, through representation on course teams, questionnaires, and in discussions with moderators. In several cases action had been taken in response to student identification of issues of concern. The moderation process and the 'kitemark' were seen by students as important external guarantees of quality.

64 HE admissions staff were familiar both with the 'kitemark' and with LOCN's requirements for achievement, and many accepted the achievement of the Access certificate as a standard entry requirement, with any specific requirements included in the rules of combination for the award (for example, GCSE equivalence in mathematics or science). While none of the HE institutions represented specifically monitored the progress of Access students, anecdotal evidence indicated that Access students did as well as, and sometimes better than, other 'standard' and 'non-standard' entrants.

Programme equivalence and outcome standards

65 While there are no formal requirements for establishing programme equivalence across the AVA, many of the AVA's standard processes help to ensure consistency. In particular, tutors pointed out that the move to large modular Access programmes had ensured a much greater degree of consistency and comparability within each providing institution. Access moderators often moderate across programmes in more than one institution as well as being themselves providers. A common bank of Access and other units is available for delivery by different providers, facilitating consistency and equivalence.

66 Data on learner achievement is monitored on an annual basis through figures presented to the autumn meeting of the AVC. Recently, providers requested benchmarking information specifically in relation to retention and achievement on Access programmes for the purposes of comparison with FEFC data, and to assist in the process of evaluating their own programmes and pathways. Figures provided by LOCN on the basis of NOCN data are a useful guide, but contain a number of caveats as to their interpretation. Nevertheless, these benchmarks show a fairly consistent performance across LOCN, with retention and achievement rates appearing to be higher than the national figures.

67 Data provided for QAA indicated that, over the past three years, the total numbers of Access students registered for and receiving the 'kitemarked' Certificate have remained relatively stable. For the past three years around 48 per cent of students enrolled on Access programmes have been awarded the 'kitemarked' Certificate and although this is higher than the national NOCN average of 38 per cent, the issue of successful completion rates is identified as a matter of concern in LOCN's Account, as is the issue of retention. What is not identifiable from the aggregate data is any information on learner profile in relation to retention or achievement, inhibiting the more detailed monitoring which might inform the planning processes of the AVA.

Progression

68 While progression data has been returned to LOCN by Access providers, this has not to date been entered on to the database. As of 1999-2000, progression information will be entered on to the database so that a more systematic analysis can be undertaken. Discussions with HE admissions staff confirmed that there was a wide range of progression opportunities available to Access students, with some historical links between FECs and HEIs still in operation. The main concerns voiced in relation to progression included the variation in rates of progression into certain subjects and particular HEIs, and the making of unconditional offers to Access students prior to the completion of their programme, which might result in students being inadequately prepared, as well as in the distortion of local and national data in respect of retention and achievement.

Conclusions

69 In terms of numbers of members and of programmes, London Open College Network is considerably bigger than any other AVA and, with just under 6,000 registered students, its validated provision accounts for approximately 16 per cent of all Access students on recognised programmes in England, Wales and Northern Ireland. The work of the AVA therefore makes a substantial contribution to the progression of under-represented groups to higher education.

70 LOCN is professionally and effectively managed and its users seem to have confidence that, on the whole, they receive an efficient and responsive service. As far as its AVA functions are concerned, LOCN appears to have maintained a strong sense of identity, commitment and involvement with its members. There is no evidence that its size has a negative impact on its operation in terms of its awareness of its members' concerns and ability to respond flexibly and sensitively to their needs; or in terms of its awareness of the detail of the Access programmes for which it has responsibility as an AVA.

71 The governance and management structures of the OCN are well established, and Access has a readily identifiable place within these structures. Though the Board of Trustees is the locus of authority for the AVA, it is the two Access Validating Committees, separately representing north and south London, which are most explicitly and directly involved in Access matters. At an operational level, this generally works well in the AVA's management of its AVA responsibilities. However, the reporting line through the Quality Assurance Committee has led to some lack of clarity, in particular, about responsibilities in relation to programme approval for Access programmes. The existence of two separate committees enables the wide geographical constituency to be represented, and the necessary detail of quality assurance of large numbers of Access programmes to receive appropriately close attention. The AVCs also act as forums for discussion of Access matters more generally, and they provide a valued opportunity for networking, and sharing good practice and information about Access. They provide a source of experience and expertise on Access matters on which the OCN can draw, although, to date, limited use is made of this expertise in the process of the development of Access-related policy and strategy.

72 LOCN has developed quality assurance arrangements which are thorough and effective for the most part, supported by clear and detailed documentation. The development of modular Access programmes has proved popular with providers as an efficient way of managing their Access provision, and this development has considerably changed the shape of Access provision in London in recent years. The development of MAPs also has the support of the OCN as a means of achieving greater consistency across programmes. However, the rate at which this development has taken place has somewhat outpaced the quality assurance structures - in particular moderation - designed to support them. At the same time, developments in internal quality assurance in FE in recent years have changed the context of moderators' responsibilities. Although LOCN has recognised that structures and processes for moderation need re-consideration and revision, the OCN has yet to address this in a systematic way that will assure the continuing quality and standards of learner achievements across all Access programmes.

73 Access providers have welcomed the development of statistical data for benchmarking purposes. Although some comparative data has been prepared, indicating performance in relation to other AVAs, detailed analysis has yet to be undertaken for the purpose of the AVA's own critical evaluation. Such work will be important for the AVA in the development of policies for Access provision within LOCN.

74 The review team commends LOCN for:

the introduction of a certificated programme of training for moderators;
the range and comprehensiveness of documentation about quality assurance processes produced for its users;
the efficiency and effectiveness of its communications with providers.

Recommendation to the ARLC

The review team recommends that LOCN be granted a conditional renewal of licence with conditions to be met by the dates specified below.

Conditions

75 Subject to the approval of the ARLC, LOCN's licence is renewed on condition that the OCN:

i clarifies which body has final authority for approving and withdrawing approval for Access programmes. The OCN should ensure that the bodies and individuals involved are informed of this, and should initiate a process to ensure that the position is reflected accurately and consistently in the relevant documentation (by 1 December 2000);

ii undertakes a thorough and systematic review of the moderation process, particularly in relation to modular Access programmes, to ensure that effective systems and guidance are in place which clarify, in particular, LOCN's requirements for:

minimum requirements for internal moderation;
articulation of internal and external moderation processes;
nature and level of communication between Lead and Pathway Moderators co-ordination of the moderation process;
conduct and membership of final moderation meetings;
(by 1 April 2001);

iii reviews the process of administrative checking prior to the issue of 'kitemarked' Access certificates, to ensure that all programme-specific achievement or successful completion requirements, as established at validation, have been met
(by 1 December 2000).

Recommendations

76 The review team recommends that LOCN:

i develops the process of strategic and policy planning in relation to the development of LOCN's role and activities as an AVA, identifying the roles of committees and officers within this, and considering mechanisms to secure comprehensive ownership of Access aims and objectives across the AVA;

ii establishes procedures for systematic monitoring of the achievement of the Access aims, objectives and operating statements;

iii clarifies the AVCs' formal reporting line to the Board of Trustees;

iv undertakes regular systematic critical evaluation of the operation of the AVA as a whole, through the planned annual monitoring process, and considers how this process might be reflected in the preparation of the AVA annual report to QAA;

v develops the analysis of Access-related data, in order to inform the planning process, and policy and strategies in relation to targeting;

vi evaluates the development of modular Access programmes to date, and develops an overall strategy in respect of their continuing development across the AVA;

vii reviews the requirements for the presentation of the specification for the achievement of the 'kitemarked' Access Certificate on validation panel reports to ensure that it is identified and stated clearly and systematically;

viii considers whether systems for annual monitoring of Access programmes are appropriate and adequate.

London Open College Network August 2000

Contents