Foreword
1 The Quality Assurance Agency for Higher Education (QAA) is responsible to the Department for Education and Employment for the recognition of Access to Higher Education courses. QAA exercises this responsibility through a national network of authorised validating agencies (AVAs), which are responsible for the processes of individual course recognition and the award of 'kitemarked' Access to HE certificates to students on QAA's behalf. QAA has developed a scheme for the licensing and review of the AVAs, the principles and processes of which are described in the QAA Recognition Scheme for Access to Higher Education in England, Wales and Northern Ireland. The Recognition Scheme is regulated and administered by the Access Recognition and Licensing Committee (ARLC), a sub-committee of the QAA Board of Directors.
2 This is a report of the consideration of the application for an AVA licence made to QAA by the Open College Network (North and East London and Hertfordshire)(OCN(NELH)). The licensing team appointed by the ARLC to consider this application consisted of Professor Steve Bristow, Principal and Chief Executive, Shrewsbury College of Arts and Technology, and Dr Pete Johnston, Head of Access and Associate Director of Overseas Relations, University of Essex. The licensing event was coordinated for QAA by Ms Kath Dentith, Assistant Director (Access), Institutional Review Directorate.
3 The procedure for the consideration of the OCN's application for an AVA licence included an initial meeting between OCN(NELH) officers and the QAA Assistant Director to discuss the requirements for the Licence Application and the process of the licensing visit; the preparation and submission by OCN(NELH) of its Licence Application together with a selection of supporting documentation; a meeting of the licensing team to discuss the Licence Application and other documentation and to establish a draft programme for the licensing visit; and negotiations between QAA and OCN(NELH) to finalise the programme and other arrangements for the visit. Following the licensing visit, the reviewers' report was presented to a meeting of the ARLC by the licensing team. The Committee made its decision about the award of an AVA licence on the basis of the Licence Application, further requested documentation, and the licensing team's report and presentation.
4 The licensing visit took place on 10 February 2000. The visit to OCN(NELH) consisted primarily of meetings with representatives of the OCN, including OCN officers; members of the Board of Trustees, Quality Committee and Access to HE Sub-Committee; Access moderators; and representatives of member institutions. The Agency is grateful to OCN(NELH) and to those who participated in the preparation of the licence application and in the licensing visit for the willing cooperation provided to the licensing team.
Background and previous experience
5 The Open College Network (North and East London and Hertfordshire) was formally created on 7 September 1999 by the combined membership of the Hertfordshire Access Consortium (HAC) and the North and East London Accreditation Federation (NELAF). Both HAC and NELAF were formed in 1989, and both have been licensed AVAs since 1990.
6 HAC was created as a partnership between Hertfordshire LEA, Hatfield Polytechnic (now the University of Hertfordshire), and the nine Further Education and two Specialist Colleges in Hertfordshire. It received approval as an AVA in January 1990 and was reviewed by the Higher Education Quality Council (HEQC) in April 1995, by which time it was responsible for 25 'kitemarked' Access programmes, many of which were large, multi-pathway programmes, offered at eight institutions but spread across numerous college sites.
7 At the time of that review, HAC was considering development as an Open College Network, largely in order to bring all validated courses into a nationally recognised credit framework. This position was reflected in the review panel's recommendation to HEQC that HAC should be required, as a condition of continued approval, to provide a further report to HEQC setting out any OCN developments, and their implications for local Access provision and for the AVA's structure and processes. In every other respect, the HEQC review report was extremely positive about the AVA's work, listing 17 main achievements as worthy of commendation.
8 HAC achieved Affiliate status of the National Open College Network (NOCN) in 1997 but then determined that, because of slower progress towards full OCN status than expected, such full OCN status 'could best be achieved through partnership with a similar organisation' (HAC Annual Report. 1998-99). During the 1998-99 session, discussions took place with NELAF (with whom HAC was already registering certain learners for credit award purposes) and it was agreed that a new organisation with AVA and OCN status should be created to serve the membership of both existing organisations. This was agreed formally at a general meeting of members of the two organisations, held on 7 September 1999. At this time, HAC had 11 full and two associate members, all but one of whom were offering Access to HE programmes, and had 25 such programmes in approval, 21 of which were running and 12 of which were general, modular schemes. Just under 1,000 learners were registered and 433 'kitemarked' Access certificates were awarded. HAC's budget for 1998-99 was £60,350 and it received support from the University of Hertfordshire in a number of practical and positive ways.
9 NELAF's original membership comprised two polytechnics (Middlesex and East London) and nine FE colleges. It achieved Associate member status of NOCN in 1995, and full OCN status in January 1997. Between 1995 and 1999, two other HEIs (London Guildhall University and Queen Mary and Westfield College) joined the AVA, and two other FE colleges joined for OCN, though not AVA, purposes, together with a significant number of other providers. By July 1999 it had a membership of 30 organisations, nine of whom offered Access to HE programmes; had recognised 83 Access to HE programmes; and had just over 12,000 learners, 1,427 of whom were registered on Access to HE programmes, with 750 'kitemarked' Access Certificates awarded. NELAF's turnover in 1998-99 was £255,348 and its financial surplus £94,627. Its running costs were slightly depressed by favourable rental and other charges by Waltham Forest College.
10 NELAF was last reviewed as an AVA in November 1992. The review report commended a number of aspects of NELAF's work, particularly in relation to its 'long-standing, principled commitment to the equal opportunities dimensions of Access Courses' and to the 'significant initiative' which NELAF had taken in producing a strategic development plan. The report also made seven recommendations, covering a range of matters and some specific concerns.
11 When the two AVAs established the new company in September 1999, the new organisation was thus in a position to build on the best practice of two very different, but experienced, AVAs: one much smaller, enjoying the strong support of a single HEI; the other larger, with a more diverse membership and with experience of operating as an OCN.
Legal status and constitution
12 The OCN was incorporated on 24 January 2000 as Open College Network (London Regional and Hertfordshire), but has since changed its name to Open College Network (North and East London and Hertfordshire) to satisfy NOCN licensing requirements. Although this change took place subsequent to the licensing visit, the OCN is referred to here by its new name and acronym for the sake of clarity and consistency.
13 The Company is limited by guarantee and has as its Objects 'the advancement of learning and education of the public in or around the London region and Hertfordshire and elsewhere by in particular, but not exclusively, one or more of the following ways:-
- promoting and increasing participation in education and training, particularly by those who have benefited least from existing provision;
- improving the quality and flexibility of education provision for the public benefit, particularly for educationally disadvantaged adults, through the National Open College Network (NOCN) accreditation system;
- facilitating progression for learners by enabling them to achieve Open College Network credits, as defined by NOCN, and to accumulate and, where appropriate, to transfer these towards a qualification;
- improving access to higher education by establishing, carrying on and conducting the work of the National Open College Network accreditation system and operating as an Authorised Validating Agency under licence from the Quality Assurance Agency for Higher Education;
- encouraging and assisting organisations within its operational area to provide a wider range of accredited learning opportunities'.
14 Membership of OCN(NELH) 'is open to any organisation which perceives a benefit from being a member and agrees to abide by the Constitution, Charging Policy and Service Level Agreement'. Membership is by application to the OCN's Executive Officer, who has the power to decide whether to confirm membership 'where an organisation meets the above criteria and presents neither the possibility for conflict of interest nor the potential to bring the OCN into disrepute'. OCN(NELH) 'reserves the right to refuse, suspend or withdraw membership where necessary'. The exercise of this right will be by the Board of Trustees of the OCN, recognising the need to ensure that 'all learners registered for accreditation at the time of suspension or withdrawal of membership are not penalised'. An organisation which has been suspended, refused membership, or had its membership withdrawn may appeal to the Chair of Trustees, who may either convene an Appeals Committee or recommend striking out the appeal to the Board of Trustees. Although such an Appeals Committee, other than its Chair, 'must not have been party to the original decision, against which the appeal has been made, and they must also be independent of the organisation making the appeal', this independence would not be similarly assured if the Chair of Trustees took the second possible course of action. The licensing team suggests that OCN(NELH) might reconsider this procedure to ensure that, in all cases, appeals conform to the requirements of natural justice.
15 The articles of association provide for the business of the company to be exercised by trustees, elected at each Annual General Meeting for a term of three years, with one third of trustees retiring at each Annual General Meeting. The article relating to the composition of the trustees is quite complicated in its attempt to secure representativeness, specifying a minimum of five and maximum of 12 trustees, 'with at least one representative drawn from the members of the Charity being a Higher Education Institution, one representative from a Further Education and Tertiary College, and with reasonable endeavour used to obtain in addition one representative from an Adult and Community Education Centre, one representative from a Voluntary Organisation and one representative from a Private Training Provider'.
16 In every other respect, the articles of association are quite standard for a company limited by guarantee and with (or seeking) charitable status, and the licensing team considered that they provided an appropriate foundation for the requirements of a body with AVA status.
Members, Access to HE providers and provision
17 In 1999-2000, OCN(NELH) had 49 members. Five of these were HEIs (University of East London, University of Hertfordshire, London Guildhall University, Middlesex University and Queen Mary and Westfield College). Twenty were FEFC-sector colleges, from Barking College in the east to Aylesbury in the west and from Hitchin (North Hertfordshire) in the north to Southwark in the south. The other 24 members were LEA adult education centres, training providers, health authorities, charitable trusts and voluntary associations (including one independent FE College). The Universities of Brighton and North London, while not being members, were involved in the work of the OCN's Quality Committee. The OCN is currently seeking to distinguish between member organisations and user organisations (the latter having fewer obligations under the Service Level Agreement), although 'since user organisations will be required to pay a 50% surcharge in all categories of charges it is not anticipated that many organisations will opt for this status and certainly it is not expected that any Access to HE providers would do so'.
18 Of its 49 members, 18 provide Access to HE programmes. All but one are FEFC-sector colleges. The exception is the University of Hertfordshire, which offers an Access to Art and Design programme. In broad terms, most of the colleges which were former HAC members offer generic Access to HE programmes, whereas former NELAF members offer specifically named programmes within a common modular structure. Altogether, the OCN has 323 recognised programmes, 103 of which are Access to HE programmes.
19 The licensing team was satisfied that the size of the OCN's membership provided reasonable assurance of future viability for the organisation, and that the involvement of members from different sectors provided an appropriate reassurance that the OCN would be able to exercise the responsibilities of a licensed AVA.
Governance and committee structures
The Board of Trustees
20 The Board of Trustees has complete responsibility for all aspects of OCN(NELH)'s work. At least for its first year of operation, the Board was established with 12 trustees, four of whom have come from organisations formerly in membership of HAC and four from NELAF. Its quorum was set at five members. At the time of the licensing visit, the minutes of four meetings of the Board of Trustees were available. These demonstrated that the Board was operating properly within its terms of reference.
21 The Board of Trustees has the power to appoint sub-committees, to include trustees and other persons, and to delegate any of their powers to such sub-committees, 'provided that all acts and proceedings of any such sub-committee shall be fully and promptly reported to the trustees'. The Board has established three sub-committees, one of which (the Quality Committee) has a further sub-committee structure. The Finance and General Purposes Committee, consisting of four members, is 'responsible for overseeing, and ensuring, the efficient operation of OCN(NELH)'. The Committee's first two meetings covered a range of corporate matters (constitution, finance, accommodation, staffing) and considered the relationship between the scope of powers and duties delegated to it and those resting with the Board of Trustees. It appeared to the licensing team to be fulfilling the remit delegated to it by the Board.
22 The Strategic Planning and Development Committee (also with four members) has been charged with 'the responsibility for the development of a realistic and coherent medium to long term strategic plan'. The first meeting of this Committee had considered and amended a draft strategic plan for 1999-2002 and a draft annual operating statement for 1999-2000 for subsequent endorsement by the Board of Trustees. It also discussed NOCN and QAA licensing and membership, and business development matters. The licensing team noted that in the inaugural year of operation of the new organisation, the Chair of the Access to HE Sub-Committee served as one of the four members of the Strategic Planning and Development Committee. The team recommends that this practice be formalised and continued in order to secure the interests of Access to HE at a senior level.
23 The Quality Committee has responsibility for 'developing, maintaining and reviewing policy and procedures on quality issues' and combines compliance objectives (eg 'To ensure that quality procedures meet NOCN and QAA requirements') and enhancement objectives. It has a larger number of members (between 10 and 14) than the other two committees, and supervises the work of three sub-committees: the Approvals Sub-Committee; the Moderation Sub-Committee and the Access to HE Sub-Committee. It is charged, in particular, with a specific duty 'to delegate to the Access to HE sub-committee primary responsibility for quality procedures relating to Access to HE programmes'.
The Access to HE Sub-Committee
24 The Access to HE Sub-Committee is required to report to and advise the Quality Committee on matters relating to quality procedures and requirements for Access to HE programmes, and in relation to action on national developments in Access to HE. The Sub-Committee noted this relationship at its meeting of 17 November 1999, recording that 'The Access sub-committee had primary responsibility for the quality assurance and quality enhancement of Access to HE provision, while it was recognised that major policy decisions fell within the remit of the Quality Committee'.
25 The operation of this relationship is demonstrated in the process involved in establishing a consistent credit requirement for the achievement of a 'kitemarked' Access certificate for the OCN. Essentially, the need for this matter to be addressed arose for the AVA because NELAF had required a minimum of 16 credits (at least 12 of which had to be at level 3), whereas HAC had required 20 (at least 15 of which had to be at level 3). After 'considerable discussion' in which 'the Access Committee did not feel able to reach a decision' the issue was referred to the Quality Committee, which itself had a 'lengthy discussion' before agreeing to recommend a target of 20 credits, 15 of which (minimum and maximum) should be at level 3. Although the minutes of this meeting record that the Quality Committee 'recommended' a target of 20 credits, the OCN has since informed QAA that 'The Quality Committee had in fact made the decision to adopt 20 credits as the standard requirement for the award of a kitemarked Access Certificate'. The Board of Trustees, in its subsequent discussion of this matter, occurring as a result of its receipt of the Quality Committee's minutes, was then able to advise that clarification should be sought in relation to any national requirements. The licensing team concluded that the OCN's committee structure provided for different decisions about Access to be made at appropriate levels and with appropriate authority.
26 To some extent, however, the remit of the Access to HE Sub-Committee places it in potential contention with the other two sub-committees of the Quality Committee. Its responsibility for 'scrutinising and confirming Access Recognition Panel decisions' is essentially the same as the Approvals Sub-Committee's responsibility for 'considering reports of Recognition Panels and ratifying panel decisions, taking action where appropriate'. The Moderation Sub-Committee's remit specifically precludes it from recommending moderator appointments for Access to HE programmes but charges it with the task of 'reviewing procedures for moderation', including Access moderation procedures, where the Access to HE Sub-Committee is enjoined to 'review and develop moderation procedures, in liaison with the Moderation sub-committee'. In reality, the licensing team was assured, there would be a sensible split of duties such that quality procedures relating to Access to HE would be dealt with by the Access to HE Sub-Committee in consultation with the Moderation Sub-Committee, while substantive matters relating to Access to HE, including national developments, staff development and promotional activities, would all fall clearly to the Access to HE Sub-Committee. It was intended that the necessary consultation would be achieved through some overlap of membership, linking the three sub-committees. However, the Moderation Sub-Committee itself apparently had some uncertainty about the precise details of the effective operation of this relationship, and had, according to the minutes of the Access to HE Sub-Committee of 25 January 2000, 'expressed some concern that responsibility for moderation was divided between the Moderation and Access Committees, although it was recognised that the overlap in membership would assist in promoting parity'. At this meeting, the Access to HE Sub-Committee had agreed to the Moderation Sub-Committee's proposal that 'the minutes of the Access sub-committee be circulated to them for information, and vice-versa'.
27 The licensing team considered that it was important for the security of the quality assurance and standards on Access to HE programmes that this relationship, and the committees' relative responsibilities, should be absolutely clear. The team therefore agreed that, as a condition of licence, the OCN should produce a statement defining the inter-relationships between the committees concerned with quality assurance of Access to HE programmes.
28 The membership and remit of the Access to HE Sub-Committee will provide the OCN with an adequate mechanism to enable it to discharge many of its responsibilities as an AVA. However, the licensing team noted that the remit made no explicit reference to standards of student achievement, in spite of the Sub-Committee's evident involvement in this area of work, as was demonstrated at the Sub-Committee's January 2000 meeting when the subject of 'assessment - promoting comparability of standards' was fully discussed and a number of worthwhile related proposals made. The team considered that the Sub-Committee's remit might, with benefit, be reconsidered to make it clear that standards as well as procedures were included among its responsibilities.
29 The creation of an Access to Higher Education Forum as a practitioner network with a strong best practice dissemination brief represents an important tranche of OCN Access development which the licensing team wishes to commend. Open to Access Co-ordinators/Course Leaders and with a formal reporting line through to the Access to HE Sub-Committee, the Access Forum has the potential to play an important part in OCN(NELH)'s quality enhancement programme.
Management and organisational structures
30 A major challenge for the OCN in bringing two pre-existing AVAs together lies not only in harmonising systems and procedures but also in ensuring that members and staff work together in a disciplined and harmonious way. It is quite clear to the licensing team from both documents and discussions with members and officers that the commitment to involve those from the two predecessor bodies in broadly equal measure, and to build on best practice from both HAC and NELAF, is succeeding. The new organisation is to be commended on the care with which it has approached the task of securing strong common ownership across the whole of its new membership.
31 The staffing structure for such a complex operation as OCN(NELH) is fairly slender, although the licensing team considered that it was sufficient to meet the AVA needs of an OCN. The Executive Officer reports to the Board of Trustees through the Chair of Trustees and is responsible for the overall operation of the OCN. The Development and Quality Manager, supported by Accreditation Advisers (1.5 FTE), reports to him on all accreditation and quality assurance matters, while the Operations Officer does so in respect of administration, financial and office support matters, including registration and awards. The Operations Officer is supported by a Quality Systems Administrator, two Administrative Assistants and an Accounts Clerk (0.4 FTE). The employer is OCN(NELH), and staff contracts are based on college support staff contracts, pensionable via the Local Government Superannuation Scheme. At the time of the licensing visit, the OCN had not been able to appoint to the Quality Systems Administrator or Accounts Clerk posts. Staff appeared to the licensing team to be highly regarded by members and committed to the success of the new organisation.
32 Accommodation was proving to be a rather more difficult issue. At the time of the licensing visit, staff were working from two bases - at Waltham Forest College and at the University of Hertfordshire, although negotiations were underway to take a lease on office property in Enfield. While representing a greater charge on the OCN's resources, this was considered to be a solution which would both offer an improved working environment for the larger team, and allow all members of staff to share a common office base. QAA has since been informed that these negotiations have been completed successfully, and the OCN has moved to appropriate independent office premises.
Aims, targets and planning
Mission, AVA aims and strategic planning
33 The organisation's mission statement 'to provide a high quality, responsive and inclusive accreditation service that encourages lifelong learning, recognises diversity and widens participation within a national framework of qualifications' encompasses an implicit Access to HE mission. The licensing team concurs with the view expressed in the Licence Application that all of the specific aims accompanying the above mission statement apply to the Access to HE agenda for action.
34 In furtherance of its mission, the organisation operates to a three-year strategy plan and an annual operating statement which sets specific targets against each general and specific objective outlined in the strategic plan. These targets and actions, as specified in the Operating Statement 1999-2000, include specific explicit references to Access to HE systems and developments, as well as to general OCN systems, improvements or developments which impact on Access to HE provision. Notably, these indicate the introduction of a 'comprehensive system of institutional/centre approval for accreditation', the development and establishment of 'more extensive and effective systems and procedures in relation to standardisation of assessment and moderation, and verification of assessment across programmes and institutions' and expansion of 'moderator training arrangements'. The licensing team found these targets to be entirely appropriate to the Access to HE agenda.
Planning mechanisms
35 The responsibility for the development and annual revision of the strategic plan and for the production of the annual operating statement, setting out the specific targets and actions against objectives within the strategic plan, is held by the Strategic Planning and Development Committee of the Board of Trustees. The Strategic Planning and Development Committee reports are standing items on the Board of Trustees' agendas. This arrangement provides an effective mechanism for policy development and review for the general OCN operation. With regard to representation of the particular interests of Access to HE provision, the licensing team considered that this would be adequately secured with the representation on the Committee of the Chair of the Access to HE Sub-Committee, as recommended at paragraph 22, above.
Target groups: policy and implementation
36 The organisation's statement on target groups satisfactorily addresses not just the identification of relevant target groups but the practical reality that an AVA policy can only be implemented by provider institutions and hence the need for the AVA to require provider institutions to evaluate the success of their targeting policies in their annual review/report.
Finances
Financial planning and management
37 Although the Board of Trustees has responsibility for all aspects of OCN(NELH)'s work, including financial planning and management, the Board delegates to the Finance and General Purposes Committee certain of these responsibilities. The Finance and General Purposes Committee has responsibility, in particular, for the formulation of proposals on finance to the Board of Trustees, and the approval of financial expenditure within agreed guidelines to ensure the efficient operation of the company. At its inaugural meeting in October 1999, the Committee, in discussing a further item in its remit relating to empowerment to take executive action, agreed that its 'main tasks in relation to this were running the budgetary plan and licence applications to NOCN and QAA'.
Funding model
38 The organisation has adopted a funding model which places the emphasis in income generation on volume of activity and where the dominant component of fee income is accrued through learner registration fees, with only minor contribution from membership and programme registration fees. In so doing, the new organisation has made a carefully considered decision to move from very different charging policies adopted by the predecessor organisations, in particular rejecting a model in which membership fees played the dominant role. For 1999-2000, of the total projected income of £351,630, the projected relevant component totals are:
- learner registration fees - £297,580;
- membership fees - £18,000;
- programme registration fees - £28,800.
39 OCN(NELH) has produced a detailed financial projection for 1999-2002, agreed by the Finance and General Purposes Committee, which sets out the projected growth in business and expenditure needs. These detailed figures, with accompanying notes on the underlying assumptions and analysis, provide evidence of a detailed planning process and identification of relevant factors to be considered, for instance recognition of the increasing costs of moderation.
40 The licensing team noted, however, that there was no budget line covering the establishment and development of a specific contingency fund and recommends that OCN(NELH) establishes and operates a contingency or reserves policy. With regard to the OCN's capacity to establish a contingency fund, the team noted that the predicted balance of income and expenditure for 1999-2000 to 2001-02 projected an opening surplus of approximately £40,000 (transfer from NELAF and HAC), and annual operating surpluses of approximately £11,500 for each of the years 1999-2000 and 2000-01, and an annual operating surplus of approximately £13,500 for 2001-02.
Viability
41 The Board of Trustees and the officers recognise the risks in establishing a company with long-term commitments through lease of premises (11 years) and the award of permanent contracts to staff. The Trustees believe business success will be achieved through the provision of a cost-effective and high-quality accreditation service operating to a carefully considered financial plan, with the Finance and General Purposes Committee charged with the responsibility for monitoring and responding to significant changes, actual or impending, in the adult education and training market.
42 The licensing team formed the view that the OCN's financial position was sufficiently robust, and its financial management sufficiently sound, to meet the responsibilities of a licensed AVA.
Arrangements for the development and validation of Access to HE programmes
Development and validation
43 A potential provider will receive a copy of OCN(NELH)'s general Programme Submission Handbook as well as its recently-approved document Access to Higher Education Programme Criteria and Quality Standards. This addresses all relevant matters, including the criteria for the award of the Access 'kitemark'. The OCN provides appropriate levels of support by experienced officers for organisations developing programme submissions. (The principle of operation 'so that member organisations receive unlimited developmental support with programme submissions', given in the Licence Application, is tempered by more realistic expressions in the draft service level agreement.)
44 Recognition panel memberships are appropriate, with cross-sector representation (in particular HE participants), appropriate curriculum expertise, and externality addressed. The organisation also has a substantial pool of experienced Chairs. Panels operate to standard OCN procedures.
Post-panel processes
45 The remit of the Access to HE Sub-Committee includes a responsibility 'to scrutinise and confirm Access Recognition Panel decisions'. The licensing team sought clarification of the means by which scrutiny was effected, noting the statement made in the Strategic Plan 'that the new OCN starts its life being able to build on the good practice developed in both NELAF and HAC in relation to quality assurance', and that the NELAF Access to HE Committee had, in October 1998, 'considered the nature of the information it required in order to ratify panel decisions'; had 'agreed that panel reports on their own provided little useful information for the committee'; and had 'agreed that in future the committee would receive a list of programmes for ratification; a sample of panel reports; and that submission documents would be available before and during the meeting'.
46 The licensing team concluded that, at the time of the review visit, there was insufficient clarity about the process through which recognition panel decisions for Access programmes would be approved. It therefore agreed that OCN(NELH) should provide an explicit definition of the process whereby the Access to HE Committee ensures that it can discharge its responsibility to scrutinise Access Recognition Panel decisions.
Assessment, moderation and certification of Access to HE programmes
Assessment
47 Appropriate policies and guidance for providers and requirements relating to assessment methods or judgements are set out in the OCN Programme Submission Handbook and in the Access to Higher Education Programmes Criteria and Quality Standards document.
Appointment, employment and training of moderators
48 The arrangements described in the Licence Application, comprising appointment by the OCN with specific contractual requirements; the requirement that a programme moderator has to possess relevant experience and be external to the programme; a normal three-year term of office; receipt of fee only after submission of the moderation report to specified requirements, are appropriate. Provision of training of moderators through an annual briefing session is adequate. This will be particularly important for those whose most recent experience has been with HAC programmes operating to a standardised grading system which had been developed for operation across that Consortium.
Moderation system
49 The Licence Application provides details of moderation arrangements at the time of the licensing visit, whereby HAC appointed two moderators to each of its programmes (one from an FE background and the other from an HE background), whereas NELAF appointed one moderator to each discrete programme, or to each pathway within a large modular programme. (The licensing team received confirmation that these arrangements had been continued for the academic session 1999-2000.) This section of the Licence Application also noted that 'It is intended that practice will be harmonised for 2000-2001, adopting the best practice from both systems. The structure of moderation for large modular programmes will be particularly reviewed, partly in the light of the OCN's plans to introduce an accredited institution/centre approval system during 1999-2000'. Further indicators as to the desired model of moderation included references to the aim to hold subject moderation meetings; the intention 'to establish Quality Circles' involving tutors and possibly moderators; and the intention to 'appoint subject lead verifiers'.
50 Additionally, the Access to HE Sub-Committee at its meeting of 25 January 2000, recognised that 'any new [Access moderation] system should take into account the requirements of QAA and NOCN regarding comparability and parity of standards between programmes' and 'agreed that the discussion on moderation should be integrated with the following agenda item on assessment'. The minutes of that agenda item record:
- recognition of the value of subject moderation sessions;
- the importance of the institutional moderator role, when introduced;
- a specific proposal to hold subject moderation sessions for social science initially, but with no indication of timescale;
- endorsement of a proposal to appoint lead verifiers;
- endorsement of a proposal 'to establish working groups to agree common GCSE Maths, English and Science Equivalences across the OCN' with the agreement 'that these groups should be convened shortly'.
51 The licensing team considered that, although the above proposals provided particular indicators of potentially relevant and desirable features of a moderation system, the Licence Application lacked a definite, coherent description of the overall system of moderation to be operated by the new organisation. Furthermore, in discussion with different groups to be involved with moderation, the team was not presented with a consistent view of the model of moderation to be adopted by the OCN for Access programmes. The team also observed that there was no indication of the timescale for the development, approval and implementation of the new system of moderation. In considering the establishment of a realistic timescale for the implementation of such a system, the team noted that the most recent NELAF Accreditation Handbook, (of September 1997), specified a system which indicated many of the features desired by the new organisation for its moderation system (for example 'programme moderators', 'subject verifiers' and 'institution moderators') but OCN(NELH) representatives confirmed that this comprehensive system had not been implemented by NELAF.
52 The licensing team also had some initial concerns over what appeared to be an emphasis on the verification of procedures, rather than a fuller interpretation of moderation. For example, the team noted that the Accreditation Handbook referred repeatedly to the 'verification' of assessment by moderators, and the Moderation Checklist within the Handbook strongly emphasised the checking of aspects of the process, rather than the outcome, of assessment. In addition, recalling the established subject moderation work of HAC and the new organisation's declared aim that the harmonised system should incorporate the best practice from the predecessor systems, the team was concerned about a lack of emphasis given to subject moderation mechanisms, as delineated in the original information provided about moderation.
53 Given the above limitations of information about the OCN's moderation arrangements, subsequent to the licensing visit, the licensing team asked OCN(NELH) to provide a definitive statement describing in full the proposed new moderation arrangements for Access programmes, together with a timed action plan for the introduction of the single harmonised system.
Further specification of moderation arrangements (following licensing visit)
54 In response, OCN(NELH) produced proposals whereby 'Access moderation will operate through a structure of Institutional and Programme/Pathway Moderators. In addition to these roles, further standardisation activities will occur on an incremental basis'. In this structure, 'Pathway moderators may have responsibility for named pathways on a modular Access programme, for subject clusters on modular programmes which do not have named pathways, or for discrete Access programmes with particular curriculum routes'. The proposals also stated that it was intended that the institutional moderator would monitor, and report on, the operation of the institution's provision in relation to Centre Approval standards, including, in particular, quality systems and internal verification systems.
55 OCN(NELH)'s Implementation Plan proposed the introduction of these two roles for operation in academic year 2000-01. However, the target given for the achievement of full Approved Status for all Access provider institutions was September 2001, and the role of institutional moderator was linked to the Centre Approval system. OCN(NELH) therefore proposed an interim arrangement whereby those Access to HE provider institutions which would not have Centre Approved Status by September 2000 would submit details of their internal verification procedures for approval by OCN(NELH)'s Quality Committee to enable operation of the Institutional Moderator system in 2000-01.
56 As indicated in the above, the major thrust of these arrangements is to ensure compliance with requirements, as specified through Centre Approval and Access programme validation processes, relating to institutional quality assurance systems, programme delivery and assessment, application of internal verification systems, and application of successful completion conditions.
57 The documentation received provided evidence of a welcome change of emphasis, whereby a focus on verification was replaced by a wider concept of moderation, to include both standardisation and verification. However, the licensing team considered that statements on the important matter of subject moderation remained imprecise. These were subordinated to a section entitled 'Additional procedures to promote parity of standards' and included a statement that subject consultants would be appointed on a 'rolling programme', but with no indication of the scale of the operation nor of a timed implementation plan. Furthermore, these subject consultants would be appointed only 'in an advisory capacity'. There was further reference to 'subject moderation sessions for tutors and moderators' and the possibility of the introduction of Quality Circles with 'blind' marking, but again, specific reference to these events/processes having only 'advisory status', and again no indication of an implementation plan. Given OCN(NELH)'s declared purpose to develop a harmonised moderation system adopting the best practice from the preceding HAC and NELAF systems, and the particular reference in this respect to the established subject moderation work of HAC, the licensing team was concerned to note that these revised proposals did not integrate subject moderation mechanisms as a central feature of the moderation structures.
58 The licensing team was not satisfied that the further information supplied provided sufficient reassurance that OCN(NELH) had yet developed a comprehensive system of moderation for its approved Access provision which would assure standards and which was ready for implementation. Given the central importance of moderation for the assurance of quality and standards on Access to HE programmes, the team therefore was unable to recommend to the Access Recognition and Licensing Committee (ARLC), at that stage, that OCN(NELH) should be awarded an AVA licence.
59 Nonetheless, in view of the fact that the OCN appeared to be working towards a satisfactory set of arrangements which could meet the requirements of the Access Recognition Scheme, the ARLC deferred a final decision on the award of AVA licence, and the OCN was required to produce a further statement on the moderation arrangements which addressed the need to integrate subject moderation mechanisms with the institutional and programme moderator system. The ARLC further requested that this statement be accompanied by statements on progress achieved towards the targets set in the Implementation Plan accompanying the definitive statement of moderation arrangements, in particular, documentation relating to the role of the Institutional Moderator, and confirmation that the financial implications of the comprehensive moderation system had been fully agreed.
Further specification of moderation arrangements (following deferral)
60 The documentation received in response to the ARLC's request provided:
i an updating statement on progress against the previously submitted Implementation Plan. This specified progress against the targets set for the complete implementation of the moderation scheme, and recorded systematic progress with some targets, some condonable slippage, and verification of some achievements through the provision of relevant documentation;
ii a document specifying appropriate criteria for selection, and procedures for appointment, of institutional moderators, as agreed by the Moderation Sub-Committee;
iii a document specifying additional measures to promote parity of standards on Access to HE programmes, 2000-01.
61 This final document provided the required detail of mechanisms to promote parity of standards, through a number of specific measures:
- Subject consultants
The role of subject consultants was specified, namely 'to inform the enhancement of the process of assessment and verification on an ongoing basis', through the mechanism of scrutiny of a 'cross-sample of evidence of learner achievement from each Access provider'. The authority of the consultants relative to that of Programme/Policy moderators was specified: 'their reports may well inform assessment decisions on the part of both Programme teams and Moderators but will not overturn decisions made'; and their relationship to Institutional Moderators was also clarified. This statement addressed the mechanisms for the integration of this subject consultant strand with the previously defined Programme/Pathway and Institutional Moderation strands.
In addition, detail was provided of the curriculum areas and target dates for 2000-01, for the sampling of evidence and for the report to the OCN for consideration by the Access to HE Committee, the Quality Committee and a meeting of Institutional Moderators. The report would 'inform the following year's practice as appropriate and provide a basis for staff development', and any specific concerns will be 'fed back...via the institutional Moderator'.
- Subject Moderation Circles
As a further set of detailed actions, the document explained that subject moderation sessions in specified subject areas would be convened 'to moderate learners' work "blind"' and to 'share and identify parity/good practice in the nature of assignments'.
- Other activities
The document also declared an intention to 'establish a series of Quality Circles for certain subject areas that will consider more general issues of common and good practice in Access to HE', and to develop AVA-wide GCSE equivalences in Maths, English Language and Science.
62 There was thus provided sufficient detailed evidence of carefully considered and constructed mechanisms to provide parity of standards to an appropriate timescale, addressing the issue of meshing with the other, previously specified, strands of moderation and incorporating best practice from the predecessor systems.
63 Hence on this topic of moderation, appropriate development and formalisation of policy and intended practice continued after the licensing visit took place. From the evidence available from the Licence Application, the licensing visit, and responses to requests for supplementary documentation, (and assuming full approval through the OCN Quality Assurance System of all relevant documentation), it was agreed that an appropriate system for the moderation of Access to HE programmes would be in place for 2000-01, with further development to take place in succeeding years. However, in view of the extended period and range of documentation through which these details became available, the licensing team concluded that the OCN should provide a consolidated account of the OCN's fully approved system for moderation of Access to HE programmes.
Moderator roles and reports
64 At the time of the licensing visit, the aforementioned NELAF Accreditation Handbook had been issued to OCN(NELH) moderators in lieu of a customised handbook for OCN(NELH) moderators in general and for the Access moderators in particular. The licensing team noted that this handbook, while describing a system which had not been fully implemented, did include an appropriate moderation checklist (for the use of programme moderators). The team also noted the introduction by OCN(NELH) of an appropriate interim report pro forma.
65 The licensing team was of the view that an equivalent handbook for moderators working with the new organisation should be developed as soon as possible, and considered that this should be available as a condition of licence.
Award and issue of 'kitemarked' Access certificates
66 The detailed system for the approval of award of 'kitemarked' certification, and the administrative process for the issue of certificates, as described in the Licence Application and as evidenced through the Registration and Award Handbook (of September 1999), is appropriate for the given purposes.
Arrangements for the monitoring and enhancement of Access provision
Monitoring through moderation
67 As specified in the Licence Application and as evidenced in the 1999-2000 Appendix to Moderator Agreement, the Institutional Moderator is required to ensure that the programme is being delivered as agreed by the recognition panel, paying particular attention to any conditions and/or recommendations of the panel. As noted above, OCN(NELH) has developed an interim pro forma to facilitate this. In particular, this requires the moderator to state whether or not issues identified in the panel report or the previous moderator report have been addressed.
68 As part of its remit, the Access to HE Sub-Committee is required to 'receive and scrutinise external moderator reports, taking action where appropriate'. The licensing team sought clarification of the means by which scrutiny was effected, recalling again the intention to build on existing good practice, and noting that the HAC Quality Committee had operated a policy of scrutinising all moderator reports, with detailed procedures not only for the receipt and consideration of reports, but also specification of procedures to be applied in the event of need for any subsequent action, and noting the statement made in the Licence Application that 'Moderator reports, along with responses from providers, will be read by OCN staff and a report made to the Access to HE Sub-Committee'.
69 With respect to this matter, the team noted that the Quality Committee had, in discussing the Licence Application at its meeting of 13 January 2000, 'agreed that the QC would need to discuss and monitor the development and implementation of new quality systems/procedures described in the application'. During the same discussion, the Committee had also specifically agreed that 'the review of moderator reports should involve the Moderation and Access Committees, with sample reports available for consideration and audit trails as well as officer reports. This should be actioned by the relevant sub-committees'.
70 The licensing team considered that there was insufficient clarity or certainty about the process whereby the Access to HE Sub-Committee ensured that it could discharge its responsibility to scrutinise moderator reports. The team agreed that OCN(NELH) should be required, as a condition of the award of licence, to provide an explicit definition of these processes, with further specification of processes for any required follow up action.
Evaluation and review mechanisms
71 OCN(NELH)'s Programme Submission Guidance requires specification of institutional programme evaluation and review mechanism, to appropriate criteria (including details of appeals and complaints procedures). Additionally, OCN(NELH)'s Access to Higher Education Programmes Criteria and Quality Standards document requires an annual self-assessment report to be provided to the AVA. This is received by the Access to HE Sub-Committee.
72 The Licence Application specifies that the Access to HE Sub-Committee is required to review procedures annually against QAA requirements and relevant NOCN Codes of Practice, and to report to the Quality Committee. The Quality Committee includes in its remit responsibility 'to consider issues of quality enhancement, including the receipt of relevant reports from sub-committees' and 'to receive reports and recommendations from all the Quality subcommittees and take action where appropriate'.
Complaints and appeals procedure
73 The OCN's Programme Submission Handbook requires a providing institution to provide details of its appeals and complaints procedures. As part of its remit, the Quality Committee undertakes to 'receive and take action on appeals against those aspects of quality assurance included in the OCN(NELH)'s appeals procedure'.
74 The NELAF Appeals and Complaints Procedure is specified in the NELAF Accreditation Handbook, which is currently being used until an OCN(NELH) handbook is prepared.
Data collection
75 OCN(NELH) will utilise the OCN database to record student registration, credit achievements and award of 'kitemarked' certificates. (This builds on the existing NELAF practice as evidenced in its 1998-99 Annual Report.) The database includes data on registered learner profiles by ethnicity, age and gender, enabling measurement of performance against targeting aims.
76 OCN(NELH) also issues a monitoring form to programme coordinators to gather information on students' progression intentions, and hopes to liaise further with the HE institutions who are the major receiving institutions of Access students in its validated programmes to gather data on such students' performance in HE.
Conclusions
77 OCN(NELH) is able to draw on the considerable experience of its two predecessor organisations (NELAF and HAC) to develop its operations and procedures as an AVA. It is adequately resourced, and its substantial and wide-ranging membership lends the OCN appropriate support to enable the organisation to pursue its aims.
78 The inevitable challenges posed in bringing together two separate organisations with different traditions, different kinds of provision, and different sets of procedures, have been largely successfully met by the new organisation. That the organisation should have been able to achieve this is indicative of the commitment of all members to the success of the new organisation, and also of its effective management at this early stage of its life.
79 The OCN's committee and organisational structures provide the means through which effective planning and decision-making can take place. Within these structures there is a clear place for the consideration of matters relating specifically to Access to HE. With some attention given to ensuring absolute clarity about responsibility for the different parts of particular quality assurance processes, and separating out the precise functions of different committees with related responsibilities, such processes should be adequately overseen.
80 The OCN presents a coherent and thorough approach to the quality assurance of Access provision, demonstrating an understanding of the responsibilities of AVA status. Most stages of quality assurance have adequate procedures in place, but it is with regard to moderation, in which area there were probably the biggest differences of practice between the two previous AVAs, that most work remains to be done. Nonetheless, since the licensing team's visit to the OCN, adequate systems, founded on sound principles, and taking proper account of the consideration of standards of student achievement, have been developed and are now fully described. The effectiveness of these procedures, and of those systems designed to enable the OCN to monitor the procedures' operation will be tested in the first years of the AVA's operation.
Recommendation to the ARLC
81 The licensing team recommends that OCN(NELH) should be awarded an AVA licence, subject to the standard requirements of the QAA Recognition Scheme for Access to Higher Education in England, Wales and Northern Ireland, and with conditions to be met as specified below.
Conditions
82 Subject to the approval of the ARLC, OCN(NELH) is awarded a licence to operate as an authorised validating agency until the time of its next review, on condition that the OCN submits the following documentation:
i the revised Moderation Handbook;
ii a consolidated account of the OCN's fully approved system for moderation of Access to HE programmes;
iii a consolidated statement of inter-relationships between the committees concerned with quality assurance of Access to HE programmes;
iv a consolidated explicit definition of the processes whereby the Access to HE Committee ensures that it can discharge its responsibilities to scrutinise Access Recognition Panel decisions and Moderation Reports.
Recommendations
83 The licensing team recommends that OCN(NELH):
i reconsiders the procedure for appeals against refusal or suspension of membership to ensure that, in all cases, appeals conform to the requirements of natural justice;
ii formally confirms the place of the Chair of the Access to HE Committee on the Strategic Planning and Development Committee in order to secure the long term interests of Access to HE at a senior level within the OCN;
iii reconsiders the Access to HE Sub-Committee's remit to make it clear that standards as well as procedures are included among its responsibilities;
iv establishes and operates a contingency or reserves policy.
Award of licence
Conditional approval of the award of licence was agreed by the ARLC on 21 July 2000.
All conditions, as detailed in paragraph 82, above, were subsequently met and OCN(NELH) was duly awarded a full licence to operate as an AVA, under the terms of the QAA Recognition Scheme for Access to Higher Education in England, Wales and Northern Ireland, on 3 October 2000.
Top