Open College Network - North West Midlands
January 2000

Foreword

1 The Quality Assurance Agency for Higher Education (QAA) is responsible to the Department for Education and Employment (DfEE) for the recognition of Access to Higher Education courses. QAA exercises this responsibility through a national network of authorised validating agencies (AVAs), which are responsible for the processes of individual course recognition and the award of 'kitemarked' Access to HE certificates to students on QAA's behalf. QAA has developed a scheme for the licensing and review of the AVAs, the principles and processes of which are described in the QAA Recognition Scheme for Access to Higher Education in England, Wales and Northern Ireland. The Recognition Scheme is regulated and administered by the Access Recognition and Licensing Committee (ARLC), a sub-committee of the QAA Board of Directors.

2 This is a report of a major review of the AVA function of the Open College Network North West Midlands (OCNNWM) undertaken by QAA. The Agency is grateful to OCNNWM and to those who participated in the review for the willing co-operation provided to the review team.

Aims and objectives

3 The aims of the system of AVA review are:

i to provide the basis for an informed judgement by the ARLC about the fitness of the AVA to continue as a licensed agency;

ii to promote public confidence in Access to HE as a properly regulated and respected route into higher education by assuring:

the quality and adequacy of AVAs' systems and procedures;
the quality, comparability and range of AVAs' operations;
the adequacy and comparability of AVAs' standards for approval, moderation and monitoring of programmes;
consistency across AVAs in the operation of criteria for the granting of the Access to HE award;

iii to stimulate reflective and self-critical perspectives within AVAs, as an instrument to promote quality enhancement;

iv to provide an opportunity to identify and disseminate good practice of AVA operations.

4 The objectives of each AVA review are:

i to examine, assess and report on:

the development of, and changes in, the AVA since its last review or initial licence, and its plans and targets for the future;
the organisation's continuing viability and robustness and the ways in which the AVA demonstrates sound governance;
the efficiency and effectiveness of the AVA's operational and quality assurance systems;
the range and scope of the AVA's activities, and the appropriateness and value of these activities;
the ways in which the AVA approves and monitors programmes and the ways in which these processes take account of the need for consistency and comparability;
the ways in which the AVA satisfies itself of the adequacy and comparability of standards achieved by students gaining the 'kitemarked' Access award;
the evidence available to indicate the AVA's success in achieving its aims and targets;

ii to identify and report on:

strengths and good practice in procedures and operations;
areas which would benefit from further development;
areas requiring attention.

Outcomes

5 Following the review of an AVA, the reviewers' report is presented to the ARLC by a member of the review team. The Committee then makes one of six decisions:

unconditional renewal of licence for a specified period;
conditional renewal of licence with conditions to be met by specified date;
provisional renewal of licence with conditions to be met and further review visit by specified date;
suspension of licence until specified conditions are met;
withdrawal of licence for operation as an AVA;
temporary renewal of licence with request for further information by specified date (decision suspended).

The review process

6 The review was conducted in accordance with the process detailed in the QAA Recognition Scheme for Access to Higher Education in England, Wales and Northern Ireland and approved by the ARLC. The preparation for the review of OCNNWM included an initial visit to OCNNWM offices by the QAA Assistant Director with responsibility for Access to HE matters to discuss the requirements for the Analytical Account (the Account) and the process of the review; the preparation and submission by OCNNWM of its Account, together with a selection of supporting documentation; a meeting of the review team to discuss the Account and supporting documentation and to establish a draft programme for the review visit; and negotiations between QAA and OCNNWM to finalise the programme and other arrangements for the review visit.

7 The review visit took place on 17 and 18 November 1999. The visit to OCNNWM consisted principally of meetings with representatives of the OCN including: OCN officers; members of the Management Board, Quality Committee and Access to HE Committee; moderators for Access programmes; Access programme co-ordinators; HE admissions staff; and former Access students now studying in HE.

8 The review team consisted of Dr Pete Johnston, Head of Access and Associate Director of Overseas Relations, University of Essex, and Mr Alan Smith, Quality Assurance Officer, North Wales Access and Credit Consortium . The review was co-ordinated for QAA by Ms Kath Dentith, Assistant Director (Access), Institutional Review Directorate.

The AVA context

General background

9 The Open College Network North West Midlands (OCNNWM) originates from a partnership established in the early 1980s between the former Staffordshire Polytechnic and four local FE Colleges. An initial body, the Polytechnic and Colleges Access Network (Staffordshire and Shropshire), (PCAN), was approved as an AVA in April 1991. Shortly thereafter, PCAN's membership expanded to include the University of Keele and a further four local FE colleges. In June 1992, the partnership was renamed the Universities and Colleges Access Network (UCAN).

10 The year 1995 marked two significant decisions regarding the development of UCAN. First, in July 1995 the UCAN Management Board decided to pursue OCN status, and second, UCAN formalised a merger from 1 September 1995 with Cheshire Access Federation (CAF) and accordingly adopted the signifier UCAN (Staffordshire, Cheshire and North Shropshire). In essence, this process involved the dissolution of CAF, a small AVA which operated a cashless economy, with individual members of CAF becoming members of UCAN. From 1 January 1998, the AVA adopted the new title of OCNNWM, and gained full membership of the National Open College Network (NOCN) in March 1998.

11 OCNNWM is based in Stoke-on-Trent and operates within the counties of Staffordshire, Cheshire and parts of Shropshire.

Staffing, finance and physical environment

12 OCNNWM maintains a staff team of seven, including four full-time and three part-time (0.5 FTE) members. All staff are employed by Staffordshire University, and the University provides payroll and personnel services. It has achieved this level of staffing through a policy of gradual increase in line with availability of finance and demand for services.

13 OCNNWM appears to operate comfortably within its annual budget. Its budget and operating statement are approved and monitored by its Management Board. The OCN's finances are managed through the University's financial system but in a separate account accessible only by OCNNWM authorised personnel. The OCN's accounts are audited through the University's standard procedures while remaining separate from the University's own accounts. The costs of both internal and external audit of OCNNWM accounts are borne by the University.

14 The University also provides office accommodation, currently free of charge.

Relationship with Staffordshire University

15 Throughout its existence, first as UCAN and then as OCNNWM, the AVA has benefited from strong support from Staffordshire University. The Account not only acknowledges that the 'AVA/OCN benefits from access to University resources (charged to the AVA/OCN Cost Centre) and conditions of service, while retaining a distinct identity' but specifies that 'this has been particularly important to give stability as the AVA/OCN matures'.

16 In 1995, a Memorandum of Understanding (MoU) was established with the University, specifying the services provided to OCNNWM (then UCAN) as an unincorporated association within the University. That MoU also specified the lines of accountability of the AVA chief officer (then Co-ordinator, now Director) through the following clauses: 'The UCAN Co-ordinator would report to the Academic Registrar for issues relating to his/her employment by the University, and, in consultation with the Management Board Chair, for the purposes of general supervision where appropriate. The UCAN Co-ordinator would, however, be operationally responsible to the UCAN Management Board and would work closely with its Chair.'

17 The University demonstrated increased commitment to OCNNWM in 1998, through the agreement of permanent contracts for the posts of Director, Senior Development Officer and Administrative and Systems Manager (previously Senior Administrator). Other staff remain on fixed term contracts in accordance with the still current 1995 MoU, which specified that the (then UCAN) staff could only be offered fixed term contracts and the proposal that initially a two year contract be offered.

18 Both the University and OCNNWM recognise the need to update and renew the MoU, and the review team recommends that this matter is progressed to ensure that the MoU fully reflects current circumstances, including the nature of contracts, and that it re-iterates the essential independence of the OCN.

Membership, provision, registration and achievement

19 OCNNWM's 25 member organisations are drawn from a range of sectors including further education; higher education; borough, city and county councils; and community, voluntary and private organisations. Additionally, OCNNWM works in partnership with a further 14 organisations including, among others, local Training and Enterprise Councils; Staffordshire Careers Service and Health Service; prisons; and the Midland Region of the Trades Union Congress.

20 Currently, 13 OCNNWM-recognised Access programmes are delivered, all by the further education sector. The great majority of these are offered both in full-time (one year) and part-time (two year) modes. A high proportion also feature an evening mode of delivery, or are delivered on an outreach basis. Most of these programmes are generic with designated pathways into degree programmes. It is the view of the AVA that, notwithstanding the successful recent establishment of two specialist routes (Access to Computing and Professional Access to Health and Welfare), the generic programme has proved the more resilient model. Over the period 1996-99, although two programmes were discontinued, both of them in Art and Design, five new programmes were introduced.

21 OCNNWM has experienced rapid growth in the overall number of learner registrations in recent years, with 20,000 in 1998-99 compared to 14,000 in 1997-98. It has also seen significant growth in the number of learner registrations on Access to Higher Education programmes over the past three years. Access to Higher Education learner registrations for 1998-99 totalled 943 whereas the corresponding figures for 1997-98 and 1996-97 were 678 and 501 respectively. The AVA has therefore sustained the recovery noted in the AVA's 1997-98 Annual Report which observed that the 1997-98 figures reversed what had been a slow decline in Access registrations.

22 That report, in commenting on the 1997-98 figures, states that 'the apparently low achievement rate (48%), expressed as a percentage of registrations, will need careful monitoring'. The actual number of Access to Higher Education certificates awarded in 1997-98 was 318, which as a percentage of numbers registered gives 47 per cent. The most recent figures for 1998-99, which became available at the time of the review visit, emphasised the need for this careful monitoring, since of the 943 registered, 367 achieved a 'kitemarked' award ie 39 per cent. The 1998-99 Resumé of Access Moderation Reports notes 'a strong trend towards part-time study, often over more than one year' and this might well be a major contributing factor in what appears to be a declining completion rate.

23 The 1998-99 figures do, however, record a substantial proportion (31 per cent) as withdrawn with no accreditation (289 of 943). These figures emphasise the need for fuller investigation of the reasons for non-completion, and OCNNWM itself recognises that retention is a matter which requires further attention. The review team would recommend that the OCN not only monitors but also analyses the data, and considers how it might advise or otherwise support providers in helping them to address any of the factors which contribute to the underlying causes of non-completion.

Mission, strategy, targeting policies and strategies

24 OCNNWM's combined AVA/OCN mission is succinctly expressed as 'working in partnership to offer enhanced learning opportunities to all adults'.

25 OCNNWM operates firstly to a three-year strategic plan and secondly to annual operational plans both for the OCN as a whole and also specific to the AVA role, with progress recorded against set targets. The existence of this separate Access to HE operational plan, (instituted in 1997-98) with measurement of achievement against targets, provides, in the view of the review team, an effective mechanism by which the OCN can discharge its AVA responsibilities within the overall process of determining appropriate strategic planning and target setting for the OCN as a whole. The team was also supplied with Key Action Points for the AVA, indicating areas for action to be taken forward for the 2000-2003 strategic plan. These key action points have arisen from an analysis of the OCN's AVA operation conducted as part of the preparation for the present review.

26 The Account draws attention to the extensive range of under-represented groups able to access programmes throughout the OCNNWM region, varying from those in isolated rural communities, to shift-workers, unemployed and de-skilled workers. The Account describes the OCN's targeting policy as 'essentially client led'. It might be argued that a 'client-led' targeting policy is a contradiction in terms, and it is clear that, to date, targeting has been the responsibility of providers, with different providers targeting a variety of different disadvantaged groups in varying geographical settings.

27 Although the OCN does not include any reference to targeting in the Access to HE Operational Plan for 1999-2000, it is included among its Key Action Points for the AVA as: 'identify groups still under-represented in Higher Education locally, and explore strategies for reaching these groups more effectively, through Access programmes'. The review team recommends the development by the OCN of this action point, and its incorporation into the forthcoming Access to HE Operational Plan.

Developments since the last review

28 As UCAN, the AVA was reviewed by the former Higher Education Quality Council (HEQC) in 1993. The review panel recommended that the AVA's licence should continue subject to normal terms and conditions. It also identified a set of recommendations for the AVA's attention. Although these are not explicitly addressed in the Account prepared for the present review, they have in fact either been addressed by specific AVA action and/or been overtaken by national developments in Access to HE provision, and in particular by the AVA's move to OCN status and the delivery of credit-based unitised provision.

29 The process of conversion of all Access programmes to a credit-based unitised format has been a major focus of activity in recent years with the process to be completed this academic year. Although Access is a statistically small proportion of the work of OCNNWM, the unitisation of the Access provision has been seen as a priority for allocation of staff time. In spite of the fact that the AVA annual report for 1997-98 records that 'many programme tutors...see the move from "traditional" Access to credit-based, unitised programmes as more of a chore and potential source of anxiety than a positive benefit, at least in the developmental stage', the review team noted a positive attitude to unitisation in its meeting with Access programme leaders.

30 The 1998-99 Resumé of Access Moderation Reports advances the view that 'the availability of study over an extended period, and the gradual accumulation of credit, signals a shift away from the "old system" of aiming for the kitemark in one year and should aid retention and achievement'. This view was also supported by Access to Higher Education programme leaders in the course of the review, although it is not yet borne out in the statistics.

Governance and Committee Structures

Legal status and governance structures

31 OCNNWM is an unincorporated association governed by a constitution. The central authority in its governance structure is the Management Board which holds overall responsibility for OCNNWM policy, planning and financial management. The Management Board is also responsible for ensuring the operation and development of all quality procedures and standards relating to its accreditation service. To assist it in discharging these responsibilities, the Management Board maintains a Quality Committee, which reports to the Management Board, and an Access to HE Committee, which reports to the Quality Committee. It further convenes, when necessary, time-limited task groups with a specific remit and time-scale within which to make recommendations to, or decisions on behalf of, the Management Board, reporting to the Management Board or Quality Committee as appropriate. Finally, the Constitution provides for the establishment of network fora. The Basic Skills and Mathematics (GCSE Equivalence) groups are currently active.

Management Board

32 Membership of the Management Board is specified through a formula designed to ensure cross-sector representation. In particular, this entitles each member FE or sixth form college and each member HEI to a representative on the Management Board, and with specification in the case of the FE and sixth form college representatives that these should be senior representatives. This formula produced a Board membership list, current at the time of the review, of 21 individuals, excluding the OCNNWM Director, who is also a Board member.

33 The current membership total of 22 is less than the maximum possible; for instance, as recorded in the minutes of the November 1998 Board, there was no representative from the voluntary sector. The review team noted that the formula itself had been a topic of recent discussion. At the July 1998 Board meeting 'discussion took place regarding entitlement of new smaller members to representation on management board' with the subsequent decision 'to form [a] small working group in early September, to review [the] constitution with particular regard to membership of [the] governing body'. However, the minutes of November 1998 record the fact that this working group had not been convened after 'Various members of [the] management board had been contacted but felt it unnecessary to review the constitution'.

34 The Chair and Deputy Chair are elected for two years by the Board from within its membership, with the Deputy Chair usually becoming the next Chair. Decision-making is by a simple majority vote with the Chair having a casting vote.

35 The Management Board is required to meet at least twice a year but has adopted the practice, last year and this, of meeting three times a year. If representatives are unable to attend, they may send a substitute. Despite this, attendance at meetings has been a matter of concern. The February 1999 meeting, for example, which dealt with important business, in particular the AVA Annual Report, was attended by only five Management Board members, one of whom was the OCNNWM Director. The fact that decisions could technically be made at that meeting was itself a consequence of the fact that the official quorum is, in fact, five. Members of the Management Board acknowledged that the low quorum was 'historic' relating to a much smaller number of members of the AVA. The review team considered that very low level of attendance could undermine the sense of authority with which important OCNNWM decisions are seen to be taken, and reduce the representativeness that is built in to the membership of the Board.

36 In dealing with the need for executive action between Management Board meetings, and in the absence of an executive committee which might make decisions about day to day matters on its behalf, OCNNWM adopts the pragmatic approach that decisions are made by the Chair of the Management Board after consultation with, and/or briefing by, the Director. And in the case of policy development, it utilises Task Groups, for instance a Funding Group in 1998 and the newly established Strategy Working Group. It is intended that this should deal with budget, staffing and strategic planning, in particular the Strategic Plan for 2000 onwards, with incorporation of key action points relating to the AVA role.

Quality Committee

37 The Quality Committee acts with delegated authority from the Management Board and is required to meet at least three times a year. It is responsible to the Management Board for the general oversight of the Network's quality assurance processes in respect of its accrediting role, covering recognition, moderation, review and evaluation. The Constitution requires that the Chair is both (i) elected from the Management Board; and (ii) elected for two years by the Committee from within its membership.

38 In addition to the Chair, one further member is elected by the Management Board. The Constitution further requires two representatives from HE; two members from the Access to HE Committee; four further representatives from any sector nominated by the Management Board; one representative from the voluntary sector; OCN officer representation; and allows for five co-opted members to ensure cross-sector representation and from specific areas of expertise. Current membership totals 15, excluding OCN officer representation, and one vacancy.

39 The Quality Committee, as the Management Board, has a quorum of five and, like the Board, it has experienced problems with low attendance at meetings. In particular, the January 1999 meeting, which received the AVA Annual Report, was attended by only six members other than OCN officers.

40 As regards the business and working methods of this committee, minutes display evidence of enhancement in formal operating methods, notably with reference to a resumé of moderation reports and the introduction for its 1998-99 review of an additional resumé, specifically of Access to HE moderation reports. While these are valuable additions to the Committee's overall mechanisms for its efficient consideration of quality matters, it is clear that such resumés have been necessary because of the considerable increase in volume of the Committee's work.

41 The Quality Committee, through its Chair, reviews recognition panel and moderation reports on a sampling basis. The Quality Committee's commitment (October 1998) that the 'committee would sample [an] average of six moderator reports per year' established an important principle, but in this sampling there is no formal requirement to include Access to HE moderation reports in the sample. The review team was informed that common practice is to include one Access moderation report per sampling occasion. The review team would expect the newly re-constituted Access to HE Committee to undertake a more comprehensive sampling of Access programme moderation reports, and for this to be specified in the Committee's formal responsibilities.

42 The OCN acknowledges that with the substantial growth in its business, the Quality Committee is now overburdened. This observation has been a significant factor in the development of the proposed new role for the Access to HE Committee.

Access to HE Committee

43 The Access to HE Committee, which meets three times a year, takes some of its functions from the original AVA Validation and Monitoring Sub-Committee and some from the corresponding original Tutors' Forum, in that it acts in an advisory capacity for the quality assurance matters associated with the licence of the network as an AVA, but also provides a focus for networking and the sharing of good practice, and promotes staff and curriculum development opportunities.

44 The membership comprises Access programme leaders from each member organisation; one representative from each member HEI; one member of the Management Committee; one member of the Quality Committee; and OCN officer representation, with powers of co-option of three additional members in order to ensure a balance of expertise in curriculum and progression routes. The Chair is elected for two years from within the Committee membership. The membership list of September 1999 totalled 16, excluding OCN officers. As with the Management Board and Quality Committee, the quorum is again five members.

45 The existence of this committee is seen by the OCN as ensuring the discrete role and status of the AVA within the larger organisation and it has the potential to fulfil this role. The Committee is described as having an 'active and committed membership'. While it may be the case that certain individual members are active and committed, the number attending, with due allowance for replacements, was consistently less than the number of absentees during 1998-99.

46 It was clear to the review team that the Committee played an important advisory role in the development of the OCN's Access to HE policy and practices, and that its membership provided a pool of relevant experience and expertise on Access to HE matters, in particular quality assurance matters. Given the previous comments regarding the overload on the Quality Committee, the team therefore concurs with the view expressed in the Account that the 'current remit and role of the Access to HE Committee limits its effectiveness'. The team further concurs with the view expressed in the Account, and included as a key action point for the AVA, that the committee should have 'a specific remit for quality rather than simply an advisory function'.

47 Given the impact of such a change of remit on the overall governance structure, the review team considers that this will provide an appropriate opportunity for OCNNWM to review its committee structures more widely and address matters identified above, in particular quorums and their impact on attendance to ensure members' commitment to, and involvement in, the discharge of the OCN's responsibilities as an AVA.

Management, organisational structures and administrative arrangements

48 The staff team consists of a Director; one full-time and one part-time Development Officer; one Administrative and Systems Manager; and three administrative and clerical staff, two of whom are part-time (with one of these casually contracted). This small team has experienced some pressure because of the recent growth in business, and staff absences and shortages led to difficulties in meeting its own service standards for the issue of certificates for the 1998-99 academic year.

49 A recently established part-time post of Quality and Moderation Officer, to be filled this academic year, is clearly a much-needed addition to the staff team. The review team also noted the suggestion in the Key Action Points for the AVA that the OCN should consider making a further appointment 'with a specific remit for AVA/HEI liaison'.

50 Roles and remits, both with reference to the OCN in general and to the AVA operation in particular, are clearly specified and well understood by the cohesive staff team.

Quality assurance functions

51 The Director undertakes key responsibilities for quality assurance. She oversees, and has direct in involvement in, operational matters; chairs Access to HE recognition panels; prepares the annual review of Access to HE moderation reports; prepares the AVA Annual Report; and plays a lead role in the development of the Access to HE Operational Plan. She also acts as the principal link between the day-to-day operation of the AVA and its committee structure, in particular maintaining frequent contact with the Chair of the Management Board and also alerting the Chair of the Quality Committee to any need for action.

52 The Senior Development Officer plays a significant role in relation to the development and recognition of Access to HE units, and facilitates the external moderation processes. The review team was informed that the Quality and Moderation Officer will, when appointed, play a key role in quality assurance and, in particular, undertake implementation and further development of moderation processes.

53 The Quality Committee, acknowledged to be overburdened, places considerable responsibility on officers to ensure that they bring to its attention matters arising from validation, moderation and annual review processes.

Communications with providers

54 OCNNWM communicates with providers through a variety of means. In terms of written communication, the annual Access to HE Operational Plan and the AVA Annual Report play significant roles in reporting achievement and key changes and trends, and in the setting of targets. At the operational level, OCNNWM provides various information packages: for Access to HE tutors; for potential students; for potential new providers, or other interested bodies.

55 The Access to HE Committee, through its three meetings per year, brings together Access programme co-ordinators and other practitioners, to network and share good practice; promote staff and curriculum development; and provide advice on quality assurance matters. Clearly, the valuable functions currently performed by the Access to HE Committee will need to be considered if it is intended that the Access to HE Committee itself should have a change of function.

56 There is also close liaison between OCNNWM staff, Access providers, and HE admissions staff. Meetings are held between AVA staff and admissions staff, for whom the OCN supplies a set of briefing notes. The review team was satisfied that the OCN had effective communication, about a range of matters, with providers of Access courses and receivers of Access students.

Development, validation and evaluation of Access programmes

Development

57 Since the amalgamation of UCAN and CAF, the main thrust in the curriculum development of Access programmes has been towards the credit-based unitisation of existing programmes. There has been some slippage in the original schedule to have all the changes in place by September 1999, but it is intended that all programmes will be unitised and credit based by the beginning of the academic year 2000-01. The work has been led by the Director and the Senior Development Officer, who have worked closely with programme teams to co-ordinate and implement the radical changes that have been necessary in documentation, strategies and philosophy. Standard procedures, documents and thorough guidelines exist for the conversion processes and, throughout the developments, the OCN has provided an underpinning programme of staff development for practitioners. Access programme co-ordinators whom the review team met spoke highly of the level of support that they received from the OCN staff; the help they received in writing units and in identifying appropriate units that were already in Access programmes; and of the help provided to prepare staff for the changes.

58 The development of a common core of units for GCSE equivalence in Mathematics and English has been achieved by working parties of FE and HE tutors, and similar work is in progress towards the recognition of a core of Science units. The review team recognised the value of working with neighbouring OCNs in this particular area, and its contribution to consistency in provision and greater certainty of standards through moderation processes.

59 The review team was informed that it is the OCN's intention to turn its attention to the moderation process, and the new appointment of Quality and Moderation Officer is intended to help the OCN to carry forward its intentions in this area.

Validation

60 Recognition panels are attended by both FE and HE practitioners; are chaired by an OCN officer; and are minuted by a member of the OCN's administration staff. Detailed notes of meetings are sent to the submitting provider to assist with the amendment of programme documents for re-submission to the OCN by agreed dates. Moderators proposed for programmes are invited to be members of the relevant panels, thus providing them with first-hand working knowledge of the programme and agreed amendments. Thorough records of the documentation of these processes, together with the developmental briefing notes prior to submission, are retained by the AVA. There did not, however, appear to be a definitive document to record that all the requirements of the recognition panel had been met and that the programme was approved for a specified period of time. The review team recommends the addition of such documentation to the recognition process.

61 A member of the Quality Committee, in her role as 'quality auditor', has responsibility for acting as an observer at recognition panels. This system is designed to assure standardisation of procedures, although it has not been in place long enough to impact on Access recognition procedures.

62 Formal procedures for transfer, and criteria for acceptance of programmes from other AVAs, have not yet been developed. The review team recommends that, in the light of its recent experience, the AVA should develop such procedures and criteria.

Monitoring, evaluation and review

63 Course team meetings with student representation, internal and external moderation, all inform the end of year review of programmes which is supplied to the OCN by providers. The OCN incorporates this information into its Annual Report to QAA along with its statistics and matters raised by the Quality and Access to HE Committees.

Assessment, moderation and award of certificates

Assessment

64 The appropriateness of programmes' assessment methods is considered as a part of the scrutiny of the programmes at recognition panels. Moderation reports seen by the review team, and feedback from HE admissions staff, indicated that, in general, the range and rigour of assessments met the criteria necessary for preparation for study at HE level.

65 OCN staff have delivered staff development events for Access staff dealing with the subject of assessment.

Moderation

66 Moderators are appointed by, and contracted to, the OCN according to a checklist of specified criteria, and appointments are confirmed by the Quality Committee. All moderators are external to the providers for which they moderate programmes, and the OCN has now adopted a policy of appointing moderators who work in HE, where possible. The OCN also has a policy of encouraging moderators to take responsibility for a 'portfolio' of programmes to contribute to consistency of standards.

67 Although the OCN is responsible for triggering the payment of moderators only after the receipt of moderators' reports, it is the providers themselves who are responsible for making payments to moderators. The review team considered that it was important to ensure that responsibility for the making of payments to moderators did not lie with a body which had an interest in the outcome of the moderation process. The team therefore took the view that OCNNWM should revise the current system, to ensure that, in its role as the awarding body, it had full control of the payment of moderators.

68 The length of time for which moderators serve on any one programme does not appear to be kept actively under review. Although the moderator's contract specifies that 'All appointments are assumed to be for the duration of the recognition of the programme which is usually five years', some current moderators have evidently been moderating the same programme for longer. The review team recommends that the OCN reviews the process by which it identifies the length of time serving moderators have been with any one programme, in order to ensure the application of the OCN's policy on this matter.

69 Induction of moderators is the responsibility of an AVA officer, and during this process moderators receive their information pack which details procedures for the moderation visit, report-writing requirements, and common moderator queries. The OCN has identified further training of moderators as a problem, with limited numbers attending any events provided.

70 Two moderation visits are made each year, with a summative report being forwarded to the OCN in July, which is then copied by the OCN to the provider. Any matters identified by the moderator are expected to be addressed in the provider's action plan subsequently forwarded to the OCN. During the first visit of the new academic year, the moderator confirms that all matters raised have been addressed. Generally, this system seems to work well in identifying and resolving issues of concern, but there does not appear to be a documented system in place to deal with any matters that have not been addressed by providers. At present they are referred via the officers of the OCN to the Chair of the Quality Committee if they have been unable to obtain the necessary response from the college. The review team would recommend to the OCN that written procedures are developed to address this contingency.

71 At the end of their final visit, moderators check tutor records to confirm that the learners have achieved the requisite number and levels of credit for the award of the 'kitemarked' certificate before signing the recommendation for credit. OCNNWM administrative staff re-check these details on their receipt and, on rare occasions, amend them if necessary. The review team considered that these procedures were well understood and rigorously applied.

72 The 'cluster' model of moderation operated by OCNNWM, places moderators in the role of 'arbiter of process', although the Account observes that moderators are also required to undertake 'thorough and rigorous sampling of complete portfolios, amounting almost to double-marking'. In this model, it is necessary to have only one (or occasionally two) moderators for a programme, even where the programme includes pathways or subjects in which the moderator has no specialist knowledge or background. Moderators whom the review team met were keen to emphasise the benefits of this model, in terms of the moderator's ability to view the student 'holistically' and to contribute to consistency of assessment across the range of a student's achievement. Moderators also emphasised that the use of clear assessment criteria in the units enabled them to form reliable judgements about levels of achievement. In particular, they felt confident of their ability to confirm that students had demonstrated level 3 skills such as the processes of analysis, synthesis and critical evaluation.

73 Although there could be concerns that, with such a system, the level of subject-specific skills and knowledge could receive insufficient attention, moderators advised the review team that external moderation is supported by interaction with colleges' internal verification systems, which consider the detail of subject-specific standards of achievement. The Account acknowledges that this model of moderation 'relies heavily on effective systems of internal moderation/verification', but also recognises that this position has not yet been achieved. Indeed, the OCN's current guidance for moderators suggests a developing model, stating that 'Increasingly, an organisation's internal moderation (or verification) system will become a key quality indicator and the moderator's role ... will become less of a curriculum specialist and more of a generic function'. Discussion with various participants in review meetings confirmed that the two systems are not yet working to complement each other in all cases.

74 OCN officers, while acknowledging the right of providers to determine their own internal verification systems, have worked with Access co-ordinators to assist in the development of systems acceptable to both parties. However, there is not yet evidence to suggest that all providers have rigorous systems of internal verification in place. The responsibilities of the Quality and Moderation Officer will include a review of the model of moderation to ensure fitness for purpose, and a review of internal verification systems with providers in order to introduce models of best practice. Nonetheless, the review team concluded that, although there was clear evidence that the OCN had begun to address this matter, the stage had not yet been reached where it could be assumed that moderation was supported by reliable and fully operational systems of internal moderation or verification.

Learner experience, standards and progression

75 The HE admissions staff and former Access students met by the review team considered that the nature of preparation for the academic experience was appropriate, and that students from Access programmes were thoroughly prepared for higher education. Admissions staff expressed the view that standards and workloads of different Access courses were comparable, and local HE tutor confidence in the 'kitemarked' certificate appeared to be well established in most instances. Students understood the national standing of the 'kitemarked' Access certificate, and had been made aware of the requirements for the award of the certificate.

Student feedback and evaluation

76 The former students who participated in review meetings were aware of the quality assurance systems in operation throughout their Access programmes, including second-marking of work, meetings with the moderator, completion of questionnaires and representation at course team meetings. The review team heard that student representation at course team meetings had resulted in several positive outcomes for the student body. Formal feedback by learners to OCNNWM is via meetings with moderators and through the minutes of course team meetings.

Conclusions

77 OCNNWM maintains effective partnership between further and higher education, and takes account of its Access to HE responsibilities in its operation and planning. Since the previous review by HEQC in 1993, the AVA's development into an OCN has enabled it to increase its range of provision, and engendered considerable growth in learner registrations. One result of this overall growth in the OCN's provision and the range of its activities is that the organisation has, to some extent, outgrown its own structures and processes in a number of areas. If further growth for the OCN is envisaged or planned, it is especially important that these areas are now addressed in a systematic manner that takes a long-term view of the OCN's development and its responsibilities as an Authorised Validating Agency.

78 There are indications that the OCN's committee structures, for example, are under some strain. While individuals are undoubtedly interested and involved, there has been poor attendance at some key meetings. At the same time, it has been possible for decisions to be taken by a very small group because of outdated quorum requirements. In addition, executive decision-making frequently has to take place between committee meetings, following discussions between the OCN's Director and the Chair of the Management Board or Quality Committee. There is no indication that business has been conducted improperly, but there is an inevitable additional, and perhaps inappropriate, burden of responsibility placed on officers, in particular on the Director.

79 A more fundamental consideration about governance concerns the committees' abilities to carry out fully their specified remits. While there is no evidence of neglect of duty, the Quality Committee, in particular, is not able to address fully all of its duties as described in the Committee's remit. The OCN acknowledges the need to re-consider the role and remit of the Quality Committee and has brought forward proposals to address the difficulty, as it affects Access to HE provision, through the designation of a more developed role within the OCN for the Access to HE Committee. This would bring fuller and closer attention to the accreditation processes of the OCN's AVA activities, but to create a wholly effective and efficient governance structure, the OCN would benefit from a more radical revision of all of its committees' structures and terms of reference.

80 Although AVA work is a small minority of the OCN's total business, there has been some increase in learner registrations on the OCN's Access to HE programmes in recent years. If the OCN does not confirm the more developed role for the Access to HE Committee, the current governance structures will need adjustment in order for Access interests to be protected. There is an over-reliance on the presence of the Access background and experience of key individuals on committees for representation of Access interests. The commitment and experience of these individuals is valuable and evident, but the place of Access is not secure within the governance structures, as currently constituted.

81 In terms of staffing, development has also outstripped structures. There has been a recognition that the current staffing base is small and has recently been very stretched. To address this, there is some planned expansion with the appointment of a new officer, who is to have responsibility for the oversight of quality assurance matters generally, as well as moderation in particular.

82 The OCN itself suggests that the necessary and worthwhile development work required for the unitisation and re-validation of programmes may have been completed at the expense of other developments. Now that this process is almost complete, however, the OCN's development agenda is able to address other matters. Chief amongst these is moderation and its relationship with internal moderation or verification in colleges. The OCN's model of moderation rests very heavily on the assumption of certain support provided by strong internal moderation or verification. Although many providers have such systems in place, it is not comprehensive. There is a danger that there may be too much reliance on systems which the OCN does not control and for which it has yet to formulate clear guidance and policy of its own. The OCN's current emphasis on moderation at the programme level leaves assurance of student standards at the subject level vulnerable.

83 The Account submitted for this review demonstrated an admirable willingness and capacity to examine the organisation critically, and to provide suggestions for improvements and ways forward in a number of areas. The implementation of these plans will bring undoubted benefits to the quality of the OCN's operation and to its assurance of quality of provision and standards of student achievement on Access programmes.

84 The review team commends OCNNWM for:

i the development of a specific operational plan for Access to HE;

ii the support offered to providers in the process of unitisation of programmes;

iii co-operation with other OCNs on work on GCSE-equivalent programmes.

Recommendation to the ARLC

85 The review team recommends that OCNNWM be granted a conditional confirmation of its AVA licence with conditions to be met by the dates specified below.

Conditions

86 Subject to the approval of the ARLC, OCNNWM's licence is confirmed on condition that it:

i reviews its committee structures and membership, and makes revisions as necessary in order to address and clarify, in particular, matters of quorums and members' voting rights; accountability through reporting lines; and the optimum numbers and representation required for effective and accountable governance;

ii clarifies the remit and decision-making powers of the Access to HE Committee to enable the OCN to discharge fully its responsibilities for quality assurance matters, with reference to the QAA Recognition Scheme for Access to HE in England Wales and Northern Ireland (Part B);

iii prioritises the key action points for Access to HE where already identified, and incorporates them within the forthcoming Strategic Plan to enable action to be initiated on these matters;

iv clarifies its relationship with Staffordshire University through an updated Memorandum of Understanding, to reflect, in particular, changes related to length of staff contracts;

v revises its method of payment of Access to HE moderators to ensure that, from the academic year 2000-01, payments are made directly by the OCN, as the awarding body.

(All conditions to be met by 1 August 2000.)

Recommendations

87 The review team recommends that OCNNWM:

i monitors and analyses data about student withdrawal from Access programmes;

ii develops a strategic approach to targeting in order to support providers in identifying and recruiting appropriate groups;

iii reviews and fully documents the criteria and mechanisms for the transference of Access programmes from other AVAs;

iv clarifies and formalises moderation procedures for Access to HE to include:

practice on sampling moderators' reports by committees;
monitoring the period of service of moderators;
written procedures to describe action to be taken in the event of providers failing to address matters raised in moderators' reports;
written guidance on best practice internal verification/moderation models, to include reference to subject-level responsibilities, and on how these should dovetail with existing external moderation systems.

Open College Network - North West Midlands January 2000

Contents