Open College Network South West Wales
July 2000

Foreword

1 The Quality Assurance Agency for Higher Education (QAA) is responsible to the Department for Education and Employment (DfEE) for the recognition of Access to Higher Education courses. QAA exercises this responsibility through a national network of authorised validating agencies (AVAs), which are responsible for the processes of individual course recognition and the award of 'kitemarked' Access to HE certificates to students on QAA's behalf. QAA has developed a scheme for the licensing and review of the AVAs, the principles and processes of which are described in the QAA Recognition Scheme for Access to Higher Education in England, Wales and Northern Ireland. The Recognition Scheme is regulated and administered by the Access Recognition and Licensing Committee (ARLC), a sub-committee of the QAA Board of Directors.

2 This is a report of a major review of the AVA function of the South West Wales Open College Network (SWWOCN) by QAA. QAA is grateful to SWWOCN and to those who participated in the review for the willing co-operation provided to the review team.

Aims and objectives

3 The aims of the system of AVA review are:

i to provide the basis for an informed judgement by the ARLC about the fitness of the AVA to continue as a licensed agency;

ii to promote public confidence in Access to HE as a properly regulated and respected route into higher education by assuring;

the quality and adequacy of AVAs' systems and procedures;
the quality, comparability and range of AVAs' operations;
the adequacy and comparability of AVAs' standards for approval, moderation and monitoring of programmes;
consistency across AVAs in the operation of criteria for the granting of the Access to HE award;

iii to stimulate reflective and self-critical perspectives within AVAs, as an instrument to promote quality enhancement;

iv to provide an opportunity to identify and disseminate good practice of AVA operations;

v to provide a mechanism for ensuring necessary, and encouraging desirable, improvements and developments in AVAs.

4 The objectives of each AVA review are:

i to examine, assess and report on;

the development of, and changes in, the AVA since its last review or initial licence, and its plans and targets for the future;
the organisation's continuing viability and robustness and the ways in which the AVA demonstrates sound governance;
the efficiency and effectiveness of the AVA's operational and quality assurance systems;
the range and scope of the AVA's activities, and the appropriateness and value of these activities;
the ways in which the AVA approves and monitors programmes and the ways in which these processes take account of the need for consistency and comparability;
the ways in which the AVA satisfies itself of the adequacy and comparability of standards achieved by students gaining the 'kitemarked' Access award;
the evidence available to indicate the AVA's success in achieving its aims and targets;

ii to identify and report on:

strengths and good practice in procedures and operations;
areas which would benefit from further development;
areas requiring attention.

Outcomes

5 Following the review of an AVA, the reviewers' report is presented to the ARLC by a member of the review team. The Committee then makes one of the following decisions:

i unconditional renewal of licence for a specified period;

ii conditional renewal of licence with conditions to be met by specified date;

iii provisional renewal of licence with conditions to be met and further review visit by specified date;

iv suspension of licence until specified conditions are met;

v withdrawal of licence for operation as an AVA;

vi temporary renewal of licence with request for further information by specified date (decision suspended).

The review process

6 The review was conducted in accordance with the process detailed in the QAA Recognition Scheme for Access to Higher Education in England, Wales and Northern Ireland. The preparation for the review included an initial meeting between SWWOCN officers and the QAA Assistant Director with responsibility for Access to HE matters to discuss the requirements for the Analytical Account (the Account) and the process of the review; the preparation and submission by SWWOCN of its Account, together with a selection of supporting documentation; a meeting of the review team to discuss the Account and supporting documentation, and to establish a draft programme for the review visit; and negotiations between QAA and SWWOCN to finalise the programme and other arrangements for the review visit.

7 The review visit took place on 14 and 15 March 2000. The visit to SWWOCN consisted principally of meetings with representatives of SWWOCN, including OCN officers; members of the Executive Committee and Quality Group; moderators for Access programmes; Access programme course tutors; HE admissions staff; and former Access students now studying in HE.

8 The review team consisted of Dr Geoffrey Copland, Vice-Chancellor and Rector, University of Westminster, and Mr Steve Babbidge, Co-ordinator, South of England Open College Network. The review was co-ordinated for QAA by Ms Kath Dentith, Assistant Director (Access), Institutional Review Directorate.

AVA statistics 1998-99

Providers offering Access to HE programmes - 10

Access programmes available - 47

Access programmes running - 31

Access learner registrations - 885

Access to HE certificates awarded - 311

The AVA context

10 The South West Wales Open College and Access Consortium (SWWOCAC) formally adopted the name South West Wales Open College Network on 1 January 2000 in preparation for its application to hold a licence from the National Open College Network, and it recently achieved incorporation under this name. While well established as an OCN and AVA, (its first Access programme was validated in 1991) SWWOCN is thus new in its current form and much of its material, as supplied for review as an AVA, was presented with the former title and acronym, SWWOCAC.

11 The area served by SWWOCN covers 6,600 square kilometres and has a population of 723,000. Educational attainment in the area is below average for the UK, with 24 per cent of the population having no qualifications. It is an area of higher than average unemployment, with the principal sources of employment being concentrated in heavy and light industry in the main conurbations of Swansea, Llanelli, Neath and Port Talbot. Outside these conurbations, the area is predominantly rural, a characteristic which SWWOCN regards as being influential on both its own operation and on the providers and provision it services. Because of a perceived lack of facilities and opportunities, there is an outflow of young people from the area as a whole. The most under-represented group on SWWOCN's recognised Access courses is young men.

12 SWWOCN also operates within an all-Wales context, having links with the other two Welsh OCNs at both a strategic level, through the work of the Wales Access Federation (WAF), of which SWWOCN is a member, and also through more informal co-operation at an operational level between officers.

Membership and providers

13 SWWOCN is a company limited by guarantee, with standard Memorandum and Articles of Association. Members of the Company are defined as 'Further Education Members', 'Higher Education Members' and 'Other', this last classification covering organisations which offer education and training opportunities to young people and adults, including community and voluntary bodies, LEAs, and employers.

14 Membership of SWWOCN currently stands at six further education college members, five higher education members and 30 other members, including five Local Education Authorities and a number of other providers of post-16 education. Of this total, all six college members (Swansea College, Neath Port Talbot College, Gorseinon College, Carmarthenshire College of Technology and Art (CCTA), Pembrokeshire College and Coleg Ceredigion); two HE members (University of Wales, Lampeter, and Swansea Institute of Higher Education); and one other member (Swansea Council for Voluntary Service) were engaged in the provision of Access courses at the time of review.

15 In 1998-99, there were just under 900 Access students registered with SWWOCN, out of a total of approximately 16,000 registered learners.

Governance and committee structures

General Management Committee

16 Appointed representatives of the Company's members form the General Management Committee (GMC) which meets three times a year, one meeting of which is the AGM of the Company. Although the Account records that responsibility for strategic planning and policy making lies with the GMC, this committee does not appear to have separate terms of reference. The review team was informed that this body was the same as that described in the OCN's Articles of Association as the 'General Meeting of the Company'. The review team considered that these differences in designation were a source of potential confusion about the status and responsibilities of the Committee and the suggestion made at the time of the review visit that the OCN might move to having a 'Council of Members' could add to this confusion. As the GMC was referred to at several points during the review visit as having ultimate responsibility for various aspects of the OCN's operations, including general responsibility for Access to HE, the review team considered that these terminological issues should be resolved unambiguously.

Executive Committee

17 The Memorandum and Articles state that 'The affairs of the Company shall be managed by the Executive Committee.' Members of the Executive Committee are the directors of the Company and they are appointed by and from the members at the AGM. Membership of the Executive Committee comprises four FE members, two HE members and three other members. There is no specific requirement for Access expertise or experience to be represented at this level, but, in practice, several members of the current Executive Committee do have such experience.

18 The Executive Committee is responsible for developing 'strategic direction and operational policies' of the OCN and may delegate any of its powers to sub-committees. The current sub-groups are the Quality Assurance Sub-Group, the Personnel Sub-Group, the Welsh Language Sub-Group and the Finance and General Purposes Sub-Group. The Chairs of these four sub-groups are all members of the Executive Committee and are therefore able to present matters for the Committee's attention which arise from the sub-groups' separate areas of work. The Articles of Association stipulate that any sub-committee should provide 'regular and prompt reports to the Executive Committee'.

19 The review team was told that the responsibility for the approval of the award of the Access 'kitemark' rests with the Executive Committee, although there is no formal statement in the OCN's documentation to this effect. It was not clear to the review team how the Executive Committee discharged this responsibility, as there was no systematic receipt by the Committee of panel reports or notification of panel outcomes. It would seem that the principal mechanism for reporting is through the person of the Chair of the Quality Assurance Sub-Group, who is relied on to raise any issues relating to recognition that might have been notified to him, or through the Co-ordinator's regular report to the Committee, which could include reference to panel activity. The minutes did not, however, record that any programmes had been formally approved by the Committee, nor did they record any discussion of any panel outcomes.

Quality Assurance Sub-Group

20 The Executive Committee has delegated its responsibilities for quality assurance to the Quality Assurance Sub-Group, which is responsible for 'assuring the Quality and Standards of qualifications and award of credit' by 'overseeing the systems of Recognition, Moderation and Review'. Currently, members of this sub-group all have experience of quality assurance within their own institutions, although this is not specified as a requirement in any of the OCN's documentation.

21 The Quality Assurance Sub-Group has clear, specific and appropriate terms of reference, including the development of all accreditation procedures; overseeing the composition of recognition panels and the appointment of moderators; receiving moderation and recognition panel reports, reviewing any issues arising and authorising any necessary related actions; and taking action on matters related to appeals. The particular tools to be used by the Sub-Group for overseeing systems of review are not specified.

22 While these duties give the Sub-Group a wide-ranging remit with the possibility of close-monitoring 'the quality and standards of qualifications', the emphasis of the Sub-Group's work is largely on the systems and procedures themselves, rather than on the outcomes of these procedures. Indeed, the model adopted by the Quality Assurance Sub-Group is one of 'guarded devolution', and several of the duties listed in its terms of reference are not undertaken directly. As an example, the Sub-Group does not currently see individual moderators' reports, either in full or through sampling, a summary being provided by the Moderation Adviser.

23 The Sub-Group also has a specified duty to report to the Executive Committee, and a note that this needed to be a regular item on the Executive Committee's agenda was recorded in the minutes of the first meeting of the newly constituted Quality Assurance Sub- Group. The review team was told that reports do pass from the Sub-Group to the Executive Committee, but this was not evident, either in terms of reports received or in terms of any ensuing discussion, from any of the subsequent minutes the Executive Committee. The team was also told on a separate occasion that although such reporting from the Quality Assurance Sub-Group to the Executive Committee should be the way that it happened, this had yet to be achieved. As the minutes provide no record of such reports, either oral or written, being received, SWWOCN has no formal record that this has occurred. While in the past this may have been regarded by the OCN as a technicality, since incorporation, this has now become necessary in order for the OCN to be able to demonstrate that it is complying in full with that clause of its Articles quoted in paragraph 18.

Access to HE Development Sub-Group

24 The Account identifies one further sub-group, the Access to HE Development Sub-Group. Although terms of reference and membership for the group were defined, it has never met, and it was explained to the review team that this was because it had been difficult to agree the purpose of the group, and because it was difficult to bring practitioners together. The team was informed that, as a result, consideration had been given to different mechanisms through which Access development issues might be handled. The team noted that these considerations appeared to be very recent, and there was no evidence that any alternatives to an Access to HE Development Sub-Group had been explored in detail as yet. At the time of the review, there was thus not only a lack of any reference to specific Access issues in the terms of reference of the existing committee structure of the OCN, but also, a lack of any SWWOCN forum for Access development outside of that structure. The team therefore concluded that there was an absence of any effective mechanism for the identification of Access to HE needs or concerns, or a forum for the development and sharing of good practice.

25 QAA has since been informed that SWWOCN intends to form an Access to HE group, but that its remit and membership might be different to those originally suggested.

Aims and strategic planning

26 SWWOCN has three aims, all of which are consistent with the broad aims of Access to HE. They are: 'a) to promote access to education and training at all levels, ranging from local community education groups to higher education, particularly for those who have in the past been unable to benefit from existing provision; b) to improve the quality, accessibility and flexibility of learning opportunities in the area; c) to offer all those who seek it, the chance to gain nationally recognised credit(s) for their learning.'

27 The Account states that the General Management Committee has responsibility for strategic planning and policy making, while the Executive Committee is charged with developing the strategic direction within the OCN. However, this process and these responsibilities have to be seen within the all-Wales context within which SWWOCN operates and the role of the Wales Access Federation. The three Welsh OCNs work in consort to produce a common strategic plan, to which they each work individually. SWWOCN produces its own operating plan structured around the four aims of the WAF Strategic Plan. Section 2 (Changing Context) of the WAF Strategic Plan document makes reference to the need for the OCNs, as AVAs, to demonstrate that they are widening participation and meeting defined standards. It goes on to say 'As the three Welsh Open College Networks, our renewal of licences in 2000 is crucial if we are to keep open and further develop this important progression route for learners.' The four aims of the WAF Strategic Plan ('to promote the development of lifelong learning and inclusive learning through credit; to continuously improve the services of the OCNs to their members and users; to promote Welsh OCNs; to contribute to the development of a single credit based framework') and their accompanying objectives are then spelt out.

28 WAF's Strategic Plan itself makes no specific reference, in either its aims or its objectives, to the strategic needs of Access to Higher Education. SWWOCN's Operational Plan, based on the WAF Strategic Plan, also makes no specific reference to Access to HE. It does, however, under the objective 'widen participation through FE/HE partnerships', state an intention to 'provide/support new local credit-based routes to HE'.

29 The OCN's approach to developing Access would appear to sit outside any strategic or policy context and could be characterised as reactive rather than pro-active. The review team was told that the general focus of Access developments to date had been the expansion of provision to as many providers as possible, with specific focuses deriving from the priorities and strategies of FE partners. With a perceived 'plateau' in demand, the OCN was apparently now looking at 'more innovative developments'. These appear to be the 'new local credit-based routes' referred to in the Operational Plan.

30 The OCN itself does not provide any mechanism for the consideration of the development needs of Access to HE as a whole. There are no regular OCN-wide planning/development meetings with providers and receivers to discuss and review Access provision. There is little evidence of the OCN using data analysis about Access to inform its development policies and strategies, and its officers noted that much of the data which was available was 'unreliable'. Although approximately one third of the OCN's accreditation income is generated by Access provision, there is no notion of proportional re-investment in the development (or quality assurance) of Access.

31 There is a view (expressed by members of the Executive and OCN officers) that an earlier, more discrete focus on Access to HE had 'matured' into a generalised approach to widening participation and social inclusion as part of the AVA's role within a larger Open College Network. It is therefore considered that separate strategic objectives for Access to HE would be redundant, as Access is implicitly included in the plan for OCN work. It is thus difficult for the OCN to identify and monitor specific Access developments. Consequently, it will be difficult for potential students, providers or HE recipients of Access programmes to discern the development strategy for increased participation, inclusion and quality on SWWOCN's Access programmes.

32 After assessing the WAF Strategic Plan and the OCN's operating targets, and having talked to members of the Executive, OCN officers, course tutors and staff from receiving higher education institutions, the review team considered that although new developments were in evidence and were supported by the Accreditation Advisor once she was advised of them, these were primarily initiated by providers of Access courses or receivers of Access students: there was no evidence of an understanding of the need for a licensed AVA to demonstrate that Access provision existed within a clear, focused, planned and managed policy or strategic context.

Management, organisational structures and administrative arrangements

Premises and staffing

33 The OCN is based in adequate and appropriate offices in the Hendrefoelan Study Block of the University of Wales Swansea, for which premises it pays rent. It has access to a range of purchasing and service contracts through the University.

34 SWWOCN employs five full-time and one part-time member of staff. The staff team is headed by a Co-ordinator who is responsible to the General Management Committee, through the Executive Committee. The other staff are an Accreditation Advisor, a Moderation Advisor, a Programme Administrator (30 hours per week), Administration Officer (IT and Finance) and a Data Entry and Certification Clerk. These all report to the Co-ordinator, who is responsible for their appraisal and performance review; the Co-ordinator is appraised by the Chair of the Executive Committee.

35 There is no provision for the appointment of a Chief Executive of the Company, but the review team was informed that the Co-ordinator acts as the Chief Executive. She also holds the post of Company Secretary. In the absence of clear definitions, there could be confusion as to where responsibility lies in the operations of the Company. These matters need to be properly defined and ambiguities of responsibilities removed.

Finances

36 The Company is responsible for its own financial management and for producing audited accounts, the process of external auditing being undertaken through the services of member institutions. The Finance and General Purposes Committee monitors accounts at its three-monthly meetings, although the cash flow statement is not updated to match monthly budgetary progress reports. Neither the OCN's audited accounts nor a balance sheet were seen by the review team, but the financial statement for 1998-99 suggests that, in financial terms, the Company is operating successfully, with a surplus on operational income/expenditure of some £25,000: sufficient to have allowed the OCN to provide a partial refund to members. Budgetary progress reports for the first seven months of 1999-2000 indicate that the company continues to operate successfully.

37 The Financial Regulations are clearly set out, although there is some ambiguity of responsibility. Reference is made to responsibility for monitoring budgetary control resting with the Co-ordinator and the 'Administrator'. This latter term appears to refer to the Administration Officer (IT and Finance), who has day-to-day book-keeping responsibilities. SWWOCN's responsibilities as a company make it imperative that such points of detail are clarified and a single point of accountability defined to remove any possibility of misunderstanding of who is responsible. Attention to the detail of formal accounting procedures and the production of standard financial accounts is required, not only to satisfy Companies House but also to satisfy the OCN's subscribing members that their finances are being properly managed. There was nothing to suggest that there had been any financial irregularities in the operation of the OCN.

38 Members of SWWOCN pay membership fees, and accreditation charges are charged to all providers who use this service. The fee structure is based on 70 per cent of income to come from FE members, 12 per cent from HE members, 10 per cent from LEAs and 8 per cent from others. In addition, there are accreditation charges of £5 per registration and £2.50 per credit awarded. The Access components of this produce £4,425 for Access to HE registrations and £28,581 for Access to HE credits awarded.

Developments since the last review

39 The response to the HEQC recommendations following the last review in 1995 was given in an undated short document that appeared to have been written before the existing Co-ordinator or Chair of Executive Committee were involved. Evidence gained by the review team showed that links with HEIs had been strengthened and satisfaction with these was expressed by those met during the review. HEI representatives play important roles in validation panels and in quality assurance processes. The University of Wales Swansea has initiated a sub-committee on Access on which the SWWOCN Co-ordinator sits, and entrance and progress of Access students is carefully monitored by the University. In addition, the Accreditation Advisor has led development sessions for admissions tutors.

40 The HEQC report identified the need for improved communication and profile-raising, particularly with HEIs. There seems to have been little formal progress with this, but informal networks have developed. A newsletter had been started but has since been discontinued, and SWWOCN is now considering greater use of email and establishing other electronic communications. There has also been increased activity in other parts of the geographic area outside Swansea, and Access co-ordinators reported improvements in developmental work being done locally. The geographic dispersion of the area was cited by OCN officers as being a difficulty in ensuring attendance at meetings and in establishing a sense of community. The review team recognised that this was a potential barrier, but noted that there was also an enthusiasm expressed by the co-ordinators for further opportunities to meet to discuss issues related to Access to HE.

41 Attendance at recognition panels is seen as a useful vehicle for tutors to share practice in curriculum design and development but was judged by tutors as not enough to keep them well informed. A number of tutors commented that they would be better informed of local and national developments if the OCN was to convene termly or per-semester meetings for Access practitioners. The observation was made that the fact that the Accreditation Advisor lived in west Wales and was able therefore to 'offset' what may otherwise have become too strong an emphasis on Swansea and its locale, was serendipitous rather than planned.

The development, validation and evaluation of Access programmes

42 The OCN has a range of Access provision that is consistent with the usual academic profile of general further education colleges which comprise the majority of providing institutions in membership.

43 The OCN has recently initiated some work to bring greater consistency of outcome to Access programmes, and discussions have taken place to increase standardisation of the successful completion criteria for Access programmes. A session was organised last year to consider and plan for the inclusion of key skills in all Access programmes, an initiative driven by the National Open College Network, and the OCN has successfully achieved this. A common key skills moderation scheme has been successfully introduced and moderators of other cognate subject areas are being encouraged to meet to discuss common standards. These developments are at a formative stage, and the review team was unable to form a view of their impact on the provision as a whole.

Development and design of new provision: standard requirements and advice to providers

44 Development support is offered by the OCN to its members through a dedicated role of 'Accreditation Advisor'. The Accreditation Advisor is in regular contact with providing institutions, which allows her to be sensitive to institutions' plans and ideas for developing their provision.

45 The OCN has recently revised its guidance documentation for all programmes submitted for recognition and makes specific reference to the additional requirements for Access. Much use is made of the capacity to adopt commonly shared units of assessment from within the OCN or from the all-Wales unit database, and the OCN also operates within an all-Wales framework for credit transfer, exemption and APL. There is an awareness of the perceived benefits of flexible, modularised and distance learning provision to promote study opportunities in the OCN's geodemographic context. There is also some evidence of cross-sectoral partnerships being formed to develop and deliver 'outreach' and community-based provision.

46 The requirement to include key skills as part of the programme and the successful completion criteria for the 'kitemark' is clearly indicated in the OCN's submission guidelines. Advice and support for centres wishing to design new Access provision is provided by the Accreditation Advisor and the review team noted in discussion with Access tutors that this support was well received.

Validation of new and revised provision

47 New programmes are submitted to a recognition panel comprised of colleagues from the further and higher education sectors, after a period of extensive developmental support. Alterations which can be made at panel are made directly to a disk copy of the programme submitted at the same time as the paper document. The Accreditation Advisor is in attendance at all panels, which are chaired by one of a variety of members of the OCN's Executive Committee or Quality Assurance Sub-Group. Where a moderator has been identified, he or she is invited to attend. Although common practice is to include two or more external panel members from HE, there is currently nothing set down in the OCN's procedures about panel composition.

48 The review team noted that OCN documentation was inconsistent about decision-making at panel. One recent document describing the recognition and approval process, bearing the OCN's current name and acronym, suggested there were three possible outcomes of a panel: 'recognised without amendments'; 'recognised with agreed amendments'; and 'deferred for rewriting'. The Notes for Chairs of Recognition Panels, however, refers to the possibility of 'suggestions, recommendations and conditions' being made, and also refers to the possibility of making 'minor amendments to wording as you work through'. In this document a 'recommendation' is defined as 'something which is expected to be done during the first run of the programme and reviewed with the moderator' and a 'condition' as 'something which must be done before the programme starts'. The document Information for Recognition Panel Members states that a programme may be recognised 'with agreed changes and amendments', 'when certain conditions are fulfilled within an agreed timescale' or 'with recommendations for tutors and moderators to bear in mind during the first programme run', or a programme may be 'referred for re-writing'. The Account states that 'there may be conditions, or more usually recommendations set at panel'. It was not clear to the review team which of these documents represented the definitive version of the possible recognition panel outcomes.

49 In discussion with OCN officers, the team learnt that conditions were not a preferred option, with programmes either achieving recognised status following 'minor' amendments at panel or, in exceptional circumstances, being referred for re-development. This was borne out by the team's scrutiny of panel reports, during which they were unable to identify any instances of conditions being attached to the approval of programmes. OCN officers ascribed this phenomenon to a process of submissions not being recommended as 'panel ready' until the Accreditation Advisor was satisfied that the likelihood of conditions being attached was minimal. It was not clear to the team how it was possible to ensure that a robust process of recognition, which included an appropriate degree of external scrutiny, would not identify matters which would need to be addressed before approval could be formally granted. If, in spite of the work of the Accreditation Adviser in preparing submissions so that they are 'panel ready' there are circumstances in which programmes might have to be referred, it would seem that that process cannot guarantee 'panel readiness'. It would follow that a lesser order of changes to the programme, which could be addressed by conditions being attached before programme approval could be granted, must be more likely.

50 Responsibility for approval of programmes was also unclear. The Notes for Chairs of Recognition Panels refers to the need to 'agree with panel members whether the programme can be recommended for approval' (although it does not state to whom this recommendation should be made); the other OCN document on the recognition and approval process, in its description of the possible outcomes makes no reference to recommendation of approval, but suggests that the panel itself is responsible for the recognition of the programme. This suggestion is also made in the Account in its statement that 'The panel will usually recognise the programme'.

51 The review team also noted that it was not always possible to confirm that the development/recognition process had come to a close (in the absence of formal confirmation of this) and in one particular instance was struck by the ambiguity of the panel report. This ambiguity was reflected in discussions with members of the Executive Committee, the Quality Assurance Sub-Group and OCN officers, where different opinions were held as to whether this particular programme had been recognised or not.

52 The review team concluded that the process for the approval of a 'kitemarked' Access programme was confused at several points and considered that this fundamental process, which is central to the quality assurance of programmes, could not be considered robust.

Monitoring, evaluation and review of programmes

53 The OCN's principal focus on monitoring and evaluating the Access programmes it has approved is via its moderators (further commentary is provided in the following section). There would not appear to be any systematic monitoring of programmes beyond this, with the OCN itself noting in its Account that 'The requirements of end of year course reviews need to be clearly specified and clear guidelines for providers developed. The methods of monitoring and evaluating Access Programmes need further development, in association with Providers'. The review team would concur with this observation as did Access tutors who noted a 'lack of clarity' in what was expected by the OCN at the end of a programme cycle.

Evaluation of procedures

54 The OCN has evaluated and revised some of its procedures as a result of preparation for the QAA review and previous audit by the National Open College Network (NOCN). There do not appear to be systematic procedures in place specifically to evaluate the development, validation and evaluation of Access provision within the OCN, however, and the review team could not see in the papers it scrutinised a clear link between the self-assessment process, the monitoring of this process, for example by the Quality Committee, and the OCN's planning or development process.

Assessment, moderation and award of certificates

Policies/guidance on student assessment methods

55 The adequacy and fitness for purpose of a programme's assessment methods and instruments is judged at the recognition panel. There are very helpful and accessible guidance documents which have been introduced this academic year in SWWOCN's document Assessment: A Guide for Tutors. This explains in some detail what is meant by 'assessment', 'evidence', and 'plagiarism', as well as including sections on grading, tracking achievement and ensuring consistency. Although the review team noted the general applicability of this guidance, it also noted the absence of any particular guidance or policy for Access programmes. Thus, an Access programme may or may not include examination skills, may or may not deploy a range of assessment instruments, and may or may not operate a balanced assessment loading throughout its delivery. The review team considered that OCN guidance on these matters could enhance consistency on programmes and would recommend the development of such guidance.

Selection, appointment and training of moderators

56 The OCN has a designated post of Moderation Advisor who has responsibility for all elements of the moderation process within the OCN. There is evidence of extensive networking by the Moderation Advisor across the OCN's region and clear general guidance documents are produced for all moderators. Moderators are frequently identified prior to recognition panels and are encouraged to attend. There are clear and appropriate criteria for the appointment of moderators, and the Moderation Advisor is assiduous in identifying appropriate individuals to undertake this role.

57 The review team noted, however, that the 'approval' of a moderator for a programme was the sole responsibility of the Moderation Advisor and that neither the panel, nor the Quality Committee, appeared to be involved in subsequent confirmation of this process, in spite of the specific inclusion of this responsibility in the Quality Committee's terms of reference. The OCN recognises that this aspect of the appointment of moderators represents a weakness in its quality processes and intends to address it through reports to the Quality Committee from the Moderation Advisor.

58 All moderators receive a briefing and support from the Moderation Advisor which, according to the Account, includes reference to the need to ensure that the successful completion criteria for the award of the 'kitemark' are met. In discussions with moderators, it was apparent that they were aware of the need to ensure that only allowable combinations of units were achieved for the award of the 'kitemark'. The guidance documents issued to moderators do not, however, make any references to this need, nor do the forms which are used to trigger certification.

Moderators' reports

59 There is a common document for all Access reports. The template gathers information using a clear, accessible and easily collated layout. The review team noted, however, that the range of prompts on the form is focused on what may be described as administrative processes rather than evaluative comment on the adequacy of programme delivery or the assessment of academic standards. Reports scrutinised by the team did not provide reassurance that the OCN could use them to monitor substantive quality issues such as the effectiveness of internal standardisation/verification; patterns of achievement; discontinuation and progression; transparency in the formal confirmation of the award of the 'kitemark' and conduct of an assessment board; and issues arising from tutor team/student evaluation and feedback on the efficacy of the moderation process. The Account asserts that moderators do indeed '...evidence these [internal verification] systems and report as necessary' but there was no evidence of this taking place. It also asserts that 'The Moderation Report is vital in the monitoring and review of Access Provision.' There does not appear, however, to be any formal mechanism for moderators to draw the OCN's attention to urgent issues that require immediate action. Discussions with moderators and Access co-ordinators revealed that a number concurred with these observations and would have preferred a report structure which allowed a fuller and more evaluative commentary.

60 All moderators' reports are received and read by the Moderation Advisor who determines if, and what, action may be required, and feeds back this advice to providers as he determines necessary. There is currently no provision for the moderator's report to be received formally at senior level within the provider institution, although the OCN intends to address this. The review team noted that there did not appear to be clear lines of reporting between the output from the moderation process and the Quality Assurance Sub-Group. Although officers and members of the Quality Assurance Sub-Group made reference to the existence of direct reporting into the Sub-Group by the Moderation Advisor (as does the Account), the team could not locate any evidence in minutes from the Sub-Group of substantive issues being discussed.

External involvement in assessment and moderation procedures

61 The OCN operates a model of appointing moderators who are external to the organisation offering the Access programme but not external to the OCN. Given the geographical context in which the OCN operates, this is understandable. Although the review team could not locate formal confirmation in the OCN's documentation, an additional level of externality would appear to be encouraged in the first year of a newly approved Access course through the attendance of a tutor from another Access provider.

Procedures for the award of the 'kitemarked' certificate

62 The Account refers to a process whereby 'At the final meeting the Moderator and Access Coordinator consider the achievement of the learners and the results recorded on the Recommendation for the Award of Credit form.' The OCN does not issue guidance for the procedures and conduct of a final assessment board and, as noted above, there is no reference in the formal guidance documentation issued to moderators on the process for confirming the award of the 'kitemark'.

63 From discussions with moderators it would appear that practice varies across the OCN, and depends on the traditions of different institutions rather than on any guidance from the OCN. In some instances, a final meeting is convened by the moderator, in others by the tutor or programme leader; the course team may or may not be in attendance; there may or may not be a member of the providing institution's senior management or quality team present; the meeting may or may not be formally conducted and/or minuted; all portfolios may or may not be available for scrutiny; and moderators may sample students' work through making their own selection or they may ask tutors to make the selection for them.

64 The review team noted that this considerable variation in practice was apparently acceptable to the OCN in its award of a national qualification. With such inconsistency in the process which leads to the award of the Access certificate, it was unclear to the review team how the OCN was able to assure itself of equitable treatment of learners or consistency of outcomes.

65 The team also noted that the OCN's appeal process effectively allowed an appeal by a centre against a moderator's judgement of academic standards. The moderation process is described as the two parties ordinarily 'reaching agreement' over academic standards and if not, allowing an appeal to the OCN. There did not appear to be an understanding of the difference between appeals against process and appeals against judgement, and the OCN's description of the appeal process that is in place varied between documentation provided to the team (Moderator Information pack and Award Appeals Procedure flowchart). From discussions with moderators, it appeared that there was an understanding of this distinction, and those present were clear that their decision was final if the appropriate procedures had been followed. The team noted that the appeals process had not yet been invoked.

66 Procedures are in place for checking that moderators have followed the appropriate rubric for allowable combinations in the award of the 'kitemark'. These rely on members of SWWOCN's administration team noting that they are dealing with an Access Recommendation for the Award of Credit (RAC) in a context where the majority of RACs do not have specific completion criteria. The OCN's database does not support an automatic verification process for the award of the qualification.

Learner experience, standards and progression

Nature of learner experience and preparedness for higher education

67 The review team heard from previous students that, generally, they were well prepared for higher education: a view echoed by staff from receiving HEIs. Those students who were near the end of their studies in higher education noted that, although they had received excellent support on their Access programme, the transition to higher education had been made more difficult because of a difference in assessment methods with an increased emphasis on assessment by examinations; a greater importance attached to meeting deadlines; and an increased workload. For some students, a supportive preparation, but one which also introduced them more fully to the realities of the common assessment regime of higher education, would appear to be of benefit.

Programme equivalence and output standards

68 The OCN asserts in its Account that its focus on ensuring comparability of experience and preparedness for higher education lies, in the first instance, with its Accreditation Advisor who '...is able to share good practice and ensure comparability between Access programmes in the region.' This is supported by moderators who '...need comparatively little help (from the OCN) in judging the standards the learners achieve in the Access to HE Programme' and providing institutions' quality systems where 'Consistency within institutions offering Access to HE programmes is guaranteed by the internal verification systems which most providers operate.'

69 The review team did not share this view, and could see little evidence of systematic attempts by the OCN to assess or monitor programme equivalence and output standards, in terms of fitness for purpose and academic achievement. Indeed, the evidence scrutinised by the team revealed a lack of recognition by the OCN of its responsibility as the awarding body for underwriting consistency in its awards. The over-reliance on individual judgement and other organisations' systems sat uncomfortably with the assertions made by the OCN that consistency could be 'ensured' and 'guaranteed'. The team noted that in its Account the OCN realised that 'The methods of monitoring and evaluating Access Programmes needs further development...' but could see little evidence of action planned to tackle this.

70 Where the OCN does discharge its responsibility for ensuring consistency is its OCN-wide key skills scheme which has two moderators who work as a team with tutors across all providers. There is a recognition that a similar approach would be generally beneficial to whole programmes, and the review team regarded indications of proposals for benchmarking initiatives as a positive development. The Account notes, however, that '...the difficulty of getting staff released remains a formidable barrier.'

Progression and monitoring

71 There were no summary statistics for completion and progression available and the OCN has recognised the need to improve its data collection and analysis. Registration figures were available for the period 1994-99 with an average of 780 students for the period. The OCN would not appear to place much reliance on what data it has to inform its planning process and the review team was interested to note that the Account attributed an almost 50 per cent drop in 1996 to '...apprehension surrounding the introduction of fees and loans in HE'.

72 The ease of access to, and reliability of, progression data about Access students in HE is widely held to be problematic for AVAs. The University of Wales Swansea, one of the OCN's member HEIs, does record this data about progression and achievement of Access students, and has done so for ten years. SWWOCN is in a fortunate position, therefore, as the University has convened a sub-committee (on which the Co-ordinator sits) which considers this data. This data has not been used to inform the OCN's planning processes to date, but the Co-ordinator now receives a copy of the annual report in which the data is summarised, and it intended that this, and reports that have been requested from other HE members, will be used to support the OCN's planning and development in the future.

Student evaluation and opportunities for feedback

73 In discussion with students, the review team noted that they had all had the opportunity to provide feedback to their Access tutors as part of providing institutions' course review and evaluation systems. Although this is not a SWWOCN requirement, the information thus gained by providers would appear to be under-utilised by the OCN, as it does not insist on receipt of programme reports, nor does it charge its moderators with providing summary findings, as noted above. In addition, the OCN does not provide guidance to providers as to what might be expected for the proper monitoring of Access courses, which might indicate the sufficiency of, or need to supplement, college requirements.

Conclusions

74 Access to HE provision in South West Wales shows some growth over the last three years, following a substantial drop between 1995-96 and 1996-97, and numbers of learner registrations are now close to their 1995 level. This growth has taken place in the context of more general growth in OCN-accredited provision and development work leading to the expansion of SWWOCN to include a wider range of providers. However, Access programmes are still very largely the preserve of the traditional FE sector and growth has largely taken place through the expansion of established providers' portfolios.

75 During this period, SWWOCN has continued to provide an accreditation service to its members which has allowed this development. At the same time, it has had to confront a range of challenges to its mode of operation, including the accommodation of a broader range of provision and new providers' different needs; new quality standards and licensing requirements from NOCN; and, most recently, the demands of company status. As a very young company, SWWOCN is still at the stage of recognising and adapting itself to the full requirements of its new status, and clarifying roles and procedures to ensure they are appropriate to this more formal legal status. It is also still in the formative stages of providing secure and rigorous quality assurance mechanisms for many of its operations.

76 In the context of these competing and immediate demands, SWWOCN has given little specific attention to developing its role as a licensed AVA or to its obligations as an awarding body for Access to HE awards, particularly in relation to its developmental and quality assurance responsibilities.

77 It would appear that, over this period, there has been minimal attempt to consider the OCN's distinct responsibilities as an AVA at either a strategic or an operational level, the Executive Committee regarding AVA activity as inextricably linked to its more general OCN activity. While the different elements of the OCN's work may have much in common and may complement one another, such integration holds inherent difficulties for the OCN in identifying responsibility and authority for Access. Furthermore, the assumption that the OCN's responsibilities for its Access work can be wholly accommodated within its more general OCN structures appears to have led, on this occasion, to not just a lack of clarity but an effective absence of consideration of Access development at the strategic level.

78 The totally integrated model is not necessarily an unworkable one, and may be the most appropriate in some circumstances, but it will need certain safeguards if this particular area of provision, for which the OCN was granted a licence, is to be enhanced and developed, and remain fit for purpose in changing circumstances. The strategic emphasis on alternatives to Access, where none exists for Access itself, call into question whether SWWOCN can be adequately fulfilling this part of its responsibilities. It is not necessary to have a total separation at operational level or within governance structures, but, if an OCN decides to adopt such an integrated model, the process by which the OCN had reached its decision, and the rationale for it, should be apparent.

79 The framework for quality assurance is broadly in place, but the detail of it is not fully worked out, or is inconsistently applied. There are significant weaknesses at many levels and at all of the major stages, from initial programme recognition through to confirmation of award. Over the last year, changes have been introduced to meet NOCN quality standards and licensing requirements but, in spite of the experience of the Quality Assurance Sub-Group, current structures do not provide rigorous monitoring of central procedures. This is exacerbated by weaknesses in the OCN's official documentation. There is, in general, a lack of attention to detail in its documentation, some of which results in an inconsistency between practice and theory, and some of which reflects a real confusion about the quality assurance mechanisms and procedures themselves. Although there are some indications of very recent developments to improve quality assurance processes, these are not yet embedded, and have yet to show real results.

80 SWWOCN has good relationships with local HEIs, and is able to call on appropriate HE representatives to be involved in its accreditation processes. Similarly, Access practitioners show commitment and enthusiasm for Access work and, in spite of the acknowledged travel difficulties and time restraints, express an interest in fuller involvement in the OCN's development of this area of provision. The absence, to date, of an Access to HE Development Sub-Group, or its equivalent, means that there is currently no mechanism which can deliver the functions which constituted the terms of reference which were proposed for the group: a forum for the exchange of good practice; a source of consultation over proposed changes; and a body to make recommendations to the Executive Committee on the basis of its consideration of developments in Access to HE.

81 The OCN's officers support and work individually with providers to good effect, but there is no evidence of a structure for the Access-related aspects of their work, or of a clear framework to develop and ensure quality and standards on Access programmes. Both Access providers and HE receivers of Access students are active in developing Access provision and enhancing opportunities for Access students: with fuller leadership from the OCN there is the potential for continued development in Access provision in this area.

Recommendation to the ARLC

82 The review team recommends that SWWOCN be granted a provisional renewal of its OCN licence, with conditions to be met by the dates specified below, and re-visit in spring 2001.

Conditions

83 Subject to the approval of the ARLC, SWWOCN's licence is renewed on condition that the OCN:

i locates ultimate responsibility for the award of the 'kitemark' to an Access programme with the AVA itself, so as to meet the need for monitoring and the verification of consistency within the validation process;

(condition to be met by 14 July 2000)

ii undertakes a review of its governance structures as they relate to Access to HE, to include the roles and operation of the General Management Committee, the Executive Committee and the Quality Assurance Sub-Group and, in particular;

ensures that the Executive Committee is enabled to undertake in full its responsibilities as a Board of Directors, acting in accordance with the Nolan Principles;
ensures that the locus of responsibility for the AVA licence is clearly located and defined;
clarifies the terms of reference of all committees, ensuring that these are adequate for the responsibilities of an AVA as defined in the QAA Recognition Scheme for Access to Higher Education (Part B) and are consistent with proper oversight of robust quality assurance for Access programmes;
specifies the membership of all committees to assure continuing appropriateness of representation and expertise, and to clarify the status of officer representation on committees;
provides a clear definition of reporting responsibilities and mechanisms between officers and committees, and between the Executive Committee and its Sub-Groups;

iii undertakes a review of its quality assurance procedures as they relate to Access to HE and, in particular;

clarifies quality assurance responsibilities for Access within the governance structures;
clarifies the range of possible outcomes of a recognition panel for Access programmes, to ensure that the process is robust;
develops post-panel procedures to include the development of a transparent system for monitoring the meeting of conditions prior to formal approval, and the introduction of a system of formal notification to providers of the full approval of Access programmes;
develops a system for moderation reporting which encourages fuller comment by moderators and, specifically, requires comment on standards of student achievement in relation to the level required for the award of the 'kitemarked' Access certificate;
develops guidance for providers on the appropriate composition, conduct and regulations for final assessment boards for Access programmes, including clear and unambiguous procedures for the formal award of the 'kitemarked' certificate to students;
ensures that all Access providers submit to the OCN an annual report for all Access programmes;
develops guidance for providers on what reports should included, and implements a system to identify issues arising from the annual reporting process;

iv reviews its strategic planning procedure to allow for consideration and articulation of specific strategic objectives relating to Access to HE and its development within the OCN, and to be informed by the evaluation of data pertaining to Access students and programmes.

(conditions ii, iii and iv to be met by 1 December 2000)

Provisional confirmation of licence allows the AVA to continue to operate under licence but makes provision for the immediate withdrawal of the licence if the conditions are not met in the time allowed and to the satisfaction of QAA. Were it to be necessary for the Agency to withdraw the licence, the AVA would need to put in place procedures for the transfer of registered students to an alternative licensed AVA.

Recommendations

The review team recommends that SWWOCN:

i establishes a forum, which has a place in the OCN's governance structures, in which Access tutors and receiving HEIs can discuss matters of good practice and development for Access;

ii develops a strategic approach to targeting in order to support providers in identifying and recruiting appropriate groups;

iii develops systems for recording, evaluating and using cohort data to inform the planning process and reporting to committees;

iv introduces formal structures to develop the assurance of programme equivalence and consistency of output standards between programmes;

v reviews and develops its guidance documents which impact on Access, to incorporate references to particular regulations or guidance on Access where appropriate;

vi undertakes a systematic re-badging of all its formal documentation relating to Access with the OCN's current name;

vii adopts transparent criteria for the selection and appropriate membership of recognition panel members;

viii reviews the post-panel administrative process to ensure clarity of outcomes is recorded and communicated;

ix implements transparent systems for the identification, appointment and approval of moderators;

x clarifies the purpose and scope of appeals against moderation and reviews the adequacy of its checking process for issuing certificates;

xi develops and implements a system to ensure that moderators' feedback to providers is supplied directly to heads of providing institutions;

xii re-considers the format of the tool used for the recommendation for award of credit to students on Access programmes, with a view to providing a format which can be used effectively and reliably by all Access providers.

Open College Network South West Wales July 2000

Contents