Open College Network, South East Wales
July 2000

Foreword

1 The Quality Assurance Agency for Higher Education (QAA) is responsible to the Department for Education and Employment (DfEE) for the recognition of Access to Higher Education courses. QAA exercises this responsibility through a national network of Authorised Validating Agencies (AVAs), which are responsible for the processes of individual course recognition and the award of 'kitemarked' Access to HE certificates to students on QAA's behalf. QAA has developed a scheme for the licensing and review of the AVAs, the principles and processes of which are described in the QAA Recognition Scheme for Access to Higher Education in England, Wales and Northern Ireland. The Recognition Scheme is regulated and administered by the Access Recognition and Licensing Committee (ARLC), a sub-committee of the QAA Board of Directors.

2 This is a report of a major review of the AVA function of the South East Wales Open College Network (SEWOCN) by QAA. The Agency is grateful to SEWOCN and to those who participated in the review for the willing co-operation provided to the review team.

Aims and objectives

3 The aims of the system of AVA review are:

i to provide the basis for an informed judgement by the ARLC about the fitness of the AVA to continue as a licensed agency;

ii to promote public confidence in Access to HE as a properly regulated and respected route into higher education by assuring;

the quality and adequacy of AVAs' systems and procedures;
the quality, comparability and range of AVAs' operations;
the adequacy and comparability of AVAs' standards for approval, moderation and monitoring of programmes;
consistency across AVAs in the operation of criteria for the granting of the Access to HE award;

iii to stimulate reflective and self-critical perspectives within AVAs, as an instrument to promote quality enhancement;

iv to provide an opportunity to identify and disseminate good practice of AVA operations.

v to provide a mechanism for ensuring necessary, and encouraging desirable, improvements and developments in AVAs.

4 The objectives of each AVA review are:

i to examine, assess and report on;

the development of, and changes in, the AVA since its last review or initial licence, and its plans and targets for the future;
the organisation's continuing viability and robustness and the ways in which the AVA demonstrates sound governance;
the efficiency and effectiveness of the AVA's operational and quality assurance systems;
the range and scope of the AVA's activities, and the appropriateness and value of these activities;
the ways in which the AVA approves and monitors programmes and the ways in which these processes take account of the need for consistency and comparability;
the ways in which the AVA satisfies itself of the adequacy and comparability of standards achieved by students gaining the 'kitemarked' Access award;
the evidence available to indicate the AVA's success in achieving its aims and targets;

ii to identify and report on;

strengths and good practice in procedures and operations;
areas which would benefit from further development;
areas requiring attention.

Outcomes

5 Following the review of an AVA, the reviewers' report is presented to the ARLC by a member of the review team. The Committee then makes one of six decisions:

i unconditional renewal of licence for a specified period;

ii conditional renewal of licence with conditions to be met by specified date;

iii provisional renewal of licence with conditions to be met and further review visit by specified date;

iv suspension of licence until specified conditions are met;

v withdrawal of licence for operation as an AVA;

vi temporary renewal of licence with request for further information by specified date (decision suspended).

The review process

6 The review was conducted in accordance with the process detailed in the QAA Recognition Scheme for Access to Higher Education in England, Wales and Northern Ireland. The preparation for the review included an initial meeting between SEWOCN officers and the QAA Assistant Director with responsibility for Access to HE matters to discuss the requirements for the Analytical Account (the Account) and the process of the review; the preparation and submission by SEWOCN of its Account, together with a selection of supporting documentation; a meeting of the review team to discuss the Account and supporting documentation and to establish a draft programme for the review visit; and negotiations between QAA and SEWOCN to finalise the programme and other arrangements for the review visit.

7 The review visit took place on 1 and 2 March 2000. The visit to SEWOCN consisted principally of meetings with representatives of the OCN, including OCN officers; members of the Board of Directors, Quality Assurance Committee and Access Task Group; moderators for Access programmes; Access programme co-ordinators; HE admissions staff; and former Access students now studying in HE.

8 The review team consisted of Dr Peter Easy, Assistant Director, Cheltenham and Gloucester College of Higher Education, and Mr Anthony McClaran, Deputy Chief Executive, Universities and Colleges Admissions Service. The review was co-ordinated for QAA by Ms Kath Dentith, Assistant Director (Access), Institutional Review Directorate.

AVA statistics 1998-99

Providers offering Access to HE programmes - 10

Access programmes available - 69

Access programmes running - 54

Access learner registrations - 544

Access to HE certificates awarded - 346

The AVA context

General background

10 The South East Wales Open College Network (SEWOCN) was, until 1 March 2000, known as the South East Wales Access Consortium (SEWAC). SEWAC was established in 1990, securing its licence from the Access Courses Recognition Group (ACRG) in October that year. An Open College Network (OCN) project was set up in 1992, and in 1994 SEWAC became a full member of the National Open College Network (NOCN). SEWOCN is also a member of the Wales Access Federation (WAF), which was established in 1997 to promote the work of the three Welsh OCNs.

11 SEWOCN is based in Cardiff, and is located in premises owned and provided at a nominal rent by Cardiff University. It operates across the unitary authorities of Blaenau Gwent, Bridgend, Caerphilly, Cardiff, Merthyr Tydfil, Monmouthshire, Newport, Rhondda Cynon Taff, Torfaen and the Vale of Glamorgan. The area covered includes urban, rural and valley communities which have suffered from the effects of the dramatic decline in their traditional industries. The roots of Access provision in this area go back to the 1970s, when courses were started for mature students with trade union backgrounds.

Mission, aim and objectives

12 SEWOCN's mission is set out within the Objects of its Articles of Association. These state that its mission is:

'To promote the advancement of education for public benefit, in particular by

a operating in South East Wales as a responsive accrediting body for programmes offered by all providers of education in the region

b promoting the widening of participation in lifelong access to, and inclusive forms of, educational provision

c promoting improvements in the educational welfare of learners through the award of credits and credit-based qualifications for the general benefit of the public, to improve access to learning opportunities and to facilitate progression to further learning, employment and higher education

d safeguarding and promoting the quality and standards of programmes of learning.'

13 SEWOCN also operates in an all-Wales context, the main organisational expression of which is WAF. The mission of WAF: 'to support and co-ordinate the Welsh OCNs in the promotion and development of access to lifelong learning and inclusive learning through a framework of credit', is consistent with, and reinforces, SEWOCN's own mission.

14 SEWOCN's Operational Plan for 1999-2000 is structured according to the four aims identified in WAF's Strategic Direction document for 1999-2000:

i to promote the development of lifelong learning and inclusive learning through credit;

ii to continuously improve the services of the OCNs to their members and users;

iii to promote Welsh OCNs;

iv to contribute to the development of a single credit-based framework.

15 These aims are developed through consultation between the three Welsh OCNs' chief officers, and approved by the individual OCNs. WAF's aims are stated broadly in order to accommodate the differing situations of the three different OCNs, and SEWOCN's Operational Plan shows how the OCN will specifically contribute to the achievement of the aims identified in this all-Wales context.

16 It is at the level of the OCN's operational plan, therefore, that any objectives or planned development for Access to HE, as a particular form of provision within SEWOCN, might be formally articulated. The review team noted that no such explicit references exist in the current Operational Plan. The team was unable to identify any mechanism through which the particular needs of Access to HE would be addressed, or its interests protected, either in the strategic or operational planning processes.

Membership, provision and targets

17 SEWOCN has 94 organisations in membership in six broad categories: eight colleges of further education; five higher education institutions; 18 employment and guidance organisations; 44 voluntary organisations working in the field of education; nine schools; and 10 local authorities. One HEI has withdrawn from membership because, according to SEWOCN's Account, it was not comfortable with converting its Access provision to a credit-based format, in line with the rest of the OCN's Access provision. The review team understands that relationships remain cordial, and the team would encourage SEWOCN to pursue discussions aimed at resolving differences, and to work towards the inclusion of all local HEIs.

18 According to SEWOCN's statistical return, there were 10 providers offering Access to HE programmes in 1998-99, eight of which were FEIs. There were 69 programmes available, 54 of which were running.

19 The total number of learners awarded Access certificates over the last three years has varied between 306 and 346, with total registrations showing an increase over the three-year period from 468 to 544. Access programmes now constitute only 3.1 per cent of learner registrations within the total provision of SEWOCN, although they account for a much higher percentage of credits awarded and therefore involve a much higher percentage of OCN activity and contribute to a higher proportion of the OCN's total income.

20 The AVA's statistical return reveals that the majority of learners are either registered unemployed or unwaged, and to this extent the provision is clearly meeting the needs of at least some of the target groups for Access provision. The specific targeting of other under-represented groups, such as ethnic minorities or people with disabilities, was less evident, and the Account acknowledges that even 'where providers have indicated in submissions that they are specially targeting such groups as ethnic minorities, it would be true to say that their recruitment policies do not necessarily reflect this'. 87 per cent of learners are white, with a very small number of ethnic minority registrations, although it must be noted that the AVA return shows 10 per cent of learners whose ethnic origin is classified as 'unknown'. Although a general context for targeting is provided by the aims of WAF and the objectives of SEWOCN, the review team found little evidence of specific targets for Access to HE provision. The team recommends that SEWOCN attempts to address this absence by working with providers and with receiving HEIs in order to identify and target appropriate groups for Access programmes and to support the implementation of such strategies through its major processes of development, validation and review. The Account indicates that SEWOCN also intends to support targeting through both its own marketing and by supporting the marketing efforts of individual colleges, and the team would wish to encourage this.

Developments since the last review

21 The last review of this AVA took place in March 1994. The main developments since that time have flowed from the transition to full Open College Network status, including the conversion of all Access programmes to a credit-based structure. In spite of some difficulties in all providers making the transition effectively, in the opinion of SEWOCN officers the effects of the change have been beneficial, ensuring increased flexibility for learners and a greater integration of Access to HE programmes within the colleges' overall provision.

22 The other contextual change relates to the all-Wales framework within which the AVA operates. At the time of the last review, the Wales Access Unit (WAU) was still in existence. The Unit has since been disbanded and has been replaced by the WAF (see paragraphs 13-15 above). The WAU had been established to develop the three Welsh AVAs and had performed a strong, central co-ordinating role. The WAF has a remit to promote the work of the Welsh OCNs but, unlike the WAU, plays no part in their activities as awarding bodies but rather, provides a forum by which a common strategy on all-Wales issues may be developed. Both the Account and the comments of those whom the review team met confirmed the importance of the all-Wales dimension; equally, it is clear that SEWOCN values highly its links with bodies which also operate outside Wales, such as NOCN and the Standing Conference of Authorised Validating Agencies (SCAVA).

Governance and committee structures

Legal and constitutional status

23 SEWOCN is a company limited by guarantee with charitable status. It is regulated by its Memorandum and Articles of Association, which contain the Objects of the company, a statement on membership, and the constitution and powers of SEWOCN's Council and Board of Directors. The Memorandum and Articles of Association were last amended in November 1999, chiefly to enable SEWOCN to make application to offer awards under licence from the National Open College Network. These changes included the adoption of the current name of the company, amendments to the Objects, and additions to membership categories.

Council

24 All of SEWOCN's 94 members are entitled to a single representative, and a single vote, on the SEWOCN Council. The Council meets on a minimum of two occasions each year, one of which constitutes the formal Annual General Meeting of SEWOCN as a limited company. The responsibilities of the Council are to receive the annual accounts, the annual report and, in alternate years, to elect the Board of Directors. It is also the only body which may authorise changes to the Memorandum and Articles of Association.

25 Although SEWOCN's Account points out that the organisation as a whole is accountable to its Council of members, it identifies the Board of Directors as having ultimate responsibility for SEWOCN's licence as an AVA. In discussing the evolution of the OCN, the Account observes that the role of the Council has gradually diminished with 'few issues of controversy' being raised.

26 This more marginal role of the Council was evident to the review team. It noted that attendance at meetings was poor and, in fact, discounting instances where more than one representative from a member organisation had attended, at least two of the last three meetings had been inquorate. The team noted that, although SEWOCN's Memorandum and Articles of Association state that 'no business shall be conducted at a Council meeting until a quorum is present', in July 1999 a new funding methodology had been approved and adopted at an inquorate Council. The meeting of Council in November 1999 which approved amendments to SEWOCN's Memorandum and Articles of Association was formally quorate although less than 20 per cent of member organisations were represented, and some participants appeared not to be linked with organisations in membership.

27 The review team heard that the Council is now largely perceived as a 'consultative device' rather than playing a substantial role in the governance of the OCN. In many respects, this is understandable given SEWOCN's status as a limited company. However, the Council does formally retain responsibility for some key aspects of business, and remains the only opportunity for the whole membership to contribute to the development and direction of the OCN. On the evidence of Council minutes and discussions held during the course of the review, its functions are being performed in a somewhat minimal fashion, and SEWOCN may wish to consider the role and operation of its Council in order that its position in the governance structure of the OCN is clarified and understood by all members.

Board of Directors

28 The Board of Directors is elected at the Annual General Meeting of the Council for a period of two years and is constituted to ensure cross-sectoral representation. When fully established, it has 11 members: four from further education institutions, two each from higher education institutions and voluntary organisations, and one each from the other three categories of membership. At the time of the review, there was no Director representing schools (a new category of membership introduced in November 1999) and the OCN has apparently had difficulty in securing a nomination for the TEC/employer category for the past two years. The Board has two main sub-committees: the Quality Assurance Committee and a Finance Group.

29 The only terms of reference of the Board are those general duties which are contained in the Memorandum and Articles of Association, which make it responsible for the 'control and management of the company'. The Account expands this minimal remit in describing the Board's ultimate responsibility for the AVA licence. Formally, the Board is required to meet three times a year. In practice, it has exceeded this minimum having met on five or six occasions in 1998 and 1999.

30 In reviewing the minutes of its meetings, and in discussion with current members, it was clear that the Board covered a wide range of operational issues related to the OCN's internal business including budgetary, financial and staffing matters. The review team also noted that it was kept well informed of relevant developments at a national and regional level with, for example, items concerning the NOCN, the WAF, and the Further Education Funding Council for Wales being subject to regular discussion.

31 The review team was less convinced about the Board's role in relation to quality and standards within SEWOCN and its overall responsibility for the AVA licence. Having devolved its powers in these areas to the Quality Assurance Committee, it does not appear to have received and considered that committee's minutes since May 1998 (and has only received two sets of minutes in the period from September 1997). This absence of systematic oversight appears to have affected the Board's ability to exercise appropriate authority over quality processes. As an example, in March 1998 it voiced a concern over annual review procedures and the potential conflicts of interest which might be present in its Quality Assurance Committee. Having agreed that an element of externality should be introduced in the consideration of annual reports, perhaps through the establishment of an external sub-group, no apparent action followed, and the Board does not appear to have pursued the issue (although the appointment of the College Review Manager some eighteen months later may have partially addressed these concerns).

32 In this context, it is difficult to understand how the Board is able to fulfil its remit to be the locus of responsibility for the AVA licence. While it is normal practice to devolve the operation of quality systems to a more specialist sub-committee, this should not diminish the responsibility of the parent body for the effective monitoring of those systems. SEWOCN may wish to consider how its Board of Directors can re-establish its overall control of arrangements for quality assurance and standards.

Quality Assurance Committee

33 In the absence of any clear statement of its title, full terms of reference, membership arrangements and periods of office of members, the review team encountered a number of initial difficulties in understanding the work of the Quality Assurance Committee. Referred to by that title in the Account, a structure diagram in SEWOCN's Providers Handbook cites a Quality Assurance Group, and the minutes of the body use two other appellations, Quality Assurance Meeting and Quality Committee.

34 As regards membership, the Chair of the Committee and its members are nominated by the Board of Directors. The number of members and their period of office do not appear to be set although, as with the Board itself, SEWOCN's intention is to ensure that it is cross-sectoral in representation. Currently the Committee has six members.

35 The remit of the Committee is brief. It is responsible for ensuring that 'SEWOCN's Quality Assurance systems meet the standards laid down under its licences from the National Open College Network and the Quality Assurance Agency' and to be 'the final stage in any appeals arising out of the Accreditation Process'. The review team noted that, unusually, this remit had been adopted and approved by the Quality Assurance Committee itself in October 1998 without any apparent confirmation from its parent body, the Board of Directors. The frequency of the Quality Assurance Committee's meetings is also uncertain. From the evidence of the minutes made available to the review team, it met three times in 1998 but only twice in 1999, with gaps of up to six months between formal meetings.

36 As its remit suggests, the Quality Assurance Committee is chiefly engaged in the review of systems rather than any involvement in those systems at a more detailed level. In this context, it receives a report each year on the operation of the annual review process. However, the Committee does not appear to have a method for the regular review of other major quality systems. In 1998, it did become involved in the essentially officer-led process of NOCN audit through the participation of individual members of the Committee in self-assessment exercises related to the areas of recognition, moderation, award and business systems. The review team noted, however, that reports were received by the Committee only in respect of recognition and moderation. In addition, although this pilot was considered to be successful, in 1999 the Committee decided that any similar participation of its members would not take place until the second or third year of the audit cycle, thus creating an appreciable gap in its involvement.

37 Given that since October 1997 the Committee has twice identified possible weaknesses in the recognition process (once related to the use of unapproved units which had been remedied by retrospective approval, and once related to the quality and credibility of recognition panels), it is surprising that a direct annual consideration of this process is not a feature of the Committee's work. In respect of the moderation process, the Committee's involvement has recently been strengthened by receiving the report of the College Review Manager (CRM), a new appointment made by SEWOCN on a consultancy basis, to provide, amongst other duties, a co-ordinated oversight of moderators' reports.

38 SEWOCN recognises that the introduction of the role of the CRM is a valuable addition to its quality processes, particularly given the comprehensive and insightful first report which the CRM has prepared. In some senses, however, the CRM's appointment is symptomatic of the issues facing the Quality Assurance Committee in fulfilling its remit. The Committee, for example, appears to have played no role in the discussion of the appointment of the CRM, which is surprising since the remit of the post covers areas of responsibility which are crucial to the Committee's work. This use of an external individual also highlights issues related to the Quality Assurance Committee which SEWOCN's Account discusses with a welcome openness. It comments, for example, that 'it is not always easy for an employee to sit on such as the AVA's Quality committee and criticise their own institution', and also considers the 'relationship in a membership body where the actual members are determining policy and carrying out the final stages of the quality process [which] can lead to conflicts of interest'.

39 Since almost all members of the Quality Assurance Committee are also moderators with SEWOCN and many have participated in recognition panels, this issue of impartiality and the independence of judgement of those quality systems is one which remains to be addressed, even with the introduction of the CRM.

40 In the opinion of the review team, SEWOCN should undertake a thorough analysis of the Quality Assurance Committee and its role. This might include a review of its current concise remit; the establishment of robust procedures for the appointment of members; clarity in the length of their periods of office; consistency in the reporting line to the Board of Directors; and a more thorough and systematic involvement with the major quality systems of the OCN. Without such a review, it is difficult to have confidence in the Committee's ability to provide adequate monitoring of the quality and standards of Access programmes and to assure the Board of Directors that appropriate action is being taken to remedy any identified weaknesses.

Access Task Group

41 Formally, the Access Task Group (ATG) is a sub-committee of the Quality Assurance Committee. It was established in 1998 in response to a position paper on developments in Access presented to the Quality Assurance Committee by SEWOCN's Co-ordinator. The paper proposed that a separate body was needed to focus on quality issues in Access provision, a judgement confirmed in the Account which points out that, with the growth of the OCN and the greatly increased amount of non-Access provision for which the OCN is now responsible, there was 'a danger of Access to HE quality arrangements which were being merged in the operation as a whole, being swamped'.

42 The Quality Assurance Committee approved the paper including, in the absence of any further separate record, details of the ATG's remit which was 'overseeing the Quality Assurance arrangements for Access to HE provision within SEWAC' [the then title of the AVA]. A membership of five was decided by the Board, to include representatives from further education providers, higher education receivers, and a current Access moderator.

43 The ATG has undertaken some valuable work in respect of Access provision. The original position paper proposed that it should focus on the annual report and moderation systems. As envisaged, the ATG has considered and made recommendations on annual reports although it has not to date discussed moderation in the same detail. It has also compiled and published a useful Access to HE Code of Practice for SEWOCN providers.

44 However, the review team noted that the ATG had met only three times since its establishment in 1998 (and the third of these meetings had the compilation of the Account for this review as its single agenda item). At the time of the review, no further meeting had been scheduled. Although the concise remit of the ATG may imply that it has a continuing responsibility for quality assurance of Access programmes, the team was told that, in fact, it had no brief for quality in this sense but acted more as a 'sounding board' for a variety of Access-related issues. Members of the ATG also confirmed this view in seeing the Group as a 'troubleshooting' device for Access.

45 In the view of the review team, the ATG represents a lost opportunity for SEWOCN. The original concept of a separate body of expertise to monitor and enhance quality arrangements for Access provision resulted in some initial, valuable contributions, but the infrequency of meetings, and an apparent shift in the perceived remit of the Group, has not allowed this work to be continued. SEWOCN may wish to review the nature of the ATG and the potential of its contribution to the quality of Access provision within the OCN.

Access Tutors Forum

46 SEWOCN also supports an Access Tutors Forum as a device to secure the perspectives of Access practitioners. The Forum is essentially an informal group which elects its own Chair. Its recent meetings have been poorly attended and the Forum has been discussing its own future with the possibility of a conference to stimulate interest for a relaunch. From the evidence of the notes of its meetings, the business of the Forum is focused on the dissemination of information related to Access developments both nationally and within SEWOCN. However, some of its discussions also seem to overlap, duplicate, or possibly replace business which is more properly the remit of the Quality Assurance Committee or the Access Task Group: for example, its most recent meeting in February 2000 discussed and made recommendations on the Access-related issues arising from the College Review Manager's report.

Management, organisational structures and administrative arrangements

Finances

47 SEWOCN had an annual budget in 1999-2000 of £248,000. Differential membership fees are applied which are generally related to the size of the organisation and the frequency of use of SEWOCN's accreditation service. Half its funding comes from the FEIs in membership, with the remainder coming from HEIs, local authorities, voluntary organisations and learner registration fees.

48 In 1998-99, SEWAC (as it then was) achieved a small surplus of income and invested these funds, together with a small amount from its reserves, in the employment of another Development Officer. SEWOCN hopes to generate sufficient income to maintain this post.

Staffing, roles and remits

49 As a company, SEWOCN employs its own team of staff, all but one of whom are permanent, and additional temporary staff are sometimes employed for specific tasks when finances permit. The team consists of a Co-ordinator (who has overall responsibility for the management and operation of the OCN and who is also Company Secretary), a Curriculum and Qualifications Adviser, an Accreditation Systems Officer, a Development Officer, and three further administrative and clerical staff. Each post has a clear job description and remit, and staff appraisals are conducted by the Co-ordinator (or by the Chair of the Board in the case of the Co-ordinator). The current staffing roles were established by a restructuring which took place in 1997, bringing to an end a period when a number of part-time officers had succeeded one another rather rapidly, causing some strain for a relatively small team. The appointment of the College Review Manager was made, on a consultancy basis, during the 1998-99 academic year as an addition to the main staff structure.

Quality assurance functions

50 The key officer in respect of quality assurance is the Co-ordinator, whose job description includes the responsibility to 'develop, implement and review the quality assurance of SEWAC's accreditation functions'. This includes attendance at standard quality assurance exercises such as panel recognition events, as well as attendance at - and the preparation of reports and papers for - meetings of the Quality Assurance Committee, the Board of Directors and the Council. In this sense, the Co-ordinator represents the main link and channel of communication between SEWOCN's business operation and its committee structure.

51 Other officers also carry significant responsibilities for quality assurance. One part of the remit of the Accreditation Systems Officer, for example, is to 'appoint, train and liaise with moderators'. The post-holder also receives and monitors moderators' reports. Such duties are indicative of the reliance which SEWOCN's governance structures place on the work of individual officers and emphasise the importance of the links which need to be maintained if the OCN's committee structure is to be kept fully informed on issues related to quality assurance.

Data management and collection

52 SEWOCN benefits from a Local Area Network comprising a server to which six PCs are linked. A customised management information system is also in place for tracking all programmes, learners and awards. The system appears to serve the data management needs of the OCN, and SEWOCN was able to provide comprehensive statistical information in its 1998-99 annual return to QAA. It also holds a full data record for previous years. The extent to which data is analysed and used by the OCN in order to evaluate the success of its work and for planning, for example in relation to targeting, appeared to be rather less comprehensive. The review team recommends that the OCN takes advantage of its thorough statistical record in the further development of its AVA operations.

Other administrative procedures

53 The review team would wish to comment on one unsatisfactory aspect of the administration of SEWOCN. The general operation of the OCN's committee structure and, in particular, the standard of minutes and their presentation, is an area for improvement. The team encountered a number of difficulties in using committee minutes as a resource for understanding the operation of the OCN. Reporting lines between committees are not consistent, and some proposed actions are either not implemented or, if they are implemented, the action is not routinely reported back to the relevant committee. No customary convention for minuting appears to have been adopted and, on occasion, basic information - such as a list of those members present or the correct title of the committee - is missing or in error. In one case, the team was offered two different sets of minutes for the same meeting of the Quality Assurance Committee. While perhaps adequate for internal purposes, such minutes do not fulfil their other function as a record of the OCN's business for external scrutiny and information. Currently, they do not reflect well on SEWOCN as an organisation and clearly do not reach the same level of quality apparent in other formal documents such as the 'kitemark' specification prepared as part of the validation process. The team would recommend that some staff development be implemented to improve this aspect of the OCN's work.

Communications with providers

54 SEWOCN produces a range of publications to assist its providers, including newsletters, circulars, and more formal bulletins. Bulletins are also used for the purposes of consultation on issues which may result in changes to the OCN's operating processes. Representatives from organisations in membership of SEWOCN were appreciative of such communications and confirmed their value.

The development, validation and evaluation of Access programmes

Development and validation of new provision

55 SEWOCN has a standard process for the development and validation of Access provision (the OCN customarily uses 'validation' as a term to cover its Access work, and 'recognition' for all other provision). In the initial pre-panel stages, a potential provider works collaboratively with a SEWOCN officer (usually, in the case of Access programmes, the Curriculum and Qualifications Adviser, but occasionally with the Co-ordinator) to ensure that draft submissions are accurate and comprehensive as preparation for a formal validation exercise. SEWOCN's view, as expressed in the Account, is that this extensive preparation ensures that no submissions are forwarded for validation until they are 'panel ready' and has therefore not only eliminated the referral or rejection of submissions but also greatly reduced the occurrence of major conditions being set by validation panels. The review team was able to confirm this analysis by its own consideration of a range of validation reports.

56 Validation itself is normally in two stages. Firstly, and as a reflection of OCN practice, individual units of assessment are considered by a subject specialist panel which confirms their level and credit volume. Given the existence of the Welsh CREDIS project, a large number of such units are available to providers and, as SEWOCN acknowledges, much learning is already approved in respect of its level and credit value before being considered as part of an Access programme.

57 The relevance of pre-approved units of assessment and their place within the Access programme as a whole is the responsibility of a second stage 'Kitemarking Panel' which is cross-sectoral in nature and must include at least two members from higher education institutions. This panel focuses on the ability of the proposed programme to prepare learners for higher education and checks that the completion criteria for the award of a particular title are in place. The panel also reviews pre-approved units in respect of their assessment methodologies since the OCN considers these to be particularly important in relation to appropriate preparation for higher education. Following validation, a panel report is prepared and, if conditions have been attached, providers are required to make a satisfactory response before the recognition process is complete.

58 The review team was able to scrutinise a range of programme submissions and validation reports. It also took the opportunity to discuss the validation process with a number of groups during the course of the review. The assistance given by SEWOCN officers during the pre-panel development phase was clearly welcomed and praised by providers. SEWOCN's Providers Handbook gathers together all pro formas which are required for the process in addition to supplying useful notes for the guidance of validation panels. Reports of validations are extremely thorough and include details of panel membership, a record of all modifications and amendments made by the panel, a summary of issues arising from the event, a complete 'kitemark' specification for each separate Access award, and a list of all approved units of assessment. The team considered the 'kitemark' specification to be a particularly good model, implementing the NOCN recommendations of Circular C98/70 to provide a clear and well-structured summary of all necessary details.

59 The review team found, however, one aspect of the validation process to be more problematic. SEWOCN's Account makes it clear that the validation panel is 'empowered to bestow the kitemark' and proposes that a body such as the Quality Assurance Committee, which might have carried final authority in the past, should now be responsible for the overall quality of processes rather than the detail of validation. Setting aside the issue of whether the granting of the 'kitemark' to an individual programme is a matter of detail, the team would acknowledge that the volume of business now conducted by SEWOCN indicates that more efficient processes are required. However, by their nature, validation panels are transient bodies selected for a specific purpose and, except in the case of fortuitous continuity of membership, are unlikely to be able to provide a structured consistency in overseeing the granting of the 'kitemark' to the variety of Access programmes which are approved by SEWOCN. Without promoting a return to a cumbersome and inevitably routine final approval process, the team considers it essential that the validation process should include further verification of consistency before the granting of the 'kitemark' to a programme.

60 Other than the issue of the granting of the 'kitemark' directly by the panel, the review team was able to confirm and commend the care taken in the development of Access programmes and the thorough and robust nature of the validation process.

Monitoring, evaluation and review of programmes

61 SEWOCN requires an annual review from each provider of an Access programme and separate reports where there are multiple and distinct pathways. The procedures for annual review were last amended in 1998-99 and are made available to providers through an Annual Reporting Guide. In summary, moderators submit their reports to SEWOCN whose officers highlight major issues before distributing them to Access Co-ordinators. Co-ordinators are then required to submit an annual report (which can, where appropriate, be a report compiled for an organisation's internal quality assurance purposes). An analysis of the report is passed to the CRM who undertakes a visit to each provider to discuss an action plan. The Quality Assurance Committee receives a report each February which details progress in the annual review cycle. Access programme co-ordinators met by the review team generally confirmed the value of the annual review process and commented that SEWOCN's feedback was usually provided promptly.

62 In its own consideration of a number of annual reports, the review team noted that quality and comprehensiveness varied from provider to provider. However, all reports appeared to be sufficient for the purposes required by SEWOCN. A more pressing issue is that recorded in the minutes of the Quality Assurance Committee for February 1999: 'the problem lies not so much with the standard of reports but with their non-receipt'. In its Account, SEWOCN recognises that the explicit emphasis on robust quality assurance arrangements in the QAA Recognition Scheme for Access to HE requires a rigorous and firm approach to the operation of major systems such as the annual review process. To this end, a draft sanctions paper has recently been prepared by SEWOCN officers for consideration by the Quality Assurance Committee. The review team noted that the firm stance taken in the original draft paper on the non-submission of annual reports had been diluted by the Committee before approval. Given the importance of compliance in the context of licensing, the team would advise that the sanctions paper is re-considered to ensure that it is unequivocal in its position on the measures which will be taken in the case of failure to submit necessary reports.

Assessment, moderation and award of certificates

Student assessment methods

63 Providers are required in their submissions for accreditation to give details of both the overall assessment strategy for the whole Access programme and the assessment criteria for each unit contributing to the programme. The form for the submission for accreditation asks providers to state clearly 'how the criteria for assessment (how the things people know, understand or can do) can be measured in relation to learning outcomes'. The AVA expects assessment strategies to ensure robust and adequate assessment of the units themselves while also incorporating methodologies which prepare learners for HE programmes. The Account indicates that SEWOCN officers are working with providers in the OCN to produce a set of formulae which would link specific assessment methodologies to sets of units. The review team understands that this will be discussed more fully within the Access Tutors Forum and in individual accreditation panels. The team would suggest that such formulae should be established with respect to Access provision in order to ensure a shared understanding and application of assessment processes which will provide suitable preparation for progression to, and within, higher education. Given the AVA's own concerns about the need to develop more specific targeting strategies, it will also be important to ensure that any general formulae which are developed also take account of the appropriateness of particular forms of assessment to the target groups concerned.

Moderation

64 All moderators working on programmes accredited by SEWOCN have contracts with the OCN itself and are appointed for periods of a year at a time. Moderators are drawn from the institutions within membership of SEWOCN and the OCN's officers try to ensure that no moderator is appointed to a programme which might involve a conflict of interest. There are currently no regulations or guidelines concerning the number of times that an individual moderator may be re-appointed. Good practice would suggest, and the review team recommends, that the AVA should consider introducing such regulations. Those moderators who are not full-time employees of a member institution are paid an hourly contractual rate based on the WEA part-time tutors' rate. In its meeting with Access moderators, the team was made aware of some inconsistencies in the practice of payments to moderators and therefore recommends that this issue, which has already been considered by the Access Task Group, be given further consideration to ensure fairness and equity. There is no formal training programme but moderators confirmed that they are all well briefed by SEWOCN officers prior to taking up their appointments.

65 SEWOCN has developed a system of moderation called Framework Moderating, which is based on different types of moderators performing different roles. The basis of the Framework Moderating Model is clearly set out in a document prepared by the Co-ordinator in January 1997 entitled Developing the Framework Moderating Model. It is also reflected in the sections on moderation in the Providers Handbook, last issued in autumn 1999. The system uses Thematic, Access to HE and Convening Moderators who, in theory, work together in contrasting but complementary roles. All are required to work according to SEWOCN's Guiding Principles of Moderation, which are set out in the Providers Handbook and which describe the role of the moderator as that of a 'critical friend' to the programme deliverers. Moderators are asked to see themselves not as inspectors but as supporters of the provision to which they are attached. Sensitivity to the diversity of ethos and philosophy which exists across the providing institutions is required.

66 Within Framework Moderation, thematic moderators are required to consider the consistency of credit value and level across a whole institution for a particular set of units (both Access to HE provision and other OCN programmes) within a particular subject area, or, to be more specific, within a grouping determined by Adset codes. Access to HE moderators are described by SEWOCN as the 'guardians of the kitemark' and have responsibility for looking at the operation of an Access programme, or pathway within a programme, as a whole. In addition to following the principles of moderation and guidelines issued to all moderators, Access to HE moderators are also subject to additional, specific guidelines.

67 The review team was less clear about the role of convening moderators, but to some extent this was because it is in a process of transition. The Account states that convening moderators were originally intended to oversee the overall credit-based provision within a particular providing institution, to ensure college-wide strategies for quality assurance in this area. It was envisaged that this role would be taken on by college quality managers but, in practice, this system never really worked and in the academic year 1998-99 the AVA decided to appoint a single College Review Manager, with the role of providing a report on each provider based upon its own annual report and those from the moderators. Part of the intention behind this appointment was to bring a greater degree of externality to bear upon the quality processes of the providers in relation to their OCN provision. As a result of this appointment, the current role of convening moderators is somewhat uncertain: although their original role no longer belongs to them, some apparently still play a part in the moderating process as 'systems moderators'. The team recommends that, following the evaluation of the effect of the appointment of the CRM, SEWOCN should revise both the Framework Moderation Model itself and all supporting documentation for moderators and providers to ensure that the roles of the different forms of moderator are clearly set out in the changed context.

Moderators' reports

68 SEWOCN amended its reporting process in 1998-99, coinciding with the appointment of the CRM. An annual reporting schedule is published by the OCN and sets out clearly the 10 stages of the reporting cycle, with deadlines given for the contributions of all those involved: moderators, Access co-ordinators, a CRM, accreditation officers, and the providing institutions. Reports for moderators, both thematic and convening, are sent directly to the OCN, where officers then copy reports and send them to providing institutions and the CRM. The providers then send extracts of their relevant course annual reviews to the OCN, whose officers, together with the CRM, prepare a draft report for each institution. This is followed by a visit from the CRM to providing institutions in order to discuss the report and the institutional response to it, both in terms of immediate action and developments planned for the longer term. The CRM then makes an overall report to the OCN, a copy of which is provided to the head of each providing institution. On the basis of this, each provider then sends a revised action plan back to the OCN, whose Quality Committee discusses any issues arising from the report and initiates any necessary actions.

69 In its meeting with Access moderators and the College Review Manager, the review team found unanimous agreement that the introduction of this new post had improved the OCN's quality processes because the CRM is able, unlike the former convening moderators, to take a comparative view of the strengths and weaknesses of provision across the whole OCN. The Quality Committee has stated that it regards the incorporation of the CRM into SEWOCN's moderation procedures as a 'significant step forward' and it has been agreed that, during the current academic year, the CRM will also be asked to include the non-college providers within his review and report. It is not yet clear whether the CRM's role will be extended beyond the current academic year; the team, however, recommends that the OCN gives serious consideration to confirming the role of the CRM within its quality assurance procedures or to adopting another, equally effective, mechanism for ensuring that a comprehensive and sufficiently external view of issues raised through moderation of Access programmes across the OCN can be brought to the attention of the Quality Committee.

70 The review team learnt during its visit that SEWOCN intends to establish a Moderators' Task Group, to provide a forum for the informed consideration of issues arising from the moderation process. The team commends this development.

Procedures for the award of the 'kitemarked' certificate

71 Accreditation officers within providing institutions or organisations are required to convene Thematic Moderating Panels. The panels are attended by the thematic moderator, any manager whose centre is delivering units included within the thematic area and all tutors who are making recommendations for the award of credit. Once thematic moderation has been completed, a form recommending the award of credit to specific learners must be signed by the unit tutor and counter-signed by the thematic moderator to indicate that the programme has been effectively moderated. This form must then be accompanied by a separate form recommending the award of the 'kitemarked' Access certificate to those learners who have completed successfully the criteria for the award. This form must be signed by the Access to HE Moderator for the programme. These forms are then forwarded to the Accreditation Systems Officer in SEWOCN who, having checked that the criteria for the award of a 'kitemarked' certificate have indeed been met, will then issue the Access to HE certificate to learners via providing institutions. The review team was shown an example of the AVA's certificate, which has recently been redesigned to incorporate the QAA's new visual identity for accredited Access provision. The SEWOCN office also issues credit records to each Access student. The team concluded that this process, which includes checks by thematic and convening moderators, and by the responsible officer within the AVA, was thorough and operating successfully.

Learner experience, standards and progression

Learner experience

72 The review team met Access students and HE admissions tutors and found much evidence that students on Access programmes accredited by SEWOCN had found that their courses had prepared them well for higher education. Awareness of the value and significance of the 'kitemark' was high and, in the experience of the students that the team spoke to, their qualification and experiences as mature students were valued by the HEIs to which they progressed, although some reported that other, younger, students sometimes regarded them as being there because of the need for institutions to fill a supposed 'quota' of mature students. Access students reported that their experience had proved that their Access courses had provided a better preparation for HE study than the A levels taken by many of their younger fellow students, a difference which they attributed to the particular emphasis placed on systematic study, research, IT and project management skills in Access provision.

73 Although they are not systematically identified through the recognition process, the review team found evidence of effective mechanisms for student liaison and feedback; in some providers this was achieved by means of staff-student liaison committees, in others through a more informal atmosphere in which tutors encourage students to approach them freely with any views about the course.

74 Despite their positive experiences of the Access courses which they had taken, students that the review team met all felt that the OCN, and individual providers within the network, should give more attention to effective marketing of Access provision. The team noted that the Account indicated that SEWOCN was intending to do more in the future to support the marketing of Access courses, since this issue is often raised in the reports submitted to the OCN and has been identified as a key issue by the CRM; the team would wish to encourage this development, which should be linked to a refocused attention to targeting.

Programme equivalence and standards

75 SEWOCN has a number of procedures in place which are intended to ensure programme equivalence and the maintenance of standards across the range of its Access provision. The introduction of NOCN accreditation requirements is one component. The Framework Moderating System, with its use of Thematic and Convening Moderators, obviously also plays an important role in this respect and this has been strengthened by the appointment of the CRM. Nevertheless, the CRM's report itself draws attention to some inconsistency of provision and argues that, where the same Access course is being offered by the same provider at different centres, there should be some standardisation of provision across the centres unless it can be clearly demonstrated that there is a different target group or different requirements by receiving HEIs.

76 The review team found evidence that HE admissions tutors regard the 'kitemark' awarded by SEWOCN as a sound and reliable currency and that, at least in some areas, the OCN is seen as beginning to exert a stronger standardising influence across diverse provision. The newly-established role of the Moderators' Task Group and enhanced training for moderators should further strengthen the role of moderation in ensuring equivalence across the range of provision.

Progression

77 Over the last three years, 65 per cent of registered learners, on average, have been awarded Access to HE certificates and 79 per cent of those receiving the certificate have progressed to HE. The introduction of credit-based courses has meant that very few learners now leave with no recognition of some achievement. Most learners progress to local HEIs and anecdotal evidence from the HE admissions tutors met by the review team indicates that progression within HE of Access students is good, although there is an absence of hard information in this area. Cardiff University is implementing a new tracking system during the current academic year which will give the OCN better information on outcomes for Access students and SEWOCN intends to ask the other HEIs within the network for similar information. A questionnaire is also being issued to ex-Access students in order to glean more information on their destinations after graduation and the usefulness of their courses in achieving their goals. The team supports both the development of such tracking systems within the OCN and the inclusion of the information which they will provide in the evidence considered as part of the OCN's quality assurance procedures.

78 The nature of the 'kitemark' is explained clearly in material for providers and, during the course of discussions with groups which included HE Access tutors, Access co-ordinators and former Access students, the review team found a high degree of awareness of the value, significance and identity of 'kitemarked' Access provision which, the team was able to conclude, is valued and well-understood by both practitioners and students within SEWOCN's areas of activity.

Conclusions

79 SEWOCN provides an accrediting service to a large number and variety of providers throughout South East Wales. Its rapid growth and development as an OCN, as an independent company, and as an awarding body, have brought about inevitable changes of strategic and operational emphases for the organisation, and accountability to a wider range of members and other bodies. These developments have also introduced a number of challenges to the organisation, some of which it is still in the process of confronting.

80 The Access to HE provision for which the OCN is responsible is delivered primarily by its FE members, and accounts for a small percentage of the total number of the OCN's registered learners. Although this number is relatively small, there is an evident interest in, and enthusiasm for, Access to HE from both OCN officers and Access practitioners. The enthusiasm and commitment of individuals has played a large part in the development of, and continuing interest in, the Access provision in the area, and much goodwill exists to support the aims of the OCN and the students for whom the provision is designed.

81 While this culture has brought undoubted benefits, the reliance on the goodwill of members, which is characteristic of many membership organisations, presents tensions for both the quality assurance and effective governance of the OCN unless it can be balanced by a clear and equal emphasis on members' responsibilities and obligations. SEWOCN has yet to resolve this tension and, in this respect, it is still in the process of realising the full implications of its responsibilities as an awarding body, and in developing and implementing effective quality assurance procedures which are sufficiently robust for the full range of those responsibilities.

82 Some of the OCN's accrediting procedures are thorough and well tested, for example those associated with the initial validation of Access programmes and the award of 'kitemarked' certificates to students. Moderation of Access programmes, however, is in a more developmental stage, although the appointment of the College Review Manager has gone some way to enhance rigour. With moderation in particular, and with other processes in general, weaknesses lie not only in the systems and structures themselves, but also, and more particularly, in the procedures for, and process of, their monitoring. As an example, responsibility for the award of the 'kitemark' to Access programmes has been delegated entirely to recognition panels without ratification by any further body. Regardless of the strengths of individual acts of moderation or the strengths of particular moderators, consistency and rigour are potentially compromised.

83 The recent appointment of the College Review Manager has also drawn attention to other areas for further development, particularly the lack of regular external scrutiny or monitoring within the OCN's processes. These are exacerbated by an apparent reluctance on the part of the OCN to implement the sanctions which are sometimes required for effective quality assurance. The OCN appears to have a lack of confidence in its own authority as an awarding body in applying measures which will enable it to exercise its responsibilities robustly.

84 The assertion of the OCN's authority in relation to providers is constrained in part because the developmental and support functions of the OCN are insufficiently separated from its quality assurance functions, both operationally and within its governance structures. The lack of clarity expressed by committee members about their responsibilities is compounded by an apparent uncertainty as to whether they are acting primarily on behalf of the OCN, or on behalf of their individual member organisations. The resulting tension reduces the effective operation of the OCN's committees. Moreover, the interests of the OCN as a whole are not best served where committee members act in the interests of their member institutions (which are subject to the OCN's quality assurance procedures), rather than the interests of the OCN itself (which is responsible for ensuring that procedures are applied equally rigorously to all members).

85 There is a not unreasonable concern to avoid overburdening individuals who serve the OCN as members of committees and validation panels or as moderators, and who may have significant responsibilities elsewhere. However legitimate this concern, the attempt to minimise the workload results in some aspects of governance and quality assurance being conducted with insufficient rigour. This is most clearly the case with the Quality Assurance Committee but, in general, committees are not always used to greatest effect and, in some instances, their proper responsibilities have been reduced or are not being fulfilled. The Board of Directors, for example, having decided to delegate a range of detailed responsibilities to other groups, might be expected to be exercising a strategic oversight of Access to HE. However, in spite of some evident enthusiasm and commitment, there is a lack of attention to Access at the strategic level.

86 The lack of support from robust governance and quality assurance structures increases the individual responsibility of officers and the dependence on their probity and integrity for the proper management of SEWOCN. In such circumstances, it is especially important that all formal procedures, including the quality of minuting and record keeping, are improved to protect not only the AVA licence and the company within which the AVA licence resides, but also the officers to whom decision-making powers have been delegated.

87 The review team commends SEWOCN for:

i the care taken in the development of Access programmes and the thorough and robust nature of the validation process;

ii the clarity of the 'kitemark' specification produced as part of the validation process;

iii the establishment of a Moderators' Task Group.

Recommendation to the ARLC

88 The review team recommends that SEWOCN be granted a provisional renewal of its AVA licence with re-visit in spring 2001 and conditions to be met by the dates specified below.

Conditions

89 Subject to the approval of the ARLC, SEWOCN's licence is renewed on condition that it:

i locates ultimate responsibility for the award of the 'kitemark' to an Access programme with the AVA itself, in order to meet the need for monitoring and the verification of consistency within the validation process;

(condition to be met by 14 July 2000)

ii undertakes a review of its governance structures as they relate to Access to HE, to include the roles and operation of the Council, Board of Directors and Quality Assurance Committee, and, in particular;

ensures their appropriateness with reference to the proper oversight and control of the development and robust quality assurance arrangements for Access programmes;
clarifies the terms of reference, remits and membership of all committees, including the establishment of robust procedures for the appointment of members and clarity in the length of their periods of office;
defines the hierarchy of structures, accountability lines and reporting mechanisms of committees;
clarifies the roles of OCN officers in relation to the OCN's committee structures;

iii undertakes a review of its quality assurance processes as they relate to Access to HE, and, in particular;

clarifies quality assurance responsibilities within the governance structures;
ensures that the responsibilities for aspects of quality assurance for Access undertaken currrently by the College Review Manager are specified and, whether they remain with the College Review Manager or not, are embedded within the OCN's structures for quality assurance and articulate fully with other parts of the quality assurance processes;

iv reviews its strategic planning process to take account of the planning needs of Access to HE.

(conditions ii, iii and iv to be met by 1 December 2000)

Provisional confirmation of licence allows the AVA to continue to operate under licence but makes provision for the immediate withdrawal of the licence if the conditions are not met in the time allowed and to the satisfaction of QAA. Were it to be necessary for the Agency to withdraw the licence, the AVA would need to put in place procedures for the transfer of registered students to an alternative licensed AVA.

Recommendations

90 The review team recommends that SEWOCN:

i reviews the nature of the Access Task Group and the potential of its contribution to the quality of Access provision within the OCN;

ii develops a cycle of business for its committees to regulate their work and to facilitate effective reporting between committees and ratification of decisions as necessary;

iii implements staff development to improve the quality of the minutes of its committees;

iv re-considers its sanctions paper to ensure that it is unequivocal in its position on the measures which will be taken in the case of failure to submit necessary reports;

v introduces regulations specifying the number of times a moderator can be re-appointed to an Access programme;

vi gives further consideration to the issue of payment of moderators, to ensure fairness and equity;

vii revises the Framework Moderation model and supporting documentation for moderators and providers to ensure that the roles of the different forms of moderators are set out in the changed context;

viii makes use of its own statistics and other data to evaluate the success of Access to HE programmes in identifying and reaching their target groups within the area;

ix collaborates with and supports providers of programmes and receivers of students in the development of further progression possibilities and appropriate targeting strategies.

Open College Network, South East Wales July 2000

Contents