Thames Region Accrediting Consortium Open College Network
October 2001

Foreword

1 The Quality Assurance Agency for Higher Education (QAA) is responsible to the Department for Education and Skills for the recognition of Access to Higher Education courses. QAA exercises this responsibility through a national network of authorised validating agencies (AVAs), which are responsible for the processes of individual course recognition and the award of Access to HE certificates to students on QAA's behalf. QAA has developed a scheme for the licensing and review of the AVAs, the principles and processes of which are described in the QAA Recognition Scheme for Access to Higher Education in England, Wales and Northern Ireland. The Recognition Scheme is regulated and administered by the Access Recognition and Licensing Committee (ARLC), a sub-committee of the QAA Board of Directors.

2 This is a report of a major review of the AVA function of the Thames Region Accrediting Consortium Open College Network (TRAC OCN) undertaken by QAA. The Agency is grateful to TRAC OCN and to those who participated in the review for the willing cooperation provided to the review team.

Aims and objectives

3 The aims of the system of AVA review are:

to provide the basis for an informed judgement by the ARLC about the fitness of the AVA to continue as a licensed agency;
to promote public confidence in Access to HE as a properly regulated and respected route into higher education by assuring:

the quality and adequacy of AVAs' systems and procedures;
the quality, comparability and range of AVAs' operations;
the adequacy and comparability of AVAs' standards for approval, moderation and monitoring of programmes;
consistency across AVAs in the operation of criteria for the granting of the Access to HE award;

to stimulate reflective and self-critical perspectives within AVAs, as an instrument to promote quality enhancement;
to provide an opportunity to identify and disseminate good practice of AVA operations;
to provide a mechanism for ensuring necessary, and encouraging desirable, improvements and developments in AVAs.

4 The objectives of each AVA review are:

to examine, assess and report on:

the development of, and changes in, the AVA since its last review or initial licence, and its plans and targets for the future;
the organisation's continuing viability and robustness and the ways in which the AVA demonstrates sound governance;
the efficiency and effectiveness of the AVA's operational and quality assurance systems;
the range and scope of the AVA's activities, and the appropriateness and value of these activities;
the ways in which the AVA approves and monitors programmes and the ways in which these processes take account of the need for consistency and comparability;
the ways in which the AVA satisfies itself of the adequacy and comparability of standards achieved by students gaining the Access to HE certificate;
the evidence available to indicate the AVA's success in achieving its aims and targets;

to identify and report on:

strengths and good practice in procedures and operations;
areas which would benefit from further development;
areas requiring attention.

Outcomes

5 Following the review of an AVA, the reviewers' report is presented to the ARLC by a member of the review team. The Committee then makes one of six decisions:

unconditional renewal of licence for a specified period;
conditional renewal of licence with conditions to be met by specified date;
provisional renewal of licence with conditions to be met and further review visit by specified date;
suspension of licence until specified conditions are met;
withdrawal of licence for operation as an AVA;
temporary renewal of licence with request for further information by specified date (decision suspended).

The review process

6 The review was conducted in accordance with the process detailed in the QAA Recognition Scheme for Access to Higher Education in England, Wales and Northern Ireland. The preparation for the review included an initial meeting between TRAC OCN representatives and the QAA Assistant Director with responsibility for Access to HE matters to discuss the requirements for the Analytical Account (the Account) and the process of the review; the preparation and submission by TRAC OCN of its Account, together with a selection of supporting documentation; a meeting of the review team to discuss the Account and supporting documentation and to establish a draft programme for the review visit; and negotiations between QAA and TRAC OCN to finalise the programme and other arrangements for the review visit.

7 The review visit took place on 7 and 8 June 2001. The visit to TRAC OCN consisted principally of meetings with representatives of TRAC OCN, including AVA officers and other staff; members of the Executive and Validation and Quality Committees; members of the Access to HE Forum Working Party; moderators for Access to HE programmes; Access to HE practitioners, including programme managers from providing institutions; higher education admissions tutors; and former Access students now studying in HE.

8 The review team consisted of Professor Beverly Sand, Dean of Lifelong Learning, University of Derby, and Ms Gill Evans, Director of the Open College Network North West Midlands. The review was coordinated for QAA by Ms Kath Dentith, Assistant Director (Access), Institutional Review Directorate.

AVA statistics 1999-2000

Providers offering Access to HE programmes	18
Access programmes available	37
Access programmes running	32
Access learner registrations	1,326
Access to HE certificates awarded	786

The AVA context

Background

10 The Thames Region Access Consortium (TRAC) was established in 1989 to validate Access courses, and to be responsible for their quality and overall management. It was approved as an AVA by the Access Courses Recognition Group in early 1990, and this status was confirmed in 1995, following a review by the Higher Education Quality Council. During that year, TRAC was also accepted into full membership of the National Open College Network (NOCN), and entered a period of expansion of its activities, together with consolidation and improvement of its systems and procedures. The Thames Region Accrediting Consortium Open College Network (TRAC OCN) was fully licensed by NOCN in 2000 under new arrangements which have had a considerable impact on the operational, management and governance context in which the AVA operates.

11 TRAC was originally based in Brunel University, then relocated to Uxbridge College in 1993, before moving to offices in Thames Valley University (TVU). In March 2000, TRAC OCN moved to its current premises within TVU where, in addition to increased office space, there was access to other facilities and physical resources which have allowed it to meet increasing demands.

Membership and provision

12 TRAC OCN operates across a wide geographical area across the Thames Valley, from West London to the Newbury area in Berkshire, and membership of the AVA includes the majority of eligible educational institutions within this region. Currently there are 35 organisations in membership, including 15 further education members, three adult and community education providers and four higher education institutions. TRAC OCN notes, however, that this categorisation of organisations for membership purposes may be misleading, as some further education members are also substantial providers of higher education, and one higher education institution delivers considerable further education provision, including Access programmes. Many member institutions deliver programmes on more than one site, and some are members of more than one AVA and, for historical reasons, may have different programmes accredited by neighbouring AVAs. This somewhat complex pattern of provision presents particular challenges for the AVA in terms of effective communications with providers, the management of quality assurance of programmes, and strategies for development.

13 Recent growth in TRAC OCN's Access provision is related to this context, and, in 1999-2000, the number of learner registrations on TRAC OCN Access programmes rose to 1,326 from 1,105 in the previous year, largely because of the closure of a neighbouring AVA and the transfer to TRAC OCN of its Access programmes. A similar rise in Access registrations is anticipated for 2000-01 partly because of the transfer of two providers' programmes to TRAC OCN from another neighbouring AVA. Some growth also relates to the expansion of Access provision within existing providers, however, and, by the end of 2000, 18 providers (including two adult education centres) were approved to run 23 discrete Access programmes, as well as 14 modular programmes which include 86 pathways. Although TRAC OCN has a general aim for overall growth, including some growth in Access provision, the AVA considers that it is unlikely that Access numbers will continue to grow significantly.

Financial position

14 TRAC OCN is funded primarily through annual membership subscriptions, and learner and credit registration. Historically, Access provision has provided TRAC OCN's core business, and it continues to contribute a very significant proportion of its income. In 1999-2000, TRAC OCN had a turnover of £180,638, with a surplus of £38,675. Access provision accounted for 49 per cent of its income and is projected to comprise approximately 43 per cent in the current financial year.

15 TRAC OCN is a financially independent organisation, maintaining its own bank accounts, including a business reserves account, and reporting regularly on its financial position to its Executive Committee, which ensures sufficient cash flow and cash reserves to ensure year-on-year stability. Its financial planning includes three-year income and expenditure projections, which are included within the organisation's published business plan, and finances are currently healthy. Charges are published annually in the form of a discrete leaflet, approved by the Executive Committee, and communicated to members through a variety of means including the TRAC OCN website.

Mission, aims and objectives in relation to Access to HE provision

16 TRAC OCN operates to a three-year strategic plan, and the current plan, covering the period 1998-2001, provides the OCN with nine aims, including 'To promote and ensure the quality of Access programmes as a route to higher education' and 'To facilitate the progression of students to HE by offering information and guidance'. The annual business plan identifies a series of operating statements for the organisation, and many of those listed in the Business Plan 2000-01, while designed for their applicability to the operation of the OCN as a whole would also, if achieved, be likely to lead to improvements in TRAC OCN's operation as an AVA.

17 There are two operating statements which relate specifically to TRAC OCN's AVA responsibilities, and, in addition to the stated intention to 'Fully prepare for QAA review', the Business Plan 2000-01 further identifies, under Strategic Objective 4 - Partnerships, an Operating Statement which expresses an intention to 'establish a regional Access to HE Forum'. This objective has been met, although, at the time of the review, the Access to HE Forum had met only once. Discussions with officers and committee members identified the development of further activities to support widening participation and, in particular, the development of Level 2 provision in a wider range of delivery contexts, as potential additional objectives which may support development in Access for the future.

Current client groups, target groups, targeting policies and strategies

18 TRAC OCN Access programmes have always had strong representation from minority ethnic groups, with between 42 and 45 per cent of learners coming from non-white ethnic groups over the last three years. In 1999-2000, 69 per cent of learners were female and 31 per cent were male, with the largest group of learners in the 26-35 age group. The Business Plan does not identify any specific targeting policies or strategies, although some member institutions are focusing on particular groups relevant to their catchment areas, asylum seekers being one example. The female to male ratio might also suggest some further possibilities for targeting.

19 The Account and discussions with officers identified a need for improvements to monitoring and to the more systematic collection, examination and analysis of data as a priority. In order to secure this, reviewers were informed that TRAC OCN was currently in the process of producing a policy document looking at active strategies for widening participation, and aims to develop a procedure for monitoring how well its members reach groups they identify in their programme submissions. These priorities do not, however, explicitly appear in the current business plan.

Developments since the last review

20 Developments since the last review include the accreditation of several large modular programmes; the full unitisation of all TRAC OCN programmes; the production and updating of handbooks, guidance documents and information for members, moderators, and others; the development of the TRAC OCN website; improved results reporting arrangements; revision to GCSE-equivalent units in mathematics, English and science in line with modifications to GCSE specifications; and the establishment of the Access to HE Forum (see paragraphs 29-31, below). Many of these developments have taken place within the last 18 months and TRAC OCN has identified 1999-2000, in particular, as a year of considerable change, with the appointment of a new Director, new Chairs of key committees, licensing by NOCN, and the relocation of its offices.

Governance and committee structures

Legal status and constitutional position

21 TRAC OCN is an unincorporated association governed by its Constitution and a Memorandum of Agreement with its host institution, TVU. The personnel and payroll services provided by TVU are the subject of a service level agreement and all financial and staffing decisions are independently taken by TRAC OCN, subject only to reassurances about financial viability. The Constitution was recently amended to include a clause barring the representative from the host institution from holding the post of Chair of TRAC OCN. The review team was satisfied that these arrangements provided the AVA with functional independence from the host institution.

Summary of committee structures and remits

22 TRAC OCN is governed by a Governing Council which maintains two committees, the Executive Committee and the Validation and Quality Committee (VQC). The functions of the Council and committees are set out in the Constitution and they meet according to an annual schedule designed to ensure effective operation and reporting. The Governing Council, comprising representation from all members, meets twice a year and is responsible for TRAC OCN's overall mission, policy, and financial probity. The Chair of the Council, elected from its membership, is also Chair of the Executive Committee, which has responsibility for the management of TRAC OCN and its staff. The VQC has responsibility for oversight of the development, recognition, moderation and review of all programmes, including Access programmes, and for all aspects of quality assurance. Although the Executive Committee is required to include representation from 'all educational sectors and interests comprising the membership of the Consortium' and the VQC membership should normally include two members from higher education institutions, there is no specific requirement for representation of Access provision on either committee.

23 While the Executive Committee and the VQC are independent of each other and report separately to the Governing Council, the Constitution allows for the Chair of the VQC to attend meetings of the Executive as a participating member and for the Chair of the Executive Committee to attend the VQC but not participate in its decision-making processes. Reviewers heard that the Chairs of the two committees did not currently make use of this provision, and while it was thought useful for the Chair of the Governing Council to observe both sets of meetings for the purposes of information, there was some lack of clarity as to why this particular arrangement had been included in the constitutional arrangements for TRAC OCN. The review team recommends that the AVA considers whether there is an important purpose to be served in these arrangements and, if so, how they might operate more effectively in practice.

Operation of the committee structure

24 Evidence indicated that the Executive Committee exercised strong leadership, planning and financial control, a forward-looking and robust sense of direction, and a clear awareness not only of its own role in relation to the management of the OCN, but also of current national and local policy issues impacting on TRAC OCN and its future operations. The Committee's Chair plays an active role in staff management and in communicating with member providers to represent the OCN's policies or concerns.

25 By contrast, the VQC has experienced some difficulties in operation this year, due in part to some changes of membership and the resignation of its Chair and uncertainty over a replacement. The Business Plan identifies disappointing attendance at committee meetings as an issue to be addressed, although the Account notes that 'despite concern being expressed in the past about levels of attendance', committee meetings are now 'in the main well attended'. The review team noted from its scrutiny of committees' minutes that, while Executive Committee meetings were, indeed, on the whole well-attended, this was not the case in relation to VQC meetings. The team noted that the Constitution did not provide the VQC with a formal quorum, although the review team was assured that OCN officers were able to advise the VQC as to whether meetings were quorate. Nonetheless, the team considered that the weak record of attendance over the last two years had weakened the effectiveness of its operations, and risked undermining its authority.

26 The VQC appears to oversee a substantial quantity and range of business and the committee's minutes suggest that its business is conducted efficiently. While officer reports keep the committee informed about matters which are pertinent to its remit, the review team was concerned to note a lack of clarity by the Committee about some of its particular powers and duties. As an example, in spite of the Committee's regular attention to the approval of programmes and additional programme units, members appeared unclear of the process or criteria applied to their role in this activity. In addition, although the Committee had discussed 'how to ensure that Moderators' recommendations are followed up by the centres concerned' at a recent meeting, the Committee appeared uncertain of the AVA's own role and authority with regard to this issue. Perhaps because of the lack of continuity in membership, there seemed to be a lack of awareness of the current status or stages of progress of agreed Committee actions, and of the relative responsibilities of officers and the Committee in relation to agreed actions, such as the 'rationalisation' of moderators, or of how decisions taken by the Committee impacted on the formation and implementation of TRAC OCN policy.

27 The review team was also concerned that there were no means, either through the committee's membership or though its remit or standard procedures of ensuring that the particular quality assurance needs and interests of Access to HE were specifically addressed. The consideration of moderation of Access programmes, for example, does not receive any separate attention, and discussion of plans to amend the model of moderation have not, as yet, considered the possible distinctive needs of moderation of Access programmes.

28 The review team consider that the operation of this Committee needs rapid attention, and while there was some recognition within TRAC OCN that the replacement of the outgoing (Acting) Chair was a matter of urgency, reviewers would also wish to point out that if standards are not to be compromised then a more thorough review of the membership and operations of this committee needs to be undertaken. Any revision to the membership, including the Chair, should take account of the considerable expertise in quality assurance matters currently available at senior management level within TRAC OCN's member organisations, and should consider how best to ensure that the particular needs of Access are explicitly provided for.

Access to HE Forum and Working Party

29 The Access to HE Forum is open to representatives from all Access providers and from local HEIs. The review team heard that an Access forum had existed previously but had not been active for some time, and that a decision had been made to revive it, a decision which was incorporated into the Business Plan for 2000-01, following earlier discussions by the Executive Committee about an identified need to 'review kitemarks and the general quality of Access courses'. The OCN describes the Forum's current remit in very broad terms: it is intended to improve communications between further and higher education members and provide a focus for the consideration of all Access to HE issues. However, the Access to HE Forum currently stands outside the OCN's formal committee structure and does not have formal terms of reference or report formally to any committee.

30 The first meeting of the Forum was invited to discuss 'the size and shape of the kitemark', and it identified a need to increase consistency among Access programmes. The principle outcome of this meeting was a decision by the Forum to establish a working party to 'propose a range of principles/guidelines to assure the appropriateness and quality of Access programmes', although the review team noted that the Executive Committee considered that it had mandated the Working Party to undertake this specific piece of work, and that it was expected that the outcomes of the Working Party's work would be reported to the Committee. The Executive Committee also indicated that it had plans to mandate the Working Party to undertake other identified pieces of work, and in these respects, it would appear that the Working Party is acting more in the capacity of a sub-group to the Executive Committee than as a working group of the Access to HE Forum. Thus far, however, it is the VQC which has received the Working Party's draft proposals and been responsible for deciding that a consultation of Access providers should be undertaken.

31 The Access to HE Forum has been identified as key in the future determination of Access, and the Executive Committee referred to the Working Party as a crucial mechanism in relation to the development of Access policy. Given the importance of both the Access to HE Forum and the Working Party to the development of the AVA's policy, strategy and operations, and in the light of the centrality of their current work to Access programmes, the review team was concerned about the uncertainty about the place of these groups within the organisation. The team considered that TRAC OCN should review the reporting and accountability arrangements for their work, and their location outside the formal governance structures, as part of a more general review of the effectiveness of the way in which policy for Access is determined and implemented.

32 Enthusiastic and committed to its task, the Working Party has produced what is currently seen as a key TRAC OCN document, intended to act as an AVA-wide template for all new and revalidated Access provision. The Principles for the Preparation of Access to Higher Education Programmes is an attempt to improve consistency on Access programmes through standardising the range of requirements for the successful completion of Access programmes recognised by TRAC OCN. The paper addresses credit requirements as well as a number of other critical issues in relation to Access programme design, subject and skills development, assessment strategies, and the general nature and purpose of Access programmes.

33 The version of this document seen by the review team was described on its front cover as a 'consultation draft', and its introduction stated that it had been 'ratified by the AVA's Validation and Quality Committee', although, at the time of the review, it had not been presented to the VQC for ratification. The Committee had, however, seen the document and approved its distribution to centres for consultation. Subsequent to the review, the AVA explained that the draft had been prepared with the exact wording which it was intended should appear in the final document, in the anticipation that either the document would be ratified or, in the event that, following the consultation process the VQC did not agree to ratify it, the document woud be withdrawn. In spite of these assurances, the review team considered that the position described created a potential for some uncertainty about the status of this document among those to whom it had been circulated for consultation, and that this could be misleading. In view of its draft status, it was not clear to the team why reference to it had already been included in the Access Guidebook as a note for guidance for Recognition Panels.

34 The review team was informed that the Principles document had been generally well received, and the team also heard a range of largely positive views in relation to this paper and its attempt to achieve an appropriate balance between improving consistency and preserving diversity. The team considered that the move to increased consistency was to be welcomed, but that there were some restrictions in the model proposed for programme design, which could limit the AVA's ability to recognise legitimate models of programmes developed to serve particular needs. In considering the results of the consultation process, the AVA will wish to assure itself about the implications of the proposals for the potential impact on flexibility and innovation, and bear in mind the importance of preserving the responsiveness of the AVA's Access provision to serve the interests of all potential Access students and the broader purpose of widening participation.

Organisational structures

Management and staffing

35 The Business Plan 2001-02 sets out the strategic context, the key strengths and weaknesses of the organisation, and the main operational targets for the year. Achievement of these targets and the work of TRAC OCN is currently facilitated by a very small staff team comprising a Director, an Operations Manager, a 0.5 administrator, and a 0.5 Development Consultant. It has been generally recognised within TRAC OCN that, while extremely cost-effective, the size of the team inhibits future business development, and the Executive Committee has commissioned from the staff team a thorough review of roles and responsibilities and some proposals for future staffing for the organisation, both in terms of specific pressure points during the year, such as the processing of results and issuing of certificates, and for the more general improvement in consistency of operations.

36 With the exception of the Development Consultant, all staff have job descriptions which outline broadly the nature of the tasks allocated to each individual. The review team noted the extensive nature of the Director's job description although, given the size of the team, it was not entirely clear which were intended to be managerial and which were intended to be operational duties. Whatever the intention, reviewers heard that the Director undertook a range of responsibilities and activities which in a larger team might more properly be undertaken by an administrator, freeing the Director to deal with more strategic matters associated with the implementation of policy, planning, business development and the achievement of targets. The Development Consultant, working on a freelance basis, has worked for the AVA since he was appointed to help bring about the organisation's establishment as an OCN, and the document describing his formal remit dates back to that time. However, this document is neither current nor in use.

37 A process of appraisal is planned for this year for all staff (with the exception of the Development Consultant) to ensure that their work meets the aims and targets of the organisation and that they are adequately supported. Reviewers heard on a number of occasions about, and have no reason to doubt, the value to the organisation of the Development Consultant. However, his anomalous position outside the appraisal structure, and the lack of a contract, job description, current brief or remit, work targets, or any other formal designation of duties relating his activities to the Business Plan suggest a lack of formal accountability within the organisation which reviewers consider TRAC OCN needs to address in the interests of organisational effectiveness, clarity and consistency.

38 Administrative procedures are in place for each element of the accreditation process, including general organisation, programme development and approval, moderation, and certification, and different members of the team are responsible for different aspects of each of these stages. Inevitably, in such a small team offering such an extensive customer service, tasks are likely to overlap. In pressured circumstances, this may lead to a lack of clarity about where responsibilities lie for specific activities, and difficulties in ensuring these are properly and consistently carried through to the high quality standards that TRAC OCN has set for itself. Reviewers endorse the Executive Committee's proposals to review and improve the staffing base within TRAC OCN's current robust financial situation.

Data management, collection and analysis

39 NOCN and QAA requirements for the collection and analysis of data have significantly improved this aspect of TRAC OCN's work over the last two years, although it was recognised in both the Business Plan and in discussions with officers and committee members that further improvements could be made, particularly in relation to monitoring targeting and widening participation, to extend the breadth and depth of analysis for the purposes of evaluation and informing planning. It is anticipated that the Access to HE Forum and reinvigorated relationships with HEIs will assist in providing information about the progress and success of Access students in HE.

Communications with providers

40 TRAC OCN provides extensive documentation to its members including standard letters generated through its database, information about its processes and procedures in guides and handbooks, calendars of events, and termly newsletters for institutional contacts, committee representatives, and others. Staff development is provided in a range of areas, especially where TRAC OCN is introducing new procedures or requirements such as the introduction of internal verification systems. An annual report is presented to the Governing Council at its December meeting which contains information (although no analysis) about Access provision, the highlights of which are published in a subsequent newsletter. A recently updated TRAC OCN Access Guidebook has been produced, and a web site has been established, with plans for its development being included in the Business Plan. The TRAC OCN Handbook is available both as an electronic and paper document. The AVA should note that the cover of this document uses an unauthorised version of the Access logo which contravenes the guidelines for its use, and it should be amended at the earliest opportunity.

41 The review team heard widespread commendation and praise for the AVA in its effective communication with providers, the efficient and supportive way in which it dealt with issues and queries, its ability to respond rapidly and to offer timely and appropriate guidance in a responsive and friendly manner, and its attention to detail in all matters pertaining to quality. Excellent customer service is clearly a priority for TRAC OCN and the team recognised this as one of the AVA's key strengths.

Effectiveness and efficiency

42 Compliance with recent NOCN licensing requirements, such as the process of self-assessment, formal production of an annual business plan and record-keeping requirements within a specific format, are likely over time to improve the effectiveness and consistency of TRAC OCN operations. However, some of these arrangements are in the early stages of development, and reviewers concluded that these were not yet embedded across the organisation. For example, even those participants most closely involved with delivering the objectives of the Business Plan appeared unfamiliar with some of its contents or the process for its development. TRAC OCN may wish to consider more effective methods of ensuring that such systems and documents are more widely owned and understood across the organisation. In particular, the review team considered that there was greater potential to involve members in the initial stages of development and planning for the AVA, perhaps through the mechanism of the Access to HE Forum, so that Access providers are able to contribute more explicitly to the setting of objectives and targets.

The development, validation and evaluation of Access programmes

Development and validation of new provision

43 New Access provision is normally initiated by the provider, and TRAC OCN currently undertakes no formal role in the planning of new provision. However, cross-sector membership and active involvement in the AVA, through moderation and the activities of the Access to HE Forum, for example, afford opportunities for collaboration between higher education institutions and Access providers. With the further development of data collection and analysis, the AVA will be in a better position to advise and support providers seeking to develop new Access programmes.

44 The process of programme development is supported by comprehensive documentation, relating both to the particular nature of Access programmes and to the requirements of credit-based, unitised provision. Development work is normally undertaken by the TRAC OCN Development Consultant and may include telephone guidance, as well as visits to the provider and comments on an initial draft. Providers commented favourably on the nature and level of support offered. Programmes are developed to standard criteria and programme documentation is submitted for recognition on a standard submission pro forma.

45 The recognition process for Access programmes requires approval by the VQC following the recommendation of a recognition panel. Panel membership includes, normally, a representative from higher education, the prospective moderator, appropriate subject specialists and a member of the VQC. The review team was informed that this last requirement had not, of late, been consistently met, perhaps, it was suggested, because of the small number of members currently active on the VQC. The review team considered that representation of a member on the VQC was commendable in principle, but that the current position suggested that the OCN needed to reflect on this requirement and amend either its practice or its procedures to ensure consistency between the two.

46 The TRAC OCN Handbook specifies that 'The Chair of the panel will normally be a Development Officer from the Open College Network or a member of the Validation and Quality Committee'. TRAC OCN does not have a member of staff with the designation 'Development Officer' and, in practice, although the Development Consultant is present at recognition panels, it is the Director (or occasionally a member of the Validation and Quality Committee) who chairs recognition panels. The review team would wish to observe that the chairing of recognition panels by a development officer would not be considered good practice if that officer had been involved in the development of the programme concerned. However, given the pressures on the Director's time, the AVA may also wish to consider as a part of its staffing review whether this task (which is not currently listed among the director's duties) needs to be among the responsibilities of the OCN's director, or whether there are appropriately experienced personnel within the AVA's membership who might be able to act as recognition panel Chairs to ensure that the essential principle of the impartiality of recognition panels is not compromised.

47 The TRAC OCN Handbook includes a section entitled Guidance for Members of Recognition Panels. However, members of the Access to HE Forum Working Party expressed some concern that recognition panels had not always applied consistent criteria across all programmes and that further guidance would be useful, particularly with regard to the standardisation of programme documentation and to the relationship between credit and volume of work, and the QAA-recognition specification. The composite moderation report for 1999-2000 also suggests that 'recognition panels should be more rigorous in this regard'. The Principles for the Preparation of Access to Higher Education Programmes (see paragraph 32, above) was designed to help address some of the tensions around diversity and standardisation.

Validation of revised provision

48 An issue of particular concern to the review team is the process by which revisions may be made to Access programmes, and, in particular, the process by which the QAA-recognition specification may be amended. The TRAC OCN Handbook and the Access Guidebook contain guidance on revision to programmes, which makes a distinction between the process for adding units and pathways, requiring reference to a recognition panel, and the process for modifying unit details, which is referred to a recognition panel 'if necessary'. The Chair of the VQC is also empowered to approve minor changes to programmes on the basis of advice from the OCN's officers and the moderator of the programme concerned, but there was some uncertainty as to the criteria for approval of amendments. While it appeared that the addition of units and pathways conformed to this procedure, the team was not able, from its interviews, to confirm a consistent interpretation of the criteria by which modifications were judged to require reference to a panel, or exactly where the responsibility lay for making such judgements. An examination of Access programme documentation and associated correspondence suggests that, on occasions, an inappropriate responsibility for making such decisions may rest with officers, particularly with respect to amending credit targets. Where this affects the QAA-recognition specification for an Access programme, it poses a risk to the integrity of the award, the quality process, and potentially undermines the authority of the AVA. The AVA should address this matter as a matter of priority to assure that changes to the programme specifications of QAA-recognised programmes can only be approved through a process which accords with clearly stated and consistently applied procedures and criteria.

Monitoring, evaluation and review of programmes

49 TRAC OCN has developed a five-year cycle of evaluation and reporting for Access programmes. This constitutes a requirement on providers for a full evaluation report in years one and four, and a short, summary report in year five, in addition to regular action plans. The evaluation cycle is designed to complement the external moderation process. The TRAC OCN Handbook identifies the role of the VQC in receiving action plans prepared by institutions resulting from this process. In practice, the evaluation cycle does not appear to be operating with any consistency and the boundary between evaluation by providers and monitoring by external moderators is unclear. A review of programme files revealed inconsistency in the provision of evaluation reports. In addition, the VQC appeared unaware of its role in the process, and although programme managers were generally familiar with these responsibilities, the opinion was expressed that TRAC OCN and college quality cycles were not operating in tandem in relation to programme evaluation. Those reports that are received by the OCN are not currently dealt with in any systematic way and it was acknowledged that this was an area of the AVA's procedures that needed development. The review team considered that this system had the potential to provide a valuable source of monitoring, evaluation and reflection, particularly in the context of QAA's requirements for annual AVA reports. In developing this process, the AVA may wish to consider how its own requirements could complement providers' existing internal systems, and how this information can best be monitored and utilised.

Assessment, moderation and award of certificates

Guidance on assessment methods and standards of student achievement

50 In addition to published documentation, guidance is given during the development process about assessment, and there is some opportunity for sharing good practice. TRAC OCN prides itself on its unit template which specifically relates assessment tasks to designated learning outcomes, and moderators commented on the helpful nature of this requirement to their task of sampling. TRAC OCN is to be commended on its recognition of the central importance of assessment to the quality of Access programmes and preparedness of learners for higher education.

51 A formal process of standardisation currently exists only for GCSE-equivalent units. Consistency of practice is supported by moderator training and other forums, and also where moderators hold a 'portfolio' of programmes. Nonetheless, the review team recommends that a more rigorous and systematic approach be taken to ensuring consistency of standards of student achievement. The AVA may wish to consider, for example, the introduction of benchmarking exercises in relation to its Access programmes as a means of improving consistency.

Selection, appointment and training of moderators

52 Access moderators are drawn from both the higher and further education sectors, and are appointed by, and contracted to, the AVA. From a database of contacts, and with reference to a checklist of specified criteria, the Operations Manager makes a recommendation for appointment to the Director. There appeared to be no recognition of the role of the VQC in formally appointing moderators, as identified in the Constitution, either from members of the VQC itself or from other participants in the review with whom the review team discussed this matter. The team considers that the AVA should address this matter to ensure that its practice is consistent with its constitutional obligations.

53 Attendance at a training session is a requirement for new moderators, and annual meetings also provide opportunities for further moderator development. Moderators commented on their appreciation of the chance to share both issues of concern, and good practice, and considered that TRAC OCN provided good support mechanisms.

The moderation process

54 The moderation process is the key mechanism through which the AVA assures itself of the quality of Access programmes and the standards of learner achievement. Clear guidance is given by TRAC OCN in its Guidelines on Moderation and the review team was confident, from discussions with moderators, that this process was well understood. For Access programmes, three moderation visits are normally made each year. The Guidelines describe the responsibilities of moderators at initial and interim visits, as well as at the final verification visit, and the AVA provides a standardised aide-memoire to assist moderators to address the requirements of interim visits and to keep a record of information that might need to be integrated into their final reports. In addition, moderators notify the AVA by standard postcards when a visit has taken place, and may use this card to signal to TRAC OCN any issues of immediate import. An example of good practice by some moderators is the custom of undertaking some sampling activity early in the academic cycle, so that discrepancies can be identified and resolved, although this is not currently a standard TRAC OCN requirement. Following the final visit, a summative report is required by TRAC OCN, which is structured to a prescribed format and includes a section for the moderator to verify his/her satisfaction that internal verification has been undertaken. A list of recommendations for action is designed to be left with the centre for immediate attention, and the full report is forwarded to the AVA and, from there, copied to a senior manager in the provider institution.

55 Moderators are generally clear about their role in following up recommendations made in their reports. However, there is a lack of clarity among those responsible for the oversight of moderation on the VQC about where the locus of authority lies and the procedures to be followed in circumstances where providers persistently fail to address issues which have been raised. This would include the identification of circumstances leading to withdrawal of validation. The review team considers that this matter requires immediate attention.

56 TRAC OCN operates a model of moderation which appears to be in transition between a subject specialist model and one where the moderator becomes, to a great extent, an arbiter of process, heavily reliant on the successful implementation of an internal verification system. While both programme managers and moderators who met the review team expressed broad confidence in the system, the team noted that, in the composite moderation report prepared for the VQC for 1999-2000, there were a number of comments on specific moderation reports which indicated that some aspects of internal verification continued to cause concern, and that moderators may be asked to take on trust that the college system is working, with minimal evidence provided to support a tutor's assertion. The TRAC Handbook gives guidance on minimum criteria for the successful implementation of an internal verification system. The AVA may wish to supplement this by exploring individual providers' practice more fully in order to disseminate models of good practice in relation to internal verification.

57 A further area of development concerns the moderation of modular Access programmes consisting of several pathways, a model which the Account describes as being increasingly popular. A single moderator commonly takes responsibility for more than one pathway within a programme, and this will support consistency. In addition, each pathway moderator should attend the final moderation meeting. Nevertheless, the review team found difficulty in understanding how the AVA was able to come to an overall judgement of the quality or effectiveness of the programme as a coherent whole, where there is no mechanism for coordination or overall analysis. The potential introduction of 'lead moderators', and a concern to rationalise the overall number of moderators has been identified recently by the VQC and a discussion paper on lead moderation is to be submitted to the VQC soon. The review team recommends that, in its review of the moderation process, TRAC OCN reflects on the specific needs of Access to identify what remains fit for purpose and what will assure quality and consistency in the context both of the AVA's strategic aims, and of developing models of course delivery and management.

58 The VQC's responsibility for 'all aspects of moderation', as identified in the Constitution, is exercised chiefly through the Committee's annual receipt of a detailed composite report on moderation, compiled by the Development Consultant. Part One of the report provides a brief resume of findings for each provider organisation, across all programmes, including Access to HE. Comments are helpful, but necessarily either selective, or very general, in that they apply to provision overall. Part Two of the report highlights issues which have general applicability for TRAC OCN, together with recommendations for improvement of the process, and this Part is also forwarded to moderators. Given the current high proportion of Access programmes among the OCN's recognised programmes, comments on Access programmes feature quite prominently. There is, however, no section within the report specifically dedicated to Access and no specific occasion during the monitoring process when issues relating to moderation of Access programmes are separately reviewed or assessed. The Committee does not appear routinely to receive or sample individual moderation reports, though it can request that these be provided. In the light of issues raised by moderation, the Committee may wish to include some opportunity for sampling in its annual monitoring cycle.

Procedures for the award of the Access certificate

59 It is the responsibility of the Operations Manager to check that the moderator has confirmed by signature that the learner's work is authentic; that samples have been inspected according to TRAC OCN regulations; and that the names of learners who are eligible for certificates have been recorded. The review team was confident of the security of this process. However, given the present shortage of resource on the administration team, there may be undue pressure on staff to ensure that the award is checked by two members of staff, as outlined in the Account, and the Executive Committee may wish to take this into account in its review of staffing.

Learner experience, standards and progression

Nature of learner experience and progression to HE

60 The HE admissions staff and former Access students met by the review team considered that the nature of preparation for the academic experience was appropriate, and that, on the whole, students from Access programmes were well prepared for higher education. Students were considered to be particularly strong in transferable study skills and in preparedness for coursework and project work, though, in some instances, less well prepared in examinations and other forms of timed test.

61 While admissions staff expressed familiarity with the QAA-recognised Access certificate, and some enthusiasm for receiving successful Access students on to their courses, the number of AVAs in and around the area in which TRAC OCN operates is reflected in the range of offers routinely being made to Access students. Some of these offers would not be appropriate for students progressing from Access courses recognised by TRAC OCN, with, for example, offers from some institutions or individual departments being expressed in terms that relate to non credit-based systems. At present, most liaison about Access happens on an individual basis, with higher education staff communicating with different Access providers. Although admissions staff do receive some information updates from TRAC OCN, the review team would support the view expressed in the Account that there is a need for continuing dialogue with higher education institutions to minimise the potential for misunderstanding and to improve progression opportunities for students.

62 A number of admissions staff met by the review team reported that a tracking system was in place for following the progress of Access students as a group, either on an institutional or departmental basis. As a cohort, their performance was reported as being at least as good as that of their peers, and some also reported a better continuation rate among Access students than among other groups. However, data is relayed to TRAC OCN only on an ad hoc basis and the Account identifies as an aim the more systematic collection and analysis of such data in the future. The review team considered that this development could usefully inform the planning process for the AVA.

Conclusions

63 The traditional focus for TRAC OCN's work has been the Access provision for which the AVA was originally established, and this still provides the OCN with its core business and, to a large extent, its financial security. Bordering on a number of other AVAs and OCNs, the OCN is concerned not only to maintain but also to increase the amount and scope of its activity. Having recently devoted considerable time and attention to preparing for the processes of NOCN licensing and audit and QAA review, and to enhancing and documenting its structures and procedures, it now intends to turn its attention to developing a more entrepreneurial approach to its work in order to promote and market the OCN's services more actively. To date, its collection of data has not provided more than has been required to fulfil its licensing obligations at a minimal level, and has not yet led to any substantial analysis or review of its operation. This is a clear area for development if the AVA is to be able to evaluate the achievement of its objectives in relation to Access and widening participation more generally. It therefore has mission-related, as well as business-related, reasons to seek to extend its services to new providers and contexts, and to improve its monitoring activities in order to provide a more intelligence-driven focus for development.

64 TRAC OCN provides a responsive and professional service for its members from a small but efficient staff team. Providers receive both individual support, in relation to development and quality assurance matters, and are appreciative of a rapid response to requests, which also gives careful attention to detail. While the staff team works effectively to meet the requirements of the AVA's responsibilities, it is clear that staff resources have been stretched and have limited the kind of strategic development which the OCN is now planning, as well as placing limits on other activities. The team has had to work flexibly to ensure all tasks are completed, but this may not always have made the best use of staff abilities and experience. The OCN is seeking to address this through undertaking a review of staffing, and will need to give some careful thought to allocation of current responsibilities and how these are monitored to ensure accountability of all staff and a clear relationship between staff responsibilities and the operational targets identified within the OCN's business plan.

65 The AVA's quality assurance procedures relating to the recognition and monitoring of Access programmes are, in the main, thorough, appropriate and clearly stated within the AVA's published documentation. Some of these documents have been relatively recently produced or updated, with the result that some procedures described are not yet fully embedded, and their effectiveness may not yet have been evaluated. Quality assurance procedures are generally efficiently managed by officers, and practitioners and moderators involved in their implementation are familiar with them and have confidence in them. However, the oversight of this activity within the AVA's committee structures shows some weaknesses. While the Executive Committee provides appropriate direction and leadership for the AVA, as well as responsible management of the AVA's affairs, it does not have any responsibilities in relation to the AVA's quality assurance activities and some attention to the VQC's membership and operation is needed if the AVA is to be able to exercise its responsibilities in relation to quality assurance in full.

66 The chief vehicle for development of Access within the AVA is the recently re-established Access to HE Forum and Working Party. These have a potentially valuable role for practitioners in cross-sector communication and networking, and important work on consistency has sprung from the Forum's first meeting. However, the Forum stands outside the AVA's formal committee structure, and there is currently no clear identification of how the outcomes of its deliberations are integrated into TRAC OCN's wider strategic development or policy. With no specific place for Access elsewhere in the formal structures, there is a risk that the Forum could come to carry an undue weight of expectation without clear direction or reporting responsibilities. As the OCN develops its non-Access activities, it will become increasingly important that it has a clear strategy for how it will preserve the particular attention needed for Access provision and identify points in its committee structures and quality assurance procedures where a clear distinction may need to be drawn, so that it can ensure that it retains the specialist experience and attention required for the effective quality assurance of Access programmes.

67 The review team commends TRAC OCN for:

its customer service;
its recognition of the central importance of assessment to the quality of Access programmes, and the design of unit templates to assist the rigour of the assessment process and consistency of results.

Recommendation to the ARLC

68 The review team recommends that TRAC OCN be granted a conditional renewal of its AVA licence, with conditions to be met by the dates specified below.

Conditions

69 Subject to the approval of the ARLC, TRAC OCN's AVA licence is renewed on condition that it:

reviews the operation of its committee structure, in the context of a consideration of how policy and decisions relating to Access are determined and implemented and, in particular:

considers how best to ensure that Access interests are represented and secured within the formal committee structure;
clarifies the specific duties and responsibilities of the Validation and Quality Committee, develops its membership specification to include experience of the operation of quality assurance procedures, and specifies an adequate quorum to ensure its effective operation;
clarifies the role and status of the Access to HE Forum and its relationship and reporting line to the AVA's formal committee structure;
clarifies the status of any working groups that may be established, and clarifies the reporting lines for any such groups;

ensures that the review of staffing addresses appropriateness of staff roles and ensures that individual staff targets are related to the TRAC OCN Business Plan, to support effectiveness and efficiency of operation;
ensures that its practice in relation to moderator appointments is consistent with its formally documented procedures;
clarifies and implements the role of TRAC OCN as the locus of authority for ensuring that moderators' recommendations are followed up and action taken as necessary;
clarifies and implements the procedure for amendments to an Access programme's QAA recognition specification;
develops a clear process for the receipt of, and response to, programme evaluation reports, and clarifies the relationship of this process to other monitoring processes;
ensures that its use of the Access logo conforms to the guidelines provided by QAA.

(conditions to be met by 31 December 2001)

Recommendations

70 The review team recommends that TRAC OCN:

formalises its plans to develop its collection and analysis of data, and develops mechanisms to make use of data to inform the AVA's planning process;
develops its strategic planning process to involve Access providers in the initial stages of the development of objectives and targets;
within its development of TRAC OCN's moderation process as a whole, takes account of the specific requirements of the effective moderation of Access programmes;
develops coordination of pathway moderation;
considers providing an opportunity for sampling of moderators' reports by the Validation and Quality Committee;
develops the process of programme evaluation to take account of providers' internal systems;
identifies and disseminates good practice models for internal verification;
considers whether it wishes to retain the Constitution's provision for the Chairs of committees to have attendance and/or voting rights on other committees and, if so, to consider how to make effective use of this provision;
develops its dialogue with local higher education institutions to enhance understanding about the nature of the AVA's awards and to improve progression opportunities for Access students.

Thames Region Accrediting Consortium Open College Network October 2001

Contents