Open College of the South West
August 1999

Foreword

1 The Quality Assurance Agency for Higher Education (QAA) is responsible to the Department for Education and Employment (DfEE) for the recognition of Access to Higher Education courses. QAA exercises this responsibility through a national network of authorised validating agencies (AVAs), which are responsible for the processes of individual course recognition and the award of 'kitemarked' Access to HE certificates to students on QAA's behalf. QAA has developed a scheme for the licensing and review of the AVAs, the principles and processes of which are described in the QAA Recognition Scheme for Access to Higher Education in England, Wales and Northern Ireland. The Recognition Scheme is regulated and administered by the Access Recognition and Licensing Committee (ARLC), a sub-committee of the QAA Board of Directors.

2 This is a report of a major review of the AVA function of the Open College Network of the South West (OCNSW) undertaken by QAA. The Agency is grateful to OCNSW and to those who participated in the review for the willing co-operation provided to the review team.

Aims and objectives

3 The aims of the system of AVA review are:

i to provide the basis for an informed judgement by the ARLC about the fitness of the AVA to continue as a licensed agency;

ii to promote public confidence in Access to HE as a properly regulated and respected route into higher education by assuring

the quality and adequacy of AVAs' systems and procedures
the quality, comparability and range of AVAs' operations
the adequacy and comparability of AVAs' standards for approval, moderation and monitoring of programmes
consistency across AVAs in the operation of criteria for the granting of the Access to HE award
the sufficiency and consistency of standards achieved by students granted the Access to HE award;

iii to stimulate reflective and self-critical perspectives within AVAs, as an instrument to promote quality enhancement;

iv to provide an opportunity to identify and disseminate good practice of AVA operations;

v to provide a mechanism for ensuring necessary, and encouraging desirable, improvements and developments in AVAs.

4 The objectives of each AVA review are:

i to examine, assess and report on

the development of and changes in the AVA since its last review or initial licence, and its plans and targets for the future
the organisation's continuing viability and robustness and the ways in which the AVA demonstrates sound governance
the efficiency and effectiveness of the AVA's operational and quality assurance systems
the range and scope of the AVA's activities, and the appropriateness and value of these activities
the ways in which the AVA approves and monitors programmes and the ways in which these processes take account of the need for consistency and comparability
the ways in which the AVA satisfies itself of the adequacy and comparability of standards achieved by students gaining the 'kitemarked' Access award
the evidence available to indicate the AVA's success in achieving its aims and targets;

ii to identify and report on

strengths and good practice in procedures and operations
areas which would benefit from further development
areas requiring attention.

Outcomes

5 Following the review of an AVA, the reviewers' report is presented to the ARLC by a member of the review team. The Committee then makes one of the six following decisions:

i unconditional confirmation of renewal of licence for a specified period;

ii conditional confirmation of licence with conditions to be met by specified date;

iii provisional confirmation of licence with conditions to be met and further review visit by specified date;

iv suspension of licence until specified conditions are met;

v withdrawal of licence for operation as an AVA;

vi temporary renewal of licence with request for further information by specified date (decision suspended).

The review process

6 The review was conducted in accordance with the process detailed in the QAA Recognition Scheme for Access to Higher Education in England, Wales and Northern Ireland and approved by the ARLC. The preparation for the review of OCNSW included a preliminary visit to QAA offices by the Co-ordinator of OCNSW to discuss the requirements for the Critical Report and the process of the review; the preparation and submission by OCNSW of its Critical Report, together with a selection of supporting documentation; a visit by the QAA Assistant Director to OCNSW to discuss the arrangements for the review visit and to identify further documents for the review team; a meeting of the review team to discuss the Critical Report and supporting documentation and to establish a draft programme for the review visit; and negotiations between the QAA and the OCN to finalise the programme and other arrangements for the review visit.

7 The review visit took place on 20 and 21 May 1999. The visit to OCNSW by the review team consisted principally of meetings with representatives of the OCN including: AVA officers; Chairs of committees; members of the Quality Committee; Access to HE moderators; Access to HE programme co-ordinators; former students from Access to HE programmes; and representatives of the University of Plymouth.

8 The review team consisted of Dr Peter Easy, Assistant Director (Academic), Cheltenham and Gloucester College of Higher Education and Ms Miriam Griffiths, Director, The Open University in Wales. The review was co-ordinated for QAA by Ms Kath Dentith, Assistant Director (Access), Institutional Review Directorate.

The AVA context

General background

9 The development of the Open College Network South West (OCNSW) began in the late 1980s through a joint initiative between the University of Exeter and the former Plymouth Polytechnic to provide alternative routes for mature student entry to higher education. An initial body, the South West Access Validating Agency (SWAVA) was recognised as an AVA in 1990. At the same time, the South West Open College Federation (SWOCF) was also established. The two bodies merged in September 1993 to form the South West Access Federation (SWAF) which assumed the AVA licence and, in 1994, also gained full membership of the National Open College Network. In 1998, SWAF adopted the new title of OCNSW.

10 OCNSW is based in Plymouth and operates across Cornwall, Devon, Dorset, parts of Somerset and the Channel Islands. It thus serves a very large, mainly rural area and, as a result, distance and travel factors influence the nature and scope of its provision as well as the day-to-day work of its members and officers. OCNSW is aware of the influence of geography on its operations and communications and, to some extent, takes these into account. However, for practical reasons, most activities and meetings take place in Plymouth.

Staffing and physical environment

11 OCNSW has three officers and three administrative or secretarial staff, all of whom are on fixed-term contracts ranging from one to three years. The staff comprise a Co-ordinator, a Development and Moderation Officer, a Development Worker with a particular brief for Access, an Operations Manager, an Accreditation Officer and a Clerical Assistant. The AVA is also in the process of creating a fractional post to cover revalidation issues. All staff are employed by the University of Plymouth and work closely with staff in the University's Department of Academic Partnerships.

12 The University also provides office accommodation and other services including finance, payroll and personnel. At the time of review, OCNSW's compact office space was being increased although this will entail a split-site operation. OCNSW appears to operate comfortably within its annual budget and has a prudent approach to contingency funds. Its budget and operating statement are approved by its Council and monitored by its Executive Committee. The University of Plymouth assists with production of budget reports and forecasting mechanisms.

Membership, provision, and target groups

13 OCNSW's 78 member organisations are drawn from a range of sectors including further education, higher education, local authorities, and community, voluntary, and private organisations. All further education and Access to Higher Education providers in the area are members. At present, there are five higher education institutions in membership from a possible total of seven in the region: the only pre-1992 university, Exeter, has recently withdrawn from membership; and the specialist Dartington College is not a member but has some links with the AVA. The review team found the membership range to be appropriate and OCNSW active in monitoring its membership categories. However, given the limited number of higher education institutions in the area, OCNSW may wish to investigate fully the reasons for the withdrawal from membership of the University of Exeter.

14 OCNSW currently has over 400 approved programmes, of which 36 are Access to Higher Education programmes, representing about 9 per cent of its provision. However, of its overall total of some 6,000 learners, 1,200 were registered on Access programmes in 1997-98, representing 20 per cent of learner registrations. Of those 1,200 initially registered students, 580 were awarded Access certificates on completion of their programmes. Access programmes are delivered in 21 centres across the region. The development of large, multi-modular programmes, and the move away from discrete courses, has meant that the overall number of programmes has decreased in the last few years, although the range of opportunities and options for learners has been expanded.

15 By the end of the current academic year, all Access programmes will be credit-based. The process of converting programmes to a credit basis has been prolonged but has been mostly been undertaken as part of the periodic review of programmes with appropriate attention being paid to staff development issues. In the main, credit-based programmes are viewed positively by providers, although OCNSW is aware that there is continuing development work to do regarding some providers' and practitioners' approach to credit-based provision.

16 OCNSW undertakes analyses of learners on Access programmes with regard to socio-economic and demographic factors, and makes available to providers information and statistics to allow them to focus their recruitment on appropriate groups. Data made available by OCNSW revealed that the number and nature of learners and programmes were not always easily gathered and interpreted. Difficulties have been experienced with the specialist AREV database used to record and analyse data, and OCNSW recognises that it needs to enhance its capacity to collect accurate data and to link it to a strategic approach in respect of targeting policies. Targeting is currently perceived as primarily the responsibility of member organisations and there is no evidence of a deliberate strategic approach to following through any identification of low or non-participant groups with targeting policies or advice. As an example, men are not well represented overall but do not feature as a possible target group in the OCNSW Equal Opportunities Guidelines which accompany submission documentation. The review team noted that although OCNSW's Strategic Plan includes the encouragement of developments of programmes for validation in curriculum areas that are under-represented, there is no equivalent strategic priority to support the development of programmes in areas or in forms of delivery that might attract particular groups that are under-represented.

Mission in relation to Access to Higher Education and the identity of the 'kitemark'

17 OCNSW's mission statement does not include explicit reference to Access to Higher Education programmes. However, much of its practice demonstrates that its general aims and principles are consonant with those expressed in the Recognition Framework. OCNSW maintains a Development Officer with a specific remit in relation to Access and, in its documentation for providers and practitioners, it distinguishes and addresses those characteristics and requirements which are specific to Access programmes.

18 The nature of the 'kitemark' is explained in clear terms in OCNSW's general literature. During the course of the review, discussions with a number of groups, including former Access students and Access Co-ordinators, demonstrated a positive understanding of the nature and requirements of the 'kitemark'. The review team was able to conclude that Access is adequately represented and defined within OCNSW's aims and objectives, and that the 'kitemark' is understood both in theory and practice.

Developments since the last review and current AVA practice

19 As SWAVA, the AVA was amongst the first to be reviewed by the former Higher Education Quality Council in 1992. The recommendations made in 1992 were satisfactorily considered by OCNSW in its Critical Report prepared for this review. However, with the seven intervening years, and the nature and breadth of changes and developments in Access generally, the review team accepted that this aspect of the review process assumed less significance than would be the case when more regular monitoring became established under the QAA's new Recognition Framework.

20 In general, OCNSW has responded appropriately to the more recent key developments in Access, for example, the growth of unitisation and credit-based programmes. It has also been successful in developing and modifying its practices in response to the implications of the merger of the AVA and OCN from which it was created.

Management, organisational structures and administrative arrangements

Roles and remits

21 Although peaks in workloads and geographical factors have created some difficulties, the staffing structure of OCNSW appears to be appropriate given the size of the operation and the volume of its work. Each post has a clear job description and remit and the general deployment of staff is kept under review. Currently, for example, there are some changes envisaged in the methods for dealing with the large amount of work generated by the moderation process.

22 The review team noted that OCNSW had recently experienced a relatively high degree of staff turnover, and that all OCNSW staff were employed on fixed-term contracts. Although there may be no causal connection between these two observations, the common use of fixed-term contracts lends a degree of impermanence to the professional staffing base. The team was told that budget uncertainties and the need for flexibility were the chief reasons for this staffing policy, which was also dictated to some extent by the personnel practices of the University of Plymouth. Whilst accepting these reasons, the team would recommend that OCNSW, through its Council, reviews this position to consider whether it would be advantageous to employ some of its core staff on a more permanent basis.

Quality assurance functions

23 Key staff of OCNSW, including the Co-ordinator and the Development and Moderation Officer, carry significant responsibilities for quality assurance. The Co-ordinator acts as the principal link between the day-to-day operation of the AVA and its committee structures and, in particular, works with the Council and its Executive Committee to ensure that these bodies are fully informed on the achievement of operational and financial targets, and can monitor the satisfactory functioning of key processes.

24 The Development and Moderation Officer plays a significant role in relation to the systems for validation, moderation and review and is thus closely involved in the work of the Quality Committee. The methods adopted by the Quality Committee to undertake its duties (which are further discussed in paragraph 32 below) place considerable responsibility on the Officer to ensure that issues arising from the validation, moderation and annual review processes are brought to the attention of the Committee.

Communications with providers

25 The review team noted the use of the OCNSW Annual Report as a primary method of communication with members and providers. The Report presents factual information on OCNSW staffing, membership and resources. It also includes an analysis of programme development and accreditation, a brief report on the work of each OCNSW committee, and discusses the key changes and developments which have occurred regionally and nationally during the year under report. A newsletter is also circulated to members, which keeps them informed of particular issues on a regular basis. The team noted the usefulness of the annual Access Forum which brings together practitioners from across the region to discuss developments in Access. OCNSW is to be commended on its attention to communications with its members.

Governance and committee structures

Legal status and the role of Council

26 OCNSW is an unincorporated association with no formal legal status. The chief authority in its governance structure is the Council which meets on two occasions each year and is constituted on the basis of one nominated representative from each member organisation. The Chair and Vice-Chair are elected by the Council itself. The Council has overall responsibility for OCNSW including policy and its implementation. It maintains two sub-committees to assist it in carrying out these functions: an Executive Committee and a Quality Committee.

Executive Committee

27 The primary task of the Executive Committee is to oversee the more routine operation of OCNSW and to make appropriate recommendations to the Council. In particular, it considers issues relating to staffing, finances, and general resources. It also has a remit to advise Council on strategic planning and policy development. The membership of the Executive Committee includes the Chair and Vice-Chair of Council, the Chair of the Quality Committee, and 12 other elected members representing OCNSW's geographical and sector spread.

Quality Committee

28 Until recently, OCNSW had maintained two separate committees for quality matters: the Validation Committee dealt with all business in respect of the development of programmes and the validation process itself, whilst the Quality Assurance Committee was responsible for post-validation matters. In 1997, these committees were merged into a single Quality Committee. The remit of this committee covers all aspects of validation, revalidation, moderation, and review. It also has a responsibility to oversee general issues related to quality assurance.

Committee structures: summary

29 It was apparent to the review team that OCNSW has experienced a number of problems in achieving an efficient and effective committee structure. The breadth of the region which it covers does not assist in assuring regular and full attendance at committees and, on occasion, a more pragmatic approach appears to have been adopted to ensure that essential business is not delayed. In some cases, this has proved to be an effective solution. As an example, a poorly attended Curriculum Support Committee was disbanded and replaced by a system in which individuals with specialist knowledge were invited to work directly with OCNSW officers. More generally, however, this situation has the potential to reduce the authority which should properly be exercised through OCNSW's committee structure.

30 With regard to the Executive Committee, for example, the review team noted that, while the Committee was constituted broadly according to the terms set out in the document, Structure of the OCNSW, its current membership of nine did not afford it the representativeness of its intended full membership of 15. This position was exacerbated by poor attendance at meetings; minutes provided for the review team indicated that no more than six members had been present at any Executive Committee meeting in the last two years and that the Committee had been inquorate more often than it had been quorate. The OCN is attempting to address this, and elections for the Executive Committee are now in hand, but at the time of the review there appeared to be no transparent process for the Committee's election and no clear rationale for membership, other than for ex-officio members. It was also difficult from the format and coverage of the Committee's minutes to identify clearly how decisions with respect to reports are taken, and how consequent actions are pursued consistently. To a greater or lesser extent, similar comments also apply to the papers of the Council and the Quality Committee.

31 The review team was also concerned with the operation of the Quality Committee. Whilst this relatively new single body for quality issues appears to have the potential to be more efficient, the team did not find that, at present, it represented an authoritative locus of responsibility for quality and standards for the AVA. Although a paper is in preparation, there is currently no formal and full statement of the membership and terms of reference. Attendance at the Committee, which has a relatively small membership, has been disappointing and, on some occasions, marginal given the importance of its responsibilities. A number of strategies have been introduced to ameliorate this particular problem but with little apparent effect.

32 These concerns are increased by the working methods adopted by the Committee which, by dint of the volume of its business, have progressively become more reliant on sampling. Moderators' reports, for example, are no longer received in full by the Committee but are read on its behalf by the Development and Moderation Officer who then raises any significant issues. An earlier plan for the Committee to receive a random sample of such reports does not appear to have been implemented. Whilst representing a pragmatic response to the problem of abundant paperwork, these practices tend to place an over-reliance on the Officer and to threaten the balance of responsibility which should be maintained by OCNSW between its officers and its deliberative structures. The Quality Committee may wish to reflect on its working methods and to design processes by which it can assure the OCNSW Council that it has authority in respect of the quality assurance and standards of programmes.

33 In its Critical Report, OCNSW identified that further work needed to be undertaken on the documentation of committees, election processes and membership categories. A recent audit of OCNSW on behalf of NOCN reached similar conclusions. The review team would concur with these analyses and would propose that OCNSW address these issues with some urgency.

Relationship with the University of Plymouth

34 OCNSW's Critical Report acknowledges a 'considerable debt of gratitude' to the University of Plymouth for its original and continuing support. It also notes that, whilst OCNSW is an autonomous body, it is also 'part of the [University's] Department of Academic Partnerships'. The relationship between OCNSW and the University of Plymouth is not governed by any formal agreement and, although clearly effective on a day-to-day basis, its nature and extent has been an issue which has been discussed by the OCNSW Council on a number of occasions.

35 The review team also considered the ways in which the independence of OCNSW might be influenced by its association with the University. It is clear that OCNSW derives considerable benefit from this relationship, not the least in the range of paid and unpaid services which the University supplies for it. Representatives of the University also confirmed the value of their close links with OCNSW.

36 However, there remains a lack of clarity regarding relative authority in some aspects of the management of the AVA. There is, for example, a fine distinction between OCNSW simply using the University for payroll services, and the University as the employer of all OCNSW staff. From the papers of the OCNSW Council and Executive Committee, it was clear to the review team that the permission of the University needed to be sought in respect of a range of employment issues, including the nature and the issue of contracts both to core staff and to moderators working on behalf of OCNSW. The job descriptions of OCNSW staff also state that they are responsible to the Head of the University's Department of Academic Partnerships as well as to the OCNSW Council. The most senior officer of OCNSW, the Co-ordinator, has an annual staff appraisal conducted by the University rather than by the Chair of the OCNSW Council.

37 In terms of financial management, OCNSW's annual budget and operating statement is prepared by the Co-ordinator and approved by the Executive and Council. The Co-ordinator also makes regular financial reports to these two bodies. However, the review team heard that, in using the University's finance systems, budgetary monitoring was also carried out within the Department of Academic Partnerships.

38 It would be unreasonable to expect OCNSW, as a relatively small organisation, to establish its own systems for payroll and key financial processes and, on balance, OCNSW concludes that the benefits which it derives from its partnership with the University outweigh any disadvantages which may be perceived as potential threats to its independence and impartiality. However, the review team believes that OCNSW should, as a minimum, seek to sign a Memorandum of Co-operation which sets out the nature of the relationship and, in particular, clearly states the boundaries of the University's control and influence. This would remove any potential confusion as to the relative authority of the University and the OCNSW Council.

The development, validation and evaluation of access programmes

Development and validation of new provision

39 OCNSW operates thorough systems for the development and validation of new Access programmes which are supported by a well-designed and comprehensive Submission Guide, Panel Handbook, and Tutor Handbook. Documentation made available to the review team demonstrated the extensive development work which precedes submission to a validation panel: this now includes a requirement for an outline proposal through which approval to progress to the development stage is granted. Whilst the Critical Report commented on the difficulty which has been experienced in establishing validation panels, the review team found that such panels were constituted appropriately and were assiduous in their work, often commenting in substantial detail on submissions.

Monitoring, evaluation and review of programmes

40 In the systems employed by OCNSW, moderators have an important role to play in continuous monitoring and review. This begins immediately following the validation process where it is the responsibility of the moderator to ensure that any conditions which may have arisen from a validation report have been fulfilled before the programme begins. This particular arrangement is greatly assisted when the potential moderator has been a member of the validation panel. In some circumstances, the moderator can also authorise minor changes to existing programmes.

41 OCNSW also requires each Access programme to compile an annual review, and issues guidelines to assist in its production. From the documentation seen by the review team, this process has not been consistently effective. There appears to be little uniformity between such reports with some providers submitting a duplicate of that required by their own institution's quality assurance processes, others producing a separate document and, in a minority of cases, there has been a failure to submit a report at all. The AVA is aware of these differences and has been discussing how, in the interests of efficiency, it might match its own requirements with those of its providing institutions. The team would encourage these discussions to continue and for OCNSW to ensure, as a minimum, that it is collecting sufficient data and information to enable it to produce the overall annual review which will be required of it as an AVA under the new QAA Recognition Framework.

42 The treatment of annual reviews following their receipt by OCNSW might also be re-considered. On submission, they are read by the Moderation and Development Officer and, in theory, any significant issues are placed before the Quality Committee. However, there is little evidence from the recent minutes of the Quality Committee that it has been involved to any great extent in this process. Access Co-ordinators met by the review team seemed uncertain of the value of annual review and OCNSW may wish to review this aspect of its quality assurance systems to confirm its purpose and usefulness.

Assessment, moderation, standards and the award of certicates

Assesment

43 The validation process ensures that assessment of students on Access programmes is appropriate in terms of methods, nature and fitness for purpose. Assessment criteria are linked to learning outcomes, and assessment processes complement teaching and learning methods and provide suitable preparation for progression to higher education.

Moderation

44 Moderation is essentially based on a team approach, with a lead moderator for each programme and additional subject moderators where appropriate. Criteria and specifications for the moderator role have recently been developed and, at the time of the review, were about to be implemented. Externality to the providing institution is a requirement, and moderators are drawn from both further and higher education. There appears to be no limit upon length of time an individual may act as moderator for a programme although OCNSW requires all moderators to attend regular updating sessions. A small number of Access providers continue to employ external examiners for their programmes in addition to OCNSW moderators. This practice is not supported by OCNSW since it appears to create a potentially divisive perception that moderators are peer reviewers (that is, representatives from further education), whilst the external examiner is a representative from higher education with a particular brief for academic standards. However, some moderators with whom the team met appeared to believe that it was 'national policy' that Access moderators should be drawn from further education rather than from higher education.

45 In common practice, but not universally, potential moderators are identified at a point in the development of a new Access programme and are frequently members of the appropriate validation panel. A new or intending moderator must submit a curriculum vitae to the Development and Moderation Officer who then initiates the appointment. Although the Quality Committee monitors this appointment process, it does so only by its customary sampling. Given the key role played by moderators in the maintenance of appropriate academic standards, OCNSW may wish to consider whether the Quality Committee should enhance its involvement in the processes of selection and appointment. Currently, OCNSW officers are taking a leading role and it is not evident that the Quality Committee is fully exercising its responsibilities to take either a strategic view of, or an active lead in, this key area in relation to quality and standards.

46 Moderators are required to undertake initial training and also periodic updating. OCNSW issues a helpful and supportive Moderator Handbook and those moderators met by the review team endorsed the quality of the training and support they received. They also demonstrated a clear understanding of the role and its function both in terms of quality assurance and standards.

47 Moderators' visits are carried out with sufficient frequency and encompass an appropriate range of monitoring activities. OCNSW issues a pro forma for moderator reports which are sent to the Development and Moderation Officer following each visit. The Officer extracts issues and information arising from the reports which are submitted to the Quality Committee especially in cases where particular problems have been identified. Moderators receive feedback from the Officer on their reports on progress with addressing of issues raised by the reports. The review team found the frequency, content and form of moderators' reports to be appropriate. As with other work undertaken by the Quality Committee, the system for processing, monitoring and scrutinising the reports is such that decision making rests too closely with the officers and staff rather than the Quality Committee.

Standards

48 Moderators expressed some reservations as to the extent to which the moderation processes can fully ensure consistency in standards and the award of credit, especially in minority or less common subject areas. The Critical Report highlights the need to ensure equivalence between programmes and the requirement that credit and other specifications are applied in common across all programmes. The ability of the moderation system to ensure commonality of standards was not fully evident and the review team would encourage OCNSW to continue to create opportunities for increased training and benchmarking exercises.

Award of certificates

49 Recommendations for the award of credit and for certificates are signed by the appropriate moderator and sent to the OCNSW office. The Accreditation Administrator checks the recommendations against the programmes' approved credits and levels, and the Development and Moderation Officer checks the reports before certificates are issued. There is a set timescale for this process which appeared to the review team to be operating successfully.

Learner experience and progression

Experience

50 On the evidence of the Critical Report, and accompanying documents and discussions, the review team found that learners on Access programmes were well prepared for higher education in terms of skills, curricula, assessment methods, and academic standards. They also appear to experience high levels of personal support and guidance. The OCN requires providers to describe methods for programme evaluation and review in programme submission documents, and such processes can provide learners with valuable opportunities to give feedback on their Access programme experience. However, discussions with both former Access students and with Access Co-ordinators suggested that, perhaps as a result of the dependence on the requirements of the provider organisations, there is wide variety in the degree of formality of methods used and inconsistency in the rigour with which they are applied. While the review team would not wish to suggest that the OCN should stipulate that providers use a common method of student evaluation, the team considers that there is merit in the OCN clarifying for providers its baseline expectations, in terms of acceptable forms and areas for consideration for student evaluation. While some providers will already meet these expectations, others might need to supplement the standard practices or requirements of their organisations. Such a statement of expectations about evaluation could also contribute to greater consistency in annual reporting, providing information in a way which would make it more amenable to comparison and analysis.

Progression

51 Former students met by the review team identified their Access programme as the key enabling factor in their progression to higher education. Several had received conditional offers without interview, testifying to the recognition within higher education institutions (both locally and nationally) of the 'kitemark' and the quality of the programmes within the AVA. OCNSW organises various events and activities to support progression planning and to facilitate the access of learners and providers to information about the transition to higher education.

52 OCNSW attempts to monitor learners' progression in HE but, as is the case nationally, it is difficult for the OCN to obtain complete relevant data both from Access providers and from receiving higher education institutions. However, data on learners' progression intentions are collected systematically by OCNSW and, together with information collected on registrations, withdrawals and completions, this furnishes an overall picture of progression trends. The links between data on progression and the ability to improve targeting strategies are understood by OCNSW and it has identified this area of activity as one which requires further development.

Conclusion and Recommendation to the ARLC

Conclusion

53 The success of OCNSW's operation as an AVA is influenced by two key factors. Firstly, as an organisation it is relatively recently established and some of its structures and processes are not yet fully mature. Secondly, its situation, covering a large, mostly rural, geographical area with a widespread membership, presents certain challenges in ensuring the effectiveness and rigour of its processes and organisation. OCNSW has been successful in maintaining a reasonable geographical spread of members throughout the south west region and has validated Access to HE programmes across most, though not all, of this region. Its membership also includes appropriate representation from different sectors, although the small number of higher education institutions in the region presents some limitations to the AVA's activities, and to the development of the its relationship with the higher education sector in general, and with the institution which hosts it in particular.

54 These characteristics lie behind the review team's concerns about the relative roles of officers and committee structures. OCNSW is well-supported by a small team of committed officers who work hard to compensate for some of the difficulties arising from its situation and structures. However, while the committees appear to conduct themselves with integrity and commitment, their effectiveness, and their capacity to assure the quality of AVA activities and processes, is hampered by a general lack of clear terms of reference, membership, remits and lines of reporting.

55 The processes for the development and validation of courses are well-established, thorough and well-understood. Processes for the monitoring and review of programmes, however, are less robust and less consistently applied. In addition, while there are strengths in the model for training and updating of moderators, the procedures for their selection and appointment, and the process of moderation itself, need some attention to assure the reliability and consistency of moderation.

56 Although the lack of data makes the statistical picture incomplete, with tracking of students being a recognised difficulty, learners progressing from OCNSW Access to HE programmes appear to have received an appropriate and adequate preparation for higher education.

57 The review team commends OCNSW for:

i the effectiveness of its methods for communications with its members;

ii its thorough and supportive processes for the development and validation of new programmes;

iii its arrangements for the training of moderators and the quality of the supporting documentation;

iv the effectiveness of the support it offers to learners in their progression to higher education.

Recommendation to the ARLC

58 The review team recommends that OCNSW be granted a conditional confirmation of licence with conditions to be met by the dates specified below.

Conditions and recommendations

Conditions

59 Subject to the approval of the ARLC, OCNSW's AVA licence is confirmed on condition that it:

i clarifies its relationship with the University of Plymouth in order to confirm its essential independence (by 1 November 1999);

ii undertakes a thorough review of its governance structure and, if there are to be committees, to ensure that all committees have formal and approved remits, terms of reference, membership, and frequency of meetings (by 1 November 2000, with an evaluation of the operational effectiveness of structures by 1 July 2000).

Recommendations

60 The review team recommends that OCNSW:

i reviews its staffing policy to consider whether it would be advantageous to employ some of its core staff on a more permanent basis;

ii reviews the operation of its Quality Committee with particular reference to its involvement in the selection and appointment of moderators and the treatment of their reports;

iii reviews the system for the receipt and review of annual reports from Access providers to ensure consistency of reporting and processing of reports, and to ensure that reports supply adequate information for the compilation of the AVA's annual report required by ARLC;

iv develops a strategic approach to targeting in order to support providers in identifying and recruiting appropriate groups;

v continues to develop methods to encourage consistency of standards through subject benchmarking and moderator training;

vi develops advice to providers to indicate the OCN's minimum expectations with regard to student evaluation and feedback;

vii considers its legal status, with a view to ensuring that it is able to meet all of its public obligations.

Open College of the South West August 1999

Contents