Open College Network of the West Midlands (OCNWM)
May 2003

Foreword

1 The Quality Assurance Agency for Higher Education (QAA) is responsible to the Department for Education and Skills for the recognition of Access to Higher Education programmes. QAA exercises this responsibility through a national network of authorised validating agencies (AVAs), which are licensed by QAA to recognise individual Access to HE programmes, and to award Access to HE certificates to students. The AVAs are responsible for implementing quality assurance arrangements in relation to the quality of Access to HE provision and the standards of student achievement.

2 QAA has developed a scheme for the licensing and review of the AVAs, the principles and processes of which are described in the QAA Recognition Scheme for Access to Higher Education in England, Wales and Northern Ireland. The Recognition Scheme is regulated and administered by the Access Recognition and Licensing Committee (ARLC), a committee of the QAA Board of Directors. The ARLC is responsible for overseeing the process of AVA review through which AVAs are periodically relicensed. The criteria applied by the ARLC and by review teams operating on the Committee's behalf, in reaching judgements about whether and under what terms an AVA's licence should be renewed, are provided within the Recognition Scheme documentation. These criteria are grouped under the seven principles that provide the main section headings of this report.

3 Following the review of an AVA, a member of the review team presents the team's report to the ARLC. The Committee then makes one of five decisions:

i unconditional renewal of licence for a specified period;

ii conditional renewal of licence with conditions to be met by specified date;

iii provisional renewal of licence with conditions to be met and further review visit by specified date;

iv withdrawal of licence for operation as an AVA;

v temporary renewal of licence with request for further information by specified date (decision suspended).

4 This is a report of a review of the AVA function of the Open College Network of the West Midlands (OCNWM) undertaken by QAA. The Agency is grateful to OCNWM and to those who participated in the review for the willing cooperation provided to the review team.

The review process

5 The review was conducted in accordance with the process detailed in the QAA Recognition Scheme for Access to Higher Education in England, Wales and Northern Ireland. The preparation for the review included an initial meeting between OCNWM representatives and the QAA Assistant Director to discuss the requirements for the Analytical Account (the Account) and the process of the review; the preparation and submission by OCNWM of its Account, together with a selection of supporting documentation; a meeting of the review team to discuss the Account and supporting documentation and to establish a draft programme for the review visit; and negotiations between QAA and OCNWM to finalise the programme and other arrangements for the review visit.

6 The review visit took place on 13 and 14 May 2003. The visit to OCNWM consisted principally of meetings with representatives of OCNWM, including AVA officers; members of the Management Committee and Quality Assurance Group, moderators for Access to HE programmes; Access to HE coordinators; representatives from higher education; and former Access students now studying in HE.

7 The review team consisted of Dr Peter Easy, Vice-Principal, University of Gloucestershire and Mr Alan Smith, consultant for quality matters, North Wales Open College Network. The review was coordinated for QAA by Ms Kath Dentith, Assistant Director (Access).

The AVA context

8 OCNWM is based in Wolverhampton and was first licensed as an AVA in 1995 by the Higher Education Quality Council (HEQC), following a merger between two former AVAs: the Black Country Access Federation and the Birmingham Access Federation. The merger created an AVA which now covers a region including Birmingham, the Black Country, and Shropshire.

9 In the period since its establishment, OCNWM has grown considerably with around 40,000 learner registrations with the OCN as a whole, of which about 2,600 are Access students. In terms of major developments, a review of governance structures was undertaken in 1998-99 and, as a result, the OCN adopted a new constitution in September 1999. As well as making more explicit the independent status of OCNWM, one of the aims of the new constitution was to clarify the processes and procedures of the AVA function within the overall organisation. Further adjustments to the constitution and to governance arrangements have followed and, in some areas, continue to be made.

10 Access provision currently validated by the AVA comprises 29 programmes delivered by 19 providers. The programmes range from small specific Access courses to large multi-pathway modular programmes.

AVA Statistics (as provided by the AVA in its annual report to QAA for 2001-02)

11 Providers offering Access to HE programmes 19

Access programmes available 30

Access programmes running 29

Access learner registrations 2,650

Access to HE certificates awarded 924

12 Although there was some decline in the intervening years, a comparison between the AVA's figures for 1998-99 and 2001-02 shows little difference in the number of providing institutions, registered learners, and Access certificates awarded. There has been a rise, however, in students progressing to higher education (466 to 575). Learner withdrawals have also increased in the period since 1998-99, and the AVA has been particularly proactive in analysing the performance of its providers in this context. Figures showing three-year trends, together with regional and national comparisons, have been produced for consideration within the AVA and discussion at the Access to HE Forum and Regional Access Forum. The level of analysis, which was a feature of both the AVA's annual report for 2001-02 and the Account prepared for this licence report, demonstrates the notable efforts made by the AVA to understand its retention and achievement rates and to use its understanding to inform future strategy.

Principle 1

The organisation has a structure which is based on a partnership of members, including institutions which provide Access to HE programmes and institutions of higher education.

13 In its constitution, OCNWM is described as 'an unincorporated association of member institutions'. Its current membership comprises over 100 organisations including voluntary and community bodies, public and private training organisations, all further education colleges in the region which it covers, and six higher education institutions: Birmingham College of Food & Tourism, Newman College of Higher Education, and the Universities of Aston, Birmingham, Central England and Wolverhampton.

14 Membership of OCNWM is open to 'all organisations that support or provide learning opportunities' and who subscribe to a set of four principles which promulgate inclusiveness, multicultural approaches, accessibility, and the enabling benefits of education. Those organisations which intend to offer accredited programmes through OCNWM must also demonstrate their ability to meet other criteria related to quality standards, finance and resources, and general infrastructure. Although OCNWM operates differential subscriptions related to the size of an organisation, or the extent of its activities in respect of the number of programmes and learners registered, there are no separate categories of membership. All organisations are thus subject to the same rights and responsibilities, and have equal representation within the governance structure.

15 Applications for membership are made on a pro forma which is contained within a larger Membership Application/Renewal document. This comprises a comprehensive statement on membership including details of eligibility and criteria, fees and charges, method of application, and a statement of rights and responsibilities. Applications are considered by the Quality Assurance Group (QAG) which makes a recommendation to the OCNWM Council. The minutes of the QAG demonstrated that appropriate attention was paid to applications and that QAG also monitored and amended the membership application documents when necessary. The AVA is to be commended on its provisions for admitting organisations into membership.

16 In summary, the review team considers that OCNWM meets in full the criteria expressed under Principle 1.

Principle 2

The organisation has governance structures which allow it to discharge its AVA responsibilities securely.

17 OCNWM's constitution sets out its formal governance structures. Each member organisation is entitled to nominate a representative to the Council of Members (the Council) which is responsible for 'overseeing the overall management and organisation of the Network', including responsibility for the approval of the strategic plan, budgets and financial reports. The Council also performs an important function in electing members to its sub-committee structure. At the time of the review, OCNWM had made further changes to its constitution which affected the scope of the Council's responsibilities. These are discussed in paragraphs 20 to 22 below.

18 The Council has one sub-committee, the Management Committee, which deals with the day-to-day operation of the AVA, including delegated responsibility for financial and staffing matters. Two sub-committees report to the Management Committee. The Finance & Resources Group (FRG) considers the detail of financial and staffing matters. The QAG monitors and reviews the accreditation, recognition and validation functions of OCNWM. In addition, it undertakes a range of other quality assurance functions including the annual review of programmes; the appointment of moderators; and receipt and consideration of moderators' reports. The QAG has quality assurance responsibility for all provision accredited by OCNWM, including Access, and 'Access to HE Issues' is a standing item on its agenda. The Account notes that, whereas HE representation on the Management Committee is required, this is not the case for the QAG, and that the AVA considers this requirement should 'apply also to the Quality Assurance Group'. The review team regards the AVA's intention to make such an amendment as an essential clarification of the QAG's membership specification.

19 In addition to these committees, which have a function within the formal governance structures, an Access to HE Forum has been established. Consisting of Access to HE coordinators and representatives from higher education institutions, the purpose of the Forum is to act as a consultative body in respect of the development of Access provision and the application of regulatory requirements related to the AVA licence. The review team noted the expertise and experience of Access which was clearly evident in this Forum and, without suggesting that there should be any impact on governance structures, would encourage the AVA to make full use of the potential of the Forum in its Access activities. (See also paragraph 23, below.)

20 In the Account prepared for this review, which was submitted in March 2003, the AVA expressed 'a general satisfaction with the present structure of governance and little indication, at present, of the need for change in the near future'. However, the review team noted that at the Annual General Meeting of the Council in March 2003, changes had been made to the constitution which devolved substantial powers from the Council to the Management Committee. These changes appeared to the team to be a recognition on the part of OCNWM that its governance structure, particularly in respect of the Council, was increasingly unable to respond effectively to the needs of the organisation. In the papers scrutinised by the team, for example, it was clear that the Council had not had a quorate meeting for some years (including the meeting at which the constitution was changed), and that the pattern of its meetings was somewhat irregular.

21 The changes made to the constitution and to the powers of Council appeared to the review team to have effectively ended the Council's role as a governance body. Its meetings have been reduced to one per annum, and all of those responsibilities which were carried out at the ordinary meetings of Council have been transferred to the Management Committee: these include approval of reports sent to external bodies (for example, the AVA annual report prepared for QAA), the receipt and approval of financial reports in relation to the strategic plan including projected budgets, the monitoring of the annual operating statement, and the handling of complaints and appeals. In the view of the team, the Council - although left with its original remit of the oversight of the management and organisation of OCNWM - could no longer fulfil that remit.

22 In discussion with senior staff of OCNWM, it was clear to the review team that the changes to the constitution had been made with an understanding of their consequences. In this sense, the organisation has acted responsively to a situation in which the Council could not realistically undertake those responsibilities which the constitution had allocated to it. The team believes, however, that the consequences of the constitutional changes now need to be followed through. The transfer of significant responsibilities to the Management Committee effectively establishes it as the senior body in the governance structure. However, the relatively small size of the Management Committee, which would appear essential for its efficiency, leaves open the question of how OCNWM is generally held accountable as an organisation to its full membership. This might be resolved by a clear statement on the future role of the Council as a consultative body but the retention of its annual task to elect the members of the Management Committee.

23 While considering these matters related to its constitution and governance, there are a number of other issues which the AVA is asked to take forward in relation to its committee structure. These are:

to establish a common approach to quorum regulations for its committees. Currently such regulations range from none (FRG) to differing proportions (for example, approximately one-third of members for the Management Committee, and one-half of members for the QAG);
to review the membership of the FRG, which currently comprises four members of the Management Committee, to consider whether a measure of independence would be gained from the inclusion of a member external to the Management Committee, or even external to the organisation. Such an amendment would reflect good practice in relation to committees which have financial and resource responsibilities;
to revise the membership specification for the QAG to make it clear that there should be specific representation from higher education (currently, such representation exists in practice but the membership regulations are restricted to generally ensuring 'adequate representation from each sector of education and training');
to consider whether the Chair of the Access to HE Forum should sit as-of-right on the QAG. While acknowledging that the Forum is not part of the formal committee structure, it does report to the QAG. The presence of its Chair would assist in this reporting function; it would also allow the AVA to benefit fully from the expertise contained within the Forum in relation to the Access-related work of the QAG.

24 Since 1998-99, OCNWM has been reviewing its constitutional and governance arrangements. A number of important changes have been made, and continue to be made, as the organisation seeks the most effective and efficient structure. The review team is confident that there exists within the AVA an appropriate understanding of these governance issues, as well as the willingness and capacity to ensure that they are satisfactorily resolved. While the criteria expressed under Principle 2 were therefore considered to have been met, there remains some progress to be made in this area. It is, therefore, recommended that a further review is undertaken of the constitution and governance arrangements which clarifies the constitutional position of the Council and which takes into account the points raised in paragraph 23, above.

Principle 3

The organisation is aware of, and in a position to meet, its legal and public obligations.

25 OCNWM's position as 'an unincorporated association of member organisations' does not provide it with a legal identity which necessarily separates the organisation's liability from that of its members. In recognising this position, the AVA has taken legal advice and, as a consequence, has taken out an indemnity which limits the liability of its individual members. These actions have also led to a position where it is recognised as having an identifiable basis for entering binding legal contracts, examples of which were given to the review team.

26 The review team also discussed the relationship between OCNWM and the University of Wolverhampton. Currently, OCNWM rents office accommodation from the University at a commercial rate and receives personnel, finance and ICT services from the University in lieu of an annual membership subscription. This long-standing relationship is governed by a Memorandum of Agreement which has a specific section, Recognition of Respective Authority, in which the University acknowledges the independence of OCNWM and agrees that it will 'in no way seek to influence or constrain the powers of the OCN...beyond its entitlement to do so as an ordinary member'.

27 The review team noted that this formal declaration does not, however, exempt OCNWM from the restrictions which accompany the services it receives from the University. Hence it must, as the Memorandum of Agreement also states, 'adhere at all times to the financial, personnel, health and safety and other policies and regulations of the University that may apply to the OCN's operation, as if the OCN were a department or operating Unit of the University of Wolverhampton'. In using the services of the University, it is unavoidable that the AVA will be subject to the University's operational policies and practices. However, the team noted that the essential independence of the AVA had been protected at key points, for example, the line of accountability of the OCN Director is agreed in the Memorandum as OCNWM's Management Committee, and the Director's appraisal is carried out by the Chair of that committee.

28 The review team considers that the relationship between OCNWM and the University does not constitute any significant infringement of the AVA's independence. However, the team noted that there were occurrences in the AVA's formal documents where the University appeared to be afforded specific privileges (an example would be clause 9.4 of the constitution which enjoins the AVA to 'advise the University of Wolverhampton of any changes made to the constitution by the Council of Members'). This does not sit easily with the concept of the University as, in most respects, an ordinary member of the organisation, and the AVA is recommended to remove them to avoid any potential misunderstandings.

29 The review team considers that OCNWM meets the criteria expressed under Principle 3.

Principle 4

The organisation is able to manage effectively its AVA responsibilities and the structure which supports them.

Aims

30 The OCNWM constitution contains a concise statement of broad aims which, in emphasising the promotion of opportunities to traditionally disadvantaged groups, and the flexibility and quality of those learning opportunities, is congruent with the aims of the QAA Recognition Scheme. The review team noted that a helpful expanded set of aims also appeared in OCNWM's Welcome to OCN Accreditation leaflet.

31 The current Strategic Plan includes 12 strategic aims, several of which could benefit the general efficiency and effectiveness of OCNWM and support the development of provision in the area, and one of which is specific to the organisation's work as an AVA. This aim states the AVA's intention to 'Extend the range and degree of opportunities of the Access route into higher education and contribute to developments in Access to HE to promote quality provision that is consistent with national standards set by QAA'.

Strategic planning

32 OCNWM operates a two-year strategic planning cycle which incorporates annual operational plans. The strategic plan is developed in the first instance by the staff of the AVA and is then approved by the Management Committee. The Committee is also responsible for the monitoring of the annual operating plans.

33 In its scrutiny of this process, the review team noted that the general strategic aims which emerge from the text of the current Strategic Plan are translated into more specific objectives; these are in turn used to produce specific targets and performance indicators in the annual operational plans. In the current planning documents, the team was able to trace this logical progression as it applied to the strategic aim for Access provision which culminated with a detailed action plan for OCNWM's AVA activities. It was clear from its minutes that the Management Committee devoted an appropriate amount of its time to the discussion and monitoring of the Strategic Plan and the related annual plans.

34 In its Account, the AVA reported that an internal analysis had revealed the need for greater involvement on the part of its staff in planning processes. To this end, it has established a Strategy Group to inform the strategic direction of the AVA. Such a development can only enhance further what, in the belief of the review team, is an already effective strategic planning process.

Financial management

35 The financial management of the AVA is located initially with the FRG, which makes recommendations to the Management Committee. The minutes of both bodies demonstrated an appropriate level of discussion and scrutiny of the financial affairs of the AVA. Overall, OCNWM is a well-resourced organisation and, for a number of years, it has accumulated surpluses which it has built into a substantial reserve. There is evidence of a healthy tension in the discussion of the use of these reserves for investment purposes or as a contingency against unforeseen circumstances. In effect, the AVA's financial prudence enables both aims to be achieved. The organisation maintains a more than adequate contingency, and the review team noted a number of well-funded developments which were aimed to improve the efficiency of the AVA and the quality of services of its members.

Communications

36 The AVA uses a number of methods to maintain its communications with providers. These include its web site, an occasional newsletter, the circulation of papers issued by national bodies, annual and other staff development events, and the work of the Access to HE Forum. The success of the AVA's communications was confirmed by Access coordinators met by the review team, who also expressed their satisfaction with the responsive and supportive service offered by the AVA's development officers.

Equal opportunities

37 The AVA has an appropriate policy statement on equal opportunities, the implementation of which is apparent in relation to its major activities. The promotion of equal opportunities is also embedded in its mission statement and its statement of aims. The review team noted examples of the ways in which the commitment to equal opportunities was implemented in the AVA's activities. These include the checklist contained in the document Submitting a Programme for Recognition where, following a restatement of the policy, providers are offered comprehensive guidance under the headings of 'curriculum', 'recruitment', 'course structure', 'the learning environment', and 'learner support', which is designed to ensure awareness of equal opportunities in all aspects of programme design and delivery. Moderators are also asked to comment on evidence that programmes are continuing to meet the equal opportunity requirements of the original submission document. The AVA is to be commended on its commitment to equal opportunities and the ways in which this commitment is evident in its activities and procedures.

Self-assessment and risk assessment

38 OCNWM undertakes a formal annual self-assessment which covers its major activities. This process is supported by a quality development cycle through which staff quality teams monitor the work of the organisation under the three themes of management and governance, accredited provision, and learner achievement. The outcomes of this monitoring activity inform the AVA's self-assessment and also lead to the production and monitoring of service standards.

39 The AVA's approach to risk assessment is not as well developed. The review team was told that an analysis of risk was a feature of the AVA's financial management processes. However, a similar analysis was not evident in other aspects of management or operation, and the team recommends that the AVA considers a more overt risk assessment approach in its general strategic planning.

Annual reporting to QAA

40 OCNWM has satisfactorily met its general annual reporting commitments to QAA, although queries have been raised in the past in two areas. Concerns raised by QAA relating to the appropriate body for the confirmation of the report may be resolved as part of the continuing review of governance arrangements, and, in particular, through the AVA's further consideration of the responsibilities of the Council of Members and Management Committee and their positions within its structures. (See paragraphs 20-22, above.) There has also been some confusion as to the extent and nature of the requirements for financial information. The AVA may wish to note that the financial information supplied as part of the documentation for this review would be entirely appropriate for inclusion in its annual report to QAA.

Office resources and staffing

41 The AVA is located in spacious and well-equipped premises, situated centrally within the region that it serves. There is an open-plan office, well-resourced with IT facilities, supplemented by private offices and a small meeting room. Additional meeting rooms are available within the building.

42 The activities of OCNWM are supported by a team of 17 staff: all posts have clear job descriptions. The Deputy Director is charged with overall responsibility for Access matters, and also has line management responsibilities for key staff. In addition to the Director and Deputy Director, there is a team of officers with responsibility for committees, finance, operations, moderation and development. This team is well supported by appropriate administrative staff. The Account records that there are regular staff team meetings at which 'operational and policy matters relating to Access' form a standing item on the agenda.

43 Staff development opportunities are available to all staff, with access available to further and higher education programmes free of charge through an agreement made with providers in the Black Country. Staff development needs are identified through the annual appraisal process.

Operational procedures

44 All staff belong to one of three quality teams: management and governance; accredited provision; or learner achievement. The purposes and processes of each team are thoroughly documented with monitoring, reporting, self-assessment and action planning being the key functions. Each team has a set of service standards against which to assess its effectiveness. Their findings are incorporated into the annual self-assessment report, which is then presented to the QAG. The review team commends the AVA for its thorough and self-critical approach to internal quality assurance, and the systems it has developed.

45 The administrative systems were considered by the review team to be thorough and secure. Documents available to providers addressing matters such as registration, programme submission, and moderation were comprehensive and clear.

46 The AVA has recently transferred from Advanced Revelation to OPUS as the database for the management and manipulation of data related to learner registrations and award of credit. Staff training sessions have been available for the new system but there are still some occasional problems, which are dealt with as they occur. The review team recommends that the AVA regularly checks the data output from the new system to ensure that it is error free.

47 OCNWM has clear policies for monitoring provider satisfaction with recognition panels, moderation procedures and staff development activities. The feedback from providers is regularly analysed by nominated members of staff as part of the self-assessment process and separate reports are submitted to QAG on panel evaluations and moderation reviews. The very few grievances that are reported are tracked and followed to a conclusion. Those providers met by the review team considered that there were sufficient opportunities to address and resolve problems informally, so that it was rare for grievance procedures to be implemented. Indeed, response from providers about the support received from the AVA was, in general, very favourable, both in relation to individual contact and the provision of staff development activities

48 The review team concluded that OCNWM had adopted clear procedures for the operational management of the AVA and, notwithstanding the above recommendations, considered that the licensing criteria expressed under Principle 4 had been met.

Principle 5

The organisation is able to assure the quality and fitness for purpose of Access to HE programmes at the point at which they are granted formal recognition.

Programme development

49 OCNWM procedures for programme development and recognition are detailed in the documents Welcome to OCN Accreditation and Submitting a Programme for Recognition. These wide-ranging and well-produced documents explain to members the requirements of the AVA for submission and recognition of programmes and also the procedures necessary to modify existing programmes. Similarly, there are guidance notes for tutors and moderators about policies governing the achievement of credits, credit transfer, credit exemption and accreditation of prior learning.

50 The majority of Access to HE programmes are developed by the provider and the development officer with responsibility for that particular member organisation. Tutors from HEIs will be involved with the development of units' content and, where there are modifications occurring to an existing programme, there will also be moderator involvement.

51 For those members submitting Access to HE programmes for recognition, and those attending recognition panels for those programmes, the AVA provides additional guidelines and notes. These highlight the importance of clear identification of target groups; admission and guidance and support requirements; assessment methodology and regulations; and the minimum credit requirement necessary for the award of the QAA-recognised Access to HE certificate.

52 The review team considered that there was a degree of uncertainty over the minimum credits required for an Access certificate awarded by OCNWM. The AVA has produced a helpful statement on minimum requirements for the award of the Access to HE qualification, which is recorded in its guidance documents for moderators, and that for submitting tutors and panel members. This states that 'learners must achieve either 80% of the credits available within a programme or a minimum of 16 credits, whichever is the higher; a minimum of 12 credits must be at level 3', and further explains that 'This minimum requirement does not preclude submitting organisations proposing or recognition panels requiring, higher levels of achievement in particular circumstances'. The guidance for moderators provides some detail about the circumstances in which requirements in relation to achievement in Core Skills may vary. There is no such detail provided, however, in relation to acceptable variations to the overall credit requirement, or the rationale for variations, although the policy does make clear that 'The specific entry requirements of particular degree pathways should be reflected in the offer made by the HE institution, rather than the Access course regulations'.

53 The review team considered that, while the AVA's current policy provided some security in relation to the national minimum requirement for achievement on QAA-recognised Access programmes, it could, nonetheless, allow substantial inequity of demands on students for the award of the Access certificates on different programmes recognised by the AVA. On reviewing the programmes currently recognised by the AVA, it was clear that the number of credits required differed from one programme to another and that there was no clear rationale for those differences. The team concurs with the view expressed by officers of the AVA that there should be greater standardisation of the number of credits necessary to achieve the Access certificate, and recommends that the AVA continues to address this, with a view to clarifying the AVA's requirements.

Programme validation

54 The document Submitting a Programme for Recognition refers to the need for Access panels to have HE representation, although the review team considered that the statement of the AVA's expectations in this regard lacked the clarity and precision needed to secure appropriate expertise, externality and objectivity within the recognition process. The review team would recommend that the AVA develops its formal requirements in relation to panel membership for Access recognition panels. The AVA may wish to consider some of the indicative evidence statements provided in the QAA Licensing Criteria, principle 5, criterion 2, in relation to such developments. While panel reports viewed by the team indicated that the AVA's current minimum requirement for HE representation was being met in practice, the Account draws attention to the difficulty experienced by the AVA in securing HE representation on panels and notes the relatively low proportion of panel members from HE. The AVA has recognised 'the need to increase the pool of HE members, overall HE representation on panels, and to establish more balanced representation from the Network's HE members', and the Account reports on its efforts to address this problem. The team would encourage the AVA to persist in these efforts to develop the involvement and participation of its HE members in the recognition process.

55 In order to promote a standardised approach to the recognition process, Access panels are normally chaired either by the Director or Deputy Director of the AVA, although this role is occasionally played by a member of the Development Team. The Development Officer who has been involved in the development of the programme convenes and acts as scribe to the panel. A standard approach is also supported by the use of very clear guidelines for Access panel members and submitting tutors to indicate the standards necessary within the documentation for approval of programmes and a checklist for panel chairs which provides a summary of main points to be addressed.

56 The Chair's Checklist is supplemented by the full detail given in Submitting a Programme for Recognition. Both documents include a note on the 'ethos' of an OCN panel, Submitting a Programme for Recognition noting that 'Everyone present takes an equal part in all the discussions of the panel' and the Checklist stating that 'All panel members have equal status'. The review team considered that the former statement provided a reasonable encouragement to all panel members to participate fully, whereas the latter might suggest an inappropriate involvement in the decision-making process by those submitting the programme. The team recommends that greater distinction is made between the role in the recognition process of external panel members and those submitting programmes to the AVA for recognition; that the AVA's documentation on the recognition process includes a statement which clarifies that responsibility for making recommendations for the formal approval of the programme (including confirming any conditions and recommendations) lies with the non-submitting members of the recognition panel; and that panel reports clearly distinguish between those submitting and those recognising the programme.

57 Reports of the panels clearly identify all outcomes and the timescale for the implementation of any conditions, and they are sent to all members of the panel and to those submitting programmes for recognition. The Accreditation Officer monitors compliance with conditions and this information is forwarded to the QAG in circumstances where conditions are not being met. When all conditions are met, the programme is approved by QAG and a certificate to this effect is sent to the provider.

58 The AVA recognises GCSE equivalences in maths, English and study skills, and science. The appropriateness of the units involved is regularly scrutinized by curriculum groups of subject experts from further and higher education, which meet at intervals to ensure that there is continuing equivalence with changing GCSE syllabus and assessment requirements.

59 The review team was satisfied that the recognition panel process was robust and secure, with the responsibility for final recognition resting clearly with the QAG. The team therefore concluded that the criteria expressed under Principle 5 had been met. The AVA should also note the team's recommendations relating to standardisation of the credit specification for the award of the Access certificate, and to the documentation relating to panel composition and roles of panel members.

Principle 6

The organisation is able to safeguard the continuing quality of Access to HE programmes, and to secure the standards of achievement.

Moderator appointment and training

60 Moderator appointments are made by the Moderation Coordinator, following a formal application process which is overseen by the QAG. The QAG's remit includes responsibility for monitoring and confirming moderation appointments, a responsibility which is exercised through the periodic sampling of moderators' application forms and CVs. Any concerns which the officer may have relating to the suitability of potential moderators are referred to the QAG for final confirmation. Moderators are required to sign an annual contract which clearly identifies that they must have no contractual connections to the organisation at which they have been asked to moderate.

61 All new moderators receive induction prior to their first visits and can refer to the moderation or development teams for advice and support where necessary. Those moderators met by the review team spoke highly of the quality and thoroughness of the training received on induction, which involved not simply being taken through the Access to HE Moderation Guidelines but also given 'what if' scenarios to discuss and consider. Moderators receive regular update sessions, in addition to the other Access-related staff development opportunities offered by the AVA.

Moderation process

62 OCNWM operates a common system of moderation for Access programmes with module moderators and overall programme moderators, with the latter being responsible for the final signing off of the QAA-recognised certificate, subject to the confirmation of the results of the modules.

63 All moderators are issued with the Access to HE Moderation Guidelines which give thorough details of their roles and responsibilities. The Guidelines provide agendas for each moderation visit by both module and overall programme moderators, clearly differentiating their roles and relationships. The Guidelines also provide a pro forma letter confirming exactly what the moderator expects to see on any visit. Details of sample sizes from different sized groups are provided, together with the other criteria that must be considered when selecting samples of learners' work for moderation. Moderators receive guidance on the completion of reports, and there is comprehensive information about when to moderate and when to submit reports.

64 The AVA, through the QAG, is considering the appropriateness of the current system of moderation with two visits per year from all module moderators and also from the overall programme moderators. It is thought that, in view of a considerable reduction in the number of curriculum-related issues over the past few years and the increasing rigour of providers' internal moderation systems, it may be possible to reduce the level of review by external moderators of learners' work, relying more on internal moderation processes, thus possibly reducing the number of visits necessary by external moderators. Currently, both module and overall programme moderators review internal moderation processes to confirm their effectiveness rather than to utilise their outcomes. The review team notes this worthwhile review of the moderation arrangements and recommends that the QAG gives serious consideration to its outcomes.

65 Guidance notes and the report forms themselves direct moderators to review matters of assessment; meeting validation panel requirements; verification of standards of achievement; and the quality of the internal moderation processes. Moderators are provided with a copy of the AVA's Equal Opportunities Checklist, which should also be used by providers preparing submission documents. This can be used by moderators to confirm that the issues of fairness, equity, objectivity and impartiality are checked during their visits. A specific section of the final moderation report form enables the moderators to feed back to the AVA on this aspect of quality assurance.

Standardisation activities

66 Several attempts have been made over the past two years to establish standardisation activities which will allow comparisons of level and achievement to be made across different programmes. Lead moderators have been appointed to curriculum groups including creative studies, business/law, mathematics, humanities and health/biological sciences. The remit of these lead moderators is stated and the agenda for each meeting is laid down together with the criteria for sampling work. Unfortunately, these events have not been well attended by either moderators or providers, and the exercise has not been able fully to achieve its aims. In an attempt to take the matter forward, the AVA has produced a report on these activities with recommendations based on the deliberations of the lead moderators and the evaluation forms of those who attended the events. The review team recommends that every effort is made to develop and continue with these events in some form, and that their importance for the integrity of the Access qualification continues to be emphasised to providers. The review of the moderation system may enable this matter to be addressed by the appointment of cluster moderators who will work across different providers, thus ensuring a greater degree of standardisation than is available at present.

Moderation reports

67 All moderator reports are reviewed on receipt by the AVA and, where there are issues with the quality or timing of the report, they are addressed with the moderator in question. These issues are highlighted in the Moderation Coordinator's annual report to the QAG, and the AVA reported in its Account that it had achieved significant improvement in the quality and promptness of reports through its tightening of monitoring procedures.

68 All reports received by the AVA are scrutinized by the Moderation Coordinator before being forwarded to the college principal or equivalent senior representative of the provider; programme coordinator; the OCNWM Development Officer responsible for the programme; and the overall moderator, if the report is from a module moderator. Where issues have been raised, the provider is asked for an action plan to address them. This is forwarded to the moderator when received by the AVA. The Moderation Coordinator produces a report for the QAG which details any provider who has not responded to the issues raised. If, after further correspondence with the provider, there is still no response, a meeting will be arranged between a QAG representative and the provider. The QAG has the power to suspend the recognition of a programme until issues are appropriately addressed.

69 The Moderation Coordinator produces an annual report for QAG which gives details of the number of moderation reports received, the percentages of those with no issues to be raised with the provider (category 1), those with sound provision but some issues to be raised with the provider (category 2), and those with serious issues of concern to the AVA (category 3). For each area in which concerns are raised, a more detailed analysis is developed for the QAG. The report also includes an analysis of moderator satisfaction with the AVA. The review team considered that this was a thorough and systematic process, with appropriate division of responsibilities between officers and the QAG.

Award and issue of Access to HE certificates

70 The award of the QAA-recognised Access to HE certificate is the direct responsibility of the Overall Programme Moderator, although the QAG's remit includes a responsibility to 'oversee the arrangements for QAA-recognised Access to HE certificates' and to 'monitor and review the processes of...The Award of Credit'. The Access to HE Moderation Guidelines give details about the conduct of the final moderation meeting at which the Overall Programme Moderator signs the Recommendations for the Award of Credit forms, on the basis of advice of module moderators, who have agreed the achievement of the necessary credits in their subject areas. Module moderators are encouraged to attend this meeting or to provide their reports if unable to attend. The Guidelines draw the attention of the Overall Programme Moderator to the need to check achievement against the criteria for successful completion specified in the submission document, a process which is especially important, given the variation in credit requirements between programmes (see paragraphs 53-54, above).

71 When the signed claims for credit are received by the AVA, they are further checked against the requirements stated in the submission to ensure compliance. The complexity of Access programmes has led in the past to problems with the accurate documenting of the achievement of credits. Work with providers to agree a set of forms that both provider and AVA can accept has done much to reduce this problem.

72 Successful candidates receive the certificate and a separate transcript listing the credits achieved. The sample certificates made available to the review team carried the incorrect version of the QAA logo for use on certificates, with no reference to the fact that the programme or certificate was recognised by QAA. The review team would expect this matter to be addressed as a matter of urgency, and the AVA should ensure that all Access certificates issued in the future bear the version of the logo which includes the words 'recognised by the Quality Assurance Agency for Higher Education'.

73 The review team concluded that the moderation process was thorough and that the process for certification was essentially secure. Nonetheless, in order for the AVA to be able to demonstrate that it has a properly regulated process for Access certification, it is a condition of the renewal of the licence that the AVA revises its standard Access certificate to ensure that the correct version of the Access logo is used, in accordance with the guidelines provided by QAA.

Principle 7

The organisation is underpinned by structures and processes which enable it to review, evaluate and develop the Access to HE provision for which it as responsibility.

Revalidation of Access programmes

74 Programme revalidation takes place every five years. Providers are required to resubmit their programmes to an Access recognition panel, following the same procedure as for a new submission. Outcomes of all resubmissions are reported to the QAG for final approval. The review team saw no direct evidence to confirm that the revalidation panel requires information on the monitoring and review procedures to which the programme has been submitted, nor any formal guidelines specifying the requirements of revalidations. The team concluded that there was scope for more detail to be given to providers to assist them when submitting programmes for revalidation. Similarly, the team recommends that additional guidance be available for panel members which clarifies the role of programme monitoring and review in the revalidation process.

Review of Access programmes

75 The AVA requires providers to produce annual reviews of Access programmes, which are used as the basis for the annual report to QAA. Late submission of annual reviews by some providers has caused problems for the AVA in compiling its report and statistics. In order to reduce the effect of this problem, the final deadline for submission of annual reviews has been brought forward from October to July.

76 The AVA's evaluation report of annual reviews (2001-02) comments that '...the better reports reflect self-critically on Access provision...while others did not and more needs to be done to achieve greater consistency in approach.' Although the AVA publishes explicit guidelines for the preparation of annual reviews, the review team noted that the current guidelines did not include reference to responses to moderation reports and processes to address any issues raised, together with the appropriate action plans. The team considers that the inclusion of such requirements would assist in developing the self-critical approach in reviews which the AVA is seeking. The team therefore recommends that the AVA develops its guidance on annual reviews further, to include these areas, and considers the indicative evidence statements provided under Principle 7, criterion 1, in so doing.

77 Significant levels of statistical data are collected and collated by the AVA for the annual report to QAA. While evidence available to the review team provided very little indication of any role played by the AVA's committees in the use of this data in planning and monitoring procedures, it was nonetheless clear from the Action Plan presented for 2002-03 that the data had been analysed and the conclusions had informed that plan. (See paragraph 12, above.)

78 The review team was satisfied that the criteria expressed under Principle 7 had been met by OCNWM.

Conclusions

79 Open College Network West Midlands (OCNWM) serves the predominantly urban areas of Birmingham and the Black Country, as well as more rural areas in Shropshire. It is one of the largest AVAs in terms of the number of learners registered on Access programmes and has one of the highest averages of number of registered learners per provider. The organisation is now a substantial, well-resourced business, which can make forward plans to further develop its activities with a reasonable degree of confidence and security.

80 OCNWM's resourcing is also reflected in well-equipped premises, and relatively generous staffing levels. It is an efficiently managed organisation, and its professional service standards are recognised and appreciated by providers, who receive consistent support from the AVA across a range of areas and activities. The team work ethos within the organisation assists in providing a strength of officer and systems support. The AVA produces clear and well-presented documentation, which provides an appropriate level of detail to support its major processes.

81 The organisation benefits substantially from its relationship with the University of Wolverhampton, from whom the organisation rents its premises. The nature of the relationship is supportive, with the University providing a range of other services of significant value to the AVA. The relationship is defined by an agreement which appropriately separates the different roles of the two organisations, and all parties are clear about their separate responsibilities. The formal document which describes the terms of the relationship needs some minor amendments for the sake of absolute clarity, but the actual independence of the AVA to govern and manage its own business is not in question.

82 Overall, current governance structures are appropriate to OCNWM's needs as an AVA. Recent changes to the AVA's constitution reflect the increasing size and complexity of operation of the AVA over recent years, as well as the professionalism and expertise of its officers. While these changes were necessary for the effective and efficient oversight of organisation's affairs and responsibilities, they now raise some queries about governance, particularly in relation to the role and function of the Council of Members. The necessary changes, while important, are not great in number, and the position may well be clarified as the AVA follows through the consequences of the decisions it has already taken.

83 OCNWM has well-established systems and procedures for review and development. The outcomes of these are integrated in a systematic approach to strategic planning, which includes appropriate consideration of the development of Access provision. The AVA's senior officers are involved in a range of HE widening participation initiatives, although the AVA is aware that it could make greater use of the potential for fuller involvement of representatives from all its HE members in its routine activities, and it will need to develop its links with each of its member HEIs in order to achieve this.

84 The AVA's operational systems and quality assurance procedures are detailed and rigorous, having been subjected to systematic and regular evaluation, review and updating. It is continuing to develop its moderation system and, in considering this particular area, it is likely that the AVA will be faced with issues relating to standardisation. OCNWM needs to give particular attention to this area. While the AVA is to be applauded for its attempts to address the issue through standardisation events with providers and moderators, little real progress has been made and the AVA has not, as yet, been able to achieve its aims in this area, and continuing efforts need to be made here. As important, are the AVA's actions in relation to credit targets for Access, where the current policy leaves implicit the acceptance of substantial variety of practice.

85 There are clear areas and opportunities for further development of Access provision in the West Midlands: OCNWM has the experience and expertise; the organisational and resource capacity; and robust and established systems and structures to be able to make significant contributions towards this.

Commendations

86 The AVA is commended for (i) its provisions for admitting organisations into membership; (ii) its commitment to equal opportunities and the ways in which this commitment is evident in its activities and procedures; and (iii) its internal quality assurance systems.

The AVA licence

Review outcome

87 Open College Network West Midlands (OCNWM) is awarded a conditional renewal of its AVA licence, with conditions to be met by the dates specified below.

Conditions

88 OCNWM's AVA licence is renewed on condition that the AVA:

i revises its standard Access certificate, to ensure that the correct version of the Access logo is used, in accordance with the guidelines provided by QAA (paragraphs 72-73).

Condition to be met by 1 August 2003

Recommendations to the AVA:

89 The review team recommends that the AVA

i completes its review of the constitution and governance arrangements which clarifies the constitutional position of the Council of Members and which:

establishes a common approach to quorum regulations for its committees;

reviews the membership of the FRG;
revises the membership specification for the QAG to make it clear that there should be specific representation from higher education;
considers whether the Chair of the Access to HE Forum should sit as-of-right on the QAG (paragraphs 23 and 24);

ii remove occurrences in the AVA's formal documents where the University of Wolverhampton appears to be afforded specific privileges to avoid any potential misunderstandings (paragraph 28);

iii considers a more overt risk assessment approach in its general strategic planning (paragraph 39);

iv regularly checks the data output from the new data management system to ensure that it is error free (paragraph 46);

v further develops the AVA's policy in relation to the number of credits necessary to achieve the Access certificate, with a view to clarifying the AVA's requirements (paragraphs 52 and 53);

vi gives serious consideration, through the QAG, to the outcomes of the review of the moderation arrangements (paragraph 65);

vii makes available additional guidance for panel members which clarifies the role of programme monitoring and review in the revalidation process (paragraph 74);

viii requires, as a part of its requirements for providers' annual reviews, the inclusion of action plans, which indicate how issues identified though moderation or other means are to be addressed (paragraph 76).

Appendix

Aims and objectives of AVA review

The aims of the system of AVA review are:

i to provide the basis for an informed judgement by the ARLC about the fitness of the AVA to continue as a licensed agency;

ii to promote public confidence in Access to HE as a properly regulated and respected route into higher education by assuring:

the quality and adequacy of AVAs' systems and procedures;
the quality, comparability and range of AVAs' operations;
the adequacy and comparability of AVAs' standards for approval, moderation and monitoring of programmes;
consistency across AVAs in the operation of criteria for the granting of the Access to HE award;

iii to stimulate reflective and self-critical perspectives within AVAs, as an instrument to promote quality enhancement;

iv to provide an opportunity to identify and disseminate good practice of AVA operations;

v to provide a mechanism for ensuring necessary, and encouraging desirable, improvements and developments in AVAs.

The objectives of each AVA review are:

i to examine, assess and report on:

the development of, and changes in, the AVA since its last review or initial licence, and its plans and targets for the future;
the organisation's continuing viability and robustness and the ways in which the AVA demonstrates sound governance;
the efficiency and effectiveness of the AVA's operational and quality assurance systems;
the range and scope of the AVA's activities, and the appropriateness and value of these activities;
the ways in which the AVA approves and monitors programmes and the ways in which these processes take account of the need for consistency and comparability;
the ways in which the AVA satisfies itself of the adequacy and comparability of standards achieved by students gaining the Access to HE certificate;
the evidence available to indicate the AVA's success in achieving its aims and targets;

ii to identify and report on:

strengths and good practice in procedures and operations;
areas which would benefit from further development;
areas requiring attention.

Open College Network of the West Midlands (OCNWM) May 2003

Contents