Foreword
1 The Quality Assurance Agency for Higher Education (QAA) is responsible to the Department for Education and Skills for the recognition of Access to Higher Education courses. QAA exercises this responsibility through a national network of authorised validating agencies (AVAs), which are licensed by QAA to recognise individual Access to HE courses and to award Access to HE certificates to students. The AVAs are responsible for assuring the quality of the individual Access to HE courses which they recognise and the standards of student achievement on those courses.
2 QAA has developed a scheme for the licensing and review of the AVAs, the principles and processes of which are described in the QAA Recognition Scheme for Access to Higher Education in England, Wales and Northern Ireland. The Recognition Scheme is regulated and administered by the Access Recognition and Licensing Committee (ARLC), a committee of the QAA Board of Directors. The ARLC appoints review teams with appropriate experience of Access to HE provision, who undertake reviews of AVAs and report to the ARLC on their findings. The Recognition Scheme includes the detailed criteria applied by the ARLC and by review teams operating on the Committee's behalf in reaching judgements about whether, and under what terms, an AVA licence should be awarded and renewed. These criteria are grouped under the seven principles that provide the main section headings of this report.
3 Following the review of an AVA, a member of the review team presents the team's report to the ARLC. The Committee then makes one of six decisions:
i unconditional renewal of licence for a specified period;
ii conditional renewal of licence with conditions to be met by specified date(s);
iii provisional renewal of licence with conditions to be met and further review visit by specified date(s);
iv suspension of licence until specified conditions are met;
v withdrawal of licence for operation as an AVA;
vi temporary renewal of licence with request for further information by specified date (decision suspended).
4 This is a report of a review of the AVA function of the Open College Network West (OCN West) undertaken by QAA. The Agency is grateful to OCN West and to those who participated in the review for the willing cooperation provided to the review team.
The review process
5 The review was conducted in accordance with the process detailed in the QAA Recognition Scheme for Access to Higher Education in England, Wales and Northern Ireland. The preparation for the review included an initial meeting between OCN West representatives and the QAA Assistant Director to discuss the requirements for the Analytical Account (the Account) and the process of the review; the preparation and submission by OCN West of its Account, together with a selection of supporting documentation; a meeting of the review team to discuss the Account and to establish a draft programme for the review visit; and negotiations between QAA and OCN West to finalise the programme and other arrangements for the review visit.
6 The review visit took place on 5 and 6 March 2002. The visit to OCN West consisted principally of meetings with representatives of OCN West, including AVA officers; members of the Board of Trustees, Quality Committee and Access Forum; moderators for Access to HE programmes; Access to HE programme coordinators from providing institutions; representatives from receiving higher education institutions; and former Access students now studying in HE. The review team also had discussions with members of the joint Strategic Board for OCN West and Gloucestershire Open College Network (GOCN).
7 The review team consisted of Mr Steve Babbidge, Director, South of England Open College Network and Mr Anthony McClaran, Deputy Chief Executive, Universities and Colleges Admissions Service. The review was coordinated for QAA by Ms Kath Dentith, Assistant Director (Access).
8 AVA statistics 2000-2001 (as provided by the AVA in its annual report to QAA for 2000-2001
| Access programmes available |
17
|
| Providers offering Access to HE programmes |
11
|
| Access programmes running |
17
|
| Access learner registrations |
737
|
| Access to HE certificates awarded |
425
|
The AVA context
9 OCN West is currently based in Weston-super-Mare, and serves a diverse metropolitan and rural membership base across the former county of Avon, and in Wiltshire, Somerset and South Gloucestershire, with Access programmes approved by OCN West offered in all areas except South Gloucestershire. The AVA licence is held by OCN West as an integral part of its overall function as an Open College Network, and its recognised Access courses form a substantial part of its business, providing something between a third and a half of all revenue.
Major developments since the previous review
10 The AVA was last reviewed in 1996 in its earlier guise as Western Counties Access Partnership (WCAP), after a deferment from 1994-95 because of serious difficulties experienced by WCAP in relation to staff illness and other internal matters. The report on that review by the Higher Education Quality Council (HEQC) recommended that the AVA's licence be conditionally renewed subject to a review by WCAP of 'the new committee structure...(that it) develop the student tracking system...builds on its existing links with HEIs...implements revised arrangements ...for entry to teacher training...(and) takes effective steps to ensure consistency in the quality of units of assessment in Access Courses...'. The Account does not specifically address the AVA's response to these recommendations, commenting rather on progress against its own statement of aims provided to the HEQC review team in its critical self-appraisal. The review team noted a continuity of concerns during the current review with a number of these issues, which are commented on elsewhere in this report.
11 During the period following the HEQC review, the AVA changed its name to Open College Network West (in 1997-98) in response to wider developments in the National Open College Network (NOCN). The AVA also gave some scrutiny to the development of common GCSE equivalence units for use across the AVA and standardising of unit writing and assignments. A major review of moderation processes was undertaken during 1999-2000 to allow the AVA to issue contracts to its moderators and replace the previous 'good will' model. This year also saw the loss of one of the AVA's three founding member HEIs, Bath Spa College (now University College).
12 The year 2000-01 is seen by the AVA as something of a watershed in its development, with a new constitution and committee structure which had been implemented in response to licensing by NOCN and issues raised by QAA in its scrutiny of the AVA's annual report; the leaving of the then Director; the promotion to acting Director of one of the AVA's officers; the inclusion on the AVA's Management Board of a trustee of the NOCN Board (and her subsequent appointment from December 2001 as Acting Chief Executive); a new Chair of the Board; the commencement and achievement of plans to create an incorporated company, and separation from its former host institution (the University of Bristol); a further revision to the constitution and committee structure in February 2002; and a proposed merger with a neighbouring OCN in Gloucestershire. Understandably, a large amount of time between 2000 and this review has been spent by staff and committee members addressing major structural and organisational issues, which the AVA attributes in its Account to 'poor management and a lack of clarity of the role of the host organisation and the governance structures within the OCN.'
Statistics and trends
13 The AVA's data for the period since the last review shows a reduction from 28 to 17 Access programmes, but a largely static number of Access learners over the period (an average of 745 for the period, with a high of 764 in 1998-99 and a low of 731 in 1997-98), with 737 learners recorded in the most recent report to QAA. This position can largely be accounted for by college mergers and the move to generic programmes with subject specific pathways in preference to the individual programmes, during this period. The proportion of under-21 learners registered dropped from 9 per cent in 1999-2000 to 1.5 per cent in 2000-01, reflecting the AVA's recognition of QAA requirements in the most recent reporting period. The recruitment of learners from minority ethnic groups has held at a broadly consistent level over the period (at 10 per cent of registrations) but the numbers of 'not knowns' reported in this category has increased from 5 per cent in 1996-97 to 13 per cent in 2000-01. There has been a steady decline in the proportion of learners achieving an Access to HE certificate (from 69 per cent of learners enrolled in 1996-97 to 39 per cent in 2000-01) and a recently increasing proportion leaving programmes with no accreditation at all (averaging 41 per cent of all learners reaching the end of their programme in the last two years). The Account provided progression data only for the last two years, which showed a small drop (from 44 per cent to 40 per cent of enrolments) of learners progressing to HE. The recent trends in recruitment to particular curriculum areas within Access programmes are reported as increasing in vocationally related areas and declining in broad-based areas such as humanities.
Principle 1
The organisation has a structure which is based on a partnership of members, including institutions which provide Access to HE programmes and institutions of higher education.
14 OCN West is a consortium of members which includes institutions providing Access to HE programmes and higher education institutions (HEIs). Currently, there are two HEIs (the University of Bristol and the University of the West of England) in full membership, with the Open University South West region acting as a 'supporting member'. OCN West also has 10 further education colleges (FECs) in membership, as well as over 40 members from voluntary, community, non-statutory and private sector providers. All of the AVA's current Access provision is offered through the member FECs. Membership is sufficiently broad and cross-sectoral to ensure, in principle, the AVA's operational viability, and to be able to secure informed decision-making about Access.
15 The conditions for membership are stated briefly in the new constitution which was approved at the AVA's AGM on 26 February 2002. Membership is open to those organisations which subscribe to the AVA's aims and support the use of credit; offer, or wish to provide, or support the provision of OCN-accredited learning opportunities; are willing to support the OCN's activities through providing representatives for its committees and key quality assurance functions; are committed to quality assurance systems and quality improvement; and agree to pay all charges, including membership subscriptions, promptly. The rights and responsibilities of members are set out in more detail in the membership agreement which each member must sign. The agreement gives clear criteria for membership of OCN West and also states the grounds on which organisations might be excluded from membership. It is supported by a service standards document which indicates the service levels to which the AVA is committed, the member of staff responsible for delivering each of these services and the expectations which the AVA holds of its member organisations. This document is admirably clear, and includes a section which specifically deals with the process of applying for membership.
16 As part of the process of establishing its separate legal identity as a company limited by guarantee, OCN West has of necessity made arrangements to provide those services, primarily in the areas of financial and personnel administration, which were formerly the responsibility of the University of Bristol (see paragraph 12, above). The AVA has a lease agreed on its current accommodation until July and is seeking to extend this until December 2002. Contracts are in place with accountants, solicitors and an IT consultancy.
17 The review team was satisfied that OCN West was currently meeting the licensing criteria under Principle 1.
Principle 2
The organisation has governance structures which allow it to discharge its AVA responsibilities securely.
18 Reference has already been made to the new constitution which OCN West adopted at an AGM on 26 February 2002. This constitution sets out the organisation's aims and its legal status as a company limited by guarantee, and establishes a new governance structure. It now sits alongside the Memorandum and Articles of Association adopted by OCN West when it achieved incorporated status towards the end of 2001.
19 All full members of OCN West have, according to the constitution, 'equal rights and responsibilities'. Among these is the right to send one representative to the company's Annual General Meeting, a gathering hitherto described as the 'Council of Members', although that description is not found in the new constitution.
20 The constitution establishes a governance structure which consists of a Board of Trustees and, below it, a Quality Committee and Higher Education Access Committee (HEAC). The constitution also envisages that from time to time informal tutor groups or forums, including an Access Tutor Forum, will meet with the support of the AVA's officers, to consider specific issues of common interest. Such forums and groups have, however, no authority within the AVA's governance structures and their function is purely advisory.
21 In addition, it should be noted that, at the time of the review, there was in existence a so-called Strategic Board composed of the senior members of OCN West and the Gloucestershire Open College Network (GOCN). This Strategic Board is responsible for planning the proposed merger between the two organisations (see paragraph 12, above). Constitutionally, the Strategic Board has no executive authority of its own but reports back to the boards of the two parent organisations. It does not appear in the AVA's new constitution. The review team was informed that if the merger proceeded then it was intended that the constitutional model to be found in the new OCN West constitution would be adopted by the merged organisation.
22 The terms of reference for the Board of Trustees, which are set out in the constitution, include responsibility for the overall policy direction, and financial and business management, of OCN West. Other responsibilities include the oversight of the management of the OCN's operations, monitoring its quality assurance processes and ensuring that it meets the requirements for its licences with NOCN and the QAA. It receives minutes from the Quality Committee and will also receive minutes from the HEAC when the latter begins to meet. The size and balance of the Board's membership are specified in the constitution and the rules for election to membership are set out in the company's Articles of Association. The rules on the balance of membership ensure that HEIs, FECs and the community/adult education/voluntary sectors are all represented: out of a membership of no fewer than nine and no more than 12 elected representatives, at least two must be from each of these sectors and, in the case of HEIs and FECs, it is specified that the representatives must be 'senior'. The constitution specifies that the Board must meet at least three times a year and that the Chief Executive will attend all meetings ex officio.
23 The Quality Committee consists of at least nine members. The Chair of the Quality Committee is nominated by the members of the Committee and attends Board meetings by virtue of the post in an ex officio capacity. The Chief Executive (or his/her nominated representative) is a member ex officio. Rules on the frequency of meetings and arrangements for a quorum are set out in the constitution. The Committee's terms of reference state that it has 'delegated authority from the Board of Trustees' and is responsible for overseeing all quality assurance matters, excepting that relating to QAA-recognised Access to HE provision. Under OCN West's previous constitutional arrangements, the Quality Committee had a remit which included Access and it is one of the noteworthy elements of the new constitution that this is no longer the case, such responsibilities having been transferred to the HEAC. The Quality Committee has a particular responsibility for ensuring that the OCN meets all the requirements necessary for its NOCN licence. It is also responsible for the approval of moderator appointments for non-Access provision. It is envisaged that the Committee will produce jointly with the new HEAC the OCN West annual quality report for presentation to the Board of Trustees and also that, more generally, it will liaise with the HEAC to ensure a consistent approach to quality assurance across all the OCN's activities. In order to facilitate this liaison, the constitution specifies that the Committee's minutes will be sent to the HEAC for information and that its membership shall include at least one member from the HEAC.
24 The remit and terms of reference for the HEAC mirror those for the Quality Committee but in respect of OCN West's activity as an AVA rather than as an OCN. Like the Quality Committee, the HEAC is chaired by the nominated representative from the Board, but the structure of its membership differs in some respects. Whereas the Quality Committee is structured to ensure strong representation from FECs, the community and voluntary sectors, the employment sector and adult education, the HEAC's membership brings together a senior representative from each HE institution in membership of the AVA and three representatives of FECs offering Access programmes. The Access Tutor Forum, although not part of the formal constitutional arrangements of the OCN, has the right to nominate a representative to the HEAC and, as already noted, the Committee is chaired by a member of the Board of Trustees. The OCN's Chief Executive and/or the officer responsible for Access to HE programmes attends ex officio. Again, in furtherance of liaison between the HEAC and the Quality Committee, the former's minutes are to be sent routinely to the latter. It is intended that the HEAC will be responsible for appointing moderators for Access courses. It is important to note that in the future the HEAC will also have responsibility, delegated by the Board, for the final recognition of Access provision, a responsibility which, until the adoption of the new constitution, lay with the Quality Committee.
25 The new constitution sets out a governance structure which, if it operates successfully, should enable OCN West to discharge its AVA responsibilities securely. However, at the time of the review, the new constitution itself was only a week old and one of its key components, the HEAC, was not yet in existence. The terms of reference for each of the committees as given in the constitution are broad and relatively few in number, and may need to be given more detailed expression when the new committees begin to meet and exercise their roles within the new constitutional arrangements. Specifically, further attention should be given to clarifying the respective functions of the Quality Committee and HEAC to ensure that, in the current absence of an explicit quality policy for the OCN as a whole, a consistent approach in relation to quality assurance is taken or that, if inconsistencies develop for what might be very good reasons, then the OCN is able to ensure that the terms of reference of each of its key Committees shows clearly how such developments can be evaluated and approved.
26 It has already been noted (paragraph 20, above) that the new constitution makes provision for informal tutor groups and forums which have no authority within its governance structures but which act in an advisory capacity. The constitution makes specific reference to the Access Tutor Forum, a group which is already in existence and which, the review team was informed, has made significant contributions to the development of Access within the AVA's geographical area. The new constitution appears to recognise the Forum's importance by including a representative as a member of the HEAC. In addition, minutes of the Access Forum are currently routinely sent to the Quality Committee, and it is intended that this will be the case when the HEAC is formed. Further consideration might be given to formalising this practice and to the status of the Forum within OCN West's structures, in order to ensure that best use is made of the Forum's output.
27 The review team concluded that the new constitution was a commendable attempt to put in place a governance structure which would ensure that OCN West's responsibilities as an AVA were properly discharged, and that it provided for the criteria under Principle 2 to be met. However, given that the constitution has only just been approved, there is, as yet, no evidence available that the structures and procedures described will operate as intended, and the team considered that there were some areas of potential confusion about the roles of different bodies in relation to Access to HE. In particular, the team had concerns about the separation of responsibilities between the Quality Committee and the new HEAC, and the possible impact of this on decision-making in relation to Access and the quality assurance of Access to HE provision. The team therefore agreed that it should be a condition for the renewal of the AVA's licence that the AVA reviews its constitution and develops terms of reference for its committees in order to ensure clarification between the functions of the Quality Committee and the HEAC. The review should include consideration of where responsibility for the development of Access to HE lies within the AVA, and, if the Access Forum is considered to have a role in this, the Forum's relationship to the formal committee structures should be more clearly identified to ensure that best use is made of its output.
Principle 3
The organisation is aware of, and in a position to meet, its legal and public obligations.
28 There is no doubt that the process through which OCN West has gone in recent months - negotiating its independence from the University of Bristol; its establishment as a company limited by guarantee; and its adoption of a new constitution - have provided a constitutional basis which is clear and readily identifiable. The Board of Trustees (formerly the Board of Management) has been active in this process. The process of legal detachment from the previous relationship with the University of Bristol has been handled constructively and with full support from the University, and new arrangements to ensure that the operational needs of the OCN with respect to financial and personnel matters are adequately supported have been put in place on a contractual basis.
29 In undertaking these organisational and governance changes, OCN West has addressed the problems which it had identified during a major review conducted in 2000-01. In particular, the AVA was keen to address weaknesses in governance as far as they affected monitoring and planning, and some difficulties in the relationship with the former host organisation, particularly in relation to line management of the Director and the lack of clarity in defining responsibility for the management of the AVA.
30 OCN West had also, at the time of the review, embarked upon a process which was intended to lead to its merger with GOCN. The adequacy of the governance arrangements of any merged organisation will be a matter for a future review but if, as was indicated to the review team, the constitutional and governance arrangements now adopted by OCN West are likely to be those of the new merged organisation, then there would be confidence, in principle, that the legal and public obligations of the new organisation could be met.
31 The processes described above, through which the AVA has addressed the very significant difficulties it had faced, should provide a successful remedy but, as noted previously, the new arrangements are so recent that it was impossible for the review team to arrive at a firm conclusion as to their effectiveness. Nonetheless, the team considered that, on the basis of the evidence available in terms of intended structures and documented agreements, the AVA was aware of, and in a position to meet, its legal and public obligations.
Principle 4
The organisation is able to manage effectively its AVA responsibilities and the structure which supports them.
Aims
32 The aims of OCN West, as set out in its new constitution, include commitments to 'widen participation and promote social inclusion in education, training and learning through flexible credit-based programmes including QAA-recognised Access to Higher Education provision' and to 'meet the regulatory requirements of ... the Quality Assurance Agency for Higher Education'. These aims are set in the context of OCN West's overall mission statement, which is 'to support and promote the widening of access to high quality, flexible education, training and learning in a variety of contexts in its [the OCN's] geographical area, particularly for learners who have benefited least from formal provision'.
Strategic planning
33 The AVA's strategies for achieving these aims are embodied in the OCN Strategic Plan and Operational Targets 2001/2002, a document which has been approved by the Board and was updated in February 2002. This plan identifies four strategic priorities, each of which is broken down into a series of objectives and operational targets. Target dates, resources and staff or committee responsibilities are assigned to each target.
34 Also included in the evidence submitted by OCN West to the review team were the OCN West Action Plan, dated September/October 2001, and a business plan for the period August 2001-July 2002. It is also the case that, because of plans for merger with GOCN, there is also a merger action plan, the fourth version of which was issued on 29 January 2002. The development staff team has recently produced a draft marketing and public relations strategy, with an associated 'action plan'. The review team would recommend that, as the AVA's new governance arrangements become embedded, the AVA should consider clarifying the precise relationship of these various plans to each other and, perhaps, rationalising the number of plans themselves.
35 The current strategic plan makes relatively few references to Access provision, although one Strategic Priority (Priority 4: 'Maintain the current activity and continue to meet regulatory requirements') does include a commitment to 'produce the QAA review document and set up process for [the] review visit' and to 'produce ... the QAA annual report and UCAS requirements [for the Access database] by required timescales'. There is also an operational target within this Priority, which commits the AVA's staffing team to the regular and systematic assessment of OCN West's performance against QAA criteria. These references cannot, however, be regarded as constituting in themselves a strategy for Access provision within the AVA and, although the team was left in little doubt about the importance of Access provision, it did not find clearly stated anywhere an Access strategy against which monitoring of objectives relating to the development and indeed enhancement of Access could be measured. It is true that the Account makes reference to six 'key points of action' adopted by the Management Board in September last year, four of which refer to Access: the successful completion of the QAA review and the retention of the AVA licence; the further support and development of flexibility within Access to HE provision to meet the needs of target groups; support for tutors with particular reference to assessment and delivery methodologies to meet the particular needs of Access provision and external agencies; and consideration of ways in which Access to HE can effectively meet the widening participation agenda locally and nationally. However, it is not clear what status these action points have in relation to the strategic, operating and action plans to which reference has already been made. The review team therefore concluded that it should be a condition of the renewal of the AVA's licence that it establishes systematic procedures for the production and monitoring of a strategy for Access.
36 If OCN West is successful in developing an effective strategic planning process for Access, then the structures which have been adopted in the AVA's new constitution should support mechanisms for effective monitoring, evaluation and report. As has been noted, the AVA's committees include appropriate representation from Access to HE providers and receivers through the Access Tutors Forum and the proposed Higher Education Access Committee. However, further attention will need to be given to developing more detailed terms of reference for the AVA's committees to ensure that responsibilities for these key functions are clearly located within the committee structure and related to the strategic planning documents that the AVA chooses to adopt.
Financial management
37 One of the reasons which prompted OCN West to separate from its host institution was a wish to assume responsibility for, and achieve clarity about, its financial affairs. There is considerable evidence that the structures to achieve these aims are now in place: OCN West is a limited company and has appointed accountants. The new constitution states that the Board is responsible for the overall financial management of the OCN and the Board receives and considers financial information, including profit and loss, balance sheet and cash flow statements. The Board agreed at its meeting in October 2001 that 'a decision needs to be made on the type of regular monthly reporting required' and discussed what format it should take, although there is no record of consideration of financial reporting at the following month's meeting. The review team recommends that the AVA defines the scope and format of regular financial reports to the Board, to ensure that these matters are addressed systematically.
Staffing and operational procedures
38 The activity of OCN West is currently supported by seven staff, the most recent of whom is the newly appointed Chief Executive. In addition to the Chief Executive, there are three development officers, to one of whom specific responsibility for Access provision has been returned after a more generalised sharing of responsibilities during the recent period of acute staffing difficulties. These three officers all report directly to the Chief Executive, as does the Administrative Officer, who is supported in her task by a Clerical Officer and a recently appointed Finance Officer. In addition, the AVA's staff have been supported by the Acting Chief Executive (the Chief Executive of OCN Central England). The AVA intended that, following the appointment of the Chief Executive, there should be a handover period of several weeks during which the Acting Chief Executive would continue to provide support and assist with the induction of the Chief Executive into his new role. Administrative systems will be strengthened by the adoption of the new NOCN software for data management, OPUS.
39 The current staffing structure is itself the result of a review conducted during 2001-02. The AVA believes that staffing is now adequate for delivering its requirements; this may prove to be the case but the review team found some evidence that those who use the AVA's services have not found this to be true in the recent past. The draft marketing strategy produced by the Development Team identifies 'low staff morale' and 'understaffing' as weaknesses and 'staff stress [and] losses' as a threat. Difficulties have arisen, particularly with the length of time taken for Access programmes to receive final recognition, a process which in some cases seems to have taken as long as two years. In view of the difficulties that the AVA has itself identified, and as the adequacy of staffing for reformed structures and processes has not yet been tested, the AVA will need to provide evidence of adequate staffing prior to a successful renewal of its licence.
40 Conscious of the lack of documented procedures to underpin the operations of the AVA in the past, OCN West has recently been using an operations manual from OCN Central England. Clearly this has been of assistance in identifying for staff the sorts of procedures which would be covered by such a manual but it is now appropriate that OCN West produces an operations manual which is specific to the AVA's own context, structures, processes and procedures. The review team considered that the production of such a manual should be a condition for the renewal of the AVA's licence. The development of an AVA-specific operations manual could usefully build on and cross reference with the material in the OCN West Accreditation Handbook, which is comprehensive and clear.
41 The AVA's Account refers to the difficulties which have been experienced in its collection of data and statistics. This has had an impact on the AVA's effectiveness in monitoring its achievements in relation to Access provision and, indeed, even in reporting on them satisfactorily to the ARLC. OCN West has instituted a review of the collection and use of data and hopes that the OPUS system will improve the situation further when it is introduced. Since the review, the AVA has informed QAA that, owing to changed circumstances, OPUS will not be implemented until March 2003 at the earliest. It is therefore a condition of licence that the AVA reviews its manual system for data collection and demonstrates its adequacy for meeting QAA's requirements by the time the AVA is revisited.
42 The review team found evidence that the AVA is regarded as communicating effectively with its members and other stakeholders. OCN West's staff are regarded as accessible and helpful, and the various documents which make up the Accreditation Handbook are comprehensive and clear. Regular mailings from the AVA to its members were instituted at the beginning of this year to reinforce the channels of communication but further improvement in this area will depend upon the strengthening of operational procedures to which reference has already been made. This strengthening will need to ensure that, for example, the long delays in processing of requests for recognition are reduced and that a lack of clarity which is apparent even in matters as fundamental as the formal notification of programme recognition can be avoided.
43 OCN West has an equal opportunities statement which may be found in its Accreditation Handbook. The application of this policy is developed in practice through an equal opportunities checklist, also to be found in the Handbook, which is designed to help tutors who are developing Access courses to be aware of issues of equality in relation to their target groups and to help recognition panels assess whether the needs of target groups have been fully considered. There is also a policy in place which provides for complaints and appeals to be made to the Chief Executive, who will make arrangements for it to be heard by the Quality Committee. The detailed procedures used to support this process need to be updated to take account of the new governance structures (the existing policy refers to the 'Director' and the 'Quality Assurance Sub Committee', for example). The organisation's grievance procedure for staff is based on that which would have applied when staff were employees of the University of Bristol, and is supported by University documentation. There is an evident need for this to be replaced by documentation which reflects the OCN's new status. It is therefore a condition of the renewal of the AVA's licence that it updates its appeals procedures and produces clear grievance procedures, as necessary for its employer status.
44 OCN West has been through an enormously rapid period of change, all of which has demonstrated its capacity to assess self-critically its effectiveness, the risks to its operation and the adequacy of its mechanisms. This has resulted in significant changes to its legal and constitutional status, and its arrangements for governance and management. The AVA now needs to ensure that it embeds within its new structures procedures which will ensure that the processes of monitoring and assessment undertaken within the last eighteen months take place effectively on a continuing basis. The working out in detail of the ways in which, in particular, the Quality Committee and the HEAC will work together in producing an annual quality report for the Board of Trustees will be particularly key in ensuring effective procedures in this area and may in turn help to ensure that OCN West's annual report to QAA will in future meet the requirements of the ARLC. It is important that a drive to ensure the accurate and comprehensive presentation of data relating to the AVA's activities is strongly linked to the development of a strategy for Access with appropriate and clear targets, to which reference has already been made.
45 The review team concluded that, while OCN West had made significant and valuable improvements to its organisational structures, there were areas where this process had not been completed. The team was, therefore, unable to have full confidence in the AVA's ability to manage all of its AVA responsibilities effectively at this stage. In particular, the team considered that licensing criteria under Principle 4 had not yet been met in full in relation to strategic planning; the documentation of essential operational procedures; and data collection, management and analysis.
Principle 5
The organisation is able to assure the quality and fitness for purpose of Access to HE programmes at the point at which they are granted formal recognition.
Programme development
46 Documents scrutinised by the review team confirmed that OCN West had established processes to advise those developing Access programmes about the AVA's requirements with regard to general procedures and programme design. The AVA produces a set of comprehensive, if dated, guidance documents for the development and subsequent initial approval of Access programmes, through the use of specific prompts within a general OCN submission document. The documents submitted with the Account and during the review still referred to the FEFC and its funding framework and 'kitemarked' Access programmes. The team would recommend that these documents be updated as a matter of priority. The AVA's submission documents are supported in the field by its development officers, and the team heard that the input of these officers is well regarded but that on occasions, long delays had been experienced in returning draft submissions (see paragraph 39, above). There was a general acceptance that this was a direct product of the demands OCN West staff had been under in the last 18 months and an expectation that recent changes to staffing structures would ameliorate this.
47 A policy on the use of credit exemption and AP(E)L has been adopted by the AVA (in keeping with other AVA-OCNs) and this is reflected in the submission documentation. It is clear from documents perused by the team, however, that the policy is not uniformly understood by submitting organisations and that potential exists for some confusion at the point learners are recommended for the award of the Access certificate.
48 The team also noted that, other than the adoption of the NOCN standard completion requirement of 16 credits (with 12 at level 3), the AVA did not appear to have a clear policy or common achievement specification for the award of the Access to HE certificate and, as a consequence, successful completion was widely defined across its Access programmes, from 16 to 32 credits. The AVA is itself aware of the possible unequal demands this places on learners and indeed has advised its Access coordinators that this could be unfair to learners. One of the AVA's HEI representatives on the Access Forum is leading a review of this issue, through a comparison of completion requirements across the country as well as within OCN West. The team noted this important development and would expect the AVA to develop a clear policy on successful completion as a matter of priority, as a significant step in ensuring consistency of outcome on Access to HE programmes.
49 The submission documentation for initial recognition requires the explicit involvement of representatives with relevant expertise working in higher education, including the signature of one or more 'HE link tutors' who have been consulted during the development process. The documentation is unequivocal in confirming the AVA's expectation that progression links and opportunities should be clearly stipulated.
50 The review team noted in a number of documents that OCN West staff were permitted to author programme submissions and that a consultancy practice, with charges of '£350 per day pro rata...(for) authorship by OCN West staff or programmes and/or units...to include...time spent writing' is available. On talking to development staff, the team heard that their involvement in the drafting of course documents for Access to HE programmes entailed no more than the usual iterative process of providing feedback to submitting organisations on their drafts. The team would therefore expect the AVA to make clear in its published documentation that the provision of bespoke authorship by AVA officers of course documents does not extend to Access to HE programmes.
Programme validation
51 The review team heard from the development team that it is a requirement that HE staff are represented on an initial recognition panel for an Access programme and this would normally include the HE link person referred to above (paragraph 49). It was not clear from documents scrutinised, nor from discussions with AVA officers, however, how the AVA ensured that there was relevant curriculum expertise at a recognition panel for the programme being validated. The AVA's processes and documentation do not appear to stipulate a requirement for a person from HE with relevant curriculum expertise to be present at a panel, nor the means by which such attendance may be monitored. Nor is it clear by what means a recognition panel is guided to reach a judgement about the 'fitness for purpose' of the Access curriculum under consideration. The review team considered that the AVA should address this and the agreed procedure be clarified in its validation process documentation. In talking to development officers and Quality Committee members, the team was further informed that some scrutiny of the curriculum occurs through the individual actions of members of the Quality Committee, thus confusing the role of such a committee in its scrutiny of validation output.
52 The AVA's guidance documentation for panels allows for Chairs to be drawn from its two main committees (although it uses obsolete nomenclature) and from AVA officers, and it is common AVA practice to use development staff as Chairs. There did not appear to be guidance documentation prohibiting the chairing of panels by AVA staff involved in the development of programmes under scrutiny, and development officers confirmed that this had happened on occasion. Although this was presented as happening only in adverse circumstances, the team noted the absence of a clear set of procedural guidelines and would expect the AVA to clarify this in an early review of its validation process documentation.
53 The AVA may also wish to consider when conducting a review of its guidance documentation, whether the integrated approach to OCN and Access validation provides sufficient focus on the requirements of Access. One area, for example, in which a distinction would be particularly important would be in appropriate guidance to panel members to consider the curriculum on Access programmes in relation to its fitness for the purpose of Access to HE.
The locus of authority for the recognition of Access to HE programmes
54 In talking to the Acting Chief Executive, the review team heard that the Quality Committee operated with delegated authority in approving an Access to HE programme, ultimate authority therefore resting with the Board. Other staff and committee members spoken to were not so clear, however, whether a decision reached by the Quality Committee could be overturned by the AVA's Board. The AVA's guidance documentation is also unclear in that it makes no reference to Access panel decisions having the status of recommendations to the Quality Committee, and instead notes that 'The (panel) report will be sent to the Quality Assurance Committee (sic) for discussion of any issues that might have arisen.' Nor was it clear to the team from its scrutiny of Quality Committee minutes and other documentation, how the formal approval process was confirmed and subsequently actioned to ensure, for example, that any conditions attached at panel are met prior to a programme commencing. Although it was clear to the team that officers and most committee members spoken to appreciated the need for such clarity, the AVA's procedures and supporting documentation had not yet reached the stage that they reflected this, and the team would expect the AVA to clarify this in its next review of its validation process documentation.
55 The review team considered that while in outline the process for Access programme recognition was satisfactory, there were weaknesses or areas of uncertainty in a number of particulars and these weaknesses made the process vulnerable. While the team considered that some matters could be addressed by updating and clarifying documentation, some would need more fundamental consideration by the AVA before all criteria under Principle 5 could be seen to have been met.
Principle 6
The organisation is able to safeguard the continuing quality of Access to HE programmes, and to secure the standards of achievement of students awarded the Access to HE certificate.
Moderation processes
56 OCN West has spent a significant amount of time in the last 18 months overhauling its moderation model, and it has established a system of regular external programme monitoring and assessment through which the quality, comparability and fitness for purpose of Access to HE programmes, and the consistency and sufficiency of standards of student achievement are, to some extent, addressed. One of the AVA's officers has half of her full time post allocated as 'moderation officer' to support the processes. The review team noted several areas, however, that may warrant further attention with regard to OCN West's function as an AVA.
57 The AVA issues composite OCN/AVA guidance documentation to its moderators and, as a more recent development, contracts, outlining their duties and responsibilities. The documentation is comprehensive in its explanation of most moderation processes but makes little explicit reference to Access, either by way of explaining the role and function of a final assessment board, or of the nature of scrutiny of academic standards/learner achievement on Access programmes. The only specific reference to guide Access moderators is a requirement on them to record the names of any learners under 21 years in their report to the AVA.
58 Although the review team heard from AVA officers, committee members and moderators that final assessment boards were held to determine the award of the Access to HE certificate, there would not appear to be any formal documentary guidance to tutors, college staff or moderators which ensures consistency and equity of practice. In order to ensure this, the team would expect the AVA to review its moderator guidance documentation to ensure that this is in place.
Moderator selection, appointment and induction/training
59 According to the recently adopted constitution, moderators are formally appointed by the Quality Committee. This was not clear in either the previous constitution, or in the minuted business of the Quality Committee. Current application forms, guidance documents and contracts do not make clear how and where the Quality Committee exercises this authority and the review team would expect the AVA to clarify this, and amend its procedural and guidance documentation accordingly.
60 The AVA invites moderators to attend the validation of the programme they are to validate and offers training in support of their role. There is a recognition of the importance of training which the Account describes as '...a key activity in developing excellence in this area of work and this has been developed significantly over the (last two years)'. The review team noted, however, that although the moderation contract specifies attendance at training events as an expected duty, the AVA recognised it still had some challenges in ensuring that all moderators attend relevant briefings. The team would recommend that a robust policy on the monitoring of both current and new moderators' contractual obligations be adopted to enable the perceived benefits of training to be achieved.
Monitoring and responding to moderation output
61 Moderators' reports are received by the AVA, where they are scrutinised by AVA officers and issues of concern referred on to the Quality Committee. The Committee considers these issues under a standing agenda item and is advised by AVA officers of both issues and progress made in addressing them. The review team noted that although this process appeared clear to all concerned, a number of instances had been identified in the scrutiny of moderator reports and committee minutes, where actions had been reported but not followed through. Although the AVA is clearly identifying issues raised by its moderators and communicating these to institutions, greater attention to detail is required in the monitoring of actions taken (or not taken) by providing institutions in order that the AVA is able to exercise its authority as an awarding body in full. The team considered that this could be achieved by a more systematic use of an appropriate reporting tool to the Quality Committee.
The award of the Access to HE certificate
62 In considering the adequacy of learner achievement for the award of the Access to HE certificate, moderators reach their decisions about the whole cohort based on a sample, the size of which is determined by the AVA according to a set formula. The award of an Access to HE certificate is confirmed by the moderator at a final assessment board and via the use of standardised proformas. Tutors are obliged to check that each individual learner's entitlement to the award is matched against the completion criteria for the programme and recorded accurately on the AVA's paperwork. The moderator cross checks this and confirms its accuracy through her or his signature. Issues concerning the means by which an assessment board might reach an equitable and consistent judgement at the final assessment board have been noted above (paragraph 58).
The issue of Access to HE certificates
63 The AVA issues separate Access certificates and credit transcripts to successful students, consistent with QAA's requirements. In addition to the 'front line' checking of individual entitlement to the award referred to above, the AVA's administrative team cross checks all signed proformas against the original programme specification as the current data system does not allow automated checking. The review team would anticipate that when OCN West migrates to the new data system (see paragraph 41, above), some automation will be built into operational practice. The team also noted that where administrative staff were uncertain in this process, advice from AVA officers could be sought. This line of clarification and authority was not clearly detailed through an operational procedures document specific to the context of OCN West, however, (see paragraph 40, above) and the team would expect clear and detailed procedures relating to the issue of Access to HE certificates to be included in such a document.
64 The review team concluded that the moderation process itself included appropriate procedures by which the quality and standards of Access programmes could be assured, and recent developments to moderation had brought significant improvements to the AVA's management of the process. Some matters need further clarification or more detailed specification, however, for all criteria under Principle 6 to be fully met.
Principle 7
The organisation is underpinned by structures and processes which enable it to review, evaluate and develop the Access to HE provision for which it has responsibility.
Monitoring the quality of Access programmes
65 As mentioned above, the AVA operates systems for the monitoring of its Access provision through its Quality Committee. This function will, under the recently adopted constitution, pass to the HEAC (see paragraphs 23-24, above). The need for clarification between this body and the Quality Committee is commented on elsewhere in this report, but there are some specific matters relating to the monitoring of Access to HE provision which will need particular consideration by the HEAC.
66 The AVA makes clear through its submission documents that Access programmes should be specifically targeted at those under-represented in HE, and its moderation guidance requires commentary that learners recruited to programmes should be those detailed in the definitive programme document. The Account makes reference to a range of data evaluated by the AVA in determining a position on targeting and connects this to wider policy issues. The review team was not clear, however, how far guidance documentation and observations had been transferred into a clear policy objective to ensure Access is offered to those who are deemed to need it most. The AVA itself recognises that it needs to develop its data capture and analysis techniques and the team recommends that this be tackled in parallel with the development of a clear targeting policy which can be effectively monitored.
67 Data capture and its analysis to inform planning at a more general level represent an area of significant weakness. While this is acknowledged by the AVA, a number of questions posed by the data sets available to the review team are left unanswered. As an illustration of this, the steady decline in the proportion of learners achieving an Access to HE certificate (from 69 per cent of learners enrolled in 1996-97 to 39 per cent in 2000-01) and a recently increasing proportion leaving programmes with no accreditation (averaging 41 per cent of all learners reaching the end of their programme in the last two years) is attributed in the Account to variations in full and part time modes of attendance, but without substantiation. Nowhere would there appear to be the translation of the recognition of a possible problem identified through analysis of available data, into a specific action. The review team would concur with the AVA that '...it must develop the depth and range of statistical reporting...' and would add that this should be underpinned by a clear communication to its members that data reporting is an obligation, and that evaluation leads to planning.
68 The review team was equally unclear as to the means by which the AVA was able to connect the output from its annual report to QAA with its own planning and review process for Access. The annual report to QAA would appear to be the output of the most significant evaluation of Access provision in the AVA, but there did not appear to be a clear link between assertions made in the Account, those made in the annual report and key targets established for Access. In each there was a disconnection between data, evaluation and planning and the team would concur with the AVA that 'OCN West has identified a significant weakness in its collection of data and statistics...(and)...that the standards of the Annual Report to QAA are currently not acceptable.' The team is assured that in reaching this recognition and planning action to address weaknesses, the AVA is taking the necessary steps to develop current practice and would welcome confirmation that appropriate milestones have been met.
The revalidation of Access programmes
69 As noted above (paragraph 51), the review team was assured of the involvement of staff from HE in the initial design and validation of Access programmes, but it would appear that, once an Access programme is initially validated, it is possible to add new pathways through the use of 'unit panels', where units are judged both in isolation from the programme as a whole and without the explicit requirement for involvement by staff from HE, thus creating a situation of uneven scrutiny. The detachment of the method of assessment (the unit) from the curriculum and delivery structure, and those most able to judge its fitness for purpose as a preparation for entry to HE (HE staff), through the use of unit panels would appear to allow a structural inequality in the process used and validation outcomes reached in post initial approval panels. The team would expect the AVA to review this practice to ensure consistent and equitable scrutiny of additions to programmes and programme reviews.
70 The AVA operates a process of 'quinquennial programme review', for which the Access course team is required to submit a 'review report'. It was not clear how the process, as described, would take account of the outcomes of programme monitoring in order to build on and enhance the initial validation process, and the review team noted that some Access practitioners were uncertain about the purpose or value of programme review. The process does not carry the same status as an initial recognition panel and the outcomes of review panels are apparently merely 'noted' for report to the Quality Committee. The Account asserted, however, that the Quality Committee was 'very involved' in the programme review process, explaining that two members 'scrutinise newly reviewed documents in addition to the documentation that goes to panel'. The team considered that the nature of the Quality Committee's involvement, as with its involvement in the process for initial recognition (see paragraph 51, above), appeared to confuse the Committee's role with that of the panel. This confusion will need to be resolved in order to ensure that the new HEAC is clear about its responsibilities.
71 The review team concluded that the AVA's structures and processes for the review, evaluation and development of the Access to HE provision for which it has responsibility were under-developed, and that the AVA should review this area of its work in order to ensure that the criteria under Principle 7 were met in full.
Conclusions
72 Open College Network West (OCN West) suffered an extended period of weak governance and poor management, exacerbated by an uncertain relationship with its host institution, for much of the period since the last review. As a consequence, its position appeared precarious and its future uncertain until very recently. The organisation has now, however, not only recognised but confronted its problems and, with outside intervention and internal support from its members, has effected some significant changes in the last year, with rapid and real development in recent months.
73 The scale of change required - to its legal status; its constitution; its governance structures; its staffing and management; its operational procedures; its quality assurance arrangements - meant that OCN West had to prioritise its activities and, quite properly, prioritised the broader structural changes needed to prevent organisational collapse. The inevitable result is that other areas in which development is also much needed have, as yet, not been addressed or have been addressed only in outline. The AVA is aware that its work is not yet complete and is continuing to address areas of weakness, but this will need sustained and systematic work in order to provide sound quality assurance in all areas of the AVA's responsibilities for Access. There are weaknesses throughout its procedures for the recognition and monitoring of Access programmes, but the current picture is somewhat uneven and reflects the resources and attention which the AVA has devoted to different areas: moderation, for example, has been recently reviewed and procedures revised and, as a result, the process is largely sound; programme review and revalidation, on the other hand, is uncertain and unreliable. Documentation in some areas is in need of revision and updating and there are some absences, and as a result, there is some lack of clarity about certain of the AVA's procedures among those responsible for applying them. In addition, there are severe weaknesses in data collection and management, and, as a result, there has been little evaluation of the AVA's operation or the success of the Access programmes.
74 OCN West is heavily reliant on Access as a substantial element of its business, but it would appear that, as the organisation weakened over several years, so that core business stagnated, and the AVA has not been able to develop the business either in volume or variety. This appears to be as much a result of the AVA's view of its role as any insufficiency of funding. There has been little sense of the AVA as a proactive organisation seeking and encouraging new areas for development, perhaps illustrating the absence of effective leadership within the organisation, and this is reflected in an absence of strategic aims and undeveloped planning procedures for Access to HE. The AVA is aware that it is currently 'punching under its weight' and the appointment of a new Chief Executive and new Board of Directors provides an opportunity for attitudinal change and re-direction within the organisation. The AVA also now enjoys a strong relationship with HE, and has derived significant support from this sector of its membership, particularly in recent months. In addition, some current initiatives demonstrate the potential benefits of partnership for both the AVA and the HEI in the wider aim of improving access to higher education for adults from under-represented groups. It remains to be seen whether the organisation's ambitions for growth build on this work to the benefit of Access to HE provision.
75 At the time of the review, many new structures and procedures had been barely implemented, still less embedded, tested or evaluated. There are also some areas which have received very little attention and where action by the AVA is still needed. It is not, therefore, possible at this stage to have full confidence in the AVA's management of the quality assurance of the Access courses for which it has responsibility. Nonetheless, the work done to date has established a sound framework for further development and there is evidence that the AVA has proceeded rapidly and effectively and has made considerable progress. The largely appropriate and well-considered solutions to the AVA's difficulties, together with evident commitment of members and staff to take this work forward, provides some confidence in the organisation's ability to sustain and build on these developments.
Commendations
76 The AVA is commended for the clarity and detail of its service standards document.
The AVA licence
Review outcome
77 Open College Network West is awarded a provisional renewal of its AVA licence, with conditions stipulated below to be addressed by the dates specified. The AVA will be revisited in December 2002.
Conditions
78 The AVA's licence will be renewed on condition that the AVA:
i reviews its constitution and develops terms of reference for its committees, in order to ensure clarification between the functions of the Quality Committee and the Higher Education Access Committee, and amends procedural and guidance documentation, as necessary (paragraph 27);
ii establishes systematic procedures for the production and monitoring of a strategy for Access and ensures that responsibilities for planning and development functions are clearly located within the committee structure and related to the strategic planning documents that the AVA chooses to adopt (paragraphs 35 and 36);
iii ensures that the level of staffing is adequate to meet its AVA responsibilities in full, and demonstrates this in a summary of Access-related responsibilities and the allocation of these responsibilities to specific posts (paragraph 39);
iv produces an operations manual for OCN West which is specific to the AVA's context, structures, processes and procedures (paragraph 40);
v reviews the adequacy of data collection systems and demonstrates their adequacy for the requirements of AVAs' statistical reports to QAA, and considers the need to interrogate and evaluate the data on which it must report to QAA when reviewing the AVA's staffing (paragraph 41);
vi provides clear appeals and grievance procedures through the production of:
- updated procedures for appeals and complaints lodged by members in respect of the AVA's decisions, and in respect of academic and academic-related appeals lodged by students registered for programmes validated by the AVA; and
- procedures for grievances and complaints lodged by its employed staff (paragraph 43);
vii establishes a policy and rationale for credit targets for Access programmes recognised by the AVA (paragraph 48);
viii confirms that it does not offer a consultancy service which provides authorship for Access to HE programmes and/or units by its own staff, and ensures that this position is reflected in its published documentation (paragraph 50);
ix reviews procedures for validation and makes revisions, as necessary, to its validation process documentation, which:
- stipulates the attendance at Access to HE recognition panels of HE personnel with curriculum expertise to assess and advise on the appropriateness of the programme's curriculum as a preparation for HE, and identifies the means by which such attendance is monitored;
- clarifies the means by which a recognition panel is guided to reach a judgement about the 'fitness for purpose' of the Access programme under consideration;
- ensures that development officers are not permitted to chair recognition panels for programmes for which they have had development responsibility; (paragraphs 51-53);
x clarifies the procedure for the final approval of programmes to ensure that the recognition of an Access to HE programme is made with the full authority of the AVA, and produces systematic procedures for notification to the provider of programme recognition, which ensures that any conditions attached at panel are satisfied before final approval has been granted by the AVA (paragraph 54);
xi reviews its moderator guidance documentation to ensure consistency and equity of practice in the function and conduct of final assessment boards (paragraph 58);
xii reviews its procedures for the review, evaluation and development of Access provision and, in particular:
- develops systematic procedures for addressing and monitoring the outcomes of external moderation (paragraph 61);
- extends the depth and range of statistical collection, analysis and reporting in order to improve its capacity to monitor current Access to HE provision and its ability to plan for the future (paragraphs 66 and 67);
- reviews the practices of unit approval and programme review to ensure consistent and equitable scrutiny of additions to programmes and programme reviews with initial validation (paragraphs 69 and 70).
Conditions to be met by 1 November 2002
Recommendations
79 The review team recommends that the AVA:
i clarifies the precise relationship of its various plans for development to each other and considers rationalising the number of plans themselves (paragraph 34);
ii defines the scope and format of regular financial reports to the Board (paragraph 37);
iii updates its submission guidance documents to reflect current funding and policy contexts, in particular, the replacement of the FEFC by the Learning and Skills Council, and the withdrawal by the QAA of its use of the term 'kitemark' in 2001 (paragraph 46);
iv adopts a robust policy on the monitoring of current and new moderators' contractual obligations to enable the perceived benefits of training to be achieved (paragraph 60);
v confirms that milestones in raising the standard of the annual report to the QAA have been met and that connections are made to the AVA's planning process (paragraph 68).
Top