Foreword
1 The Quality Assurance Agency for Higher Education (QAA) is responsible to the Department for Education and Skills for the recognition of Access to Higher Education (HE) courses. QAA exercises this responsibility through a national network of authorised validating agencies (AVAs), which are licensed by QAA to recognise individual Access to HE courses, and to award Access to HE qualifications to students. The AVAs are responsible for implementing quality assurance arrangements in relation to the quality of Access to HE provision and the standards of student achievement. QAA has developed a scheme for the licensing and review of AVAs, the principles and processes of which are described in The QAA Recognition Scheme for Access to Higher Education in England, Wales and Northern Ireland. The Recognition Scheme is regulated and administered by the Access Recognition and Licensing Committee (ARLC), a committee of the QAA Board of Directors.
2 The ARLC is responsible for overseeing the processes of AVA licensing and periodic review and relicensing. The criteria applied by the ARLC and by review teams operating on the committee's behalf, in reaching judgements about whether and under what terms an AVA licence should be confirmed or renewed, are provided within the Recognition Scheme documentation. These criteria are grouped under the four principles that provide the main section headings of this report.
3 Following an AVA licence confirmation review, a member of the licensing team presents the team's report to the ARLC. The committee then makes one of four decisions:
i confirmation of the licence for a specified period
ii extension of the provisional licence with conditions to be met and further review visit by specified date
iii withdrawal of the provisional licence for operation as an AVA
iv temporary renewal of the provisional licence with request for further information by specified date (decision suspended).
4 This is a report on the licence confirmation review for Open College Network (OCN) Wales. QAA is grateful to OCN Wales and to those who participated in the review for the willing cooperation provided to the team.
The licence confirmation review process
5 The licence confirmation review was conducted in accordance with the process detailed in The QAA Recognition Scheme for Access to Higher Education in England, Wales and Northern Ireland. The preparation for the event included an initial meeting between OCN Wales representatives and QAA's Assistant Director to discuss the requirements for the Overview Document (the Overview) and the process of the event; the preparation and submission by OCN Wales of its Overview, together with a selection of supporting documentation; a meeting of the review team to discuss the Overview and supporting documentation and to establish the main themes and confirm the programme for the review; and negotiations between QAA and OCN Wales to finalise other arrangements for the review.
6 The licence confirmation review visit took place on 26-27 January 2006. The visit to OCN Wales consisted principally of meetings with representatives of OCN Wales, including AVA officers; members of the Board of Directors, the Quality Assurance Committee and Access to HE Development Committee; moderators for Access to HE programmes; and Access to HE coordinators.
7 The review team consisted of Mr Steve Babbidge, Chief Executive and Regional Director, Open College Network South East Region, and Dr Peter Easy, Vice-Principal (Academic Quality and Customer Service), Napier University Edinburgh. The review was coordinated for QAA by Kath Dentith, Assistant Director (Access).
The AVA context
8 OCN Wales was created on 1 October 2004 through the merger of the three OCNs then operating in Wales (South East Wales OCN (SEWOCN), South West Wales OCN (SWWOCN) and North Wales OCN (NWOCN)) At the time of the merger, SEWOCN and NWOCN also held AVA licences. Prior to the merger, an application was made for a provisional AVA licence for the proposed single organisation, based on the practices and processes of the two licence-holding AVAs.
9 The provisional licence was granted in September 2004, following scrutiny by the ARLC of an initial outline and a subsequent full application. A number of conditions were attached to the award of the provisional licence, including the amendment of some aspects of governance, a clarification of the processes for the development, validation and approval of Access to HE programmes, the introduction of a credit specification for Access to HE programmes, and a statement of the confirmed staffing structure of the AVA. These conditions were met in November 2004 and the provisional licence was confirmed.
10 The merger of the three OCNs into one organisation has created, in effect, a national AVA covering the whole of Wales. On its establishment, OCN Wales retained three 'regional' offices in Bangor, Cardiff and Llanelli but, since October 2005, has rationalised its operations and works from two centres in Bangor and Cardiff. At the time of the review, there were plans for further office relocations designed to increase the AVA's efficiency and to ease the difficulties which it experiences in travel and communication across its large operational area.
11 Prior to the scrutiny for the provisional licence, the individual AVAs which now comprise OCN Wales had been subject to licence review by QAA. The licences for SEWOCN and NWOCN were renewed following reviews in July and November 2000 respectively. Reviewed in July 2000, SWWOCN surrendered its AVA licence in August 2002 and did not thereafter pursue a further renewal.
AVA statistics (2004-05)
Providers offering Access to HE programmes 23
Access to HE programmes available 97
Access to HE programmes running 74
Access to HE learner registrations 2,184
Access to HE certificates awarded 1,071
12 In 2004-05, OCN Wales had 80,000 learner registrations of which just over 2,000 were for Access to HE courses (of these, about 50 per cent were following health-related programmes). Given the relatively recent establishment of the AVA from three individual organisations, year-on-year statistical comparisons are problematic. However, between 2003-04 and 2004-05, there was an increase of about 5 per cent in the number of enrolled Access to HE students. In the same period, the proportion of students successfully completing (based on those who were in a position to complete) has remained level, although the number of Access to HE certificates awarded rose from just over 900 to just under 1,100. The AVA recruits above the representative proportions for ethnic groupings in Wales, although the actual numbers of students remain very low (just over 4 per cent of total registrations in 2004-05).
Principle 1
The organisation has governance structures which enable it to meet its legal and public obligations, to render it appropriately accountable, and to allow it to discharge its AVA responsibilities securely
Legal identity and membership
13 OCN Wales is a company limited by guarantee and a registered charity. By definition, the OCN thus has a readily identifiable legal identity, and its Memorandum and Articles of Association formally describe the functions, aims and structures of the organisation. On the establishment of the OCN in 2004, a small number of subscribers became the original Trustees of the company and formed a transitional Board of Directors. These arrangements were intended to be temporary and they have now been superseded by the permanent governance arrangements that are set out in the company's Articles of Association and are described below.
14 Membership of OCN Wales is open to any organisation willing to support and promote the objects of the company and currently falls into three categories: further education (FE) colleges; HE institutions; and a third grouping which includes voluntary organisations, private companies and schools. At the time of the review, there were approximately 350 members of whom 23 were providers of Access to HE programmes. Virtually all HE institutions located in Wales are in membership.
15 The responsibility for considering new applications for membership is included in the remit of the Board of Directors. The membership of the AVA has not changed since its establishment in October 2004 and there is, therefore, no direct evidence of the operation of the system to admit new members. However, the Articles of Association of the company set out clearly the basis on which membership may be granted and terminated.
16 The OCN's FE and HE members account for most of its Access to HE providers. In view of the very large number of other members, the OCN may wish to consider whether there is further potential for delivery of Access to HE provision among some of these members, as a means of promoting and developing Access to HE opportunities.
Governance structures
17 It is the responsibility of the General Council of Members (in effect, an annual general meeting of the company) to elect the members of the Board of Directors. (The OCN itself uses interchangeably the titles 'Board of Directors' and 'Executive Board': in this report, it is referred to as 'the Board'). The Council comprises one voting representative from each member and, as well as acting as an electoral college, is responsible for approving amendments to the company's Memorandum and Articles of Association or any other changes in its governance structure.
18 Other than these retained responsibilities, the Council delegates all matters of governance and operations to the Board. The Board maintains three committees to assist it in conducting the business of the AVA: a Quality Assurance Committee (QAC), a Finance and General Purposes Committee (FGP), and an Access to HE Development Committee. There is also a Personnel Sub-Committee, which reports to the FGP. Given its extensive responsibilities, the Board is the key committee in the structure of the AVA and is described as 'the locus of authority for the governance of the organisation'. Its terms of reference include responsibility for policy and strategy formulation and monitoring, proper operation of the committee structure, all legal and financial controls, and acting as the employing body. Included in these terms of reference is a clear acknowledgement that the Board is responsible for ensuring that 'the strategic direction and operation of OCN Wales is in accord with its licence obligations from...QAA'.
19 The Board has met regularly since the establishment of the AVA and, from the evidence of its minutes, has managed the creation of the merged organisation - and the continuing business of that organisation - in an efficient and effective manner. Inescapably, perhaps, in the context of a recent merger, the Board has also experienced some problems, including a number of meetings being inquorate. The Overview prepared by the AVA discusses these matters and makes clear that there have been active attempts to overcome them, including the increased use of video-conferencing.
20 The review team also noted that the AVA had introduced changes to the constitution of the Board since the granting of the provisional licence. One of the original licensing team's concerns, which had been discussed at the licensing meeting with the OCN's representatives, was the number of co-options permitted to the Board. This concern was addressed by the OCN in its response to the conditions for the award of the licence by a reduction in the number of permitted co-options. However, the OCN has recently moved from a position in which the Board was composed of 15 elected members and three possible co-options, to one in which there are now 20 members of which nine are elected and 11 may be co-opted. Co-opted members are also given full voting rights. In addition, the quorum for meetings of the Board has been reduced and is now any one third of its membership.
21 Purely in terms of standard principles of governance, the review team did not consider it appropriate that unelected and co-opted members might possibly form a majority of the Board's membership. In addition, the current regulations would permit a portion of the co-opted members to form a quorum without the presence of any elected members. Such occurrences may appear to be hypothetical although, in the past year, the Board has met with as few as five members present. In discussion with the AVA, the team heard that this increase in co-options was a response to a wish to increase the stakeholder involvement in the work of the AVA in a context where such stakeholders were not formal members. An increase in such involvement is clearly desirable, but the team considered that this could be achieved without the extensive co-option on the Board which is currently permitted. The criteria for holding an AVA licence call for governance arrangements which ensure that the organisation is protected from any possibility of undue influence by its stakeholders. In order to fulfil this expectation, it is a condition of licence that the AVA reviews and revises its regulations for quoracy and co-options in respect of its Board of Directors and sub-committees to ensure that unelected members alone are unable to form a quorate meeting.
22 Elsewhere, the quoracy of committees has been a problem for the AVA. To some extent, this problem is self-imposed. The AVA's preference for relatively small committees, with relatively high quorum requirements, almost inevitably leads to inquorate meetings especially given the geographical context in which the AVA operates. In particular, the QAC has a maximum membership of nine and a quorum of six. While recognising that the committee is not yet at full strength, the review team noted that it has rarely had a quorate meeting in the last year. In the view of the team, this has not significantly affected the quality of the work undertaken by the committee. However, it is clearly not desirable for such a committee to operate in this context for the long term. In undertaking the required revision of quorum and co-option regulations discussed above, the AVA may wish to consider, in particular, the size and quorum of the QAC.
23 The QAC is responsible for all major quality-related processes within the AVA, including programme validation, moderation and annual review. As required by the AVA licensing criteria, the QAC retains direct authority in the approval of programmes which have been recommended to it through a panel process. In exercising its other responsibilities, greater reliance is placed on 'highlight' or synoptic reports prepared by officers of the AVA for scrutiny by the QAC. In discussion with members of the QAC, the review team heard that full copies of individual provider reviews or moderator reports could be called for, although this had not yet been required. The team also noted that members of the QAC had recently undertaken a thorough 'internal audit' of Access to HE arrangements and heard that a similar audit function might well be incorporated into the standard operating procedures of the QAC.
24 From its establishment, it has been the intention of OCN Wales to create an Access to Higher Education Development Committee (AHEDC) within its governance structure. The remit of the AHEDC gives it an important role in advising the Board (to which it reports directly) on strategic planning and the general development of Access to HE. The AHEDC also has a brief to collaborate with the QAC on matters of quality and standards within Access to HE provision. However, at the time of the review, the AHEDC had met on only one occasion. The Overview was open in expressing the disappointment at this situation within the AVA. It highlighted the difficulties which it had experienced in securing the participation of members from HE institutions as a particular obstacle to the successful formation of the AHEDC. This is a more general issue within the AVA leading, for example, to the QAC having to use co-option to obtain a representative from HE and to a general reluctance on the part of HE members to participate in validation panels in some parts of the country. However, having identified the issue, the AVA has recognised that further efforts need to be made to secure the involvement of HE in its work and this will form a more substantial part of the work of the Access Co-ordinator in the near future. The review team also noted that, notwithstanding the single meeting of the AHEDC, both the Board and the QAC had been active in the discussion of Access to HE.
25 Three Access Tutor Forums, matching the areas covered by the three former Welsh OCNs, are facilitated by the AVA. Although they do not form part of the formal governance structure, notes of their discussions are passed to the AHEDC to inform its work. The review team noted that the Forums had met relatively frequently and had covered a range of topics including discussion of some operational processes, the dissemination of information, and general debate on Access to HE matters. The Forums have also found it difficult to attract members from HE although a recent decision has been taken to change their title to the simpler Access Forum to encourage more participation. In the team's view, the Forums were undertaking valuable work on behalf of the AVA and, particularly given the difficulties in establishing the AHEDC, the team would wish to encourage their continuation and, in due course, the creation of strong links between them and the AHEDC.
26 In summary, the review team concludes that the AVA meets almost all of the requirements expressed through the criteria for Principle 1, but will need to review and revise its regulations for quoracy and co-options in respect of the Board and sub-committees.
Principle 2
The organisation is able to manage its AVA responsibilities effectively, and to maintain an appropriate structure to support them
Aims and targets in relation to Access to HE provision
27 As a limited company, OCN Wales expresses its general educational aims through the company's objects. These emphasise the promotion and widening of participation, particularly among those who may have been previously excluded; the improvement of the quality and flexibility of educational opportunities; and the facilitation of progression in FE and HE. These objectives are consonant with the principles of The QAA Recognition Scheme for Access to Higher Education, from which are derived the criteria for the licensing of AVAs. The review team noted that these general objectives also clearly informed the more specific targets contained in the AVA's current strategic plans for its Access to HE activities.
Strategic planning
28 As a relatively new organisation, OCN Wales is still developing its strategic planning processes. While the framework for the formulation and monitoring of strategy is clear in theory (it is firmly within the responsibilities of the Board), in practice there have been complications in the 18-month period since the creation of the organisation. Some of these have been operational issues resulting from the merger; others were linked to the pressing need to secure provisional licences both as an AVA and an OCN. In this context, the description in the Overview of the past year as one of 'stabilization and consolidation' is understandable.
29 The busy nature of the AVA's existence to date is perhaps reflected in the range and number of plans presented as part of the documentation for the licence application. Beginning with an Action Plan to deal with the consequences of the merger, they included a statement of the relationship between OCN strategy and Access to HE, a Strategic Plan for Access to Higher Education 2004-2007, a Draft Strategic Plan for Access to Higher Education 2005-2006, a statement of Key Priorities 2004-2008, and an Operational Plan 2005/2006. The review team discussed the nature of these various plans with staff of the AVA and heard of the intention to settle into the pattern of a three-year plan for the organisation with an Access to HE component embedded in it. This latter component would be formulated with advice and guidance from the AHEDC. The team welcomed this clarification and would encourage the AVA to continue to simplify its strategic planning processes, and the nature and range of the formal plans which emerge from them.
Promotion of Access to HE
30 The immediate tasks which faced OCN Wales on its creation have led to a prioritisation of its work and a focus on securing important aspects of moderation arrangements and the awarding function of the organisation. This has left relatively little time for reflection on the more general promotion of Access to HE opportunities and the AVA itself would concede that it is not yet undertaking as much development work as it would wish. Nevertheless, the review team noted the successful summer conference held by the AVA in 2005 and the developmental aspects of the work of the Access Tutor Forums. The individual work carried out by the current Access Coordinator in visiting each provider should also not be undervalued: the team heard many endorsements by providers of the importance of this contribution. In discussion, the possibility of further Welsh language provision and of bilingual teaching and assessment was raised as a clear developmental aim of the AVA. These various activities suggest that the AVA has recognised and accepted its responsibility for the promotion of Access to HE in Wales and that the prognosis for further expansion of appropriate developmental activities is positive.
Financial management
31 Reporting to the Board, the FGP has responsibility for preparing the AVA's budgets and for their monitoring and review. Accommodation and the provision of appropriate management information systems are also within the remit of the FGP. Although the FGP has experienced the AVA's common problems with quoracy, its minutes demonstrate an appropriate level of financial scrutiny and control. The AVA's intention is to maintain a reserve equivalent to three months' operating costs and to aim for a cover period of six months with any further surplus being used to fund developments. Currently, the AVA benefits from substantial reserves inherited from the constituent organisations and has also produced a healthy surplus in its first year. The review team was told that, until the AVA's budget position became entirely settled, the intention was to act with prudence in the use of any surpluses.
Staffing
32 The staffing of the AVA falls into two broad groups. One deals chiefly with operations and is headed by a manager reporting to the Chief Executive: its functions include finance, office systems and information technology (IT) provision. The Deputy Chief Executive manages the other group which includes the three quality and curriculum officers (QCOs), the Access Coordinator, and other staff devoted to quality and moderation activities. This structure, which was introduced in 2005, is intended to resolve some of the initial operational difficulties encountered by the AVA at its formation.
33 The review team heard that the level of support to providers had improved markedly since the creation of the all-Wales AVA in general, and as a result of the work of the dedicated Access Coordinator role in particular. Improvements were noted in accessibility and response times, and the level of knowledge, expertise and professionalism provided by the AVA across the country. Staff in the AVA have clearly made a significant difference to providers' perceptions of quality in service generally, and are to be commended for this.
34 Since it had contributed to a condition placed on the granting of a provisional licence, the review team paid particular attention to the provision of staff resources for the management of Access to HE activities. At that time, OCN Wales had proposed a 0.5 full-time equivalent Access Coordinator but agreed to keep the adequacy of this provision under review. In practice, the post has become 'front-loaded' and has been full-time since the start of the academic year, in response to the need to support Access to HE providers, to revise procedural and operational guidance, and to respond to strategic developments relating to the creation of the new AVA and to external drivers. The team was reassured to hear that this frontloading would not mean that the AVA would be without an Access Coordinator for the second half of the year, but that the role would operate at 0.5 for the remainder of the year.
35 It is intended, however, that the current level of dedicated support for Access to HE should be reduced in the future, with a shift to a dispersed model which will involve other staff more closely in Access to HE work. The adoption of this model is intended to increase knowledge and awareness of Access to HE in OCN Wales, to spread the workload and to reduce the reliance on the Access Coordinator as a key individual. While this approach offers the AVA clear advantages with regard to risk management, succession planning and responsiveness to changing patterns of provision, in its further consideration of staffing models, the AVA should weigh these advantages against the benefits that have been brought by the present model.
36 The review team therefore recommends that the AVA conduct a thorough risk assessment exercise that models a range of approaches for meeting the staffing needs associated with its Access to HE activities and responsibilities. The outcome of this exercise should be to ensure that there is no dilution of focus or service standards achieved by the current Access Coordinator, and to ensure that responsibilities for its Access to HE activities and operations are clearly specified in any future staffing strategies.
Operational procedures
37 Bringing together and rationalising the operational procedures of three different organisations has offered some particular challenges to OCN Wales, which have not been eased by the OCN's split office locations. The AVA is to be commended for the ways in which it has successfully and efficiently managed this rationalisation. All of the major processes undertaken by the AVA are appropriately documented and the review team noted the AVA's set of clear service standards with specific undertakings on delivery in respect of approvals, post-panel processes, moderation and certification. Evidence that the AVA was meeting its obligations was offered by the endorsement of providers and programme coordinators met by the team.
38 In one area, the nature and quality of minutes, the AVA may wish to review its practice. The review team encountered a variety of formats for minutes and differences in minuting conventions. Although the basic pattern of minutes is appropriate, particularly the provision of space for document references and noted action, in some cases the minutes themselves were difficult to follow as a record of a meeting and the decisions which had been reached. The shorthand nature of some minutes also prevented them from being fully useful for the purposes of external scrutiny of the business of the organisation. In discussion with the AVA, the team heard that this area of the AVA's work was currently being evaluated with the possibility of changes being introduced. The team would recommend the adoption of a more informative and systematic model for committee notes and minutes.
Data collection and management
39 The chief responsibility for data management rests with the OCN's operations team and, in particular, the IT Officer. An IT Assistant has also been appointed to each of the two offices of the OCN. All of the data operations associated with learner registrations and awards have been located in the Bangor office which has contributed to better control and accuracy of data. Such issues as the control of data collection and inputting have been a further challenge to OCN Wales with the necessity to merge and reconcile computer-held data from three separate organisations. This operation was not helped in the first instance by uncertainties over the accuracy of such data from South Wales or by the reported unreliability of the OPUS software used by the OCN. The Overview noted, however, that significant progress had been made with data reconciliation and that the OPUS software had become more robust.
40 In discussing the AVA's provision and learners, the Overview demonstrated that the organisation has the capability to present data accurately, simply and with confidence, although the level of analysis applied to such data was more hesitant. In general terms, and on the evidence of the strategic plans and other papers seen by the review team, the AVA does not yet appear to be making the most thorough use of the data at its disposal in its strategic planning.
Support for members and communications
41 A number of the ways in which the AVA supports and communicates with members have already been discussed above: these include the facilitation of regional Access Tutors Forums and the individual visits to providers made by the AVA's Access Coordinator. To this might be added the periodic Access to HE newsletter aimed at Access to HE tutors, programme coordinators and moderators which, if a little prosaic, is nevertheless informative and helpful. In respect of providers' promotional literature and their use of appropriate logos, the AVA has issued written guidelines and is currently following up those guidelines with visits to clarify its expectations. In discussion with representatives from members and providing institutions, the review team heard an appreciation of the high level of support which was being received from the AVA with an emphasis on the speed of response and the level of expertise being offered. The AVA is to be commended on the levels of support which it has offered its providers during and after the merger process.
Transparency of procedures
42 The AVA has established an appropriate grievance procedure for staff. It has also developed a thorough appeals and complaints process which involves external participation at appropriate points. The AVA's policy on equal opportunities is comprehensive.
43 In summary, the review team concludes that the AVA meets all of the requirements expressed through the criteria for Principle 2.
Principle 3
The organisation is able to assure the quality and fitness for purpose of Access to HE programmes at the point at which they are granted formal approval, and to have effective means to develop, evaluate and review the Access to HE provision for which it has responsibility
Programme development
44 The AVA's process for developing and submitting an Access to HE programme is set out in a series of guidance documents available via the AVA's website, email or in hard copy. A submission template is supported by guidance documentation that is specific to Access to HE. Providers wishing to develop an Access to HE course liaise directly with the Access Coordinator, who ensures that providers understand the submission and validation process and have easy access to the guidance notes and submission templates.
45 A process of iteration between the AVA and the provider is undertaken until the submission documentation has reached the standard of clarity and completeness required for its submission to panel. Submission documents contain all information needed by validation panel members to ascertain fitness for purpose.
Consultation with HE
46 Documentation scrutinised by the review team confirmed that the AVA has implemented a rigorous process of matching the details of each Access to HE submission to the requirements of the current QAA licensing criteria. Providers are given explicit guidance to liaise with staff from HE institutions at an early point in the development process. This stage is intended to support progression opportunities and allow HE institutions to confirm that units of assessment are appropriate, both individually and in combination with other units, to provide a coherent programme leading to the award of the Access to HE qualification.
47 The review team noted from its scrutiny of panel reports that the involvement of staff from HE institutions at the pre-panel stages appeared systematic, and was assured that the expertise available to the AVA was used effectively.
Guidelines to providers submitting proposals
48 OCN Wales has assessed the potential for integrating its Access to HE submission requirements and processes into those it uses for OCN provision. Following a trial period, it has opted to produce documents and supporting guidelines for providers which are specific to Access to HE. These have been thoroughly mapped to both QAA's licensing requirements and to the specific requirements of the AVA's policy and funding context. There is a commendably clear identification of the AVA as a QAA licensed awarding body throughout the documentation, and the locus of authority in determining key elements of programme design is consequently well articulated. This extends to areas that the AVA recognises as being transitional, such as the requirement for a common credit target. The documentation comprises a helpful general preamble, the submission criteria and a checklist for providers.
49 From its scrutiny of course documents, the review team was able to establish the efficacy of the submissions process generally and the submission documents in particular. The team was assured that the submission guidelines gave providers a level of clarity and support to enable AVA officers to make effective use of their time assisting with the development process, and for validation panels to be clear subsequently in their assessment of fitness for purpose.
Specification of criteria for approval
50 The criteria for the approval of an Access to HE course are clearly set out in the submission guidelines. They incorporate criteria for programmes, pathways, new modules, additional units and procedures for revalidation. Successful completion criteria are governed by the current all-Wales credit target (66 credits with at least 48 at level 3 and a maximum of 18 at level 2), with clear guidelines to govern the means by which units are identified as mandatory and specified within the rules of combination. A common pro forma approach is used for each pathway to ensure that moderators and AVA staff are able to confirm the accuracy of the recommendation for the award of the qualification.
51 The review team noted in its scrutiny of course documents and in its discussions with AVA, FE and HE staff, that the status of the current credit target was uncertain. The overall thrust of the submission documentation was unequivocally to establish fitness for purpose, but the team noted a number of examples in course documents where completion criteria varied from the national target. This practice will need to be reconsidered in the context of the AVA's implementation of the new qualification specification.
Implementation of the Access to HE Diploma and credit specification
52 The AVA has submitted an implementation plan to QAA, confirming its capability to migrate from the current criteria to the new common credit framework for Access to HE but, at the time of the review visit, had not reached a stage of any detailed planning for dealing with the consequences of the migration. The current all-Wales tariff of 66 credits was apparently agreed relatively straightforwardly, and the AVA asserts in its implementation plan that migration to the new, common tariff of 60 credits is 'in hand'. However, the AVA suggested that some departments of receiving HE institutions might be reluctant to offer entrance to HE, based solely on the new tariff, and it was not clear that the AVA had a strategy that would address difficulties that may result from this expectation.
53 Of particular concern to the review team was the AVA's apparent expectation that the new qualification specification would operate in much the same way as the previous specification, with the common credit requirement for the Access to HE Diploma indicating a minimum requirement, but with some programmes specifying additional credit requirements for successful completion, according to the expectations or demands of local HE institutions. The adoption of a single, credit requirement which defines the volume of the Access to HE qualification, wherever it is awarded, signals a more fundamental and significant change, the implications of which have not yet been fully worked through by OCN Wales. Among these implications may be the recalibration or revalidation of some programmes, but there will also need to be direct engagement by the AVA with admissions staff and appropriate managers in its partner HE institutions, to explain the changed basis of the qualification and to ensure that it is fully understood.
54 In order to demonstrate that its award of the Access to HE Diploma will be consistent with the published Access to HE Diploma and credit specifications, the AVA will need to develop a revised, more detailed implementation plan. The plan should include appropriate strategies to develop a common understanding across the AVA of the new credit requirement for the Access to HE Diploma as a fixed tariff that defines both fitness for purpose of a programme of study, and successful completion criteria for the national qualification.
Programme validation and recognition
55 There are well established criteria set out in the AVA's guidance documentation for the composition, conduct and chairing of validation panels. As with other AVA documents, there is a clear identification of the AVA as a QAA licensed awarding body throughout the documentation, and the locus of authority in determining key elements of the validation process is consequently well articulated.
Panel membership and procedures to assure externality, objectivity and consistency of process
56 Panels normally comprise at least two representatives from HE (one of whom acts as the Chair), an Access to HE lead moderator, two Access to HE coordinators and an AVA quality reviewer. Additional panel members are invited where a need is identified for a greater level of curriculum expertise than is available through the initial core group. Lead moderators may absent themselves from the panel, but must submit written comments. Conflicts of interest are avoided by the AVA's stipulation that no panel member may have any connection with the submitting organisation.
57 The panel meeting is preceded by an agenda setting session before being joined by the submitting team. The panel's decision is reached during a meeting from which the submitting team withdraws. AVA staff are present but have no role in the decision making process. Chairs, for whom there are separate guidelines, are responsible for ensuring that the process is understood by all panel members.
58 Having scrutinised panel guidelines and reports, the review team was assured that the externality, objectivity and consistency of the validation process were consistent with the licensing criteria.
Process for formal recognition by AVA, including meeting conditions
59 Decisions reached at validation panels are subject to formal approval by the QAC. Providers are supplied with written confirmation of the panel decision once any conditions that have been set have been met, and prior to the next scheduled meeting of the QAC. The process of formal confirmation of the panel decision by the QAC is staged, to guarantee that approval is not granted where there may be outstanding conditions. The AVA's Deputy Chief Executive is charged with the responsibility of ensuring that any conditions have been met prior to paperwork being submitted to the QAC. The Chair of the QAC then acts with delegated authority from the OCN's Board in confirming that QAA's criteria for approval have been adhered to.
60 From its scrutiny of guidance documentation, minutes, panel documentation and discussions with AVA staff, the review team was assured that the process for granting formal recognition was consistent with the licensing criteria.
Guidelines/policy on assessment
61 The AVA clearly specifies in its programme submission documentation the information it requires providers to give about assessment on proposed Access to HE programmes. Areas to be covered include assessment strategies and their fitness for purpose for subsequent study in HE; providers' policies on plagiarism, resubmissions, accredited prior learning (APL) and credit exemption, and late submissions; and procedures for confirming that the successful completion criteria for the award of the Access to HE qualification have been met. There are clearly set out expectations in the guidance documents, which stipulate that unit/module assessment strategies should either mirror those used in HE, or otherwise prepare students for HE assessment regimes.
62 In addition to the above areas that are stipulated as requirements in the submission process, the AVA provides specific policy guidance documents for APL (which include experiential and certificated learning). All other policies relating to assessment are determined by institutional practice in providing institutions.
63 The AVA therefore operates two different approaches to policies for assessment on Access to HE courses: in some areas it devises specific policies itself, to which providers are expected to adhere (for example, in relation to the accreditation of prior learning); in other areas, providers develop their own policies (for example, in relation to plagiarism and resubmissions). The AVA's rationale for generating some policies itself, and accepting or endorsing others produced by providers, is not clear and the review team was concerned that this position could lead to some unintended inconsistencies of practice, with consequent potential inconsistencies in assessment outcomes. The team therefore recommends that the AVA reviews its approach to assessment and produces an overarching policy statement on assessment that explains the rationale for its approach. If, as a result of this review, the AVA should identify any areas in which new AVA-wide assessment policies are needed, it should agree the means by which it will take forward the necessary developments.
Revalidation procedures
64 The AVA publishes clear guidelines for the revalidation of Access to HE courses with approval usually granted for a period of five years from the date of the initial validation panel. Once the initial approval period is nearing termination, the AVA engages in a dialogue with the provider to confirm the different requirements for revalidation and to establish that these are clearly understood. This results in the submission of an analytical account covering the operation of the course during its validated period. Extensive guidance and a summary table are made available to assist providers in compiling an evaluative reflection of their provision. As with initial submissions for validation, AVA staff assist providers through an iterative process until the submission is judged as 'panel ready'. The panel process then follows that for initial validations.
65 The review team noted that the AVA had experienced some difficulties in the application of the initial trial of its new submission documents and revalidations had often run behind schedule. As a result, the AVA was identifying appropriate measures to rectify these problems. Having scrutinised panel reports and spoken with AVA staff, the team was assured that the AVA's revalidation process and procedures were as robust as those for initial validation, and consistent with the licence criteria.
Programme monitoring and review
66 The AVA monitors and reviews the quality of its Access to HE provision via annual reporting from both providers and moderators. It also organises standardisation events on an annual basis as part of its procedures for assuring the consistency of academic output.
67 The requirements for annual reporting are set out in a letter from the AVA's Access Coordinator, which also serves to remind providers of the annual deadline and the importance of full and timely responses. The letter establishes the key areas of reporting required to enable the AVA to meet its obligations under the licence, and confirms that providers may submit college-produced annual monitoring reports, as long as the AVA's criteria are met and are cross-referenced. The AVA evaluates each report upon receipt and provides feedback to the originator's institution at senior level. The 'highlight' report is then prepared by officers for the QAC.
68 The AVA has identified that further work is required to receive reports by the deadlines set; to draw out more evaluation, rather than description from report writers; and that a more empirical approach needs to be taken with the capturing and representation of students' views of their learning. The review team concurred with these observations and noted in its scrutiny of documents and discussions with AVA staff that there were further opportunities to develop the use of the output from quality monitoring processes in order to assure the consistency of learner achievement. The team noted that the AVA had conducted a large number of standardisation events across Wales, with significant numbers of staff attending, as part of the means by which it assured itself of the consistency of student output. However, the team also noted that the AVA needed to increase the level of scrutiny of the output from these events, and the composite reporting, at QAC level, in order to ensure that the significant observations that may be drawn, could be embedded in the AVA's plans for quality improvement.
Programme development and enhancement
69 The AVA operates a robust enhancement and staff development programme, including its regional induction events at the start of each year, practitioner meetings, and an annual conference. Development events have ranged across sessions on internal moderation, portfolio development, assessment and unit writing, with the all-Wales conference covering policy as well as practical concerns such as credit tariffs, standardisation and retention.
70 The review team noted the commendably wide-ranging programme of events across Wales to support the development of the AVA's Access to HE courses but returned to its earlier observation that a more structured engagement of HE staff, in informing policy related developments, would be valuable. Nonetheless, the team was assured that the AVA's approach to programme development and enhancement was consistent with the requirements of the licensing criteria.
Principle 4
The organisation is able to manage effectively its AVA responsibilities and the structure which supports them
External moderation
71 The AVA organises its Access to HE moderation model around two broad principles: scrutiny of courses (or programmes) and subject content. The former is covered by the role of lead moderator, and the latter by sector moderators (who are also known as thematic, subject or pathway moderators). Wherever possible, moderators are appointed from HE institutions. The AVA has been careful to separate out its Access to HE moderation function from general OCN provision and appoints an Access Lead Moderator at each institution that offers Access to HE. It is their role to ensure that the quality systems relating to the whole of the Access to HE course are robust and as described in the approved validation document. It is also their role to evaluate the AVA's core skills provision and sign off achievement of the Access to HE qualification. Sector moderators are responsible for reviewing the standard of students' work as required in the units of assessment. They are guided not only to assess the standard of achievement, but also comment on the suitability (or otherwise) of combinations of units.
72 The review team noted in its scrutiny of documents and in discussions with AVA staff and moderators, that although there was some confusion in terminology for the moderation function responsible for assuring the subject-specific achievement of learners, this was largely historical and a product of a common (ie OCN and AVA) contracting process. The team was assured that the roles of lead and sector moderator were sufficiently well demarcated and understood by those appointed, to ensure that the quality and fitness for purpose of courses; and consistency and sufficiency of standards of student achievement, were assured.
73 Moderators are initially selected by AVA staff and approved by the QAC. They are then obliged to attend an initial induction/training event and an annual updating session. On initial appointment, moderators are normally allocated an experienced moderator as a mentor, who provides feedback to the new moderator and the AVA on performance. The AVA also conducts a random sampling exercise of accompanied visits with some moderators, as well as receiving telephone feedback from providers. Moderators are given feedback on both these processes. The AVA's Moderation Officer is responsible for monitoring the effectiveness of individuals and the process as a whole and can offer further training to individuals as required. Interventions leading to removal and non reappointment may (and have been) made, where moderators are not be operating to the standards expected by the AVA.
74 In its discussions with moderators, the review team noted that not all were aware of the extensive monitoring that was undertaken regarding their performance, but acknowledged the high standard of support provided by the AVA. Where discrepancies were noted, they were largely attributable to changes in procedures introduced on an all-Wales basis and not experienced by all longer serving moderators. The team found the AVA's overall approach to ensuring the suitability of its moderator pool thorough, and focused on continuous quality improvement. The team would wish to commend the AVA for this thoroughness and its diligence in monitoring standards.
Final assessment boards
75 The AVA has recently introduced a further step - a requirement that final assessment or examination boards be held - to ensure that the award of the Access to HE qualification is conducted in a uniformly fair and equitable manner. Lead moderators are required to check that successful completion requirements for the award have been met both in terms of general credit targets, and specific combinations of units. The appropriate paperwork must be signed by course tutors and internal moderators, prior to the Lead Moderator confirming that the qualification (or partial achievement only) has been granted. A further check is then conducted by AVA staff upon receipt of the paperwork in the office.
76 The review team heard of the benefits that this new process had afforded, including ensuring a more uniform approach, but noted that the AVA's guidelines on the conduct of assessment boards were silent on the matter of a formal record of assessment board meetings which would allow the AVA to monitor this process. The team would therefore recommend that the AVA
amends its guidance notes to allow it to confirm that its requirement for an assessment board has been successfully carried out.
77 Notwithstanding this recommendation, the team was assured that the AVA's approach to ensuring a robust quality, comparability and fitness for purpose of programmes, and the consistency and sufficiency of standards of student achievement, was consistent with the requirements of the licensing criteria.
Internal moderation
78 The AVA has a robust approach to internal moderation on its Access to HE provision, through the establishment of seven quality standards assessed through its moderator report form and supplementary guidance. These standards cover the usual range of effective internal moderation: a clearly identified role of internal moderator; evidence of a planned process; assessments standardised before issue to students; a valid range of work selected; a well documented process; complete records; and feedback to learners. Lead moderators are guided to both capture evidence about the extent to which providers meet the standards, and to report on it explicitly in their reports.
79 The AVA has itself identified that further work is required to ensure that greater consistency is achieved by some providers in their approach to internal moderation and, from its scrutiny of moderation reports, the review team would concur with this view. The team was assured in its discussions with AVA staff and moderators, however, that the staff development events and the individually customised meetings that it offers, evidence the AVA's determination to ensure that its quality standards are adhered to. The team was therefore assured that the AVA's approach to internal moderation was consistent with the requirements of the licensing criteria.
Action and feedback to providers on the outcomes of the moderation process
80 The AVA ensures that providers have feedback from the moderation process. Oral feedback is provided at the end of the moderation visit, as a conscious act designed to overcome problems identified by the AVA as attributable to poor communication at this stage of the reporting process. Written reports are sent to the AVA after the visit has concluded. All reports are received and assessed by AVA staff and feedback provided to providers at senior, normally head of institution, level. Where action is required, an action plan is also provided. Moderators are copied in on this correspondence and charged with monitoring providers' responses to it.
81 The review team concurs with the AVA's own observation that this model can rely too heavily on moderators to carry any action forward into the following year, as opposed to requiring providers to respond proactively to issues identified. Having scrutinised a range of moderation reports and associated documentation, the team is assured that the AVA's stated intentions of setting tighter standards with its own staff to encourage responses to be made by providers in 2006 is likely to produce the anticipated improvements.
82 The AVA feeds the output of its scrutiny of moderation reports into its QAC, via its Deputy Chief Executive and Access Coordinator. Key issues are then identified to take forward as quality improvement actions, a process which has been well evidenced since the AVA came into existence. The review team noted that there was an extensive development programme that did indeed respond to issues identified. The team was less confident, however, that the AVA's QAC was sufficiently involved in monitoring both the summative output from the evaluation of the moderation round, and actions that followed from it. The team would recommend that closer attention be given to the receipt of, and commentary on, the annual summative report. Overall, however, the team was assured that the AVA's approach to monitoring and feeding back the outcomes of the moderation process was consistent with the requirements of the licensing criteria.
Award and issue of certificates
83 The AVA has centralised the function of learner registration and certification into one of its offices. It is moving from a position where providers can generate their own award notification form to one where there will be a common template, in order to speed up the certification process and ensure greater accuracy in both completion and checking.
84 The AVA operates a 35-day service standard for issue of certificates and reported that this was met in all cases. It also reported an error margin of 0.3 per cent (10 out of 2,000) for the incorrect issue of Access to HE certificates (including partial completion transcripts). This represents a dramatic improvement in the speed and accuracy of production of award certificates. The AVA has taken steps to reduce operational difficulties in checking queries for awards over the summer period by requiring providers to nominate a named person(s) who, at all times over the summer period, will be available to AVA staff needing to seek clarification prior to issuing certificates. Three tiers of staff are involved in ensuring the accuracy of certificates awarded, including the Access Coordinator and Deputy Chief Executive.
85 While not wishing to diminish the AVA's achievements with regard to its significantly improved error margin, the review team was not clear whether the organisation's identification of this margin formed part of a systematic standard setting and monitoring role by the AVA's QAC. The team would recommend that the QAC take an active role in both establishing and monitoring acceptable benchmarks for certification standards.
86 Notwithstanding this recommendation, the team was assured that the AVA's approach to securing the standards of its certification processes was consistent with the requirements of the licensing criteria.
Conclusions
87 Since the award of its provisional AVA licence in September 2004, OCN Wales has made significant advances in a number of key areas and has succeeded in establishing itself as a viable new organisation which has the confidence of its stakeholders. In the early stages, it quite properly prioritised those tasks which related to ensuring that sound legal and governance foundations were in place for the new organisation, and enabling the organisation to become fully operational. This has been largely achieved, although there are some details that will need to be reconsidered. In particular, the AVA will need to review its regulations for co-options and quoracy, in order to maintain an appropriate balance between the involvement of a wide range of interested bodies in its deliberative processes and the security of the organisation's governance.
88 The AVA has also rationalised and developed quality assurance mechanisms for Access to HE and operated its revised processes robustly. In particular, the AVA has developed comprehensive moderation procedures which are sound in their conception and thorough in their implementation. It has also made major improvements in meeting clearly specified service standards, most notably, for providers, in relation to processes for student certification.
89 At the same time as it was developing and embedding these arrangements, and throughout an extended transitional period when the AVA was being created as a single organisation from three predecessor bodies in which there were some notable weaknesses, the AVA maintained, and indeed improved, the quality of the service it offered to providers. The AVA's success and achievements in these areas are due in no small part to the experience, commitment and hard work of the AVA's staff, as well as the determination and support of its advisers and stakeholders.
90 OCN Wales is now embarking on the next phase of development. As it turns its attention towards the more strategic considerations that will allow it to develop further, it will need to give particular attention to its strategy for the development of Access to HE provision. Central to its success in this area will be the extent to which it succeeds in its aim to engage more fully with its HE partners at strategic and advisory levels. There are evident challenges for the AVA in developing links and it will need to be clear about what it can offer to strategies for widening participation in HE in Wales, through the development of its Access to HE provision. In particular, the AVA will need to be clear about how it can respond to the opportunities and challenges brought by the specifications of the new Access to HE Diploma.
91 OCN Wales has demonstrated that it has the capability and capacity to build on the firm foundations it has put in place. Continued emphasis on the rigorous quality assurance of its Access to HE provision, coupled with an active strategy for the development of that provision, derived from a strong evidence base, could enable it to make a unique contribution to HE progression opportunities in Wales.
Commendations
92 The AVA is commended for:
i the ways in which it has successfully and efficiently managed the rationalisation process following merger
ii the levels of support and improved service and professionalism which it has offered its providers during and after the merger process
iii the AVA's overall approach to ensuring the suitability of its moderator pool, including its thoroughness and diligence in monitoring standards and focus on continuous quality improvement.
The AVA licence
93 The AVA licence awarded to Open College Network Wales in September 2004 is confirmed, with conditions to be met by the dates specified below.
Conditions
94 OCN Wales's AVA licence is confirmed on condition that the AVA:
i reviews and revises its regulations for quoracy and co-options in respect of its Board of Directors and sub-committees to ensure that unelected members are unable to form a majority at a quorate meeting (paragraphs 20-22 and 26)
ii reconsiders and further develops its strategy for the implementation of the Access to HE Diploma and credit specifications, to be reflected in a revised, more detailed individual implementation plan. The plan should include specific actions to effect the development of a clear and common understanding across the AVA of the new credit requirement as a fixed tariff that defines both fitness for purpose of a programme of study, and successful completion criteria for the national Access to HE qualification (paragraphs 52-54).
Conditions to be met by 1 December 2006.
[conditions met and licence confirmed 30 November 2006]
Recommendations to the AVA
95 The review team recommends that the AVA:
i considers whether there is further potential for delivery of Access to HE provision among members outside the FE and HE sectors, as a means of promoting and developing Access to HE opportunities
ii continues to simplify its strategic planning processes, and the nature and range of the formal plans which emerge from them
iii conducts a thorough risk assessment exercise that models a range of approaches from the current one full-time equivalent dedicated role, to a fully integrated approach to Access to HE development and support, and ensures that the responsibilities for its Access to HE activities and operations are clearly specified in any future staffing strategies
iv adopts a more informative and systematic model for committee notes and minutes
v reviews its approach to assessment and produces an overarching policy statement on assessment that explains the rationale for its approach. If, as a result of this review, the AVA should identify any areas in which new AVA-wide assessment policies are needed, it should agree the means by which it will take forward the necessary developments
vi reviews and amends its guidance notes to allow it to confirm that its requirement for an assessment board has been successfully carried out
vii gives closer attention to the receipt of, and commentary on, the annual summative report
viii through the QAC, takes an active role in both establishing and monitoring acceptable benchmarks for certification standards.
Appendix
Aims and objectives of AVA review
The aims of the system of AVA review are:
i to provide the basis for an informed judgement by the ARLC about the fitness of the AVA to continue as a licensed agency
ii to promote public confidence in Access to HE as a properly regulated and respected route into higher education by assuring:
- the quality and adequacy of AVAs' systems and procedures
- the quality, comparability and range of AVAs' operations
- the adequacy and comparability of AVAs' standards for approval, moderation and monitoring of programmes
- consistency across AVAs in the operation of criteria for the granting of the Access to HE award
iii to stimulate reflective and self-critical perspectives within AVAs, as an instrument to promote quality enhancement
iv to provide an opportunity to identify and disseminate good practice of AVA operations
v to provide a mechanism for ensuring necessary, and encouraging desirable, improvements and developments in AVAs.
The objectives of each AVA review are:
i to examine, assess and report on:
- the development of, and changes in, the AVA since its last review or initial licence, and its plans and targets for the future
- the organisation's continuing viability and robustness and the ways in which the AVA demonstrates sound governance
- the efficiency and effectiveness of the AVA's operational and quality assurance systems
- the range and scope of the AVA's activities, and the appropriateness and value of these activities
- the ways in which the AVA approves and monitors programmes and the ways in which these processes take account of the need for consistency and comparability
- the ways in which the AVA satisfies itself of the adequacy and comparability of standards achieved by students gaining the Access to HE certificate
- the evidence available to indicate the AVA's success in achieving its aims and targets
ii to identify and report on:
- strengths and good practice in procedures and operations
- areas which would benefit from further development
- areas requiring attention.
Top