Tyne and Border Counties Access Partnership
October 2000

Foreword

1 The Quality Assurance Agency for Higher Education (QAA) is responsible to the Department for Education and Employment (DfEE) for the recognition of Access to Higher Education courses. QAA exercises this responsibility through a national network of authorised validating agencies (AVAs), which are responsible for the processes of individual course recognition and the award of 'kitemarked' Access to HE certificates to students on QAA's behalf. QAA has developed a scheme for the licensing and review of the AVAs, the principles and processes of which are described in the QAA Recognition Scheme for Access to Higher Education in England, Wales and Northern Ireland. The Recognition Scheme is regulated and administered by the Access Recognition and Licensing Committee (ARLC), a sub-committee of the QAA Board of Directors.

2 This is a report of a major review of the Tyne and Border Counties Access Partnership (TBCAP) undertaken by QAA. The Agency is grateful to TBCAP and to those who participated in the review for the willing co-operation provided to the review team.

Aims and objectives

3 The aims of the system of AVA review are:

i to provide the basis for an informed judgement by the ARLC about the fitness of the AVA to continue as a licensed agency;

ii to promote public confidence in Access to HE as a properly regulated and respected route into higher education by assuring:

the quality and adequacy of AVAs' systems and procedures;
the quality, comparability and range of AVAs' operations;
the adequacy and comparability of AVAs' standards for approval, moderation and monitoring of programmes;
consistency across AVAs in the operation of criteria for the granting of the Access to HE award;

iii to stimulate reflective and self-critical perspectives within AVAs, as an instrument to promote quality enhancement;

iv to provide an opportunity to identify and disseminate good practice of AVA operations;

v to provide a mechanism for ensuring necessary, and encouraging desirable, improvements and developments in AVAs.

4 The objectives of each AVA review are:

i to examine, assess and report on:

the development of, and changes in, the AVA since its last review or initial licence, and its plans and targets for the future;
the organisation's continuing viability and robustness and the ways in which the AVA demonstrates sound governance;
the efficiency and effectiveness of the AVA's operational and quality assurance systems;
the range and scope of the AVA's activities, and the appropriateness and value of these activities;
the ways in which the AVA approves and monitors programmes and the ways in which these processes take account of the need for consistency and comparability;
the ways in which the AVA satisfies itself of the adequacy and comparability of standards achieved by students gaining the 'kitemarked' Access award;
the evidence available to indicate the AVA's success in achieving its aims and targets;

ii to identify and report on:

strengths and good practice in procedures and operations;
areas which would benefit from further development;
areas requiring attention.

Outcomes

5 Following the review of an AVA, the reviewers' report is presented to the ARLC by a member of the review team. The Committee then makes one of six decisions:

i unconditional renewal of licence for a
specified period;

ii conditional renewal of licence with conditions to be met by specified date;

iii provisional renewal of licence with conditions to be met and further review visit by specified date;

iv suspension of licence until specified conditions
are met;

v withdrawal of licence for operation as an AVA;

vi temporary renewal of licence with request for further information by specified date
(decision suspended).

The review process

6 The review was conducted in accordance with the process detailed in the QAA Recognition Scheme for Access to Higher Education in England, Wales and Northern Ireland. The preparation for the review included an initial meeting between TBCAP officers and the QAA Assistant Director (Access) to discuss the requirements for the Analytical Account (the Account) and the process of the review; the preparation and submission by TBCAP of its Account, together with a selection of supporting documentation; a meeting of the review team to discuss the Account and supporting documentation, and to establish a draft programme for the review visit; and negotiations between QAA and TBCAP to finalise the programme and other arrangements for the review visit.

7 The review visit took place on 14 and 15 June 2000. The visit to TBCAP consisted of meetings with senior managers and admissions staff from member higher education institutions; AVA officers; course coordinators; moderators; former Access students; and members of the Quality Committee and TBCAP Council.

8 The review team consisted of Professor Janet Finch, Vice-Chancellor, University of Keele, and Mr Malcolm Barry, Director, Professional and Community Education (PACE), Goldsmiths College (University of London). The review was coordinated for QAA by
Ms Kath Dentith, Assistant Director (Access), Institutional Review Directorate.

The AVA context

AVA statistics 1998-99

Providers offering Access to HE programmes 14

Access programmes available 7

Access programmes running 6

Access learner registrations 1,644

Access to HE certificates awarded 351

Background

9 The Tyne and Border Counties Access Partnership (TBCAP) has been in existence in its present form, and has held the AVA licence, since 1996. The structure of the AVA has evolved over time, with the forerunner organisation, the University of Northumbria and Partners, having held the AVA licence between 1989 and 1996. During that period, the AVA was responsible for just one programme, the Higher Education Foundation Course (HEFC), which was delivered by a number of colleges and community high schools, and was managed and coordinated through the HEFC Consortium, which included both providers and the University of Northumbria at Newcastle. In 1996, at the time of the AVA's last review by the Higher Education Quality Council (HEQC), the AVA put forward a proposal for a licence for a new AVA, which would allow it to validate provision outside the HEFC framework, envisaging a regional Access network working, in particular, with a number of local Open College Networks (OCNs) in the region. In 1996, when the new AVA was formed, the membership included three OCNs, but with the departure of one and the merger of the remaining two, the North East Open College Network (NEOCN) is now the only remaining OCN partner.

Membership

10 TBCAP's constitution allows it to admit both associate and full members, and provides criteria for both categories of membership, although at the time of the review visit, there were no associate members. Full membership is available to an organisation which is 'a provider of adult learning and/or is involved in the accreditation of adult learning and/or is an institution of higher education'. The list of current member organisations comprises two accreditation bodies (the HEFC Consortium and NEOCN); three HE institutions (the University of Newcastle upon Tyne, the University of Northumbria at Newcastle [University of Northumbria] and the Open University); 10 FE sector colleges, and six community high schools (in respect of their adult provision). As the organisation is evolving, the membership structure is, in practice, becoming based less on individual membership than upon consortia (the 'accreditation bodies'), which then form the membership of TBCAP. There are currently no provider members which are not associated with either the HEFC Consortium, or with NEOCN.

11 Currently, the two member consortia have different forms of relationship to TBCAP, which holds the AVA licence. The HEFC Consortium operates in 'devolved mode'. This means that responsibility has been devolved to the HEFC Consortium for carrying out many of the functions of an AVA itself, and it is responsible to the Council of TBCAP for the discharge of these. Other provision - currently comprising six free-standing validated programmes - operates in 'centralised mode'. These six programmes, though validated by both TBCAP and NEOCN, remain the direct responsibility of the central body. The distinction between these 'devolved' and 'centralised' modes is crucial to understanding the present structure and will be referred to at various points throughout this report.

12 TBCAP envisages a further evolution for September 2000, when it is expected that NEOCN will also operate in devolved mode. TBCAP has laid down procedures for considering the suitability of any member to operate in devolved mode, and it is intending to conduct the evaluation of NEOCN in July 2000. If NEOCN is granted approval to operate in devolved mode, no provision will remain operating in centralised mode and, as the Account states, 'TBCAP will essentially comprise a partnership of...the HEFC Consortium and NEOCN...supported by a Core administration and Committee structure which is the locus of authority for the AVA licence'. The resulting structure of TBCAP envisages the 'recognition that all of the co-ordination, development and administrative functions of the HEFC and NEOCN Access to HE provision are performed by the partners rather than by the central administration'.

Provision

13 The HEFC (which is delivered by eight of the FE members and all of the school members) remains the major part of TBCAP's validated provision and, in 1998-99, TBCAP recorded 1,644 learner registrations, of which 1,453 were registered on the HEFC. The HEFC is jointly validated by TBCAP, which awards the 'kitemarked' Access certificate, and the University of Northumbria, which awards its own certificates to students for successful completion of individual HEFC modules. There are 24 modules currently available, in a range of subjects, which, in different combinations, according to the resources and individual marketing and targeting of different providers, can lead to a range of progression opportunities. The six non-HEFC programmes, delivered by four of the FE members, provide a variety of progression routes, although one of these programmes has never recruited sufficient numbers to run.

Aims and objectives

14 The TBCAP Constitution includes the AVA's five aims: 'to enhance access to HE opportunities for those who have not participated previously, and particularly for those groups under-represented in HE'; 'to establish a collaborative partnership between the local OCN, the HEFC Consortium, their providers and other interested bodies'; 'through that partnership to build and map a wide range of coherent and flexible pathways to HE within a common credit framework'; 'to quality assure access to HE provision and to seek and maintain a current licence from the Quality Assurance Agency to act as an Authorised Validating Agency;' and 'to promote equal opportunities'. The Constitution also describes the AVA's seven objectives, which include the validation of Access programmes, of pathways and of individual learner routes to HE; the award of 'kitemarked' Access certificates; the establishment of a credit framework; the facilitation of staff and curriculum development; and the provision of support to member organisations in respect of the design and validation of Access provision.

15 TBCAP's governing body, the Council, has a responsibility to 'institute and approve the Development Plan at appropriate intervals', and the plan for the period 1997-98 to 2001-02 includes a detailed operational plan for 1997-98. Formally, this remains the AVA's current plan, although there have, inevitably, been changes in the context of the AVA's operation which have affected its priorities for development. Responsibility for monitoring the plan, and the process of developing the new plan were not clear to the review team, and operational plans for the years since 1997-98 did not appear to have been developed. The review team was informed that it was likely that the detailed financial plan, which looks forward to fully devolved status, would form the basis of a new development plan.

Developments

16 Within the Council of TBCAP, and among its individual members, there is a consciousness of the potential of the AVA to address new and emerging regional agendas. As a structure which links major HE and FE institutions in the Newcastle area, TBCAP is felt to be well placed to work in partnership with the Regional Development Agency and to address the Government's widening participation agenda. Its focus on access to HE for adult learners is particularly important for the North East region, which has a low level of post-compulsory participation in education.

17 While continuing to provide well-established Access to HE programmes, in recent years TBCAP members have also responded to the needs of specific groups in the North East region. Plans for ICT developments in course delivery are being developed, to make the programmes more accessible to learners who live in rural areas. In the urban areas, there has been some targeting of ethnic minorities and of people who live in disadvantaged neighbourhoods. Some of this work has been undertaken with funding from the National Health Service, whose educational needs have become a more prominent part of the strategy of providers in recent years.

18 The review team heard that several of the TBCAP partners had also been linked separately through the Credit Alliance in a successful bid for regional funding from the Higher Education Funding Council for England. In taking forward this development of a regional credit framework, it is intended to build on substantial work previously undertaken - though not brought to fruition - by TBCAP. This work, undertaken during 1998-99, had explored the development of a credit framework which would have included both HEFC and NEOCN provision. While the Credit Framework Project provided a detailed and well-considered theoretical basis for further work, it did not reach a stage where it could be implemented.

19 In terms of the AVA's own internal development, the main area of recent deliberation and activity for TBCAP has been its preparations for the anticipated change to fully devolved mode of operation. There has been detailed consideration of the impact that this would have on the AVA's structure and operations, and agreement has already been reached over new financial and staffing arrangements which anticipate this change. The review team noted that the assumption was being made that NEOCN's application would be successful, and plans were well advanced for the implementation of substantial revisions to the AVA's structures.

20 A process for considering this application has been set out. However, no criteria have been developed for determining whether applications should be approved, and some details of the approval process did not appear to the review team to have been fully thought through. As an example, although review documentation and participants in review meetings referred to the expectation that this change would occur 'with effect from September', it was acknowledged that this would be difficult to achieve, given that the TBCAP Council, which would have formal responsibility for approving such a change, was not due to meet until October. Thus the timescale was not only unclear but also suggested to the team that the TBCAP Council was not fully in control of the process by which this extremely important decision was to be made.

Context for the review

21 There have been a number of interactions between TBCAP and QAA since 1997, when the latter assumed the responsibility for the Access Recognition Scheme that had been held previously by HEQC. At that point, there were a number of issues unresolved from the HEQC review and consideration of the proposal for the new AVA in 1996. The proposal itself had not been fully developed, and 'in principle' approval only had been sought. QAA had inherited follow-up reports from this review which indicated that the situation was not wholly satisfactory, and in February 1998, requested responses from TBCAP to a number of the points raised. At that stage TBCAP was operating under a temporary licence.

22 This response was considered at meetings of the Access Recognition and Licensing Committee in September 1998 and, following a further request for clarification, in November. While satisfied that the response was appropriate on a number of points, QAA remained concerned about the organisational structure of TBCAP and its consequent ability to exercise full authority over quality matters. It was also recognised that a number of new developments had recently been put in place, and should be given time to become established. Therefore a second temporary licence was issued, with a limit of 18 months, at which point TBCAP would be put into QAA's own review schedule.

23 In confirming this in December 1998, QAA indicated to TBCAP that four specific issues would be central to this review. The letter stated that 'the team will particularly wish to consider whether there is clear evidence of the following:

efficient, effective and rational committee and management structures;
transparent processes which pay due attention to accountability in decision making and reporting procedures of committees and of individuals;
progress by the Quality Committee in carrying out its stated intentions and programme of work;
evidence that TBCAP is exercising its authority as an AVA, and carrying out its responsibilities properly and fairly in relation to all TBCAP partners'.

24 The team treated these four points as the framework for the review.

Organisational and operational structures

25 The University of Northumbria acts as the host institution for TBCAP, and the AVA's premises are located in the offices occupied by the University's Access and Continuing Education Service. The AVA's staff are employed by the University, which also provides administrative support for TBCAP committees. The AVA budget is operated within the budgetary structure of this Department, through a University finance code.

Finances

26 Sources of income for TBCAP, other than funding for specific projects, are membership and registration fees. In 1998-99, the most recent completed financial year, TBCAP's accounts indicate a trading loss of £11,134 but this was offset by substantial brought forward balances, meaning that a balance of £56,866 was achieved at the end of the year. The AVA also receives support in kind from the University of Northumbria which makes no charge for premises and provides some administrative staffing. In 1998-99 this contribution in kind amounted to £38,836.

27 At the time of the review, TBCAP was planning to introduce new financial structures, proposed by the AVA's Financial and Strategic Planning Group, which would underpin the new operating arrangements planned, and would take effect from September 2000. The proposals, accepted by the TBCAP Council in April 2000, mean that registration fees would be paid direct to the two accreditation bodies, HEFC and NEOCN. Each Consortium would retain these fees in full to fund activities which it undertakes on behalf of TBCAP. Membership fees would continue to be paid to TBCAP, but at a reduced rate to reflect the reduced amount of work to be done by TBCAP itself. As part of this plan, the TBCAP balances would be run down to, and sustained at, the level of £20,000.

28 The organisational structures of TBCAP reflect the particular governance and operational arrangements of this AVA. The Constitution states that 'at operational level TBCAP implements its role as an AVA through the expertise and mechanisms of those of its members which are accreditation bodies'.

Data collection and analysis

29 Arrangements for data collection, analysis and management operate separately within the structures of HEFC and NEOCN; they use the same software but have adapted it differently. The devolved mode uses a University-based system. The centralised mode of provision uses a data management system also used by many Open College Networks, which has not been totally satisfactory, either in ease of use or providing outputs, and thus data collection on progression remains a staff-intensive activity. TBCAP records are kept in the office located in the University of Northumbria. The HEFC and non-HEFC learners are registered separately, but by the same individuals, operating in the same office.

30 While detailed statistical data has been kept about HEFC students, equivalent data has not been available in relation to students on non-HEFC programmes to date. Some providers reported a dip in enrolments on Access programmes, and thus progression, in the late 1990s, but overall the situation appeared stable. A range of withdrawal rates across programmes was reported (up to 40 per cent was reported in some cases) and various factors were adduced as contributors to this range.

31 The absence of complete data records about the AVA's programmes and student profiles has impeded the use of hard data for planning purposes. QAA's statistical requirements have led to the development of common requirements for all courses, and could provide the basis for improvements in this area.

Learner experience

32 The evidence available to the review team from meetings with students and various categories of staff, including receiving HE admissions tutors, indicated that Access courses generally prepared students well for progression to higher education. This was apparent across the discipline range, and former Access students reported that their programmes had developed their skills and, especially, their confidence. The guidance and support received in FE colleges in relation to their study on the Access course was praised, although in some areas related to progression to HE, students felt that they could have been better informed. This guidance, encompassing HE admissions requirements and procedures, financial regulations and opportunities, and, in some cases, familiarisation opportunities, is provided by the FE colleges. The review team considered that TBCAP could play a more active role in facilitating communications between FE and HE partners to prepare students better in these areas.

Communications with providers

33 TBCAP communicates with providers through a variety of means: the committee structure, its staff, paper-based and electronic communications. Providers are represented on all committees though, since these meet three times a year at most, they cannot be regarded as a regular means of communication.

34 Providers whom the team met indicated that they felt well informed through regular correspondence from officers, and through their accessibility to offer advice when requested, although this seemed to be most effective where they were part of the HEFC Consortium. They also regarded the MADG meetings (see below, paragraph 66) as an important means of communication. These are part of the HEFC arrangements, although at the time of the review, a recent decision had been taken to open them up to non-HEFC provision under the auspices of TBCAP.

35 The responsiveness of officers was praised by providers, who found ready and relevant support to enquiries about new developments, the management of programmes, and quality assurance issues. This was particularly apparent for members of the HEFC Consortium. To date, a comparable level of support has not been available to provision offered in centralised mode. The plan to place this provision under the responsibility of NEOCN is intended to address this.

Staffing

36 Arrangements for employment of staff rest with the host organisation. The staffing structure of TBCAP reflects the particular constitutional structure of this AVA. At the time of the review, this comprised a Development Officer, supported by officers of the HEFC and NEOCN consortia. In reality, the former Development Officer had been seconded by the University to other duties in 1999-2000, thus leaving TBCAP without a dedicated officer. During 1999-2000, the role of TBCAP Development Officer was being shared by three individuals (two being officers of the HEFC Consortium, one of NEOCN). At the end of the secondment, however, the secondee will not return to TBCAP. It is planned that changes in the structure of the AVA to be put in place in September 2000 - anticipating the granting of devolved status to NEOCN - will reduce the staffing requirement of TBCAP itself to one half-time administrative post at a junior level.

37 The duties of TBCAP staff differ in respect of the programmes offered in centralised and devolved mode. For the six programmes offered in centralised mode, professional and administrative support is provided by TBCAP staff. During 1999-2000, when there was no dedicated TBCAP officer in post, the NEOCN Development Worker was performing TBCAP functions in respect of these centralised programmes. His role is to offer guidance to providers in the development and monitoring of programmes, and to support the work of moderators. This is seen as a transitional phase, until September 2000, when it is anticipated that NEOCN will assume full responsibility for these programmes on behalf of TBCAP.

38 Currently, all officers supporting TBCAP are combining this with supporting either the HEFC or NEOCN, leaving them with a dual role - and potentially a conflict of interest - in interactions with providers. It is not clear, for example, how they are expected to respond to incoming enquiries about new programmes which might be channelled through either of the present 'modes'.

39 Staff undertaking TBCAP duties are managed by the Head of the Access and Continuing Education Service (ACES) at the University of Northumbria. In the current year, when TBCAP duties are being shared, this includes all three staff who share the TBCAP role, but the NEOCN Development Worker's reporting line is accepted as nominal and is not reflected in a written contract. The other two staff, employed both on HEFC and TBCAP duties, report to the same individual in both capacities; it is accepted that the distinction between the two accountability lines is not formally separated. TBCAP staff are appraised within the University's procedures. However, during the current year, the University Head of ACES appraises only those staff who are also employed in supporting the HEFC. The NEOCN Development Worker, currently performing some TBCAP duties, is appraised within the NEOCN staffing arrangements.

40 The key lines of accountability for TBCAP policy issues are defined within the committee structure. The formal position is that officers report to, and take instructions from, the Council of TBCAP. Evidence from both meetings and written documents suggests that the officer role in this AVA is seen as operational and responsive, rather than one which provides significant leadership to the Council in policy development.

41 Accountability to committees - including the Council - is indirect in this AVA. The previous Management Committee structure has been discontinued, since its business was found to overlap with that of the Council. Whatever the merits of this change, it has left TBCAP without a formal mechanism for holding officers to account, save through meetings of Council itself. These are quarterly at most. Although the Head of ACES is technically acting on behalf of the Council in his management of TBCAP staff, there are no formal reporting lines to the Chair or to the Council as a whole.

42 The Account described the management and administrative arrangements for HEFC and TBCAP as 'very similar'. After the anticipated devolution to NEOCN, the remaining central resources for TBCAP will still be located at the University of Northumbria. The team felt that there was little evidence to sustain the claim, made in the Account, that while functioning as a unit within the University, the AVA is nevertheless independent from it because lines of accountability are to the members through the constitutional arrangements.

43 The review team considered that current arrangements for both devolved and centralised modes made it difficult to accept that, at an operational level, TBCAP had an existence separate from its component parts. The organisational support structures are weak, despite valued services which providers receive from officers. There is currently no single professional officer whose sole responsibility is to TBCAP. From September 2000 this position will be weakened even further by the reduction of TBCAP staffing to half a junior administrator. It is very difficult to see how this structure - however talented the individuals who occupy it - can support the responsibilities vested in it by being granted a QAA AVA licence.

44 Referring to the framework for this review, the review team concluded that it had failed to find clear evidence of management structures which were effective and rational, though they appeared to be efficient. The team also concluded that reporting procedures of individuals were not transparent and did not have clear lines of accountability. The principle that TBCAP relies on the procedures of its members which are accrediting bodies has, in practice, left this AVA without clear lines of accountability back to its own Council, at the operational level.

Governance and committee structures

Overview of structures

45 TBCAP is an unincorporated association and is described in the Account as being 'jointly owned and run by its members'. The formal locus of the authority for the AVA licence lies with its governing body, the TBCAP Council. The Account specifies that the Council is 'accountable to QAA in respect of all those matters surrounding the quality assurance of its Access programmes and the award of the ''kitemarked'' Access certificate'. The Council can institute sub-committees and ad hoc working groups, which have a responsibility to report to the Council.

46 The Constitution records two sub-committees, the Management Committee and the Quality Committee, each of which 'reports to the Council'. However, at its Annual General Meeting on 12 April 2000, a number of amendments were made to the committee structures, designed to facilitate the organisation's intended move to a fully devolved structure. In particular, the Management Committee (which had been under review for some time and had met only once during the period January 1998 to April 2000) was dissolved and the Financial and Strategic Planning Group (previously an ad hoc group) was established as a permanent part of the TBCAP committee structure. TBCAP's committees and reporting lines appear to provide an adequate structure for the management and quality assurance of those programmes which are operating in centralised mode. The review team had a number of concerns about their operation, however, which are detailed later in this report.

47 The HEFC Consortium, operating in devolved mode, maintains its own committee structure, including a HEFC Board. The Board oversees the arrangements for the operational management and development of the HEFC, as well as having a number of functions relating to the quality assurance processes of the HEFC and to the confirmation of awards to students. Although it was not a part of the review team's remit to consider the detailed operation of the HEFC Board, the team considered, on the evidence of minutes presented, that, in relation to its quality assurance responsibilities for the HEFC, the Board was working well.

48 There is a lack of clarity about reporting lines between the HEFC Consortium and TBCAP, which holds ultimate responsibility for the provision offered under the AVA licence. The HEFC Board oversees the devolved AVA responsibilities, but the review team noted that the terms of reference of the HEFC Board make no reference to a responsibility to report to TBCAP. They do, however, indicate a reporting line for the HEFC Board to the University of Northumbria's Committee for Quality Improvement and Academic Standards (QIAS). The HEFC academic committee structure shows a relationship between the QIAS and three bodies: the Consortium Principals' Group, the University's Academic Board, and the TBCAP Council. There was, however, no evidence of a direct relationship between the QIAS and the TBCAP Council, and how reporting lines to TBCAP might operate in practice was unclear to the review team. Although there is an overlap in membership between the HEFC Board and the TBCAP Council, in the persons of the University of Northumbria's Head of ACES, and the HEFC Co-ordinator and Development Officer, they do not represent the HEFC Board in their positions on the Council.

49 The lack of clarity of these reporting lines, and the indirect nature of the relationship between the University of Northumbria's academic committee structure and the structure of TBCAP, gave rise to concerns among the review team about the effectiveness and autonomy of the latter. The team could find no evidence in the description of committee structures, or in the accounts of the business of the different committees, to support the claim made in the Account that 'the HEFC Board is accountable to Council for the effective discharge of its devolved responsibilities'. The statement that 'the arrangements for devolution to NEOCN will reflect the same lines of accountability' gave the team little confidence that TBCAP would be able to exercise its authority as an AVA in relation to the devolved partners within the future structures that are envisaged.

The TBCAP Council

50 The Council meets at least twice annually, one of these meetings including the AVA's Annual General Meeting. All member organisations are represented on the Council, which is responsible for the policy and management of TBCAP, as well as for its particular functions as a licensed AVA. The Constitution specifies a number of particular responsibilities for the Council, including overall direction of the policy of the AVA; instituting and approving the development and financial plans; the approval of quality assurance arrangements and subsequent modifications; the appointment of paid and honorary officers and arrangements for their employment; the receipt of reports from sub-committees and working groups; and responsibility to QAA in respect of the national arrangements for the recognition of Access courses.

51 Evidence available to the review team suggested that the Council was fully informed and had been properly involved in the consideration of the AVA's development, and matters related to financial planning and staffing. It had also kept an appropriate overview of national and regional developments relating to Access, and had provided appropriate direction in some matters such as the Credit Framework Project. In addition, it had given close attention to matters of validation and modification of modules for programmes operating in centralised mode. However, the team had substantial concerns in relation to the Council's exercise of its overall quality assurance responsibilities, and these were related both to the concerns about the Council's relationship to the HEFC Board, described above, and also to the receipt of reports from its own sub-committees.

52 One of the responsibilities of the Council is '...to require regular reports and recommendations' of its sub-committees, and the Quality Committee's own responsibilities also make reference to the provision of advice and recommendations to the Council. The Council does not formally receive the minutes, or regular written or oral reports from the Quality Committee. During the 18 month period prior to the review, each committee had met five times but the Council had apparently received an oral report for one meeting only of the Quality Committee. The team concluded that the required reporting mechanism was not clear, and that reporting was not taking place regularly or systematically. The team was told that the Council relied on the Quality Committee to ensure that the AVA's quality assurance procedures were effective and that, at the time of the review, they felt confident in the latter's ability to discharge this function. It was unclear to the team, however, how the Council received sufficient information to sustain this confidence.

Financial and Strategic Planning Group

53 The Financial Strategy Group was established as an ad hoc group to consider the future financial structures of the AVA in fully devolved mode. Its contribution, in terms of financial planning and strategic considerations, was found to be constructive and it has now been formally constituted within TBCAP structures as the Financial and Strategic Planning Group. It is intended that this group will look at the strategic direction of the AVA and, with the move to a fully devolved model of organisation, take forward the strategic plans of TBCAP. It was too early, at the time of the visit, to evaluate how effective this group would be were that intention to be realised.

Quality Committee

54 The Quality Committee's specified membership of 11 includes two representatives of Council (currently the University of Northumbria's Head of ACES, who chairs the Committee, and the NEOCN Development Worker); two representatives of the Management Committee (currently the HEFC Co-ordinator, with the other position being vacant); six representatives of validated Access programmes; and an external member from another AVA. There is also the provision for up to three co-options, although these were not filled at the time of the review. Attendance at meetings during 1998-99 and 1999-2000 varied between four and seven members, and no more than half of those attending were representatives from providers of programmes on any occasion. While this may not inhibit the process of effective decision-making for the Committee, it does not appear to reflect the cross-section of representation suggested by its membership specification or the intention stated in the Account that the membership is 'constructed to ensure that there is equal representation and input from both HEFC and NEOCN providers'. In addition, there appears to be a lack of distinction between the roles of officers and committee members. The Committee has recently reconsidered its composition, and has decided to invite people 'with an understanding of quality assurance' to join the Committee 'in order to focus activity'. However, no formal changes have yet taken place.

55 The Committee's responsibilities, as defined in the Constitution, are 'to act on behalf of the Council in the formulation and implementation of audit procedures', and the purpose of these procedures, and of the Committee itself, is described as being the review of the quality assurance process; the provision of advice to Council about these procedures; and the consideration of strategic and policy issues arising from the quality assurance process. The Committee also has a responsibility to 'consider issues to do with the comparability of standards between Access programmes, and to initiate mechanisms to establish comparability of standards'. The AVA's Quality Assurance Procedures also emphasises the importance of quality audit and, in its description of the audit process, identifies a number of activities of scrutiny through sampling of documentation received from providers, including validation reports, moderators' reports, annual reviews and statistical data. The Quality Assurance Procedures stipulates that 'all programmes in all modes' participate in these procedures and that they are 'the particular responsibility of the Quality Committee'.

56 The review team sought to clarify the nature of the Quality Committee's contribution to quality assurance in the context of these responsibilities. Members of the Committee referred to the intention to have a rolling programme of activities to consider different quality assurance procedures, with validation having been already considered, and a validation questionnaire having been designed for validation panel participants. The specific outcomes of this work were not clear. Moderation was intended for consideration in the coming year. However, the Committee had not yet undertaken any of the specified activities of scrutiny, and had not considered any validation reports or moderators' reports. While the Committee had received statistics about student registrations, and had considered annual course reports in the context of preparation for the AVA report to be submitted to QAA, there was no evidence that this had led to a consideration of the quality of the provision or of the 'effectiveness of TBCAP's operations'. The Committee had also received reports from officers about national and regional developments, and about progress with the Credit Framework Project; and had discussed matters relating to the present review. The Committee acknowledged that, although the framework for quality assurance had been in place for some years, the audit stage of the procedures was not yet being operated.

57 The review team concluded that, while some of the Quality Committee's activities were related to some of its responsibilities, the Committee was not fulfilling its purpose within the AVA's structures of providing a means by which the quality assurance process could be effectively reviewed, either in relation to centralised or devolved mode operations. In addition, although there was a recognition that the AVA needed to develop more common procedures, there was no indication that the Committee itself had made any recommendations to the Council which might initiate mechanisms to establish comparability of standards.

58 The AVA's current development plan envisaged that the Quality Committee would 'begin implementation of Audit and Review procedures' in the period January-April 1998, and the importance of progress being made in this work was underlined by QAA in its letter of December 1998. By the time of the review, however, there was no evidence that audit procedures had been formulated or were being implemented. The team therefore judged, with reference to the framework for this review, that there was no clear evidence that progress had been made by the Quality Committee in carrying out its stated intentions and programme of work.

The quality assurance of Access to HE programmes

Overview of procedures

59 TBCAP has set down an eight stage procedure for operating quality assurance across both devolved and centralised modes. The eight stages are

1) Programme Development
2) Documentation
3) Validation
4) Approval
5) Moderation
6) Awards
7) Review
8) Audit

These procedures are set down in the TBCAP document Quality Assurance Procedures. Although the document itself was undated, the review team was told that these procedures had been established at, or subsequent to, the time of the HEQC review of the AVA in 1996.

60 The Quality Assurance Procedures states that, for programmes operating in centralised mode, 'the responsibility for the operation of QA procedures lies with TBCAP centrally'. In the case of courses operating in devolved mode, Stages 1-5 are devolved, but Stages 6-8 ('awards', 'review' and 'audit') are retained to TBCAP. Currently this arrangement applies just to the HEFC, but, if NEOCN were to be awarded devolved status from September 2000 as expected, it would apply to all TBCAP programmes. The Account anticipates that, in these changed circumstances, 'The quality assurance processes and procedures of the HEFC Consortium and NEOCN will therefore constitute the quality assurance and validation processes and procedures of TBCAP itself'.

61 The HEFC Definitive Course Document (DCD) provides full details of the procedures for which the HEFC Consortium has devolved responsibility. Although it was not within the review team's remit to assess the detailed operation of HEFC systems directly, it appeared that these were essentially robust. However, the procedures were, in essence, the internal procedures of the University of Northumbria. The review team was unable to discern any procedures that were being operated at TBCAP's, as opposed to the University's, behest. While this might be deemed unnecessary if TBCAP were satisfied with the University's own procedures, the team noted that there had been no evaluation of HEFC quality assurance arrangements before, or since, the decision had been taken to operate in devolved mode, nor was there a planned process for the review of the operation of devolved quality assurance arrangements by TBCAP.

62 The quality audit function of the Quality Committee has a more important role in the fully devolved structure envisaged for the future operation of the AVA. Under this structure, with the accreditation partners having responsibility for operating most of their own quality assurance procedures, the audit function is identified as the mechanism through which TBCAP will monitor these devolved arrangements. The review team considered that the fact that the audit procedures were not in place by the time of the review indicated that TBCAP would not be properly exercising its responsibilities if it were to devolve quality assurance of its recognised Access provision to another body over which it had no effective monitoring mechanisms. It was of equal concern to the review team that the licensed AVA had already devolved responsibility to another body and allowed it to operate without effective monitoring processes in place since 1996. It was not clear to the review team how, in a fully devolved system, the AVA as a whole could be confident of the adequacy of the quality of programmes or standards of student achievement with the current mechanisms and structures.

Development, validation and approval

63 Curriculum development is generally initiated by providers in response to perceived need, but there are also examples where a programme has been developed as a result of regional developments or in response to a request from a higher education partner. For programmes operating in centralised mode, this stage is supported by the TBCAP officer, who also provides guidance to providers about the development of documentation required for the validation process.

64 In centralised mode, the documentation submitted is considered by a validation panel whose composition is specified in TBCAP guidelines. Final approval that panel membership is in line with the template rests with TBCAP, albeit this is delegated to an officer. Evidence available to the review team suggested that the template for panel membership had been adhered to in most recent validations, though the revalidation of one programme had been conducted with a panel much smaller than that required by the template. Members of the TBCAP Quality Committee, who have responsibility for oversight of this stage of the process for programmes operating in centralised mode, were unable to explain how this had come about. They surmised that it occurred in the context of a revalidation of elements of a programme which had been validated only two years previously; they accepted that a modification to their procedures, to allow for variations in the template in such circumstances, would be desirable.

65 The panel reviews the documents produced for the purpose, meets the course team and may view the facilities for the delivery of the course before making its recommendation to the TBCAP Council. All centralised mode validations must be approved by the Council before they can run and there is a standing item on its agenda for this purpose. Minutes of the Council gave evidence that these arrangements were being followed in the period covered by the minutes (December 1998 to April 2000). The review team concluded that arrangements for validation and approval appeared, in general, to be working effectively in centralised mode.

66 Operating in devolved mode, the HEFC arrangements for programme development and validation are set out in the DCD and the development process is supported by HEFC officers. Practitioners whom the review team met considered that both the mechanisms for course development and the support of HEFC officers provided a responsive and flexible system. A central feature of the development process is a well-established system of Module Assessment and Development Groups (MADGs), which are expected to meet at least three times a year and which undertake both development and monitoring work. These groups include practitioners from providing centres as well as representatives of the University of Northumbria. Papers from the HEFC Board suggested that these groups work effectively and their value in relation to development work was also confirmed by both providers and moderators whom the team met.

Assessment and accreditation

67 For programmes in centralised mode, the Quality Assurance Procedures specifies that arrangements for assessment should be 'effective and appropriate'. There is further guidance for those developing a course for validation which is to be submitted for NEOCN recognition. This points to the need to specify for all units learning outcomes, assessment criteria, the method of assessment and evidence of learning. As all programmes operating in centralised mode are currently also NEOCN programmes, the review team assumes that they all have this common approach to assessment. These programmes have learning recognised through the award of credit, consistent with the National Open College Network credit framework, as well the award of the 'kitemarked' certificate. These credits are not graded.

68 For the HEFC, the DCD includes a discrete section about arrangements for assessment. This provides a description of all assessment regulations and procedures, as well as outlining the principles of assessment underlying the HEFC. The HEFC has a unitised structure, and achievement required for the award of the 'kitemarked' certificate is described in terms of OCN levels and credits, although, as the HEFC Consortium is not an OCN, it is not in a position to award OCN credits.

69 Although HEFC/OCN dual accreditation was discussed in May 1996, the HEFC Board did not approve this development, expressing its concerns about potential additional costs. Rather, in addition to the award of the 'kitemarked' certificate, students on the HEFC are awarded University of Northumbria certificates for the successful completion of individual HEFC modules or units. These certificates indicate graded achievement in banded categories ('Pass', 'Merit', 'Credit' and 'Distinction') which relate to percentage marks awarded for individual units. The HEFC DCD looks forward to the development by TBCAP of 'a coherent regional credit-based access to HE and higher level skills framework' which, it envisages, will 'link the HEFC with other kinds of accredited provision, qualification and experience'.

70 There are clearly fundamental tensions between these two systems of accreditation, and the Account notes, in relation to the development of new structures, that 'Concerns about the implications of abandoning the HEFC system of differentiated achievement have not been resolved'. This situation presents this AVA with particular difficulties in assuring equivalence of assessment outcomes for those to whom TBCAP awards the 'kitemarked' certificate.

Comparability and standardisation

71 The AVA has approached the difficulty of standardising across these two very different accreditation regimes through support for the TEC-funded Credit Framework Project (see above, paragraph 18), which was undertaken by the TBCAP Development Officer on a half-time contract during 1998-99. The review team was told that, in addition to the work already done on credit, developing procedures for ensuring comparability was a key priority for the coming year and that the AVA had set aside funding for work to be done in this area. The Account makes reference to plans to develop procedures and systems for comparability of standards across provision operating in both modes, and to the intention to appoint a part time (0.3 FTE) Development Officer for 6 months to undertake this work. It was not clear to the team, however, what form this work would take, and although this was described as a 'project', there did not appear to be any developed plans which indicated how this was to be taken forward by the AVA.

72 Within the HEFC Consortium, Curriculum Sub-groups, the Moderators' Forum, examination boards and MADGs all have a role to play in this work for the HEFC. Standardisation is one of the functions of the MADGs and the inclusion of non-HEFC members in MADGs is one way in which comparability across the AVA has begun to be addressed. The review team noted that this appeared to be working well for the HEFC both in terms of individual students and across the course but, given the complexity of relationships between the HEFC Board and TBCAP, it was unclear how this approach to standardisation benefited TBCAP as an AVA.

73 The work on credit equivalence is not now to be taken forward in the way that was originally envisaged, although it might contribute to larger developments at a regional level. Other work, such as the work on moderation, intended to include direct comparison of student portfolios between HEFC and non-HEFC practitioners, is at an early stage. At the present time, therefore, the AVA remains in a position where it has no means of assuring the comparability of the 'kitemarked' Access awards made in its own name.

Selection, appointment and training of moderators

74 In centralised mode, a potential moderator is identified during the development process, usually by the Development Officer, and then agreed with the provider. There are clear and appropriate criteria for the appointment of moderators. The prospective moderator is then invited to attend the validation panel and is formally appointed by TBCAP subsequent to this panel.

75 In devolved mode, there is a MADG for each module of the HEFC course, to which a moderator is attached. When a module is proposed, the Head of Department or School at the University of Northumbria from a relevant academic discipline is invited to propose a moderator. Newly appointed moderators are provided with full information on the programme, the module and expectations of their role. There are no formal criteria for appointment, nor a stipulated limit to length of service.

76 The review team noted that, although moderators for programmes operating in both modes received a fee for their work, there were no formal contracts for this activity. For University of Northumbria staff moderating the HEFC, it was regarded as a part of their roles as academic staff within the University, and was included within appraisal interviews as a topic for performance review. For those moderating programmes operating in centralised mode, it was recognised that this was a matter to be addressed. The team noted that an appointment as moderator in this mode may be terminated on either side at any time; the team was concerned at the contractual situation that this represented, with possible implications for learners and award of certificates should a moderator withdraw at particular times of the year.

77 In devolved mode, moderators are invited to an annual Moderators' Forum, organised by the HEFC Co-ordinator, and are provided with a mentor, also by the HEFC Co-ordinator. The Forum was described as an opportunity for discussion of issues of mutual concern, for example the workload of moderators, and an opportunity to consider comparability. In centralised mode, moderators receive induction training, and are also invited to attend HEFC Moderators' Forum meetings.

The moderator's role

78 The role of the moderators for centralised mode programmes is set down in the Quality Assurance Procedures and the responsibilities of moderators for the HEFC are set down in the DCD. These are broadly comparable, with the exception of those responsibilities for HEFC moderators which refer to internal HEFC procedures. In devolved mode, the moderator, as a member of a MADG, is fully involved in agreeing a schedule for the assessment and moderation process, a common marking scheme, agreeing the examination paper and the conduct of the Examination Board. In both modes moderators are expected to visit their programme twice per year and, on both occasions, to meet with teaching staff, students (informally) and to see a representative sample of written work, the latter to confirm the overall standard within the programme and to assure, as far as possible, comparability with other programmes. A draft moderators' handbook has been produced, although it was not clear whether this had yet been issued to any moderators. This handbook includes guidance for both HEFC and centralised mode programmes, with the procedures for the two different systems generally being presented separately.

79 The manner in which moderators exercised their roles and responsibilities did not seem entirely consistent. The review team was interested to learn, for example, that there were differences in the proportion of sampling undertaken, and in the level of scrutiny of assessment, with variations in the extent of 'second marking' of work.

Moderators' reports and responses

80 There are guidelines for moderators' reports for both modes which specify areas for comment for moderators, although these appear more fully developed for programmes operating in centralised mode. HEFC moderators' reports are scrutinised by the HEFC Co-ordinator and immediate action may be taken at this stage, if necessary. Points of concern are incorporated within a summary of issues that forms part of the HEFC annual review document. This document is considered by the HEFC Board and subsequently by the TBCAP Quality Committee.

81 Copies of the moderator's report for programmes in centralised mode are received by the TBCAP office and forwarded to the relevant provider. Should any matters of concern arise, the Development Officer is responsible for taking these forward. The review team noted that the documentation provided by TBCAP stated that the Quality Committee has a remit to ensure consistency regarding reporting and action following receipt of the report. The documentation detailing the AVA's quality assurance procedures also specifies that copies of moderators' reports 'go to the TBCAP Quality Committee to feed into the audit process'. In this context, the team noted that there was no record of consideration of any moderation reports at any meeting for which agendas and minutes were supplied, and members of the Quality Committee were unable to explain why this was so. The failure of TBCAP to implement Stage 8 of its own quality assurance procedures, namely Audit, appears to be confirmed by this omission.

82 The review team concluded that it was difficult to see how moderation, as an entire process, was being used by TBCAP, either to monitor standards of student achievement, or to review and enhance the quality of provision.

Award and issue of certificates

83 In devolved mode, the MADG Examination Board makes recommendations for awards, on the basis of marks achieved, and any extenuating circumstances, to the HEFC Board for the Board's approval. Programmes operating in centralised mode also have examination or assessment boards, held at the providing institution, which confirm assessment results and the award of 'kitemarked' certificates. Both the HEFC Board and the individual course boards for centralised mode programmes then forward results to the University of Northumbria's Access Office. This office oversees the processes leading to the issue of certificates, which involve appropriate checking mechanisms by the HEFC Development Officer and the TBCAP Officer, for the two different parts of the operation, and which appeared to the review team to be appropriately careful and thorough.

84 The award of certificates is the first of the three quality assurance procedures which, according to TBCAP's own regulations, cannot be devolved. The Quality Assurance Procedures document states that 'The award of kitemarked access certificates to students is made by TBCAP, in all modes of operation...'. The review team was therefore surprised to note that, although the HEFC DCD confirms this situation, the terms of reference of the HEFC Board also include responsibility for 'the award of Kitemarked Access Certificates on behalf of TBCAP'. The review team was unable to find any evidence that this AVA responsibility had been formally given over to the HEFC Board.

85 It was apparent that it was the HEFC Board that awarded 'kitemarked' certificates in the devolved mode of operation, without reference to the TBCAP Council or other body, in spite of the regulations of TBCAP's own quality assurance procedures that this is a responsibility that it reserves to itself. Indeed, in relation to this matter, representatives of the AVA with whom the team spoke seemed unaware of this contradiction between the AVA's regulations and its operation, and readily acknowledged that the sequence of meetings of the TBCAP Council and HEFC Board would not allow results which had been approved by the HEFC Board to then be forwarded for approval by the TBCAP Council.

86 The review team concluded that 'kitemarked' Access certificates were being awarded by a body which did not have the formal authority to do so. That this was being allowed by the licensed AVA, without any apparent awareness of the contravention of its own quality assurance procedures or QAA's requirements, led the team to further conclude that it could not be confident that TBCAP was exercising its authority as an AVA.

Progression to HE

87 All three member HEIs reported strong commitment to Access courses and to encouraging progression to their institutions by Access students. Within the context of various factors such as diversity of requirement by discipline, this seemed to the review team to be borne out in practice. The close involvement of HE staff in some TBCAP programmes facilitated progression, since the receiving institution had close involvement in the order and delivery of the Access curriculum. While this was particularly true for the HEFC programme, it was also the case with the Access to Engineering programme which has close links with the Engineering Department at the University of Newcastle.

88 The review team noted the positive impressions of HE admissions staff concerning the comparative motivation, preparation and level of achievement of Access students once in HE, as compared with 'A' level entrants. The team also noted the view expressed that student withdrawal was, in almost all cases, caused by personal rather than academic circumstances.

89 While the 'kitemarked' Access award appeared generally to be understood by those representatives of local HE institutions met by the review team, the team noted that the currency for offers to students on the HEFC course was commonly the successful completion of a certain number of HEFC modules, often at particular grades (for example, 'merits' or 'distinctions'), rather than the achievement of the 'kitemark'. The common unit of currency for those involved in the HEFC course appeared to be the HEFC module, rather than the Access programme as a whole. This common currency was also apparent among students, who spoke of 'doing HEFCs'. While this appears to work well in the local HE context, where the HEFC is well known and respected, students reported some difficulty when applying elsewhere where the HEFC was unknown.

90 The review team was concerned that TBCAP may not have yet fully addressed the implications of this situation, and its responsibility as a licensed AVA to promote Access to HE. The team considered that there was more work to be done among the AVA's partners in relation to the promotion of the value of the 'kitemarked' Access certificate.

Monitoring, review and evaluation

91 At course delivery level, provision is monitored by moderators, who report to the AVA. Evidence available to the review team suggested that moderators had close involvement with courses operating in both centralised and devolved modes. In certain respects this role was appropriate and robust. However, the team had some concerns about the adequacy of the monitoring of this process by TBCAP, and about the use made of its outcomes in the review and evaluation of the AVA's recognised provision as a whole (see above, paragraphs 81-82).

92 The Quality Assurance Procedures identifies two elements under 'Stage 7: Review': 'Annual Review', and 'Periodic Review and Revalidation'. As Review is one of the three procedures identified as the direct responsibility of the AVA, both of these elements are the direct responsibility of the AVA for all programmes, regardless of operational mode. The evidence available to the review team suggested that programme revalidation had been operated appropriately in relation to courses operating in centralised mode, including the consideration of revalidation decisions by the TBCAP Council. The team was informed that the periodic review and revalidation of the HEFC was planned to take place during 2000-01.

93 The AVA guidance on the requirements for Annual Review is presented in a separate appendix to the Procedures, and consists of a single sentence: 'All programmes will have arrangements for Annual Review'. The monitoring of this process involves the submission of course reports to TBCAP and, during 1998-99, the TBCAP Development Officer introduced a standardised annual review report form. In 1999, this was used for the submission of annual reports by providers of courses operating in centralised mode, although the HEFC continued to use its own, much more extensive, format for annual course review. The HEFC annual course report reflects a wide spectrum of monitoring and evaluation activities undertaken by the HEFC Board, and suggested that careful consideration was being given to data on individual students, examination outcomes, recruitment figures and progression statistics, as well as the review of modules. The report also includes a section on 'action arising from annual course review' for which there is no comparable requirement in TBCAP's standardised annual review form. The review team concluded that HEFC provision is subject to rigorous annual monitoring, deriving principally from University requirements.

94 For non-HEFC courses, there is no equivalent process of scrutiny and evaluation being undertaken by a body external to the provider. This may take place among individual course teams but, as the reporting mechanism does not require it, the AVA has no means of knowing whether the courses for which it holds direct responsibility are subject to such detailed consideration. The review team observed that, while the HEFC Consortium appeared to have well-established and effective procedures for monitoring, evaluation and review, these owed little to TBCAP, whose own procedures were limited and under-developed. Paradoxically, therefore, there is much tighter control of standards and quality in the devolved mode than in the centralised mode of operation.

95 According to the Quality Assurance Procedures, annual course reports are forwarded to TBCAP for inclusion in the Audit process through which all the other stages of quality assurance are also reviewed. The procedures for Audit include specific mention, among the list of documents to be sampled, of annual review reports, as well as validation reports and moderators' reports, and a range of other documentation related to the quality assurance of courses. However, during the review, the Quality Committee confirmed that the Audit process had not yet been established and no such sampling activity had taken place.

96 For the academic year 1997-98, non-HEFC course reports were submitted to the TBCAP Access office; the TBCAP Development Officer prepared a summary report based on the information received; and the Quality Committee received the summary report, without reference to the reports themselves. The HEFC annual report was not, however, included in the summary report. It was presented to the Quality Committee as a separate document by the HEFC Co-ordinator, and the presentation of the two reports occurred under two separate agenda items at a meeting of the Committee in May 1999. There was thus no direct comparison of all courses, or of the quality of course reports, either by the Development Officer or by the Committee. Neither report appeared as a discussion item on the agenda, and the relevant minutes make no reference to any discussion having taken place. At the time of the review, there had been no consideration by the Quality Committee of the course reports for 1998-99.

97 In addition to the question of whether the receipt of a summary report provides the same level of monitoring for those courses for which the AVA has direct responsibility for quality assurance, as characterised by the oversight by the HEFC Board of the devolved mode provision, the review team had concerns about how the AVA intended to manage this stage of its quality assurance procedures in the future. The intention to replace the Development Officer post with a junior administrative post would provide this process with limited support in the AVA's envisaged model of fully devolved operation. The team concluded that the current process of course monitoring was vulnerable under the proposed changes to fully devolved status.

98 Though TBCAP Council members whom the team met expressed confidence in the Quality Committee to undertake monitoring of provision on their behalf, evidence from the Quality Committee's minutes indicated that not only was it failing to implement the AVA's approved quality assurance procedures, which would involve the direct scrutiny of a range of programme documentation, but also that it was not operating an effective alternative process for the critical evaluation of centralised mode programmes, nor for the scrutiny of annual reports for all programmes. The agenda for the Council meeting on 20 October 1999 includes the consideration of 'the Annual Reports from TBCAP validated programmes for 1998-99', but there is no minute which refers to this item. As the TBCAP Council does not receive the Quality Committee's minutes, it did not apparently receive any information about the review of courses. The review team therefore concluded that TBCAP was not exercising its responsibilities as an AVA in relation to course review.

99 In the absence of the kind of mechanisms envisaged under Audit, TBCAP is left without an effective way of ensuring the continuing appropriateness of programmes, the quality of their delivery or the standards of their outcomes. It may well be that there is no problem with any of these in any of the programmes validated; however, TBCAP has no secure way of knowing whether or not this is the case. It is also left without a method of fulfilling its obligation under the QAA scheme to ensure consistency between programmes leading to the 'kitemarked' award, a particular issue for this AVA given its preference for a highly devolved structure.

100 The devolved structure adopted by this AVA requires special mechanisms for monitoring the activities of devolved partners, if TBCAP is to exercise its responsibilities as an AVA under the terms of the licence. While the HEFC annual review report provides a detailed account of the HEFC as a course, and includes some information about the course's management, there is no formal requirement for the HEFC Consortium to report to TBCAP as a devolved body (see above, paragraph 48). The quality assurance activities of the HEFC Consortium are conducted as an internal matter by the Consortium, operating under devolved arrangements, and TBCAP's knowledge of the operation of HEFC procedures is confined to the receipt of the HEFC annual report, the contents of which are not specified by TBCAP. In relation to the HEFC's procedures, the Account states that 'The HEFC is a mature course with well tried and tested procedures which are subject to regular refinement and change rather than formal evaluation'. In the light of this absence of self-evaluation by the HEFC Consortium of its own procedures, the review team considered the absence of any evaluation by the AVA to be a significant omission in its responsibility for the oversight of the devolved part of its operation. The team concluded that TBCAP had not put in place any specific safeguards, either through annual monitoring or through periodic review, which would ensure that it received regular assurance about the responsibilities which the HEFC Consortium was exercising on its behalf.

101 In considering the framework for the current review, the review team concluded that TBCAP was exercising only limited authority as an AVA in the implementation of most of its quality assurance procedures. In relation to the HEFC, the team concluded that, having devolved most stages of quality assurance, and having failed to implement others, TBCAP was exercising very little authority over the HEFC Consortium at any stage.

Conclusions

102 The review team considered the full range of the AVA's structures, systems and activities, in accordance with the procedures for AVA review specified in the Recognition Scheme, and reached its conclusions after consideration of the totality of the AVA's operations and procedures.

103 The context for this review, with an early date being set to follow up a number of outstanding concerns after the last review obliged the review team to consider especially carefully those matters of particular concern to QAA, expressed in earlier correspondence with TBCAP. The review team therefore considered its conclusions with particular reference to the four matters that the AVA had been informed in December 1998 would form the focus of the review.

i 'efficient, effective and rational committee and management structures'

104 TBCAP's committee and management structures appear to be efficient, in the sense that they are not unnecessarily burdensome or wasteful of resources. However, the efficiency of both types of structure is less certain if measured in terms of their fitness for their particular purpose: the provision of a successful means of exercising the responsibilities of a licensed AVA.

105 While the committee structures are adequate for certain aspects of the oversight of programmes operating in centralised mode, they do not provide a clear connection to those parts of the AVA's operation which are devolved. As the devolved operation accounts for the major part of the provision which is recognised by TBCAP, this omission seriously diminishes the ability of the committee structures to provide effective control for the AVA as a whole. TBCAP has devolved to the HEFC Consortium most of the responsibilities of an AVA, and has allowed it to assume other responsibilities which have not been formally devolved. The claim that the HEFC Consortium is accountable to TBCAP is not supported by any structures which can assure this. The central body which holds the AVA licence has no means of holding the devolved body to account for the decisions it takes in relation to the Access provision which is recognised under the aegis of TBCAP. Committee structures which have neglected to recognise or address the accountability of a body exercising responsibilities on the AVA's behalf, cannot be considered effective.

106 In the absence of a developed operational base for the work of the AVA, the successful operation of the committees is key to the management of the AVA's work. The committee structure is light; it thus falls to the effective operation of two or three committees to ensure the AVA is properly managed. The Council's inattention to its responsibilities for overseeing the work of its sub-committees has allowed severe weaknesses in the operation of the Quality Committee to pass apparently unnoticed. The Quality Committee itself has proved an ineffective mechanism for the development and implementation of the AVA's quality assurance procedures at a number of points. The committee structures also fail to provide an effective means of linking the governance and the day-to-day management of the AVA, resulting in inadequate monitoring of the operation of the AVA by its governing body.

107 The management structures provide limited independence for the AVA, as the management of the AVA is inseparable from its relationship with the University of Northumbria. Without the University's support in terms of premises, some staff support and administrative costs borne, it is doubtful that the AVA would be viable. The choices of the AVA about its management structures are necessarily constrained by this position, rather than by considerations of the most effective or rational structures for the work of an AVA. Given the University of Northumbria's direct responsibility for, and involvement in, the HEFC, and the common operational management of the two bodies, the effective independence of the AVA must be called into question.

108 The proposed new structures would exacerbate the present difficulties by weakening the AVA's operational support and would compound the problem identified by devolution to a second body (NEOCN). They would also extend to another body the unsatisfactory situation which pertains to the current devolved partner, through a process which itself is under-developed. The entirely devolved model, as it is currently conceived, would leave at the centre a body holding an AVA licence, but which would have little real power and no effective means of exercising that power.

ii 'transparent processes which pay due attention to accountability in decision-making and reporting procedures of committees and individuals'

109 Members of the TBCAP Council may be aware of decisions being taken by the devolved partner, through the cross-membership of committees. However, this does not provide a formal reporting mechanism, and there is no transparent process for the reporting of decisions made. There is no clear mechanism that ensures that a devolved partner is accountable to the AVA for the decisions that it takes. The plans for the future appear further to devolve authority to partner bodies with no effective controlling mechanism by the central body which carries the licence.

110 The reporting procedures of the AVA's committees are unclear, and are weak in operation. Sub-committees do not make regular reports, either written or oral, to the TBCAP Council. In particular, the TBCAP Council is not properly aware of the actions of the Quality Committee, and does not effectively direct its activity or monitor its achievements.

111 Individuals employed by the AVA are not subject to transparent reporting procedures that demonstrate their accountability to the AVA (as opposed to the organisation which employs them), and their roles on committees are not always clear. For example, the Council is the only committee in which the membership of paid officers of the AVA is specified. However, as the principle of the AVA's operation involves reliance on 'the expertise of those of its members which are accreditation bodies', the exact nature of the role of officers of those bodies on TBCAP committees is unclear.

112 According to the terms of its constitution, TBCAP 'will seek to enter into partnership...with one of the member institutions which is already an employer' in making arrangements for employment of staff. In practice, however, it is, and has always been, the University of Northumbria which acts as the employer. Although the AVA's employees are accountable formally to the Council, responsibility for their everyday management and appraisal lies with the University's Head of Access and Continuing Education Service, who in this instance is acting on behalf of the TBCAP Council. This same individual sits on the TBCAP Council as a representative of the University of Northumbria; chairs the TBCAP Quality Committee (representing Council) and sits on the Financial and Strategic Planning Group (as Chair of the Quality Committee). He also has a number of roles in relation to the HEFC Consortium, being Vice-Chair of the HEFC Board; chairing the HEFC Committee, the Senior Managers' Steering Group and the Development Project Sub-Committee; and sitting on the HEFC Principals' Group and HEFC Board Examination Sub-Committee. This network of responsibilities vested in a single individual, who represents a range of different bodies with potentially different interests at different points, provides an extraordinarily intricate set of lines of accountability which is not conducive to the transparency of the AVA's operations and decision-making.

iii 'progress by the Quality Committee in carrying out its stated intentions and programme of work'

113 The correspondence from TBCAP which prompted this requirement included the statement that 'the Quality Committee has particular responsibility for ensuring that audit procedures are fully operationalised'. In this respect, it is evident that the Quality Committee has made no identifiable progress in carrying out its stated intentions and programme of work, nor are its current activities clearly directed towards this.

114 The AVA also stated in 1998 that it was the Quality Committee that would be comparing the results of each year's operation as outlined in a range of reports, and would 'seek to standardise processes and evidence'. The review team found that the Quality Committee had not directly considered reports, nor was there any evidence that it had been involved in a comparison of results, although the Development Officer may have engaged in this activity on the Committee's behalf. The standardised annual course report appears to be a result of this activity.

115 While the Quality Committee has considered some matters of importance to the AVA and to its quality assurance, it has not fulfilled its remit, nor has it made progress in the specific areas which the AVA identified as its immediate priorities. Where some progress in quality assurance procedures has taken place, initiatives have come from the Development Officer. Without this post, as would be the case in the planned new structures, the potential of this committee to address such matters is questionable.

116 Outside of the work of the Quality Committee, there are no effective mechanisms through which the quality of the provision or of the operation of the AVA as a whole is being overseen by TBCAP.

iv 'evidence that TBCAP is exercising its authority as an AVA, and carrying out its responsibilities properly and fairly in relation to all TBCAP partners'

117 TBCAP is failing to exercise its authority as an AVA in three major respects. First, it is allowing the award of 'kitemarked' certificates to students to be made by a body which does not hold an AVA licence, and which has not been granted devolved authority to make awards. Second, it is not adequately monitoring provision, in part because it has failed to implement all stages of its own quality assurance procedures. While this is 'the particular responsibility of the Quality Committee', the inadequacy of reporting procedures has led to a failure of the Council, which carries ultimate authority for the licence, to recognise and address this. Third, it has devolved AVA responsibilities to a separate body, without putting in place adequate mechanisms to monitor or review the way in which that body exercises its devolved responsibilities, and without ensuring the accountability of that body to TBCAP, as the AVA licence holder.

118 Although the AVA is not acting unfairly to its partners, the devolved partner is effectively being allowed to act independently. The intention to devolve responsibility for all the remaining part of its recognised provision to the other accreditation partner on the same basis may result in a fairer situation, in that all parts of the AVA's recognised provision would be subject to equivalent treatment. This situation would not, however, assist the AVA to carry out its responsibilities properly, as the body which actually held the AVA licence would not be able to exercise the responsibilities associated with that licence.

119 After full and careful consideration, the review team formed the view that there were substantial weaknesses in all four of the areas which QAA had informed the AVA it should address. These four matters go to the heart of the responsibilities of an AVA and the team therefore concluded that it could not recommend the renewal of AVA licence to TBCAP. In reaching this decision, the review team acknowledges some positive features of TBCAP. The team also emphasises that the recommendation does not imply that the provision of Access programmes is necessarily inadequate. It was not the team's role to evaluate the provision directly; rather it was charged with evaluating TBCAP's discharging of its responsibilities as a validating agency.

Recommendation to the ARLC

120 The review team recommends that the licence held by TBCAP to act as an authorised validating agency should be withdrawn.

Tyne and Border Counties Access Partnership October 2000

Contents

Foreword

Aims and objectives

Outcomes

The review process

The AVA context

AVA statistics 1998-99

Background

Membership

Provision

Aims and objectives

Developments

Context for the review

Organisational and operational structures

Finances

Data collection and analysis

Learner experience

Communications with providers

Staffing

Governance and committee structures

Overview of structures

The TBCAP Council

Financial and Strategic Planning Group

Quality Committee

The quality assurance of Access to HE programmes

Overview of procedures

Development, validation and approval

Assessment and accreditation

Comparability and standardisation

Selection, appointment and training of moderators

The moderator's role

Moderators' reports and responses

Award and issue of certificates

Progression to HE

Monitoring, review and evaluation

Conclusions

Recommendation to the ARLC

Tyne and Border Counties Access Partnership
October 2000