QAA 086 09/05Annex A
Schedule for GOsC review
Review timing
1 The timing of the GOsC reviews and their reporting will be such as to enable the GOsC to pass its own recommendation to the Privy Council in a timely fashion for a statutory decision of recognition. In particular, the planning of reviews will normally be made in consultation with the GOsC in respect of its timetable for Education Committee meetings.
2 The GOsC has a protocol stipulating the minimum period that must elapse between a failed application and review for initial recognition of a programme and any subsequent application and review. Intending providers of osteopathic courses should consult the GOsC on the protocol currently in force.
Annex B
Aide-mémoire for the GOsC review
1 This aide-mémoire is intended to support:
- the provider's preparation of an SED
- the visitors' analysis of the SED before examining the provision
- the visitors' collection of evidence during the review.
2 The aide-mémoire covers the main features of the review process, but it is not intended to be prescriptive or exhaustive. The provider's SED, the statement of aims and the intended learning outcomes of programmes may all raise questions particular to the programme under scrutiny. The visitors will use SEDs in any reasonable form, provided that they contain the information that the visitors need to plan and conduct the review.
Guidance for providers when preparing self-evaluations
3 Each review will begin with the supply by the teaching institution of an SED. The teaching institution's SED should pay particular attention to any comments or conditions present in any recent previous reviews. The SED should be accompanied by the definitive course document and/or by the programme specification.
4 In 2005-06 reviews, the annual report of a provider may form the SED, or a first SED to which the provider may add fresher elements.
5 The range of activities and documents reviewed will vary from one review to another. Providers should take care to ensure that their SED meets the needs of the review and answers the visitors' questions. In the interests of keeping preparation to a minimum, QAA encourages each provider to send a collection of current review and evaluation documents as the SED rather than a purpose-written document. Providers will find it helpful to refer to the prompts and questions in the aide-mémoire below. Providers should also refer to the GOsC's Standard 2000 and to the components of the Academic Infrastructure, such as the Code of practice and the FHEQ.
6 The SED should discuss both the strengths and the areas for improvement the provider has identified in the subject to be reviewed. It provides an opportunity for the provider to demonstrate how previously identified strengths have been built upon and how any identified areas for improvement have been addressed. Where areas for improvement remain, existing plans for addressing these should be discussed. The visitors will give credit where appropriate remedial plans are in place to address acknowledged areas for improvement, provided students are experiencing a programme of the adequate standards of, at least, adequate quality.
7 Effective self-critical evaluation makes reference to the sources of evidence, the analysis of which has informed reflection. Sources of evidence should be cited and could usefully be included as a list of references. These references will help the visitors in gathering evidence, and avoid the need for providers to include merely descriptive material in the evaluation. The references quoted in the SED should be made available to the visitors during the review period so that they may triangulate evidence gathered with evidence cited.
8 The process of review focuses on governance and management (including the maintenance of academic standards by the provider), the achievement of standards by students and the quality of the learning opportunities offered. Governance, standards or quality cannot be reviewed in isolation: they are interrelated and must be reviewed as such.
9 The visitors will need factual information about the programme under review. There must be a clear statement of the overall aims of the subject programme, which should not exceed 200 words in length. The visitors will use this to assess whether the programme achieves its broad aims. The statement of aims will be reproduced at the start of the academic review report. Overall aims will reflect the distinctive mission of the provider's osteopathic programme(s).
10 Statements of aims should be succinct but convey clearly the parameters of the subject programme. They may be presented as narrative statement, bullet points, or a mixture of the two. Precise, definitive programme titles should be stated.
11 The evaluation of the programme(s) should not normally exceed 6,000 words in length. The programme specifications/definitive course document form part of the SED and should be appended to provide the detail of programme aims, intended learning outcomes and the means by which they are achieved. Factual material provided in the programme specifications/definitive course document need not be repeated elsewhere. Where the programmes in the subject/unit of review are organised and delivered through different departments or schools, clarification of the academic structures and scope of responsibility for the programmes will be helpful to the visitors. This should include information about relevant modular structures, internal and external collaborative arrangements and relationships with local, regional or national employers.
12 The visitors need statistical data on student achievement and progression for the last three student intakes that have completed the programme(s). This requirement necessitates more than three years of entry data so that the visitors can see the data for the students who have completed the programme(s). For example, for a three-year degree programme:
| 2000-01 | 2001-02 | 2002-03 | 2003-04 | 2004-05 | 2005-06 | |
| Year one | Intake A | Intake B | Intake C | Intake D | Intake E | Intake F |
| Year two | Intake A | Intake B | Intake C | Intake D | Intake E | |
| Year three | Intake A | Intake B | Intake C | Intake D |
13 Statistical indicators should include data on recruitment and admissions; entry profile (qualifications, age, gender, ethnicity - profile data relevant to mission and aims); rates of progression and completion; student achievement in summative assessment; and progression of completing students to employment and further study. There is a need to distinguish between students who complete and gain the award, and students who complete without gaining the award. Visitors will need to know the number of students who enrol for the first year of each programme and progress to complete the programme and gain the award.
14 The data should distinguish between those students in the first-year entry cohort, those joining directly at subsequent stages, withdrawals (including reasons for withdrawal), referrals (showing those subsequently failing and those passing), failures and those achieving the award. It is important that such data are presented in a suitably disaggregated form to clearly inform the visitors of the levels of achievement, such as the number of passes in each degree classification. Referrals and failures should also be included. The visitors will be interested to know how the provider uses such data to evaluate, manage, plan and enhance academic quality and standards.
Guidance for review teams
15 Visitors will respect the principle of proportionality in the degree of scrutiny and the weight of reporting. A well-written SED and the efficient provision of evidence by the provider should allow visitors to confirm many SED statements quickly.
Aims and outcomes
16 The SED should address the appropriateness of the intended learning outcomes in relation to the overall aims of the provision, Standard 2000 and the FHEQ. The SED should discuss the effectiveness of measures taken to ensure that staff and students have a clear understanding of the aims and intended learning outcomes of the programmes.
17 The visitors will consider:
- how well the intended learning outcomes relate to the overall aims of the programme and whether they enable the aims to be met
- the extent to which they are aligned with external reference points, including the FHEQ, to provide an appropriate level of challenge to students
- the extent to which they are aligned with Standard 2000
- how well the intended learning outcomes of a programme and its constituent parts are communicated to staff, students and external examiners/verifiers.
18 The definitive course document/programme specification will provide details of the overall aims and intended learning outcomes for each award. Module or unit descriptors will provide a further level of detail. Review activities may include analysis of module and level descriptors, consideration of the balance and range of outcomes relating to subject, professional, practical and transferable skills, and the relationship between the intended learning outcomes for each award and its title. The visitors will engage in discussions with staff and students, and review the reports of external examiners/verifiers. Student handbooks and curricular documents will provide factual information and will illustrate how aims and outcomes are communicated to staff and students.
19 The visitors should be able to judge whether intended learning outcomes are clearly stated and are appropriate to the level of the award, meeting the requirements of relevant external reference points.
Curricula
20 The SED should review the effectiveness of curriculum design and content in enabling the intended learning outcomes to be achieved.
21 The visitors will consider:
- how the provider plans the curriculum design and content and how decisions about contributing modules and their sequencing are made
- whether the design and content of the curricula encourage achievement of the intended learning outcomes in terms of knowledge and understanding, cognitive skills, subject-specific skills (including practical/professional skills), transferable skills, progression to employment and/or further study, and personal development
- the extent to which curricular content and design are informed by recent developments in techniques of teaching and learning, current research, scholarship or consultancy and by any changes in relevant occupational or professional requirements
- how the provider ensures that the design and organisation of the curriculum provide appropriate academic and intellectual progression and are effective in promoting student learning and achievement of the intended learning outcomes.
22 The design and content of each programme will be evaluated in relation to its potential for enabling students to achieve the intended learning outcomes. Sources of information may include curricular documents, review reports, reports from professional bodies, placement reports from employers, course and student handbooks and module descriptions. From September 2005, the visitors will enquire into how appropriate sections of the Code of practice have been considered. For example, the approval, monitoring and review of programmes and placement learning, where appropriate.
23 The visitors will be able to judge whether the curriculum is appropriate to each stage of the programme and to the level of the award.
Assessment
24 The SED should review the effectiveness of student assessment in measuring the achievement of the intended learning outcomes of programmes.
25 The visitors will consider:
- the extent to which the overall assessment strategy has an adequate formative function in developing student abilities, assists them in the development of their intellectual skills and enables them to demonstrate achievement of the intended learning outcomes in all learning settings
- the assessment methods selected and their appropriateness to the intended learning outcomes, and to the type and level of work
- the criteria used to enable internal and external examiners/verifiers to distinguish between different categories of achievement, and they way in which criteria are communicated to students
- the security, integrity and consistency of the assessment procedures, the setting, marking and moderation of work in all learning settings, and the return of student work with feedback
- how employers and other professionals contribute to the development of assessment strategies, where appropriate.
26 The visitors will seek sources of evidence to help them evaluate whether the overall assessment process is appropriate and effective, and will seek evidence of clear guidance about the assessment arrangements. Where appropriate, the visitors will evaluate the contribution of assessment of any placement learning to the final award and the responsibilities of the parties involved. The review of samples of marked student work, annual review reports, external examiners'/verifiers' reports, statistical data and discussions with staff and students will help the visitors to evaluate the contribution of assessment to their overall view on standards. The visitors will evaluate the appropriateness of assessment tools, the competence of those involved with the assessment arrangements and whether the assessment strategy reflects progression, integration and coherence. From September 2005, the sections of the Code of practice dealing with assessment of students (Section 6) and external examining (Section 4) will be important points of reference, as may be the section on collaborative provision (Section 2).
27 As a result of these activities, the visitors should be able to judge whether assessment procedures securely measure the achievements of the intended programme outcomes.
Achievement
28 The SED should review evidence of the extent to which students achieve the learning outcomes set.
29 The visitors will consider:
- the evidence that students' assessed work demonstrates their achievements of the intended learning outcomes
- the evidence that standards achieved by learners meet the minimum expectations for the award as measured against the FHEQ, Standard 2000 and subject benchmark statements
- whether students are prepared effectively for their subsequent employment roles
- the levels of achievement indicated by the statistical data, whether there are any significant variations between modules and the successful progression to employment
- how the provider promotes student retention and achievement.
30 The visitors will then evaluate whether student achievement meets such expectations. Sources of information will include external examiners'/verifiers' reports, any placement or clinical practice supervisors' reports, assessment board minutes, samples of student work, and statistical data on achievement and career destinations. Review activities will include discussions with the teaching team, internal examiners/verifiers and students. Standard 2000, subject benchmark statements and the level descriptors of the FHEQ will be important points of reference.
31 As a result of these activities, the visitors should be able to judge whether appropriate standards are being achieved.
Teaching and learning
32 The SED should review the effectiveness of teaching and learning, in relation to programme aims, the intended learning outcomes and curriculum content.
33 The visitors will consider:
- the range and appropriateness of teaching methods employed in relation to curriculum content and programme aims
- how staff draw upon their research, scholarship, consultancy or professional activity to inform their teaching
- the ways in which participation by students is encouraged and how learning is facilitated
- how the materials provided support learning and how students' independent learning is encouraged
- student workloads
- how quality of teaching is maintained and enhanced through staff development, peer review of teaching, integration of part-time and visiting staff, effective team teaching and the induction and mentoring of new staff.
34 The visitors will evaluate the overall effectiveness of the teaching and learning activities on the programmes under review; in particular, the breadth, depth, pace and challenge of teaching, the effectiveness of the teaching of subject knowledge and the effectiveness of the teaching of subject-specific, transferable and practical skills. Sources of evidence may include student evaluation of their learning experience, internal review documents, staff development documents, course and student handbooks and discussions with staff and students. Review activities will normally include direct observation of both clinical and non-clinical teaching. A note on the observation of teaching and how it is recorded is in Annex K.
35 As a result of these activities, the visitors will be able to judge the quality of learning opportunities offered to students through the programme of teaching and learning and its contribution to achievements of the overall aims of the programme.
Student progression
36 The provider should evaluate the effectiveness of strategies for recruitment, admission and academic support and guidance to facilitate students' progression and completion of the programme.
37 The visitors will consider:
- the effectiveness of arrangements for recruitment, admission and induction, and whether these are generally understood by staff and students
- the overall strategy for academic support and its relationship to the student profile and the overall aims of the programme
- how learning is facilitated by academic guidance, feedback and supervisory arrangements
- the arrangements for academic tutorial support, their clarity and their communication to staff and students, and how staff are enabled to provide the necessary support to students
- the quality of written guidance
- the extent to which arrangements are in place and effective in facilitating student progression towards successful completion of their programmes.
38 The visitors will evaluate the appropriateness of the overall strategy for academic support, including written guidance and the extent to which it is consistent with the student profile and the overall aims of the programme. They will evaluate whether there is appropriate matching of the abilities of students recruited to the demands of the programme and whether there are appropriate arrangements for academic guidance and support to facilitate progression, completion and non-completion. The visitors will consider progression within programmes as well as non-completion rates. Sources of evidence may include statistical data on application, admission, progression and completion, policy statements on admission and learning support, course and student handbooks, student evaluation of admission, induction and tutorial support and discussions with staff and students. From September 2005, sections of the Code of practice on recruitment and admissions (section 10), students with disabilities (section 3), placement learning (section 9), academic appeals and student complaints on academic matters (section 5), and career education, information and guidance (section 8) are relevant.
39 As a result of these activities, the visitors will be able to make judgements about the quality of learning opportunities in support of student progression.
Learning resources
40 The SED should review the adequacy of human and physical learning resources and the effectiveness of their utilisation. In particular, the evaluation should demonstrate a strategic approach to linking resources to intended learning outcomes at programme level.
41 The visitors will consider:
- staffing levels and the suitability of staff qualifications and experience, including teaching and non-teaching staff
- professional and scholarly activity to keep abreast of emerging, relevant subject knowledge and technologies
- research activity
- staff development opportunities, including induction and mentoring for new staff, and whether opportunities are taken
- library facilities including relevant and current book stock
- journals and electronic media
- access times and arrangements, and induction and user support provision
- computing hardware, both general and subject-specific software availability, and currency
- accessibility, including times of opening and opportunities for remote access, and induction and user-support provision
- specialist accommodation, equipment and consumables
- adequacy, accessibility, induction, user-support and maintenance
- suitability of staff and teaching accommodation in relation to the teaching and learning strategy and the provision of support for students.
42 The visitors will evaluate how effectively learning is facilitated through the overall deployment of resources, including whether appropriate technical and administrative support is made available, and the appropriateness of staff development strategy and practice. Sources of information will include direct observation of physical resources, internal review documents and minutes of meetings, equipment lists, library stocks, staff curricula vitae, external examiners'/verifiers' reports and staff development documents. The visitors will meet staff and students.
43 As a result of these activities, the visitors will be able to judge whether the learning resources available successfully underpin the programmes and whether there are appropriately qualified staff who are contributing effectively to the achievements of the intended learning outcomes.
Governance and management
44 The provider should evaluate its governance and management, including financial and risk management, and the effectiveness of measures taken to maintain and enhance academic standards and the quality of learning opportunities.
45 The visitors will consider the provider's governance, management, financial control and quality assurance arrangements in order to determine whether they are sufficient to manage existing operations and respond to development and change.
Indicative evidence
46 The provider should be able to demonstrate that:
- its academic and financial planning, quality assurance and resource allocation policies are coherent and relate to its mission, aims and objectives
- there is a clarity of function and responsibility in relation to its governance and management systems
- across the full range of its activities, there is demonstrable strength of academic and professional leadership
- policies and systems are developed, implemented and communicated in collaboration with staff and students
- its mission and associated policies and systems are understood, accepted and actively applied by staff and, where appropriate, students
- it is managing successfully the responsibilities vested in it by its validating university and the GOsC
- its operational policies and systems are monitored, and it identifies where, when, why and how changes might need to be made
- there is reliable information to indicate continued confidence and stability over an extended period of time in the organisation's governance, financial control and quality assurance arrangements, and organisational structure.
Maintenance and enhancement of standards and quality
47 The visitors will consider:
- the provider's approach to the quality assurance of its provision and the effectiveness of this approach for the programmes under review
- the use made of quantitative data and qualitative feedback from students, external examiners/verifiers and other stakeholders in a strategy of enhancement and continuous improvement
- the provider's responsiveness to internal and external review and assurance processes
- the accuracy of the SED.
48 The visitors will evaluate how well the internal mechanisms for assuring academic standards and quality are working. Sources of evidence will include student and staff feedback, external examiners'/verifiers' reports, quantitative data, employers' views, previously published subject review reports, other reports if available and internal review reports.
Submission of the self-evaluation for analysis by visitors
Introduction
49 This section sets out the procedures for handling the SED prior to the start of a review. This detail has been included in this annex to help providers understand the processes that their SED passes through before the visitors begin the review.
50 In 2005-06 monitoring and renewal reviews, the annual report of a provider to the GOsC may form the SED, or a first SED to which the provider may add more recent and/or more self-evaluative elements. It is open to a provider to submit as its SED an entirely new document without inclusion of the annual report. QAA understands that the provider's context and the review timing will differ between providers, occasioning different views among providers on the best method of supplying self-evaluative evidence to QAA.
Initial scrutiny of the SED
51 Providers are asked to send three printed copies and one electronic copy of the SED (ie annual report and/or any other documents that the provider wishes to supply at this stage) to QAA's Gloucester office with the appropriate form completed and attached to the front of the document. The first part of this form allows the provider to state clearly the documentation being submitted as an SED. When QAA has accepted the document as an adequate basis for review the Review Coordinator or QAA Officer will ask the provider to send to each review visitor an electronic and paper copy of the SED.
52 The SED is scrutinised as follows:
- QAA will check the word length and, if it exceeds a reasonable length (200 words for the statement of overall aims: 6,000 words for evaluation of the subject programme) will return it to the provider for editing. In this case, QAA will inform the Review Coordinator. Sometimes, however, there will be instances (particularly in initial recognition review) where some flexibility regarding word length is required.
- QAA will check that the definitive course document and/or programme specifications are attached as annexes. If the course document and/or programme specifications are missing, the QAA will inform the Review Coordinator and ask the provider to supply the missing material. QAA will also check that statistical data are included. These checks are for completeness of the SED and not an evaluation of its reliability as the basis for review.
Stage 1: Submission of the self-evaluation
53 The form below is emailed to the provider during the period of arranging the review. The provider should complete the sections indicated, attach the form to the front of the SED and send the documents to QAA for checking.
54 Providers complete the form as a checklist, before submitting their SED to confirm that all relevant material is included. QAA uses it to cross check that the submitted SED contains the range of material needed to support the review. The SED is then forwarded to the Review Coordinator for analysis (see Stage 2).
55 A model form is provided below. Sections in italics are examples and should be overwritten. The form will expand as text is entered. Rows may be added to the tables to accommodate the information to be provided.
56 The document(s) submitted include the following:
| For completion by the provider | For Logistics & Deployment (L&D) use only | ||
| Section | Confirm | Tick | Comment |
| Aims (Maximum 200 words) | 187 words | ||
| Evaluation of the provision (maximum 6,000 words) | 5,095 words | ||
| Aims and outcomes | Yes | ||
| Curricula | Yes | ||
| Assessment | Yes | ||
| Achievement | Yes | ||
| Teaching and learning | Yes | ||
| Student progression | Yes | ||
| Learning resources | Yes | ||
| Governance and management: general governance | Yes | ||
| Governance and management: finance and risk management (eg insurance, contingency) | Yes | ||
| Governance and management: maintenance and enhancement of standards and quality | Yes | ||
| Information regarding modular structures/ collaborative arrangements | No | ||
| Page count = | |||
Stage 2: Analysis of the self-evaluation
Scrutiny by the Review Coordinator
57 When Stage 1 is successfully completed, the SED and the completed form are passed to the Review Coordinator to ensure that the SED contains sufficient information to plan and conduct the review. The Review Coordinator will check that it broadly follows the guidelines in this handbook and that it is sufficiently self-evaluative. The Review Coordinator will notify QAA of the result of the checks by completing the form set out below. Providers are advised to address the form themselves to test whether their SED is fit for purpose. The Review Coordinator will return this form electronically to QAA's L&D officer and copy it to the QAA officer responsible for the review (see Stage 3 below) within seven days of receipt of the SED.
58 If it becomes apparent on completion of the form that the SED does not contain the information needed to plan and conduct the review, or if there are significant uncertainties about its quality, the Review Coordinator and the QAA officer will discuss this and decide upon an appropriate course of action. The review should normally proceed using the SED as originally submitted. Questions of content or quality should normally contribute to the dialogue between the provider and the visitors and should influence the visitors' lines of enquiry. Occasionally, it may be necessary for QAA to ask a provider to add to, or rewrite, the SED. Responsibility for this final decision rests with the QAA officer (see Stage 3 below). The preliminary meeting of visitors and the institution should not normally take place until it has been agreed that the SED provides a satisfactory basis for the conduct of the review.
59 Providers will appreciate the demanding timescales above and will need to consider the security of their SED.
Review Coordinator's analysis of the self-evaluationReview Coordinator: Date self-evaluation and submission and analysis form forwarded to Review Coordinator: (date to be entered by the L&D officer) |
|
1 Are the programmes to be reviewed clearly identified within the self-evaluation? Yes/No Notes |
|
2 Are the overall aims clear and do they provide a reasonable basis for the planning and conduct of the review? Yes/No Notes |
|
3 Does the self-evaluation address clinical and academic standards, and in particular:
Yes/No Notes
Yes/No Notes
Yes/No Notes
Yes/No Notes |
|
4 Does the self-evaluation address the quality of learning opportunities and, in particular:
Yes/No Notes
Yes/No Notes
Yes/No Notes |
|
5 Does the self-evaluation address the governance and management of the provision, including financial and risk management issues and the maintenance and enhancement of standards and quality in the subject? Yes/No Notes Does the self-evaluation address both strengths and areas of importance of the programme(s) under review? Yes/No Notes Is there evidence for the strengths and an action plan for areas of importance? Yes/No Notes |
|
6 Are any programme specifications missing? Yes/No Notes |
|
7 Do all programme specifications contain learning outcomes? Yes/No Notes |
|
General observations 8 Please comment here on the quality of the self-evaluation, for example, is it evaluative? Is it helpfully structured? Is any essential information missing? |
|
9 Do you accept that the review can proceed on the basis of this self-evaluation? Yes/No If no, please summarise reasons |
|
10 Proposed date for the preliminary meeting with the institution: (insert date) |
|
Date: Date form returned to L&D: Please email this completed form to the L&D officer and QAA officer responsible for GOsC review within seven days of receiving the self-evaluation. |
|
Stage 3: Confirmation that the review may proceed 60 When the Review Coordinator agrees that the SED forms the basis for the review to proceed, the QAA officer with responsibility for the review considers the documentation and decides if the review can proceed. The form below is used for this purpose. For completion by the QAA officer responsible for the review Date received: Should this self-assessment be returned to the institution for amendment? Yes/No Notes Does the initial scrutiny of the self-evaluation suggest that visit support should be provided? Yes/No Notes Signed: Date: Once completed, this form is retained by the QAA officer with responsibility for the review and sent to the L&D Team and to the electronic folder for the review. |
Annex C
Visitors
Introduction
1 QAA operates an equal opportunities policy in suggesting visitors to the GOsC. Applications for appointment as a visitor are scrutinised by both the GOsC and QAA. The GOsC remains the appointing body in the recruitment, selection and deployment of visitors. QAA policy on the appointment of review visitors and the application procedures are available on the QAA website. All applicants will be considered on the basis of their ability to meet the specifications outlined below. Matters of commercial sensitivity in the osteopathy sector suggest that principals and vice-principals of providing institutions should not normally be review visitors in this method during 2005-06. For similar reasons, the heads of osteopathic programmes, departments, schools and faculties in publicly-funded institutions are discouraged from applying to be visitors in the 2005-06 academic year. In consultation with QAA, the GOsC will review this policy during the calendar year 2006. More generally, before offering nominations for future visitors, providers are asked to reflect on whether the potential visitor will be commercially acceptable to other providers undergoing review. QAA evaluates the performance of all visitors, using feedback from review visits. The Osteopaths Act 1993 states that:
- no person appointed as a visitor may act as a visitor in relation to any place at which s/he regularly gives instruction in any subject or any institution with which s/he has a significant connection
- a person shall not be prevented from being appointed as a visitor merely because s/he is a member of the General Council or any of its committees.
2 There are three types of visitor used in the review:
- specialist osteopath visitors, with current teaching experience in the discipline concerned, and/or experience of relevant professional or occupational practice
- review coordinators, who lead academic reviews and have extensive experience of quality assurance and programme approval of HE programmes, usually gained by working with such procedures in more than one discipline. In monitoring reviews, a QAA officer may take the place of a Review Coordinator
- lay visitors are non-osteopaths who have proven an interest in academic standards, quality and management in HE.
Qualities required in all visitors
3 Effective visitors will possess the following qualities:
- demonstrable commitment to the principles of quality assurance in educational provision
- an enquiring and sceptical disposition
- powers of analysis and sound judgement
- personal authority and presence coupled with the ability to act as an effective team member
- good time-management skills
- experience of chairing meetings
- the ability to make appropriate judgements in the context of the college or university being reviewed, and recognising that it is different from their own place of work
- experience of organisation and management, particularly in relation to teaching and learning matters
- high standard of oral and written communication, preferably with experience of writing formal reports for publication to deadlines.
4 In addition, visitors are expected to have a clear knowledge and understanding of QAA's GOsC review process and the Academic Infrastructure. QAA expects that visitors will be familiar with the Code of practice and aware of the precepts in the sections relevant to the provision under review. All sections of the Code of practice are operational and will be taken into account by visitors from September 2005.
Recruitment, training and role of specialist and lay visitors
5 Specialist and lay visitors are recruited from individuals nominated by providers or other organisations, and from individuals who reply to advertisements. Specialist and lay visitors are recruited and trained to ensure that they are capable of carrying out their duties effectively. In particular, specialist and lay visitors who undertake reviews are expected to:
- possess the knowledge and skills set out in detail below
- have completed successfully QAA's training programme
- ensure that they are available for the whole period of a review for which they have been selected and have a commitment to complete all processes of a review once they have embarked on it.
6 Initial training of visitors is carried out by QAA by means of a two-day residential course.
7 QAA maintains a database of visitors and other reviewers and auditors. The primary purpose of the database is to show, for each visitor, the main areas of HE and/or teaching and learning that s/he is qualified to review.
8 The key purpose of acting as a specialist or lay visitor is to contribute to the maintenance and enhancement of standards in HE by reporting to the GOsC through QAA on the governance of the provider and the standards and quality of programme(s) scrutinised. Specialist and lay visitors are expected to agree individual timetables of activity with the Review Coordinator or QAA officer, with a view to making the most effective contribution to the review. The responsibilities of visitors include:
- reading and analysing the SED submitted by the provider and any other documents sent in advance of a review
- adhering to the review schedule agreed by the provider and the Review Coordinator/QAA officer
- participating in visits to the provider in order to gather, share, test and verify evidence
- making judgements on the governance and management of the provider, the clinical and academic standards achieved and the quality of the learning opportunities provided
- contributing to and commenting on the completion of the report of the review to agreed schedules and deadlines.
9 Specialist and lay visitors analyse and evaluate the SED, with particular emphasis on curricular content and its suitability for achieving the programme outcomes. They review and evaluate the assessment process designed for the programme and determine whether they are suitable to assess programme outcomes as stated in the programme specifications.
10 Specialist and lay visitors consider and evaluate overall student achievement, including progression to employment, the contribution made to student achievement by the quality of teaching, opportunities for learning, academic support intended to ensure effective progression of students and learning resources and their deployment (including staffing).
11 Specialist and lay visitors judge the overall standards for subjects.
12 Specialist and lay visitors judge the overall governance and management of the provision, including financial management and the procedures associated with the maintenance and enhancement of clinical and academic standards and quality.
13 Finally, visitors contribute to the compilation of a report made to the GOsC. Each specialist and lay visitor will be expected to prepare material for the various sections of the report and relevant draft sections of the report, with specific references to the sources of evidence considered.
Knowledge and skills required of specialist and lay visitors
14 To carry out the role outlined above, for each review, specialist and lay visitors will need to demonstrate:
- experience, knowledge and understanding of educational provision
- at least five years' experience of providing HE-level teaching and learning. In the case of professionally-based visitors, familiarity with HE teaching and learning
- familiarity with academic support strategies and the functions of academic tutorials
- experience of examining and/or verification (and preferably external examining or verification)
- knowledge of the quality assurance processes employed by providers of HE
- knowledge and understanding within the subject area
- knowledge of, and familiarity with, Standard 2000 and any subject benchmark information produced for osteopathy
- professional and currently registered expertise in osteopathy (specialist osteopath visitors)
- familiarity with health subject matters and/or financial analysis and/or quality assurance and review in HE (lay visitors)
- familiarity with comparable programmes and standards of awards in other providers (specialist osteopath visitors)
- understanding of external examiners' reports and internal documents such as internal verification, second-marking and second reading
- understanding of programme entry requirements and an ability to interpret progression statistics for each stage of the programmes, including withdrawal, transfer and failure rates
- understanding of programme learning outcomes
- familiarity with destinations data and employment statistics
- ability to conduct meetings and interviews with staff
- ability to conduct meetings with a range of current and former students
- ability to write succinctly and coherently
- ability to meet exacting timescales and deadlines
- ability to work effectively as a member of a team
- ability to communicate electronically, including emails, attachments and use of webmail.
Recruitment, training and role of review coordinators
15 Review coordinators are also recruited from individuals nominated by colleges, universities or other organisations, and from individuals who reply to advertisements. They may be seconded or independent consultants. It is expected that they will possess extensive experience of HE and of the assurance of standards and quality. They will be expected to perform a number of duties, of which managing reviews and writing reports are the major responsibilities. Opportunities to contribute to other activities such as editing reports, training specialist visitors and drafting overview reports may also be available.
16 Because of the relative complexity of the Review Coordinator role, the individuals recruited will undergo a longer induction and training process than that provided for specialists. Induction into the review method will normally include attendance at, and participation in, at least one visitor or reviewer training course, as well as attendance at workshops and conferences arranged by QAA.
17 Reviews take place throughout the academic year and are variable in length. Review coordinators will need to organise their time and reach agreement with the providers and their teams of visitors, about the pattern of review activities in such a way as to ensure effective use of the time available.
18 All reviews consist of four main activities:
- preparation for review
- visits to the subject provider
- analysis of documentary evidence
- report writing.
19 Each Review Coordinator is responsible for maintaining an overview of the range and balance of these activities, and for helping visitors to divide their time effectively. The achievement of an appropriate balance between the various activities requires planning in advance of, and coordination throughout, the review. Above all, it is essential that it enables the visitors to develop a robust evidence base on which to make judgements.
20 The following criteria for selection will apply.
Knowledge and skills required of review coordinators
21 In order to carry out their role, review coordinators will need to demonstrate:
- recent knowledge and understanding of current HE issues
- awareness of current teaching methods and curricula
- knowledge and understanding of the assurance of standards and quality
- experience of liaison with senior management and staff at other levels
- ability to manage small teams (with experience in HE, FE or industry)
- ability to work within tight timescale and to strict deadlines
- ability to lead a team of experts
- ability to communicate effectively in face-to-face interaction
- ability to produce clear and succinct reports to time
- experience of word processing
- ability to communicate electronically, including emails, attachments and use of webmail.
22 The essential qualities above might be reinforced by experience of a wide range of teaching at HE level and by experience of programme accreditation by professional, regulatory or statutory bodies, programme approval or validation events, quality audits, quality assessment/academic review or educational inspection.
Annex D
Timeline of review schedule
1 The timing of reviews will enable reports and recommendations to be passed to the GOsC and then on to the Privy Council so that decisions on approval can be made and transmitted in a timely fashion. In particular, the planning of reviews will normally be made in consultation with the GOsC in respect of its timetable for Education Committee meetings.
2 The review period will start with visitors reading and commenting on the SED supplied by the provider. The period of review will normally be about six weeks for recognition and renewal reviews and four weeks for monitoring review. Within the six weeks review period, recognition and renewal visitors will normally spend two days on site at the teaching institution, with a further half-day assigned to a final meeting of the visitors. Within the four weeks' review period, monitoring visitors may spend at least one day on site at the teaching institution. Where a programme operates under approval with conditions in teaching and learning, student achievement or assessment, or where the GOsC wishes particular scrutiny of a programme feature monitoring visitors will normally spend time at the programme provider's site(s). In the 2005-06 academic year, it is expected that all monitoring reviews will include at least one day of site visit. Visit days may not necessarily be consecutive days.
3 It is anticipated that in 2005-06 reviews, the visit will normally occur at the end of the review period, immediately before the final meeting of the visitors and the oral feedback to the provider. However, this pattern is not a requirement of the review method. The visitors may visit the provider at any time during the period of review, but always by mutual agreement with the subject provider and within the overall number of visitor days allocated by QAA. This may involve the whole team visiting together, as for the initial meeting, or it may involve two or more visitors visiting for specific observations, meetings or review of documents. Individual visitors will not normally visit providers alone, or hold meetings with staff and students on their own, except in the case of teaching observation. The table below assumes that the visit of all the visitors will take place at the end of the review period.
4 The review period may be extended where it is felt that enquiry has not been completed. In these circumstances the final oral feedback to the provider will necessarily be postponed. However, through the Review Coordinator or QAA officer, visitors will always debrief the provider on completing a visit to the provider.
| Start of review minus 12 to minus three months | QAA contacts the provider to agree three months dates for a review period and a specific date for the visit part of the review |
| Start minus eight weeks | Submission by the provider of the SED (a) to QAA, (b) later to visitors. |
| Start plus or minus approx two weeks | At least one preliminary meeting between the Review Coordinator/QAA officer and the provider. |
| Start plus two-six weeks | Submission by the provider of any extra documents arising from the preparation for the visit, including the preparation of the sample of student work. |
| Visit (Start plus four-six weeks) | Visit of one or two days may include meetings of visitors with the course team, and activities such as meetings with current and former students, and employers, and scrutiny of documents and student work, and visits to site(s) of clinical learning. |
| Final meeting of visitors (Start plus four-six weeks, often immediately after one-two days of visit) | Final meeting of the visitors |
| Oral feedback to the provider (Start plus four-six weeks, often immediately after one-two days of visit and the final meeting) | Notification of main findings to the provider, without indication of formal recommendation to be made to the GOsC. |
This timeline is indicative. Individual events may be varied to accommodate specific circumstances such as Christmas, vacations or examinations.
See Part one of this Handbook for the report production schedule.
Annex E
Review teams
Team composition
1 The review team in recognition and renewal review will normally consist of a Review Coordinator, two osteopath visitors and at least one lay visitor. The review team in a monitoring review will normally consist of one QAA Assistant Director or Development Officer and two visitors, these being either two osteopath visitors or, more exceptionally, one osteopath visitor and one lay visitor. In advance, QAA will communicate to the GOsC for approval the suggested review team. The GOsC has appointed QAA to the task of submitting names of provisional team members to the providing institution. A specialist adviser may be appointed to conduct the financial scrutiny. Providers to be reviewed will have the opportunity to comment on suggested review team composition about eight weeks before the review starts.
2 QAA will take account of conflicts of interest declared by specialist and lay visitors.
3 Providers are expected to comment on the composition of teams of visitors in writing to QAA. Occasionally, a provider will be unable to accept the proposed team. It is essential that any concerns, for example, a conflict of interest or the team's match to the programmes to be reviewed, are made in writing and discussed with QAA officers as soon as possible after notification, and no later than two weeks from notification. Delay in the above may result in undue delay for the review.
Team function
4 Specialist (osteopath) and lay visitors assume a collective responsibility for gathering and verifying evidence in relation to clinical and academic standards, the quality of learning opportunities and governance and management issues. However, visitors are expected to respect the particular expertise of their visitor colleagues. The Review Coordinator will deploy specialist and lay visitors in the most appropriate way to complete the review activities professionally and in the time available. Sometimes the visitors will work together as a team, for example, at the initial meeting. At other time, visitors may work in pairs or, for direct observation of teaching and learning, singly. The Review Coordinator will ask each visitor to write a brief commentary based on the SED and the evidence gathered during the review. These commentaries will make full reference to the aims of the subject provider and identify matters for which additional evidence is required. They will inform the visitors' priorities and the balance of activities undertaken. The commentaries will be revised as the review continues, so forming an early draft report as the review continues. The revised commentaries that result from the review of documents, visitor discussions and the visit will inform the collective findings and recommendation reached at the end of the review.
5 All visitors will be expected to identify, share, consider and evaluate evidence related to the programmes under review. The visitors will keep notes of all meetings with staff and students, their observations and comments on student work and its assessment for two years from the end of the review period. Circulation of these confidential notes within the review team will help develop a collective evidence base on which the judgements can be made. The visitors communicate with each other and share their analyses and notes through the electronic folder system developed by QAA to support its reviewers. The Institutional Contact is able to post material to the electronic review folder, by agreement with the Review Coordinator, but is not able to access documents posted there by the visitors. The visitors will be expected to evaluate how the accumulating evidence compares with that provided in the SED, and to test the strength of the evidence produced to support the findings and recommendations. It is essential that discussions of the emerging view on the provision involve the whole review team.
6 Draft summaries written by the visitors during the course of a review will focus on the evaluation of evidence related to their particular responsibilities, as agreed by the Review Coordinator/QAA officer. Summaries are analytical rather than descriptive and refer to sources of information as well as to direct observations. Any written evaluation will summarise the relevant strengths and areas for improvement of the provision and, overall, underpin the judgements made. A final meeting of the visitors will be used to review any additional evidence, to agree particular strengths and areas for improvement in relation to standards, quality, governance and management to determine precisely what is to be reported and to finalise findings and recommendation to GOsC.
7 Occasionally, the visitors may consider that they need additional time to complete a review. The Review Coordinator and Institutional Contact discuss this proposal. If they agree that extra time is needed, the Review Coordinator applies for this to the QAA officer responsible for the review. QAA grants extra time for a review only exceptionally.
Reports
8 The Review Coordinator (usually QAA officer in monitoring review) prepares the first draft of the report from the visitors' summaries immediately after completion of the last day of the visit to the provider. Specialist and lay visitors check and comment on this draft. Following their final meeting, the visitors check the draft report before the Review Coordinator sends it to the provider. As the reports provide the main feedback about reviews to providers, it is particularly important that visitors check their accuracy carefully.
9 The reports supplied by visitors to the GOsC via QAA are the main documented outcomes of the process. The timescale for the completion and checking of reports is demanding. Reports should be characterised by succinct, accurate writing and a clear, consistent style with findings and recommendations clearly related to evidence.
Practical arrangement for visitors
10 Practical arrangements made by QAA for visitors include:
- hotel accommodation, where this is required
- travel and subsistence reimbursement
- administrative support.
11 The visitors will need to have access to word processing facilities. Visitors compile and transfer written summaries electronically. The visitors must conform to the QAA's procedures described in the information technology guidelines for the use of electronic information, as these are designed to protect against damage and computer viruses.
Annex F
Preliminary meeting agenda
1 There will be at least one preliminary meeting between the provider and Review Coordinator. It is important that providers are prepared to discuss each item on this agenda by, for example, ensuring that they have up to date student data and timetables available at the meeting.
2 The preliminary meeting gives the provider's staff a valuable opportunity to clarify their understanding of the review method.
3 The agenda below is indicative and QAA considers it the minimum necessary to enable the provider and the Review Coordinator/QAA officer to establish the requirements of the review. The Review Coordinator/QAA officer and the provider may feel it appropriate to include additional items.
Agenda
- Introductions
- Purpose of preliminary meeting
- Clarification of review method
- Clarification of the scope and nature of the provision
- Questions arising from initial analysis of the SED
- Confirmation that the statistical data are correct and accurate
- The visitors' requests for information to date
- Establishing the programme of review activities during the initial meeting, including the agenda
- Clarification of the availability of documents, including student work
- Clarification of the availability of academic teaching observation
- Clarification of the availability of clinical teaching observation
- Questions from the provider's staff
- Housekeeping arrangements
- Next steps
ISBN 1 84482 220 6
© 2005 General Osteopathic Council
© Quality Assurance Agency for Higher Education 2005
Top