Introduction
1 This document provides an operational description of the special review that will apply for postgraduate research degree programmes (RDPs) in Wales. The review, announced in the Higher Education Funding Council for Wales (HEFCW) publication Postgraduate research degree programmes: minimum standards and funding (HEFCW WO4/64HE) [1] as part of the broader strategy for securing the quality of RDPs, will take place in 2005-06.
2 The main purpose of the review is to ensure that all higher education institutions (HEIs) receiving funding from HEFCW for RDPs have policies and procedures in place that are robust and effective in securing and enhancing the quality and standards of RDP provision. In September 2004, QAA published a revised Code of practice for the assurance of academic quality and standards in higher education (Code of practice), Section 1: Postgraduate research programmes [2]. The review of RDPs will draw upon the Code of practice, Section 1: Postgraduate research programmes, to see the extent to which institutional policies and procedures are in alignment with the Code of practice, Section 1: Postgraduate research programmes and its precepts.
3 The review will not address standards of research more generally.
Scope
4 The review method described in this paper will apply to all HEIs in Wales which are in receipt of HEFCW funding for RDPs. The special review includes all RDP students registered for an award of the institution. This includes HEIs which have registered RDP students and are without research degree awarding powers.
5 The review will cover:
- institutional arrangements
- the research environment
- selection, admission and induction
- supervision
- progress and review arrangements
- development of research and other skills
- feedback mechanisms
- assessment
- student representation
- complaints
- appeals.
Defining research programmes
6 The review is intended to apply to the PhD (including the New Route PhD and PhDs awarded on the basis of published work), and all forms of taught or professional doctorate, and research master's degrees where the research component (including a requirement to produce original work) is larger than the taught component when measured by student effort.
Relationship to institutional review
7 The special review of RDPs is a one-off exercise. HEFCW, if necessary, will consult with HEIs on any changes which may be required to QAA’s existing arrangements for institutional review in the light of the outcomes of the special review of RDPs.
8 In order that the review can provide a comprehensive baseline of information, there will no exemptions from the special review for institutions which have recently undergone an institutional review.
Consultation and pilots
9 The draft operational description and questionnaire was circulated at the beginning of February, with a deadline for comments by 22 April 2005. The responses have been carefully considered and, where appropriate, revisions have been made to the operational description.
10 A pilot project involving three HEIs has also taken place. The outcomes of the pilot have fed into developing the review method and the guidance to institutions.
The review method
Overview
11 The review method adopts a desk-based approach, which involves the evaluation by reviewers of an institution’s response to a questionnaire about how the institution has aligned its polices and procedures with the revised Code of practice, Section 1: Postgraduate research programmes. Each submission will result in a report to the institution and HEFCW. There is also provision within the method for institutional visits where a follow-up visit is considered necessary. A visit by reviewers will only take place normally where there are particular causes for concern raised by the institutional response to the questionnaire.
Review teams
12 A peer review team will judge the extent to which a HEI’s processes and procedures for its RDPs are appropriate and satisfactory and align with the expectations of the Code of practice, Section 1: Postgraduate research programmes. The reviewers will work in teams, with the work of each team coordinated by a review coordinator. The review coordinators will help ensure consistency both within and across the review teams. A QAA Assistant Director will assume overall responsibility for the management of the review, providing guidance and support as required.
13 Prior to the review, institutions will be informed as to the membership of the review team allocated to consider their response to the questionnaire and draft the report.
14 Reviewers will be selected by QAA from nominations made by awarding and other appropriate institutions. Existing reviewers in Wales and Scotland and institutional auditors for England and Northern Ireland, who meet the criteria, may be invited to participate in the review of RDPs. The criteria for the selection of reviewers includes:
- extensive experience of the development and management of research students in higher education, eg acting as a supervisor of research students
- wide experience as a researcher in higher education, including the management of research programmes and developing the institutional research environment
- participation in the quality assurance of research programmes and of the research postgraduate learning environment
- personal and professional credibility with heads of institutions and senior managers in the higher education sector
- ability to assimilate a large amount of disparate information; to analyse and draw reliable conclusions about complex arrangements; and to undertake research and investigation into documentary and oral evidence in order to form judgements
- clear oral and written communication skills
- ability and willingness to use electronic communications.
15 Training for reviewers will be undertaken by QAA.
Questionnaire
16 The primary method for gathering information for the review will be a questionnaire. The questions are based on the areas covered by the precepts in the revised Code of practice, Section 1: Postgraduate research programmes, and focus on the institution’s approach to the management and support of RDP activity. When responding to the questionnaire institutions will be asked to provide supporting evidence to substantiate their claims and to reference it to their responses. The questionnaire is attached as Appendix 1.
Visits
17 The review method includes provision for review teams to visit an institution where there is a cause for concern. A cause for concern may be generated by the evidence that the institution has not:
- considered the intentions of the revised Code of practice, Section 1: Postgraduate research programmes
- reflected on its own practices and taken the steps, where necessary, to ensure that appropriate changes are being made to meet the intentions of the Code of practice.
18 Any visit to an institution is likely to take the form of discussions between the review team, staff engaged with the supervision and management of RDPs, and between the team and a representative group of students. Discussions will focus on specific issues derived from the outcomes of the survey and the supporting evidence provided by the institution. The exact nature, scope and timing of any visit will be determined by agreement between the institution and QAA following the reviewers’ evaluation of the survey.
19 Before any visit is made, an institution will be given the opportunity to provide additional evidence to address any concerns the reviewers may have.
Judgements
20 Using the outcomes of the survey and the supporting documentation, the review team will report on the institution’s ability to secure and enhance the quality and standards of its RDP provision through its alignment with the Code of practice. The judgement will be expressed in terms of the quality of RDP provision as being appropriate and satisfactory or, where this is not the case, further action is required.
21 The judgement made, whether it be appropriate and satisfactory or further action required, will be an holistic judgement.
Reports and sign-off
22 The review team will produce a report for each institution on the outcomes of the RDP review. The reports will not be published. The final reports on RDP will be submitted to the institution and to HEFCW.
23 The draft report is prepared and submitted to the institution as soon as possible following the reviewers’ final consideration of the outcomes of the survey. Copies of the draft reports should be with institutions by mid-May 2006. The institution is invited to provide QAA with corrections of errors of fact, within three weeks of receipt of the draft report. The final report is prepared in light of the institution’s response. Where a visit to an institution takes place, the draft report will be submitted as soon as possible following the visit.
24 The review is completed when it is formally signed off. Where the report makes a statement of appropriate and satisfactory provision, the review is signed off when the report is sent to HEFCW. In the event of a judgement of further action required, the provider will be asked to send an action plan to QAA within three months of the report being signed off, setting out how the matters raised in the report have been addressed. Twelve months after the end of the review, a QAA officer will visit the provider to confirm the effectiveness of the action plan. The means of removing a cause for concern will be through the institutional review process.
25 QAA will also produce an overview report for the exercise as a whole, which will be published.
Evaluation of the process
26 QAA will encourage institutions and reviewers to contribute to the evaluation of the RDP review process by inviting comment on the reviews in which they have participated.
Complaints and representations
27 Complaints about the conduct of the special review and representations about the judgements made by the review team are considered by QAA in accordance with the formal procedures published on its website (www.qaa.ac.uk).
Review timetable
| Academic year | Month | Task |
|---|---|---|
| 2004-05 | February-April | Consultation with higher education sector on the Operational description |
| May-June | Consideration of the outcomes of the consultation | |
| June-September | Pilots to test processProduction and distribution of revised operational description and questionnaire | |
| 2005-06 | September-November | Selection and training of reviewers |
| October-November | Briefing for institutions | |
| February | Responses from HEIs due Friday 10 February 2006 | |
| March-April | Analysis of responses by review teams | |
| May-June | Production of draft reports, circulation to HEIs, and responses to QAAAny review visits will take place now | |
| July | Final reports to HEIs and HEFCW |
Appendix 1 - questionnaire
Introduction
1 The questionnaire is intended to provide QAA with information on the institution’s quality assurance framework for RDPs. Particularly, how the framework aligns with the precepts of the revised Code of practice, Section 1: Postgraduate research programmes.
2 The review process recognises the validity of differences across disciplines in the characteristics of the research environment and in the management of quality and standards. Where differences exist, institutions may want to make this clear in their response, and show how the institution manages and monitors differences of approach in relation to quality and standards.
3 The institutional responses should include a description of key features and a commentary on the institutional approach to assuring quality and standards under each of the precepts along with evidence to substantiate the claims. It would be helpful if institutions could indicate, under each section, as precisely as possible where the evidence to support claims about alignment to the revised Code of practice, Section 1: Postgraduate research programmes can be found in a particular document. Occasionally you may wish to add an explanatory note to the evidence source.
4 Your response and the supporting evidence (see below, paragraph 9) should consist of material from existing documents. It should not be necessary for new material to be written for the review.
5 Where institutions have introduced new policies and procedures in response to the revised Code of practice, Section 1: Postgraduate research programmes, but where completed evidence is not yet available (eg annual monitoring), it would be helpful if an explanation of the change is given.
6 It is anticipated that the average response to the questionnaire will typically be around 6,000 words, although there is no penalty for a shorter or longer submission.
7 It is left to the discretion of an institution to decide the evidence it considers demonstrates its alignment with the Code of practice, Section 1: Postgraduate research programmes.
8 To avoid having to send too much supporting evidence, in cases where there may be multiple examples of evidence (eg discipline level student handbooks), the institution is invited to submit a representative sample to support their submission.
9 As a desk-based review method has been adopted, it is important that institutions provide evidence in a form which is easily accessible. It would, therefore, be very helpful if documents are provided electronically (eg on a CD-ROM and labelled clearly) and where appropriate access to relevant documentation on the institution’s intranet. Where evidence is not available electronically, it can be submitted in hard copy.
10 QAA needs three copies of your completed questionnaire and CD-ROM. Please send these to Julian Ellis at QAA no later than Friday 10 February 2006. An electronic version of the questionnaire is available - MS Word version
11 Please send your institutional submissions and CD-ROMs to:
Julian Ellis
Quality Assurance Agency for Higher Education
Southgate House
Southgate Street
Gloucester
GL1 1UB.
If you have any further queries please send them to the following address: RDP.specialreview@qaa.ac.uk.
[1] www.elwa.org.uk/elwaweb/doc_bin/he%20circulars/w0464he%20circ.pdf
back
[2] www.qaa.ac.uk/academicinfrastructure/codeofpractice
back
Top