Educational and Student Policy
Report of LTS Lunch 2: Friday 22 July, 2005
- The University and the Cambridge University Student Union (CUSU) have raised serious concerns with the methodology and implementation of HEFCE's National Student Survey);
- The response from students at Cambridge to the National Student Survey has been too low for any data relating to the University to be made public (see the national Teaching Quality Information website).
- Several Faculties and Departments already have on-line student feedback systems: these have been developed locally to suit the needs of the individual institution;
- CARET is in the process of developing a University-wide on-line student feedback system, for use by any Faculty or Department free of charge: this should be available for the 2005-6 year;
- Electronic questionnaires with standard formats are easier to analyse and summarise, saving time on collection of data, chasing respondents and collating summaries;
- The tensions between access and security, anonymity and availability of results need consideration before any system is implemented.
Other points, which also relate more generally to feedback, were noted:
- questionnaire fatigue: response rates tend to diminish over time for the same student cohort;
- response rates can be buoyed if students are clear about what happens to their feedback and can visibly see the impact it has;
- questionnaires are responded to more if there are other perceived benefits to the students (e.g. if it encourages self-reflection on the purpose of the course and what they have learned; if it is entered into a draw);
- free-form text boxes often provide the most useful information.
National Student Survey
Dr Matthew Russell (Education Section; email@example.com) alerted those present to the National Student Survey (NSS), a nationwide survey of graduating students instigated by HEFCE. A copy of the questionnaire given to students (who had been contacted either by email or telephone) is available as a PDF file. Further information on the initiative could be see on the NSS website (see link above).
The University had been notified recently that the response rate for the University was just over 20% (the response rate required by HEFCE for publication is 50%); the data provided for students graduating in 2005 would not therefore appear on the national Teaching Quality Information website.
Dr Russell also indicated that the General Board and the University's student union (CUSU) had had serious reservations about the methodology and implementation of the Survey, and had communicated those to HEFCE. It was likely that the NSS would be repeated annually.
Example: the Computing Laboratory
Dr Neil Dodgson (Computing Laboratory; firstname.lastname@example.org) presented the mechanisms used by that Department in the gathering of student feedback. A summary of his presentation is available as a PDF file, outlining all of the mechanisms by which feedback is collected. The summary includes an example of the on-line lecture course questionnaire used, and its eventual output.
The on-line course questionnaire has been used since 1995 and has been successful to the extent that paper copies of the form were abandoned in 2003 (whereas until then, both has been managed in parallel). Dr Dodgson acknowledged that paper forms resulted in a higher response rate than on-line (20-30%), but that this was largely due to time being given within a lecture to complete them. The introduction of the on-line form had saved considerable time and effort in the collation of the data collected, and allowed the results to be displayed quickly and easily to everyone with access to the Department's server (cl.cam.ac.uk). Students provide their email ID, more in order to monitor who has responded: the answers given are anonymised on submission.
The Department also operates an instant feedback facility, which relays urgent messages to key members of staff (e.g. Director of Teaching, Departmental Administrator, Head of Department), and which may be anonymous on request. This facility enables the Department to respond rapidly to teaching-related problems that cannot wait until feedback forms at the end of a lecture course. Interestingly, the facility is more useful in reassuring students (they all like its existence), but it has in fact only ever been used twice.
Example: Clinical and Biomedical Computing Unit
Mr Kim Whittlestone (CBCU; email@example.com) presented the on-line feedback questionnaires used by students taking clinical attachments as part of the medical course. It is part of the Educational Resources Web (ERweb) site, which is a personalised web-based teaching and learning environment and allows teachers to make resources available to support the course: from course objectives and information, to formative and summative assessments.
In order to maintain high response rates for student feedback, CBCU operate a system whereby students may only review their results for an attachment after they have provided feedback. It has not yet had to deal with a case where the student has refused to comply with this request.
The future of on-line student feedback within the University
Dr John Norman (CARET; firstname.lastname@example.org) has been part of an initiative to review the use of on-line feedback systems used by Faculties and Departments, with a view to providing a University-wide service. Recognising that current systems tend to have been biased towards local needs, CARET are collaborating with an external company (Ostrakon; CamTOES and now have a University-wide licence for using the software already developed and used by a number of Departments in the School of Biological Sciences. It is intended to launch this system in October 2005.
The CARET/CamTOES system allows customisation of questionnaire forms to suit the individual Department (or course). In that sense, it will allow for the questionnaire designers to include Department-wide questions and course-specific questions. There is also the potential for University-wide questionnaires.
This facility will, in time, become part of a portal-based learning support tool to be launched by CARET in October 2006. The system will incorporate a calendar/course timetable, announcements and news, document repository, chatroom and email lists, and personal development planning tools.
CARET have undertaken some background research as WIKI files (RAVEN access required).