Case Study 1. Source: Oakley, B. (2004). Assessment Diversity: Learning Lessons from a Pilot Viva-Style Assessment in Level 1. LTSN Hospitality, Leisure, Sport & Tourism LINK Newsletter – Issue 10 Student Diversity Summer 2004.
Assessing students by getting them to talk to tutors about what they know, rather than having them write exam scripts is widely used as postgraduate level, and is known as the ‘viva’. In terms of student diversity, this type of assessment could be well suited to those with special learning needs. This article report on the challenges of introducing such viva-style assessment for large numbers of level 1 students.
Traditionally, the Department of Sport & Exercise Science at the University of Portsmouth has made widespread use of multiple-choice examinations in a number of Stage 1 units within its Sports Science and Sports Development degree programmes. However, it seemed that multiple-choice was not allowing real probing of the subject matter and was not stretching the more able students. Therefore, staff members expressed concern over whether some learning outcomes were being met at Level 1.
Viva-style assessments have been widely adopted in the medical and legal professions, and given the reliance on interpersonal skills in the Leisure, Sport and Tourism sectors, it seemed appropriate to pilot and evaluate their use.
Practicalities of the viva-style assessment
The issues of catering for more than 100 students, using a new assessment method (for both staff and students), and ensuring consistency led to the following initial actions:
- Unit leaders set aside 2-5 hours in their teaching programmes for a mock assessment in weeks 8-10 of the 12 week semester.
- The Head of Department encouraged all staff to be involved on assessment days with the provisions that non-subject (or unit) specialists would still have an important support role.
- A staff meeting was used for training on the general protocol and methods to be used. This was developed, in writing, for staff with a subsequent version released to students.
- Unit leaders prepared a ‘question bank’ of 10-20 questions for each learning outcome to be assessed. Former multiple choice exam questions were adapted and other questions added that would allow a student response to be graded on a scale of 0-5 marks. A threshold of 2 out of 5 marks was recorded as the ‘pass’ marking criteria for each question.
Students were allocated a 30-minute period for the assessment, which they attended in pre-allocated groups of three. Usually, three learning outcomes were assessed using two random questions from each of the question banks – making a total of six questions for each student. Written prompts or clarifications were permissible with each question.
A pro-forma marking sheet was used to grade (0-5 marks) the student’s response to each question using the written marking criteria and thus, an overall mark was obtained for each student. The overall pass mark of 12 (40%) was used and a mechanism was established for those students who were borderline to be allocated a final pass/fail question.
Students with Dyslexia
Advice was taken from the University’s disability advice centre about whether those with special needs (mostly dyslexia) should be assessed separately and/or allowed extra time. The conclusion reached was that they should be integrated with the main cohort, with the only proviso being that they should be allowed sufficient time to respond.
An adjustment to the method of asking questions to students was made following the mock. The verbal reading of the question was changed to a written form, which was handed to the student. In the mock, the verbal questions often had to be repeated and students struggled to remember the precise question while assembling their response, thus adding to the pressure. It was thought that this multiple processing of information was particularly intimidating for dyslexic students and the change was received positively. However, a further request has subsequently been made from a student with dyslexia:
“Allow rough notes to be written as an aide-memoir when thinking about our answer.”
Clearly for some students at Level 1 the processing, ordering and recall of information in the viva format is quite a challenge, and evidently, this may be a particular issue for students with dyslexia. This adjustment may be tried next year.
By far the greatest amount of time and effort applied in the whole exercise was the development of the question bank for each of the four units involved with the pilot, as the questions needed to be phrased appropriately to allow an open response that could be graded.
Below are a few of the written feedback responses that have been received to date:
“It was a good way of testing knowledge as you had to be really prepared and have answers ready to say.”
“I felt intimidated when faced with questions in front of other people and this hindered my performance; I am more used to written exams!”
“It made me study more, however, I found the questions were at various levels and were not all of the same difficulty.”
“It was a good way of getting me to think on the spot and it will help me with my interviewing skills.”
Early evidence suggests that there is wider support for this type of assessment in units such as Sports Development, where social and organisational issues can be discussed more readily than, for example, the role of different types of blood cells in Physiology (Sports Science). It is also clear that the more socially confident, and in particular, mature students thrive in this situation; however even those that are not so competent at expressing themselves orally concede that this is a vital skill.
Implementing this change in assessment is clearly intimidating to the less confident students and therefore, some mock element is needed in order to show participants what to expect; this requires effort from staff members who need to try and create a non-threatening atmosphere. The point raised above about equality in terms of question difficulty can be tempered in the future with careful preparation and wording of questions, and an adjustment to the protocol used.
Evidently then, this style of assessment offers one way of countering the written-bias of traditional undergraduate assessment which is acknowledged as being disadvantageous to some students. Equally though, feedback from students (including those with dyslexia) has shown that the viva is not a problem-free alternative. However, the useful and interesting thing about this type of assessment is that it offers exactly that – an alternative.
Case Study 2. A Dyslexic Student with a Sports Health and Exercise Science Degree. The information used in this case study is taken from the Staffordshire University Strand 2 project SIDE-STeP (Staffordshire Inclusive Disability Education – Sport TEaching Practice) http://crwnpro2.staffs.ac.uk/sidestep/
K graduated in 2000 with a degree in sports health and exercise science. He then went onto to do a masters in human nutrition and is currently doing a postgraduate in human nutrition and dietetics.
K has been receiving support for his dyslexia since school and as a result has experience of working in a number of different institutions and is well aware of what his needs are and what adjustments will help him overcoming challenges with his learning. He finds he has to communicate this regularly to his tutors because they are not always sure what adjustments he will need. He admits it is sometimes difficult to get tutors to understand his particular needs, and doesn’t like to be made to feel like he is being a nuisance by continually asking for adjustments if his needs are not being met.
His most necessary requirement is having copies of lecture notes in advance of a lecture, this is because he cannot read, listen and write at the same time. In the past he has also had extra time for assignments, access to an academic support tutor and a reader and use of a computer during examinations. He has also used student support when he was struggling with a final dissertation. He believes that a lot of the help he has received has been due to his ‘strong personality’ and the fact that he has not been afraid to request help when he has needed it, despite sometimes feeling that he was ‘pestering’ tutors. He has also used a Dictaphone to record lectures but this does depend on whose lecture he is attending and whether he feels that the tutor is ‘comfortable’ with the Dictaphone. Some tutors can draw attention to it which causes embarrassment, he also feels a shy student might find this difficult.
The institution where K is currently studying has a system in place whereby lecture notes are posted onto an intranet for access by all students in advance of lectures. There are also reading lists posted onto the intranet to enable students to prepare for lectures by doing some background reading. He emphasises the point that students with dyslexia very often have to do a lot more background work than other students and tutors might not always fully appreciate this.
K discusses the transition between school and university and the fact at school there was no choice about whether to disclose or not and support was automatically given, whereas at university the student suddenly has a choice about whether or not to disclose their dyslexia and can make their own choices about the implications this will have on the level of their support received.
K highlights the situation of another student he knows who has chosen not to disclose her dyslexia to her current institution. This is due to her experiences at a previous institution where she felt that she wasn’t taken as seriously as other students. She felt that during discussion groups her comments were not taken on board by the tutors in the same way that comments made by other students were. He makes the point that if you are withdrawn and worried about what people will think of your dyslexia, then you are likely to go through university without any assistance at all.
K believes that in order for students to be successfully supported through university, they need to be fully informed about what support is available to them before they make the decision about whether to disclose their dyslexia or to go for a screening/needs assessment. This is the only way to give students the confidence to manage their own learning needs successfully.
K felt that sport science is a good subject for a student with dyslexia to study because it is very practical. However, there is still the issue of reading load and obtaining lecture notes in advance as there is with any subject. He finishes by reiterating the point that the only way to successfully assimilate the information is by having lecture notes and reading lists in advance, and recording lectures where necessary, in order to refer back to the information at a later date.
Case Study 3 – Hartley, H. Dr. Understanding Presentation of an Essay – Formative tasks: An Introduction to Philosophy and Sport. School of Sport & Leisure Studies. Leeds Metropolitan University. http://www.hlst.heacademy.ac.uk/resources/cases/case8.html
Lecture and Seminar tasks on Essay Assessment Criteria, with a focus on presentation
Group: 120 level One students in lecture; 5 seminar groups on written task and feedback.
Aim: The lecture and seminar in week 6 of level one is designed to assist understanding of presentation of academic writing in the area of paraphrasing, direct quoting, paragraph structure and referencing. It aims to get them writing and doing philosophy using this formative feedback as early as week 6 and to enhance self confidence in checking their own draft work.
Context / Background: An increasingly diverse group of students, very anxious about academic writing in a new subject area. The monitoring of past work was showing some weaknesses in presentation of essay and concise writing skills.
Example: At the end of session one – after covering an introduction to philosophical methods, students are asked to write two paragraphs (150-170 words per paragraph) on the nature of conceptual, empirical and evaluative questions, with examples from the theme of drugs in sport, for week 6. In week six, after completion of the main subject themes, the lecture is on all aspects of the assessment criteria, with a particular focus on presentation and with cross-reference to the material and tasks in the Skills for Learning System. In the seminar, of 1 hour, they are asked to bring two copies of the above two paragraphs and working in pairs, with their study partner in the group:
- Read and mark on the copy any presentation weaknesses and comment on good practice.
- Provide feedback on the criteria of use of relevant literature and the skills of direct quoting and paraphrasing.The member of staff gives feedback on philosophical aspects of the writing.
Results / Feedback: All students complete the task. They were very good at paragraph structure, paraphrasing and selective quoting. In marking they gave thorough feedback and showed good understanding of the work covered in the lectures and in Skills for Learning. Still a little dependent on tutor regarding philosophical work, and problems getting round whole group, but students gained in confidence with room for development well before essay hand-in date.
Other comments: Material used in this pilot lecture on assessment criteria/essay wiring guide, was later developed for a broader group of students accessing the Skills for Learning system. Essays have improved since this work was developed.