Ungrading my teaching

I have been extremely lucky to participate in a Carpentries instructor training as an early career researcher. By then, I had only taught a few workshops, and was very impressed by the content of the training, and by Greg Wilson, who delivered it with touching sincerity and dedication. A couple of years later, I managed to secure some funding from the R Consortium to organise an instructor course in Cambridge, where I worked at the time, and was delighted to be able to revisit the content. The Carpentries approach to lesson design and teaching has stuck with me since then. I have applied these principles and some of the material to my main university courses (WSBIM1207 and WSBIM1322 for the second and third bachelor courses respectively and WSBIM2122 for the master’s course), and am now, as part of the Bioconductor Teaching committee, developing new lessons.

Delivering good courses at workshops isn’t an easy task, but everybody can get there with the right preparation and support (such as proposed by the Carpentries). There is however one aspect that makes it easier than teaching a university curriculum: as an workshop instructor, you focus on the day (or days) you teach, giving your best, and then it’s over - no need to think about tomorrow. Participants will hopefully apply what they have learnt, some might get back to you with questions (extremely rare in my experience), but others might not and you’ll never hear from them. In a university setting, on the other hand, you have to think about assignments, test, exams, marks, … and possibly even about pre-requisites for follow-up classes. You have to be able to assess the students, i.e. assess whether they have acquired (enough of) the material, and whether they are sufficiently well prepared for next year’s class and, more generally, their future activities. In my opinion/experience, this makes for a more complex teaching experience, more careful planning and a whole different level of responsibilities.

Criticising my own pedagogy

I have adapted how I test/assess students during the 4 years, 3 course and 10 semester classes I have taught since I moved to my current position. Following the Carpentries philosophy, my courses are hands-on and are based on repeated practice, so that students get it and have a chance to apply what they have learnt. As opposed to other courses, it isn’t quite possible to leave it to the last week(s) before the exam to cram all the material in short term memory - students need to attend and practise exercises in class and (ideally also) at home. To promote students’ work, I organise weekly tests which, for a couple of years now, allowed those that earned good marks, to transfer their mean test scores to their final grade, and thus get a dispense for the exam. Unfortunately, this hasn’t proven as successful as I had hoped.

The reason for the failure of this test/dispense approach is well known to those that have studied to effect of marking and grading - the focus of the student is switched from learning new skills to obtaining a good mark. And, as clearly stated by Marcus Schultz-Bergin in his chapter Grade Anarchy in the Philosophy Classroom (p. 175) (from ¹): Grades do not track learning (or anything else of importance) and, to add insult to injury, grading reduces student learning.

This indeed matches my observation. My students have access to any material they see fit for the tests and exams. Some focus their preparation on gathering as much material as possible (such as answers from previous tests and exercises) and then apply the minimax strategy to maximise the marks while minimising their own work by matching the answers they have gathered to the questions of the day. The focus of the tests has become the tests’ marks, and most criticism/feedback at the end of a lesson (I use the Carpentries famous colour sticky notes to increase interactions and collect feedback after each class) was the lack of time to finish/get good marks, with little questioning about the understanding/mastering of the material (such as being able to answer the questions in a limited but doable amount of time).

It is important to clarify here that I don’t blame the students. Students, as a collective entity, will adapt to rules and constrains that are set to them. Maximising the success in exams is a natural adaptation to the current system, that has focused on grades as a mean of standardisation. Even if the goal of this standardisation is (or was) equality and fairness (in the face of an increasing number of students), it has led to the acquisition of grades to become the main (and often only) assessment of successful learning and teaching.

UNgrading, why rating students undermines learning

During the last summer holidays, I read UNgrading, why rating students undermines learning (and what to do instead) ¹, a collection of chapters edited by Susan D. Blum, highlighting the issues with grading and sharing the experiences of some instructors that decided to drop them in their classes. To summarise the main arguments made by the many authors in the book, I’ll cite Marcus Schultz-Bergin again:

1. Grades do not track learning (or anything else of importance).
2. Grading reduces student learning.
3. Only receiving feedback (and no grades) increases student learning.
4. Self-evaluation and self-reflection improve student learning.

The last point refers to the concept of meta-cognition, thinking about how we think, and learn, and the ability to self-assess. It is paramount for students to learn how to receive and use the feedback they are given. Self-evaluation of one’s own learning process should be the ultimate goal of any teaching enterprise, as it essentially provides the means for anyone to learn any new subject they are interested in.

The authors provide more details about points 1 and 2 above, but my own observation made it abundantly clear that, despite my best intentions, tests offering a possibility to get a dispense for the exam wasn’t working. I took plenty of notes during my reading of the ungrading book and have devised alternative approaches that I’m looking forward to apply this semester. I will drop marking and put as much emphasis as possible on providing feedback, and leading students to assess their own work, focusing on understanding and mastering the material, rather than on obtaining grades. Before I describe my plans, I must specify that I will have to provide a grade for the end of year assessment, that students will earn with an exam. Even though I am convinced that marking isn’t an appropriate assessment of learning, I will first need to implement this in my classes and assess if/how this affects student learning and (hopefully) how they perform in that final exam. There is a long learning curve ahead (for me) and battle to extend this further. I have exciting plans for my bachelor and master courses. Read on…

First steps in ungraded land

I will be dropping grades in all the tests that are given throughout the term (typically at each or most lectures but the first one) in both bachelor courses. The first implication of this is that there won’t be any opportunity for students to be dispensed from the final exam, which is an opportunity several counted on. I will thus need to make sure that the reasons for this, described above, are clear (see point 1 below). The goal is to promote self-evaluation, feedback on their understanding of the material, and a reflection on their learning process. Instead of grading each individual test (a time-consuming activity), we will answer test’s questions together, in class, right after the test itself. This is something I have done previously, albeit rather swiftly. Here, I intend to take more time and involve the students as much as possible by asking for their active participation. After the collaborative correction, I will ask them to answer a quick poll, assessing whether (1) they understood the material but need more work to really master it, (2) they mastered the material and are ready to move on or (3) they neither understood nor mastered it and will ask for help (see point 2 below). Teaching assistants (TAs) and myself will spare some considerable time by not grading tens of copies (sometime over 60) and will seize it to put this new pedagogical strategy into place. After the poll, students will be invited to review their work after the lecture, in the light of the correction and their self-assessment and come to us to ask specific questions about their work, things they struggled with, or advice on how they could improve their learning (see point 3 below). The goal of this is to steer away from grades and re-focus on their understanding/mastering of the material, offer opportunities for personalised feedback, and promote self-evaluation and meta-cognition. Fingers crossed!

Self-evaluation works best in an environment where we trust each other. To emphasise this, I have also decided to change the attendance formalities. Since the beginning, I have made attendance mandatory. At my institution and in Belgium in general, attendance is only mandatory for practicals, and optional for formal lectures. Given that in my courses, theory and practice blend into each other, that I try to give as much hands-on work as possible during the classes, and that it is unlikely to master the material without that practice anyway, presence was mandatory, and TAs circulated attendance sheet that students had to sign. I will be honest that I didn’t check them systematically and didn’t follow up on absences. But I thought that this pressure would force students to attend and get them a chance to practise. I decided to change this. Attendance is still mandatory for the reasons mentioned above, but I will not use presence sheets anymore. If a student can’t make it, I’m asking for a short email to let me know that they aren’t or weren’t able to attend. I do not need any reason for the absence. I will just ask them to specify, in that email, how they plan to make up for the missed lesson, including asking me or the TAs for extra help. I think this approach builds on trust and self-motivation, invites students to take responsibility for their own learning, and promotes reflection of their progress, notably when they miss a class (whatever the reason).

First self-assessments (theirs and mine)

Here’s how things look like after a couple of weeks:

1. A short live poll (using sticky notes) during the first class of the year, after explaining the reasons for dropping the dispense and implementing the new strategy, was seen in a positive light by the great majority of the students. A handful of the 50 students regretted not being able to get a dispense. My impression is that these are the students that were aiming to get one (some of them probably already got one for the previous course), and consider this a loss (for them, personally) compared to the opportunities offered by the change.

2. The first post-test polls of the year indicate that the majority of students understand the material but needed more work to fully master it (from 55 to 85%). Smaller fractions of students don’t understand the material and conceded they need more help (8 to 14%) or fully master it (from 0 to 17%). The proportions are variable, depending on the chapter, but the fraction of students understanding the material is always the largest, by a comfortable margin.

3. We set up office hours during the week and just before the lectures, making sure that these didn’t conflict with the student schedule. So far, for the first weeks this has been in place, the number of students that booked and met with us was not in line with the poll - we meet with 1 to 3 students per week, out of a total over 40.

4. I have received a couple of emails from students not being able to attend. Most were related to clashes in their schedule (for students that had lectures from second and third bachelor years). Some of these emails did follow the guideline and explained how they intended to make up for missing the classes: work by themselves, use the recordings we provide, interact with class mates who do attend the lectures. Along the same lines, I had one student that explicitly wrote that they would revise the sections they struggled with on their end-of-class red sticky notes.

Adjustments for smaller classes

The master’s course provides more freedom to experiment further, as there are less students in the class and allows for more one-to-one interactions.

• So far, students had to prepare and present two data analysis reports throughout the semester course, focusing on two different high-throughput data types (transcriptomics and proteomics). We devote 3 to 4 successive session to each project: (1) lectures and exercises for them to learn and practice the theory, (2) a session where they test what they have learnt on their own data and can ask questions about the material and their data and (3) a short presentation with questions from the audience (other students and instructors), followed by, the following week, the submission of a detailed report that takes into account the questions and discussion during the respective presentations. I am considering to let students choose whether they prefer to do the second project, or propose improvements and extension to the first project and, if they wish, implement these during the next practice sessions.

• I am also considering to ask students to comment on each others written reports, in addition to the question and answer sessions, to further emphasise peer assessment and promote peer/self-evaluation. This could also be a good source of inspiration for improvements and extension to their first project (see previous point).

• During the master course’s oral exam, we focus primarily on the questions and discussions from the written reports. I systematically ask what students preferred in the course or felt they have mastered best, and what they struggled with. I would like to go a step further and ask them to propose and motivate their marks.

Ideas for the future

Depending on how the attempts above are received and how they influence the students’ learning, I might consider propose a more ambitious project to the 3rd bachelor students. This might only really apply to students that are re-taking the class and thus have already an overview of the whole material, and are in a position to anticipate the kind of data, analyses and interpretation that would be expected.

1. One or a group of students will be able to choose a project they want to work on throughout the year. Choosing a project would entail to identify a dataset and one or several questions they want to address with that data, confirm that the project would be written up as a (reproducible) report, and, if they work as a group, some sort of agreement that all members of the group will contribute.

2. They have total freedom in the complexity of the data and questions they choose to work on. It could either be a rather ambitous project, that would account for all credits/marks of the course, or choose something smaller. There could be an option to make up for the rest of the marks with (parts of) the final exam (that those that didn’t choose a project would have to take anyway), although I haven’t figured out the details of this.

3. The students and the education team (i.e. TAs and myself) would discuss their project (data and questions), the feasibility and the adequacy of their proposal, and the marks they would obtain upon successful completion. All these are open to discussion and amendments during the iterations in the next point.

4. Students and the education team would meet regularly to discuss the project’s advancement and provide guidance to help them to successfully reach their goals. These goals and the marks/credits could be revisited accordingly: an overly ambitous project could be reduced and, conversely, if the goals are attained earlier than expected, students could propose extensions if they wish so.

5. I think students should have the freedom to back out whenever they want to, if at any point they prefer to take the exam. They should also have the freedom to propose a project that counts for, say 50% of the marks, and settle with that without taking the exam if they reach their goals.

6. The completion of the project would be officialised by the final submission of the report. Importantly, the education team wouldn’t read/discover that report for the first time at that point, but would have followed its progress throughout the year so that, at the time of submission, little doubts persist as to the successful completion of the project.

7. If students worked as a group, they would need to provide a section describing their respective contributions. Again, these would have been laid out early on, and possibly updated, throughout the project. Every student of the group would get the same final mark but individuals would get recognised for their specific contribution.

Final thoughts

There is quite some excitement on my part with these new approaches, which has, I believe, a positive impact on my teaching. The lectures seem to have become more interactive: I try to make sure that all/most students get asked questions about the material we see and the exercises we do. Based on the sticky notes feedback that are collected at the end of each lecture, they seem to appreciate it because it improves their involvement and attention. In previous years, sampling students at random for to answer my questions was reported as being stressful on the sticky notes. It feels like clarifying and verbalising the goals gives me more freedom to reach out to the students and them to take it as a genuine learning opportunity. The latest set of sticky notes goes along these lines and gives hope for the future.

Edit:

• Here’s a first ungrading assessment, posted on 15 December 2022.
• A short interview, in French: Evaluation dans l’enseignement supérieur : « UNgrading » ou évaluer et apprendre sans points.
• A podcast (in French) on “Et si on oubliait les notes?”