This page collects the practical steps for teachers who would like to participate in the Open Pool Exam study. The aim is to keep the process manageable while collecting enough information to evaluate student experience, engagement, and learning outcomes. The student assistant for the study is Tessa Holzhaus.
The study is easiest to understand when the tasks are grouped by phase of the course. Some steps happen before teaching starts, some during the course, and some after the course has finished.
These are the main preparations that help set up the Open Pool format and make the study possible from the beginning.
Prepare a list of Open Pool Exam questions for the course. This can still be done while the course is already running, but ideally the main structure is ready early.
The list should cover the full set of learning goals of the course, while remaining short enough that students can realistically study the whole pool within the available time.
Tell students that at least X% of the points on the final exam will come from the Open Pool question list.
A reasonable value for X is between 30% and 70%. The exact percentage can be chosen based on the course structure and your own judgment.
Share anonymized grade data from previous years. We are interested in the same kinds of aggregate information as for the current year.
This is especially relevant if the Open Pool Exam method was not used in those previous years, since it allows a more meaningful comparison.
If that is not possible, a useful alternative is comparable data from a parallel course of similar difficulty.
Optionally, include one exam question that was also used in a previous year, without drawing attention to it as a comparison item.
This can make year-to-year comparisons more precise than comparing only final grades, since overall exam difficulty may vary between years.
These tasks support recruitment, consent, and data collection while the course is running.
Give the student assistant the opportunity to give a short presentation of around 10 minutes to explain the study to the students.
Ideally, this happens within the first two weeks of the course. This step is important for the ethics procedure.
Give the student assistant the opportunity to collect signatures for the consent forms.
Ideally, this happens directly after the short presentation, when the study is still fresh in the students’ minds.
Give the student assistant the opportunity to get in contact with students, ideally in person, in order to conduct five interviews.
These interviews take place outside classroom hours, so this mainly concerns access and coordination rather than using teaching time.
Optionally, measure a learning-related attitude both at the beginning and at the end of the course using a single WooClap question. This should not be anonymized.
For example, in Discrete Mathematics we used the statement:
“I am excited to learn more about Discrete Mathematics.”
Students then rate their agreement on a five-point scale.
Give the student assistant the opportunity to run a survey during the last lecture or at the end of the course, for example using WooClap or similar software.
The survey can also be administered by the teacher. If there is really no time during class, it can be distributed online, although that usually leads to a noticeably lower participation rate.
After the course, the focus shifts to evaluation, outcome data, and reflection on the teaching experience.
Share the student evaluations from the current year and, if possible, from the previous three years.
These help us compare how the course is experienced over time and whether the Open Pool format changes student perceptions.
Share the final grades of the students. Ideally, this includes a breakdown that separates points earned on Open Pool questions from points earned on surprise questions for each exam question.
If available, it is also useful to provide intermediate grades such as homework, quizzes, or midterm results.
If full individual data is difficult to provide, aggregate data is also valuable.
We would also like to interview you as the teacher about your experience with the course and the Open Pool approach.
These interviews typically take between 30 minutes and one hour.
If you are considering participation, the process can be adapted to the practical constraints of your course. It is completely fine to discuss what is easy to implement and where a lighter version may be more realistic.
Tessa Holzhaus
For questions about the study, participation, or how to adapt the setup to a specific course, please get in touch.