These preliminary results come from a first-year Logic course in the second block of the Bachelor Computer Science at Utrecht University. The course lasts 10 weeks, with the final week reserved for the exam. The Open Pool Exam method was used for the first time in this course during the period from November 2025 to February 2026.
Students in this course report relatively low levels of motivation. The reasons appear to include a general aversion to mathematics and a perceived lack of usefulness or connection to their broader study goals. The course itself has been taught consistently over many years, although the co-teacher and teaching assistants change frequently.
The study combines interviews with students and teachers, analysis of grades, a student survey administered in the final lecture, and comparison with annual student evaluations from previous years. In total, 52 students participated in the survey, while about 150 students followed the course.
We investigated several theoretical expectations about the method. Many of them are supported by the available data. Some remain non-conclusive, either because they are difficult to measure or because the current evidence is mixed.
From the student evaluations, we have the following self-reported average study hours:
| 2022 | 12.6 |
| 2023 | 12.5 |
| 2024 | 12.5 |
| 2025 | 13.6 |
This suggests that students spent more time on the course with the Open Pool Exam method.
We define the passing rate as the proportion of students who passed the course without resits among those who participated in the final exam.
| 2021 | 68% |
| 2022 | 78% |
| 2023 | 70% |
| 2024 | 74% |
| 2025 | 79% |
Compared with an average of 72.5% in earlier years, this suggests an increase of about 8 percentage points.
At the same time, the teacher indicated that the exam may actually have been graded more strictly, because students had already seen a fraction of the questions in advance.
Expectancy Value Theory suggests that students are more likely to invest time in studying when they value the outcome and believe success is attainable.
“Studying the Open Pool Questions is a good time investment.”
| 5 – I completely agree | 52% |
| 4 – I somewhat agree | 35% |
| 3 – Neutral | 8% |
| 2 – I somewhat disagree | 4% |
| 1 – I disagree | 2% |
This indicates that students generally regarded studying the pool questions as a worthwhile investment.
It is important that a method is accepted by students if it is to work well in practice.
“I like that the course uses Open Pool Exams.”
| 5 – I completely agree | 52% |
| 4 – I somewhat agree | 29% |
| 3 – Neutral | 0% |
| 2 – I somewhat disagree | 0% |
| 1 – I disagree | 4% |
Students generally liked the method. The Open Pool Exam format was also mentioned many times in the student evaluations.
One theory behind the method is that the format encourages students to actively test themselves instead of passively reading solutions, and test-enhanced learning is generally known to be more effective.
While students indicate that they do test themselves rather than immediately looking at the answer, this might also happen without the method. At the moment, the evidence for this mechanism is therefore non-conclusive.
One of the main criticisms of the Open Pool Exam method is the fear that students may memorize answers without understanding them. This is perhaps the strongest criticism of the method, but also the point where the current evidence is strongest.
Both the interviews and the survey suggest that rote memorization is not a meaningful issue in this setting.
“I prefer to understand the solution of the Open Pool Questions rather than memorizing the solution.”
| 5 – I completely agree | 71% |
| 4 – I somewhat agree | 25% |
| 3 – Neutral | 2% |
| 2 – I somewhat disagree | 0% |
| 1 – I disagree | 2% |
This is a strong indication that students prefer understanding over memorization when working with the pool.