The Proctor will See You Now
Featuring contributions from UCC’s Head of Digital Education Tom O’Mara (@tomomara), UCC Student Intern Catherine Dawson and IUA Student Intern Ruairi O’Gallchoir.
In a year where Covid-19 and its consequences have complicated many aspects of our lives, education has probably undergone the biggest transformation. Several months in, and lectures and students alike are finally becoming accustomed to online learning. But now as assessment time approaches, there is a whole new set of controversies and difficult decisions brewing over how to conduct these exams remotely, and how to ensure that academic integrity remains intact. There have been relatively benign solutions proposed, such as getting students to sign ‘honesty contracts’ and ensuring that they are aware of the strict consequences of cheating, but nothing has quite captured the headlines in the way the proctoring has.
Proctoring at its core means ensuring that no student cheats. In the real world, we all know the form that this takes. During in-class tests, lecturers leave space between students to prevent copying and keep an eye on them from the top of the room to ensure there are no wandering sets of eyes. During assessment time, students in an exam hall are used to seeing invigilators doing laps of the hall, answering questions and making sure that all centre rules are adhered to.
But when exams are moved into a students home environment, the need to ensure academic integrity can sometimes go too far. We have already covered in our EDTL Webinar series both an academic and student perspective on how to create effective assessment. However, if a module is offering a “closed-book” exam, the lecturer might still feel it necessary to go further than just taking students at their words that they will follow all the regulations surrounding the test.
Enter remote proctoring. With almost every device having a camera of some sort built-in these days, why not use this to simulate a real-life exam environment, where students take the test under the watchful eye of an invigilator, or machine? The reasons not to built-up very fast once proctoring was introduced in some universities worldwide, with students reporting a range of issues from the technological to the personal.
Since remote learning became the primary form of learning earlier this year, what was previously a niche industry of online proctoring tools has become the centre of attention. Many people would probably be surprised at the variety of remote proctoring providers that exist and the variety of features that each offers. If you want your students to experience remote proctoring at the bleeding edge of innovation, there are many providers which use Artificial Intelligence to try and automate exam integrity. If you don’t have the time to review each student’s work manually, a proctoring company will be able to run it through their automated system or even have an employee watch over the footage from start to finish and report back to you with any infractions. So it is clear that remote proctoring has moved with the times, but that has not stopped several of the same problems cropping up as more and more universities implement this form of invigilation.
For example, a regular problem that emerges from this software is that black students often have trouble accessing their exams. Most proctoring solutions offer automatic biometric scanning to ensure the correct person is taking the test, however this has posed problems for many black students worldwide. The MIT Technology Review wrote of a black student in the US who had such trouble identifying herself for the proctoring software (despite following all instructions to a tee) that she had to go to her professor and request alternative exam arrangements, just on the account of poor software design.
There is also the personal privacy aspect. Many people say they are uncomfortable with broadcasting with Zoom and other video conferencing software, but many proctoring solutions take this to the next level. In order to ensure that there are no other devices or exam materials around the student, a ‘room scan’ is required by many providers. This involves the student bringing their device around the room with them and letting the software record their surroundings. These are then stored on the provider’s servers and depending on the service offered, can be viewed or downloaded by both remote invigilators or lecturers long after the student has finished their exam.
The ability of proctoring services to make students uncomfortable is not limited to the realm of race and privacy however. A Washington Post article covering how students are fighting back against proctoring software mentions how one of these movements was sparked by an associate professor sending out an email to all students who took a test commenting angrily on the head and eye movements of one student. Head and eye movements are often picked up by proctoring software as red flags that could be indicative of cheating, and this particular student was being publicly called out for having 776 head and eye movements in the space of 6 minutes.
The fact that something as simple as a shift in a student’s gaze can be detected is both a testament to the advancement of remote learning technology and a warning signal for universities looking to implement remote proctoring. Taking an exam is already a stressful time, and students having to worry about explaining away a high-tech piece of software’s assumptions about their honesty is just another layer compounding the anxiety and strain many students are feeling during remote learning.
These are just some of the problems that remote proctoring can cause, and the UCC EDTL Intern, Catherine Dawson, shares her perspective here on how this is only the tip of the iceberg.
The feeling of being watched can set off a flight and fight response in people. It’s only natural, but it can be too much for people already in the high-stress environment like an exam. Many students find this time of recording very intrusive and treating everyone as suspicious, as a criminal. Students aren’t the enemy, and working with us on our academic integrity will lead to a decrease in plagiarism.
An advantage of online assessment is that it can assess students in a more comfortable environment. Open book assessments can allow students to use the resources they would have in the modern world and the workplace. This can ensure that students are in the best mind frame to display all the knowledge they have been learning. Adding the stress of every move being monitored isn’t kind to students.
On another note, many students require privacy taking an exam due to mental health reasons like anxiety or a disability. They would get those accommodations from their institution, a private room and staff to help them. Monitoring those students in the privacy of their homes would be extremely inappropriate and can exacerbate conditions. It is vital to acknowledge the range of students taking your exams and realised that remote proctoring isn’t kind or fair to a majority of students, and you end up throwing the baby out with the bathwater.
Catherine Dawson – ucc edtl student intern
That is not to say that proctoring has no redeeming qualities, and when done in a non-invasive manner can even be reassuring to students. This is demonstrated by the EDTL Intern from Maynooth University, Michaela Waters.
My personal experience with proctoring has been extremely positive. Thankfully, my lecturers were very mindful of this new assessment type. Students’ worries and possible confusion with the adapted exam formats were taken into account. We received very in-depth instructions for the exam regarding submission and what to do in case we could not submit on the virtual learning environment. I was relieved to have a practice test run of submitting a sample assignment/exam on Moodle.
Lecturers allocated one class for students to ask questions regarding the exam. During the timed exam, all students were invited to be in an optional MS Teams call, so that any questions could be answered immediately through chat. Extra time was also given to allow for internet issues and formatting exam scripts into PDF format.
I felt prepared for my online exams because of my lecturer’s attention to detail and listening to student feedback. I believe effective proctoring can be empowering for students. It can also help build trust and develop the staff and student relationship as a partnership.
Michaela waters – mu edtl student intern
Assessing students online brings with it a whole new set of challenges that universities will need to adapt to. Tom O’Mara, Head of Digital Education in UCC, offers the following advice to academics when considering proctoring their assessment.
Use of remote invigilation needs to be carefully considered by staff. As we provide learning and assessment online, consideration should be given to assessment design:
– Is it possible to move away from a traditional exam and provide other opportunities for students to demonstrate knowledge and competence across learning outcomes?
– Where the only way to test knowledge is through an MCQ-style quiz, then randomised question banks provide a way to ensure no two quizzes are the same.
– Learning analytics provides a way for staff to identify outliers in performance – if a student suddenly achieves a very high grade in a summative assessment exercise while previous continuous assessment performance was of a lower standard, then this might point towards something that warrants further investigation.
· If remote invigilation is required, then would lighter touch options using existing technologies and university staff suffice?
tom o’mara, head of digital education, ucc
Tom also notes that the above options are not always suitable, especially in cases where external according bodies are involved and there are very specific requirements to assure them of examination integrity. In these cases, remote invigilation software may be the only option where on-campus exams are not feasible. Saying this, it should be noted that UCC does not currently have an agreement in place with providers of remote invigilation software, although a pilot project should be in place during 2021 for two schools who require this for professional accreditation purposes.
It is clear that ensuring academic integrity in a remote setting is a tricky and sometimes thorny issue. There are many alternatives to remote proctoring, a lot of which are explored in our EDTL Webinar on Consider Technology to Support Academic Integrity. A lot of talk about remote examinations comes down to creating an environment where academic integrity is encouraged, and remote proctoring in many cases replaces this with a feeling of distrust and surveillance.
Follow the #IUADigEd social platforms for more: