Online Proctoring With AI: The Issues and Solutions

Test proctoring began with human proctors overseeing exams to keep the integrity exams intact. In recent years, it has become apparent that artificial intelligence can play a key role in the exam process by replacing—and even expanding upon—some of the functions of human proctors.

Online proctoring with AI can provide the scalability and affordability needed to accommodate students in remote settings. But test validity can be called into question and the student experience can be compromised when institutions rely solely on artificial intelligence.

Human proctoring should be a key component of exam security whether tests are administered online or in person. Humans have the wisdom to quickly recognize whether the student behavior flagged by the AI is benign or a real concern.

Rick Hebson understands this. Rick has worked as a proctor for over eight years and has overseen more than 10,000 tests. During that time, he has witnessed many students go to great lengths to cheat on exams, from checking mobile devices strategically placed out of webcam view to using fake IDs to having twins standing in for each other. Despite the state-of-the-art technology used to prevent students from cheating, some will still find a way around the system, whether through common or more creative.

But Rick believes that proctoring is not all about catching students cheating. As he says “No one in this business gets up and goes to work excited about the idea of catching people doing bad things.” In fact, he is primarily driven by positive outcomes that came from proctoring exams.

Countless times proctors serve as impartial judges, not to catch students but to help them. I can’t tell you how many times a cat has jumped on a keyboard and prematurely submitted a test. Or how many times a student has had to leave a test because they’re sick. We are able to say to the school or their testing body, ‘yes, the cat really did eat their test.’ Having that record, that neutral observer can be very important when the stakes are high.

Artificial intelligence does not have the capability to understand a cat jumping on a keyboard or a student leaving the testing environment due to illness. The threat of being flagged for benign actions without recourse can leave test-takers feeling overwhelmed in an already stressful experience. This leads to increased anxiety amongst students and questionable results.

Students are increasingly critical of online proctoring with AI due to the ethical issues it raises. For many, those begin at the very beginning of the test, when fully automated systems use facial recognition software to verify identity.

AI is only as good as the dataset on which it is trained—and most leave a lot to be desired. Without a large and diverse dataset, facial recognition software may be unable to recognize the student, effectively stripping them of their right to learn. In recent years, for example, some proctoring solutions with AI have failed to recognize students with dark skin tones and certain medical conditions.

But the bias doesn’t end there. Students whose appearance and behaviors aren’t represented within the dataset may be incorrectly flagged for violations because the system doesn’t recognize their patterns. This bias leaves minority populations vulnerable to being branded as cheats.

Developers of online proctoring solutions can minimize the risk of discrimination by training their AI on a dataset that includes a wide range of skin tones, ethnicities, genders, and neurotypes. With a system capable of recognizing a diverse array of test-takers, functionality is improved for all students. As the dataset expands further, a dynamic system can learn from what it gets wrong, which means it becomes more accurate with every use and more valuable for exam proctoring.

However, even the most sophisticated AI works best as a supplement to humans rather than as a replacement. This improves accuracy and assures students that their success is not dependent solely on potentially flawed software.

Some online proctoring systems with AI have burdensome technological requirements. From only working on a few specific OSs to requiring high-speed internet, such systems can present significant barriers, particularly to economically disadvantaged students. AI-driven proctoring solutions with low tech requirements can give students better, more equitable online exam experiences. This includes compatibility with a broad range of operating systems and good functionality in low bandwidth environments.

Rosalyn believes that combining AI technology with human wisdom is the best way to create positive student experiences and valid results students and faculty can count on. Our human-in-the-loop (HITL) proctoring solution offers the functionality needed to provide fair and comfortable proctoring that respects student privacy and dignity.

• All potential violations are sent to a human proctor for review, which means students are never accused of cheating based on flawed AI.
• We train our AI on our own proprietary database of nearly 250,000 sessions that represent test-takers from all over the world, minimizing the risk of algorithmic bias.
• Our system continually learns from test sessions and becomes more intelligent over time.
• Rosalyn is user-friendly and has low tech requirements.
  

Online proctoring AI issues must be overcome to ensure successful testing in remote environments. With Rosalyn’s dynamic HITL system, students, educators, and institutions alike can benefit from more ethical testing practices and more reliable results.