Choosing Online Test Administration Software With Monitoring That Is Not Too Intrusive

Aisha Kamran, a Junior at the University of Illinois at Urbana-Champaign and a member of Rosalyn’s Student Advisory Board, had her computer bricked by an overly aggressive remote proctoring service.

"It took over my entire computer. I got a Mac in the first place so I could airdrop, message, and have my computer connected to my iPhone. The proctors’ system came in and disconnected it completely. Somehow my whole iCloud was deleted, including my photos. They shouldn't have to control my laptop completely. You can see it changing your entire settings. I tried to change it back to default, and it didn't work out."

Kamran had to send her laptop back to Apple to have it sorted out.

Reddit is also replete with anecdotes of overly aggressive proctoring.

• On r/UniversityofArkansas, LtDominator says, “As far as the privacy goes, I agree, they also have access to your browser history and saved passwords...It's absolutely an invasion of privacy, but schools insist on using it.”
  
• On r/OSUonlineCS, RabbitwithADHD says, “I have the luxury of having a laptop that I’ve mostly just used for proctored exams. Once I graduate, I plan on just wiping the entire laptop clean just to make sure. I really doubt that the proctoring software will cause me any issues as it is right now, but you can never be too careful about these things.”
• On r/Professors, Vakieh says,“[It] is in essence rootkit spyware that takes over the target (almost exclusively student-owned) computer. The level of control it requires over the machines it is installed on is insane.”

A page of online examination stories from WGU shares anecdotes where students had their exams stopped after being flagged for innocuous behavior.

• “I had mine stopped because when I am alone I read the questions out loud when I am taking a test on my computer. I never realized I was doing it until I literally had my test stopped three times!”
• “I had my proctored exam stopped because I stretched my cramping hand out of camera range. Trust me - THEY are watching! :)”
• “I had my test stopped because of an inconsistent rumbling noise that they believed could be someone trying to hum/signal answers. I just moved the camera (while on the phone) to show that it was my snoring cat! I'm glad that they watch. I want my hard work to be worth it!”
• “I've had tests stopped a couple of times for bad camera position and because I looked out the window. LOL”

These online test administration examples point out that there are real impacts of overreach in remote proctoring technology. Beyond concerns for test-taking anxiety and unease at being observed while taking exams, taking too little care evaluating the intrusiveness of proctoring systems can take more than an emotional toll. To be more confident of choosing online test administration software with monitoring that is not too intrusive, select a system that only asks for the minimum permissions needed to invigilate the test and nothing more.

Invoke the personal agency of the student in configuring the test computer and the disposition of the test session data. Students want to feel at least a modicum of control of the online exam process and that the invigilation process is responsive to their actions.

1. Give students control. Let students change their computer settings to comply with the rules instead of taking over control of the computer. Let the student know what programs they need to shut down and what devices they need to unplug to be ready for the test.  Asking the student to close apps avoids the mysterious reconfigurations of the computer like Aisha Kamran’s.
2. Tell students how it works. Communicate to students what the remote proctoring system is looking for and why. Apprise test-takers of what kinds of behavior are flagged and how to avoid inadvertently triggering attention to your test session.
3. Use the best technology. Choose a sophisticated proctoring technology that uses human-in-the-loop artificial intelligence (AI) to monitor exams. The best of these systems use purpose-built AI’s that are explicitly trained on test sessions.  These systems can likely tell the difference between a snoring cat and a nefarious audio feed. Here you are minimizing the interruptions of the test-taker. Having a real person dismiss or confirm a suspicious event flagged by the AI before pausing or stopping a test lowers the intrusiveness of proctoring.
4. Give students a choice. Proctoring systems need to capture audio and video of the test session to work.  What happens to that data after the test is where the intrusiveness of remote proctoring tends to bother students. Proctoring companies that use AI  want to hold onto test session data. Not to sell to advertisers, but because more data makes the AI better at proctoring. We are all daily users of AI.  If you’re like me, you are continually amazed at how much better things like voice recognition, driving directions, and internet search have gotten.  All those improvements have been driven by AI and access to enormous amounts of data to train them.  The benefits of improved AI are so clear. If given a choice, most students would gladly give their consent to use their data for that specific purpose for a specific period of time.

Choosing online test administration software with monitoring that is not too intrusive is easier if you start with the principles of minimization and personal agency. At Rosalyn, we follow these principles throughout the development of our proctoring platform.

A great example of this effort is the founding of Rosalyn’s Student Advisory Board (SAB). A first in the industry, this panel of undergraduate students from esteemed institutions all over the world advises the company on the student experience of taking proctored exams.  Insights from the SAB help shape the design of our proctoring solutions and ensure that we balance the control we exert over the test environment with respecting students’ rights, dignity, and agency.