US colleges turn to oral exams to combat AI
The assignment involves no laptop, no chatbot and no technology of any kind. In fact, there’s no pen or paper, either.
Instead, students in Chris Schaffer’s biomedical engineering class at Cornell University are required to speak directly to an instructor in what he calls an “oral defense.”
It’s a testing method as old as Socrates and making a comeback in the artificial intelligence (AI) age. A growing number of college professors say they are turning to oral exams, and combining a variety of old-fashioned and cutting-edge techniques, to help address a crisis in higher education.
“You won’t be able to AI your way through an oral exam,” says Schaffer, who introduced the oral defense last semester.
Getting more sophisticated
Educators are no longer naively wondering if students will use generative AI to do their homework for them. A big question now is how to determine what students are actually learning.
College instructors across the United States are noticing troubling new trends as generative AI becomes more sophisticated. Take-home essays and other written assignments are coming back perfect. But when students are asked to explain their work, they can’t. The long-term impact of AI use on critical thinking remains to be seen, but educators worry students increasingly see the hard work of thinking as optional.
In-person tests
At the University of Pennsylvania, Emily Hammer, an associate professor of Middle Eastern Languages and Cultures, now pairs oral exams with written papers in her seminar classes.
“It comes across as if we’re trying to prevent cheating,” Hammer says. “That’s not why we’re doing this. We’re doing this because students are actually losing skills, losing cognitive capacity and creativity.”
Hammer forbids AI use on all writing assignments but tells her class she knows she can’t enforce that. However, if they haven’t written their papers themselves, defending the material face-to-face will likely be “a very stressful situation.”
Faculty workshops
Hammer’s class is part of “a massive shift toward in-person assessments,” both written and oral, at Penn, says Bruce Lenthall, executive director of the school’s Center for Teaching and Learning. The Ivy League school is one of a small but growing number of universities that have started running faculty workshops on oral exams.
Oral exams are not traditionally part of the modern American undergraduate system, unlike certain European universities. For instance, in the Oxbridge tutorial system in England, students meet faculty for weekly discussions. Some US colleges saw a move toward oral exams during the COVID-19 pandemic to address concerns about online cheating, and interest has intensified since the launch of ChatGPT in 2022.
During the pandemic, engineering professor Huihui Qi launched a three-year study at the University of California, San Diego, on how to scale oral exams. Several universities have since invited her to provide faculty workshops or discuss her research.
Fight ‘fire with fire’
At New York University, several types of oral assessments are on the rise. More faculty are requiring office hours, assigning presentations and cold-calling on students in class. Instructors are saying, “I need to look my students in the eye and ask, ‘Do you know this material?’” says Clay Shirky, vice provost for AI and technology in education.
One NYU professor has put a modern spin on the traditional oral test.
Panos Ipeirotis, a professor at NYU’s Stern School of Business, unveiled an AI-powered oral exam last semester for the final exam in a class on AI product management. He calls it “fighting fire with fire.”
Students log in from home, at any time that fits their schedule. A voice cloned from a business school professor greets them.
“Hi there,” says the voice on their screen. It asks for the student’s name and school ID number and then says, “I’m ready to conduct your exam today.”
AI voice agents
The chatbot starts with questions about a final group project and drills into details based on each student’s answers. If the student stumbles, the AI agent gives them clues, along with criticism and positive feedback. Ipeirotis grades the exams separately, also with the help of AI.
“We wanted to check: Do you know what your team did? Were you a free rider? Did you outsource everything to AI?” says Ipeirotis, who designed the tool with ElevenLabs, a company that develops generative AI voice agents to conduct job interviews.
Students in the class this semester are redesigning the AI agent to smooth out some kinks, and Ipeirotis plans to use it in all his future classes.
“I want oral exams everywhere now. I want to pair it with every single written assignment,” says Ipeirotis. “I don’t trust written assignments anymore to be the result of actual thinking.”
Mixed feedback
Feedback from students last semester was mixed.
Business major Andrea Liu found the chatbot’s voice to be surprisingly human, but the conversation felt choppy with odd pauses. It asked multiple questions at once, which was confusing. And it was jarring to hear a voice but not see a person.
“It felt kind of awkward to be talking to what was pretty much a blank screen,” says Liu, 21.
But she agreed with worried educators: “There is no perfect world where AI exists and kids are not abusing it.”
Benefits for shy students
Across the humanities and STEM disciplines, like computer science, educators worry that students who skip the mental struggle that is necessary for problem solving won’t develop the skills they need to advance in upper-level classes and careers.
That’s why Schaffer, the Cornell professor, introduced the oral defense in his biomedical engineering class. He requires students to sign up for 20-minute sessions of Socratic-style questioning after submitting written problem sets, which are assigned several times each semester.
With a class of 70 students, Schaffer splits the job with his teaching assistants. They no longer grade the written problem sets, just the oral defenses.
He calls it “incentivizing” his students to do the work, or at least understand it enough to explain.
