A British university has turned the microscope on itself, carrying out a Turing test under real conditions.
The test was developed by Alan Turing in 1950. The leading mathematician and computer scientist proposed the following criteria for the level of artificial intelligence: If a machine can fool humans into believing it is human, then it must be at least as intelligent as a human. Turing even predicted that by the year 2000, artificial intelligence would have been developed that could fool 30% of those interviewed, after five minutes of discussion.
In a new study from the University of Reading, which was published in the journal “PLOS One” the experiment is presented.
33 fake student profiles were created by the university’s technology team. The “students” were all completing their bachelor’s degrees in psychology, in different years of study. Researchers helped… fake students complete a series of online assignments and exams, using raw answers written by OpenAI’s GPT-4 artificial intelligence chatbot.
In what is believed to be the largest study of its kind, researchers found that 94% of AI submissions went unnoticed by examiners. Furthermore, the AI students performed better than their peers. Their grades were, on average, about half a point higher than those of real students, with an 83.4% chance of outperforming a random selection of real student submissions.
Awakening and assessment
Professor Peter Scarfe, one of the study’s authors, said its findings should serve as a “wake-up call” for educators, with a recent UNESCO survey revealing that less than 10% of schools worldwide had policies or guidelines for the use of genetic intelligence, which has been leading the charge in recent years in the technology industry and the field of artificial intelligence, leaving behind the machine learning AI branch that was leading before the emergence of ChatGPT and its rival programs that appear every day.
“Many institutions have moved away from traditional exams to make assessment more inclusive. Our research shows that it is of international importance to understand how AI will affect the integrity of educational assessments. We won’t necessarily go back to handwritten exams entirely, but the global education sector will need to evolve in the face of AI,” says Scarfe.
Naftemporiki.gr