The writer is a science commentator
Large language models like ChatGPT are purveyors of plausibility. The chatbots, many based on so-called generative AI, are trained to respond to user questions by scraping the internet for relevant information and assembling coherent answers, churning out convincing student essays, authoritative legal documents and believable news stories.
But, because publicly available data contains misinformation and disinformation, some machine-generated texts might not be accurate or true. That has triggered a scramble to develop tools to identify whether text has been drafted by human or machine. Science is also struggling to adjust to this new era, with live discussions over whether chatbots should be allowed to write scientific papers or even generate new hypotheses.
The importance of distinguishing artificial from human intelligence is growing by the day. This month, UBS analysts revealed ChatGPT was the fastest-growing web app in history, garnering 100mn monthly active users in January. Some sectors have decided there is no point bolting the stable door: on Monday, the International Baccalaureate said pupils would be allowed to use ChatGPT to write essays, provided they referenced it.
In fairness, the tech’s creator is upfront about its limitations. Sam Altman, OpenAI’s chief executive, warned in December that ChatGPT was “good enough at some things to create a misleading impression of greatness . . . we have lots of work to do on robustness and truthfulness.” The company is developing a cryptographic watermark for its output, a secret machine-readable sequence of punctuation, spellings and word order; and is honing a “classifier” to tell the difference between synthetic and human-generated text, using examples of both to train it.
Eric Mitchell, a graduate student at Stanford University, figured a classifier would take a lot of training data. Along with colleagues, he came up with DetectGPT, a “zero-shot” approach to spotting the difference, meaning the method requires no prior learning. Instead, the method turns a chatbot on itself, to sniff out its own output.
It works like this: DetectGPT asks a chatbot how much it “likes” a sample text, with the “liking” a shorthand for how similar the sample is to its own creations. DetectGPT then goes one step further — it “perturbs” the text, slightly altering the wording. The assumption is that a chatbot is more variable in its “likes” of altered human-generated text than altered machine text. In early tests, the researchers claim, the method correctly distinguished between human and machine authorship 95 per cent of the time.
There are caveats: the results are not yet peer-reviewed; the method, while better than random guessing, did not work equally reliably across all generative AI models. DetectGPT could be fooled by making human tweaks to synthetic text.
What does all this mean for science? Scientific publishing is the lifeblood of research, injecting ideas, hypotheses, arguments and evidence into the global scientific canon. Some have been quick to alight on ChatGPT as a research assistant, with a handful of papers controversially listing the AI as a co-author.
Meta even launched a science-specific text generator called Galactica. It was withdrawn three days later. Among the howlers it produced was a fictitious history of bears travelling in space.
Professor Michael Black of the Max Planck Institute for Intelligent Systems in Tübingen tweeted at the time that he was “troubled” by Galactica’s answers to multiple inquiries about his own research field, including attributing bogus papers to real researchers. “In all cases, [Galactica] was wrong or biased but sounded right and authoritative. I think it’s dangerous.”
The peril comes from plausible text slipping into real scientific submissions, peppering the literature with fake citations and forever distorting the canon. The journal Science now bans generated text outright; Nature permits its use if declared but forbids crediting it as co-author.
Then again, most people don’t consult high-end journals to guide their scientific thinking. Should the devious be so inclined, these chatbots can spew an on-demand stream of citation-heavy pseudoscience on why vaccination doesn’t work, or why global warming is a hoax. That misleading material, posted online, can then be swallowed by future generative AI to produce a new iteration of falsehoods that further pollutes public discourse.
The merchants of doubt must be rubbing their hands.