top of page
Dr. Stackhouse's blog has moved to as of August 2023 visit there for the latest posts and news!
  • Writer's pictureJohn Stackhouse

ChatGPT and the Turing Test: Biblical Studies Version

Updated: Mar 3, 2023

My inventive colleague, Dr. Isaac Soon, recently offered a constructive counterpart to all the handwringing in academia about ChatGPT. Instead of bemoaning ChatGPT’s potential to mimic student papers for fear of widespread cheating, why not use ChatGPT as a teaching and research tool?

Isaac put up on his weblog a process he could supervise, as a teacher of the New Testament, to help students dive deeper and deeper into the study of a text.

With a series of iterative steps, students could get a general sense of the passage, an overview of the main critical question, help with the actual Greek text, and even a bibliography for further study.

ChatGPT as a graduate assistant! The whole exercise seemed almost too good to be true.

And it was.

Soon after he published his blog post, a scholarly friend of Isaac’s tipped him off. One of the sources adduced by ChatGPT was incorrect. Just page numbers, but still.

So Isaac, assiduous scholar that he is, submitted all the references to scrutiny. And he became dismayed.

Real authors—as notable as Paula Frederiksen and James D. G. Dunn—were cited, but with the wrong cities for the publisher cited, or the wrong publisher, or even simply fabricated articles in toto. The more Isaac dug, the more he found bogus material.

ChatGPT really was just mimicking human work. (Indeed, I concluded darkly that ChatGPT demonstrated that it truly was capable of writing student papers—but not in a good way.)

Isaac announced his finding in a rueful update to his blog post, pronouncing himself “idiotic.”

But Isaac Soon is no idiot. He is a young scholar with a master’s degree from Oxford and a Ph.D. from Durham. We turned down a fine candidate with a Princeton Seminary Ph.D. to hire Isaac Soon instead. Barely graduated from his doctoral program, Isaac has already published a handful of journal articles, including one in the Journal of Biblical Literature, and has a book contract with OUP.

And he was fooled.

So here’s to a clever idea for how to use ChatGPT as a research tool—except not yet, it seems. Here’s to a clever scholar who, with his network of learned friends, caught and fixed a public mistake.

And here’s to lying awake at night, still, worrying about ChatGPT after all.

[Note 1: This post was originally posted on the Christian Scholar's Review weblog.]

[Note 2: If you want to read (listen and watch) more content like this, enroll in ThinkBetter Media for exclusive content on apologetics, theology, epistemology, ethics, culture, and discipleship.

Sign up today and start a two-week free trial on our Sustainer memberships HERE.]


Want more content on apologetics, theology, epistemology, ethics, culture, and discipleship?


ThinkBetter Media  was created by Professor Stackhouse to provide accessibleinformed, balanced, and practical Christian insight and direction around crucial issues in contemporary culture.

Start a two-week free trial of our Sustainer memberships.

bottom of page