From a different section of that Max Raskin article in the Free Press:

Most people initially reject this premise because they think they can tell whether an essay was written by a human. Enough data and anecdotes are pouring in to show this confidence is an illusion. In a study published in August by Villanova University researchers, 1,682 adults read short fiction written either by AI or human beings. They not only preferred the stories written by AI, but in follow-up experiments, they did no better than a coin flip in determining authenticity.

The educated literary elite performed no better—in a separate study, researchers at Columbia, Michigan, and Stony Brook universities had readers with MFAs try to distinguish between passages written by AI and passages written by writers with MFAs. When the models were trained on a writer’s body of work, the MFA readers preferred the AI-generated passages to MFA-generated passages, and the best detection tools only flagged the AI text 3 percent of the time.

I’ve been running my own little test on the writers and editors I have been discussing this idea with. These are literary minds at some of the country’s most prestigious writing institutions. What I do is I pick a handful of authors they would be familiar with and then create a test—one real passage and one fake.

The first time I ran the test earlier in the year, I had Anthropic’s Claude generate an imagined passage and a real passage from Pride and Prejudice. I asked if two of my literary-minded friends could tell the difference. One could. But that’s because she knew the book chapter and verse. We then tried the same test with Moby-Dick, and although they were both confident in the answer, they both got it wrong.

The models have improved even since then. I went to the most well-read person I know—a rapacious (I used that word even though voracious is better because you have to know it wasn’t an AI who wrote this) consumer of 18th- and 19th-century British historians. He had a failure rate of 100 percent of the passages he did not already recognize. He was unable to tell the difference between genuine and faked Gibbon and Macaulay.

For reasons I can only speculate about, the “educated literary elite” — I will resist paraphrasing Voltaire here — have decided to stake the honor of thousands of years of literary achievement on a parlor game. They have foolishly attempted to claim territory they haven’t a chance of holding. In order to defend the merits of organic human prose, they have committed themselves to the proposition that AI prose is so clearly inferior and predictable that a trained eye can spot it at a glance. Thus they suffer the embarrassing routs described above. All because they decided to tie their credibility to a cheap conjuror’s trick. They act as if the essence of literature depends on them being able to “read” maximum information from a minimum of clues.

I picture a gathering of the educated literary elite in a darkened room, seated around a table draped in black velvet. Incense curls through the air. A bust of Shakespeare presides from the mantelpiece. Their most distinguished colleague, eyes closed, presses a scrap of paper to his forehead. “I’m sensing a presence,” he murmurs. “A grave, magisterial presence. Long, rolling cadences. An eighteenth-century energy.” The others lean forward. His brow furrows. “I’m seeing ruins…columns…a great empire in decline. A man is trying to communicate with us. His name begins with G. Goblin? Gaboon? Gibbles?” He begins to tremble. “Gibbon!” A gasp goes around the table. Someone faints. Somewhere in the darkness, a tambourine rattles. The medium collapses backward in his chair, exhausted by his exertions. After a respectful silence, a janitor changing the wastebasket asks, “Why don’t you just look up the passage?” He is immediately expelled for disrupting the vibrations.

Seriously, though. Just concede the indefensible territory. AI can already convincingly imitate style and voice, and it will only improve. Unless you, too, know an entire oeuvre by heart, you will be capable of being “taken in” by a pitch-perfect pastiche presented in isolation. I am by no means a Nietzsche “expert,” just an enthusiast who is familiar with most of his writing. I often see quotations attributed to him that are better credited to that prolific author Apocryphal. What do I do when I see one that doesn’t sound familiar? I look it up. I ask an LLM to verify it and link to a source. If I knew any actual Nietzsche scholars, I’d ask them. If the purported quotation names a particular book as the source, I’ll check it myself. If it doesn’t name any particular source, I’m already suspicious of it anyway. The point is, there is absolutely no need to make instant recognition the gold standard here. There is no prize for being the first to cry fraud. A paragraph may be superficially plausible while not standing up to further scrutiny. Further scrutiny should always be the standard, whether the work in question is a purported quotation of a famous author, or just the office blowhard sounding off about something he saw on Facebook.

Suppose AI eventually becomes capable of imitating Gibbon so perfectly that no scholar can reliably tell the difference. What exactly will have been lost? Gibbon’s achievement was not merely his ability to arrange subordinate clauses in a particularly recognizable fashion. It was his historical imagination, his skeptical temperament, his judgments about Rome, Christianity, and civilization, the intellectual undertaking that produced The Decline and Fall. None of that disappears because a machine can reproduce his cadences. We don’t accuse Frank Caliendo of stealing John Madden’s soul when he imitates him; we laugh and applaud precisely because the imitation is so good. If someone passes off a counterfeit passage as genuine Gibbon, that’s simple fraud, hardly a novelty in literary history. We have perfectly mundane ways of dealing with such things. But an imitation honestly presented as an imitation? Where’s the injury? Someone still has to want to make it, for whatever purpose: an argument, a joke, an experiment, a bit of harmless amusement. Someone has to care.

I’m reminded of the people who insist that without God, an immortal soul, or an afterlife, human existence would be meaningless and scarcely worth living. I’ve never understood the logic. Why should the absence of some supernatural guarantee invalidate the pleasures and satisfactions of the life I’m already living? Now I’m confronted with people who seem equally convinced that literature loses its value unless some irreducible human essence can be found lurking inside every paragraph. Once again, I’m baffled by the non sequitur. If a sentence is beautiful, an observation perceptive, an argument persuasive, why must my appreciation of it depend on discovering a ghost in the machinery?

The point of writing, surely, is to have something worth saying, not to possess a skill that nobody else can reproduce. If the latter is what we’re really defending, perhaps we should stop invoking the soul of literature and admit that we’re worried about the price of labor.