How to spot AI writing

[mklabgr]

The widespread use of artificial intelligence (AI) has transformed written communication, with large language models (LLMs) increasingly producing emails, essays, scientific papers, journalism and other forms of content. Although AI can imitate different literary styles and generate large quantities of text rapidly, its writing can exhibit distinctive linguistic patterns. Commonly suggested indicators include em dashes, sophisticated vocabulary and expressions such as “delve” and “rich tapestry.” However, the article stresses that there is no single linguistic feature that reliably proves that a text was written by AI. Identifying AI-generated prose therefore requires examining several characteristics together rather than focusing on individual words or punctuation marks.

To investigate these characteristics, The Economist compared human-written articles with texts generated by ChatGPT, Claude, Gemini and Grok. The models were asked to reproduce versions of the magazine’s articles without access to the internet, using AI-generated summaries as prompts. The resulting dataset contained 55,940 sentences and 1.2 million words. The researchers also compared the AI-generated material with journalism from CNN, The New York Times and The Washington Post, as well as bestselling fiction published between 1950 and 2022. The analysis found differences between AI and human writing in vocabulary, punctuation, sentence length and paragraph structure. At the same time, these characteristics can change considerably when AI models are updated.

One notable characteristic is vocabulary. Contrary to the popular belief that AI consistently overuses words such as “delve” and “tapestry,” contemporary models often favour longer and more sophisticated words, including “significant,” “increasingly” and “consequences,” as well as technical terminology such as “methodology” and “interdependence.” AI-generated texts also make frequent use of nominalisations, turning verbs into nouns, and words with Latin-derived suffixes. The article also examines punctuation and challenges the idea that AI universally overuses em dashes. ChatGPT's newer versions use fewer dashes, whereas Claude continues to use them relatively frequently. More persistent differences include the comparatively limited use of commas, semicolons and parentheses, alongside longer sentences and fewer quotations from experts.

The article identifies further patterns in sentence structure and rhetoric. AI-generated prose tends to contain longer sentences, less variation in sentence length and fewer short, emphatic statements. Models also frequently use rhetorical constructions such as “not X but Y,” “not only but also,” and the rule of three, in which three related ideas are grouped together for emphasis. ChatGPT and Claude use these constructions particularly often compared with human writing. AI-detection tools can help identify machine-generated text, but they are imperfect and can produce false positives. The article's central conclusion is that AI writing may display a combination of formulaic structures, sophisticated vocabulary and limited stylistic variation, but these characteristics are changing rapidly. As AI models become increasingly capable of producing human-like prose, reliably distinguishing AI-generated writing from human writing is becoming more difficult.

Key Takeaways

1. AI writing is increasingly widespread, appearing in journalism, education, science and everyday communication.
   
2. No single feature reliably identifies AI writing; several linguistic characteristics need to be considered together.
   
3. Sophisticated vocabulary and technical terminology are common characteristics of AI-generated prose.
   
4. Nominalisations and longer, more formal words can occur more frequently in AI-generated writing.
   
5. Punctuation patterns vary between AI models, so the presence of em dashes alone is not a reliable indicator.
   
6. Longer sentences and limited variation in sentence length are recurring characteristics of AI-generated prose.
   
7. AI models frequently employ rhetorical patterns such as “not X but Y” and the rule of three.
   
8. AI-detection software is imperfect and can produce false positives.
   
9. AI writing changes as models are updated, meaning that detection techniques based on older models may become less effective.
   
10. The increasing ability of AI systems to produce human-like prose makes reliable identification progressively more challenging.
    

ARTICLE