ClaudeFishing and the Purity Test

How will Substack survive?

Last week, Substack released a new feature to detect AI-written text. They partnered with Pangram to let readers scan a post and see how much of it is AI-generated. I both love and hate this update, and I want to share my views with you, along with some research on how this will impact authors and Substack.

Here’s my takeaway upfront: I do not see this as an authenticity measure; it’s a purity test. This is yet another shift in a platform that’s already drifted pretty far from what it initially promised.

How We Use AI at Decode Econ

First, I believe in transparency, so let’s get to that first.

Substack now lets writers post a “How I make this” statement. But it is limited in length. So here’s a more detailed Decode Econ AI policy.

• We believe our ideas and our ability to connect dots are what we do best. Every idea is ours. The way we connect topics and help you understand the economy is ours too. We lean heavily on economic models and research to shape those ideas; it is all original content.

• We always write our own drafts. We love writing and putting our thoughts on paper. The writing process is how we shape our understanding and form our arguments.

• We do use AI, Claude and Grammarly, for editing and flow. We ask what to cut for clarity, and how to make our writing land for a non-academic audience, because our first drafts tend to read at too high a reading level. We lean on Claude for headings and titles too, but we consider that a search-optimization choice, not an authenticity one. We also try to use it for formatting consistency between posts, though honestly, that’s one of our bigger weaknesses.

• Every word we publish is written, read, and evaluated by us, and we stand by it.

Writing Confidence

Here’s the personal side of this statement.

I’ve been told my whole life that I’m a good writer and communicator, but I never quite believed it. Something about growing up between Oman and the U.S., and missing some of the basics of writing through those transitions, made me feel less confident about my skills. My writing is heavily influenced by what I read, and less by the rules of writing. I often feel like I am an imposter.

My wife, parents, friends, and my colleagues used to proofread my writing before it went out, because I didn’t trust my own drafts. Sometimes I hire editors to help me feel more confident about what I have written. The introduction of AI didn’t replace my judgment, but it gave me the confidence to trust my writing and allowed me to reduce the burden of proofreading I imposed on my family and friends.

Substack's investment in AI detection

Now, why is Substack doing this? In their announcement, they said:

“Many people complain their feeds are filled with soulless slop. Others don’t yet realize when they are reading something written by no one.”

As I read it, they’re attempting to solve for two problems: the rise of AI slop, and reader trust. Both are real concerns. However, there is a big difference between AI slop and AI-enhanced posts, and Substack AI score does not make a distinction between those two.

With that said, Substack is upfront that Pangram isn’t perfect, just that “independent research suggests it detects AI-generated text with a high degree of accuracy.” Fine.

But they also admit the actual limitation:

“Pangram can only detect whether AI was used to make the text, not whether great human care went into creating it, nor whether AI tools were used as a source.”

Anyone who’s spent time in higher education already knows this. Detection tools flag patterns, not quality, and not authorship in any meaningful sense. Moreover, there are good and bad uses of AI, and that nuance is lost in the Substack AI score.

The problem I have with this is that the score measures how much AI touched the writing and editing, not how much AI shaped the thinking, whether it was used for research, or whether a human sat with the final draft and meant every word.

The Substack measure doesn’t capture when AI is used for the following

• Ideation

• Research and sources

• Images and visuals

It strictly captures the use of AI in text.

Is This a Good Move for Substack?

So here’s where I land on the bigger picture. This feature isn’t happening in isolation. It’s the latest step in Substack moving away from what it originally promised. I joined because it offered a clean way to write a newsletter, distribute it without fighting contact management, and, most importantly, actually own my email list. Over the past several years, Substack has pushed hard toward social media mechanics: feeds, follows, discovery, and now a badge system for “real” writing.

Until recently, I considered Substack a newsletter distribution and monetization tool. If I evaluate its AI policy based on that premise, the introduction of the AI score is a horrible policy. Why should Substack care about what I distribute to my contacts? This has been Substack’s position too, as they have come under attack for hosting Nazi newsletters.

To be clear, I’m not defending Substack’s decision to host Nazi content, and I’m not equating AI-assisted writing with it. I’m pointing out an inconsistency: Substack has chosen not to police one and is now choosing to flag the other, and that choice tells you what the company actually cares about.

Substack’s co-founder, Hamish McKenzie, addressed its decision to host Nazi newsletters by saying

“I just want to make it clear that we don’t like Nazis either – we wish no one held those views,” he wrote. “But some people do hold those and other extreme views. Given that, we don’t think that censorship (including through demonetising publications) makes the problem go away – in fact, it makes it worse.”

So if harmful and extreme positions aren't something Substack manages, why is AI use getting a badge? Both are business decisions dressed up as principle. Substack is worried that readers and users care more about AI use than the writer's views. Substack has been clear that it is shifting from newsletter distribution to a social media platform with the goal of keeping you on the platform and attracting more people onto its pages. Readers have stated they have a problem with AI slop (not AI use), and Substack has responded by giving you a signal. They do not want you to associate Substack with AI use because people say they do not like it.

Do Readers Care About AI Use?

AI hate is real. The purists argue that AI lacks authenticity. A recent study examined how AI disclosure impacts reader perceptions. They analyzed 990 text evaluations and found that disclosing AI use erodes perceived trustworthiness, caring, competence, and likability of the writing and author. They attribute these negative shifts to a “perceived loss of human sincerity, diminished authorial effort, and the contextual inappropriateness of AI”.

Nakano, H., Takezawa, J., Matulic, F., Yang, C. L., & Yatani, K. (2026, March). Understanding reader perception shifts upon disclosure of AI authorship. In Proceedings of the 31st International Conference on Intelligent User Interfaces (pp. 2131-2146).

Therefore, this platform signal will cause a shift in the perception of authors who use AI.

But without the signal, can people tell when something is AI-Assisted, AI-generated, or fully human-generated? This is where it gets complicated: most people can’t tell if something is written by AI, but they can identify overused AI words and phrases, and the structure of sentences and flow that sound repetitive or mechanical. Just look at how we have treated the em dash, which, by the way, is a great punctuation mark we now all avoid to seem more authentic. Otherwise, most people aren’t great at detecting AI-generated text. According to Russell, Karpinska, and Iyyer (2025), how good you are at detecting AI is correlated with how much you yourself use AI.

Russell, J., Karpinska, M., & Iyyer, M. (2025, July). People who frequently use ChatGPT for writing tasks are accurate and robust detectors of AI-generated text. In Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) (pp. 5342-5373).

Dr. A’s Take

If you’re a writer, don’t let a purity score decide how you use AI. Decide that yourself, publish your own “how I make this,” and let readers judge the work. But know that some readers will judge you for the score, even if they had not noticed AI use in the past. If you’re a reader, don’t treat a low AI score as a proxy for quality; read the piece and decide if the person behind it actually has something to say. Do recognize that we are making a value judgment that writing with AI deserves a signal but using AI in other ways is not worthy of identification.

And if you’re on Substack because you want to distribute a newsletter, Substack has made it clear that this is no longer their goal. They are now a social media platform like Facebook, LinkedIn, or X.

To Our Readers

We are happy to share our AI policy and use with you. I hope you continue to read our work and support it the way you always have. We invest a lot of time to bring you economic news, policy changes, and help you make sense of what it means to you. We will continue to invest in pursuing that mission of helping you see the economy clearly— and your place in it.

Update

This weekend, LinkedIn introduced a new feature that lets users flag content that “seems like AI slop.” As the number of low-quality posts floods the platform, LinkedIn is stepping in to control some of the content.