Skip to content
×

Can AI improve writing quality? First define “good,” then distinguish perceptions from evidence

AI may help improve writing, but the sources discussed here do not establish that it generally improves the quality of finished work. A better-supported claim is that AI can provide feedback and support repeated revisions. To judge whether a piece has improved, first define its purpose, then examine its strengths, wordiness, clarity, and credibility. Finding a tool helpful or choosing to keep using it does not necessarily mean an author is producing better work.

First, clarify: what should this piece accomplish?

Before comparing quality, specify whether the work is fiction, nonfiction, or content serving another purpose. A vague sense that it is “more readable” is not enough. In an interview published by Animalz in 2025, Nathan discussed the ABCD feedback method, which helps readers comment on specific passages:

  • Awesome: Which parts are worth keeping?
  • Boring: Which parts make readers lose interest?
  • Confusing: Which parts are difficult for readers to understand?
  • Didn’t believe: Which claims or plot developments do readers find unconvincing?

Nathan noted that “Didn’t believe” applies to both fiction and nonfiction, but what readers assess differs: in a story, the question is whether the plot developments are believable; in nonfiction, it is whether the arguments are convincing.

Writing better now does not mean AI deserves all the credit

Nathan felt that his annual letter was better than many pieces he had written before AI became available, but he also suggested other possible reasons: he had more writing experience, or he had more time and energy when writing the letter. This is his personal comparison of his work. It does not rule out other factors, so it cannot establish AI’s effect.

He noted that people generally do not write two versions of the same piece during the same period, one with AI and one without. That makes it difficult to know what they would have written without AI. Comparing newer and older work can still offer clues, but any interpretation must account for differences in when the pieces were written, the author’s experience, and the time invested. Improvements cannot simply be attributed to the tool.

Treat AI feedback as suggestions to consider, not a verdict on quality

Nathan described repeatedly seeking feedback and then selecting the suggestions he considered useful, rather than accepting them all. This supports a more cautious approach: ask AI to identify specific problems, then let the author judge whether the revisions serve the piece’s purpose.

Tenten’s 2024 article recommended providing clear prompts and context, working section by section, proofreading and editing, and prompting again when the results fall short of expectations. These are practical recommendations, not guarantees of effectiveness. When evaluating writing quality, explain which practices were used and show how specific passages changed, rather than simply stating that “AI was used.”

Continued use reflects perceived value, not the quality of finished work

Nathan regarded retention—whether people keep using a tool—as the primary empirical indicator of its value, provided that users had gained some experience and found it helpful. But he also acknowledged that retention does not directly measure writing quality. An author’s willingness to keep using a tool can therefore help explain how they feel about it. To determine whether the writing is clearer, more engaging, or more convincing, the work itself still needs to be assessed against each criterion.

AI and writing quality: perspectives from across the web

Should you accept all of AI's revision suggestions?

No. Nathan described repeatedly seeking feedback and then selecting useful suggestions; he estimated that he found only about one in five useful. This reflects his personal experience, not a general rule about the accuracy of suggestions. Before accepting a suggestion, you still need to judge whether it fits the piece's purpose and whether the revised passage is better.

How can you structure prompts when generating a long article?

You can use separate prompts for each section. Tenten's 2024 article recommended working section by section, specifying a word count for each part, and cleaning up the formatting when combining them. Prompts should also provide context and keep the task focused on writing. The completed draft still needs proofreading and editing, and you can prompt again if it does not meet expectations. These practices can help organize the workflow, but the article does not establish that they necessarily improve the quality of the finished work.

If an author switches to another AI tool, does that mean the original tool did not help their writing?

You cannot draw that conclusion from a switch alone. In discussing retention, Nathan noted that people who find AI helpful may continue using AI while switching to another product. When examining retention for a single product, it is therefore important to consider whether users have moved to other tools. Even continued use does not directly establish that the quality of their work has improved.

References

  1. 如何使用 Perplexity AI 撰寫文章tenten.co · 2024-10-17T15:28:09.000Z — Discusses: How can you structure prompts when generating a long article?
  2. AI as a Personal Trainer for Writers: A Conversation with Nathan Baschez of Lexwww.animalz.co · 2025-03-04T00:00:00+00:00 — Discusses: Should you accept all of AI's revision suggestions?; If an author switches to another AI tool, does that mean the original tool did not help their writing?
Keep the conversation going

Join the discussion 0 comments

What are you working through? Share a question or experience and let’s find the next step.

No comments yet. Start the conversation.

Share your thoughts

← Keep exploring
LET’S TALK

Start with your brand.

Tell me about your website and what you want to achieve.

WhatsApp tony@procrustes.info
Scan to chat on WhatsApp