What can content testing evaluate, and how should it be conducted?
Content testing can evaluate whether readers understand information, perceive the intended tone, and need explanations of particular terms. Feedback can also help improve content production processes and standards. To understand how users research topics, compare options, and make decisions, you can use moderated usability testing to observe their reading and ask follow-up questions. To compare how different versions affect user behavior, you can use A/B testing. These methods answer different questions, so establish your purpose before choosing an approach.
Break “Is the content good?” down into questions you can test
In its 2021 article on content testing, NN/g identifies comprehension, tone, and terminology as key areas to evaluate. Go beyond asking whether participants like the material by asking them to explain:
- Comprehension: Can they accurately summarize what they have just read in their own words?
- Terminology: What does a particular technical term mean to them, and does it need further explanation?
- Tone: If someone said these words to them, what kind of person would that be? This helps reveal the impression the writing creates.
Being able to paraphrase easily and accurately is a good sign of understanding. If participants have to go back and read the text word for word, they may not yet understand it. Finishing the text alone does not establish that the content is effective.
Recruit the right participants, and make sure researchers understand the material too
Participants need relevant knowledge or firsthand familiarity with the situation the content addresses. This is especially important when testing scientific, medical, and technical information. NN/g gives the example of testing an article about adult non-Hodgkin lymphoma: participants should be people diagnosed with the condition or their primary caregivers. You cannot expect people unfamiliar with that situation to stand in for the intended readers and assess the content’s value after a quick read of background information.
Researchers should also be familiar with the text being tested and its subject area. They should first understand the type of content and its intended readers, and consult its creators and subject-matter experts. They do not need to become experts themselves, but they should know enough about what participants are reading to understand their difficulties and ask follow-up questions.
Leave room for quiet reading during moderated sessions
NN/g recommends moderated testing when studying how people research topics, compare options, and make decisions. Moderators can provide appropriate feedback to prevent participants from becoming uncomfortable with prolonged silence and rushing to finish. This does not mean continually asking them to speak.
- Allow extended periods of quiet reading without appearing impatient or interrupting frequently.
- Keep any necessary questions during reading neutral, such as “What are you looking for?” After participants answer, let them continue reading.
- Once they finish reading, follow up on their reactions, for example: “You paused at this passage earlier. What were you thinking at the time?”
When speaking with participants, say “this information” or “this article.” Avoid the broad term “content,” which ordinary readers may understand differently from content professionals.
Decide which behaviors to measure before comparing versions
Google’s guidance on website testing explains that A/B testing can compare two or more versions of a single change and track user behavior, such as the number of clicks on different button versions. Multivariate testing examines multiple changes and their possible interactions simultaneously.
When planning, clarify your purpose: do you want to find out which version generates more interaction, or identify information that is difficult to understand? The first focuses on user behavior. For the second, you can observe participants as they read, ask them to paraphrase, and follow up with questions.
Content testing: perspectives from across the web
When is multivariate testing appropriate?
Consider multivariate testing if you want to understand the combined effects of multiple changes, such as headlines, images, or calls to action. For example, the same headline might work well with one image but not another. However, this type of testing involves more complex analysis and requires substantial traffic to produce accurate results. It may not be suitable for websites with limited resources or traffic.
What kinds of websites are suited to SEO content testing, and how should the results be interpreted?
If a website is stable and its traffic is relatively consistent, consider testing content changes before a full rollout to give decisions a stronger basis in data. However, fluctuations in search rankings, seasonality, and algorithm updates can all affect results. Keywords, content, and competition also vary across pages. So while SEO testing is feasible, it remains difficult to fully rule out other variables when interpreting the results.
How long do tests usually need to run?
There is no fixed number of days that works for every situation. SEO tests may run from a few days to several months, depending on factors such as traffic and search volume. Testing low-search-volume keywords may require more time to gather enough data. Before starting, choose pages and methods that fit your goals and resources, and allow time to analyze the results rather than drawing conclusions too early.
Can clicks and traffic alone explain why readers respond as they do?
Clicks and traffic can show what happened, but they do not necessarily explain how readers felt or what difficulties they encountered. One author argues that although A/B or multivariate testing can be used to investigate causes, quantitative experiments have limited efficiency and accuracy when used to understand visitors’ feelings. If the questions concern terminology, comprehension, or impressions of tone, participant feedback and follow-up questions can provide additional insight.
Can you test whether content is more likely to be used in AI answers?
You can develop hypotheses to test, but there are limits to what the results can tell you. Tests can explore which AI conversations mention the content, where it appears, or how appealing it is. However, AI conversations do not have the same concept of rankings as traditional search results. For example, whether adding a concise feature summary to a product page increases citations and qualified clicks remains an untested hypothesis. If a model does not use external search, the knowledge it acquired during training is updated slowly, and it is almost impossible to isolate the effect of a single website change through statistical testing.
References
- 針對搜尋進行 A/B 測試的最佳做法 | Google 搜尋中心 | Documentation | Google for Developersdevelopers.google.com — Source used in the article above.
- How to Test Content with Userswww.nngroup.com · 2021-02-07T17:00:00+0000 — Discusses: Can clicks and traffic alone explain why readers respond as they do?
- SEO Testing: What It Is & How to Test SEO in 2025 (+ 6 Ideas)www.semrush.com · 2024-03-21T10:44:00Z — Discusses: When is multivariate testing appropriate?; What kinds of websites are suited to SEO content testing, and how should the results be interpreted?
- Generative Engine Optimization (GEO): Stop Guessing and Start Testingwww.searchpilot.com · 2025-09-26T14:51:14.000Z — Discusses: Can you test whether content is more likely to be used in AI answers?
- 那些 Google Analytics做不到的事,談質性研究與量化研究www.yesharris.com · 2019-04-02T01:22:23+08:00 — Discusses: Can clicks and traffic alone explain why readers respond as they do?
- Content Strategy: Study Guidewww.nngroup.com · 2024-08-23T17:00:00+0000 — Source used in the article above.


Join the discussion 0 comments
What are you working through? Share a question or experience and let’s find the next step.
No comments yet. Start the conversation.