Can You Trust Keyword Search Volumes Provided by ChatGPT? Check the Source and Time Period First

Precise numbers from ChatGPT should not automatically be treated as reliable keyword search volumes. A comparison published by awoo in 2023 showed substantial differences between the version described in the article—which had no internet access and a data cutoff of September 2021—and two search-volume tools. However, this historical test cannot directly represent all current versions. When interpreting the figures, you still need to check the data source, the measurement period, and whether you can verify them against the source.

A complete set of numbers does not mean the source is verifiable

In an awoo article published on May 16, 2023, Fenix Li asked ChatGPT to generate ten keywords related to “moving” (“搬家”) and list their search volumes in Taiwan. Although the response presented a complete table, the author questioned the reliability of the figures and compared some of the keywords with results from awoo 天下無狗 and Ahrefs.

The distinction here is between numbers supplied by a model and data traceable to a tool. If the source and update date of the former cannot be checked, the figures should not be treated as verified data. The latter can be checked against their source, but that does not mean they are the only correct search-volume figures.

Discrepancies in the comparison were not limited to trending keywords

The article listed the following monthly search volumes, in the order of ChatGPT, awoo 天下無狗, and Ahrefs:

• “Moving company” (“搬家公司”): 14,800, 40,500, and 25,000.
• “Moving services” (“搬家Services”): 8,100, 90, and 50.
• “Moving reviews” (“搬家評價”): 4,400, 10, and 10.

The figures for “moving company” differed by a factor of a few, while those for “moving reviews” differed by more than a hundredfold. These discrepancies cannot be generalized into a single pattern of error across all keywords. Meanwhile, the figures for “ChatGPT” in the same comparison were 1,300, 823,000, and 55,000, respectively. The author considered ChatGPT’s figures substantially lower than those of the other two tools for topics that began attracting widespread discussion around early 2023.

However, the two tools also differed from each other. This comparison supports the conclusion that the figures are inconsistent and need verification. On its own, it cannot establish which tool’s figures reflect the actual number of searches, nor can it yield a universally applicable error rate for ChatGPT.

Check three things before using the figures for marketing decisions

Given the limitations revealed by this case, you can start with the following checks when you receive a search-volume table:

1. Check the source: Which tool or report supplied the figures? Can you verify them against the original data rather than relying solely on what the response says?
2. Check the period and scope: Confirm the measurement month, region, and whether the figures represent average monthly search volumes. Do not mix figures from different periods.
3. Cross-check: Check other tools for the keywords you intend to use in decision-making. If the figures differ substantially, acknowledge the uncertainty rather than rushing to choose one as the answer.

The author cautioned that inaccurate search volumes could affect marketing strategies and goal-setting. A more prudent approach is to label unverified figures separately from sourced data, rather than letting a seemingly precise table become the direct basis for decisions.