AI can produce a 2,000-word blog post in seconds.
It can also produce 2,000 of them.
That is exactly what makes the recent 2,000 AI Blogs Experiment from Otterly.ai so interesting for marketers and SEOs. Rather than debating whether AI-generated content is “good” or “bad,” the experiment put large-scale AI content to the test: What happens when thousands of AI-generated articles are published across new websites, and how do traditional search engines and AI platforms respond?
The answer should make any marketer think twice before turning an AI content generator loose on an important domain.
The experiment found that both test sites initially gained visibility. But that success did not last. One site lost its Google visibility after about two weeks, while the other continued to receive impressions for roughly three months before experiencing a dramatic collapse. The sites were subsequently deindexed by Google, according to the experiment’s reporting.
At the same time, the content produced very different results across AI platforms. Some platforms crawled the sites extensively but produced few or no citations, while others generated substantial citation activity.
So, should you use only AI-generated content on your domain to increase AI visibility?
Our answer is no.
But that doesn’t mean you should avoid AI.
The more useful lesson is that AI should be part of your content and local SEO strategy—not the strategy itself.
The Problem Isn’t Simply “AI Content”
It is tempting to look at the experiment and conclude that Google is simply penalizing AI-generated content.
That’s not really the lesson.
The more important variable is how the content was created and what value it provides.
There is an enormous difference between:
“Use AI to write 1,000 articles and publish them.”
and:
“Use AI to help research, organize, analyze, and draft content that contains original expertise and value.”
The first approach treats content as a commodity. The second treats AI as a productivity tool.
That distinction matters because search engines and AI systems have a difficult job: They need to decide which information is useful enough to surface, summarize, or cite.
Simply producing more content doesn’t necessarily make your website more valuable.
In fact, the experiment suggests that scaling low-value content can put the entire domain at risk.
Lesson 1: Don’t Experiment With Your Most Valuable Domain
The first recommendation is perhaps the simplest:
Don’t use your primary business website as the laboratory for an untested scaled-content strategy.
The Otterly.ai experiment used two new domains and published approximately 1,000 AI-generated articles on each. Both sites initially gained visibility, demonstrating that large quantities of AI-generated content can be discovered and indexed.
But that initial success proved temporary. One site lost its Google visibility after approximately two weeks. The other appeared to perform successfully for months before experiencing a sudden collapse.
For a marketer, this is an important reminder:
Initial indexing is not the same thing as sustainable search visibility.
A new content program might look successful in Google Search Console during its first few weeks. That doesn’t necessarily mean the strategy is working.
Watch what happens over time.
If you are experimenting with AI-assisted content, establish a baseline for:
- Organic impressions
- Organic clicks
- Conversions
- AI citations and mentions
Then monitor those metrics consistently.
Lesson 2: Don’t Confuse Early Growth With Long-Term Success
One of the most interesting parts of the experiment was the timing.
The test sites didn’t immediately disappear from Google.
Instead, they initially received impressions and demonstrated measurable visibility before one site collapsed after about two weeks and the other after roughly three months.
That’s an important lesson for SEO reporting.
A content strategy shouldn’t be judged solely on the first few weeks of performance.
This is particularly important when you’re experimenting with something as new as AI-generated content.
Don’t ask only:
“Did Google index it?”
Also ask:
“Did the visibility persist?”
“Did AI search platforms cite it?”
“Did people click?”
“Did the content attract qualified visitors?”
“Did it generate conversions?”
Those are much more meaningful measures of success.
Lesson 3: AI Visibility Is Not One Number
Here’s where the experiment gets especially interesting for marketers working on Generative Engine Optimization (GEO) and AI visibility.
The same content did not perform consistently across AI platforms.
The experiment monitored platforms including ChatGPT, Microsoft Copilot, Perplexity, and Google AI Overviews/Gemini. The results varied significantly.
For example, the experiment reported approximately an 8% citation share on Microsoft Copilot while seeing no citations from ChatGPT, Gemini, or Google AI Overviews during the measured period. It also reported more than 1,000 citations from Perplexity despite substantially fewer crawler visits than ChatGPT.
The takeaway isn’t that one platform is “better.”
It’s that AI visibility Optimization is platform-specific.
If you’re reporting visibility to a client, an aggregated score can hide important differences. A better report might include AI citations, as well, for example:
| Platform | What to Measure |
|---|---|
| Google AI Overviews | Mentions and citations |
| ChatGPT | Brand mentions and cited URLs |
| Microsoft Copilot | Mentions and citations |
| Perplexity | Citations and source inclusion |
| Gemini | Mentions and citations |
This gives marketers a much clearer picture of where their content is actually being discovered and referenced.
Lesson 4: A Crawler Visit Isn’t a Citation
This may be one of the most important lessons from the experiment.
AI crawlers visiting your website can be exciting. After all, it means an AI system is discovering your content.
But discovery isn’t the same as citation.
According to the experiment, ChatGPT’s crawler visited the test sites more than 5,000 times, yet the test content generated zero ChatGPT citations during the measurement period. Perplexity, by contrast, generated more than 1,000 citations from a much smaller number of crawls.
That difference illustrates why marketers need to be careful about what they call a success metric.
A crawler hit tells you:
“Something looked at this page.”
A citation tells you:
“This content was considered useful enough to reference.”
Those are very different outcomes.
For AI visibility reporting, prioritize mentions, citations, source inclusion, and referral traffic over raw crawler activity.
Lesson 5: Original Value Still Matters
If there is one principle marketers should take away from this experiment, it is this:
Don’t use AI to manufacture more content. Use AI to help you create better content.
AI models are exceptionally good at summarizing information that already exists.
That creates a problem when your entire content strategy consists of asking an AI system to summarize what other websites have already published.
Why would an AI search system need to cite your version? This is where marketers can create an advantage. Add information that AI cannot simply recreate from generic web content:
Proprietary data
Research, customer surveys, campaign results, sales data, and original analysis.
First-hand experience
Examples from your customers, projects you’ve managed, problems you’ve solved, and lessons you’ve learned.
Expert perspective
Don’t simply explain what everyone else says. Explain why you agree, disagree, or approach the problem differently.
Unique analysis
Take existing information and add your own interpretation.
This is the difference between AI-generated content and AI-assisted expertise.
So, Should You Use AI-Generated Content?
Absolutely.
Just don’t make it your entire content strategy.
AI can be incredibly useful for marketers. It can help with:
- Topic discovery
- Keyword clustering
- Content outlines
- Internal linking
- Redirect mapping
- Content gap analysis
- Repurposing existing content
But the final product should still provide something worth finding.
Think of AI as an exceptionally capable marketing assistant—not as a replacement for your organization’s expertise.
That distinction becomes even more important as AI search evolves.
If Google’s AI Overviews, Open AI’s ChatGPT, Perplexity, and other systems are deciding which sources to cite, producing another generic article may not give them a reason to choose you.
Giving them something useful, authoritative, and different might.
The Bigger Opportunity: Measure Before You Scale
The 2,000 AI Blogs Experiment also highlights a broader change in SEO.
For years, marketers primarily measured:
Rankings → Traffic → Conversions
Now we’re adding another layer:
AI Visibility → Mentions → Citations → Referral Traffic
That doesn’t mean traditional SEO metrics are obsolete.
It means marketers need a broader view of visibility.
Before launching hundreds of AI-generated pages, establish a baseline. Then measure what happens across Google and the major AI platforms. Traditional search tools, such as Google Search Console and Bing Webmaster, now provide AI visibility reports.
And don’t assume that success in one environment automatically transfers to another.
The experiment demonstrated just how differently platforms can treat the same content.
The Black Kite Marketing Takeaway
The temptation of AI-generated content is understandable.
Why spend weeks producing content when AI can produce hundreds of articles in a day?
Because content volume isn’t the same thing as content value.
The future of search isn’t necessarily about who can publish the most pages. It’s about who can provide information worth discovering, ranking, summarizing, and citing.
Use AI to:
- make your marketing team more efficient
- analyze your content
- uncover opportunities.
- accelerate production.
Don’t outsource your expertise to an LLM.
AI-assisted content with original expertise is a much more compelling strategy than an entire website filled with AI-generated articles.
And before you scale an AI content strategy, measure what is actually happening.
At Black Kite Marketing, we help marketers navigate this transition—from traditional SEO measurement to the emerging world of AI visibility. We can help you determine what to measure, identify opportunities to improve your visibility, or outsource the measurement and reporting process entirely.
Wondering how visible your brand is across Google and AI search? Reach out to Black Kite Marketing. Let’s measure it before you scale it.