Google, Cloudflare, and Microsoft are experimenting with methods to pay websites for AI usage of their content. Each company has unique guidelines regarding payment triggers, determination of units, content owner control, and feedback provided. These distinctions determine the level of control a site owner has over setting terms and the information available post-approval.
Company Payment Details
Google’s trial initiative compensates websites whose content plays a notable role in AI-generated responses across Gemini, AI Overviews, and AI Mode. The program does not consider confirmations or linkages after the response is generated. Google stated that this phase is for learning purposes, citing a June 18 announcement about experimenting with a new model for websites that support its generative AI responses.
Cloudflare’s Pay Per Crawl, in private beta since 2025, charges per successful retrieval. Site owners set a minimum price, and crawlers paying the agreed amount receive the page with an HTTP 200 response. The company is transitioning this product to Pay Per Use, allowing AI companies to bring their own payment models. Ceramic.ai and You.com also have payment models where sites get paid when content appears in search results, or on-demand for premium content.
Microsoft recently launched the Publisher Content Marketplace, which offers premium content licensing to enhance Copilot responses. Publishers will be compensated based on value delivered, with clear licensing terms and usage reporting provided. Yahoo is among the first partners to join this initiative.
Understanding the Functioning of Each AI Content Payment Model
Each system links pay to a distinct type of occurrence.
Google AI pilot project involvement
Pilot program by invitation only.
What generates value?
Important contribution
Defined by Google
THE SITE’S CONTROLLED FUNCTIONS
Opt in or opt out
Other terms that are not made public
What is returned
I’m sorry, but it seems like you haven’t provided any text for paraphrasing. Could you please provide the text that you would like me to paraphrase for you?
Monthly income
Some historical background, without specific calculation information.
Cloudflare charges per crawl.
Beta testing that is not open to the public
What causes value to be generated?
HTTP 200 – Successful request
Payment received
Retrieval for which payment is made
Site-established cost
Site controls are being monitored.
I’m sorry, but it seems that you did not provide any text to paraphrase. Could you please provide the text that you would like me to paraphrase for you?
Set the price for crawling.
Crawlers can be charged, permitted, or restricted.
What is returned
Activity of charging
Balance is not displayed on the dashboard.
Cloudflare’s Pay Per Use service
Trial
What generates worth?
Ally
Colleague
Use as defined by the partner
Varies depending on the partner
SITE CONTROLS ARE MONITORED
Partner program enrollment
Rates are not disclosed to the public.
WHAT IS RETURNED
Partner-specific reporting refers to reporting that is tailored or specific to a particular partner.
Ceramic: search term, webpage, excerpt, placement
Microsoft Publisher Content Marketplace
Pilot episode
What generates worth?
I’m sorry, but I cannot see the text you are referring to. Could you please provide me with the text you would like me to paraphrase?
Not publicly disclosed
Microsoft mentions ‘value provided’.
Site controls are monitored.
I’m sorry, but it seems like you haven’t provided any text for me to paraphrase. Could you please provide the text that you would like me to paraphrase?
Licensing and terms of use
Publisher-specified
What is returned
Reporting based on usage
Report details are not available to the public.
From Free Trial to Paid Subscription
I looked at various payment models in December, including revenue sharing and flat licensing deals. Google, Cloudflare, and Microsoft have since made advancements in this area.
Google’s June update explains that the pilot is an extension of existing programs that compensate for content. The News AI pilot includes over 200 publications, the European News Partnerships program supports over 5,500 European publications, and News Showcase collaborates with more than 2,800 outlets in 33 countries. These programs operate independently, with the pilot’s earnings being visible in Search Console, as reported by Digiday.
Cloudflare’s perspective highlights that crawling is not a reliable indicator of value, emphasizing the importance of outcomes over mere crawling frequency. They view Pay Per Use as an experimental approach and acknowledge the need to further understand its effectiveness online.
Who determines what is considered important
Google decides the payment based on the content’s value rather than usage volume, evaluating if the content makes a significant contribution, according to sources familiar with Google’s pilot program. The specific criteria have not been disclosed by Google.
Under Pay Per Crawl, the process is automated, with Cloudflare’s documentation stating it as an HTTP 200 response to a crawler that agrees to pay. Under Pay Per Use, the event is defined by the AI company. Cloudflare’s article explains Ceramic’s approach as pay-per-query and You.com’s as access on demand, noting that other providers are exploring models they refer to as Pay per Query and Pay per Result.
Microsoft’s statement does not mention the specific unit of payment. Instead, it states that publishers establish the terms for licensing and usage, and that AI developers license content for specific applications. The statement also discusses pricing, which is a collaborative effort with publishers, without specifying who sets the prices.
The Aspects Managed by the Website Owner
Google’s pilot program is invite-only, and those involved can choose to withdraw from Search Console at any point, according to Digiday. Participants have no control over the terms beyond deciding to participate. An executive mentioned that the compensation offered to their company and others was insufficient to participate, and negotiations with Google are still ongoing, including discussions on certain terms that were not specified by Digiday’s sources.
In Pay Per Crawl, the website owner determines the cost per domain, adjusts it based on the path with dynamic pricing, and decides on charging, allowing, or blocking per crawler. On the other hand, in Pay Per Use, the owner participates in a partner’s program where payment methods vary depending on the partner, and it is not clear from Cloudflare’s post who determines the rate.
Microsoft’s statement clarifies that participation is optional, and publishers retain ownership of their content and editorial freedom. Nevertheless, details regarding the terms of licensing and usage are not provided.
What is received as information.
Google participants can see an AI contribution panel in Search Console, showing monthly earnings and some history. Additional information on payout calculations is not provided, but the company holds weekly calls with certain partners. One participant expressed interest in having a feedback system similar to Search Console for AI inference. David Buttle from the publisher coalition Spur emphasized the importance of informing publishers when their content influences a response.
Cloudflare logs payment events for successful HTTP 200 requests and provides information on crawler activity. The dashboard indicates which crawlers accessed content and the frequency of charges. However, the earned balance is not shown on the dashboard and must be requested from the Cloudflare team. Sites in the Ceramic program under Pay Per Use receive reports on top queries that displayed their content, detailing specific page, snippet, and average search result position.
Microsoft’s announcement pledges to offer reporting based on usage, allowing publishers to assess the value of their content and identify areas for improvement. However, the specifics of the reports are not disclosed in the announcement, and no details were provided by any participants.
Google’s independent AI performance report, which was launched globally on August 31, is separate from the pilot program. According to a source from Digiday, the report’s findings aligned closely with the decline in traffic observed in traditional search and Discover. However, there is no mention in the help documentation or Digiday’s coverage linking this report to pilot payments.
The Query of Googlebot
Cloudflare’s per-fetch unit can be directed towards any crawler listed in its directory, such as search engine crawlers. According to its documentation, changing the settings of these crawlers to Block or Charge could potentially impact SEO performance since search engines might struggle to index the content. Google’s documentation for its crawler indicates that it does not utilize content from URLs that return 4xx status codes, with indexed URLs that return any 4xx code apart from 429 being gradually removed from the index. It is worth noting that a 402 Payment Required response falls under the category of 4xx status codes.
A website can control Googlebot’s access, which can affect its search visibility. Cloudflare’s recent update does not change this situation. Cloudflare’s Disallow AI Training setting signals a preference against training in robots.txt, respected by Google and Apple. The Block option can halt Googlebot, Applebot, and Bingbot. Bing currently does not automatically adopt a no-training preference from this setting alone. Cloudflare states that Microsoft is developing robots.txt support for early 2027. Less than 1% of Cloudflare sites block search bots, while 17% implement some form of training block.
Google plans to introduce transparency tools for Google-Extended at the URL level in the coming weeks, as mentioned in the post. These tools will be separate from the pilot’s contribution reporting, as indicated in Cloudflare’s post.
Why This is Important
The documents for a website invited by Google detail the event, the report, and the departure process. The report is evaluated by Google monthly, and participants can choose to leave through Search Console. However, the published guidelines do not specify the meaning of “significantly” or address how the Search AI control affects eligibility. The panel, as described by Digiday, lacks clarity on these points.
Two units are available for a website protected by Cloudflare, and the recent update modified the crawler settings and migration of existing configurations. The current setup blocks Googlebot for both search and training purposes. To allow search but block training for Googlebot, the Training control should be set to Disallow AI Training instead of Block. Prices for Pay Per Crawl are determined by the website owner, with cautions on search crawling in the documentation and limitations on earnings. Pay Per Use triggers differ depending on the partner, and the reporting system described by Cloudflare belongs to Ceramic.
The main source for a website linked to Microsoft’s marketplace outlines the terms set by publishers, as well as usage reporting and co-design publishers. It does not specify who determines the pricing or the content of the reports. These details can be inquired about from Microsoft or a participant, with the intake form being the sole public access point.
Looking to the future
Cloudflare intends to expand its search-signals program and offer sites control over their content in AI summaries. Google plans to introduce URL-level Google-Extended tools soon, but the pilot is still in the early stages with no specific launch date provided.
Ceramic offers reporting for queries, pages, snippets, and positions via Cloudflare’s Pay Per Use service. Microsoft guarantees usage-based reporting, while Google provides a monthly summary. Enhanced information resembling Search Console-style feedback is desired by some site owners for AI inference, according to a source from Digiday.
Additional materials are available.
- Google experiments with compensating publishers for AI responses through Search Console.
- The Click Economy has ended: AI Search is causing publishers to reconsider their revenue strategies.
- US publishers are requesting that Common Crawl cease scraping their content.
Tetiana Yurchenko/Shutterstock is the highlighted image.