For almost thirty years, robots.txt could express exactly two ideas: crawl this, do not crawl that. The AI era broke that binary. A publisher might happily appear in search results, accept being cited in AI answers, and still refuse to feed model training. Classic robots.txt has no vocabulary for that distinction. Content Signals is the attempt to add one, and because it shipped as a default on one of the largest networks on the internet, there is a good chance your website is already broadcasting these signals without you knowing.

What Content Signals Actually Are

Content Signals is a policy extension for robots.txt introduced by Cloudflare in late 2025. It adds a single new line type that declares how the content of your site may be used after it is accessed. The line distinguishes three separate uses:

Content-Signal: search=yes, ai-input=no, ai-train=no

search covers the classic case: building a search index and showing your pages as results with links. Saying yes here keeps normal Google and Bing behavior working.

ai-input covers retrieval for AI answers: a system fetching your content as grounding material for a generated response, the way answer engines cite sources. This is the signal that controls whether you want to appear inside AI generated answers.

ai-train covers model training: using your content to train or fine tune the models themselves.

Each signal is independent. The most popular combination among publishers is exactly the one above: stay searchable, and make a separate decision about answers and training.

Why Your Site Might Already Have It

Cloudflare did not just publish a specification. It enabled a managed robots.txt feature that injects a Content Signals policy block into the robots.txt of zones that opt in, and parts of it became default behavior for new domains. The result is that millions of robots.txt files today contain a policy block their owners never wrote. The served file and the file sitting on your origin server can be two different documents, a phenomenon we documented in detail in our article on CDN level AI bot blocking.

Checking takes ten seconds: open yoursite.com/robots.txt in a browser and look for a Content-Signal line or a comment block mentioning content signals. If it is there and you never added it, your CDN is speaking on your behalf.

A Signal, Not a Wall

The most important thing to understand about Content Signals is what they are not. They are not enforcement. A signal line in robots.txt stops nobody by itself, exactly like a Disallow rule stops nobody by itself. Compliance is voluntary, and as of mid 2026 the major AI companies have not publicly committed to honoring the ai-input and ai-train signals.

So why bother? Three reasons. First, the signal documents your intent in a machine readable, timestamped, public way, which matters in a period when the legal framework around AI training is being actively fought over. Second, well behaved crawlers increasingly parse the extended syntax, and adoption tends to follow the largest networks. Third, the signal costs one line. Low cost, real upside, no downside is a rare combination in technical SEO.

For actual enforcement, the stack remains what it always was: robots.txt directives for the polite bots, and firewall or CDN rules for the rest. Our complete list of AI crawlers explains which bots respect what.

How to Set Your Own Policy

Step 1: Decide your three answers. The framework forces a useful clarity. Do you want search visibility? Almost always yes. Do you want to be citable inside AI answers? For most businesses yes, because citations bring qualified traffic. Do you want to feed model training? This is the genuinely contested one, and reasonable owners land on both sides.

Step 2: Add the block to robots.txt. The signal line sits alongside your normal rules:

User-agent: *
Allow: /

Content-Signal: search=yes, ai-input=yes, ai-train=no

Sitemap: https://example.com/sitemap.xml

Step 3: Align your bot rules with your signals. A policy that says ai-train=no while robots.txt still allows GPTBot and CCBot everywhere sends mixed messages. If you opt out of training, also disallow the pure training crawlers, and consider firewall rules for the ones with a history of ignoring robots.txt.

Step 4: Check what is actually being served. If you are behind a CDN with managed robots.txt, your edits may be merged with or overridden by the managed policy. Always verify the live file, not the one on disk.

The Honest Outlook

Content Signals might become the standard grammar for content permissions, or it might be remembered as one proposal among several in the great robots.txt reform era. Competing efforts exist, and standards bodies are still debating the final shape. What seems certain is the direction: the binary crawl or no crawl model is finished, and some machine readable way to separate search, answers and training will win. Adopting the current syntax costs you a line of text and puts your preferences on the record today.

The CheckMy.site scanner reads your live robots.txt, detects Content Signal policies including CDN injected ones, tests real bot access with live fetches, and shows you the complete picture of what machines are being told about your content, all within its 158 point report. One scan tells you whether the signals your site broadcasts are actually the ones you chose.