Jarrett Sidaway
CEO & Co-Founder, FetchRight
If you asked most publishers who their largest audiences are, they would instinctively think in terms of human segments: subscribers, casual visitors, search-driven readers, social traffic, direct loyalty. Very few would answer with the truth that is rapidly emerging: their biggest and most consistent "readers" are no longer human at all.
They are AI systems.
Crawlers, answer engines, and retrieval models are continually scanning, ingesting, and interpreting publisher content at a scale and cadence that no human audience can match. These systems draw on news, analysis, reviews, explainers, and evergreen reference material to power everything from chat assistants to enterprise tools to consumer-facing AI products.
And yet, despite this enormous and growing dependency, most publishers have almost no visibility into this interaction. They do not know which AI systems rely most heavily on their content, how often it is accessed, or what portions are being reused in answers and recommendations. They are fuelling the AI economy, but largely blind to the scope and nature of that contribution.
That gap between usage and visibility is the core strategic problem of the AI era for publishers. Closing it is the first step toward something more important: not just seeing what AI is doing with your content, but deciding how it should.
In the legacy web model, publishers could see most of what mattered. Analytics tools told them who arrived at their pages, how those users got there, what they read, and where they dropped off. Even if the signal was imperfect, it was at least legible. Traffic was something you could observe, model, and optimize.
AI systems operate differently. They may hit your domain from a changing pool of IPs. They may go through proxies, aggregators, or third-party scrapers. They may access your content for very different reasons: some are training models, others are building retrieval indexes, others are constructing vertical search engines or domain-specific copilots.
From the publisher's perspective, this can all look like generic bot traffic. In log files, the most important readers of your content are effectively indistinguishable from low-value crawlers. They come, they extract, and they leave without any reliable mechanism for you to understand the value of their visits, the purposes they serve, or the downstream products they power.
This is not simply a technology nuisance. It is a business blind spot. When the entities consuming your content most intensively are also the least understood, you lose the ability to align your editorial, commercial, and brand strategies with how your work is actually being used.
For years, publishers have relied on a handful of mechanisms to manage automated access to their sites. Robots.txt offered a coarse-grained way to signal what should or should not be crawled. Rate limits and firewalls tried to keep abusive or overly aggressive bots at bay. Terms of service and legal agreements provided a framework for negotiated use in specific situations.
In the AI era, these tools are showing their limits.
Robots.txt was designed in a context where search engines were the dominant crawler and the primary use of content was indexing links. It does not express nuanced use cases, such as "this content may be used for real-time question answering, but not for model training," or "this path can be summarized, but only with explicit citation and limited caching."
Logs and security tools, meanwhile, can tell you that traffic occurred but not what kind of value it created. They cannot distinguish between a bot that scraped a page once and a system that repeatedly returns to your domain to feed a high-traffic, revenue-generating AI product.
The result is a structural mismatch. Publishers are trying to manage a new class of interaction with old tools, and the outcome is predictable: lack of control, lack of insight, and a growing sense that AI is "doing things with our content" without any meaningful governance.
Faced with this asymmetry, many publishers have reached for the bluntest tool they have: blocking. They are tightening access at the firewall, turning away bots, or broadly disallowing AI-associated user agents. In some contexts, particularly where abuse is clear, this is an understandable response. It protects short-term interests and sends a message that unmanaged exploitation is unacceptable.
But blocking alone is not a defensible long-term strategy.
Audiences are already moving into AI discovery channels. They are asking questions of assistants, copilots, and answer engines that publishers are uniquely qualified to address. If publishers remove themselves entirely from these channels, they may win a measure of tactical privacy but lose strategic relevance. The conversation will continue without them, fuelled by lower-quality sources or legacy caches.
The more durable strategy is not to retreat, but to manage participation on your terms. That means seeing AI access in detail, deciding what is acceptable, and creating structured pathways that let you support high-value use while restricting or monetizing others.
In other words, the goal is to move from blocking to governance.
If governance is the goal, where should it live?
The answer is not in the CMS, nor in the ad stack, nor in a patchwork of page-level scripts. Those tools were built for human-facing experiences, not for machine-facing access. By the time an AI crawler reaches the application layer, it has often already retrieved what it came for.
The only defensible place to manage AI access systematically is at the edge of your infrastructure, where all traffic enters your domain through your content delivery network (CDN). At the CDN layer, you see the raw request before it has been transformed by downstream logic. You can fingerprint bots, enforce rules, and make decisions about what to serve.
More importantly, you can begin to differentiate.
You can treat an AI system that is willing to play by structured licensing rules differently from a scraper that ignores your policies. You can serve one a structured preview that invites deeper, licensed interaction, while throttling or denying access to the other. You can log and audit what is happening in a consistent way, across all of your properties, without rewriting your existing content systems.
This is where FetchRight operates: at the publisher's edge, translating abstract policy into concrete enforcement, and transforming undifferentiated bot traffic into a governable interaction between publishers and AI platforms.
Visibility is not an end in itself. It is the foundation for strategy.
When you can see which AI systems are accessing your content, how frequently they are doing so, and what they are trying to retrieve, you can begin to make informed decisions. You can decide which relationships you want to strengthen, which you want to regulate, and which you want to discourage.
You can also begin to align your investments with actual usage. If you know that certain parts of your archive are particularly valuable to AI systems, you can prioritize structuring, enriching, or summarizing those areas. If you see a pattern of repeated access around specific topics, you can consider licensing frameworks that match that demand. If you discover models relying on your content without proper attribution or terms, you can either pull them into a structured relationship or tighten controls.
In other words, once AI reading becomes visible, it becomes manageable. And once it is manageable, it becomes monetizable.
The same infrastructure that gives you visibility and enforcement can underpin new economics.
On the revenue side, structured licensing allows you to define different tiers of access. Limited, low-frequency retrieval to power occasional answers may be permitted under basic terms. Higher-intensity use for enterprise tools, training, or domain-specific copilots may carry licensing fees that reflect actual value. Instead of arguing abstractly about the "worth" of content, you can have concrete discussions based on observed usage patterns.
On the risk side, governed access reduces exposure. When AI platforms obtain content under clear, enforced terms, they face less uncertainty. They are less likely to rely on questionable intermediaries or to ingest data with unclear provenance. This makes them more receptive to structured relationships and more inclined to view publishers as partners rather than obstacles.
For the publisher, this combination of revenue opportunity and risk mitigation is powerful. It turns AI from an uncontrollable force at the edge of the business into a channel that can be managed with the same discipline you apply to other parts of the operation.
None of this happens by accident. Regaining control of how AI uses your content is not solely a technical project. It is a leadership issue that requires alignment at the top of the organization.
Editorial leaders need to be clear about which uses of their content align with their mission and which do not. Commercial leaders need to design pricing and partnership frameworks that reflect the value of structured, licensed access. Legal and policy teams need to codify acceptable use in ways that can be implemented at the edge. Technology teams need to integrate enforcement mechanisms without disrupting existing workflows.
What ties all of this together is a simple shift in perspective: from viewing AI as an external threat to viewing it as a new class of user that must be governed like any other. You would not allow a human partner to reuse your content at scale without an understanding of purpose, terms, and compensation. The same logic now has to apply to machine partners.
The most important readers of your content are no longer the ones you see in your browser analytics. They are the invisible super-readers behind AI systems that shape how people ask questions, form opinions, and make decisions. Ignoring them does not make them go away. Blocking them blindly does not answer the strategic question of how your expertise will matter in the AI-driven future.
The path forward begins with visibility at the edge: seeing who is accessing what, under which conditions, and for what implied purposes. From that foundation, you can assert control, structure relationships, define licensing, and ultimately turn AI from an opaque risk into a governed, value-generating part of your business.
Publishers who seize that opportunity will not only protect their rights. They will shape how their journalism and expertise live inside the systems that increasingly mediate public understanding. In an era where AI is reading more than any human ever could, the publishers who decide how that reading happens will decide how their brands and businesses evolve.
The audience has migrated to AI-driven discovery channels. Publishers who structure and govern what they contribute can become essential partners in the AI economy.
AI upends the logic of pages as the atomic unit of value. Publishers can allow their content to be atomized without input, or they can define how their expertise is represented in the new answer layer.
When discovery moves from pages to answers, the value of expertise increases. Publishers have an opportunity not just to participate in the AI ecosystem, but to shape its trajectory.