Amazon Blocked Meta’s Muse, And Robots.txt Had Nothing To Say
Any website can write Amazon’s block-page sentence into its own terms. Spotting an agent that looks like a shopper is the hard part.
Slobodan “Sani” Manić is a website optimisation consultant with over 15 years of experience helping businesses make their websites faster, more usable, and more accessible for humans and the AI agents increasingly acting on their behalf. He takes an uncompromising approach to technical audits, addressing every issue rather than just the critical ones.
Slobodan is the creator of Machine-First Architecture, a methodology for building websites that work for machines first and humans always. The framework spans four pillars, identity, structure, content, and interaction, and reframes web design around what AI systems need to discover, understand, and transact with a brand.
Since 2021, he has hosted the No Hacks podcast, exploring how AI systems like ChatGPT, Claude, and Perplexity are reshaping the web. He works with a small number of companies through the No Hacks Advisory, starting with a fixed-scope Machine-First Architecture audit. He is also the co-founder of Web Performance Tools, Inc. Earlier in his career, Slobodan served as CTO of Search Engine Journal and worked as an e-commerce CRO strategist and Klarna integration specialist in the Nordics.
Any website can write Amazon’s block-page sentence into its own terms. Spotting an agent that looks like a shopper is the hard part.
Strip the visual layer and a machine can still use a website. Strip the structure, as markdown does, and it can only read about it.
Visa, Mastercard, PayPal, and Stripe name no live merchant. Google names three. Here is what the agentic commerce documentation still refuses to say.
Bot Preference Sync sets AI crawler policy per category, not per crawler. Allow GPTBot and block Bytespider, and no setting describes what you do.
Meta published two Muse documents on the same day. One never mentions risk, attack, or prompt injection. The other uses them 39 times.
The machine-access fight is aimed at Google. Meta reads more, returns almost nothing, and no one is asking it to stop.
One company measures the bot problem, frames it, and sells the fix. Cloudflare now owns both the meter and the valve. Check your own logs.
Cloudflare announced wallets and optional identity handles for AI agents on August 4, extending paid access past crawlers to any caller and any resource.
Clarity now splits AI citation queries into branded and non-branded. The two mean different things, and only one counts as discovery.
AI search only feels like a new sport to the people who were never really training. Here’s what actually changed, and what didn’t.
If you run CRO or SEO, the oldest mistake in your discipline just came back wearing a new referrer string. Here’s the fix.
No agent reads your OKF bundle. Write one anyway: v0.2’s provenance and staleness fields expose every claim your website cannot defend.
Cloudflare and three rival browsers agreed on one protocol. That’s rare. It still only solves half of the agentic access problem.
Google’s DMCA case against scraper SerpApi collapsed because SearchGuard protects ad revenue, not copyright. The same reasoning weakens every site’s anti-bot defense against AI crawlers.
AI bot paywalls are framed as new income. The real question is which AI answers you’re willing to disappear from.
OpenAI killed Atlas because AI browsers were always a workaround. The real fix is a website machines can actually read.
AI visibility tools count citations. Citations aren’t recommendations. Here’s what to measure instead, and why the gap keeps widening.
WebMCP gives AI agents named tools to call, and a clean route for prompt injection. Here’s what Chrome says to lock down first.
For website owners chasing AI visibility: LLMs.txt answers who you are. WebMCP answers what your site can do.
Open AI Overview rankings won’t change the work. What a machine can read, parse, and verify on your site is what matters now.
Google’s Open Knowledge Format turns your website’s ideas into a linked graph agents can traverse. Here’s what that means for how machines understand your site.
Google filed AI visibility under search. Stop treating it as a separate workstream with its own budget and its own quarterly report.
Six audits. One failure. The fix took five minutes. Here’s what Lighthouse 13.3’s Agentic Browsing category actually tells you, and what it doesn’t.
Website owners: Apple and Google both shipped phone agents this month. Only Google’s visits your site to act. Here’s what that demands.
Bought Reddit citations will be filtered the way Penguin filtered bought links. The only strategy that compounds on the agentic web is being real.
The “separate AEO strategy” consultants have been selling collapses when Google itself says it’s one playbook. Here’s what that means for how you build.
Most AI visibility advice skips the rendering step. A study of 274 fintech homepages shows why that’s the most expensive mistake to make.
Accessibility used to be a compliance checkbox. In 2026 it’s the interface the majority of your visitors use to read your website.
A Munich court found AI Overviews are Google’s own speech, not search results – that changes who pays when the answer about you is wrong.
Is your website invisible to AI agents? Six companies just signaled the channel is real. Here’s the window that’s still open.