Denying reading is self-sabotage
Letting answer systems read who you are, what you do, your hours and your phone number is exactly what a service business wants. It is your storefront in front of the question.
[ note ] Analysis · 2026-08-24
Closed by default: half the internet is saying no to its own customers.
Over the past weeks we measured dozens of real business websites, in Miami and beyond. One pattern keeps repeating: the site works perfectly in a browser and shuts the door on AI crawlers. It is almost never a decision. It is almost always a default nobody reviewed. And it is costing customers to the people who least suspect it.
[01] The finding
Websites perfect for people, mute for machines.
We measured it with real requests, not opinions: sites that return 403 (forbidden) when the caller identifies as the crawler behind ChatGPT or Claude. Restaurant menus that, without executing JavaScript, collapse into a "load more" button. Platforms selling AI acronyms on their homepage while their sites serve no llms.txt at all. The owner sees a complete, beautiful website on their phone; the system that now answers their customers' questions sees a closed door.
[02] The origin
Nobody decided to close. That is the serious part.
Since 2025, a large share of internet infrastructure blocks AI crawlers by default: it ships that way unless someone deliberately opens it. Fear wrote that default and inertia keeps it. Behind it sits an outdated mental model: every bot is a content thief or an attack vector. That category has expired. Today's AI crawler is not coming to steal your website: it is coming to decide whether you exist when someone asks where to eat, who fixes their car, or who can build them a website. Closing the door protects you from nothing; it erases you from the conversation.
Limiting action is hygiene
Making sure an agent cannot submit your form a hundred times or drain your inventory is sensible. These are two different layers. Panic merges them and throws away the first along with the second.
[03] The litmus test
Do you win or lose when AI answers with your information?
That is the only question that matters, and it has an answer by type of business. If you sell services (a restaurant, a shop, a studio, a practice), you always win: the assistant's answer does not replace your product, it brings you the customer, because the transaction happens at your table, in your garage, in your chair. For you, opening is not an opinion: it is arithmetic. If instead your product IS the content (a newspaper, a stock photo library, a paid course), the answer can replace the visit, and there restricting is a legitimate business decision, not a mistake. Full honesty: nobody controls what an assistant finally answers. What the door decides is something else: whether you are even eligible to be the answer.
[04] Check it yourself
Your website, in one minute, taking nobody's word.
Do not believe us: ask your own server. From any terminal, request your homepage identifying as ChatGPT's crawler:
curl -A "GPTBot" -o /dev/null -s -w "%{http_code}" https://yourdomain.com/
Open door
Your server delivers the page. Next question: is what it delivers understandable without executing JavaScript, and does your robots.txt avoid banning AI crawlers from the whole site?
Closed door
Your business is saying no to every customer who asks an assistant about you. And most likely: nobody at your company ever made that decision.
The layer on top
Structured data that states who you are without ambiguity, and an llms.txt with your business profile for assistants. They do not guarantee the citation; they make you easy to cite. The difference between luck and eligibility.
[ → ] And your site?
We tell you how AI sees you. Measured, not guessed.
Our audit opens your website the way crawlers open it and hands you the diagnosis in writing: what they read, what they miss, and which door you have closed without knowing.
Let's talk →