Thirty-Four Out of Thirty-Four. Then Zero Out of Ten.

My last article ended by admitting something my own dashboard could not do. Four days before it went out, I had already fixed it — and the fix had an answer waiting. The article was about twenty requests for an .env file that does not exist on my site. Every one of them said it was ClaudeBot. I could not tell you whether they really were, because my dashboard verified crawlers, not paths: it would report that ClaudeBot as a whole was 68% verified that week, and stop there. Which of its requests were in the verified 68% and which were not, it had no way to say. That is now a column. Here is what it says. The fix is smaller than it sounds, and it is really two fixes. The first one I owe to a reader. When I published an article in early September about crawler verification, a commenter pointed out that my "unverified" number was doing two jobs at once: it counted requests that had been checked against the vendor's published IP ranges and failed, and it counted requests that had never been checked at all. A crawler with no published ranges and a crawler caught wearing a false name both came out as 0% verified. That is not a measurement. That is two different facts wearing the same number. So the plugin now records whether a check actually ran, separately from whether it passed. The second fix follows from the first: once you know which individual requests were checked, you can total them by path instead of by crawler. The page list grew a column reading verified / checked, and paths where nothing has been checked yet say so in words rather than showing a zero. This is the top ten pages for the seven days ending September 22, in the order the dashboard shows them. Path (requests) Verified / checked / (40) 22 / 22 /tag/news/ (21) 1 / 1 /.env (14) 0 / 6 /topics/ダバオの運転事情/ (9) 6 / 6 /read-document (7) 0 / 3 /tag/知育教室/ (7) 2 / 2 /api/designer/v1/file-content (6) 0 / 1 /attachment_tag/favicon/ (6) not checked /tag/lego/ (6) 2 / 2 /tag/イスラム/ (6) 1 / 1 Read down the right-hand column and the table sorts itself into two piles. Most of it is a small Japanese site about living in Davao being read the way you would expect: the front page, some tag listings, an article about driving there. Three rows are not like the others, because three of these paths do not exist and never have. /.env is where a web application keeps its database passwords and API keys. /read-document and /api/designer/v1/file-content are endpoints belonging to software this site has never run. All three return 404. Now add up each pile. Checks on the real pages. Every one of them matched the vendor it claimed to be. Six paths had been checked at least once. The seventh, /attachment_tag/favicon/, had not been checked at all, so it is in neither pile. And the other pile. Checks on the three paths that do not exist. Not one matched. All three name ClaudeBot as their most frequent visitor. Same site. Same week. Same instrument, running the same comparison against the same published list of addresses. The only thing that changes between the two piles is whether the thing being asked for is really there. It would be easy to read that as an accusation, so let me put the rest of the week's numbers next to it. Over those same seven days ClaudeBot made 599 requests to this site. 94% of them verified as Anthropic. 3% had not been checked yet. Which means that of everything I was able to check, almost all of it was exactly what it said it was. Anthropic began publishing its IP ranges on August 18, and on the evidence of this site that list works. The overwhelming majority of traffic calling itself ClaudeBot here is Anthropic. That is the point. A name that is usually honest is a much better disguise than one that is usually suspect, and the ten failures are not distributed evenly across the site's traffic — they are sitting on the three paths where somebody would go looking for credentials. One more detail, small but hard to unsee. Last week /read-document was in this same table, and the name on it was PerplexityBot. This week the same non-existent path is being requested under ClaudeBot. The path did not change. The costume did. Splitting "checked and failed" from "never checked" had a second effect I did not plan for: it made visible how much of this traffic never reaches the gate at all. Here is the whole week, sorted by what the instrument was able to do with each request. Requests Share What the instrument could do 1,809 65% No verification method published by the vendor ~800 29% Checked, matched the vendor it claimed ~85 3% Checked, did not match ~70 3% Not checked yet AI crawler requests to davaonline.net, seven days ending 2026-09-22. The first row is exact: Meta's two crawlers, Bytespider and Amazonbot publish neither IP ranges nor reverse DNS, so no check is possible. The other three are derived from the dashboard's rounded percentages and are accurate to within a few requests. Two thirds of it cannot be checked by anyone, because the vendors have published nothing to check against. Meta alone accounts for most of that. An absent failure is not a pass: for those 1,809 requests I have no more idea who sent them than I did before I built any of this. Ten checks is ten checks. The three paths received 27 requests between them; only 10 had been checked by the time I looked. The other 17 arrived before the column existed and will never be checked, because the plugin does not re-examine a request it has already processed. I am not counting them, in either direction. A failed check is not an identification. It means the address did not appear in the vendor's published list. It does not tell me who did send the request, and nothing here should be read as a claim about Anthropic, whose name was used without its involvement. My own list was three weeks old. A verification result is only as fresh as the snapshot it compares against, and mine dated from September 3. I refreshed it on September 22 before writing this. The published ranges for Anthropic, Perplexity, Mistral, Common Crawl and DuckDuckGo had not changed by a single entry — so none of the failures above are explained by a stale list. OpenAI's had: 22 ranges added for ChatGPT-User. Which means ChatGPT-User's 35% that week is an undercount, and since checked requests are never re-checked, it is an undercount I cannot repair. It will simply be right from now on. Verified means the company, not the crawler. Anthropic publishes one combined list covering all three of its crawlers, so a match confirms the request came from Anthropic — not which of its crawlers it was. Cached pages are invisible. When a page cache answers, PHP never runs and nothing is recorded. Every count here is a lower bound. The instrument is a WordPress plugin called AILYS Lens, and it is free. I build it, which you should factor in. Every calculation runs on your own server, nothing is transmitted anywhere, and human visitors are never recorded. If you install it, the thing worth doing is the one I did here: read the page list first and the crawler list second. A crawler name tells you what something claims to be. A path tells you what it wanted. When a path you have never heard of appears on that list, the new column will tell you whether the name attached to it survived contact with the vendor's own records — and when it says not checked, that is an honest blank rather than a zero pretending to be a verdict. I spent two articles calling this gap a defect in my own tool. It was more useful to close it than to keep describing it. AILYS Lens is free permanently. That is a design commitment rather than a pricing stage — every feature runs locally on your server, so there is nothing for a paid tier to unlock. The diagnostic service alongside it, AILYS Doctor, is a different thing. Lens tells you what happened; Doctor tells you why, and what to change. It is free to use during its data-collection period, and any future pricing will be announced on the site. See a sample diagnosis · AILYS Doctor Illustration generated with AI and selected by the author.