# See what AI sees on your site. > Enter a web address. We fetch the page like a crawler, with no scripts run, and show what people, search crawlers and AI answer engines get from it. Page: https://adjltechnology.com/scan/ Updated: 2026-11-12 Free site scan Type in a web address. We fetch the page the way a crawler does, without running its scripts, and show what people, search crawlers and AI answer engines get from it. Then we list what to fix first. One page, plus its robots.txt, sitemap and llms.txt. We don’t keep the address or anything we fetch. The scanner runs from your browser, so it needs JavaScript. With it off, you can still read a recorded scan of this site below. It shows everything a scan of your page would. [Read the recorded scan](https://adjltechnology.com/scan/#report) People read the top layer, crawlers follow the middle one and machines parse the bottom one. The report ## Every page has three readers. People read the words. Crawlers follow the links and the rules. Machines parse the structured data. The report reads your page the way each one does, then lists what to fix first. Until you scan a page, it shows a scan of this site’s home page, recorded on Nov 12, 2026. Scan report adjltechnology.com/ Nothing to fix. Here’s what we checked. ### What people read The words on the page before any script runs, and what a search result shows. ### What crawlers follow Whether search and AI crawlers may read the page, and where it sends them. ### What machines parse Structured data, share tags and the files written for machines. ### AI crawlers and your robots.txt What your robots.txt tells each crawler about this page. A crawler with no rules of its own follows the rules for all crawlers. | Crawler | Run by | What it does | This page | | --- | --- | --- | --- | | Search engines | | | | | `Googlebot` | Google | Crawls pages for Google Search. AI Overviews in Search use the same index. | Allowed by the rules for all crawlers `Allow: /` | | `Bingbot` | Microsoft | Crawls pages for Bing search, which Copilot’s web answers draw on. | Allowed by the rules for all crawlers `Allow: /` | | `Applebot` | Apple | Crawls pages for search in Siri, Spotlight and Safari. Apple says it may also use them to train its models. | Allowed by the rules for all crawlers `Allow: /` | | AI search and answers | | | | | `OAI-SearchBot` | OpenAI | Finds pages to show and link in ChatGPT search. OpenAI’s separate GPTBot does the training crawls. | Allowed by the rules for all crawlers `Allow: /` | | `Claude-SearchBot` | Anthropic | Indexes pages to improve the results Claude’s search gives. | Allowed by the rules for all crawlers `Allow: /` | | `PerplexityBot` | Perplexity | Finds and links pages in Perplexity’s answers. Perplexity says it isn’t used for training. | Allowed by its own rules `Allow: /` | | `Meta-WebIndexer` | Meta | Reads pages to improve Meta AI’s search results and citations. | Allowed by the rules for all crawlers `Allow: /` | | Fetches a page when a person asks | | | | | `ChatGPT-User` | OpenAI | Visits a page when someone asks ChatGPT about it. OpenAI says robots.txt may not apply. | Allowed by the rules for all crawlers `Allow: /` | | `Claude-User` | Anthropic | Visits a page when someone’s question to Claude needs it. Anthropic says it follows robots.txt. | Allowed by the rules for all crawlers `Allow: /` | | `Perplexity-User` | Perplexity | Visits a page when someone asks Perplexity about it. Perplexity says it generally ignores robots.txt. | Allowed by the rules for all crawlers `Allow: /` | | AI training | | | | | `GPTBot` | OpenAI | Collects pages that may be used to train OpenAI’s models. | Allowed by its own rules `Allow: /` | | `ClaudeBot` | Anthropic | Collects pages that may be used to train Anthropic’s models. | Allowed by its own rules `Allow: /` | | `Google-Extended` | Google | Not a crawler. A switch for whether Google may use your pages to train Gemini, and to ground Gemini’s answers in them. Google says it doesn’t affect whether your pages appear or rank in Google Search. | Allowed by its own rules `Allow: /` | | `Applebot-Extended` | Apple | Not a crawler. A switch for whether Apple may use what Applebot collects to train its models. Apple’s search isn’t affected. | Allowed by the rules for all crawlers `Allow: /` | | `Meta-ExternalAgent` | Meta | Collects pages for uses such as training Meta’s AI models. | Allowed by the rules for all crawlers `Allow: /` | | `CCBot` | Common Crawl | Builds Common Crawl, a free public archive of the web that many AI models were trained on. | Allowed by the rules for all crawlers `Allow: /` | | `Bytespider` | ByteDance | The crawler of TikTok’s parent company. ByteDance says little about it. It’s widely reported to gather data for AI models. | Allowed by the rules for all crawlers `Allow: /` | What it does ## A first look, not an audit. The scanner fetches one page from our server and reads the HTML it gets back, with no scripts run. Many AI crawlers read pages the same way. It also reads your robots.txt and looks for a sitemap and an llms.txt. It can’t see your rankings, your traffic or what an AI answer says about you today. It doesn’t check every page or judge your writing. We don’t store the address you scan or anything we fetch. The report goes back to your browser and nowhere else. Go further ## Want the whole site read properly? The website audit starts from these three layers and goes much deeper, across every kind of page on your site. You get a written report with the fixes in order of impact, and a call to walk through it. It’s priced by quote, after a look at the site. [Ask for an audit quote](https://adjltechnology.com/start/?topic=audit#form) [What the audit covers](https://adjltechnology.com/services/seo/audit/) Start here ## Tell us what’s slow, or what can’t be found. One paragraph is plenty. Tell us what you want built, the step that costs the most, or the page that should rank and doesn’t. If you know why the obvious fix hasn’t worked, say that too. [Start a project](https://adjltechnology.com/start/) [daniel@adjlcapital.com](mailto:daniel@adjlcapital.com) [Email Daniel](mailto:daniel@adjlcapital.com) Daniel reads and answers these himself, usually within a day or two. No sequence, no drip, and nothing sold to anyone. [ADJL Technology](https://adjltechnology.com/) Today’s investment, tomorrow’s legacy. ADJL Technology builds AI systems, websites and apps, and the technical SEO that gets them found. Founded and led by Daniel Laskowski. [daniel@adjlcapital.com](mailto:daniel@adjlcapital.com) Built 2026-11-12 from commit 6afca01. No cookies, no trackers, no third-party scripts. ADJL Technology. Nothing on this site is an offer to sell or a solicitation of any security, and nothing here is investment advice. Descriptions of internal systems are general by design and omit parameters and thresholds.