About the Kennext crawler

What it is

Kennext is an SEO and AI-search audit. Our crawler reads the public pages of a website the way a search engine does, then reports what is holding the site back in search and in AI answers, such as missing descriptions, broken links, thin pages and missing structured data.

Why it visited

Someone asked for a scan of your site. That is usually the site's owner or team using Kennext, someone running a free scan from kennext.ai, or Kennext preparing a short audit to share with the company that owns the site.

How it identifies itself

Every request carries this user agent: Kennext/1.0 (+https://kennext.ai/bot)

How it behaves

It only reads public pages. It never logs in, submits forms or reads anything behind a password. It checks your robots.txt and follows it, and if there is no section for Kennext it follows your User-agent: * rules. It keeps its pace low, at most 5 requests at a time, with a 10-second timeout and no retries. It honours Crawl-delay up to 1 second between requests; higher values are treated as 1 second. A scan reads up to a few thousand pages, and renders a small number of JavaScript pages in a real browser so it sees what visitors see.

How to block it

Add these two lines to your robots.txt. Use the bare word Kennext (capitals don't matter); Kennext/1.0 or the full user-agent string will not match. To keep it out of only part of your site, use Disallow: /path/ for the sections you want skipped.

User-agent: Kennext

Disallow: /

Questions or problems

Email [email protected] with your site address and roughly when you saw the visits, and we'll look into it within one business day.