About the Gatefee crawler

If you run a website and found this address in your logs: this is who fetched your pages, and how to stop it.

How it identifies itself

Our crawler identifies itself: a User-Agent naming us and pointing to a page about it, and a signature on every request (Web Bot Auth, RFC 9421) that can be checked against published keys. It honours robots.txt, and a named-AI-crawler disallow is recorded as the publisher's stated intent.

Concretely: the User-Agent begins Gatefee/1 and links here, and the signing keys are published at https://gatefee.ai/.well-known/http-message-signatures-directory.

What it does, and does not do

Our crawler reads only the feeds, sitemaps and article pages of publishers who registered them with us.

We never serve content from a publisher that has not signed.

How to block it

User-agent: Gatefee
Disallow: /

If a site's robots.txt disallows named AI crawlers, we record that as its stated intent; nothing from any publisher is served before that publisher has signed an agreement.