LinkRotBot

LinkRotBot fetches single pages on behalf of LinkRot users who want to know whether a link they placed on that page is still live. It identifies itself as Mozilla/5.0 (compatible; LinkRotBot/0.1; +https://linkrot.io/bot).

• It fetches one page at a time, at most a few times a day per URL, and reads at most 2 MB of it.
• It does not crawl, follow links beyond redirects, or store page content.
• To keep it away, add User-agent: LinkRotBot with Disallow: / to your robots.txt, or email [email protected].