About WebtraxBot
The crawler behind Webtrax website checks: broken links and page performance.
Who runs it?
WebtraxBot crawls a site on behalf of its owner. If it is visiting your site, somebody added your URL to Webtrax to be told when links break or pages slow down.
What it does
- Follows links within your site from the start page, up to a page limit, and reads
/sitemap.xml. - Checks each link once per scan with
HEAD, falling back toGET. A temporary error (5xx, timeout) is retried once. - On some plans, loads pages in a real browser to measure performance.
How to identify it
Mozilla/5.0 (compatible; WebtraxBot/1.0; +https://webtrax.dev/bot) How to slow it down or block it
WebtraxBot honours robots.txt groups for WebtraxBot (or * when there is none), including Crawl-delay up to 30 seconds.
User-agent: WebtraxBot
Crawl-delay: 5
Disallow: /private/ To block it entirely:
User-agent: WebtraxBot
Disallow: / Contact
Questions or abuse reports: bot@webtrax.dev. Include your domain and, if you can, a sample request with its time, and we will stop scans of your site.