ForgeAIBot crawler

ForgeAIBot indexes websites at their owners' request so their AI support chatbots can answer from that content. If you're seeing our crawler, someone asked ForgeAI to index that site.

User agent

ForgeAIBot/1.0 (+https://forgeai.com.au/bot)

How it behaves

• Respects robots.txt, including Disallow rules and Crawl-delay.
• Uses sitemaps when available instead of deep link-following.
• Rate-limits itself and pauses between requests.
• Never bypasses logins, paywalls, CAPTCHAs, or anti-bot systems.
• Only fetches HTML pages; skips media and binary files.

Blocking the crawler

Add this to your robots.txt to opt out entirely:

User-agent: ForgeAIBot
Disallow: /

Contact

Questions or takedown requests: crawler@forgeai.com.au. Include the affected domain and we'll remove indexed content promptly.