This repository contains up-to-date lists of IP addresses of known bots and crawlers, useful for whitelisting or filtering network traffic. They can also be used as blacklists.
Do you have any questions or want to receive notifications about important changes or new features in my repositories? Join my Discord server! If you don't use Discord, you can also open an issue on GitHub.
The project is released under the MIT license — you can do whatever you want with it.
If you like this repository, leave a star ⭐. Thank you!
| 🔄 Update frequency | 📋 Changelog |
|---|---|
Every 6 hours |
CHANGELOG.md |
Search engines, SEO tools and web testing bots.
| Service | IPs & CIDRs | Sources | Downloads |
|---|---|---|---|
| GoogleBot | 0 - 311 | developers.google.com | TXT • CSV • JSON |
| Google Special Crawlers | 0 - 266 | developers.google.com | TXT • CSV • JSON |
| Google User-Triggered Fetchers | 0 - 1,504 | Fetchers & Google | TXT • CSV • JSON |
| BingBot | 0 - 28 | www.bing.com | TXT • CSV • JSON |
| DuckDuckBot | 0 - 319 | duckduckgo.com | TXT • CSV • JSON |
| YandexBot | 0 - 16 | yandex.com | TXT • CSV • JSON |
| FacebookBot | 0 - 1,041 | RIPEstat & RADB | TXT • CSV • JSON |
| Applebot | 0 - 12 | search.developer.apple.com | TXT • CSV • JSON |
| Kagi | 4 - 0 | Custom list | TXT • CSV • JSON |
| AhrefsBot | 9,870 - 0 | api.ahrefs.com | TXT • CSV • JSON |
| Semrush | 0 - 7 | RIPEstat & RADB | TXT • CSV • JSON |
| WebPageTest Bot | 35 - 0 | www.webpagetest.org | TXT • CSV • JSON |
AI crawlers from large language model providers.
| Service | IPs & CIDRs | Sources | Downloads |
|---|---|---|---|
| ClaudeBot | 0 - 20 | claude.com | TXT • CSV • JSON |
| GPTBot | 0 - 21 | openai.com | TXT • CSV • JSON |
| OpenAI SearchBot | 0 - 35 | openai.com | TXT • CSV • JSON |
| ChatGPT User | 0 - 240 | openai.com | TXT • CSV • JSON |
| PerplexityBot | 0 - 8 | perplexity.ai | TXT • CSV • JSON |
| Perplexity User | 0 - 4 | perplexity.ai | TXT • CSV • JSON |
Uptime monitoring services and internet scanners.
| Service | IPs & CIDRs | Sources | Downloads |
|---|---|---|---|
| BetterStack | 34 - 0 | uptime.betterstack.com | TXT • CSV • JSON |
| PingdomBot | 152 - 0 | IPv4 & IPv6 | TXT • CSV • JSON |
| Pulsetic | 48 - 2 | IPv4 & IPv6 | TXT • CSV • JSON |
| UptimeRobot | 232 - 0 | uptimerobot.com | TXT • CSV • JSON |
| Censys | 0 - 36 | RIPEstat & RADB | TXT • CSV • JSON |
| Modat Scanner | 0 - 35 | scanner.modat.io | TXT • CSV • JSON |
| Shodan | 95 - 6 | Custom list | TXT • CSV • JSON |
CDN providers, hosting networks, DNS resolvers and web services.
| Service | IPs & CIDRs | Sources | Downloads |
|---|---|---|---|
| Cloudflare | 0 - 22 | IPv4 & IPv6 | TXT • CSV • JSON |
| Bunny CDN | 906 - 0 | IPv4 & IPv6 | TXT • CSV • JSON |
| Canonical | 0 - 37 | RIPEstat & RADB | TXT • CSV • JSON |
| NASK PL | 0 - 642 | RIPEstat & RADB | TXT • CSV • JSON |
| Palo Alto Networks | 0 - 93 | RIPEstat, RADB & Custom list | TXT • CSV • JSON |
| DNS Resolvers | 72 - 0 | Custom list | TXT • CSV • JSON |
| Stripe | 15 - 0 | stripe.com | TXT • CSV • JSON |
| TelegramBot | 0 - 14 | core.telegram.org | TXT • CSV • JSON |
| RSS API | 1 - 1 | rssapi.net | TXT • CSV • JSON |
| Baidu | 0 - 616 | RIPEstat & RADB | TXT • CSV • JSON |
