Skip to content

Commit 800ae1e

Browse files
committed
Pattern updates
1 parent 819c051 commit 800ae1e

10 files changed

Lines changed: 123 additions & 126 deletions

File tree

.github/workflows/periodic.yml

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -22,6 +22,8 @@ jobs:
2222
continue-on-error: true
2323
run: npm t
2424
- name: Create issue on test failure
25+
env:
26+
GH_TOKEN: ${{ github.token }}
2527
if: steps.test.outcome == 'failure'
2628
run: |
2729
gh issue create \

CHANGELOG.md

Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1,5 +1,9 @@
11
# Changelog
22

3+
## [5.1.38](https://github.com/omrilotan/isbot/compare/v5.1.37...v5.1.38)
4+
5+
- Pattern updates
6+
37
## [5.1.37](https://github.com/omrilotan/isbot/compare/v5.1.36...v5.1.37)
48

59
- Better checking for non empty strings in the interface functions

fixtures/crawlers.yml

Lines changed: 5 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -614,6 +614,8 @@ Pinterest:
614614
- Mozilla/5.0 (compatible; Pinterestbot/1.0; http://www.pinterest.com/bot.html)
615615
Placid.app:
616616
- placid.app/v1
617+
Playwright:
618+
- Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/120.0.0.0 Safari/537.36 Playwright/1.40.0
617619
PocketParser:
618620
- PocketParser/2.0 (+https://getpocket.com/pocketparser_ua)
619621
Postman:
@@ -693,6 +695,8 @@ ScreenerBot:
693695
- ScreenerBot Crawler Beta 2.0 (+http://www.ScreenerBot.com)
694696
Seekport:
695697
- Mozilla/5.0 (compatible; Seekport Crawler; http://seekport.com/)
698+
Selenium:
699+
- Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/120.0.0.0 Safari/537.36; Selenium
696700
Semrush Bot:
697701
- Mozilla/5.0 (compatible; SemrushBot/0.97; +http://www.semrush.com/bot.html)
698702
- Mozilla/5.0 (iPhone; CPU iPhone OS 6_0 like Mac OS X) AppleWebKit/536.26 (KHTML, like Gecko) Version/6.0 Mobile/10A5376e Safari/8536.25 (compatible; SiteAuditBot/0.97; +http://www.semrush.com/bot.html)
@@ -991,6 +995,7 @@ ZZZ Miscellaneous bots:
991995
ZZZ Miscellaneous Glitches and Errornous User Agent Strings:
992996
- "123"
993997
- <class 'fake_useragent.fake.FakeUserAgent'>
998+
- acunetix-product/wvs
994999
- Chrome
9951000
- default_user_agent
9961001
- ipad

fixtures/downloaded/downloaded

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1 +1 @@
1-
Mon, 09 Mar 2026 15:45:48 GMT
1+
Mon, 13 Apr 2026 10:34:26 GMT

fixtures/downloaded/kikobeats.json

Lines changed: 4 additions & 103 deletions
Large diffs are not rendered by default.

fixtures/downloaded/monperrus.json

Lines changed: 48 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1194,5 +1194,52 @@
11941194
"Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) HeadlessChrome/87.0.4280.88 YextBot/Java Safari/537.36",
11951195
"Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/138.0.7204.168 Safari/537.36 DatadogSynthetics",
11961196
"Mozilla/5.0 AppleWebKit/537.36 Chrome/139.0.7258.127 Safari/537.36 Google-Ads-Conversions",
1197-
"Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/134.0.6998.35 Safari/537.36 (Checkly, https://www.checklyhq.com)"
1197+
"Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/134.0.6998.35 Safari/537.36 (Checkly, https://www.checklyhq.com)",
1198+
"ALittle Client",
1199+
"AliyunSecBot/Aliyun AliyunSecBot@service.alibaba.com",
1200+
"Claude-Web/1.0 (web crawler; +https://www.anthropic.com/; bots@anthropic.com)",
1201+
"anthropic-ai",
1202+
"Mozilla/5.0 AppleWebKit/537.36 (KHTML, like Gecko; compatible; Claude-User/1.0; +Claude-User@anthropic.com)",
1203+
"Claude-User (claude-code/2.1.86; +https://support.anthropic.com/)",
1204+
"Mozilla/5.0 AppleWebKit/537.36 (KHTML, like Gecko; compatible; Claude-SearchBot/1.0; +https://www.anthropic.com)",
1205+
"Mozilla/5.0 (compatible; Google-Extended/1.0; +http://www.google.com/bot.html)",
1206+
"cohere-ai",
1207+
"Timpibot/0.9 (+http://www.timpi.io)",
1208+
"Mozilla/5.0 (compatible; Timpibot/0.8; +http://www.timpi.io)",
1209+
"Mozilla/5.0 (compatible; Timpibot/0.9; +http://www.timpi.io)",
1210+
"Mozilla/5.0 (compatible; SERankingBacklinksBot/1.0; +https://seranking.com/backlinks-crawler)",
1211+
"Mozilla/5.0 (compatible; CMSChecker/1.0; +https://cmschecker.net)",
1212+
"Mozilla/5.0 (compatible; archive.org_bot; Wayback Machine Live Record; +http://archive.org/details/archive.org_bot)",
1213+
"Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/120.0.0.0 Safari/537.36 Playwright/1.40.0",
1214+
"Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) HeadlessChrome/120.0.0.0 Safari/537.36 Puppeteer",
1215+
"Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/120.0.0.0 Safari/537.36; Selenium",
1216+
"Mozilla/5.00 (Nikto/2.1.5) (Evasions:None) (Test:Port Check)",
1217+
"Mozilla/5.0 (X11; Linux x86_64) Nikto/2.5.0 (Evasions:None) (Test:Port Check)",
1218+
"sqlmap/1.7.8#stable (https://sqlmap.org)",
1219+
"ZmEu",
1220+
"masscan/1.0 (https://github.com/robertdavidgraham/masscan)",
1221+
"WPScan v3.8.22 (https://wpscan.com/wordpress-security-scanner)",
1222+
"Mozilla/5.0 (compatible; WPScan; +https://wpscan.com/wordpress-security-scanner)",
1223+
"Mozilla/5.0 (Windows NT 6.1; WOW64) acunetix-product/wvs (Acunetix Web Vulnerability Scanner - Free Edition)",
1224+
"acunetix-product/wvs",
1225+
"Mozilla/5.0 (compatible; Nessus; http://www.nessus.org)",
1226+
"DirBuster-1.0-RC1 (http://www.owasp.org/index.php/Category:OWASP_DirBuster_Project)",
1227+
"StatusCake_Uptime_Checker/1.0",
1228+
"colly - https://github.com/gocolly/colly",
1229+
"colly/2.1.0",
1230+
"Mechanize/2.9.1 Ruby/3.1.2 (http://github.com/sparklemotion/mechanize/)",
1231+
"air.ai/scanning Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) Chrome/126.0.0.0 Safari/537.36",
1232+
"asnriskscorer/1.0",
1233+
"OICrawler/Nutch https://openindex.ai",
1234+
"Mozilla/5.0 (l9scan/2.0; +https://github.com/LeakIX/l9scan)",
1235+
"SlaccaleBot",
1236+
"CustomAsyncHttpClient",
1237+
"HTTPie/3.2.2",
1238+
"Mozilla/5.0 AppleWebKit/537.36 (KHTML, like Gecko; compatible; Gemini-Deep-Research; +https://gemini.google/overview/deep-research/) Chrome/135.0.0.0 Safari/537.36",
1239+
"Mozilla/5.0 AppleWebKit/537.36 (KHTML, like Gecko; compatible; Perplexity-User/1.0; +https://perplexity.ai/perplexity-user)",
1240+
"Mozilla/5.0 AppleWebKit/537.36 (KHTML, like Gecko; compatible; PerplexityUser/1.0; +https://perplexity.ai)",
1241+
"meta-webindexer/1.1 (+https://developers.facebook.com/docs/sharing/webmasters/crawler)",
1242+
"DuckAssistBot/1.2; (+http://duckduckgo.com/duckassistbot.html)",
1243+
"Mozilla/5.0 AppleWebKit/537.36 (KHTML, like Gecko; compatible; MistralAI-User/1.0; +https://docs.mistral.ai/robots)",
1244+
"webzio (+https://webz.io/bot.html)"
11981245
]

0 commit comments

Comments
 (0)