Faisal Salman
|
9003fe5724
|
[extensions] Add new bots: Algolia Crawler, contxbot, HubSpot Page Fetcher, Kagibot
|
2025-08-15 20:42:41 +07:00 |
|
Faisal Salman
|
975c4860f4
|
[extensions][helpers] Add some new AI bots: Bravebot, Cotoyogi, FirecrawlAgent, HuggingFace-Bot, Kangaroo Bot, PanguBot, Replicate-Bot, RunPod-Bot, Together-Bot, xAI-Bot
|
2025-08-14 20:42:17 +07:00 |
|
Faisal Salman
|
647b6232bd
|
[extensions][helpers] Add some bots from Vercel: v0bot, vercel-favicon-bot, vercel-screenshot-bot, vercelflags, verceltracing
|
2025-08-14 19:34:14 +07:00 |
|
Faisal Salman
|
95485f7b5d
|
[extensions][helpers] Add new bots: cohere-training-data-crawler, Gemini-Deep-Research, kakaotalk-scrap, TikTokSpider
|
2025-08-11 13:14:45 +07:00 |
|
Faisal Salman
|
0bb6e24837
|
[extensions] Add new bots: Blueno, BufferLinkPreviewBot, Claude-SearchBot, Claude-User, Coveobot, CriteoBot
|
2025-07-22 12:29:01 +07:00 |
|
Faisal Salman
|
c9d008e97a
|
Bump version 2.0.4
|
2025-06-21 21:41:00 +07:00 |
|
Faisal Salman
|
ee88f16620
|
[extensions] Add new email: Apple's Mail, DaumMail, Polymail, ProtonMail, SparkDesktop, Zimbra, ZohoMail-Desktop
|
2025-06-18 19:07:14 +07:00 |
|
Faisal Salman
|
d28d74fea7
|
[extensions] Add new apps: Discord, Evernote, Figma, Flipboard, Mattermost, Notion, Postman, Rambox, Rocket.Chat, Teams, TikTok Lite, VS Code
|
2025-06-17 01:50:33 +07:00 |
|
Faisal Salman
|
1282175bac
|
[extensions] Add new libraries: AdobeAIR, aiohttp, nutch, httpx, urllib3
|
2025-06-10 23:05:03 +07:00 |
|
Faisal Salman
|
72d0c2acb3
|
[extensions] Add new crawler bots: ChatGLM, Onespot, Startpage
|
2025-06-03 11:07:27 +07:00 |
|
Faisal Salman
|
a3549efc22
|
[extensions] Add new bots: Daumoa, iAskBot, Iframely, Qwantbot
|
2025-05-30 23:16:02 +07:00 |
|
Faisal Salman
|
64a933bae8
|
[extensions] Add new fetcher: MistralAI-User, Perplexity-User
|
2025-05-20 16:46:40 +07:00 |
|
Faisal Salman
|
e069133f51
|
Fix #787 - Avoid using lookbehind assertion in regex to ensure compatibility
|
2025-04-15 22:01:38 +07:00 |
|
Faisal Salman
|
af8acf9078
|
Bump version 2.0.3
|
2025-03-15 00:37:56 +07:00 |
|
Faisal Salman
|
f7f64a31ea
|
[extensions] Detect OS from WhatsApp user agent
|
2025-03-11 17:50:36 +07:00 |
|
Faisal Salman
|
367eae4c85
|
[extensions] Add new Vehicles: BMW, Jeep Wagooner
|
2025-03-07 10:15:47 +07:00 |
|
Faisal Salman
|
a9b31c0f7b
|
[extensions] Add bots: AdIdxBot, Linespider, LinkedInBot, MicrosoftPreview, OpenAI Image Downloader
|
2025-03-07 00:15:54 +07:00 |
|
Faisal Salman
|
8a05328ce0
|
[extensions] Improve inapp browser detection: Slack, Yahoo! Japan
|
2025-03-06 21:03:29 +07:00 |
|
Faisal Salman
|
3d41647a55
|
[extensions] Update some more bots #780: Better Uptime Bot, SemrushBot, Yahoo! Slurp
|
2025-03-04 11:16:16 +07:00 |
|
Faisal Salman
|
178e678b15
|
[extensions] Update some fetcher bots #780 - Google-PageRenderer, GoogleImageProxy, Snap URL Preview, SkypeUriPreview, TelegramBot
|
2025-03-01 08:08:44 +07:00 |
|
Faisal Salman
|
837d319630
|
[extensions][helpers] Update Semrush bot variants
|
2025-02-20 21:53:49 +07:00 |
|
Faisal Salman
|
4ea03a587d
|
Bump version 2.0.2
|
2025-02-09 07:34:43 +07:00 |
|
Faisal Salman
|
51dce55e65
|
Bump version 2.0.1
|
2025-02-04 21:49:59 +07:00 |
|
Faisal Salman
|
e8a301265a
|
[extensions] Create a new list for Vehicles user-agent: Volvo, Rivian, BYD
|
2024-12-21 12:53:06 +07:00 |
|
Faisal Salman
|
d024fe2006
|
[extensions] Fix #770 - Add new fetcher bot: Bluesky
|
2024-12-03 21:48:01 +07:00 |
|
Faisal Salman
|
fc851b40c5
|
Improve device detection: recognize MIUI as Xiaomi
|
2024-11-30 08:35:19 +07:00 |
|
Faisal Salman
|
c37f3863c0
|
[sub:extensions] Rearrange mediaplayer regexes
|
2024-11-26 20:13:07 +07:00 |
|
Faisal Salman
|
60cc84d58c
|
[sub:extensions] Add new bots: Apache-HttpClient, go-http-client, got, GuzzleHttp, Java-http-client, libwww-perl, lua-resty-http, Needle, OkHttp, node-fetch, PHP-SOAP, PostmanRuntime, superagent
|
2024-11-24 12:27:08 +07:00 |
|
Faisal Salman
|
38baf844fc
|
Update version to 2.0.0
|
2024-11-18 07:55:18 +07:00 |
|
Faisal Salman
|
70b3003344
|
[submodule:helpers] Add new method isAIBot(): detect AI bots
|
2024-11-16 22:14:14 +07:00 |
|
Faisal Salman
|
2b125c8de4
|
[submodule:extensions] Add new bots: AI2Bot, aiHitBot, anthropic-ai, cohere-ai, Diffbot, ImagesiftBot, magpie-crawler, Omgilibot, Screaming Frog SEO Spider, Seznambot, Teoma, Timpibot, VelenPublicWebCrawler, Vercelbot, Webzio-Extended, YouBot
|
2024-11-16 18:40:02 +07:00 |
|
Faisal Salman
|
2181559b01
|
[submodule:extensions] Add new email: Airmail, BlueMail, eMClient, NaverMailApp, Sparrow, Yahoo
|
2024-11-16 15:28:35 +07:00 |
|
Faisal Salman
|
14caf6d204
|
[submodule:extensions] Add new library: java, python-urllib, python-requests
|
2024-11-15 17:25:43 +07:00 |
|
Faisal Salman
|
4a12562946
|
[submodule:extensions] Add new CLI: ELinks, HTTPie
|
2024-11-14 22:51:41 +07:00 |
|
Faisal Salman
|
ee51caf422
|
Update version to 2.0.0-rc.3
|
2024-11-13 17:47:44 +07:00 |
|
Faisal Salman
|
cbe06acaf2
|
[submodule:extensions] Add email clients: Evolution, KMail, Kontact
|
2024-11-10 19:32:37 +07:00 |
|
Faisal Salman
|
a0e11b701e
|
[submodule:extensions] Add some new bots: 360Spider, Archive.org Bots, CCBot, DataForSeoBot, DuckAssistBot, Exabot, Google Bots, Meta Bots, MojeekBot, PerplexityBot, PetalBot, TurnitinBot, Yeti, YisouSpider
|
2024-11-10 13:08:21 +07:00 |
|
Faisal Salman
|
3b3361fe53
|
[submodule:extensions] Breaking change: rename module to library
|
2024-11-10 08:16:24 +07:00 |
|
Faisal Salman
|
a1644b6775
|
Update version to 2.0.0-rc.2
|
2024-11-07 21:00:04 +07:00 |
|
Faisal Salman
|
1e0175c001
|
Update version to 2.0.0-rc.1
|
2024-10-29 22:29:51 +07:00 |
|
Faisal Salman
|
a4c81a5600
|
[submodule:extensions] Add new crawler: OAI-SearchBot (OpenAI's bot for SearchGPT)
|
2024-09-28 10:33:33 +07:00 |
|
Faisal Salman
|
f00fb3a2e2
|
[submodule:extensions] Restore Bots as an umbrella term for any kind of automated browsers: CLIs, Crawlers, Fetchers, and Modules
|
2024-09-22 13:50:00 +07:00 |
|
Faisal Salman
|
b1dae13245
|
[extension][bot] Add AhrefsBot, AhrefsSiteAudit, Dotbot, Rogerbot, Uptimerobot, Coc Coc Bot
|
2024-08-20 14:06:06 +07:00 |
|
Pablo Osés
|
68ae2a76de
|
adding detection of MJ12Bot and SemrushBot into extension Crawlers including testing (#738)
|
2024-08-14 10:46:46 +07:00 |
|
Faisal Salman
|
a74ace8dd1
|
Add Yahoo! Japan to Crawler, Fetcher, & InApp browser
|
2024-06-22 00:13:13 +07:00 |
|
Faisal Salman
|
de4978e8d9
|
Add mediaplayer as a new browser.type
|
2024-06-11 09:12:36 +07:00 |
|
Faisal Salman
|
6b642e2838
|
[Extensions submodule] Add ChatGPT-User as fetcher
|
2024-06-08 20:18:56 +07:00 |
|
Faisal Salman
|
5328642e18
|
Update version to 2.0.0-beta.3
|
2024-06-08 01:17:52 +07:00 |
|
Faisal Salman
|
bdcd927304
|
Update test for extensions
|
2024-06-08 00:27:28 +07:00 |
|
Faisal Salman
|
db3423a76c
|
BREAKING - Remove bot type, divide as crawler / fetcher
Add new crawler: Baiduspider, DuckDuckBot, & Sogou Web Spider
Add new fetcher: Mastodon, Pinterestbot, Redditbot, LinkedInBot, Discordbot, Telegrambot, Twitterbot, Snapchat Bot, WhatsApp
|
2024-06-08 00:06:27 +07:00 |
|