Faisal Salman
|
e8a301265a
|
[extensions] Create a new list for Vehicles user-agent: Volvo, Rivian, BYD
|
2024-12-21 12:53:06 +07:00 |
|
Faisal Salman
|
d024fe2006
|
[extensions] Fix #770 - Add new fetcher bot: Bluesky
|
2024-12-03 21:48:01 +07:00 |
|
Faisal Salman
|
fc851b40c5
|
Improve device detection: recognize MIUI as Xiaomi
|
2024-11-30 08:35:19 +07:00 |
|
Faisal Salman
|
c37f3863c0
|
[sub:extensions] Rearrange mediaplayer regexes
|
2024-11-26 20:13:07 +07:00 |
|
Faisal Salman
|
60cc84d58c
|
[sub:extensions] Add new bots: Apache-HttpClient, go-http-client, got, GuzzleHttp, Java-http-client, libwww-perl, lua-resty-http, Needle, OkHttp, node-fetch, PHP-SOAP, PostmanRuntime, superagent
|
2024-11-24 12:27:08 +07:00 |
|
Faisal Salman
|
38baf844fc
|
Update version to 2.0.0
|
2024-11-18 07:55:18 +07:00 |
|
Faisal Salman
|
70b3003344
|
[submodule:helpers] Add new method isAIBot() : detect AI bots
|
2024-11-16 22:14:14 +07:00 |
|
Faisal Salman
|
2b125c8de4
|
[submodule:extensions] Add new bots: AI2Bot, aiHitBot, anthropic-ai, cohere-ai, Diffbot, ImagesiftBot, magpie-crawler, Omgilibot, Screaming Frog SEO Spider, Seznambot, Teoma, Timpibot, VelenPublicWebCrawler, Vercelbot, Webzio-Extended, YouBot
|
2024-11-16 18:40:02 +07:00 |
|
Faisal Salman
|
2181559b01
|
[submodule:extensions] Add new email: Airmail, BlueMail, eMClient, NaverMailApp, Sparrow, Yahoo
|
2024-11-16 15:28:35 +07:00 |
|
Faisal Salman
|
14caf6d204
|
[submodule:extensions] Add new library: java , python-urllib , python-requests
|
2024-11-15 17:25:43 +07:00 |
|
Faisal Salman
|
4a12562946
|
[submodule:extensions] Add new CLI: ELinks, HTTPie
|
2024-11-14 22:51:41 +07:00 |
|
Faisal Salman
|
ee51caf422
|
Update version to 2.0.0-rc.3
|
2024-11-13 17:47:44 +07:00 |
|
Faisal Salman
|
cbe06acaf2
|
[submodule:extensions] Add email clients: Evolution , KMail , Kontact
|
2024-11-10 19:32:37 +07:00 |
|
Faisal Salman
|
a0e11b701e
|
[submodule:extensions] Add some new bots: 360Spider , Archive.org Bots , CCBot , DataForSeoBot , DuckAssistBot , Exabot , Google Bots , Meta Bots , MojeekBot , PerplexityBot , PetalBot , TurnitinBot , Yeti , YisouSpider
|
2024-11-10 13:08:21 +07:00 |
|
Faisal Salman
|
3b3361fe53
|
[submodule:extensions] Breaking change: rename module to library
|
2024-11-10 08:16:24 +07:00 |
|
Faisal Salman
|
a1644b6775
|
Update version to 2.0.0-rc.2
|
2024-11-07 21:00:04 +07:00 |
|
Faisal Salman
|
1e0175c001
|
Update version to 2.0.0-rc.1
|
2024-10-29 22:29:51 +07:00 |
|
Faisal Salman
|
a4c81a5600
|
[submodule:extensions] Add new crawler: OAI-SearchBot (OpenAI's bot for SearchGPT)
|
2024-09-28 10:33:33 +07:00 |
|
Faisal Salman
|
f00fb3a2e2
|
[submodule:extensions] Restore Bots as an umbrella term for any kind of automated browsers: CLIs , Crawlers , Fetchers , and Modules
|
2024-09-22 13:50:00 +07:00 |
|
Faisal Salman
|
b1dae13245
|
[extension][bot] Add AhrefsBot, AhrefsSiteAudit, Dotbot, Rogerbot, Uptimerobot, Coc Coc Bot
|
2024-08-20 14:06:06 +07:00 |
|
Pablo Osés
|
68ae2a76de
|
adding detection of MJ12Bot and SemrushBot into extension Crawlers including testing (#738)
|
2024-08-14 10:46:46 +07:00 |
|
Faisal Salman
|
a74ace8dd1
|
Add Yahoo! Japan to Crawler, Fetcher, & InApp browser
|
2024-06-22 00:13:13 +07:00 |
|
Faisal Salman
|
de4978e8d9
|
Add mediaplayer as a new browser.type
|
2024-06-11 09:12:36 +07:00 |
|
Faisal Salman
|
6b642e2838
|
[Extensions submodule] Add ChatGPT-User as fetcher
|
2024-06-08 20:18:56 +07:00 |
|
Faisal Salman
|
5328642e18
|
Update version to 2.0.0-beta.3
|
2024-06-08 01:17:52 +07:00 |
|
Faisal Salman
|
bdcd927304
|
Update test for extensions
|
2024-06-08 00:27:28 +07:00 |
|
Faisal Salman
|
db3423a76c
|
BREAKING - Remove bot type, divide as crawler / fetcher
Add new crawler: Baiduspider, DuckDuckBot, & Sogou Web Spider
Add new fetcher: Mastodon, Pinterestbot, Redditbot, LinkedInBot, Discordbot, Telegrambot, Twitterbot, Snapchat Bot, WhatsApp
|
2024-06-08 00:06:27 +07:00 |
|
Faisal Salman
|
173325faa1
|
Add some well-known bot user-agents: Applebot, Amazonbot, Bytespider, Claudebot, Yandexbot
|
2024-06-06 22:36:15 +07:00 |
|
Faisal Salman
|
39590f112d
|
BREAKING CHANGE - Add new property to browser : type
|
2024-06-02 23:04:25 +07:00 |
|
Faisal Salman
|
4d950db145
|
Update version to 2.0.0-beta.2
|
2024-01-28 22:34:46 +07:00 |
|
Faisal Salman
|
ac282df13e
|
Update version to 2.0.0-beta.1
|
2023-10-02 15:11:31 +07:00 |
|
Faisal Salman
|
f57f8fa1a7
|
Update enums & extensions
|
2023-09-30 14:10:53 +07:00 |
|
Faisal Salman
|
d168b75a3a
|
Bump version 2.0.0-alpha.3
|
2023-08-17 11:29:18 +07:00 |
|
Faisal Salman
|
6e26e38247
|
[extension] Add Axios, jsdom, Scrapy. Also fixes #627 #156 #217 #227
Axios: `axios/VERSION`
https://www.zenrows.com/blog/axios-user-agent#what-is-axios-user-agent
JSDOM: `Mozilla/5.0 (${process.platform || "unknown OS"}) AppleWebKit/537.36 (KHTML, like Gecko) jsdom/${jsdomVersion}`
https://github.com/jsdom/jsdom
Scrapy: `Scrapy/VERSION (+https://scrapy.org)`
https://docs.scrapy.org/en/master/topics/settings.html
|
2023-08-15 11:46:31 +07:00 |
|
Faisal Salman
|
22eae9f70c
|
[extensions] Add GPTBot to Bots
|
2023-08-07 21:39:21 +07:00 |
|
Faisal Salman
|
4c77c5ef21
|
Revive the extensive list of MediaPlayers regexes by @leofiore as an Extension
(Original commit reference: 3fa1fe9f704d716e8f57dea46bccc5724da7b952)
|
2023-04-24 14:58:08 +07:00 |
|
Faisal Salman
|
9102871dea
|
Rearrange the structure of src folders
|
2023-04-21 09:47:51 +07:00 |
|