Faisal Salman
|
2b125c8de4
|
[submodule:extensions] Add new bots: AI2Bot, aiHitBot, anthropic-ai, cohere-ai, Diffbot, ImagesiftBot, magpie-crawler, Omgilibot, Screaming Frog SEO Spider, Seznambot, Teoma, Timpibot, VelenPublicWebCrawler, Vercelbot, Webzio-Extended, YouBot
|
2024-11-16 18:40:02 +07:00 |
|
Faisal Salman
|
a0e11b701e
|
[submodule:extensions] Add some new bots: 360Spider, Archive.org Bots, CCBot, DataForSeoBot, DuckAssistBot, Exabot, Google Bots, Meta Bots, MojeekBot, PerplexityBot, PetalBot, TurnitinBot, Yeti, YisouSpider
|
2024-11-10 13:08:21 +07:00 |
|
Faisal Salman
|
a4c81a5600
|
[submodule:extensions] Add new crawler: OAI-SearchBot (OpenAI's bot for SearchGPT)
|
2024-09-28 10:33:33 +07:00 |
|
Faisal Salman
|
b1dae13245
|
[extension][bot] Add AhrefsBot, AhrefsSiteAudit, Dotbot, Rogerbot, Uptimerobot, Coc Coc Bot
|
2024-08-20 14:06:06 +07:00 |
|
Pablo Osés
|
68ae2a76de
|
adding detection of MJ12Bot and SemrushBot into extension Crawlers including testing (#738)
|
2024-08-14 10:46:46 +07:00 |
|
Faisal Salman
|
a74ace8dd1
|
Add Yahoo! Japan to Crawler, Fetcher, & InApp browser
|
2024-06-22 00:13:13 +07:00 |
|
Faisal Salman
|
bdcd927304
|
Update test for extensions
|
2024-06-08 00:27:28 +07:00 |
|