Faisal Salman
|
2b125c8de4
|
[submodule:extensions] Add new bots: AI2Bot, aiHitBot, anthropic-ai, cohere-ai, Diffbot, ImagesiftBot, magpie-crawler, Omgilibot, Screaming Frog SEO Spider, Seznambot, Teoma, Timpibot, VelenPublicWebCrawler, Vercelbot, Webzio-Extended, YouBot
|
2024-11-16 18:40:02 +07:00 |
|
Faisal Salman
|
a0e11b701e
|
[submodule:extensions] Add some new bots: 360Spider, Archive.org Bots, CCBot, DataForSeoBot, DuckAssistBot, Exabot, Google Bots, Meta Bots, MojeekBot, PerplexityBot, PetalBot, TurnitinBot, Yeti, YisouSpider
|
2024-11-10 13:08:21 +07:00 |
|
Faisal Salman
|
b1dae13245
|
[extension][bot] Add AhrefsBot, AhrefsSiteAudit, Dotbot, Rogerbot, Uptimerobot, Coc Coc Bot
|
2024-08-20 14:06:06 +07:00 |
|
Faisal Salman
|
6b642e2838
|
[Extensions submodule] Add ChatGPT-User as fetcher
|
2024-06-08 20:18:56 +07:00 |
|
Faisal Salman
|
bdcd927304
|
Update test for extensions
|
2024-06-08 00:27:28 +07:00 |
|