Spiders
Overview
Use this dialog to configure the bots that should be tracked as spiders system trade. Enter one string as part of bot User Agent per line. The case of the characters does not matter.
For example:
This list matches with
For example:
bot
spider
SCJchecker
SafeSearch
Sogou
Yahoo
Bing
spider
SCJchecker
SafeSearch
Sogou
Yahoo
Bing
This list matches with
Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)
Mozilla/5.0 (compatible; YandexBot/3.0; +http://yandex.com/bots)
Mozilla/5.0 (compatible; bingbot/2.0; +http://www.bing.com/bingbot.htm)
Mozilla/5.0 (compatible; Baiduspider/2.0; +http://www.baidu.com/search/spider.html)
Mozilla/5.0 (compatible; YandexBot/3.0; +http://yandex.com/bots)
Mozilla/5.0 (compatible; bingbot/2.0; +http://www.bing.com/bingbot.htm)
Mozilla/5.0 (compatible; Baiduspider/2.0; +http://www.baidu.com/search/spider.html)