Understanding the Web Crawlers of the Internet
A Spider / Bot / Robot is a computer program that browses the World Wide Web in a methodical, automated manner or in an orderly fashion.
Many sites, in particular search engines, use spidering as a means of providing up-to-date data. This is not counted as a unique click or a manual visit [source: Wikipedia].
Rebrandly can recognize when a user is clicking a link vs when a bot is clicking a link based on some fundamental differences in how they present and share data to and from our servers.
In doing so, we can eliminate spider/bot traffic from our click counters and ensure the utmost accuracy for you. This means every click you see, is undoubtedly a human. Whether that human is paying attention or not is another story...
This article is about:
- What are spiders?
- What are bots?
- Do you remove spiders/bots from counting clicks?