Track Search Engine Robot Activity With SEO Crawlytics

in Blog

SEO Crawlytics is a free plugin from Elevate Local that tracks search engine robot activity on your website.

By default the plugin works with Google and Bing however via the options area you can track other search engines by simply adding the robot name, mask and reverse DNS for the site. The plugin can send you daily reports about the search engine robots that crawled your site via email.

You can also receive instant notification via email whenever a search engine robot has visited your site, though doing so wouldn’t be very practical as your inbox would be full of notifications very quickly. Additional email addresses can be set in the notification area, which is useful if there is more than one admin for your website.

SEO Crawlytics Options

One of the best features of SEO Crawlytics is the ability to track search engine robots from other parts of your website. For example, by adding the integration code into the footer of your forum, you can also track the activity of robots in your discussion forum. This is a welcome addition as most WordPress plugins don’t offer any functionality outside of WordPress itself.

SEO Crawlytics provides a variety of reports about search engine crawling activity on your site. In each graph, a different colour is used for the search engine robots that you have configured in the options area. This helps you distinguish stats for each search engine in the reports.

The hourly visits graph is a great way of seeing hourly activity over a two week period for each of the search engines. The graph will highlight spikes in crawling activity and the times of the day most search engines visit.

SEO Crawlytics Hourly Visits

The same data can be shown on a daily basis. This graph is a bit more user friendly and shows the activity for each search engine robot on a per day basis rather than showing activity spikes on an hourly basis.

SEO Crawlytics Daily Visits

One benefit of using SEO Crawlytics over non-WordPress solutions is it’s ability to track activity on a per post basis. The plugin shows you the posts and pages that are being crawled most on your site. It also shows you the post categories that are being crawled the most by search engines.

SEO Crawlytics Crawled Categories And Pages

The peak crawl times graph shows you the times of the day search engine robots have crawled your website over the last 24 hours. It’s essentially the same information as the hourly visits graph except data is shown from the last day rather than the last two weeks. Crawl times are shown on a collective basis too rather than being broken down for each search engine robot. It’s a useful graph for understanding why your server load increases at certain times of the day.

SEO Crawlytics Crawl Times

The crawl spread pie chart shows you what search engines are crawling your website the most (for most websites, Google will have the biggest share).

SEO Crawlytics Most Active Robots

There’s a lot to like about SEO Crawlytics. It’s easy to setup, produces some useful reports about search engine crawling activity on your website and best of all – it’s free to download. I recommend trying it out if you want to track what search engines are crawling your website every day.


SEO Crawlytics: Announcement | Download From ElevateLocal | Download From WordPress

Comments (1)

  • Comment by Renuka

    Can you give details of more Bot name / mask / reverse DNS