Screaming Frog
For a long time, tools like the Xenu sleuth were the secret tool in the arsenal of many SEO marketers. However, with the advent of screaming frog SEO spider, things seemed to have changed. The screaming frog SEO spider is a desktop application which crawls any site given to it and returns information to the user. Unlike other tools, the reports given here are essentially spreadsheets, and allow one to filter and reorder the results, with all the standard export options also available. There are many tools and capabilities in the screaming frog SEO spider that many, even the more experienced users, still miss or do not understand. Here are a few of the examples;
One of the most common questions users have is how the screaming frog SEO spider discovers a URL, and how one can view the in links in any particular URL. Sometimes, viewing the in links is easy, as one simply clicks on the URL and then clicks on the in links at the bottom. However, sometimes it might not be that simple. An infinite linking issue, for instance, might result in a situation where one needs to view the in links of in links. This is where the crawl path report comes in handy. This is an extremely useful tool that many have not yet discovered. Simply go to the URL, go to export and then click on the crawl path report.
The SEO spider was set up with the user in mind, though. It has numerous options, but all the default settings of the options are set to do exactly what one would most likely want to do. Using it is easy. Just like any other tool, all one has to do is load the URL and hit start. However, the difference between it and other tools like Xenu is the fact that it returns a lot more information. Depending on the server, it also crawls much faster than the typical tool.
Here are some of the information sections or types of information one might get when they use the SEO spider
- Internal pages: a summary of all the pages and other resources like images
- Any external links to the site
- Response codes: it shows response codes as well as redirected URLs for every source
- URI: URL details
- Page titles: the details on all the page titles on the site
- Meta keywords: details on all the site’s meta keywords
- H1: the site’s h1 tag details
- H2: details on the H2 tags on the site
- Images
- Directives: information on such meta tag directives like refresh and robots
- Meta descriptions
- Custom: the custom source code searches of the site
All 12 sets of information will normally have their own reports. There might be some duplication among the reports, but what most people consider important is the fact that every report has its own filters, designed carefully for the kind of final task one would want to do with the information. For every category, it is possible to drill down to an even deeper level of data.
[no_toc]
