In many instances, site crawl errors like the Soft 404 go unnoticed as SEO marketers put more effort on other components. Often, we bring to fore many aspects such as quality content and building links while the technical aspect is pushed to the back seat. It will be surprising to note that even top web developers are often oblivious of how bots work. In this post, we dig deeper into one of the top site crawlers, the Soft 404 Errors.
Table of Contents
What are Soft 404 errors?
You must have seen the message, 404 page not found ¬error when checking a page that is not available. The 404 HTTP response code means that the site’s server is not available. The code tells search engines and browsers that the respective page is unavailable. Therefore, the page can neither be crawled nor indexed.
What many people fail to appreciate is that the content on the Page not found message does not have any relationship to the HTTP response that the server generates. Simply because a page has given a 404 File Not Found does not automatically qualify it as a 404 page.
According to Google, it is like a Zebra with a tag indicated polar bear. Does the tag make it a bear? Of course no! The Soft 404 Error happens when a page is nonexistent (removed or deleted) and a browser is trying to reach it. The error can also happen when a non-existent page redirects browsers to pages that are irrelevant. Here, you need to grasp the fact that the web page/content is in no way related to the HTTP message generated by server
The issue with soft 404 errors
When a site returns a HTTP status code instead of 404/410 (for nonexistent page), the website’s performance in the local organic search will be affected negatively. One, when there is failure to serve the 404 Status Code, the site is simply telling search engines that a real page with the respective URL actually exist. Therefore, the URL that you have deleted is going to be crawled and indexed. The impact is waste of useful crawl budget.
Crawl budget means the number of times that a search engine sends bots to your page. Google only allocates a specific time for crawling your site before stopping and moving to a different site. Therefore, allowing Google to crawl even the non-existent pages will waste useful time that the bots could have taken to discover other useful URLs. This ultimately compromises visibility of your content. It is because of this that resolving Soft 404 errors yields significant improvement on organic searches.
To know the extent of Soft 404 errors problem, you need to use Google Webmaster Tools. Here, you should check the number of pages that were crawled and indexed. You should also look at the website’s XML sitemap to know the pages that a site has.
Resolving soft 404 not found errors
Google allows a total of 1000 URLs in its webmaster tools. After exporting the URL list to note those with errors, you have to move a step ahead to establish why they have the Soft 404 errors. The first thing that one should do is running the Soft 404 error pages on a HTTP status code. One of the best checkers is the httpstatus.io that helps to check the page status.
One of the main issues you should expect to get when diagnosing the causes of Soft 404 errors is the 302/301 redirects. Many are the webmasters who opt to redirect the deleted pages to the main home page that returned the errors. This is often confusing and very annoying for bots. The best thing is redirecting the deleted pages/out of stock pages to replacement pages. If the replacement page is not available, consider displaying the next best alternative product.
Remember that redirects should never be used to serve 404 error pages. Rather, you should serve a HTTP 404 response code when visitors request moved or deleted pages. This helps to prevent triggering a large number of Soft 404 errors while ensuring only the right pages are indexed and ranked.
Will addressing the soft 404 errors raise traffic to my site?
This is one question that website owners always ask when you recommend urgent solution to soft 404 errors. When working on a client’s ecommerce website a few years back, we noted that there were numerous Soft 404 errors compared to the total pages. The client’s website was giving soft 404 errors without providing HTTP 404 Status cod for the deleted products.
We liaised with the developer and the site owner who ensured that the server returned appropriate HTTP 404 status code together with the page not found message for all the pages of removed from the website. Once these changes were implemented, the traffic went up dramatically.
Soft 404 errors: Why technical SEO is so important
Technical SEO is one thing that many digital marketers rarely understand well. Even some top SEO experts also rarely appreciate it. When you fail to appreciate the importance of technical SEO, there is a risk of losing a lot of opportunities especially on raising organic visibility. Technical SEO helps to follow all the internal issues such as errors and prevent them from occurring. For example, technical SEO will help to address the issue of duplicate content, broken links and soft 404 errors.
