Solving Soft 404's - Proven Strategy
Category Management (Google’s Perspective)
Soft 404 is a relatively newer challenge faced by webmasters. Soft 404 in easy terms means "Google crawler thinks the page link is broken and the website is showing different content to cover it up". This is more common and concerning for online shopping websites, where there are lots of category pages which goes empty when these runs on product inventory.
There are certain ways to tackle this problem. One could be to keep the SOLD OUT deals intact and just show a message stating "out of stock". But according to my recent experience, i found out that Google also detects sold out deals and moves the rank down to maintain the quality of its result. That's the reason you don't see a product detail page ranking high when you do a product search on Google.
The strategy we put in place was to Disable and Noindex a category listing page when it is out of stock automatically. Being Australia's leading marketplace we are never sure when and what product we will have or how many categories we will have to create when something new comes on our platform for selling. For instance we have a wider range of lawnmowers, even bigger than some of the largest Brick & Morter retailers in Australia. Same is the case with electric and petrol pressure washers.
Auto Disable Rule
If there are 0 SKU’s in a category landing page, the landing page is automatically disabled which have following impact on the website.
Link is removed from navigation
Link is removed from sitemap_subcategories.xml (which is submitted to Google to index)
Link is taken out of website's internal search pages
Google considers empty categories to be SOFT 404, which means the link is broken and we are showing something irrelevant on purpose. To tackle this we are going to
Noindex/ Nofollow flag have following impact on the website.
Moving a category
Whenever moving one category to another top level following considerations should be made
The slug is not same as any other subcategory under the top level you are moving to (this will break the page)
Title, META Description, H1 & Category Description is transferred and tweaked (if required)
And do following for the old spot
Don’t reuse the tagid (that will leave a 404 and we will have to deal with after effects)
Put a 301 redirect to the new category link
Disable & Archive (It can also be done automatically by untagging all deals from the category)
Why we are doing this:
Google allocates a definite crawl budget to each website that it crawls. With an unhealthy website with lots of repeating broken links, soft 404’s & server errors, Google bot reduce the frequency and depth of crawl.