404 Error when checking cache of websiite

[cogxim]

My website has been indexed in google but when i check its cache its give 404 error. Why it is so i have tried every thing resubmitted sitemap, Fetch as google and many more. First it show error in some pages now 404 error showing in all pages.
My website URL is cogxim.com
Answer will be really appreciated.

 

[Shivam Tripathi]

Same problem is being faced by me also.Just you will have to update your sitemap and see your result within 15 min..try it once..

 

[Marc van Leeuwen]

Checking at the second line on your robots.txt and you need to remove it

cogxim.com/robots.txt

Remove red text and your web page will be cached by Google.

User-agent: *
[COLOR="#FF0000"]Disallow:[/COLOR]
Disallow: /finance.htm
Disallow: /contact.html

I hope it helps!

 

[cogxim]

Hi Marc
I have deleted Disallow from my robots.txt file hope google easily cache my website. Instead of Disallow can we use Allow.

 

[Marc van Leeuwen]

Marc van Leeuwen

This is what you should add to your robots.txt file

User-agent: *
Disallow: /finance.htm
Disallow: /contact.html
Disallow: /automobile.html
Disallow: /services.html
Disallow: /visage.html
Disallow: /downloads.html
Disallow: /news_events.htm
Disallow: /offshore_support.htm
Disallow: /e_commerce.htm 
Disallow: /petrogenius.htm
Disallow: /transport.htm
Disallow: /privacy_policy.htm
Disallow: /software_consultancy.htm
Allow: /

After that, resubmit your sitemap, using fetch tool and building some quality backlinks then Google will cache your site faster.
I hope it helps!

 

[cogxim]

i have resubmitted the robots.txt using allow: / hope its work.

 

[Marc van Leeuwen]

Marc van Leeuwen

For sure, It will work.
It will not work unless you still add nofollow, noindex in your source of index file. You should check around on your website if you blocked robots from crawling your websites.

 

[simicartan]

a weird thing happens to my site too. When I index as google bot, it shows up that my site has 404 errors but I cannot find where the page that has the 404 error links to to fix that, which means there is no trace of the error page at all, how can google bot still index the error pages? I know it sounds weird.

 

[cogxim]

Its Still not working Marc. I have done the changes and waited but no result found

 

[Marc van Leeuwen]

Marc van Leeuwen

Have you tried to use "Fetch as Google" for your website and what result is returned there?

After used fetch tool, you can click on Partial or Completed under status column to check details which parts on your site is blocked for robots.