How to use the meta robots tag

[hoangvu]

Hi there,

I disabled google and other search engines crawl and indexing my site by code in robots.txt

User-agent: *
Disallow: /

but in my html template I still put

<meta name="robots" content="index,follow" />

thus Google crawled and indexed my web pages.

just want to know How to use/control the meta robots tag for Google index or following my web pages?

Thanks in advanced!

 

[Jack London]

Hello,

If you disable spiders crawl your web pages then you can use this robot tag:

<meta name="robots" content="index, nofollow">

Google spider will not index page, only look at page.

<meta name="robots" content="noindex, follow">

Google spider will not index/crawl pages, not look at other pages from this page.

<meta name="robots" content="noindex, nofollow">

You can switch between index/noindex and follow/nofollow depends on your requirements on web pages to control spiders crawl/index or not.

Hope it's useful to you.