How to set up a robots.txt to control Google spider crawl my site?

johnyplex

New member
Joined
Jan 23, 2015
Messages
30
Points
0
Hi everyone,

I have just established a new site and I don't want google spider crawl some parts (folder, files) on my site
Are threre any ways to set up a robots.txt to control Google spider crawl my site?

for example:
I want google spider crawl all my site but not for /payment/ folder and /private_data/ folder
I don't want google spider crawl a image file like my_image_file.jpg
I don't want google spider crawl all file start with mydata

Any suggestions?
 
Older threads
Replies
6
Views
3,527
Replies
8
Views
3,374
Replies
3
Views
3,239
Replies
4
Views
3,127
Latest threads

Referral contests

Referral link for :

Sponsors

Popular tags

You are using an out of date browser. It may not display this or other websites correctly.
You should upgrade or use an alternative browser.

Top