Simple ways to a Robot file

hoangvu

New member
Joined
Jun 6, 2012
Messages
1,888
Points
0
Create Robots.txt file is a way to speak directly with the search engines when they visit your website. These spiders are simply one program has been programmed to perform one of the commands given

Therefore, good communication with this spider is very useful job, you probably do not want spider to visit a certain web page or folder in your website. Or you want to help control the frequency of visits to spider your website.
A simple robots.txt file as shown below, the first line of the User-Agent: *: This line describes the search engine should follow the commands below, here you are marked *, which means all search engine must follow the rules below. For example:

Code:
User-Agent: *
Disallow: / images /
Disallow: / directory / file.html
Dissallow: / images / this means that you do not allow all search engines to visit your image folder for images with SEO does not mean anything.

Disallow: / directory / file.html this means that you do not allow all search engines without reading file.html.

After creating the robots.txt file, you upload this file to the root directory of the webiste.
 
Older threads
Replies
0
Views
1,832
Replies
41
Views
12,853
Replies
0
Views
1,839
Replies
0
Views
1,920
Newer threads
Replies
24
Views
12,153
Replies
44
Views
16,472
Replies
0
Views
2,726
Replies
7
Views
4,165
Latest threads
Replies
0
Views
21
Replies
0
Views
19
Replies
4
Views
56
Recommended threads
Replies
8
Views
2,339
Replies
2
Views
2,833
Replies
0
Views
2,354
Replies
3
Views
1,077

Latest postsNew threads

Referral contests

Referral link for :

Popular Contributors - Past 30 Days

Sponsors

Latest Blog ArticlesMost Viewed Threads

Popular tags

You are using an out of date browser. It may not display this or other websites correctly.
You should upgrade or use an alternative browser.

Top