How to check robots.txt on site?

hoangvu

New member
Joined
Jun 6, 2012
Messages
1,889
Points
0
Hi Friends..


If I would like to check whether a website has robots.txt, how would I check it?
 

Top Niche Websites

New member
Joined
Jul 11, 2013
Messages
81
Points
0
Just type this into your browser and see if you get a result

[URLnf=http://domain/robots.txt]http://domain/robots.txt[/URLnf]

just change domain to whatever site you are checking

Michael
 

psavsnacks

New member
Joined
Mar 11, 2014
Messages
15
Points
0
simple way to check robots.txt file....

type home page website url/robots.txt
 

TejasWebHosting

New member
Joined
Nov 9, 2014
Messages
33
Points
0
It is important to remember that just because you can browse the site, doesn't always mean everything is okay....

I would recommend using either Bing or Google's Webmaster Tools service to check it. They have a tool that allows you to confirm if it is visible or not with the click of a button.
 
Older threads
Newer threads
Latest threads
Replies
0
Views
56
Replies
1
Views
126
Replies
1
Views
84
Recommended threads
Replies
3
Views
2,071
Replies
4
Views
3,598
Replies
2
Views
3,021
Replies
0
Views
1,797

Latest postsNew threads

Referral contests

Referral link for :

Sponsors

Latest Blog ArticlesMost Viewed Threads

Popular tags

You are using an out of date browser. It may not display this or other websites correctly.
You should upgrade or use an alternative browser.

Top