Testing for crawlability

denverdonate

New member
Joined
Dec 20, 2018
Messages
26
Points
0
For some reason Google's tools in Google Webmaster Tools aren't always able to run right then. Like if I am shown my robots.txt is blocking Google Bot and I fetch the URL, it schedules it, it's not always immediate. Do any of you use other tools for this that can test for crawlability with GoogleBot?
 

JohnWeb

Member
Joined
Apr 6, 2019
Messages
116
Points
18
Explain what you mean in more details. What do you mean "fetch the URL"? It schedules what?
 

jeffscott

Member
Joined
Dec 18, 2017
Messages
46
Points
6
For some reason Google's tools in Google Webmaster Tools aren't always able to run right then. Like if I am shown my robots.txt is blocking Google Bot and I fetch the URL, it schedules it, it's not always immediate. Do any of you use other tools for this that can test for crawlability with GoogleBot?
Google search console will help you a lot in this kind of situation, because now they are continuously updating their functions, the URL Inspection feature will help you to test crawlability.
 
Newer threads
Replies
3
Views
1,729
Replies
6
Views
7,325
Replies
6
Views
2,192
Replies
5
Views
2,380
Latest threads
Replies
1
Views
81
Replies
1
Views
176
Replies
4
Views
384
Replies
11
Views
526
Replies
2
Views
228
Recommended threads
Replies
7
Views
2,195
Replies
2
Views
2,708
Replies
17
Views
8,520

Latest postsNew threads

Referral contests

Referral link for :

Sponsors

Popular tags

You are using an out of date browser. It may not display this or other websites correctly.
You should upgrade or use an alternative browser.

Top