Why cURL Fails To fetch Page ?

[sunny_pro]

Folks,

Why is cURL failing to fetch the page ? All this time it worked.
Echoes "Page fetching problem!"

<?php 

//Required PHP Files.
include 'config.php';
include 'header.php';

//1). Set Banned Words.
$banned_words = array("asshole", "nut", "bullshit");

$url = "http://devshed.com";
// 2). $curl is going to be data type curl resource.
$curl = curl_init();

// 3). Set cURL options.
curl_setopt($curl, CURLOPT_URL, "$url");
curl_setopt($curl, CURLOPT_SSL_VERIFYPEER, false);
curl_setopt($curl, CURLOPT_RETURNTRANSFER, true);

// 4). Run cURL (execute http request).
$result = curl_exec($curl);

if (curl_errno($curl))
{
	echo 'Error:' . curl_error($curl);
}

$response = curl_getinfo( $curl );

//If page is fetched then replace banned words found on page.
if($response['http_code'] == '200' )
{   
	$regex = '/\b';
	$regex .= implode('\b|\b', $banned_words);
	$regex .= '\b/i';
	$substitute = 'BANNED WORD REPLACED';
	$clean_result = preg_replace($regex, $substitute, $result);
	//Present the banned words filtered webpage.
	echo $clean_result;
}
else
{
	//Show error if page fetching failed.
	echo "Page fetching problem!";
	exit();
}


?>

 

[Rob Whisonant]

First thing to do is echo the http_code and see what is being returned.

 

[sunny_pro]

sunny_pro

Added this line:

echo "$response[http_code]";

Result I get is this: 403.

Googling, I see that the page devshed.com is forbidden. Since I can view the page on y browser and not manage to get cURL to fetch it then I'm guessing devshed.com has put a measurement in place to foil proxies.
Switched the url to another site and that got fetched.
last night, even google did not get fetched and so I suspected maybe I accidently deleted something from my code but could not spot the deletion. Hence, opened this thread to see if anyone else spots it.
Now, I understand, both devshed.com and google were foiling the cURL fetch or proxy fetch.

How well did I do ?

I now get error that variable $url is undefined on line 72.

Notice: Undefined variable: url in C:\xampp\htdocs\......

If you check line 72, it says between double quotes:

$url = "http://devshed.com";

Even if I change the url to a url who's page cURL is able to fetch, I still get the same error.
This does not work either, with single quotes:

$url = 'http://devshed.com';

This is very very strange! If the $url variable has not been defined then how is cURL able to fetch the page who's url is on the $url variable value ?
Even though page gets fetched, I still see the error! Weird!

 

[dineshsma]

You are getting the notice because you are using double quotes for the variable in the first line of CURL options. While using variable, you don't have to use double quotes.

Old code:

curl_setopt($curl, CURLOPT_URL, "$url");

New code:

curl_setopt($curl, CURLOPT_URL, $url);

For your main CURL issue, the 403 errors can be caused by many reasons. But the exact reason can be found in server logs. As you don't have access to the server, you can experiment different things. If the website is working fine when access through the browser then its not server side issue. I think the server is configured to restrict some useragent. I will suggest you to set CURL to use your browser's useragent and give it a try.

curl_setopt($ch,CURLOPT_USERAGENT,$_SERVER['HTTP_USER_AGENT']);