How to check if a URL is online...

[Zyphr]

Hey guys,

I'm new here but plan to become a regular! I'll leave my introduction to the intro's section though.

My issue at hand is that I need to be able to check a long list of URL for a certain criteria. Ill show you what I need to do, what I've done so far, and what I really could do with some help with! Then you can either move to the next thread or put me on the right track

What I need to know

I need to know whether the site is currently online or offline, which is pretty simple. However I need it to be able to tell the difference between online and a holding page. Preferably I'd like it to class all sites with a holding page as offline also.

What I have so far

Well so far I have ready many different websites with many different implementations of code to do the online offline bit. The issue I'm having it the holding page bit. What I think needs to happen is to some how get the code to search the sites source code for certain keywords to determine whether it is a holding page or not.

The best code I came across was the below, however would be happy for your professional opinions for alternatives

Code: Select all

<?php

function Visit($url){

$agent = "Mozilla/4.0 (compatible; MSIE 5.01; Windows NT 5.0)";$ch=curl_init();

curl_setopt ($ch, CURLOPT_URL,$url );

curl_setopt($ch, CURLOPT_USERAGENT, $agent);

curl_setopt ($ch, CURLOPT_RETURNTRANSFER, 1);

curl_setopt ($ch,CURLOPT_VERBOSE,false);

curl_setopt($ch, CURLOPT_TIMEOUT, 5);

curl_setopt($ch,CURLOPT_SSL_VERIFYPEER, FALSE);

curl_setopt($ch,CURLOPT_SSLVERSION,3);

curl_setopt($ch,CURLOPT_SSL_VERIFYHOST, FALSE);

$page=curl_exec($ch);

//echo curl_error($ch);

$httpcode = curl_getinfo($ch, CURLINFO_HTTP_CODE);

curl_close($ch);

if($httpcode>=200 && $httpcode<300) return true;

else return false;

}

if (Visit("http://www.google.com"))

echo "Website OK"."n";

else

echo "Website DOWN";

?>

If you know where i should be looking to solve my issues, i would be extremely grateful

Thanks in advance,

Zyphr

[ghost107]

If you want to know if a site is online or ofline you could use fsockopen():
http://php.net/manual/en/function.fsockopen.php

I usually use sockets, plus HTTP /1.1.

Code: Select all

$fp = fsockopen("http://www.example.com", 80, $errno, $errstr, 5);
if(!$fp){
    echo "offline";
}else{
   echo"online";
    $out = "GET / HTTP/1.1\r\n";
    $out .= "Host: www.example.com\r\n";
    $out .= "Connection: Close\r\n\r\n";         //Connection: Open
    fwrite($fp, $out);
    //process file
   fclose($fp);
}

http://www.w3.org/Protocols/rfc2616/rfc2616.html

You can use ajax + php too.

[Zyphr]

Thanks for your reply!

Annoyingly enough, I have been told I must do this in javascript. Now my PHP was limited, however javascript is non existent...

Any idea how to get javascript to check if a site is online or offline, and then if online run Regex to determine if it is a holding page or not?

Tad lost here...

[ghost107]

If you want to check status of an page try using ajax, for more info read about ajax:

Code: Select all

httpRequest.onreadystatechange = ProcessMessage; 
httpRequest.open('GET', "http://www.example.com"); 
httpRequest.send(); 

function ProcessMessage() { 
    if (httpRequest.readyState === 4  && httpRequest.status === 200 ) { 
          alert('Online'); 
           //process message httpRequest.responseText
    } else { 
        alert('Offline.'); 
    } 
}

[tgoe]

@ghost107
I think you just accidentally the Same Origin Policy.

---

Looks like you're trying to write a domain squatter bot. JavaScript is neither needed nor helpful here. But I suppose you could do it with node or seed or something. Any details on why js is a requirement?

[tremor77]

This would probably be best suited for a compiled php program, rather than a webpage script. And you are going through the effort of writing PHP for an executable you may as well just do in Perl or Python.

[ghost107]

@ghost107
I think you just accidentally the Same Origin Policy.

Sorry about that, I never used ajax outside the same domain(Thanks for reminding me).