Page 1 of 1

Domain resolving ( not WHOis ) - searched phpdn.net for 1hr.

Posted: Fri Jan 14, 2005 4:18 am
by Calimero
Ok, I realy digged for a while ( almost over 1 hour - here ) and I still dont have definite answer ( on which to start ).

I want my php to check the domain existance ( something like 1st step for starting a crawler ) - and by repeating that to get the list of existing domains.

And the other condition is to check if the site exist ( is operational ) on that domain.

I already know how to create URL generator, I need functions or snippets on how to check the domain existance ( and website presense on it ): if found - echo "HERE IT IS", else if (...) echo "DOMAIN YES, WEBSITE NO" else echo"NO WEBSITE AND NO DOMAIN"

Or just if the website is present ( I will supply the URL lists )

What would be the best idea for coding this - given that I have middle to somewhat high expirience in Php ( and JavaScript ) and MySQL - so none external API's or whatever come in question.

Thanks Ahead !

Re: Domain resolving ( not WHOis ) - searched phpdn.net for

Posted: Fri Jan 14, 2005 6:54 am
by visionmaster
# Is the domain name valid, what does nslookup say?

Code: Select all

function dns_timeout($name)
{
	if ($name == '' )
	{
	   return(0);  
	}
	
	else
	{
		$res=`nslookup -sil -timeout=3 -retry=2 $name`;
		if ( preg_match('/Non-existent domain/', $res) || preg_match('/request timed out/', $res) || preg_match('/server can''t find /', $res) || preg_match('/Can''t find /', $res) )
		{
		   return(0);  	
		} 
		else
		{
		   return(1);  
		}

	}
	
}
You may also want to try out php's gethostbyname() function. Is this the direction you would like to go? The above function would at least take care of DOMAIN YES/NO.

To the WEBSITE NO/YES question, probably have to parse the first webpage and analyse the context. Anybody have a better solution, or any tricky solution to this one?

Regards,
visionmaster

Posted: Fri Jan 14, 2005 7:27 am
by Wayne
To find out if the website exist :

Have a look at the fsockopen(), fputs() and fgets() functions in php, you could use that to send a HTTP HEAD request on the domain, check the response you get from the server. You can also do something similar in Javascript using ActiveX control on the client side (browser dependant).

Posted: Fri Jan 14, 2005 8:29 am
by visionmaster
Wayne wrote:To find out if the website exist :

Have a look at the fsockopen(), fputs() and fgets() functions in php, you could use that to send a HTTP HEAD request on the domain, check the response you get from the server. You can also do something similar in Javascript using ActiveX control on the client side (browser dependant).
Hmm, the details would interest me. Could you go into the details? Howto send a HTTP HEAD request on a domain? What responses are possible?

Thanks!

Posted: Fri Jan 14, 2005 9:05 am
by feyd
curl can also do HEAD requests.