Page 1 of 1

Getting google URL Link's from search

Posted: Mon Nov 19, 2007 11:00 am
by ojsimon
Hi
I would like to enable my users to search and the script return just the links not description or title. How can i do this using google API or not.

Thanks
Olie

Posted: Mon Nov 19, 2007 7:09 pm
by rturner
This includes all links on the page so more work is necessary to strip off internal links.

Code: Select all

<?php
function curl_it($url, $user_agent, &$page, &$plen){
$ch = curl_init();
curl_setopt ($ch, CURLOPT_URL,$url);
curl_setopt ($ch, CURLOPT_USERAGENT, $user_agent);
curl_setopt ($ch, CURLOPT_HEADER, 0);
curl_setopt ($ch, CURLOPT_RETURNTRANSFER, 1);
$page = curl_exec ($ch);
$plen=strlen($page);
curl_close ($ch);
}

$user_agent = "Mozilla/4.0 (compatible; MSIE 7.0; Windows NT 5.1; .NET CLR 1.1.4322)";
$url = "http://www.google.com/search?hl=en&q=some+stuff&btnG=Google+Search";
# search for                                some stuff

curl_it($url, $user_agent, $page, $plen);
if (!$page) {
        echo "Process failed...";
        exit;
}

// build array of links
$needle='/<a[^>]*?href=[\'"](.*?)[\'"][^>]*?>(.*?)<\/a>/';
preg_match_all($needle, $page, $hyper);
$hyper = $hyper[1];
#hyper = $hyper[0];  entire link

print "<pre>";
print_r($hyper);
print "</pre>";
?>

Posted: Mon Nov 19, 2007 7:24 pm
by feyd
That's also a good way to get blacklisted by Google for abuse. ;)

Posted: Mon Nov 19, 2007 7:52 pm
by rturner
Thats true, I would use Google's Ajax api for custom search returns on a website. Never scrape directly from a domain if you want to play with scraping at all.

Posted: Wed Nov 21, 2007 9:50 am
by ojsimon
How do i do this with the api?
Thanks

Posted: Wed Nov 21, 2007 9:58 am
by feyd
Google has pretty darn good documentation for developers on its site(s).