Getting google URL Link's from search

PHP programming forum. Ask questions or help people concerning PHP code. Don't understand a function? Need help implementing a class? Don't understand a class? Here is where to ask. Remember to do your homework!

Moderator: General Moderators

Post Reply
ojsimon
Forum Newbie
Posts: 24
Joined: Sat Mar 31, 2007 5:01 am

Getting google URL Link's from search

Post by ojsimon »

Hi
I would like to enable my users to search and the script return just the links not description or title. How can i do this using google API or not.

Thanks
Olie
rturner
Forum Newbie
Posts: 24
Joined: Sun Nov 04, 2007 1:39 pm

Post by rturner »

This includes all links on the page so more work is necessary to strip off internal links.

Code: Select all

<?php
function curl_it($url, $user_agent, &$page, &$plen){
$ch = curl_init();
curl_setopt ($ch, CURLOPT_URL,$url);
curl_setopt ($ch, CURLOPT_USERAGENT, $user_agent);
curl_setopt ($ch, CURLOPT_HEADER, 0);
curl_setopt ($ch, CURLOPT_RETURNTRANSFER, 1);
$page = curl_exec ($ch);
$plen=strlen($page);
curl_close ($ch);
}

$user_agent = "Mozilla/4.0 (compatible; MSIE 7.0; Windows NT 5.1; .NET CLR 1.1.4322)";
$url = "http://www.google.com/search?hl=en&q=some+stuff&btnG=Google+Search";
# search for                                some stuff

curl_it($url, $user_agent, $page, $plen);
if (!$page) {
        echo "Process failed...";
        exit;
}

// build array of links
$needle='/<a[^>]*?href=[\'"](.*?)[\'"][^>]*?>(.*?)<\/a>/';
preg_match_all($needle, $page, $hyper);
$hyper = $hyper[1];
#hyper = $hyper[0];  entire link

print "<pre>";
print_r($hyper);
print "</pre>";
?>
User avatar
feyd
Neighborhood Spidermoddy
Posts: 31559
Joined: Mon Mar 29, 2004 3:24 pm
Location: Bothell, Washington, USA

Post by feyd »

That's also a good way to get blacklisted by Google for abuse. ;)
rturner
Forum Newbie
Posts: 24
Joined: Sun Nov 04, 2007 1:39 pm

Post by rturner »

Thats true, I would use Google's Ajax api for custom search returns on a website. Never scrape directly from a domain if you want to play with scraping at all.
ojsimon
Forum Newbie
Posts: 24
Joined: Sat Mar 31, 2007 5:01 am

Post by ojsimon »

How do i do this with the api?
Thanks
User avatar
feyd
Neighborhood Spidermoddy
Posts: 31559
Joined: Mon Mar 29, 2004 3:24 pm
Location: Bothell, Washington, USA

Post by feyd »

Google has pretty darn good documentation for developers on its site(s).
Post Reply