Page 1 of 1
All-In-One Search Engines - Do they steal result pages?
Posted: Tue Jul 25, 2006 8:07 pm
by kaYak
Looking at DogPile.com I was wondering if it just steals results from the various sources and compiles them into one list of results just by grabbing the content of search engine result pages. When processing your search does their script visit Google, Ask, Yahoo, and whatever else and just pull the results from the HTML code and parse it accordingly? It doesn't seem like this would be legal if they did. Could anyone shed some light on this?
Posted: Tue Jul 25, 2006 9:11 pm
by LiveFree
If those multi-engine searchers are, they could get into some big <span style='color:blue' title='I'm naughty, are you naughty?'>smurf</span>.
Posted: Tue Jul 25, 2006 9:13 pm
by feyd
I believe most of them do. By permission or not, I can't say. They may also cache the resultant data to ease the bandwidth needs.
Posted: Tue Jul 25, 2006 9:41 pm
by kaYak
I thought it would be interesting to create a mini-search engine and then I see that some "search engines" just gather them from other engines, so it made me wonder. I could pretty easily make something in php to steal content from Google and parse it, although it may be easy it's most likely not legal, let alone right.
Posted: Tue Jul 25, 2006 9:46 pm
by Burrito
what would be the point of that anyway?
Posted: Tue Jul 25, 2006 9:57 pm
by kaYak
The point of stealing from Google?
Posted: Tue Jul 25, 2006 10:05 pm
by Burrito
no, the point of creating a search engine that just displays results from google...why not just search on google?
Posted: Tue Jul 25, 2006 10:33 pm
by daedalus__
Selling ads on your search and giving yourself money rather than Google.
Posted: Tue Jul 25, 2006 11:01 pm
by John Cartwright
Burrito means, if I want to search google why would I go to some other site to google when I can just go to the real thing?
Posted: Wed Jul 26, 2006 6:43 am
by Jenk
These sites prey on those who are unaware of googles existance, some claiming to use multiple search sites prey on those too lazy to use them individually, or just to get the "wow, they search on all of those and put into one page!!11" factor.
The money is in the advertisement. It is also a ploy to track peoples search habits, find out what is popular, then increase advertising prices on the popular pages.
Posted: Wed Jul 26, 2006 6:48 am
by Chris Corbyn
Well, google "offer" their search services to other companies. From our point of view their are literally thousands of search engines (webistes). In reality, there are only a handful. Most search engine sites are using a more commercial source for their results.
http://searchenginewatch.com/showPage.html?page=2156401
Posted: Wed Jul 26, 2006 5:07 pm
by kaYak
Google offers their search to webmasters who wish to place on their site either a search form which links to Google, or an offsite semi-branded page. They also offer their API with limited queries per day and with limited non-commercial use. Beyond that they only offer partnered search options to large, high traffic services such as AOL (and Google has stake in AOL). Perhaps sites like DogPile have an agreement with the big search engines that they will attract the users who like the combined search, and give money to the engines they grab results from, and make some money themselves? I'm not sure, but I'm curious. I can't imagine them just pulling results without some sort of agreement.
EDIT: The search engine watch page does make it clear that they have partnerships and it appears that they buy the results from the major search engines.
Posted: Wed Jul 26, 2006 5:28 pm
by Ambush Commander
Search engines are all about making information you want to find easier to find. Even ripping off results from multiple search engines might be effective... if it turns out that you're more likely to get useful results on those engines.
Nevertheless, I have never used dogpile before (although I have had it recommended to me before).
Posted: Wed Jul 26, 2006 5:33 pm
by RobertGonzalez
There is nothing that compares to google, in my opinion. The only thing that comes close is
dmoz, but that doesn't have the sponsored listings so people that actually are willing to spend money to reach can't reach you on it.
As for who gets the money from advertising, whether someone pays google for you to find them, or someone pays you so you can pay google to help you let the user find the payer, google still inherits the earth. Their site is easy to use, so I use it and don't plan on going to a combination, multi-search engine site any time soon.