All-In-One Search Engines - Do they steal result pages?
Moderator: General Moderators
All-In-One Search Engines - Do they steal result pages?
Looking at DogPile.com I was wondering if it just steals results from the various sources and compiles them into one list of results just by grabbing the content of search engine result pages. When processing your search does their script visit Google, Ask, Yahoo, and whatever else and just pull the results from the HTML code and parse it accordingly? It doesn't seem like this would be legal if they did. Could anyone shed some light on this?
I thought it would be interesting to create a mini-search engine and then I see that some "search engines" just gather them from other engines, so it made me wonder. I could pretty easily make something in php to steal content from Google and parse it, although it may be easy it's most likely not legal, let alone right.
- daedalus__
- DevNet Resident
- Posts: 1925
- Joined: Thu Feb 09, 2006 4:52 pm
- John Cartwright
- Site Admin
- Posts: 11470
- Joined: Tue Dec 23, 2003 2:10 am
- Location: Toronto
- Contact:
These sites prey on those who are unaware of googles existance, some claiming to use multiple search sites prey on those too lazy to use them individually, or just to get the "wow, they search on all of those and put into one page!!11" factor.
The money is in the advertisement. It is also a ploy to track peoples search habits, find out what is popular, then increase advertising prices on the popular pages.
The money is in the advertisement. It is also a ploy to track peoples search habits, find out what is popular, then increase advertising prices on the popular pages.
- Chris Corbyn
- Breakbeat Nuttzer
- Posts: 13098
- Joined: Wed Mar 24, 2004 7:57 am
- Location: Melbourne, Australia
Well, google "offer" their search services to other companies. From our point of view their are literally thousands of search engines (webistes). In reality, there are only a handful. Most search engine sites are using a more commercial source for their results.
http://searchenginewatch.com/showPage.html?page=2156401
http://searchenginewatch.com/showPage.html?page=2156401
Google offers their search to webmasters who wish to place on their site either a search form which links to Google, or an offsite semi-branded page. They also offer their API with limited queries per day and with limited non-commercial use. Beyond that they only offer partnered search options to large, high traffic services such as AOL (and Google has stake in AOL). Perhaps sites like DogPile have an agreement with the big search engines that they will attract the users who like the combined search, and give money to the engines they grab results from, and make some money themselves? I'm not sure, but I'm curious. I can't imagine them just pulling results without some sort of agreement.
EDIT: The search engine watch page does make it clear that they have partnerships and it appears that they buy the results from the major search engines.
EDIT: The search engine watch page does make it clear that they have partnerships and it appears that they buy the results from the major search engines.
- Ambush Commander
- DevNet Master
- Posts: 3698
- Joined: Mon Oct 25, 2004 9:29 pm
- Location: New Jersey, US
Search engines are all about making information you want to find easier to find. Even ripping off results from multiple search engines might be effective... if it turns out that you're more likely to get useful results on those engines.
Nevertheless, I have never used dogpile before (although I have had it recommended to me before).
Nevertheless, I have never used dogpile before (although I have had it recommended to me before).
- RobertGonzalez
- Site Administrator
- Posts: 14293
- Joined: Tue Sep 09, 2003 6:04 pm
- Location: Fremont, CA, USA
There is nothing that compares to google, in my opinion. The only thing that comes close is dmoz, but that doesn't have the sponsored listings so people that actually are willing to spend money to reach can't reach you on it.
As for who gets the money from advertising, whether someone pays google for you to find them, or someone pays you so you can pay google to help you let the user find the payer, google still inherits the earth. Their site is easy to use, so I use it and don't plan on going to a combination, multi-search engine site any time soon.
As for who gets the money from advertising, whether someone pays google for you to find them, or someone pays you so you can pay google to help you let the user find the payer, google still inherits the earth. Their site is easy to use, so I use it and don't plan on going to a combination, multi-search engine site any time soon.