Page 1 of 1

Is this legal?

Posted: Sun Mar 04, 2007 10:07 pm
by wildwobby
Is it legal to make a website where one part of it involves using curl+regex to scrape stock quotes from google finance?

Posted: Sun Mar 04, 2007 10:19 pm
by feyd
Check their terms of service: http://www.google.com/terms_of_service.html

Since I'm not seeing an API on their code pages, I'll guess they may take issue in scraping.

I probably would say it's not illegal per se, but not very nice or cool.

Posted: Sun Mar 04, 2007 11:28 pm
by wtf
why boter with scraping/regex when you can http://xmethods.net/

Posted: Mon Mar 05, 2007 9:37 am
by veridicus
IANAL, but can tell you that Google has a license to the information either directly or indirectly from the stock markets. Stock markets aren't public services and their data is owned by them. Therefore you can be pretty sure there are restrictions to scraping any of their data.

Posted: Mon Mar 05, 2007 10:39 am
by Maugrim_The_Reaper
Actually the stock market doesn't necessarily own the data - remember it's an open regulated market so the data *must* be published by the stock exchange. Chances are the restrictions concern access to their underlying systems. Just as Yahoo can license access to their system, but not ownership of the data being served.

In any case, scrapping is not cool. Yes, it would work. Yes, it's probably legal. But try to find a free data source before wasting Yahoo's bandwidth for your own gain.

Posted: Mon Mar 05, 2007 10:40 am
by Jenk
But try to find a free data source before wasting Yahoo's bandwidth for your own gain.
Is that not the illegal bit? It's using Yahoo's/Google's/Someone else's service without their consent, and presumable without credit?

Posted: Mon Mar 05, 2007 10:49 am
by RobertGonzalez
veridicus wrote:IANAL
For those that do not speak AOL (Abbreviated Online Language), this means I Am Not A Lawyer. I know it is a lot to spell out, but hey, I can take one for the team...

As far as illegal... If it is data coming from a source that has a registration or fee charged for the service, you are violating their user agreement. Even if you are not violating a user agreement or breaking the law, it is still being rude to gank someone else's content without their permission. That truly is just wrong.

Posted: Mon Mar 05, 2007 11:15 am
by Jenk
I would have thought there is a user agreement even for sites that do not have a registration/fee.

Posted: Mon Mar 05, 2007 3:59 pm
by wildwobby
hmm, well im not expecting much trafic, but what if I made the php randomly choose which site to scrape from (msn, yahoo, google, cnbc, etc...) for every query therby distributing the "load"...


Already have a little mini-test:
http://rwphoto.thepeopleshost.com/gf.php?q=adbe

Posted: Mon Mar 05, 2007 4:35 pm
by s.dot
wildwobby wrote:hmm, well im not expecting much trafic, but what if I made the php randomly choose which site to scrape from (msn, yahoo, google, cnbc, etc...) for every query therby distributing the "load"...


Already have a little mini-test:
http://rwphoto.thepeopleshost.com/gf.php?q=adbe
I'd say that's just digging you a deeper hole :P

Posted: Mon Mar 05, 2007 8:05 pm
by wildwobby
aight I wonder how I can legally get it... I made it ajax-powered too.

Posted: Mon Mar 05, 2007 8:07 pm
by feyd
Legally, you need to subscribe to the various exchanges' feeds, typically.