Login and retrieve

PHP programming forum. Ask questions or help people concerning PHP code. Don't understand a function? Need help implementing a class? Don't understand a class? Here is where to ask. Remember to do your homework!

Moderator: General Moderators

Post Reply
bladecatcher
Forum Commoner
Posts: 67
Joined: Sat Mar 12, 2005 12:50 am

Login and retrieve

Post by bladecatcher »

G'day All,
I'm trying to automate a download from a site where I'm a paid member.

To do this I must log in and then fill out a form (see below).

The site is MS using a VB_Script.

Once I figure out how to log in and the url to download the data, I think I have worked out the rest from the php manual.

My questions are:
How can I log in using php run from my web server?

What is the URL to download the data for today?

-----------------------------
LOGIN FORM
------------

Code: Select all

<BODY class=Body2 onload=document.frmLogin.memberID.focus();>
<FORM name=&quote;frmLogin&quote; action=&quote;https://www.xxxx.com.au/mb_login_page.asp?_sn=ED4EEE9E-7E68-4D97-8B70-E96EB1C20895&_c=&_dt=17%2F03%2F2005+10%3A12%3A01+AM&popUpType=Login&quote; method=&quote;POST&quote;>

<INPUT type=hidden name=&quote;GoBack&quote; value=&quote;&quote;>

Member ID

<INPUT type=&quote;text&quote; name=&quote;memberID&quote; size=&quote;20&quote; class=&quote;clsFieldData&quote; value=&quote;myname&quote;>

<br><br>

Password

<input type=&quote;password&quote; name=&quote;pswd&quote; size=&quote;20&quote; class=&quote;clsFieldData&quote; value=&quote;xyz123&quote;>

<br><br>

<INPUT TYPE=&quote;checkbox&quote; NAME=&quote;remember_me&quote; VALUE=&quote;Y&quote; checked>

remember my details

<INPUT class=&quote;clsBtnGo&quote; type=submit name=&quote;subLogin&quote; value=&quote;Login&quote;>

  

<INPUT class=&quote;clsBtnGo&quote; type=submit name=&quote;subCancel&quote; value=&quote;Cancel&quote;>

</FORM>
------------------------------------------------------
DOWNLOAD FORM
-------------------

Code: Select all

<!-- http://www.xxx.com.au/MB_Download_Price.asp -->

<!-- 
http://www.xxx.com.au/MB_download_price_show.asp?mode=all&oneday=20&onemon=07&oneyear=2005&dataformat=metastock&B1=Get+Historical+Data
http://www.xxx.com.au/MB_download_price_show.asp?mode=all&oneday=20&onemon=07&oneyear=2005&dataformat=metastock

Referring URL: http://www.xxx.com.au/Shares/default.asp?_sn=568D09AF-D6DA-4BC7-867C-0618B0F3C7D9&_c=71290&_dt=21%2F07%2F2005+3%3A42%3A70+PM

 -->


All Securities -  Only last 4 weeks available.
<br><br>

<FORM name=&quote;AllSec&quote; method=POST action=&quote;http://www.xxx.com.au/MB_download_price_show.asp?mode=all&quote; target=&quote;_blank&quote; <!-- onsubmit=&quote;return ValidateAll(this)&quote; -->>

Date
 
<input type=text name=oneday size=2 maxlength=2 value=01>/
<input type=text name=onemon size=2 maxlength=2 value=07>/
<input type=text name=oneyear size=4 maxlength=4 value=2005>
(DD/MM/YYYY)

<br><br>

Data Format:
<select name=dataformat class=&quote;clsFieldData&quote;>
<option value=metastock><FONT FACE=&quote;Arial&quote; SIZE=2>MetaStock</FONT>
<option value=ezy><FONT FACE=&quote;Arial&quote; SIZE=2>Ezy Chart</FONT>
<option value=ezy2><FONT FACE=&quote;Arial&quote; SIZE=2>Ezy Chart (Indices * 100)</FONT>
<option value=insight><FONT FACE=&quote;Arial&quote; SIZE=2>Insight Trader</FONT>
<option value=insight2><FONT FACE=&quote;Arial&quote; SIZE=2>Insight Trader (Indices * 100)</FONT>

<option value=metastockvol><FONT FACE=&quote;Arial&quote; SIZE=2>MetaStock (Vol % 100)</FONT>
</select>

<br><br>

<input type=submit name=B1 value=&quote;Get Historical Data&quote; class=&quote;clsBtnGo&quote;>

</FORM>
d11wtq | Please use

Code: Select all

or

Code: Select all

tags where required!   [/color]
bladecatcher
Forum Commoner
Posts: 67
Joined: Sat Mar 12, 2005 12:50 am

Login & retrieve data

Post by bladecatcher »

G'day Guys,

Do you require more information?

Is the login process just a matter of submitting the name and password contained in a url?

then retrieve the serial number?

then submit the data request in the correct format?

tia
build
josh
DevNet Master
Posts: 4872
Joined: Wed Feb 11, 2004 3:23 pm
Location: Palm beach, Florida

Post by josh »

I think a lot of people on this forum are going to question whether what you are doing is legit or not, as a lot of the members here have strong ethics, and people have posted a lot before about downloading entire image galleries via php and stuff like that, but anyways what do you mean by how do you login? Is there a specific question, or are you at just a total loss?

For a start try opening a socket to the url that handles logins, and sending memberId with your username and send your password and send whatever other variables their login form utilizes, you would then have to read in the response and parse it for any cookies they are sending, then you would subsequently place a request to the download page, passing along any cookie data to let the server know you're logged in, then parse the response for the url you want and you said you already know how to download it once youve got the URL

Hope I can help with anything else,
Good luck
timvw
DevNet Master
Posts: 4897
Joined: Mon Jan 19, 2004 11:11 pm
Location: Leuven, Belgium

Post by timvw »

Tools like wget are build for this...
bladecatcher
Forum Commoner
Posts: 67
Joined: Sat Mar 12, 2005 12:50 am

appreciate your replies

Post by bladecatcher »

Quite legit guys, I'm a paid up member and have permission to automate the download process and permission to share the method I use.

I was going to use something like:

Code: Select all

$wpage = fopen(&quote;https://www.xxxx.com.au/mb_login_page.asp?_sn=ED4EEE9E-7E68-4D97-8B70-E96EB1C20895&_c=&_dt=17%2F03%2F2005+10%3A12%3A01+AM&popUpType=Login&quote;, &quote;r&quote;);
then parse for the stuff I need, that's why I wanted help working out the URL, the one I guessed was wrong.

Sockets huh? OK I've started with http://au.php.net/sockets

any other suggestions appreciated.

GNU wGet looks interesting too, might be a better answer but both will be educational.

Thank you both.
build
josh
DevNet Master
Posts: 4872
Joined: Wed Feb 11, 2004 3:23 pm
Location: Palm beach, Florida

Post by josh »

Regular expressions will help you find the URL once you are logged in.
timvw
DevNet Master
Posts: 4897
Joined: Mon Jan 19, 2004 11:11 pm
Location: Leuven, Belgium

Post by timvw »

I've found the http://simpletest.sourceforge.net browser to be quite handy too ;))

I use it to log in on a website, click a few links, post a form automagically...
http://timvw.madoka.be/programming/php/smscity.php.txt
User avatar
Ambush Commander
DevNet Master
Posts: 3698
Joined: Mon Oct 25, 2004 9:29 pm
Location: New Jersey, US

Post by Ambush Commander »

Most of the time, authentication is done via cookies. For instance, I use a Fetch_WikipediaRecentChanges class to automatically grab my Watchlist every five minutes. Wikipedia, when you select Remember Me, sets this cookie that you simply have cURL or SimpleTest's simple browser present during the automated downloading.

So, your first job is to figure out how the website authenticates users.
Post Reply