use curl to connect asp site with username and password ?

PHP programming forum. Ask questions or help people concerning PHP code. Don't understand a function? Need help implementing a class? Don't understand a class? Here is where to ask. Remember to do your homework!

Moderator: General Moderators

bidntrade
Forum Commoner
Posts: 33
Joined: Thu Jul 02, 2009 9:54 pm

use curl to connect asp site with username and password ?

Post by bidntrade »

I have a site that has a csv feed updated everyday ...

I have a script that i use to parse the csv and update my database
each day . the problem is i must manually download the csv
and upload to my ftp server for the script to work ...

now im using :

Code: Select all

$handle = fopen("DataFeed.csv", "r");
i would like to be able to pull it directly like this

Code: Select all

$handle = fopen("http://www.sitenamehere.com/Reseller/Feeds/GetDataFeed.asp?type=csv", "r");
that way it could be run automated each night ....

the problem is the site requires me to login before i have access to the file download...

the site owner said something about :
so what you're basically looking for is

How to implement impersonation in an ASP.net application,
not sure what they was talking about ...

are there a way around this ?
Last edited by bidntrade on Sat Jul 04, 2009 12:00 pm, edited 1 time in total.
Eric!
DevNet Resident
Posts: 1146
Joined: Sun Jun 14, 2009 3:13 pm

Re: how to fopen a site that requires a username and password ?

Post by Eric! »

You can use cURL to log in. You have to have it installed on your server. Here's an example I pulled from http://www.trap17.com/index.php/Automat ... 38162.html

Code: Select all

<?php
// INIT CURL
$ch = curl_init();
 
// SET URL FOR THE POST FORM LOGIN
curl_setopt($ch, CURLOPT_URL, 'http://www.external-site.com/Members/Login.php');
 
// ENABLE HTTP POST
curl_setopt ($ch, CURLOPT_POST, 1);
 
// SET POST PARAMETERS : FORM VALUES FOR EACH FIELD
curl_setopt ($ch, CURLOPT_POSTFIELDS, 'fieldname1=fieldvalue1&fieldname2=fieldvalue2');
 
// IMITATE CLASSIC BROWSER'S BEHAVIOUR : HANDLE COOKIES
curl_setopt ($ch, CURLOPT_COOKIEJAR, 'cookie.txt');
 
# Setting CURLOPT_RETURNTRANSFER variable to 1 will force cURL
# not to print out the results of its query.
# Instead, it will return the results as a string return value
# from curl_exec() instead of the usual true/false.
curl_setopt ($ch, CURLOPT_RETURNTRANSFER, 1);
 
// EXECUTE 1st REQUEST (FORM LOGIN)
$store = curl_exec ($ch);
 
// SET FILE TO DOWNLOAD
curl_setopt($ch, CURLOPT_URL, 'http://www.external-site.com/Members/Downloads/AnnualReport.pdf');
5DOWNLOAD)
$content = curl_exec ($ch);
 
// CLOSE CURL
curl_close ($ch);
 
?>
bidntrade
Forum Commoner
Posts: 33
Joined: Thu Jul 02, 2009 9:54 pm

Re: how to fopen a site that requires a username and password ?

Post by bidntrade »

well thanks for that code but it still not working


I dont get any curl errors but i get a white page that says:

Object Moved to Here

the login page is
and the page im trying to download is :

These pages are Asp i see are there a differant way to approach this ...

the file im trying to download is a dynamicaly generated csv...
it always contains there latest info .


any help ?

thanks
User avatar
requinix
Spammer :|
Posts: 6617
Joined: Wed Oct 15, 2008 2:35 am
Location: WA, USA

Re: how to fopen a site that requires a username and password ?

Post by requinix »

Have you tried changing the code yet? Or is that a stupid question on my part?

What code are you trying to use?
bidntrade
Forum Commoner
Posts: 33
Joined: Thu Jul 02, 2009 9:54 pm

Re: how to fopen a site that requires a username and password ?

Post by bidntrade »

Code: Select all

<?php
// INIT CURL
$ch = curl_init();
 
// SET URL FOR THE POST FORM LOGIN
curl_setopt($ch, CURLOPT_URL, 'https://www.somesite.com/DealerLogin.asp');
 
// ENABLE HTTP POST
curl_setopt ($ch, CURLOPT_POST, 1);
 
// SET POST PARAMETERS : FORM VALUES FOR EACH FIELD
curl_setopt ($ch, CURLOPT_POSTFIELDS, 'username=user@user.com&password=mypass');
 
// IMITATE CLASSIC BROWSER'S BEHAVIOUR : HANDLE COOKIES
curl_setopt ($ch, CURLOPT_COOKIEJAR, 'cookie.txt');
 
# Setting CURLOPT_RETURNTRANSFER variable to 1 will force cURL
# not to print out the results of its query.
# Instead, it will return the results as a string return value
# from curl_exec() instead of the usual true/false.
curl_setopt ($ch, CURLOPT_RETURNTRANSFER, 1);
 
// EXECUTE 1st REQUEST (FORM LOGIN)
$store = curl_exec ($ch);
 
// SET FILE TO DOWNLOAD
curl_setopt($ch, CURLOPT_URL, 'http://www.somesite.com/Reseller/Feeds/GetDataFeed.asp?type=csv');
 
// EXECUTE 2nd REQUEST (FILE DOWNLOAD)
$content = curl_exec ($ch);
 
 
// CLOSE CURL
curl_close ($ch);
 
echo $content;
 
 
?>

this is what i tried and i get the page has moved message...

i tried also changing
CURLOPT_RETURNTRANSFER to 0
and i all i see is the main page with the login fields of the other site ....

this is not a standard http login ... its a Form login in a asp page ......

I checked my server and the cookie.txt is getting this info :

Code: Select all

# Netscape HTTP Cookie File
# http://curl.haxx.se/rfc/cookie_spec.html
# This file was generated by libcurl! Edit at your own risk.
 
.www.somesite.com   TRUE    /   FALSE   1404460010  I1  96084529
.www.somesite.com   TRUE    /   FALSE   1404460010  G1  1BB81767-554D-4445-8708-CA4328C633E4
.www.somesite.com   TRUE    /   FALSE   1088927210  VisitorIDNew    
.www.somesite.com   TRUE    /   FALSE   1088927210  VisitorHashNew  
.www.somesite.com   TRUE    /   FALSE   1088927210  DV1 
 

any ideas ?
Eric!
DevNet Resident
Posts: 1146
Joined: Sun Jun 14, 2009 3:13 pm

Re: how to fopen a site that requires a username and password ?

Post by Eric! »

curl can take tweaking. I'm just tossing out some ideas. Make sure your fields match the login form exactly and try some of the following:

Code: Select all

//the following might be all you need
curl_setopt($ch, CURLOPT_FOLLOWLOCATION, 1); // often not set by default,  this will recursively follow all redirect headers
 
// use this option to output all the exchanged info for trying to see what is happening
curl_setopt($ch, CURLOPT_VERBOSE, 1);
 
//sometimes the post method doesn't work for logins and this does
curl_setopt($ch, CURLOPT_USERPWD, '[username]:[password]'); // example: 'myusername:secretpass'
 
//negociate the best HTTP authentication method(s)
curl_setopt($ch, CURLOPT_HTTPAUTH, CURLAUTH_ANY);
 
// This is occassionally required to stop CURL from verifying the peer's certificate. for https connections
curl_setopt($ch, CURLOPT_SSL_VERIFYPEER, FALSE);
curl_setopt($ch, CURLOPT_SSL_VERIFYHOST,  2); //sometimes needs to be 0 or 1...
 
//use this after a curl_exec to extract http codes
//and lots of info as documented here
//http://curl.haxx.se/libcurl/c/curl_easy_getinfo.html
//not probably needed if in verbose mode for debugging
$info=curl_getinfo($ch);//get debugging info on last call
bidntrade
Forum Commoner
Posts: 33
Joined: Thu Jul 02, 2009 9:54 pm

Re: how to fopen a site that requires a username and password ?

Post by bidntrade »

i have tried every possible combination
i could think off

i have commented out and uncommented
every combination i could think of ....

the verbose mode does nothing .. nothing changes if its set to 1 or 0 .
dont really understand why the curl debug does not work .

the only information i can get is if i print the array $info

this is the script right now with the second link disabled . i was just trying to get
it to login first.

all i get with this is the websites login page ... no failed login or anything . almost like nothing is being sent . just the login page .... does not matter how i change it ...

Code: Select all

<?php
// INIT CURL
$ch = curl_init();
 
# // use this option to output all the exchanged info for trying to see what is happening
curl_setopt($ch, CURLOPT_VERBOSE, 1);
 
 
 
// SET URL FOR THE POST FORM LOGIN
curl_setopt($ch, CURLOPT_URL, 'https://www.somesite.com/DealerLogin.asp');
 
 
 
# // This is occassionally required to stop CURL from verifying the peer's certificate. for https connections
curl_setopt($ch, CURLOPT_SSL_VERIFYPEER, FALSE);
curl_setopt($ch, CURLOPT_SSL_VERIFYHOST,  0); //sometimes needs to be 0 or 1...
#  
 
// ENABLE HTTP POST
curl_setopt ($ch, CURLOPT_POST, 1);
 
// SET POST PARAMETERS : FORM VALUES FOR EACH FIELD
$user="some@domain.com";
$pass="1password";
$fields = 'username='.urlencode($user).'&password='.urlencode($pass);
 
//curl_setopt ($ch, CURLOPT_POSTFIELDS, 'username=some@domain.com&password=1password');
curl_setopt($ch, CURLOPT_POSTFIELDS, $fields);
 
 
# //sometimes the post method doesn't work for logins and this does
//curl_setopt($ch, CURLOPT_USERPWD, 'some@domain.com:1password'); // example:'myusername:secretpass'
#  
 
 
# //negociate the best HTTP authentication method(s)
curl_setopt($ch, CURLOPT_HTTPAUTH, CURLAUTH_ANY);
 
 
curl_setopt($ch, CURLOPT_USERAGENT, 'Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; .NET CLR 1.1.4322)');
curl_setopt($ch, CURLOPT_HTTPHEADER, array('Content-Type: text/xml')); 
 
curl_setopt($ch, CURLOPT_REFERER, 'https://www.somesite.com/DealerLogin.asp');
#curl_setopt($ch, CURLOPT_HEADER, 1);
 
// IMITATE CLASSIC BROWSER'S BEHAVIOUR : HANDLE COOKIES
curl_setopt ($ch, CURLOPT_COOKIEJAR, 'cookie.txt');
 
# //the following might be all you need
curl_setopt($ch, CURLOPT_FOLLOWLOCATION, 1); // often not set by default,  this will recursively follow all redirect headers
#  
 
# Setting CURLOPT_RETURNTRANSFER variable to 1 will force cURL
# not to print out the results of its query.
# Instead, it will return the results as a string return value
# from curl_exec() instead of the usual true/false.
#curl_setopt ($ch, CURLOPT_RETURNTRANSFER, 1);
 
// EXECUTE 1st REQUEST (FORM LOGIN)
$store = curl_exec ($ch);
 
# //use this after a curl_exec to extract http codes
# //and lots of info as documented here
# //http://curl.haxx.se/libcurl/c/curl_easy_getinfo.html
# //not probably needed if in verbose mode for debugging
$info=curl_getinfo($ch);//get debugging info on last call
 
print_r($info);
 
// SET FILE TO DOWNLOAD
//curl_setopt($ch, CURLOPT_URL, 'http://www.somesite.com/Reseller/Feeds/GetDataFeed.asp?type=csv');
 
// EXECUTE 2nd REQUEST (FILE DOWNLOAD)
//$content = curl_exec ($ch);
 
 
 
// CLOSE CURL
curl_close ($ch);
 
?>

here is what is printed using the print_r ;

Code: Select all

Array ( [url] => https://www.somesite.com/DealerLogin.asp [content_type] => text/html; charset=utf-8 [http_code] => 200 [header_size] => 713 [request_size] => 295 [filetime] => -1 [ssl_verify_result] => 0 [redirect_count] => 0 [total_time] => 1.542275 [namelookup_time] => 0.718569 [connect_time] => 0.791108 [pretransfer_time] => 1.11707 [size_upload] => 48 [size_download] => 15484 [speed_download] => 10039 [speed_upload] => 31 [download_content_length] => 15484 [upload_content_length] => -1 [starttransfer_time] => 1.388848 [redirect_time] => 0 )

I can not get this to do anything no matter where i put it :

Code: Select all

curl_setopt($ch, CURLOPT_VERBOSE, 1); 
any ideas?
bidntrade
Forum Commoner
Posts: 33
Joined: Thu Jul 02, 2009 9:54 pm

Re: use curl to connect asp site with username and password ?

Post by bidntrade »

my php version is

PHP 5.2.6

curl
cURL support enabled
cURL Information libcurl/7.19.5 OpenSSL/0.9.8b zlib/1.2.3 libidn/0.6.5
Eric!
DevNet Resident
Posts: 1146
Joined: Sun Jun 14, 2009 3:13 pm

Re: use curl to connect asp site with username and password ?

Post by Eric! »

For verbose, try to direct stderr to some known location.

Code: Select all

//redirect stderr to file stream
$fp = fopen("temp.txt","w");
curl_setopt ($ch, CURLOPT_STDERR, $fp);
//don't forget fclose at the end...
Are you sure you have the form variables correct? Can you post the link to the login page?
bidntrade
Forum Commoner
Posts: 33
Joined: Thu Jul 02, 2009 9:54 pm

Re: use curl to connect asp site with username and password ?

Post by bidntrade »

bidntrade
Forum Commoner
Posts: 33
Joined: Thu Jul 02, 2009 9:54 pm

Re: use curl to connect asp site with username and password ?

Post by bidntrade »

Eric! wrote:For verbose, try to direct stderr to some known location.

Code: Select all

//redirect stderr to file stream
$fp = fopen("temp.txt","w");
curl_setopt ($ch, CURLOPT_STDERR, $fp);
//don't forget fclose at the end...
Are you sure you have the form variables correct? Can you post the link to the login page?

using that i was able to see this .. but it does not really tell me much ....

Code: Select all

* About to connect() to http://www.xsdepot.com port 443 (#0)
*   Trying 12.157.226.145... * connected
* Connected to http://www.xsdepot.com (12.157.226.145) port 443 (#0)
* successfully set certificate verify locations:
*   CAfile: /etc/pki/tls/certs/ca-bundle.crt
  CApath: none
* SSL connection using RC4-MD5
* Server certificate:
*    subject: C=US, ST=Ohio, L=Lagrange, O=SpaceBound Inc., OU=SpaceBound Inc., CN=www.xsdepot.com
*    start date: 2008-09-11 17:48:50 GMT
*    expire date: 2009-10-06 17:57:33 GMT
*    subjectAltName: http://www.xsdepot.com matched
*    issuer: C=US, ST=Arizona, L=Scottsdale, O=GoDaddy.com, Inc., OU=http://certificates.godaddy.com/repository, CN=Go Daddy Secure Certification Authority, serialNumber=07969287
*    SSL certificate verify ok.
> POST /DealerLogin.asp HTTP/1.1
User-Agent: Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; .NET CLR 1.1.4322)
Host: http://www.xsdepot.com
Accept: */*
Referer: https://www.xsdepot.com/DealerLogin.asp
Cookie: G1=B7CBBC87-009E-4CBF-9D01-A2A1239D46AE; I1=96158605
Content-Length: 60
Content-Type: application/x-www-form-urlencoded
< HTTP/1.1 200 OK
< Date: Sat, 04 Jul 2009 19:37:28 GMT
< Server: Microsoft-IIS/6.0
< X-Powered-By: ASP.NET
< X-AspNet-Version: 1.1.4322
* Replaced cookie DV1="" for domain xsdepot.com, path /, expire 1088969848
< Set-Cookie: DV1=; domain=.xsdepot.com; expires=Sun, 04-Jul-2004 19:37:28 GMT; path=/
< Cache-Control: private
< Content-Type: text/html; charset=utf-8
< Content-Length: 15483
< 
* Connection #0 to host http://www.xsdepot.com left intact
* Closing connection #0
bidntrade
Forum Commoner
Posts: 33
Joined: Thu Jul 02, 2009 9:54 pm

Re: use curl to connect asp site with username and password ?

Post by bidntrade »

with this code :

Code: Select all

<?php
// INIT CURL
$ch = curl_init();
 
 
// SET URL FOR THE POST FORM LOGIN
curl_setopt($ch, CURLOPT_URL, 'https://www.xsdepot.com/DealerLogin.asp');
 
# // This is occassionally required to stop CURL from verifying the peer's certificate. for https connections
curl_setopt($ch, CURLOPT_SSL_VERIFYPEER, FALSE);
curl_setopt($ch, CURLOPT_SSL_VERIFYHOST,  2); //sometimes needs to be 0 or 1...
#  
 
// SET POST PARAMETERS : FORM VALUES FOR EACH FIELD
$user="some@email.com";
$pass="1password";
$fields = 'username='.urlencode($user).'&password='.urlencode($pass);
 
// ENABLE HTTP POST
curl_setopt ($ch, CURLOPT_POST, 1);
curl_setopt($ch, CURLOPT_POSTFIELDS, $fields);
 
curl_setopt($ch, CURLOPT_REFERER, 'https://www.xsdepot.com/');
 
// IMITATE CLASSIC BROWSER'S BEHAVIOUR : HANDLE COOKIES
curl_setopt ($ch, CURLOPT_COOKIEJAR, 'cookie.txt');
 
# //the following might be all you need
curl_setopt($ch, CURLOPT_FOLLOWLOCATION, 1); // often not set by default,  this will recursively follow all redirect headers
#  
 
# // use this option to output all the exchanged info for trying to see what is happening
curl_setopt($ch, CURLOPT_VERBOSE, 1);
 
# Setting CURLOPT_RETURNTRANSFER variable to 1 will force cURL
# not to print out the results of its query.
# Instead, it will return the results as a string return value
# from curl_exec() instead of the usual true/false.
curl_setopt ($ch, CURLOPT_RETURNTRANSFER, 1);
 
//redirect stderr to file stream
$fp = fopen("temp.txt","w");
curl_setopt ($ch, CURLOPT_STDERR, $fp);
//don't forget fclose at the end...
 
curl_setopt($ch, CURLOPT_REFERER, 'http://www.xsdepot.com/Reseller/Feeds/GetDataFeed.asp?type=csv');
 
// EXECUTE 1st REQUEST (FORM LOGIN)
$store = curl_exec ($ch);
 
// SET FILE TO DOWNLOAD
curl_setopt($ch, CURLOPT_URL, 'http://www.xsdepot.com/Reseller/Feeds/GetDataFeed.asp?type=csv');
 
// EXECUTE 2nd REQUEST (FILE DOWNLOAD)
$content = curl_exec ($ch);
 
echo $content;
 
// CLOSE CURL
curl_close ($ch);
fclose($fp);
?>

the output of temp.txt is :

Code: Select all

* About to connect() to http://www.xsdepot.com port 443 (#0)
*   Trying 12.157.226.145... * connected
* Connected to http://www.xsdepot.com (12.157.226.145) port 443 (#0)
* successfully set certificate verify locations:
*   CAfile: /etc/pki/tls/certs/ca-bundle.crt
  CApath: none
* SSL connection using RC4-MD5
* Server certificate:
*    subject: C=US, ST=Ohio, L=Lagrange, O=SpaceBound Inc., OU=SpaceBound Inc., CN=www.xsdepot.com
*    start date: 2008-09-11 17:48:50 GMT
*    expire date: 2009-10-06 17:57:33 GMT
*    subjectAltName: http://www.xsdepot.com matched
*    issuer: C=US, ST=Arizona, L=Scottsdale, O=GoDaddy.com, Inc., OU=http://certificates.godaddy.com/repository, CN=Go Daddy Secure Certification Authority, serialNumber=07969287
*    SSL certificate verify ok.
> POST /DealerLogin.asp HTTP/1.1
Host: http://www.xsdepot.com
Accept: */*
Referer: http://www.xsdepot.com/Reseller/Feeds/G ... p?type=csv
Content-Length: 48
Content-Type: application/x-www-form-urlencoded
 
< HTTP/1.1 200 OK
< Date: Sat, 04 Jul 2009 20:16:59 GMT
< Server: Microsoft-IIS/6.0
< X-Powered-By: ASP.NET
< X-AspNet-Version: 1.1.4322
* Added cookie I1="96161994" for domain http://www.xsdepot.com, path /, expire 1404505019
< Set-Cookie: I1=96161994; domain=www.xsdepot.com; expires=Fri, 04-Jul-2014 20:16:59 GMT; path=/
* Added cookie G1="3C9AD334-DD35-40BA-8427-7FC0F16F59AB" for domain http://www.xsdepot.com, path /, expire 1404505019
< Set-Cookie: G1=3C9AD334-DD35-40BA-8427-7FC0F16F59AB; domain=www.xsdepot.com; expires=Fri, 04-Jul-2014 20:16:59 GMT; path=/
* Added cookie VisitorIDNew="" for domain http://www.xsdepot.com, path /, expire 1088972219
< Set-Cookie: VisitorIDNew=; domain=www.xsdepot.com; expires=Sun, 04-Jul-2004 20:16:59 GMT; path=/
* Added cookie VisitorHashNew="" for domain http://www.xsdepot.com, path /, expire 1088972219
< Set-Cookie: VisitorHashNew=; domain=www.xsdepot.com; expires=Sun, 04-Jul-2004 20:16:59 GMT; path=/
* Added cookie DV1="" for domain http://www.xsdepot.com, path /, expire 1088972219
< Set-Cookie: DV1=; domain=www.xsdepot.com; expires=Sun, 04-Jul-2004 20:16:59 GMT; path=/
< Cache-Control: private
< Content-Type: text/html; charset=utf-8
< Content-Length: 15483
< 
 
* Connection #0 to host http://www.xsdepot.com left intact
* About to connect() to http://www.xsdepot.com port 80 (#1)
*   Trying 12.157.226.145... * connected
* Connected to http://www.xsdepot.com (12.157.226.145) port 80 (#1)
> POST /Reseller/Feeds/GetDataFeed.asp?type=csv HTTP/1.1
Host: http://www.xsdepot.com
Accept: */*
Referer: http://www.xsdepot.com/Reseller/Feeds/G ... p?type=csv
Cookie: G1=3C9AD334-DD35-40BA-8427-7FC0F16F59AB; I1=96161994
Content-Length: 48
Content-Type: application/x-www-form-urlencoded
 
< HTTP/1.1 302 Found
< Date: Sat, 04 Jul 2009 20:16:59 GMT
< Server: Microsoft-IIS/6.0
< X-Powered-By: ASP.NET
< X-AspNet-Version: 1.1.4322
< Location: /DealerLogin.asp?redirect=%2fReseller%2fFeeds%2fGetDataFeed.aspx%3ftype%3dcsv
< Cache-Control: private
< Content-Type: text/html; charset=utf-8
< Content-Length: 194
< 
* Ignoring the response-body
* Connection #1 to host http://www.xsdepot.com left intact
* Issue another request to this URL: 'http://www.xsdepot.com/DealerLogin.asp?redirect=%2fReseller%2fFeeds%2fGetDataFeed.aspx%3ftype%3dcsv'
* Violate RFC 2616/10.3.3 and switch from POST to GET
* Re-using existing connection! (#1) with host http://www.xsdepot.com
* Connected to http://www.xsdepot.com (12.157.226.145) port 80 (#1)
> GET /DealerLogin.asp?redirect=%2fReseller%2fFeeds%2fGetDataFeed.aspx%3ftype%3dcsv HTTP/1.1
Host: http://www.xsdepot.com
Accept: */*
Referer: http://www.xsdepot.com/Reseller/Feeds/G ... p?type=csv
Cookie: G1=3C9AD334-DD35-40BA-8427-7FC0F16F59AB; I1=96161994
 
< HTTP/1.1 200 OK
< Date: Sat, 04 Jul 2009 20:16:59 GMT
< Server: Microsoft-IIS/6.0
< X-Powered-By: ASP.NET
< X-AspNet-Version: 1.1.4322
* Replaced cookie DV1="" for domain http://www.xsdepot.com, path /, expire 1088972219
< Set-Cookie: DV1=; domain=www.xsdepot.com; expires=Sun, 04-Jul-2004 20:16:59 GMT; path=/
< Cache-Control: private
< Content-Type: text/html; charset=utf-8
< Content-Length: 16850
< 
* Connection #1 to host http://www.xsdepot.com left intact
* Closing connection #0
* Closing connection #1

any ideas .... if i echo out $content i have a page saying :

You must be logged in to access the page you requested.
Eric!
DevNet Resident
Posts: 1146
Joined: Sun Jun 14, 2009 3:13 pm

Re: use curl to connect asp site with username and password ?

Post by Eric! »

You're definately getting redirected. There is something strange in the cookie too with blank values.

Code: Select all

31.* Added cookie VisitorIDNew="" for domain http://www.xsdepot.com, path /, expire 1088972219
32.< Set-Cookie: [b]VisitorIDNew=; [/b]domain=www.xsdepot.com; expires=Sun, 04-Jul-2004 20:16:59 GMT; path=/
33.* Added cookie [b]VisitorHashNew="" [/b]for domain http://www.xsdepot.com, path /, expire 1088972219
34.< Set-Cookie: [b]VisitorHashNew=; [/b]domain=www.xsdepot.com; expires=Sun, 04-Jul-2004 20:16:59 GMT; path=/
35.* Added cookie [b]DV1="" [/b]for domain http://www.xsdepot.com, path /, expire 1088972219
36.< Set-Cookie: [b]DV1=; [/b]domain=www.xsdepot.com; expires=Sun, 04-Jul-2004 20:16:59 GMT; path=/
I don't see anything wrong with the code, so there is probably something more going on that the server is checking for.

Sometimes if you don't set the user agent you get denied (like bots/spiders/etc).

Code: Select all

curl_setopt($curl, CURLOPT_USERAGENT, "Mozilla/4.0 (compatible; MSIE 5.01; Windows NT 5.0)");
Since I don't have an account there, I didn't play with it. I did check the form fields and they look correct in your code. You could download the JAVA application webscarab, start it up as a proxy and then login normally with your browser. Webscarab will output the http exchanges between your browser and the server. You'll be able to compare the difference between the curl and your browser to see what else is necessary.

Maybe someone else on here has some more suggestions..... :roll:
bidntrade
Forum Commoner
Posts: 33
Joined: Thu Jul 02, 2009 9:54 pm

Re: use curl to connect asp site with username and password ?

Post by bidntrade »

I used tamperdata since the one you suggested kept crashing ....

the info from tamper data is :
23:54:18.973[4766ms][total 4766ms] Status: 302[Found]
POST https://www.xsdepot.com/DealerLogin.asp Load Flags[LOAD_DOCUMENT_URI LOAD_INITIAL_DOCUMENT_URI ] Content Size[152] Mime Type[text/html]
Request Headers:
Host[www.xsdepot.com]
User-Agent[Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.9.1) Gecko/20090624 Firefox/3.5]
Accept[text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8]
Accept-Language[en-US]
Accept-Encoding[gzip,deflate]
Accept-Charset[ISO-8859-1,utf-8;q=0.7,*;q=0.7]
Keep-Alive[300]
Connection[keep-alive]
Referer[http://xsdepot.com/DealerLogin.asp?redi ... fault.aspx]
Cookie[I1=96191380; G1=E1E2771E-9C0C-4A2C-8623-9E46C0D63C28; __utma=244350220.2129882624068568600.1246765584.1246765584.1246765584.1; __utmb=244350220.2.10.1246765584; __utmz=244350220.1246765584.1.1.utmcsr=(direct)|utmccn=(direct)|utmcmd=(none); __utmc=244350220]
Post Data:
username[email%40domain.com]
password[mypassword]
login.x[0]
login.y[0]
login[Login]
redirect[%2Fdefault.aspx]
Response Headers:
Date[Sun, 05 Jul 2009 03:54:17 GMT]
Server[Microsoft-IIS/6.0]
X-Powered-By[ASP.NET]
X-AspNet-Version[1.1.4322]
Location[http://www.xsdepot.com/default.aspx]
Set-Cookie[DV1=8VEefLa5MN3aB1YDHnu4Fw==; domain=.xsdepot.com; expires=Sat, 05-Jul-2014 03:54:17 GMT; path=/]
Cache-Control[private]
Content-Type[text/html; charset=utf-8]
Content-Length[152]




23:54:26.930[385ms][total 1121ms] Status: 200[OK]
GET http://www.xsdepot.com/default.aspx Load Flags[LOAD_DOCUMENT_URI LOAD_REPLACE LOAD_INITIAL_DOCUMENT_URI ] Content Size[56738] Mime Type[text/html]
Request Headers:
Host[www.xsdepot.com]
User-Agent[Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.9.1) Gecko/20090624 Firefox/3.5]
Accept[text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8]
Accept-Language[en-US]
Accept-Encoding[gzip,deflate]
Accept-Charset[ISO-8859-1,utf-8;q=0.7,*;q=0.7]
Keep-Alive[300]
Connection[keep-alive]
Referer[http://xsdepot.com/DealerLogin.asp?redi ... fault.aspx]
Cookie[I1=96191380; G1=E1E2771E-9C0C-4A2C-8623-9E46C0D63C28; __utma=244350220.2129882624068568600.1246765584.1246765584.1246765584.1; __utmb=244350220.2.10.1246765584; __utmz=244350220.1246765584.1.1.utmcsr=(direct)|utmccn=(direct)|utmcmd=(none); __utmc=244350220; DV1=8VEefLa5MN3aB1YDHnu4Fw==]
Response Headers:
Date[Sun, 05 Jul 2009 03:54:20 GMT]
Server[Microsoft-IIS/6.0]
X-Powered-By[ASP.NET]
X-AspNet-Version[1.1.4322]
Cache-Control[private]
Content-Type[text/html; charset=utf-8]
Content-Length[56738]
I have no idea where to go from here ?
Eric!
DevNet Resident
Posts: 1146
Joined: Sun Jun 14, 2009 3:13 pm

Re: use curl to connect asp site with username and password ?

Post by Eric! »

The first thing that jumps out is the additional post data fields.
Post Data:
username[email%40domain.com]
password[mypassword]
login.x[0]
login.y[0]
login[Login]
redirect[%2Fdefault.aspx]
There seems to be some http and https urls thrown back and forth which confuses me. I don't know what it means exactly.
POST https://www.xsdepot.com/DealerLogin.asp Load Flags[LOAD_DOCUMENT_URI LOAD_INITIAL_DOCUMENT_URI ] Content
...
GET http://www.xsdepot.com/default.aspx Load Flags[LOAD_DOCUMENT_URI
....
Referer[http://xsdepot.com/DealerLogin.asp?redi ... fault.aspx]
Post Reply