php cURL - multiple pages & variable postfield ...

PHP programming forum. Ask questions or help people concerning PHP code. Don't understand a function? Need help implementing a class? Don't understand a class? Here is where to ask. Remember to do your homework!

Moderator: General Moderators

Post Reply
stino111
Forum Newbie
Posts: 3
Joined: Sun Oct 29, 2006 4:10 am

php cURL - multiple pages & variable postfield ...

Post by stino111 »

feyd | Please use

Code: Select all

,

Code: Select all

and [syntax="..."] tags where appropriate when posting code. Your post has been edited to reflect how we'd like it posted. Please read:  [url=http://forums.devnetwork.net/viewtopic.php?t=21171]Posting Code in the Forums[/url] to learn how to do it too.[/color]


Hi,


the last couple of days I've been learning cURL. A nice phptool I must say. Nu I wanted to write a script for a lookalike site of ebay. You can offer some of your private stuff to sell it overthere. Now I came into troubles : 

the site is 
http://www.2dehands.be  (i've to apollogize but it's in dutch :p )
i've created a test-user
login : stijnthe1st  @   hotmail.com
pw : garmin1

What is what I want to do? I want a daily renewal of my ad, so it comes on top of the ads once a day. 
For this I don't want to create a new ad. And the strange thing is, that I manage to create new ads, but not just a refresh. Also there are some pics included (actually just one) and this one isn't either included.

The difficulties I came along :

- login (this one is solved I guess)
- a variable field scratch passed in the header. I managed to extract it from the source html output, but i'm not sure this is the right way to do it
- variation in Post & Get methods
- cookies & sessions 

I hope someone can help me out. I think this one is a very challenging problem. As I said, I - as a newby - succeeded in posting new posts, but not in renewals.

I'm curious what the guru's can make of it.

Code: Select all

<?php

// first step : login - this works fine"
$ch = curl_init();   
$url3 = 'http://www.2dehands.be/login.html';
curl_setopt($ch, CURLOPT_URL, $url3);
curl_setopt($ch, CURLOPT_POST, 1); 	
curl_setopt($ch, CURLOPT_COOKIEJAR, "cookies.txt"); 
curl_setopt($ch, CURLOPT_COOKIEFILE, "cookies.txt"); 
$postfields3 = "doel=%2Fbeheer%2F&via_req=1&email=stijnthe1st%40hotmail.com&password=garmin1&password1=&password2=&naam=&telefoon=&pref_plaats=&nieuwsbrief=1"; 
curl_setopt($ch, CURLOPT_POSTFIELDS, $postfields3); 
curl_setopt($ch, CURLOPT_TIMEOUT, '30');
curl_setopt($ch, CURLOPT_FOLLOWLOCATION, 1); 	
curl_setopt($ch, CURLOPT_AUTOREFERER, 1);  
$curllogin = curl_exec($ch); 


//load the page for this product with id number 16775162 so we can extract the variable field "scratch=lsmfjsmldjfmskj" from the source HTML code

$url = "http://www.2dehands.be/plaats.html?nr=16775162";
curl_setopt($ch, CURLOPT_URL,$url);
curl_setopt($ch, CURLOPT_COOKIEFILE, "cookies.txt"); 
curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1);
curl_setopt($ch, CURLOPT_FOLLOWLOCATION, 0);  
curl_setopt($ch, CURLOPT_AUTOREFERER, 1);  
$data = curl_exec($ch); 

//extract the field scratch from the $data string
$start = strpos($data, '"scratch" value="');
$start = $start + strlen('"scratch" value="');
$end = strpos($data, '"', $start+1+strlen('"scratch" value"'));
$length = $end - $start ;
$tst = substr($data, $start, $length);
echo $tst;

//first part of the post procedure

$url= "http://www.2dehands.be/plaats.html?scratch=".$tst."&afdeling=moto&dbase=motor_access&motor_access=7&submit_1=Wijzig+advertentie";
curl_setopt($ch, CURLOPT_URL,$url);
curl_setopt($ch, CURLOPT_COOKIEJAR, "cookies.txt"); 
curl_setopt($ch, CURLOPT_COOKIEFILE, "cookies.txt"); 
curl_setopt($ch, CURLOPT_HTTPGET, 1);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, 0);
curl_setopt($ch, CURLOPT_FOLLOWLOCATION, 0);  
$data = curl_exec($ch); 

// 2nd ppart of the post procedure

$postfields2="doplaats=1&motor_access=7&dbase=motor_access&host=moto&scratch=" . $tst . "&tkgz=tk&adtekstplaats_title_nl_BE=Tomtom+Rider&adtekstplaats_text_nl_BE=Speciaal". "+ontworpen+voor+twee+wielen%3A+TomTom+RIDER+is+robuust%2C+waterbestendi" . "g%2C+past+zowel+op+motoren+als+scooters+en+geeft+duidelijk+gesproken+instruc" . "ties%2C+draadloos+via+je+helm.%0D%0AGebruikersvriendelijk%3A+de+rijweergav" . "e+geeft+eenvoudige+aanwijzingen+op+een+touchscreen+dat+speciaal+is+afgestemd" . "+op+het+gebruik+van+handschoenen.%0D%0AKaart+van+West-Europa+op+geheug" . "enkaart%3A+deur-tot-deurnavigatie+waar+dan+ook+in+heel+West-Europa.%0D%0A" . "Handsfree+bellen%3A+is+ook+te+gebruiken+als+handsfree+motorkit+via+Bluetooth%" . "E2%84%A2+technologie.+Zo+kun+je+tijdens+het+rijden+telefoontjes+aannemen+doo" . "r+simpel+met+je+handschoen+het+scherm+aan+te+raken.%0D%0ASlimme+extra%2" . "7s%3A+plan+je+rit+vooraf+op+je+computer%2C+ontvang+gratis+flitslocaties+en+m" . "otorvriendelijke+toeristische+routes%2C+zoek+je+vrienden+op+en+houdt+contact%2" ."C+en+nog+veel+meer%0D%0A%0D%0A%0D%0A%0D%0A%0D%0A%0D%0A%0D%" . "0A%0D%0A%0D%0A&adtekstplaats_title_fr_BE=&adtekstplaats_text_fr_BE=&prijs_type" . "=bedrag&prijs=649%2C00&allow_bieden=on&conditie=nieuw&chfoto1=keep&foto1=&fo" . "to2=&skip_preview=Plaats+wijziging";


$url = 'http://www.2dehands.be/plaats.html';
curl_setopt($ch, CURLOPT_URL, $url);
curl_setopt($ch, CURLOPT_POST, 1); 	
curl_setopt($ch, CURLOPT_POSTFIELDS, $postfields2); 
curl_setopt($ch, CURLOPT_TIMEOUT, '30');
curl_setopt($ch, CURLOPT_RETURNTRANSFER, 0); 
$data2 = curl_exec($ch); 

?>

feyd | Please use

Code: Select all

,

Code: Select all

and [syntax="..."] tags where appropriate when posting code. Your post has been edited to reflect how we'd like it posted. Please read:  [url=http://forums.devnetwork.net/viewtopic.php?t=21171]Posting Code in the Forums[/url] to learn how to do it too.[/color]
timvw
DevNet Master
Posts: 4897
Joined: Mon Jan 19, 2004 11:11 pm
Location: Leuven, Belgium

Post by timvw »

For such 'screenscraping' tasks i usually prefer to use SimpleTest scriptable browser instead of Curl... Here's an example: http://www.timvw.be/wp-content/php/smscity.txt
stino111
Forum Newbie
Posts: 3
Joined: Sun Oct 29, 2006 4:10 am

Post by stino111 »

sorry this is the correct number 17119446
stino111
Forum Newbie
Posts: 3
Joined: Sun Oct 29, 2006 4:10 am

Post by stino111 »

thx tim

but I would prefer to do it with cURL in order to learn it.

I just can't understand why it isn't working.
I just discoverd that it might be something with sessions 2

anybody?
Post Reply