Page 1 of 1

STRANGE Content Retrieval

Posted: Tue Dec 30, 2003 4:55 pm
by alin
hello,

I have to get content from a site and put it in a database. But in the database I have to put the data in the right place.

Let's say there is a page for each user of a site and on that page there are some information about him (name, age, list of friends, etc.).

I have to scan the site and put into a database all the users with their info.

Any suggestion on how to do that ? Is this possible with PHP ?

Thanks a lot.

Posted: Thu Jan 01, 2004 4:36 am
by JAM
Depending on how advanced the page you are retrieving there are various ways to achieve this, and it's a rather annoying (I personally think) test-and-retry thing you might embarge into.

fopen() the page and start narrowing down the results using ereg/preg/string functions (there are lots of tips in the manual on these functions). I think that is the easiest explanation so far.

The main problem is that if the one hosting the original page one days changes the layout of it, your data-retrieval expressions might be broken and you need to do the above again...

Posted: Fri Jan 02, 2004 8:59 am
by alin
Ok, I think I can do that. The only problem that remains is :

The following pages I have to put in the database.

- http://www.domain.com/display_user.php?id=xxxxxx , where xxxxx stands for the ID of the user displayed.

I have to put all users in the database so I will have to use something like this :

for ($i=i;$i<MAX_NO;$i++)
fopen("www.doma.../display_user.php?id=$i","r");

and operate on the file to get the data I need.

Is this correct ?