Page 1 of 1

Trying To Create Database

Posted: Wed Feb 24, 2010 9:04 pm
by Runtest
[URL REDACTED]

This is the website I am trying to mine.
Here is my problem.

Code: Select all

<?php
 
foreach(range('A','Z') as $a) {
     foreach(range('A','Z')as $b) {
          for($c = 0; $c < 10; $c += 1) {
               for($d = 0; $d < 10; $d += 1) {
                    for($e = 0; $e < 10; $e += 1) {
                         echo $a;
                         echo $b;
                         echo $c;
                         echo $d;
                         echo $e;
                         echo "</br>";
                    }
               }
          }
     }
}
 
?>              
 
I am trying to run through all the SKUs on the page. You will notice when you go to a product it has a SKU # in the title bar. There are several issues besides how long it takes just to run through all of the combinations above.

1. The skus vary. Some are 5 long some are 9 long with a different combination of letters and numbers.
2. Some have dashes.
3. Some are just words like 'battery105'.

How do I go about going through all of the skus available without running a loop like that for everything?

Any help is very appreciated. Thank you.

Re: Trying To Create Database

Posted: Wed Feb 24, 2010 10:34 pm
by requinix
So what do you need this for?

Re: Trying To Create Database

Posted: Thu Feb 25, 2010 1:02 am
by Runtest
Basically, I need to be able to generate every possible SKU # so I can go through the pages and pick out text to store in a database. If you notice in the url you will see SKU=AA123. My code can generate that format. But that site is very unfriendly as in, some of their SKU #s are 9 characters long, not even close to the same format some have a - in them. So I need a way to generate every possible SKU #.

Re: Trying To Create Database

Posted: Thu Feb 25, 2010 1:07 am
by Benjamin
Do the terms of service permit automated requests for the purpose of data aggregation? Does the site have an API you could use instead? Are you aware of the detrimental impacts your script can have on the performance of the site and how to negate this?

There are too many questions here. I need a good reason not to lock this topic.

EDIT:
All images and contents contained within this website are ©1998-2010
:arrow: LOCKED