working with big file array_unique() question ??

PHP programming forum. Ask questions or help people concerning PHP code. Don't understand a function? Need help implementing a class? Don't understand a class? Here is where to ask. Remember to do your homework!

Moderator: General Moderators

Post Reply
duk
Forum Contributor
Posts: 199
Joined: Wed May 19, 2004 8:45 am
Location: London

working with big file array_unique() question ??

Post by duk »

hy,

im working with a big file, with about 40000 arrays, and for example, i have a lot off blank arrays with nothing...

so i have use array_unique(); to erase arrays with double values in this case all blank arrays are gone... but now my problem is that im afraid that i could lost some information in other arrays by having the same first names ou surnames and lost that arrays...

what for me the best solution for this, and the best way to work with a big file and manipulate with arrays ??

thanks in advance
User avatar
feyd
Neighborhood Spidermoddy
Posts: 31559
Joined: Mon Mar 29, 2004 3:24 pm
Location: Bothell, Washington, USA

Post by feyd »

since php has issues with very large arrays, I'd suggest figuring a way to store the data into a database.
duk
Forum Contributor
Posts: 199
Joined: Wed May 19, 2004 8:45 am
Location: London

Post by duk »

the idea is to store everything in a database but first sort it...

but you suggest store every array in database then push data from the database and sort it ???
User avatar
feyd
Neighborhood Spidermoddy
Posts: 31559
Joined: Mon Mar 29, 2004 3:24 pm
Location: Bothell, Washington, USA

Post by feyd »

couldn't you use your RDBMS' own sorting (ORDER BY) command to sort the data?
duk
Forum Contributor
Posts: 199
Joined: Wed May 19, 2004 8:45 am
Location: London

Post by duk »

no...

becouse its like this...

array[0] => name, surname phone somedate
array[1] =>
array[3] => date: x/x/x sex:
array[4] => payed by:
array[5] =>
array[6] => more stuff here

then jumps more 12 blank arrays and goo for the next person... until reach the array[53245]...

i want to sort everything, like

array[1] => Name
array[2] => Surname
array[3] => phone
array[4] => x/x/x
array[5] => stuff1
array[6] => stuff2

blank blank blank and more sorting

and what i understand from you is store array[1] => name, surname phone somedata into DB and then get it with other script to explode it... and put again in DB ?
User avatar
feyd
Neighborhood Spidermoddy
Posts: 31559
Joined: Mon Mar 29, 2004 3:24 pm
Location: Bothell, Washington, USA

Post by feyd »

If there's a pattern to the data entry, you can process the file into a normalized database fairly easily. Post some excerpts from the file.
duk
Forum Contributor
Posts: 199
Joined: Wed May 19, 2004 8:45 am
Location: London

Post by duk »

something like this:
jhas, sdsdsd number Current
address, jkhjsdhjsh, dkjd

Added 20/12/2005 by X Changed / / by


18/12/2005
phone
phone


Date of Birth: 31/10/1980 Age: 25 Sex: Female Marital Status:


jhas, sdsdsd number Current
address, jkhjsdhjsh, dkjd

Added 20/12/2005 by X Changed / / by


18/12/2005
phone
phone


Date of Birth: 31/10/1980 Age: 25 Sex: Female Marital Status:


jhas, sdsdsd number Current
address, jkhjsdhjsh, dkjd

Added 20/12/2005 by X Changed / / by


18/12/2005
phone
phone


Date of Birth: 31/10/1980 Age: 25 Sex: Female Marital Status:












1



2


and then will be more 3 people
duk
Forum Contributor
Posts: 199
Joined: Wed May 19, 2004 8:45 am
Location: London

Post by duk »

just to say that i found a easy solution for my blank arrays...

Code: Select all

for ($i = 2 ; $i <= $total; $i++) {
      
        $var = $array[$i];
        $var = trim($var);
        if (!empty($var)) {
        echo $i ."-".$array[$i] ."<br>";    
        }
      
  }
now i just need to explode line after line... :D
Post Reply