Create 80000 users one shot

PHP programming forum. Ask questions or help people concerning PHP code. Don't understand a function? Need help implementing a class? Don't understand a class? Here is where to ask. Remember to do your homework!

Moderator: General Moderators

Post Reply
styvodiabolo
Forum Newbie
Posts: 13
Joined: Tue May 19, 2009 2:18 pm

Create 80000 users one shot

Post by styvodiabolo »

Hello,

I have a problem with an unknown server configuration.
When I try to insert 80000 users with wamp (locally) it works !
But when I execute the same script in the production server, it stop at 13591 users.

The very big problem is that no error is display despite that I wrote before the script :

Code: Select all

 
set_time_limit(0);
ini_set('memory_limit','80M');
if(!ini_get('display_errors'))
    ini_set('display_errors', 1);
error_reporting(E_ALL);
 
and so, I can't find a solution if I haven't error to work...

Do you have a suggest, an idee ?

Thanks a lot.

Steve.
User avatar
Benjamin
Site Administrator
Posts: 6935
Joined: Sun May 19, 2002 10:24 pm

Re: Create 80000 users one shot

Post by Benjamin »

Check the error log. Your webhost may be killing the process due to the CPU usage.
User avatar
Darhazer
DevNet Resident
Posts: 1011
Joined: Thu May 14, 2009 3:00 pm
Location: HellCity, Bulgaria

Re: Create 80000 users one shot

Post by Darhazer »

Show the code you are using and give us some info about the database.

I bet you are not using a delayed insert (or multiple records insert) and just insert them in a loop... this is wrong practice. You can use:

Code: Select all

INSERT INTO table(field, field) VALUES(value,value),(value,value),(value,value)...
with for example 100 records, and in this way it is much faster, because MySQL do not lock/unlock the table for each record, but only once for all of them.

P.S. You have to set ini_set('display_errors', 'on') also for displaying the errors
And if MySQL is not the reason your PHP script stops, maybe it just reach the 30 seconds default time-limit, so check this also.
styvodiabolo
Forum Newbie
Posts: 13
Joined: Tue May 19, 2009 2:18 pm

Re: Create 80000 users one shot

Post by styvodiabolo »

I am just download it from the server (65 Mo :o ) do you know an editor which feels good with very big files ?

I program with Aptana but when the file is up to 6000 lines, Aptana feels bad and I have to restart it (very often).

I will try now the other way with INSERT VALUES(),(),()

Thanks for your comments.

Steve.
User avatar
Darhazer
DevNet Resident
Posts: 1011
Joined: Thu May 14, 2009 3:00 pm
Location: HellCity, Bulgaria

Re: Create 80000 users one shot

Post by Darhazer »

What is 65 MB - the script or the database dump? How you are importing it? I thought this is a script written by yourself?
styvodiabolo
Forum Newbie
Posts: 13
Joined: Tue May 19, 2009 2:18 pm

Re: Create 80000 users one shot

Post by styvodiabolo »

65 Mb is the size of the php_errlog create by the server.

Many of the errors was Warnings with headers (already sent...), so I could find an other error on it but I can't open the log with my editor.

I really need on other editor just to open correctly big files.

I continue to download it...
styvodiabolo
Forum Newbie
Posts: 13
Joined: Tue May 19, 2009 2:18 pm

Re: Create 80000 users one shot

Post by styvodiabolo »

Finally, I will use notepad to search / replace error on the log.

I really think that the solution is the query insert with VALUES(),(),()

I have to go.

Thank you for helping me.

Steve.
Post Reply