Page 1 of 1

Create 80000 users one shot

Posted: Tue May 19, 2009 2:25 pm
by styvodiabolo
Hello,

I have a problem with an unknown server configuration.
When I try to insert 80000 users with wamp (locally) it works !
But when I execute the same script in the production server, it stop at 13591 users.

The very big problem is that no error is display despite that I wrote before the script :

Code: Select all

 
set_time_limit(0);
ini_set('memory_limit','80M');
if(!ini_get('display_errors'))
    ini_set('display_errors', 1);
error_reporting(E_ALL);
 
and so, I can't find a solution if I haven't error to work...

Do you have a suggest, an idee ?

Thanks a lot.

Steve.

Re: Create 80000 users one shot

Posted: Tue May 19, 2009 2:28 pm
by Benjamin
Check the error log. Your webhost may be killing the process due to the CPU usage.

Re: Create 80000 users one shot

Posted: Tue May 19, 2009 2:31 pm
by Darhazer
Show the code you are using and give us some info about the database.

I bet you are not using a delayed insert (or multiple records insert) and just insert them in a loop... this is wrong practice. You can use:

Code: Select all

INSERT INTO table(field, field) VALUES(value,value),(value,value),(value,value)...
with for example 100 records, and in this way it is much faster, because MySQL do not lock/unlock the table for each record, but only once for all of them.

P.S. You have to set ini_set('display_errors', 'on') also for displaying the errors
And if MySQL is not the reason your PHP script stops, maybe it just reach the 30 seconds default time-limit, so check this also.

Re: Create 80000 users one shot

Posted: Tue May 19, 2009 2:37 pm
by styvodiabolo
I am just download it from the server (65 Mo :o ) do you know an editor which feels good with very big files ?

I program with Aptana but when the file is up to 6000 lines, Aptana feels bad and I have to restart it (very often).

I will try now the other way with INSERT VALUES(),(),()

Thanks for your comments.

Steve.

Re: Create 80000 users one shot

Posted: Tue May 19, 2009 2:47 pm
by Darhazer
What is 65 MB - the script or the database dump? How you are importing it? I thought this is a script written by yourself?

Re: Create 80000 users one shot

Posted: Tue May 19, 2009 2:52 pm
by styvodiabolo
65 Mb is the size of the php_errlog create by the server.

Many of the errors was Warnings with headers (already sent...), so I could find an other error on it but I can't open the log with my editor.

I really need on other editor just to open correctly big files.

I continue to download it...

Re: Create 80000 users one shot

Posted: Tue May 19, 2009 3:10 pm
by styvodiabolo
Finally, I will use notepad to search / replace error on the log.

I really think that the solution is the query insert with VALUES(),(),()

I have to go.

Thank you for helping me.

Steve.