Page 1 of 1

submitting large amount of data

Posted: Tue Sep 04, 2007 4:19 pm
by mikeeeeeeey
Hi guys,

I'm developing a script which takes the contents of a tab-delimited text file and inputs them into a database.
However, as I've started to near actually putting the data in now I've got the text processing working fine, my browser is timing out and the script only seems to be running once.

Do any of you guys know why? And what I can do stop it from timing out i.e. how to split the upload into chunks?
Here's my code incase:

Code: Select all

//READ TEXT FILE
  	$filename = "first_quarter.txt";
  	$handle = fopen($filename, "r");
  	$contents = fread($handle, filesize($filename));
 	fclose($handle);
  
	$num = 0;
	$lines = explode("\n",$contents,50);
	$sections = array();
	foreach($lines as $row){
		$field  = explode("\t",$row);
		$Order_Code                  = $field[0];
		$Short_Description           = addslashes($field[1]);
		$Long_Description            = addslashes($field[2] . "Vat_code=" . $field[4]);
		$Sterling_Retail_Price       = $field[5];
		$KFH_Section                 = $field[14];
		$KFH_Major_Group_Description = $field[15];
		$KFH_Minor_Group_Description = $field[16];

		$sql   = "INSERT INTO products (products_price) 
			  VALUES ('" . mysql_real_escape_String($Sterling_Retail_Price) . "')";
		$query = mysql_query($sql);


		$sql   = "INSERT INTO products_description (products_name, products_description) 
			  VALUES ('" . mysql_real_escape_string($Short_Description) . "',
				  '" . mysql_real_escape_string($Long_Description) . "')";
		$query = mysql_query($sql);

		$catId  = 1;
		for($i = 0; $i < 3; $i++){
			$val   = $i + 14;
			$sql   = "INSERT INTO categories (categories_id, parent_id) 
				  VALUES ('" . mysql_real_escape_string($catId) . "',
					  '" . mysql_real_escape_string($i) . "')";
			$query = mysql_query($sql);
			$sql   = "INSERT INTO categories_description (categories_id, categories_name) 
				  VALUES ('" . mysql_real_escape_string($catId) . "',
					  '" . mysql_real_escape_string($field[$val]) . "')";
			$query = mysql_query($sql);
			$catId++;
		}
  	}
Thanks a lot.

Posted: Tue Sep 04, 2007 4:37 pm
by hawleyjr
You may want to look into taking (using) BigDump

http://www.ozerov.de/bigdump.php

Posted: Wed Sep 05, 2007 11:07 am
by xpgeek
First you can set_time_limit to force browser time limit.

Second BigDump is not a solution because mikeeeeeeey need to import data from one file to different tables.