submitting large amount of data

PHP programming forum. Ask questions or help people concerning PHP code. Don't understand a function? Need help implementing a class? Don't understand a class? Here is where to ask. Remember to do your homework!

Moderator: General Moderators

Post Reply
User avatar
mikeeeeeeey
Forum Contributor
Posts: 130
Joined: Mon Jul 03, 2006 4:17 am
Location: Huddersfield, UK

submitting large amount of data

Post by mikeeeeeeey »

Hi guys,

I'm developing a script which takes the contents of a tab-delimited text file and inputs them into a database.
However, as I've started to near actually putting the data in now I've got the text processing working fine, my browser is timing out and the script only seems to be running once.

Do any of you guys know why? And what I can do stop it from timing out i.e. how to split the upload into chunks?
Here's my code incase:

Code: Select all

//READ TEXT FILE
  	$filename = "first_quarter.txt";
  	$handle = fopen($filename, "r");
  	$contents = fread($handle, filesize($filename));
 	fclose($handle);
  
	$num = 0;
	$lines = explode("\n",$contents,50);
	$sections = array();
	foreach($lines as $row){
		$field  = explode("\t",$row);
		$Order_Code                  = $field[0];
		$Short_Description           = addslashes($field[1]);
		$Long_Description            = addslashes($field[2] . "Vat_code=" . $field[4]);
		$Sterling_Retail_Price       = $field[5];
		$KFH_Section                 = $field[14];
		$KFH_Major_Group_Description = $field[15];
		$KFH_Minor_Group_Description = $field[16];

		$sql   = "INSERT INTO products (products_price) 
			  VALUES ('" . mysql_real_escape_String($Sterling_Retail_Price) . "')";
		$query = mysql_query($sql);


		$sql   = "INSERT INTO products_description (products_name, products_description) 
			  VALUES ('" . mysql_real_escape_string($Short_Description) . "',
				  '" . mysql_real_escape_string($Long_Description) . "')";
		$query = mysql_query($sql);

		$catId  = 1;
		for($i = 0; $i < 3; $i++){
			$val   = $i + 14;
			$sql   = "INSERT INTO categories (categories_id, parent_id) 
				  VALUES ('" . mysql_real_escape_string($catId) . "',
					  '" . mysql_real_escape_string($i) . "')";
			$query = mysql_query($sql);
			$sql   = "INSERT INTO categories_description (categories_id, categories_name) 
				  VALUES ('" . mysql_real_escape_string($catId) . "',
					  '" . mysql_real_escape_string($field[$val]) . "')";
			$query = mysql_query($sql);
			$catId++;
		}
  	}
Thanks a lot.
User avatar
hawleyjr
BeerMod
Posts: 2170
Joined: Tue Jan 13, 2004 4:58 pm
Location: Jax FL & Spokane WA USA

Post by hawleyjr »

You may want to look into taking (using) BigDump

http://www.ozerov.de/bigdump.php
User avatar
xpgeek
Forum Contributor
Posts: 146
Joined: Mon May 22, 2006 1:45 am
Location: Kyiv, Ukraine
Contact:

Post by xpgeek »

First you can set_time_limit to force browser time limit.

Second BigDump is not a solution because mikeeeeeeey need to import data from one file to different tables.
Post Reply