Page 1 of 1

Memory Size - Fatal Error

Posted: Mon Nov 29, 2010 5:12 am
by onecurlyfry
Hey all -

First of all, I'm using CodeIgniter for this, but I'm pretty sure the error is strictly PHP (just giving you a heads up for syntax).

I'm consuming an xml file generated by a gnip.com social media API. This particular iteration is attempting to save about 25,000 records to my db. The code I'm using is:

Code: Select all

$this->load->model('feed_source_model');
$this->load->model('records_model');
$gnip = $this->config->item('gnip');
$feed_sources = $this->feed_source_model->get();
foreach($feed_sources->result() as $source) {
	if(!$source->since_date) {
		$date = date('YmdHis',strtotime('-6 month'));
	} else {
		$date = $source->since_date;
	}
			
	$url = $gnip['base_url'] . $source->collector_ID . '/' . $gnip['filename'] . '?max=10000&since_date=' . $date;
	$xmlStr = file_get_contents($url);
	$xmlObj = simplexml_load_string($xmlStr);
	
	foreach ($xmlObj->entry as $entry) {
		$keyword = $xmlObj->entry->source->title;
		$p = strrpos($keyword, ' - ');
		if ($p!==false) $keyword = substr($keyword, $p+3);
		$data['keyword'] = $keyword;
		$data['date'] = date('Y-m-d H:i:s', strtotime($entry->published));
		$data['source'] = $source->ID;
		$data['data'] = $entry->title;
		$this->records_model->save($data);
	}
Here is the save function:

Code: Select all

function save($record) {
		$data = array(
			'keyword' => $record['keyword'],
			'source' => $record['source'],
			'date' => $record['date'],
			'data' => htmlspecialchars($record['data']));
		$this->db->insert('records',$data);
	}
The error I get is this:
Fatal error: Allowed memory size of 33554432 bytes exhausted (tried to allocate 76 bytes) in /public_html/CI_1_7_2/system/database/DB_active_rec.php on line 1753

I have been able to mitigate this by including:

Code: Select all

ini_set("memory_limit","512M");
What I want to know is why this is happening, and if there's a better solution than just feeding my functions more RAM. Is my code just inefficient?

Thanks!

Scott

Re: Memory Size - Fatal Error

Posted: Mon Nov 29, 2010 7:04 am
by Weirdan
What is your PHP version? PHP has been including circular object graph garbage collector since v5.3
While I don't see anything in your code that would lead to a circular object graph the leak could be hidden somewhere in CI internals.

Re: Memory Size - Fatal Error

Posted: Mon Nov 29, 2010 1:30 pm
by onecurlyfry
Weirdan wrote:What is your PHP version? PHP has been including circular object graph garbage collector since v5.3
While I don't see anything in your code that would lead to a circular object graph the leak could be hidden somewhere in CI internals.
Thanks for the response.

Apparently I'm on PHP Version 5.2.14, VPS hosting with Host Gator. It sounds like I should speak with them about possibly upgrading... I assume that's the correct path to take?

It sounds like if that doesn't fix it, I might actually have to jump to the CI forum & see if any CI guru's can help me out.

Thx again,

Scott