Memory Size - Fatal Error
Posted: Mon Nov 29, 2010 5:12 am
Hey all -
First of all, I'm using CodeIgniter for this, but I'm pretty sure the error is strictly PHP (just giving you a heads up for syntax).
I'm consuming an xml file generated by a gnip.com social media API. This particular iteration is attempting to save about 25,000 records to my db. The code I'm using is:
Here is the save function:
The error I get is this:
Fatal error: Allowed memory size of 33554432 bytes exhausted (tried to allocate 76 bytes) in /public_html/CI_1_7_2/system/database/DB_active_rec.php on line 1753
I have been able to mitigate this by including:
What I want to know is why this is happening, and if there's a better solution than just feeding my functions more RAM. Is my code just inefficient?
Thanks!
Scott
First of all, I'm using CodeIgniter for this, but I'm pretty sure the error is strictly PHP (just giving you a heads up for syntax).
I'm consuming an xml file generated by a gnip.com social media API. This particular iteration is attempting to save about 25,000 records to my db. The code I'm using is:
Code: Select all
$this->load->model('feed_source_model');
$this->load->model('records_model');
$gnip = $this->config->item('gnip');
$feed_sources = $this->feed_source_model->get();
foreach($feed_sources->result() as $source) {
if(!$source->since_date) {
$date = date('YmdHis',strtotime('-6 month'));
} else {
$date = $source->since_date;
}
$url = $gnip['base_url'] . $source->collector_ID . '/' . $gnip['filename'] . '?max=10000&since_date=' . $date;
$xmlStr = file_get_contents($url);
$xmlObj = simplexml_load_string($xmlStr);
foreach ($xmlObj->entry as $entry) {
$keyword = $xmlObj->entry->source->title;
$p = strrpos($keyword, ' - ');
if ($p!==false) $keyword = substr($keyword, $p+3);
$data['keyword'] = $keyword;
$data['date'] = date('Y-m-d H:i:s', strtotime($entry->published));
$data['source'] = $source->ID;
$data['data'] = $entry->title;
$this->records_model->save($data);
}
Code: Select all
function save($record) {
$data = array(
'keyword' => $record['keyword'],
'source' => $record['source'],
'date' => $record['date'],
'data' => htmlspecialchars($record['data']));
$this->db->insert('records',$data);
}
Fatal error: Allowed memory size of 33554432 bytes exhausted (tried to allocate 76 bytes) in /public_html/CI_1_7_2/system/database/DB_active_rec.php on line 1753
I have been able to mitigate this by including:
Code: Select all
ini_set("memory_limit","512M");
Thanks!
Scott