MyActiveRecord - memory limit
Posted: Fri May 26, 2006 10:53 am
i have a page that reads from a remote MySQL DB and uses objects based on MyActive Record to populate a local MySQL DB
i read from the remote DB and if the record is different i create the objects and save()
these are inside a transaction & i commit every 5000 records (i process 5000 at a time from the remote DB) - at this point i am only finding 5-10 records that are different
i only read a key & date from the remote DB and if i need the entire dataset i read it 1 record at a time and build 1 record at a time locally
i then reuse the same objects for the next record
i have tried setting my objects to NULL at the end of every loop
i release the sql resources at the end of every loop
but, i die with memory exceeded - increasing the memory limit only extends the # of records i process until it dies
something is not releasing it's memory resources & i can't seem to find it
mark
i read from the remote DB and if the record is different i create the objects and save()
these are inside a transaction & i commit every 5000 records (i process 5000 at a time from the remote DB) - at this point i am only finding 5-10 records that are different
i only read a key & date from the remote DB and if i need the entire dataset i read it 1 record at a time and build 1 record at a time locally
i then reuse the same objects for the next record
i have tried setting my objects to NULL at the end of every loop
i release the sql resources at the end of every loop
but, i die with memory exceeded - increasing the memory limit only extends the # of records i process until it dies
something is not releasing it's memory resources & i can't seem to find it
mark