Page 4 of 4

Re: Batching an import [solved]

Posted: Sat Sep 25, 2010 4:31 am
by josh
VladSun wrote:I wonder how all of you guys have so well formatted CSV input data - it haven't happened to me yet :evil:
Well, usually because a machine generates the CSV. The CSV is more less a "transport" format for a lot of these guys, they purchase access databases, and write visual basic scripts to output the CSV (one example). Also if you wanted to chat about that thing we PM'd you about I could enlighten you a little bit more about this.
VladSun wrote:I you look at the final result table you'll see it's a full hierarchy INNER JOIN, so probably it has to be something with JOIN in its name :)
Well, that accurately describes the solution (a set of queries that effectively join data all over the place, along with update & insert it places) - but the implementation is not necessarily the pattern. The same thing could be done with a PHP array as the "container" (think multi-row insert statement), instead of the import queue table. The important part was not looping over the data rows individually - I would say.