Page 1 of 1

Large tree structure and execution time outs

Posted: Mon Sep 13, 2010 1:23 pm
by Technocrat
I have a system that builds a category tree for a select box. It works fine when the number of categories is under 1000. But I have a couple of users that have LARGE category structures, like 6-7K. The problem they have is they get an error where the execution time for the script runs out, usually 30 seconds. Obviously you could try to extend the time but that isn't going to work for everyone on different shared hosting. I have optimized the function as much as possible but that still don't get it up in time. I could try to tread it but again that wouldn't work on server that doesn't allow procs.

I am looking for thoughts on how to maybe get this working.

One thought is to use an AJAX tree.

Re: Large tree structure and execution time outs

Posted: Mon Sep 13, 2010 1:25 pm
by Gargoyle
use a cronjob to create the trees in the background and then just read them on demand.

a couple 1000 categories shouldn't be much of a problem, though. there's probably something wrong in your code.

Re: Large tree structure and execution time outs

Posted: Mon Sep 13, 2010 1:39 pm
by Technocrat
A cron would have the same problem. Plus trying to tell people how to set one up isn't really easy. Plus some hosts don't allow crons.

You have to think about all the recursion on 6-7K category tree. Imagine a category 30 children deep with hundreds of its own. The code itself is really simple. Problem is just the sheer number of recursive calls to build it with that many.

Re: Large tree structure and execution time outs

Posted: Mon Sep 13, 2010 1:44 pm
by Gargoyle
well, you can allow the script to run longer than 30 seconds if invoked by cron.

as for your tree, I can't really judge that because I haven't seen the code, but from experience I would expect that you call mysql on demand to make things easier for you and/or that you use regular expressions all over the place.

Re: Large tree structure and execution time outs

Posted: Mon Sep 13, 2010 1:47 pm
by Weirdan
Technocrat wrote: You have to think about all the recursion on 6-7K category tree. Imagine a category 30 children deep with hundreds of its own. The code itself is really simple. Problem is just the sheer number of recursive calls to build it with that many.
The problem seems to be the recursion itself. Have you considered changing you db structure to either 'nested set' or 'materialized path'? (both terms should be easily googlable if you're not familiar with them).

Re: Large tree structure and execution time outs

Posted: Mon Sep 13, 2010 1:50 pm
by AbraCadaver
Technocrat wrote:I have a system that builds a category tree for a select box. It works fine when the number of categories is under 1000. But I have a couple of users that have LARGE category structures, like 6-7K. The problem they have is they get an error where the execution time for the script runs out, usually 30 seconds. Obviously you could try to extend the time but that isn't going to work for everyone on different shared hosting. I have optimized the function as much as possible but that still don't get it up in time. I could try to tread it but again that wouldn't work on server that doesn't allow procs.

I am looking for thoughts on how to maybe get this working.

One thought is to use an AJAX tree.
Why would anyone want to scroll through so many entries in a select box??? I don't know what your data is, but you should be able to narrow it down and much like "select your country" then "select your state/territory" then "select your city" instead of just listing all cities in all states/territories in all countries! You would display the first list of countries, then when they select one you fetch all states/territories in that country, then when selected you list all cities. Do this using AJAX or a simple select one and click next for next page method.

Re: Large tree structure and execution time outs

Posted: Mon Sep 13, 2010 1:54 pm
by Technocrat
Though a cron would probably be a good solution I just dont think I can get it to work for everyone. There are to many different people using it on to many different servers. That would be more of a mess than trying to just use AJAX.

The problem only happens in two areas. One where they have to put an item into a category so the list has to be complete and when they add a new category and again the list has to be complete for them to pick a parent.

Re: Large tree structure and execution time outs

Posted: Mon Sep 13, 2010 1:57 pm
by Technocrat
Weirdan wrote:
Technocrat wrote: You have to think about all the recursion on 6-7K category tree. Imagine a category 30 children deep with hundreds of its own. The code itself is really simple. Problem is just the sheer number of recursive calls to build it with that many.
The problem seems to be the recursion itself. Have you considered changing you db structure to either 'nested set' or 'materialized path'? (both terms should be easily googlable if you're not familiar with them).
The structure is already set (not by me) and at this point they don't want to change it.
AbraCadaver wrote:Why would anyone want to scroll through so many entries in a select box??? I don't know what your data is, but you should be able to narrow it down and much like "select your country" then "select your state/territory" then "select your city" instead of just listing all cities in all states/territories in all countries! You would display the first list of countries, then when they select one you fetch all states/territories in that country, then when selected you list all cities. Do this using AJAX or a simple select one and click next for next page method.
Its customer entered categories for products. That's my current #1 solution is to do an AJAX tree system. I am just trying to see if there was something else maybe I am missing or hadn't though of.