Large tree structure and execution time outs

PHP programming forum. Ask questions or help people concerning PHP code. Don't understand a function? Need help implementing a class? Don't understand a class? Here is where to ask. Remember to do your homework!

Moderator: General Moderators

Post Reply
User avatar
Technocrat
Forum Contributor
Posts: 127
Joined: Thu Oct 20, 2005 7:01 pm

Large tree structure and execution time outs

Post by Technocrat »

I have a system that builds a category tree for a select box. It works fine when the number of categories is under 1000. But I have a couple of users that have LARGE category structures, like 6-7K. The problem they have is they get an error where the execution time for the script runs out, usually 30 seconds. Obviously you could try to extend the time but that isn't going to work for everyone on different shared hosting. I have optimized the function as much as possible but that still don't get it up in time. I could try to tread it but again that wouldn't work on server that doesn't allow procs.

I am looking for thoughts on how to maybe get this working.

One thought is to use an AJAX tree.
Gargoyle
Forum Contributor
Posts: 130
Joined: Wed Jul 14, 2010 12:25 am

Re: Large tree structure and execution time outs

Post by Gargoyle »

use a cronjob to create the trees in the background and then just read them on demand.

a couple 1000 categories shouldn't be much of a problem, though. there's probably something wrong in your code.
User avatar
Technocrat
Forum Contributor
Posts: 127
Joined: Thu Oct 20, 2005 7:01 pm

Re: Large tree structure and execution time outs

Post by Technocrat »

A cron would have the same problem. Plus trying to tell people how to set one up isn't really easy. Plus some hosts don't allow crons.

You have to think about all the recursion on 6-7K category tree. Imagine a category 30 children deep with hundreds of its own. The code itself is really simple. Problem is just the sheer number of recursive calls to build it with that many.
Gargoyle
Forum Contributor
Posts: 130
Joined: Wed Jul 14, 2010 12:25 am

Re: Large tree structure and execution time outs

Post by Gargoyle »

well, you can allow the script to run longer than 30 seconds if invoked by cron.

as for your tree, I can't really judge that because I haven't seen the code, but from experience I would expect that you call mysql on demand to make things easier for you and/or that you use regular expressions all over the place.
User avatar
Weirdan
Moderator
Posts: 5978
Joined: Mon Nov 03, 2003 6:13 pm
Location: Odessa, Ukraine

Re: Large tree structure and execution time outs

Post by Weirdan »

Technocrat wrote: You have to think about all the recursion on 6-7K category tree. Imagine a category 30 children deep with hundreds of its own. The code itself is really simple. Problem is just the sheer number of recursive calls to build it with that many.
The problem seems to be the recursion itself. Have you considered changing you db structure to either 'nested set' or 'materialized path'? (both terms should be easily googlable if you're not familiar with them).
User avatar
AbraCadaver
DevNet Master
Posts: 2572
Joined: Mon Feb 24, 2003 10:12 am
Location: The Republic of Texas
Contact:

Re: Large tree structure and execution time outs

Post by AbraCadaver »

Technocrat wrote:I have a system that builds a category tree for a select box. It works fine when the number of categories is under 1000. But I have a couple of users that have LARGE category structures, like 6-7K. The problem they have is they get an error where the execution time for the script runs out, usually 30 seconds. Obviously you could try to extend the time but that isn't going to work for everyone on different shared hosting. I have optimized the function as much as possible but that still don't get it up in time. I could try to tread it but again that wouldn't work on server that doesn't allow procs.

I am looking for thoughts on how to maybe get this working.

One thought is to use an AJAX tree.
Why would anyone want to scroll through so many entries in a select box??? I don't know what your data is, but you should be able to narrow it down and much like "select your country" then "select your state/territory" then "select your city" instead of just listing all cities in all states/territories in all countries! You would display the first list of countries, then when they select one you fetch all states/territories in that country, then when selected you list all cities. Do this using AJAX or a simple select one and click next for next page method.
mysql_function(): WARNING: This extension is deprecated as of PHP 5.5.0, and will be removed in the future. Instead, the MySQLi or PDO_MySQLextension should be used. See also MySQL: choosing an API guide and related FAQ for more information.
User avatar
Technocrat
Forum Contributor
Posts: 127
Joined: Thu Oct 20, 2005 7:01 pm

Re: Large tree structure and execution time outs

Post by Technocrat »

Though a cron would probably be a good solution I just dont think I can get it to work for everyone. There are to many different people using it on to many different servers. That would be more of a mess than trying to just use AJAX.

The problem only happens in two areas. One where they have to put an item into a category so the list has to be complete and when they add a new category and again the list has to be complete for them to pick a parent.
User avatar
Technocrat
Forum Contributor
Posts: 127
Joined: Thu Oct 20, 2005 7:01 pm

Re: Large tree structure and execution time outs

Post by Technocrat »

Weirdan wrote:
Technocrat wrote: You have to think about all the recursion on 6-7K category tree. Imagine a category 30 children deep with hundreds of its own. The code itself is really simple. Problem is just the sheer number of recursive calls to build it with that many.
The problem seems to be the recursion itself. Have you considered changing you db structure to either 'nested set' or 'materialized path'? (both terms should be easily googlable if you're not familiar with them).
The structure is already set (not by me) and at this point they don't want to change it.
AbraCadaver wrote:Why would anyone want to scroll through so many entries in a select box??? I don't know what your data is, but you should be able to narrow it down and much like "select your country" then "select your state/territory" then "select your city" instead of just listing all cities in all states/territories in all countries! You would display the first list of countries, then when they select one you fetch all states/territories in that country, then when selected you list all cities. Do this using AJAX or a simple select one and click next for next page method.
Its customer entered categories for products. That's my current #1 solution is to do an AJAX tree system. I am just trying to see if there was something else maybe I am missing or hadn't though of.
Post Reply