Page 1 of 1

Returning hug results

Posted: Wed Feb 13, 2008 12:54 am
by jsky
I'm currently developing a web application that handles huge contents. In my main page i have 3 panes, each with different functions to show dynamic results (from a search template).

I have a problem in loading the entire page because it first waits for all of the 3 divisions to be processed before showing their results. I would like to load the first two panes first, cause the last one is not that necessary and has the function that returns the largest result.

Im thinking of using AJAX but is it advisable in this case? Tnx!

Re: Returning hug results

Posted: Wed Feb 13, 2008 11:55 am
by alex.barylski
You would use AJAX for something like that...although I would suggest pagination as well to minimize the strain on your entire LAMP stack...

Keep it to 25 items per page and it should load just fine.

Re: Returning hug results

Posted: Wed Feb 13, 2008 8:24 pm
by jsky
Thanks for your reply, my code has paging in it, the process is just taking too long i think because of the huge contents. Think of them like short stories, and the div that has the longest loading time is the one that displays the highlighted search results hits per line of the story.

Would ajax assure me that the other divs would load first and won't wait for the slowest div? And my functions are all located on that page, so if i use ajax, should i transfer the function to the server request file? How about its parameters? Im sorry as i just made my first simple ajax app minutes ago. Thanks for your time.

Re: Returning hug results

Posted: Wed Feb 13, 2008 9:10 pm
by Christopher
When you say "huge contents" what size do you mean 1Mb, 10Mb, 100Mb, 1Gb?

Re: Returning hug results

Posted: Wed Feb 13, 2008 11:48 pm
by jsky
More than 1gb if the database is populated completely, but right now i have more than 50MB and the total time it takes the page to load is 8 seconds (local ftp server). And i could just imagine how long it will take when the database is complete and in a live site.

Re: Returning hug results

Posted: Wed Feb 13, 2008 11:52 pm
by califdon
I think you need to have another look at the architecture that would require that much data. I would expect that you would exceed allowable string lengths in Ajax responses.

Re: Returning hug results

Posted: Thu Feb 14, 2008 12:23 am
by jsky
So ajax responses have max allowable string lengths? Tt would be then harder for me to consider ajax..

Re: Returning hug results

Posted: Thu Feb 14, 2008 12:25 am
by califdon
jsky wrote:So ajax responses have max allowable string lengths? Tt would be then harder for me to consider ajax..
I don't actually know, but a 1GB string length blows my mind.

Re: Returning hug results

Posted: Thu Feb 14, 2008 12:28 am
by alex.barylski
AJAX is limited by only what GET/POST allow and what the browser implements...I guess

If your sending that much data to the screen for rendering...you absolutely need to re-think your design. If it's the queries which are just taking a long time...how are you retreiving the results???

Are you skipping through each record one bye one or are you dumping the resultset into a native PHP array and then iterating it?

Have you determined where the bottleneck is? Is it your PHP code, PHP memory consumption or maybe the database itself? You could probably optimize the query relatively easily with a little homework and asking questions here in the database forum.

Re: Returning hug results

Posted: Thu Feb 14, 2008 12:37 am
by Christopher
jsky wrote:More than 1gb if the database is populated completely, but right now i have more than 50MB
Wait a second ... is that the size of the whole database, or the size of one record of data sent to the browser for one page?
jsky wrote: and the total time it takes the page to load is 8 seconds (local ftp server). And i could just imagine how long it will take when the database is complete and in a live site.
8 seconds is not that bad for a large page. Having more records will not necessarily make your site any slower. Especially because you are probably using TEXT or BLOB fields which are stored separately from the row, so your rows are probably fairly small.

Re: Returning hug results

Posted: Thu Feb 14, 2008 12:40 am
by alex.barylski
is that the size of the whole database, or the size of one record of data sent to the browser for one page?
Is there even a database field which would hold 1GB???