Configuration files vs Database. Design suggestion

Not for 'how-to' coding questions but PHP theory instead, this forum is here for those of us who wish to learn about design aspects of programming with PHP.

Moderator: General Moderators

User avatar
julian_lp
Forum Contributor
Posts: 121
Joined: Sun Jul 09, 2006 1:00 am
Location: la plata - argentina

Configuration files vs Database. Design suggestion

Post by julian_lp »

Hello all. I'm fairly new to this forum, I am really impressed how good it is. I usually visit comp.lang.php as well

I've coded a catalog site, which has the hability to be configured as follows:

Number of items to display: x
Default Sort Order: x
Max price of the articles to show: x
...
...
...

Now the question.

Should continue reading those values (x) from the database, or would it be better to read them from a conf file?
Which option is better?
Note that every page load I've to have this info, so I'm thinking about perfomance

regards - julian
User avatar
Chris Corbyn
Breakbeat Nuttzer
Posts: 13098
Joined: Wed Mar 24, 2004 7:57 am
Location: Melbourne, Australia

Post by Chris Corbyn »

If you need this on every page load I'd suggest using a config file. It's easier to maintain (in most cases) in addition to the gain in performance too.

That said, it really depends upon just how many settings your config file would then contain as a result of this.

Some applications use a nice combination of config files and "defaults" stored in the database.

Next question. Do you consider your configuration directives as final, fixed entities or rather, a set of defaults which can be changed by the application? :)

:arrow: Moved to Theory & Design
User avatar
julian_lp
Forum Contributor
Posts: 121
Joined: Sun Jul 09, 2006 1:00 am
Location: la plata - argentina

Post by julian_lp »

d11wtq wrote:If you need this on every page load I'd suggest using a config file. It's easier to maintain (in most cases) in addition to the gain in performance too.
Ok.
d11wtq wrote: That said, it really depends upon just how many settings your config file would then contain as a result of this.
Some applications use a nice combination of config files and "defaults" stored in the database.
There are 15 settings or so.

d11wtq wrote: Next question. Do you consider your configuration directives as final, fixed entities or rather, a set of defaults which can be changed by the application? :)
Those values could be seen as "final", or could be changed by the owner of the store weekly, so again, they could be taken as "final". What do you think?
d11wtq wrote: :arrow: Moved to Theory & Design
I doubted posting here, I'll post here this kind of questions now
User avatar
feyd
Neighborhood Spidermoddy
Posts: 31559
Joined: Mon Mar 29, 2004 3:24 pm
Location: Bothell, Washington, USA

Post by feyd »

If it's 15 settings, and isn't expected to expand much, if at all, a file makes most sense to me. Creating a table to hold only 15 records feels... silly.
User avatar
julian_lp
Forum Contributor
Posts: 121
Joined: Sun Jul 09, 2006 1:00 am
Location: la plata - argentina

Post by julian_lp »

feyd wrote:If it's 15 settings, and isn't expected to expand much, if at all, a file makes most sense to me. Creating a table to hold only 15 records feels... silly.
Yes, in this scenario I think it'd work quite well, but what if the settings could be modified by the users?
Someone told me once, that it is not a good idea allowing users to write on a file simulaneously, that's what databases are written for...

Anyway, I'll chose the file option in this case.
User avatar
Christopher
Site Administrator
Posts: 13596
Joined: Wed Aug 25, 2004 7:54 pm
Location: New York, NY, US

Post by Christopher »

I'd say put the actual configuration stuff in a config file, and anything that users can set should probably be in the user record or records in a linked table.
(#10850)
User avatar
AKA Panama Jack
Forum Regular
Posts: 878
Joined: Mon Nov 14, 2005 4:21 pm

Post by AKA Panama Jack »

feyd wrote:If it's 15 settings, and isn't expected to expand much, if at all, a file makes most sense to me. Creating a table to hold only 15 records feels... silly.
Actually it is pretty silly to use a database to store configuration settings even if you have hundreds of settings.

The reason is because it takes MANY TIMES longer to process those settings from a database for each page load. The game that I work on, listed in my signature, used to use the database to store and process the configuration settings for each page load. Right now our configuration file has 272 variables.

What we decided to do is use a database table to store the config variables for easy manipulation by the admin.

Image

The name is the name of the variable.

The value contains the value stored in the variable.

The input_type is used by the configuration editor and controls the type of input used to edit the variable (radio, checkbox, list, number and text).

The input_selections are the valid sections to used by the input_type in the configuration editor.

The description is the description of the configuration variable displayed by the configuration editor.

The section is the section the variable is stored under as the configuration editor allows variables to be edited by section.

Image

This is the admin configuration editor that allows the editing of the configuration values based upon the data stored in the database.

Image

When the editor finishes editing any variable it writes out a variable file that is included by the page. The only difference is the admin has to use the provided editor to edit any variables. They can nolonger edit variables using something like phpMyAdmin.

I performed a wide range of speed tests using Microsofts Web Stress Test Application and pulling the configuration data from a database was horrendously SLOW compared to including a variable file. If a site is even moderately active this will put an unacceptable load on the server. I even used caching and even with query caching enabled it was horribly slow because the database data has to be processed into variables.

I even tried using a configuration class and that was just as bad if not worse because creating the configuration object is just as slow as pulling data from a database and converting it into variables.

I am a BIG proponent for using real world stress tests and in all cases using a PHP file that contains straight variables is many times faster than other methods.
User avatar
julian_lp
Forum Contributor
Posts: 121
Joined: Sun Jul 09, 2006 1:00 am
Location: la plata - argentina

Post by julian_lp »

AKA Panama Jack wrote:
feyd wrote:If it's 15 settings, and isn't expected to expand much, if at all, a file makes most sense to me. Creating a table to hold only 15 records feels... silly.
Actually it is pretty silly to use a database to store configuration settings even if you have hundreds of settings.

The reason is because it takes MANY TIMES longer to process those settings from a database for each page load. The game that I work on, listed in my signature, used to use the database to store and process the configuration settings for each page load. Right now our configuration file has 272 variables.

I am a BIG proponent for using real world stress tests and in all cases using a PHP file that contains straight variables is many times faster than other methods.
Very interesting kind of hack ;)
I'll keep it in mind for future projects !!!
User avatar
AKA Panama Jack
Forum Regular
Posts: 878
Joined: Mon Nov 14, 2005 4:21 pm

Post by AKA Panama Jack »

Not really a hack. :D

We came up with this procedure so there would be set limits on what the admin could change the various configuration values. An admin couldn't input some outlandish value and then complain about how things didn't work right. It maintains consistency. Using the database this way also makes it easier to edit by section without the need to load the entire configuration each time.
User avatar
julian_lp
Forum Contributor
Posts: 121
Joined: Sun Jul 09, 2006 1:00 am
Location: la plata - argentina

Post by julian_lp »

I called it "hack", beacause I thought once someting similar:

Code: Select all

<?php
include('config.php')

rest of the code.
?>

where config.php, would be a file uploaded by the admin (client of mine).
The problem was that he could upload a dangerous file:

Code: Select all

config.php
<?php

  Read all the f***king code and don't pay anymore for the service 

instead of

$max_items= 10;
$bla blab albb = xxxx;

?>
You solved that problem really smartly ;)
Roja
Tutorials Group
Posts: 2692
Joined: Sun Jan 04, 2004 10:30 pm

Post by Roja »

julian_lp wrote:Yes, in this scenario I think it'd work quite well, but what if the settings could be modified by the users?
Thats a key issue. Also, you didn't mention what you are hoping to accomplish - what are your requirements. Security? Memory use? Parsing speed?

By way of example, PanamaJack brings up the performance difference between config files and database loading. I've had very different experiences, with extremely similar data, primarily because we have vastly different requirements. I prefer to minimize the number of files that are writable by the server, to reduce security issues.

I don't put a heavy emphasis on memory use. Speed, however, is a concern of mine.

With those requirements, storing and retrieving ~200 config options from a database can be relatively quick. By using caching correctly (by which I mean caching individual records, not mysql's 'flush entire tables of cache data if one row changes'), you can keep the speed relatively quick. Without any caching, for the ~200 config options in my game, I was able to consistently load it with only a penalty of about two hundreths of a second per page. (The game generally renders a page in approximately 0.20-0.45 seconds). Thats hardly "horrendously slow". With proper caching, there is no measurable increase in time, except when the settings change (and then it is the previously mentioned timing).

Server load is a non-trivial concern. With proper caching, its extremely low, and without, its a reasonable increase. Its far below a 1% (of the games load - not server) increase on my server, making it within the margin of error between runs.

Put another way, you are seriously considering the performance impact of a *single* select pulling 20 rows. If THAT is horrendously slow, pretty much no dynamic site on the planet is fast enough for those absolute standards.

Making design choices based on absolutes is the epitome of the quote "Premature optimization is the root of all evil".

Set out your requirements, try different things, test, and see which meets your requirements.

My personal take, is that if the settings aren't likely to change often, and security isn't a primary concern, but memory use is, go with files. Otherwise, don't sweat a single select with 20 rows. Thats just silly.
User avatar
julian_lp
Forum Contributor
Posts: 121
Joined: Sun Jul 09, 2006 1:00 am
Location: la plata - argentina

Post by julian_lp »

Roja wrote: By way of example, PanamaJack brings up the performance difference between config files and database loading. I've had very different experiences, with extremely similar data, primarily because we have vastly different requirements.
So you've had different experiences? I think I should do a test by myself, as you mentioned...

Roja wrote: I don't put a heavy emphasis on memory use. Speed, however, is a concern of mine.
Speed of execution is my main concern in this case, specially beacause I'm back a shared server...


Roja wrote:
Set out your requirements, try different things, test, and see which meets your requirements.

My personal take, is that if the settings aren't likely to change often, and security isn't a primary concern, but memory use is, go with files. Otherwise, don't sweat a single select with 20 rows. Thats just silly.
Good advice. Thanks
Yossarian
Forum Contributor
Posts: 101
Joined: Fri Jun 30, 2006 4:43 am

Post by Yossarian »

You could consider a giving your store owner a simple ini file to control his variables.

Look at parse_ini_file() for how to use it in your scripts.

If you think he/she might abuse it, take a look in phpclasses.org, I once found a great ini script that gave the user a form to fill in that wrote the ini file. Would be a simple matter to attach some simple validation to that.

You could always get yourself a copy emailed whenever he/she changed the ini file, just in case.
User avatar
julian_lp
Forum Contributor
Posts: 121
Joined: Sun Jul 09, 2006 1:00 am
Location: la plata - argentina

Post by julian_lp »

Yossarian wrote:You could consider a giving your store owner a simple ini file to control his variables.

Look at parse_ini_file() for how to use it in your scripts.
Yes, but in this case I need parse_ini_file function call, whereas just including a php file, the assingnements are direct. (just thinking in the best speed scenario, of course)
User avatar
Christopher
Site Administrator
Posts: 13596
Joined: Wed Aug 25, 2004 7:54 pm
Location: New York, NY, US

Post by Christopher »

julian_lp wrote:Yes, but in this case I need parse_ini_file function call, whereas just including a php file, the assingnements are direct. (just thinking in the best speed scenario, of course)
First let me specifically address this comment. You need to remember that "just including a php file" is often slower than reading in a data file and parsing it. PHP's include parses the file as well.

But my more general comment, touched upon by Roja, is that this is a in part a performance discussion. And every performance discussion needs to be prefaced with the question: Is there a performance problem? Followed by the question: If so, how can we most cheaply and effectively solve that performance problem?

SImple solutions are better than complex ones. And, changing your software design should be towards the end of the list.

Premature optimization is a real problem in software development -- not a huge problem -- but a real one. And it is a problem that tends to wedge little bad design decisions here and there inside application that can effect surrounding code.
(#10850)
Post Reply