Shared memory question

PHP programming forum. Ask questions or help people concerning PHP code. Don't understand a function? Need help implementing a class? Don't understand a class? Here is where to ask. Remember to do your homework!

Moderator: General Moderators

Post Reply
itbegary
Forum Commoner
Posts: 34
Joined: Sun Jan 05, 2003 2:50 am

Shared memory question

Post by itbegary »

Here is the situation.

We have some applications writing in PHP for both CLI and web based applications. There is a table with about 20k rows in it at @ 1k in size per row. We also have a related table with about 100k rows at about 30 bytes per row. We do a lot of calculations on the data provided in these rows with multiple applications. Most if it's for reporting. Many of the reports use 5k of these rows at a time.

What we have found is the we spend most of the time loading the data for each row from the MySQL database to the application. Much of the data in the database is static for the day only but updated daily. As we load the data we apply various calculations per hour to the data as it's loaded.

We spend a lot of time loading data and applying the hourly rules to the daily data. The calculations cannot be stored in the database for a variety of reasons (business requirements not technical).

We have a class that we use when loading each of the data elements. As we apply this calculation to the data it's valid for 1 hour after it's loaded. I would like to keep this in some type of shared memory until it is expired. This way the data would be available precalculated for both the CLI and the web.

I have looked as some shared memory functions in c/c++ but I would like to find out what is best to use for PHP. I know that the first thoughts would be why not add the fields to MySQL and be done with it. I would like to but because of the type of business using this we cannot store the data to disk for a variety of business/legal reasons.

What are my best options for this in PHP?
kettle_drum
DevNet Resident
Posts: 1150
Joined: Sun Jul 20, 2003 9:25 pm
Location: West Yorkshire, England

Post by kettle_drum »

-Keep the php program running in the background and have it store the data in a var/array.

-encrypted in a file.

Even if you store it in RAM rather than on a hard disk people could still access it, by grabbing the data from certain address locations. And since your holding it for only an hour i would say an encrypted file would be your best bet.
itbegary
Forum Commoner
Posts: 34
Joined: Sun Jan 05, 2003 2:50 am

Post by itbegary »

The problem lies in the fact that it's not a single PHP program. It's multiple CLI's that are running against this data we well as multiple random web accesses of the same data.

Our prototype is also using files that we dump from an array of classes which works out well because we and deserialize the data from the file directly back into an object. But the problem is that changes that we make to the data cannot be written to the file. I'm less concerned with people access the box to get to the memory location. These are on dedicated boxes that have limited access.

As part of the information security requirements that we have we cant use a swapfile either for these systems. We have lots of physical ram (6gb per box - DL380's) to compensate though.

Are there any other shared memory ideas? We were also thinking of building a daemon to hold the data and connecting with some type of pipe on the local machine but this seems like it will waste more CPU time then just using a shared memory segment.

Gary
dave420
Forum Contributor
Posts: 106
Joined: Tue Feb 17, 2004 8:03 am

Post by dave420 »

CLIs don't have access to the shared memory, according to the documentation, so unless you're willing to set up a method for CLI to access your webserver-borne PHP code, you're not going to get access to it.

Encrypted file sounds good to me. If you're scared about performance, you can set up a quick ramdrive to store it on. You get effectively the same functionality as shared memory, yet it operates between anything that can read a drive (obviously relying on encryption to provide some security :))
itbegary
Forum Commoner
Posts: 34
Joined: Sun Jan 05, 2003 2:50 am

Post by itbegary »

I didn't think about the ram drive. Sometimes little things like that are just overseen. Security on this thing is already pretty tight so we really don't need to encrypt the files. The biggest thing is "What if the drives were yanked, could they recover the data" and now I can say definitively "no".

Thanks.
User avatar
Weirdan
Moderator
Posts: 5978
Joined: Mon Nov 03, 2003 6:13 pm
Location: Odessa, Ukraine

Post by Weirdan »

dave420 wrote:CLIs don't have access to the shared memory, according to the documentation
CLI don't have access to the shared memory on Windows systems. According to the documentation, of course ;)
itbegary
Forum Commoner
Posts: 34
Joined: Sun Jan 05, 2003 2:50 am

Post by itbegary »

Sorry, should have been more specific. The target environment is RH9 or RH EN3.
Post Reply