User defined data structure & plugin based data types

Not for 'how-to' coding questions but PHP theory instead, this forum is here for those of us who wish to learn about design aspects of programming with PHP.

Moderator: General Moderators

Post Reply
jobzesage
Forum Newbie
Posts: 5
Joined: Thu Oct 08, 2009 7:11 am

User defined data structure & plugin based data types

Post by jobzesage »

Hi, I'm working on an open source project where we're trying to build a webapp to serve as an address book. The particularity is that the data structure is fully defined by the user, further more, a user can have several data structures called contact types. To make it a little more complicated, the fields used in this address book are plug in based.

While designing this software, I quickly went for POO where plugins are classes that can extend an interface we will define.

Option 1 - the trivial
Now the tricky part, how do you retrieve the data from your database :-) the trivial solution is to load a set of Contact, to iterate through them, and to iterate through their fields. Then we can call a getString() type of methods in each field and display a list of contacts. For 10'000 contacts, 20 fields, that's 200'000 requests on the database... uhm.

Option 2 - an other more efficient approach
An other option is to have an object which is sort of a Column, and then I iterate the other way around. I go field by field and query the value for all the users. In that way the plugins can implement whatever they want. In fact they don't even need to use the actual database, but could get the data from different sources. Interesting perspective, but I don't know how important.

Option 3 - SQL request building
I ask the plugin to send me the information I need to build an SQL request that retrieves all the information at once for all the users. It's nice because it makes it easy to sort and filter, but isn't that mixing the controller layer and the model layer ? And then the plugin can actually bug the request if they are not done properly.


That's where I am at... any suggestion ??? Comments ? Idea ? Experience ?

I enclosed a document talking about the project that has some more information about this problem. But it's still a draft... just in case you wanna know more about it.
User avatar
JNettles
Forum Contributor
Posts: 228
Joined: Mon Oct 05, 2009 4:09 pm

Re: User defined data structure & plugin based data types

Post by JNettles »

I'm a little confused about your problem - why are you needing to load and iterate through all of these contacts? Is it to check them based on their different types or what? I'll give some shot-in-the-dark advice here though.

For data driven sites you'll need to implement well structured database design that makes strong use of relations. For example, let's say I'm storing information about a not-for-profit organization called Failstorm. I could easily create one massive table with a hundred fields that store everything there is to possibly know about this organization. A more likely situation though, is that I'll need to know pieces of information about the org at different times - a screen with a listing of all my orgs and some basic info to go with it, a contacts screen, etc. etc. So in my database I would have a table called Orgs with several fields indicating name, address, Org ID, etc. etc. Another separate table would be Org_Contacts and hold the Org_ID, Contact_ID, primary_contact, license_num, etc. etc.

I have two tables that relate to each other via their Org_ID's - which I can then link together via some kind of JOIN in my SQL if I need that extra contact data (likely, in most cases I won't) but I can still grab my basic org information without having the added bulk of everything else relating to the org.

Hope that helps, I'm still kind of sketchy on what you're trying to do.
jobzesage
Forum Newbie
Posts: 5
Joined: Thu Oct 08, 2009 7:11 am

Re: User defined data structure & plugin based data types

Post by jobzesage »

Thanks for your reply JNettles !

Sorry if I didn't manage to make my point very clear, but it's a quite complex problem (to my eyes at least ;-) I tried to enclose a documentation but I just realized that I can't enclose PDFs. But you can find it on the project's page http://sourceforge.net/projects/adres/ it's the file documentation.pdf

Gotta go, back in a minute
jobzesage
Forum Newbie
Posts: 5
Joined: Thu Oct 08, 2009 7:11 am

Re: User defined data structure & plugin based data types

Post by jobzesage »

OK I'm back, that was a long minute in face ;-)

What I'm trying to say, is that I cannot model something that doesn't exist. Plugins will be create by developers in the future, and I would like them to have the freedom to deal with data the way they want. The question is, how do you deal with that ?

Abstractions mechanisms in OOP lets you design a frame that designers have to fit in, but I don't want to do everything on a OOP level since the database system is much more efficient to sort, filter,...

I hope you get a little more the essence of the problem.

I hope someone will get what I'm trying to talk about ;-) but if you talk the time to have a look at the documentation, hopefully you'll get it.

Interesting design problem though
alex.barylski
DevNet Evangelist
Posts: 6267
Joined: Tue Dec 21, 2004 5:00 pm
Location: Winnipeg

Re: User defined data structure & plugin based data types

Post by alex.barylski »

Designing a plugin architecture is a complicated task, have you looked at existing systems:

WordPress
Drupal
Joomla
User avatar
PHPHorizons
Forum Contributor
Posts: 175
Joined: Mon Sep 14, 2009 11:38 pm

Re: User defined data structure & plugin based data types

Post by PHPHorizons »

Why can't you plugin a model? You've got the idea to plugin a controller, now you need to plugin a model to compliment it.

Cheers
User avatar
markusn00b
Forum Contributor
Posts: 298
Joined: Sat Oct 20, 2007 2:16 pm
Location: York, England

Re: User defined data structure & plugin based data types

Post by markusn00b »

A small nuisance: you're saying "an other" when you mean to say "another".

:D
jobzesage
Forum Newbie
Posts: 5
Joined: Thu Oct 08, 2009 7:11 am

Re: User defined data structure & plugin based data types

Post by jobzesage »

Thanks to all of you for you inputs !

@PCSpectra, yes I know it's complicated, but it's quite an important key when it comes to community development. I cannot ask everyone to understand the whole core before they can contribute. I've been writting a Wordpress plug-in, and tweaking quite some. But you're right, it's a complicated task. I'm not planing to make it as user friendly as Wordpress for now.

@PHPHorizons, thanks for this input, I'm actually quite new to the whole MVC concept. I've always separated the data from the rest, but not in three layers. The thing is that I haven't been that much in internet programming before.

But to all of you, my issue is not so much on how the plugin is implemented, but much more how my core is designed knowing the plugin is a black box. And that if I want to retrieve field by field, which would be easy to implement, as I mentioned it would take plenty of requests.

What we will most likely go for, is the option 2 or 3, together with a cache to speed up simple display of the data grid. I will do some testing and see what's the best practice.

Thanks to all of you, it's good to get some other brains to heat up on the same problem ;-)
josh
DevNet Master
Posts: 4872
Joined: Wed Feb 11, 2004 3:23 pm
Location: Palm beach, Florida

Re: User defined data structure & plugin based data types

Post by josh »

If your plugin system is going to limit the way they implement their data access, or modeling, you're probably going about it wrong. The best way IMO is an event based architecture, maximum cohesiveness, cuts down on global / "static singletons", they can use native code and subscribe to certain events, less potential for conflict in the "plugin ecosystem"
jobzesage
Forum Newbie
Posts: 5
Joined: Thu Oct 08, 2009 7:11 am

Re: User defined data structure & plugin based data types

Post by jobzesage »

Thanks Josh, some good hints about the implementation of a plugin powered system. I was planing to go event based, my problem is more on the data access level rather than how to implement the plugin.

But we'll go for building one SQL request out of some key information that each plugin will provide. A lot easier to filter and sort.

Thanks to all of you for the inputs
Post Reply