Page 2 of 4

Re: Ideas for lightweight ORM implementation

Posted: Fri Jun 05, 2009 10:32 am
by allspiritseve
inghamn wrote:I've been going the route of Active Record + Collections. The collection classes are where we can put all the SQL needed to handle returning a bunch of Active Record objects. With a flexible find function, you can do just about anything you want. And if there's some reporting you just can't figure out how to do with the Active Record object or the Collection's find function, you can admit defeat, and just write your own static function into the collection class to do whatever.
You know, you could just move all of your SQL out of that AR object and into the Collection...
inghamn wrote:Well, that's all I can think of right now. I've actually left relations off the list. I'm not sure I want to get bogged down in some custom mapping system from an ORM. The relationships fall into the easy stuff category, and best left to the developer.
Interesting. What is it about a 'custom mapping system' that you feel would bog you down?

Is this how you'd do relationships? (Assuming a DataMapper):

Code: Select all

$author = $userMapper->findById (10);
$posts = $postMapper->findByAuthor ($author);
foreach ($commentMapper->findByManyPosts ($posts) as $comment)  {
    $comments[$comment->getPostId()][] = $comment;
}
foreach ($posts as $post)   {
    $post->setAuthor ($author);
    $post->setComments ($comments[$post->getId()]);
}

Re: Ideas for lightweight ORM implementation

Posted: Tue Jun 09, 2009 3:15 pm
by inghamn
allspiritseve wrote: You know, you could just move all of your SQL out of that AR object and into the Collection...
The goal is to provide the cleanest API for the controllers and views to use. It's not necessarily to provide the most optimized, performance-based solution. As such, ActiveRecord style makes a whole lot of sense. The collections are just collections of ActiveRecord style objects.
allspiritseve wrote: Is this how you'd do relationships? (Assuming a DataMapper):
No, I don't use data mappers at all. They lead to very verbose code up the chain. Here's how I would do it:

Code: Select all

 
$author = new User(10);
foreach ($author->getPosts() as $post) {
  echo $post->getComment();
}
 
Although I'm not sure what the use case was you were trying to demonstrate with the second foreach. I guess if you wanted to add on a comment to all of an author's posts....

Code: Select all

 
$author = new User(10);
foreach ($author->getPosts() as $post) {
  $post->setComment("Something or other");
}
 
I think what you're wondering is how these relationships would show up in the class? Short example:

Code: Select all

 
class User
{
  public function getPosts()
  {
    return new PostList(array('user_id'=>$this_id));
  }
}
 
Support for passing in that associative array to the PostList classes find() method is what the developer would write as well. I've talked about this magic before here: viewtopic.php?f=19&t=99041

Re: Ideas for lightweight ORM implementation

Posted: Tue Jun 09, 2009 3:28 pm
by inghamn
In the most likely event that the post table has a user_id field, all those ActiveRecord Post objects would be hydrated by the default PostList class. This is what I'm referring to as the collection class ,which the developer is responsible for maintaining.

Code: Select all

 
class PostList extends ZendDbResultIterator
{
    public function __construct($fields=null)
    {
        parent::__construct();
        if (is_array($fields)) {
            $this->find($fields);
        }
    }
 
    public function find($fields=null)
     {
         $this->select->from('posts');
  
         if (count($fields)) {
             foreach ($fields as $key=>$value) {
                 $this->select->where("$key=?",$value);
             }
         }
  
         $this->populateList();
     }
  
     /**
      * Hydrates all the objects from a database result set
      *
      * @return array An array of Post objects
      */
     protected function loadResult($key)
     {
         return new Post($this->result[$key]);
     }
 }
 
It may be that my use of the word "collection" is not inline with the usual definition, but it seems like the closest word that applies here.

Re: Ideas for lightweight ORM implementation

Posted: Tue Jun 09, 2009 4:50 pm
by allspiritseve
inghamn wrote:No, I don't use data mappers at all. They lead to very verbose code up the chain.
I wasn't implying that you did. I just used it to show my examples, because I don't like AR.
inghamn wrote:Here's how I would do it:
Who retrieves the comments though? You implied from your earlier post that the ORM didn't fetch relationships for you.
inghamn wrote:I think what you're wondering is how these relationships would show up in the class? Short example:
Ah, Ok. So you're essentially lazy loading a call to the ORM. Our ORM will have same capability, except without the User object having to know about the ORM (and consequently, having a hidden dependency on the DB). Plans also include an Identity Map, Eager Fetching, Batch Lazy Loading, locking strategies, etc. You may not find all of those necessary, but the idea is to make them available if needed, but not required.

Re: Ideas for lightweight ORM implementation

Posted: Tue Jun 09, 2009 8:34 pm
by allspiritseve
What different ways of hydrating an object with data should we support? We have currently discussed:

Setting a property:

Code: Select all

$object->color = 'Red';
Setting methods:

Code: Select all

$object->setColor('red');
Setting generic methods (set() and get()):

Code: Select all

$object->set('color', 'Red');
Setting a constructor parameter:

Code: Select all

$object = new Object('Red');
Anything else we should support?

Re: Ideas for lightweight ORM implementation

Posted: Sat Jun 13, 2009 9:53 am
by inghamn
A handy one that I've started supporting is an associative array to hydrate:

Code: Select all

 
$object = new Object(array('color'=>'red'));
 
However, I limit this style usage to only when it completely hydrates the object, not as a way to just set a few properties. So a more typical call might look like:

Code: Select all

 
$something = new Something(array('id'=>2,'name'=>'car','color'=>'red'));
 

Re: Ideas for lightweight ORM implementation

Posted: Sat Jun 13, 2009 12:10 pm
by allspiritseve
OK. Maybe we could do something like this:

Code: Select all

$this->mapParamArray('id')->toColumn('id');
$this->mapParamArray('name')->toColumn('name');
$this->mapParamArray('color')->toColumn('color');

Re: Ideas for lightweight ORM implementation

Posted: Sat Jun 13, 2009 12:56 pm
by Christopher
Seems like both could easily be supported. Interesting thinking that you could initialize via property name in the constructor array and via setter otherwise.

Re: Ideas for lightweight ORM implementation

Posted: Sun Jun 14, 2009 8:59 am
by inghamn
Passing the associative array in the constructor was my recent performance optmization for my collection classes. It has to be done in a trusted manner, since the setters are where all the filtering happens. So, an object populated this was is being populated in an unfiltered manner.

The optimization was desired as a result of my design decisions (which ya'll are probably familiar with by now). If a collection needs to iterate over a ton of active record style objects, and the only way to instantiate them is to call the constructor with the primary key, then you will are causing n+1 database calls for populating the collection.

Code: Select all

 
// This will cause "select id from books where author_id=xx"
$books = new BookList(array('author'=>$author));
 
foreach ($books as $book) {
    // This has just caused a book to by hydrated by calling new Book(x)
    // select * from books where id=x
}
 
So the optimization was to allow for the collection classes to completely hydrate the objects to save on database calls.

Code: Select all

 
// This will cause "select * from books where author_id=xx"
$books = new BookList(array('author'=>$author));
 
foreach ($books as $book) {
    // This has just caused a book to by hydrated by calling new Book(array(....))
    // Since we selected all the data from the row, we can now completely hydrate the book
}
 

I would not use this method for processing data POSTed from forms, the controllers should still only use the setters. But model to model data passing can now potentially save on some database calls.

Granted this is at the expense of the extra memory to store the full database result. For really large result sets, I would probably use the select ID method and lazy load the objects from the database. The extra database calls would be minimized, since stuff like this is usually paginated, and we would only ever need to load a single page of book objects.

So having this gives me some targetted options when bulding stuff to match all the various user interface stuff that's desired. It's an example of the user interface desires impacting how the models are written. A collection whose objects are predominately shown in full, unpaginated form would probably use the select-*-array-load method. A collection whose objects are very numerous, or have a large number of fields thus requiring a lot of memory, might use the select-ID-with-lazy-load database call method.

Re: Ideas for lightweight ORM implementation

Posted: Tue Nov 03, 2009 12:17 pm
by allspiritseve
Hello all,

Arborint and I are just getting back into this project after a long hiatus. Here is a quick summary of where we are at. Comments and suggestions are welcome.

The skeleton framework code can be downlaoded here: http://code.google.com/p/skeleton/

The ORM is currently located in A/Orm with some supporting code in the examples/orm folder.
allspiritseve wrote:We are currently focusing on mapping to/from one object (though potentially mapping to two or more tables). As there are many ways to hydrate an object, we are trying to be flexible in what we allow developers to do.

To create your mappings, you would subclass A_Orm_DataMapper:

Code: Select all

class PostMapper extends A_Orm_DataMapper {
   public function __construct($db) {
      parent::__construct($db, 'Post','posts'); // mapping table posts to object Post. 
   }
}
Then, you would add your individual mappings in the constructor. Some examples:

Mapping properties:

Code: Select all

$this->mapProperty('title')->toColumn('title'); // maps $post->title to posts.title
Mapping methods:

Code: Select all

$this->mapMethods('getTitle','setTitle')->toColumn('title'); // maps $post->getTitle()/$post->setTitle() to posts.title
Mapping generic methods (get()/set()):

Code: Select all

$this->mapGeneric('title')->toColumn('title'); // maps $post->get('title')/$post->set('title','value') to posts.title
Mapping constructor parameters

Code: Select all

$this->mapParam()->toColumn('title'); // maps new Post($title) to posts.title
Note you can also call mapParams('id:key','title','body') which acts as if you called mapParam() for each one.

Auto mapping:

Code: Select all

$this->map('title'); // inspects $post and determines whether to access by property, by methods, or by generic methods
Note you can also call map() with multiple fields:

Code: Select all

$this->map('id:key','title','body');
Which acts the same as if you had called map() for each one.

We also intend on supporting XML mappings which essentially will be parsed and turned into calls to the above functions. I don't anticipate it being that complicated to code.
------------------------------------
Now, on the usage side, you're going to want to look in the examples/orm/ folder. HardCodedMapper currently contains all methods related to SQL. We are in the process of replacing our hard-coded SQL with calls to TableDataGateways. In order to get an example working, you will want to subclass HardCodedMapper (see JoinPostMapper for a working example).

Re: Ideas for lightweight ORM implementation

Posted: Tue Nov 03, 2009 12:41 pm
by inghamn
Rock on, guys. I'll take look here in a bit.

I'm putting the finishing touches on the addressing application I've been working on since this summer. For this project I've needed to support both Oracle and MySQL with the same code. I've been doing the Active Record+Collection classes like I've described, but have been using ZendDB instead of raw PDO.

It's been fairly successful

Re: Ideas for lightweight ORM implementation

Posted: Tue Nov 03, 2009 10:54 pm
by allspiritseve
inghamn wrote:Rock on, guys. I'll take look here in a bit.
That would be great. If there's any chance you like where it's headed, and want to help out, contact Arborint. We could use some more contributors.

Re: Ideas for lightweight ORM implementation

Posted: Wed Nov 18, 2009 10:47 am
by inghamn
Finally! Just finished that Master Address project. Way way over schedule, but what else is new?

Anyway in this project, I started using Zend_DB in my AR models, and can report that I kinda like Zend_DB. It simplified my List classes ( aka Collections) with me still using the flexible finder methods I've described.

I also ended up using Zend_Paginator. Zend_Paginator made things simpler, however it ended up moving a bunch of code and logic into the view and view partials. Basically, the partials are given a Zend_Paginator and are expected to hydrate their own objects from that. I'm not too crazy about having view partials hydrating their own.

Master Address is launched and a success here. I've started merging all the modifications I made to our scaffolding/framework back into it's trunk. I'm realizing that my framework latest stuff is relying on Zend_Framework for stuff. Not sure where I'm going from here, though. I feel like I might be slipping away from my original mandate, simple-light-fast, by relying on heavier frameworks inside my own framework.

I don't think I'm ready to adopt ZF for all my MVC stuff. I'm not crazy about how their context switching integrates with their views. And I'm still not sold on a front controller. (Well, I might still be able to be convinced of front controller).

Re: Ideas for lightweight ORM implementation

Posted: Wed Nov 18, 2009 2:54 pm
by Christopher
inghamn wrote:Anyway in this project, I started using Zend_DB in my AR models, and can report that I kinda like Zend_DB. It simplified my List classes ( aka Collections) with me still using the flexible finder methods I've described.
Can you give some quick before and after code examples of how it simplified the code?

Re: Ideas for lightweight ORM implementation

Posted: Wed Nov 18, 2009 4:26 pm
by inghamn
In my Active Record classes, it simplified my Save methods. I no longer had to do all the binding of parameters myself. Passing Zend_Db the associative array of data to be inserted or updated meant less code:

Original User->save() using PDO.

Code: Select all

 
class User
{
    public function save()
    {
        $this->validate();
 
        $fields = array();
        $fields['person_id'] = $this->person_id;
        $fields['username'] = $this->username;
        // Passwords should not be updated by default.  Use the savePassword() function
        $fields['authenticationMethod'] = $this->authenticationMethod
                                        ? $this->authenticationMethod
                                        : null;
 
        // Split the fields up into a preparedFields array and a values array.
        // PDO->execute cannot take an associative array for values, so we have
        // to strip out the keys from $fields
        $preparedFields = array();
        foreach ($fields as $key=>$value) {
            $preparedFields[] = "$key=?";
            $values[] = $value;
        }
        $preparedFields = implode(",",$preparedFields);
 
        // Do the database calls
        if ($this->id) {
            $this->update($values,$preparedFields);
        }
        else {
            $this->insert($values,$preparedFields);
        }
 
        // Save the password only if it's changed
        if ($this->passwordHasChanged()) {
            $this->savePassword();
        }
 
        $this->updateRoles();
    }
 
    private function update($values,$preparedFields)
    {
        $PDO = Database::getConnection();
 
        $sql = "update users set $preparedFields where id={$this->id}";
        $query = $PDO->prepare($sql);
        $query->execute($values);
    }
 
    private function insert($values,$preparedFields)
    {
        $PDO = Database::getConnection();
 
        $sql = "insert users set $preparedFields";
        $query = $PDO->prepare($sql);
        $query->execute($values);
        $this->id = $PDO->lastInsertID();
    }
}
 

New User->save() using Zend_Db

Code: Select all

 
class User
{
    public function save()
    {
        $this->validate();
 
        $data = array();
        $data['person_id'] = $this->person_id;
        $data['username'] = $this->username;
        // Passwords should not be updated by default.  Use the savePassword() function
        $data['authenticationmethod'] = $this->authenticationmethod
                                        ? $this->authenticationmethod
                                        : null;
 
 
        // Do the database calls
        if ($this->id) {
            $this->update($data);
        }
        else {
            $this->insert($data);
        }
 
        // Save the password only if it's changed
        if ($this->passwordHasChanged()) {
            $this->savePassword();
        }
 
        $this->updateRoles();
    }
 
    private function update($data)
    {
        $zend_db = Database::getConnection();
        $zend_db->update('users',$data,"id={$this->id}");
    }
 
    private function insert($data)
    {
        $zend_db = Database::getConnection();
        $zend_db->insert('users',$data);
        $this->id = $zend_db->lastInsertId('users','id');
    }
}