Data mapper - need to map an array of values to db w/filters

Not for 'how-to' coding questions but PHP theory instead, this forum is here for those of us who wish to learn about design aspects of programming with PHP.

Moderator: General Moderators

Post Reply
User avatar
Luke
The Ninja Space Mod
Posts: 6424
Joined: Fri Aug 05, 2005 1:53 pm
Location: Paradise, CA

Data mapper - need to map an array of values to db w/filters

Post by Luke »

I am writing an order management console (I think I have mentioned that a few times :)). I need to create an import system that is easy to extend. For now, I want to accept an array of values and map them to my data model. The thing is, I need to do things to certain columns:

- I need to filter some of the values (data comes in as YYYY-MM-DDTHH:MM:SS-(TIMEZONE-OFFSET) and it needs to map to Order::date as a YYYY-MM-DD field)
- I need to map parts of an input column to more than one model param (for instance if I get a full name for input--like "Luke Visinoni"--I need a function to break it apart and map it to Order::shipping_first_name and Order::shipping_last_name)
- Sometimes I need to do it the other way too... I need to map multiple input columns to one model param (If I get a shipping fee, a shipping tax, and a shipping discount, I need them added together and mapped to Order::shipping_fee)

I have begun this process, but I'm finding it difficult to come up with a good system that is extensible and easy to understand. I won't always be the one writing the importers, so I'd like it to be pretty straight-forward. Any ideas?

Oh, I should also mention (although it doesn't really matter because the concepts are the same) that I'm using cakephp. I should also mention that many times the data will map to several different models. For instance, the following would map to 3 seperate models (Order, OrderItem, and OrderCharge, and possibly even Customer although we aren't using it here):

Code: Select all

abstract class MC2_Importer
{
    /**
     * Maps fields to their respective function / value in this class
     */
    protected $_map = array();
    /**
     * Holds field values
     */
    protected $_data = array ();
    
    public function __construct($data) {
    
        $this->_data = $data;
    
    }
 
    /**
     * Maps input columns to functions / values
     */
    public function _map() {
    
 
    
    }
}
 
class MC2_Amazon_Importer extends MC2_Importer
{
    protected $_map = array(
        'order-id'                  => '',
        'order-item-id'             => '',
        'purchase-date'             => '',
        'payments-date'             => '',
        'buyer-email'               => '',
        'buyer-name'                => '',
        'buyer-phone-number'        => '',
        'sku'                       => '',
        'product-name'              => '',
        'quantity-purchased'        => '',
        'item-price'                => '',
        'item-tax'                  => '',
        'shipping-price'            => '',
        'shipping-tax'              => '',
        'ship-service-level'        => '',
        'recepient-name'            => '',
        'ship-address-1'            => '',
        'ship-address-2'            => '',
        'ship-address-3'            => '',
        'ship-city'                 => '',
        'ship-state'                => '',
        'ship-postal-code'          => '',
        'ship-country'              => '',
        'item-promotion-discount'   => '',
        'item-promotion-id'         => '',
        'ship-promotion-discount'   => '',
        'ship-promotion-id'         => '',
    );
}
I am not looking for anybody to write any code for me. I'm simply asking for inspiration. What design patterns would you use here? Why?
User avatar
Luke
The Ninja Space Mod
Posts: 6424
Joined: Fri Aug 05, 2005 1:53 pm
Location: Paradise, CA

Re: Data mapper - need to map an array of values to db w/filters

Post by Luke »

bump
User avatar
Christopher
Site Administrator
Posts: 13596
Joined: Wed Aug 25, 2004 7:54 pm
Location: New York, NY, US

Re: Data mapper - need to map an array of values to db w/filters

Post by Christopher »

It seems like you could do this with a standard Filter Chain that runs Filter objects. You would pass the input object through the chain and the Filters could do the modifications. It would be expandable because you could load filters by name like Helpers -- so it could be driven off of configuration data.
(#10850)
User avatar
Kieran Huggins
DevNet Master
Posts: 3635
Joined: Wed Dec 06, 2006 4:14 pm
Location: Toronto, Canada
Contact:

Re: Data mapper - need to map an array of values to db w/filters

Post by Kieran Huggins »

I'd likely start by creating the base model class, extend it for each model (nothing new here) using overloading to read and write my vars. (*Kieran ducks*) That way you can simply override certain getters and setters with custom functions.

If you need composite models, I have a feeling somethings wrong. Maybe what you really want is associations, like:

Code: Select all

$order->number
$order->customer->name
 
// rather than
$order->number
$order->customer_name
User avatar
Maugrim_The_Reaper
DevNet Master
Posts: 2704
Joined: Tue Nov 02, 2004 5:43 am
Location: Ireland

Re: Data mapper - need to map an array of values to db w/filters

Post by Maugrim_The_Reaper »

I'd go with arborint's advice. Use a filter chain to iterate across the input and manipulate a set of models in the background. Since filter chains are pretty simple to draw up, you can use a factory to build up chains from a config file once you have a working API that can be interpreted.
User avatar
Luke
The Ninja Space Mod
Posts: 6424
Joined: Fri Aug 05, 2005 1:53 pm
Location: Paradise, CA

Re: Data mapper - need to map an array of values to db w/filters

Post by Luke »

Thanks guys... that is exactly the advice I needed. :)
User avatar
Kieran Huggins
DevNet Master
Posts: 3635
Joined: Wed Dec 06, 2006 4:14 pm
Location: Toronto, Canada
Contact:

Re: Data mapper - need to map an array of values to db w/filters

Post by Kieran Huggins »

who won? :mrgreen:
User avatar
Luke
The Ninja Space Mod
Posts: 6424
Joined: Fri Aug 05, 2005 1:53 pm
Location: Paradise, CA

Re: Data mapper - need to map an array of values to db w/filters

Post by Luke »

do you mean something sort of like this, arborint?

Code: Select all

<?php
 
$rows = read_csv('/path/to/csv'); // fictional csv-reading function
 
/**
 * Step one: transform the data so that it has all the right fields (this does not do any formatting of the actual values, just the array structure)
 */
$trans = new MC2_Transform($rows);
$trans->removeColumn(12); // remove the 12th column
$trans->removeColumn('order-item'); // remove column with key of order-item
$trans->addColumn('some-column'); // adds a column called "some-column" with blank values
$trans->addColumn('some-other-column', 100); // adds a column called "some-other-column" with default value
$trans->removeRow(15) // remove 15th row
$trans->sum('shipping-fee', array('shipping-fee','shipping-tax','shipping-discount'), true); // combines all into specified name (first arg) (third arg $destructive means destroy the columns that it combined)
$trans->implode('shipping-address', array('shipping-address-1','shipping-address-2','shipping-address-1'), ' ', true); // implodes all into first arg (also has a destructive param)
$trans->explode('shipping-name', array('shipping-first-name','shipping-last-name'), ' '); // explodes shipping name into shipping-first-name and shipping-last-name
// etc
 
/**
 * Step two: filter / validate the data
 */
$data = new MC2_Input($trans->output());
$data->addFilter('purchase-date', MC2_Input_Filter_Date('Y-m-d h:g:ia')); // takes value in 'purchase-date', strtotime()'s it and outputs in format spec'd in Filter_Date
$data->addFilter(array('buyer-phone-number', 'ship-zip'), MC2_Input_Filter_Digits()); // filters out all but digits for given fields
// etc
 
/**
 * Step three: map / adapt the data to my model to fit import
 */
$map = new MC2_Mapper($data->output(), array(
    'purchase-date' => 'Order.date',
    'shipping-first-name' => 'Order.shipping_first_name',
    'shipping-last-name' => 'Order.shipping_last_name',
    'total' => 'Order.total',
    // etc
));
 
$order = new Order($map->output());
$order->save();
How could I adapt this to run off of some sort of configuration? Or could I?
User avatar
Christopher
Site Administrator
Posts: 13596
Joined: Wed Aug 25, 2004 7:54 pm
Location: New York, NY, US

Re: Data mapper - need to map an array of values to db w/filters

Post by Christopher »

I was thinking of a system with filter classes like AddColumnFilter, RemoveColumnFilter, RemoveRowFilter, SumFilter, ImplodeFilter and ExplodeFilter. You add the filter objects to the Filter Chain and then pass $rows to it. The filters would each run and modify $rows.
(#10850)
Post Reply