Mageploy

Easily keep track of changes and deploy synchronization between different Magento environments.

Download .zip Download .tar.gz View on GitHub

1. Introduction

If you are reading this documentation you are likely a developer who had the chance to work with Magento. There are several tools that were born with the intent of leveraging the development experience on Magento. Some favourite tools are N98magerun, Modman and Magentleman. Mageploy aims at becoming one of them. It's intended to facilitate the deployment process, keeping different environments synchronized; that's something new to any other tool, as far as we know.

What do we intend with the word "deployment"? If you have developed a complete web store with Magento you know that it's not all about code. It's also a matter of doing some activity (sometimes a lot of that) in the Admin Control Panel. Things like creating Attributes, associating them to brand new Attribute Sets, structuring Category trees, changing System Configuration and so on (and on). Once you have done all this kind of work in your development environment you have to be sure that it will be done exactly the same way on each other environment involved in the process: other developers' environments, staging environments and finally the production environment. So, with the word "deploy" we not only mean mantaining code consistency among different environments but also mantaining entities and configuration consistency.

Given that you already know how to mantain code consistency (you are on GitHub after all), Mageploy aims at automating the process of mantaining entities and configuration consistency without obliging you to bypass the usage of the Admin Control Panel. Infact, the only other way of mantaining such consistency (the so called "best practice") is that of writing install/upgrade scripts which create entities and change System Configuration programmatically. That's definitively boring and error prone.

1.1 About Mageploy

Mageploy has been developed on Magento CE v 1.7.0.2 and hasn't been tested with different CE or EE versions yet.

The idea of Mageploy was born within a discussion on Magento Stack Exchange named "What is the best deploy strategy?" that you can read at the following URL: http://magento.stackexchange.com/questions/252/what-is-the-best-deploy-strategy. In that discussion I asked to Magento developers what was their preferred deploy strategy and answers given to me made me understand that there wasn't any commonly adopted strategy, except for the "definitively boring and error prone" practice of doing a lot of code to mantain consistency among different environments.

So the idea of developing Mageploy took place and here we are.

The basic idea under Mageploy is that every action you perform in the Magento Admin Control Panel translates into an invocation of a method of a controller. These "action methods" take their parameters from the HTTP request object. So it's all about tracking and serializing these method calls and request parameters into a file and centralizing it under a version control system. Each action you perform on your environment not only is stored into a "global actions" file but also in a "local actions" file which is not versioned. Are you getting it? Actions performed in other enviroments result from the difference between global actions and your local actions, once you have checked out the global file.

Replicating these actions having stored method names and parameters should be a joke, isn't it? Obviously it's not so simple. There is a problem concerning the fact that every environment has its own IDs which you can't count on when they are part of a method's parameters. So we have to focus on finding a way to convert specific IDs into universally unique identifiers (UUIDs) so that we could do our data marshalling and demarshalling without worrying about specific IDs.

That's the complex part of Mageploy but that's also its strength and once we are able to perform a conversion from IDs to UUIDs and back we are able to replicate changes between different environments automatically and consistently. Converting IDs into UUIDs can sometimes force some limitations and there is also the risk of logical conflicts: think, for example, to two developers creating an attribute with the same code. Mageploy can facilitate our deployment tasks a lot but can't work magic; even Git can't automatically solve code conflicts after all.

1.2 Installation and Setup

To install Mageploy easily we suggest you to install Modman (Module Manager) by Colin Mollenhour; you can find the instructions to install Modman here: https://github.com/colinmollenhour/modman.

Once you have Modman installed, go to the root folder of your Magento project and type the following commands to initialize Modman:

$ modman init

and then, to clone Mageploy from its git repository via ssh:

$ modman clone git@github.com:pug-more/mageploy.git

After that, clear your Magento Cache, if enabled, and run Magento Compilation Process if Compilation is enabled. We assume you know how to perform such tasks. If not, you'd better know.

At this point Mageploy should be up and running but still needs some configuration changes.

Go to System -> Configuration -> Advanced -> Developer section and expand the "Mageploy" tab. Then you can change the values of the following configuration parameters:

  • Active: choose whether to activate action tracking. By default tracking is active. Activation can be performed faster through the command line, as we will see in the section dedicated to the command line tool.
  • Debug mode: if set to Yes, Mageploy will write logs even if logging is disabled in Magento.
  • Specific User Identification: by default it is set to "anonymous" and we encourage you to change this value and set a unique identifier among the different environments the project is developed and deployed on. This way, when you'll show the history of global changes, you will be able to distinguish who did what. Another reason to differentiate this value among different environments is that doing so you will enforce the probability to avoid conflicts among global actions.
  • Storage directory: specify the path where Mageploy will create CSV files. Paths starting without a slash '/' are considered relative to Magento base installation.

There is only one step remaining to begin taking advantage from the usage of Mageploy. Run the following command from the root folder of your Magento project:

$ php shell/mageploy.php --status

At this point you should see something similar to the following

Mageploy v 1.1.3 - tracking is not active - user is aronchi
There aren't any pending actions to execute.

Most important, your first invocation of Mageploy's command line tool has the effect of creating a folder named "mageploy" under a configurable folder whose default value is Magento's "var" folder and initializing the "mageploy_all.csv" and "mageploy_executed.csv" files. Put the "mageploy_all.csv" under version control and ignore "mageploy_executed.csv" from version control and you are almost done.

1.3 Basic Usage

Once Mageploy is configured and tracking is activated, actions you perform in certain sections of Magento Admin Panel are tracked both into "mageploy_all.csv" and "mageploy_executed.csv" files.

Replicating tracking in both files serves the purpose of instructing Mageploy about which actions have been executed locally (the ones in "mageploy_executed.csv" which has not to be put under version control) and don't have to be replicated while reading the global actions (the ones in "mageploy_all.csv", which has to be put under version control).

That's quite all you have to know, except that you have to learn how to use the Command Line Tool. See the following paragraph dedicated to the Command Line Tool for further instructions.

1.4 The Command Line Tool

The Command Line Tool allows you to monitor Mageploy's tracking status and history, enable or disable tracking, running pending actions that, once executed, should keep tracked areas of Magento easily aligned.

To get an overview of different commands you can issue on the Command Line Tool, you can try the following (assuming you are on the Magento root installation folder):

$ php shell/mageploy.php --help

The result will be somthing like the following:

Mageploy v 1.1.3 - tracking is not active - user is aronchi

Usage:	php mageploy.php --[options]

--h/help            to show this help
--t/track [val]     0 to disable tracking, any other value or blank to enable it
--hi/history [n]    Show the last n changes. Leave n blank to show all
--s/status          Show if there are any changes to be imported
--r/run [id]        Import changes for specified action (not recommended); leave id blank to import all

As you can see, you can use a short version for each option and it's what we suggest you to do, unless you are paid for the number of characters you type on a keyboard.

Now let's see how the different options operates.

1.4.1 Activating/Deactivating tracking

Sometimes deactivating trackers can be useful or necessary; for example you may need to change values in System Config that you don't want to propagate to other environments. Or maybe you need to create a test Attribute or Category and delete them after you have done your tests without the need to deploy these actions.

To deactivate all the trackers at the same time, you can use the following command:

$ php shell/mageploy.php -t 0

To activate all the trackers at the same time, you can use the following command:

$ php shell/mageploy.php -t 1

Note that omitting the value after the -t option corresponds to activating the tracking.

Remember that the same option can be changed via the Magento Admin Control Panel in the "Mageploy" tab under System -> Configuration -> Advanced -> Developer section. But it's far more easy to change it via the Command Line Tool, isnt' it?

1.4.2 Showing tracking history

To show the list of all tracked actions so far, type the following:

$ php shell/mageploy.php -hi

If you have tracked something so far, you should obtain something like:

Mageploy v 1.1.3 - tracking is not active - user is aronchi

Global Actions list:
ID: 1	 - Save existing Attribute Set with UUID 'Test Group' (aronchi on Thu Mar 14 11:25:25 2013)
ID: 2	 - Save existing Attribute Set with UUID 'Test Group' (aronchi on Thu Mar 14 11:29:41 2013)
ID: 3	 - Save existing Attribute Set with UUID 'Test Group' (aronchi on Thu Mar 14 11:34:54 2013)
ID: 4	 - Save existing Attribute Set with UUID 'Test Group' (aronchi on Thu Mar 14 11:35:15 2013)
ID: 5	 - Save existing Attribute Set with UUID 'Test Group' (aronchi on Thu Mar 14 11:35:45 2013)
ID: 6	 - Save Static Block 'footer_links' (aronchi on Thu Mar 14 13:43:06 2013)
ID: 7	 - Save Static Block 'footer_links' (aronchi on Thu Mar 14 13:45:55 2013)
ID: 8	 - Save Static Block 'footer_links' (aronchi on Thu Mar 14 13:48:42 2013)
ID: 9	 - Save Static Block 'footer_links' (aronchi on Thu Mar 14 13:50:18 2013)
ID: 10	 - Save Static Block 'sample_block' (aronchi on Thu Mar 14 13:55:29 2013)
ID: 11	 - Save Static Block 'customer-service' (aronchi on Thu Mar 14 14:03:17 2013)
ID: 12	 - Save Static Block 'another-sample-page' (aronchi on Thu Mar 14 14:05:22 2013)
ID: 13	 - Save new CMS Page 'another-sample-page' (aronchi on Thu Mar 14 14:19:43 2013)
ID: 14	 - Save existing CMS Page 'another-sample-page' (aronchi on Thu Mar 14 14:26:55 2013)
ID: 15	 - Save existing CMS Page 'customer-service' (aronchi on Thu Mar 14 14:35:38 2013)
ID: 16	 - Save existing CMS Page 'customer-service' (aronchi on Thu Mar 14 14:38:51 2013)

Total global actions listed: 16

You can specify a number, say 5, after the hi option in order to show the list of the last five actions tracked so far:

$ php shell/mageploy.php -hi 5
Mageploy v 1.1.3 - tracking is not active - user is aronchi

Global Actions list:
ID: 12	 - Save Static Block 'another-sample-page' (aronchi on Thu Mar 14 14:05:22 2013)
ID: 13	 - Save new CMS Page 'another-sample-page' (aronchi on Thu Mar 14 14:19:43 2013)
ID: 14	 - Save existing CMS Page 'another-sample-page' (aronchi on Thu Mar 14 14:26:55 2013)
ID: 15	 - Save existing CMS Page 'customer-service' (aronchi on Thu Mar 14 14:35:38 2013)
ID: 16	 - Save existing CMS Page 'customer-service' (aronchi on Thu Mar 14 14:38:51 2013)

Total global actions listed: 5

In case you never tracked anything before, you will simply get:

Mageploy v 1.1.3 - tracking is not active - user is aronchi

There aren't any actions tracked.

1.4.3 Showing tracking status

Say that you have just updated the "mageploy_all.csv" file from your preferred Source Control System; the first thing we suggest to do, at his point, is typing the following:

$ php shell/mageploy.php -s

Something similar to the following could appear:

Mageploy v 1.1.3 - tracking is not active - user is aronchi

Pending Actions list:
ID: 17	 - Save existing CMS Page 'about-us' (rgambuzzi on Thu Mar 15 15:17:47 2013)

Total pending actions: 1

What does it mean? Simply put, our working mate whose username is "rgambuz" has done some changes on the existing "about-us" CMS Page; he has tracked his changes and now Mageploy knows that this has to be replicated in your environment. That's where Mageploy can start showing off its potential.

1.4.4 Running pending actions

Once you have some pending actions to be executed, you can instruct Mageploy to run execute them all to keep different environments aligned. In other words, type:

$ php shell/mageploy.php -r

And you will get something similar to the following:

Mageploy v 1.1.3 - tracking is not active - user is aronchi

Action ID #17 - success The page has been saved.

Executed actions: 1/1

Note that you can also run pending actions one by one specifying the action's ID after the run option. But pay attention: doing so you run the risk to break the correct sequence in which actions were executed and this can cause problems. If, for example, you run the action that associates an Attribute to an Attribute Set before the Attribute is created, you can imagine that this will cause some errors. So we advice you not to run actions one by one unless you are really sure of what you are doing. We implemented this function mainly for debugging purposes.

1.4.5 Changing the user's name

In Mageploy v 1.1.7 we have introduced the ability to set current user's name via command line, as shown below:

$ php shell/mageploy.php -u myusername

1.5 Summary

Mageploy is really easy to install provided you know what Modman is and how to use it. Once installed configuration is a matter of flushing Magento's cache and compilation, setting up your username and putting the {{configurable_folder}}/mageploy_all.csv file under version control.

Once you switch Mageploy's tracking function on, you will get your actions recorded (depending on which trackers are implemented and activated) and have the ability to synchronize different environments running actions executed by others. That's simply done by updating the versioned mageploy_all.csv file and using the Command Line Tool.

2. Trackers

Trackers are a core part of Mageploy and are designed following "the Magento way" of doing things. Doing so, trackers can easily be added to Mageploy by external modules using configuration directives.

Mageploy comes bundled with some trackers designed to keep track of some Magento core functionalities.
Mageploy, however, can't know in advance the internals of custom modules; so trackers of custom modules should be implemented by third parties, typically the same who implements the module itself.

The key functionality for a Tracker is the one of converting specific IDs into Universally Unique Identifiers (UUIDs).
Without this ability, a Tracker couldn't even exist.
In the following sections we will describe how we implemented IDs to UUIDs conversion for out-of-the-box trackers adding some considerations about limitations when needed.

Converting IDs into UUIDs is done in the encoding method which is called every time you invoke a Controller's Action.
Converting back UUIDs into IDs is done in the decode method invoked by the Command Line Tool every time it excutes pending actions.

At the moment this is all you have to know about Mageploy's internals in order to understand what's next.
We will examine all this in more depth in the Extending Mageploy section.

2.1 About Trackers

When Mageploy was still only an idea in our minds, we needed a "proof of concept" which could demonstrate that a tool like Mageploy could be really useful to developers and, first of all, to ourselves.
So we wrote up a list of "most desirable" Trackers, that is a way of listing things that were boring to implement in Magento every time we had to start a new project.

At the top of that list there were Attributes and Attribute Sets.
Every time a new project starts on Magento, you likely have to set up new entities; creating new Attributes and associating them to Attribute Sets and Groups is a frequent task. What's also frequent is deleting them, changing their options (is it searchable? Is it visible on Frontend? Is it used for Simple or other kind of Product types? And so on), assigning them to Sets and Groups etc.

Best Magento practices impose to do all this thing via source code, that is, developing install and upgrade scripts in custom Modules which programmatically do all the sort of tasks described above.

But, you know, the knowledge of the entire system comes with time and each time you (or your customer) change your mind about some detail you have to write an upgrade script in order to propagate these changes.

Frankly: all this is boring and error prone, especially considering that Magento comes with a convenient Admin Panel which let you easily do this stuff. What's more, delegating the creation of such entities to developers means cutting away store managers and web designers from giving their contribute to setting up the store. Not realistic and, from developers' point of view, not funny.

That's where Mageploy comes to hand: you can continue using the Admin Panel without worrying about propagation and synchronization, provided a tracker is silently working in the background registering your actions.

2.2 Attribute Tracker

The Attribute Tracker tracks down the action of creating and updating an Attribute.

Implementing its core wasn't so hard because Attributes comes with a built-in universally unique identifier (or UUID) which is the Attribute Code. What's more, once you create an Attribute in Magento Admin Panel, you can't change its Code so you can be sure that the UUID won't change in time.

So, encoding and decoding of the IDs is really easy and follows the easy rule below:

  • Attribute UUID = Attribute Code

The hardest part comes when an Attribute has some Options associated, because they don't have an immutable identifier.

In order to make things work we were obliged to do our first assumption: Attribute Options Admin values should be immutable.

Thanks to the above assumption, it's easy to keep track of changes to existing options and distinguish them from new options created at a later stage.

Assuming we use a string separator called UUID_SEPARATOR in UUIDs that serves the function to split different parts while decoding them, the Options rule is the following:

  • Existing Option UUID = 'existing_opt'.UUID_SEPARATOR.'<Option Admin Value>'
  • New Option UUID = 'new_opt'.UUID_SEPARATOR.'option_<option_number>'

To decode an existing Option UUIDs we can extract the Admin Value and, knowing the Attribute it belongs to, find its specific ID. This works as far as we respect the assumption of leaving the Admin value unchanged over time.

From a new Option UUIDs, instead, we simply extract the 'option_<option_number>' and give it back to the Controller's Action. This way the new Option will be generated for the Attrbute the Option belongs to.

2.3 Attribute Set Tracker

The Attribute Set Tracker tracks down the action of creating and updating an Attribute Set, creating and updating an Attribute Group and associating existing Attributes to a (new or existing) Group of a (new or existing) Attribute Set.

This was our real proof of concept for Mageply because here difficulty scales up a bit. That's dued to the fact that in some circumstances we can't rely on an explicit UUIDs like the Attribute Code.

As far as we have to deal with creating a new Attribute Sets, things are simple because Magento requires that different Attribute Sets should have different names, so:

  • Attribute Set UUID = Attribute Set Name

From now on, we split topics into sub topics because on an Attribute Set you can associate Attribute to Groups, create new Groups, remove the association between an Attribute and a Group and remove Groups.

2.3.1 Associatig Attributes to Groups

The structure we have to deal when handling the association of each Attribute to Groups is a list of indexed arrays like the following:

0 => <attribute_id>
1 => <attribute_group_id>
2 => <sort_order>
3 => <eav_attribute_id> //empty if it's a new association

So here we have to convert different IDs into UUIDs:

  • Attribute UUID = Attribute Code
  • Existing Attribute Group UUID = Attribute Group Name . UUID_SEPARATOR . Attribute Set Name
  • New Attribute Group UUID = the <attribute_group_id> value in the array
  • EAV Attribute UUID = Attribute Group UUID . UUID_SEPARATOR . Attribute UUID

2.3.2 Creating and Updating Groups

The structure we have to deal when handling the creation of Groups is a list of indexed arrays like the following:

0 => <attribute_group_id> or "ynode-<NUM>" for a new Group
1 => <attribute_group_name>
2 => <sort_order>

So, the only ID we have to convert is the one of the Group. Using the name can't be sufficient, as Attribute Groups within different Attribute Sets can have the same name.
For this reason we concatenate the Group Name and the Attrbute Set Name as the latter can't be repeated for the same entity type (the Product, in current scope):

  • Attribute Group UUID = Attribute Group Name . UUID_SEPARATOR . Attribute Set Name

Note that as we load the Attribute Group from the DB, changing the name of the Group is possible even if it's part of the UUID.
That's because in environments where the action of renaming the Attribute Group hasn't been run yet, the Group Name is the one in the UUID, so the Group can be identified. Provided no one renames the same Group in the same time but this would be considered a conflict and conflicts can't be resolved by Mageploy; humans, in this case, have to step in and do their part.

2.3.3 Removing Attributes from Groups

The structure we have to deal when handling the deletion of Attributes from an Attribute Set is an indexed array whose values are the Entity IDs of the Attributes to delete.

We have an Entity Attribute ID because the same Attribute can be associated to different Sets and Groups, so the data we have to deal with is a bit more structured.
This means we have to identify ad Attribute within specific Set and Group and we do this by providing the following UUID:

  • Entity Attribute UUID = Attribute Group UUID . UUID_SEPARATOR . Attribute UUID

That is, as you can guess:

  • Entity Attribute UUID = Attribute Group Name . UUID_SEPARATOR . Attribute Set Name . UUID_SEPARATOR . Attribute UUID

2.3.4 Removing Groups

The structure we have to deal when handling the deletion of Groups is an indexed array whose values are the IDs of the Groups to delete.

So encoding of such IDs is the same we have already seen above:

  • Attribute Group UUID = Attribute Group Name . UUID_SEPARATOR . Attribute Set Name

2.4 Category Tracker (uncomplete)

The Category Tracker tracks down the action of creating and updating a Category.
This Tracker is not complete in Mageploy v 1.1.3 because it doesn't track the Thumbnail and the Image uploads.

We didn't deal with handling uploads at all in Mageploy 1.0.0; it will be certainly part of the very next improvements we have in mind.

The Category Tracker wasn't as complex to implement as the Attribute Set one but we had to deal with several topics:

  • Category ID
  • Store ID
  • Parent Category ID
  • Associated Products
  • Moving of the Category in the hierarchy

From now on, we split the above topics into sub topics in order to keep things more clear.

2.4.1 Category ID

Categories in Magento don't have a unique code out of the box as Attributes do. What's more, you are allowed to create two or more Categories with the same name under the same parent Category.

In order to be able to convert a Category ID into a Category UUID we had to assume that Categories with the same parent Category won't ever have the same name.

Maybe it could have been easier to use the Category URL as UUID but we didn't want to deal with URL rewrites and chose to use a different algorithm.

Provided we don't have to convert the ID for new Categories, for existing ones the UUID is the concatenation of the name of all the Categories in the Category path, ending with current Category name.

  • Category UUID = Full Named Path

In fact, assuming that there will never be two Categories with the same name under the same parent Category, such a path will uniquely identify a Category.

2.4.2 Store ID

Identifying a UUID from a Store ID is a trivial taks because Stores have unique codes in Magento, so:

  • Store UUID = Store Code

2.4.3 Parent Category ID

For the parent Category ID, refer to the conversion used for the Category ID in Paragraph 2.4.1.

2.4.4 Associated Products

Producs you associate to a Category through the "Category Products" tab are passed as product_id=position string segments concatenated with an & character; here is an example:

1=0&11=1&12=2&13=3"

So, converting such a string is a matter of converting each Product ID into a Product UUID and this is quite easy because Products have a native unique identifier represented by the SKU, so:

  • Product UUID = Product SKU

2.4.5 Moving of the Category in the hierarchy

Moving a Category in the hierarchy means handling information about the new parent Category and the Category after which current Category has to be placed. So we are again in the case of converting a Category ID and you can refer to paragraph 2.4.1.

2.5 System Config Tracker (uncomplete/experimental)

The System Config Tracker in Mageploy v 1.1.3 is not intended to be fully functional. The Magento System Config is a complex set of configuration sections, each of which should be deeply analyzed. For each section a specific tracker should be written in order to do things the right way.

Instead, at the moment, the System Config Tracker consists of a single tracker which doesn't convert any ID but simply encodes and decodes posted values as is.

Despite this big limitation, the System Config Tracker can be very useful to track changes of a lot of config values which are intrinsically independent from specific IDs.

2.6 CMS Page Tracker

The CMS Page Tracker tracks down the action of creating and updating a CMS Page.

This tracker wasn't complex to implement; the only aspect we had to deal with carefully was the association of a Page to one ore more Store Views and, thus, the conversion of Store IDs through the following rule:

  • Store UUID = Store Code

2.7 Static Block Tracker

The Static Block Tracker tracks down the action of creating and updating a Static Block.

Its implementation is equivalent to the CMS Page Tracker, so you can refer to it for further details.

2.8 Summary

Trackers are the pieces of code which let Mageploy record actions which can be rerun on different environments via the Command Line Tool in order to keep things synchronized.

In this section we explained the basic algorithms undergoing the built-in Trackers in order to give you a reference for the development of new Trackers.

3. Extending Mageploy

As we mentioned above, Mageploy is developed "the Magento way" meaning that you can extend Mageploy and deploy your own Trackers.

Trackers can be built inside Mageploy's module or inside your own module simply by extending a Configuration node and registering your own classes.

Our idea is that Trackers dealing with Magento's core functionalities should be delivered directly as Mageploy's built-in Trackers meanwhile third party Modules could provide their own Trackers.

Provided that third party Trackers are not mandatory, once you begin working with Mageploy you will desire to have a Tracker for every section. Count on it.

3.1 Tracker Basics

Before explaining how Trackers work let's explain how they are activated.

3.1.1 The Request Observer

The tracking functionality is based upon an Observer's method, observeRequest(), which is invoked each time Magento dispatches the controller_action_predispatch event.

The observeRequest(), implementation is very simple as shown below:

public function observeRequest($observer) {
    $helper = Mage::helper('pugmore_mageploy');
    if (!$helper->isActive()) {
        return;
    }

    $request = Mage::app()->getRequest();
    $funnel = Mage::getModel('pugmore_mageploy/request_funnel')
            ->init(Mage::getSingleton('pugmore_mageploy/io_file'))
            ->dispatch($request);
}

The first code block checks whether tracking is active; if not nothing has to be done. Easy.

The second code block instantiates the so called "Request Funnel" object, injects into it the object responsible of writing things into files and calls the dispatch() method, passing the Request Object as its only parameter. Done. Easy too.

The reason why the I/O object is injected is that we didn't want to bound Mageploy to the initial decision of using files to synchronize things. If one day another better method will arise, we can simply change the object responsible of writing things and all should continue working without worrying about what comes after the dispatch() invocation. This way of doing things is named "Dependency Injection" and follows the "Inversion of Control" design pattern which we are great fan of. Magento is not implemented this way but Magento 2 is, and this is a good piece of news.

3.1.2 The Request Funnel

The Request Funnel object is responsible of taking the HTTP Request object, checking Magento configuration to retrieve and instantiate all the Trackers which can be interested in the action being submitted and, for each of them, calling their encode() method.

The encode() method of each Tracker will return the result which the I/O injected Object is responsible to store.

Even if more than a Tracker can be activated upon an action submission, at the moment the Trackers we developed react once at a time.
The tracking of the System Configuration, if developed the right way, is a good candidate to expose more than a Tracker reacting at the same time to a same action submission. But actually it is not.

The Request Funnel logic con be divided into two main blocks: the initialization one and the dispatching/recording one.

The init() method's implementation is shown below:

public function init($io) {
    $this->_io = $io;

    Mage::dispatchEvent('mageploy_funnel_collect_actions_before', array('funnel'=>$this));

    $actionsInfo =  Mage::getConfig()->getNode(self::XML_ACTIONS_PATH)->asArray();

    Varien_Profiler::start('mageploy::funnel::collect_actions');
    foreach ($actionsInfo as $actionCode => $actionInfo) {
        if (isset($actionInfo['disabled']) && $actionInfo['disabled']) {
            continue;
        }
        if (isset($actionInfo['class'])) {
            $action = new $actionInfo['class'];
            $this->addAction($actionCode, $action);
        }
    }
    Varien_Profiler::stop('mageploy::funnel::collect_actions');

    Mage::dispatchEvent('mageploy_funnel_collect_actions_after', array('funnel'=>$this));

    return $this;
}

It's a very simple block of code, mainly inspired by the init() method of the Mage_Core_Controller_Varien_Front class.

It simply access Magento Configuration nodes in order to collect all registered trackers. Once a Tracker node is found and provided it's not disabled and is associated to a class it's stored in a protected list that will be iterated during the dispatching/recording phase.

The dispatch() method's implementation is ridiculously simple as shown below:

public function dispatch($request)
{
    foreach ($this->getActions() as $action) {
        if ($action->setRequest($request)->match()) {

            $this->record($action);
        }
    }
}

The dispatch() method iterates the list of collected Trackers and invoke their record() method.

The record() method's implementation is very simple too, as shown below:

public function record($action) {
    Mage::dispatchEvent('mageploy_funnel_record_action_before', array('funnel'=>$this, 'action'=>$action));

    Varien_Profiler::start('mageploy::funnel::record_action');

    Mage::helper('pugmore_mageploy')->log("Should record '%s'", $action->toString());
    $result = $action->encode();
    ksort($result);
    $this->_io->record($result);

    Varien_Profiler::stop('mageploy::funnel::record_action');

    Mage::dispatchEvent('mageploy_funnel_record_action_after', array('funnel'=>$this, 'action'=>$action));

    return $this;
}

Given a Tracker (the $action parameter), the record() method calls encode() upon it and demands the I/O Object to store the result.
The result is an array and the ksort($result) ensures that the sort order of its colums will be preserved.

As you have seen, the tracking logic is extremely simple. The hardest part, as you can guess, is demanded to the encode() and decode() methods of each Tracker.

3.2 Adding a Tracker

To add a Tracker, the first step is to register the configuration node which will be read by the Funnel Object during the init() method, as seen before.

Regardless of the fact that you are working on Mageploy's code base or in your own module, the structure of the configuration node is the same, and is shown below:

<config>
    <default>
        <mageploy>
            <actions>
                <your_tracker_identifier>
                    <disabled>0</disabled>
                    <class>Vendor_Module_Model_ClassName</class>
                </your_tracker_identifier>
            </actions>
        </mageploy>
    </default>
</config>

If you are extending the Mageploy's code base, the Vendor_Module_Model_ClassName is recommended to be PugMoRe_Mageploy_Model_Action_<YourClassName>.

The <disabled> tag allows you to disable a the Tracker by specifying values other than 0.

The <class> tag should containg the fully qualified name of an existing class extending PugMoRe_Mageploy_Model_Action_Abstract base class.

3.3 Developing a Tracker

As mentioned above, your Tracker class should extend the PugMoRe_Mageploy_Model_Action_Abstract base class.

The base class already provides an encode() method which however doesn't do anything useful. That means that, despite you should implement your own method, PHP interpreter won't complain if you don't.

On the other hand, the base class also declares an abstract function decode(...), which means that you are obliged to provide an implementation as far as you don't want the PHP interpreter to complain about a missing method in your class.

3.3.1 A sample Tracker

Let's show how to implement a StoreView Tracker in order to learn the basics of Tracker implementation.
We will implement it into Mageploy's code base, so here is our configuration node:

<config>
    <default>
        <mageploy>
            <actions>
                <store_storeview>
                    <disabled>0</disabled>
                    <class>PugMoRe_Mageploy_Model_Action_Store_StoreView</class>
                </store_storeview>
            </actions>
        </mageploy>
    </default>
</config>

And here is the very first implementation of the Tracker class:

class PugMoRe_Mageploy_Model_Action_Store_StoreView extends PugMoRe_Mageploy_Model_Action_Abstract {
    const VERSION = '1';

    protected $_code = 'system_store';
    protected $_blankableParams = array('key', 'form_key');

    protected function _getVersion() {
        return Mage::helper('pugmore_mageploy')->getVersion(2).'.'.self::VERSION;
    }

    public function match() {
        if (!$this->_request) {
            return false;
        }

        if ($this->_request->getModuleName() == 'admin') {
            if ($this->_request->getControllerName() == 'system_store') {
                if (in_array($this->_request->getActionName(), array('deleteStorePost'))) {
                    return true;
                }
                if (in_array($this->_request->getActionName(), array('save'))) {
                    if ($this->_request->getParam('store_type') == 'store') {
                        return true;
                    }
                }
            }
        }

        return false;
    }

    public function encode() {
        $result = parent::encode();
        return $result;
    }

    public function decode($encodedParameters, $version) {
        // The !empty() ensures that rows without a version number can be
        // executed (not without any risk).
        if (!empty($version) && $this->_getVersion() != $version) {
            throw new Exception(sprintf("Can't decode the Action encoded with %s Tracker v %s; current Store View Tracker is v %s ", $this->_code, $version, $this->_getVersion()));
        }

        $parameters = $this->_decodeParams($encodedParameters);
        $request = new Mage_Core_Controller_Request_Http();
        $request->setPost($parameters);
        $request->setQuery($parameters);

        return $request;
    }

}

The const VERSION = '1'; is used to keep track of internal Tracker version. This number should change every time you make a change in the encoding logic in order to prevent Mageploy to decode actions encoded with different algorithms, which will likely bring to an error.

The protected $_code = 'system_store'; should uniquely identify the Tracker; this string is used in logging and printing and is usually identical to corresponding action name even if this is not mandatory.

The _getVersion() function can't be generalized because it accesses the local VERSION constant but it's basically always the same code. We admit this is not something we like much.

The match() function should return true when current Tracker intercepts an action it is interested to track.

typically we track actions which belongs to the Mage_Adminhtml Module, whose name is admin; in particular, the Store View Tracker should intercept the save and deleteStorePost action belonging to the system_store Controller. Note that in case of save, a match should occurre if and only if the value of the store_type Request parameter equals store. This logic is implemented in the portion of the match() method implementation shown below:

if ($this->_request->getModuleName() == 'admin') {
    if ($this->_request->getControllerName() == 'system_store') {
        if (in_array($this->_request->getActionName(), array('deleteStorePost'))) {
            return true;
        }
        if (in_array($this->_request->getActionName(), array('save'))) {
            if ($this->_request->getParam('store_type') == 'store') {
                return true;
            }
        }
    }
}

At this point our Tracker will intercept and match the save action of the Mage_Adminhtml_System_StoreController but it doesn't anything interesting yet.

If we take a look at the Request parameters submitted upon saving an existing Store View, we have a structure similar to the following:

[form_key]	string	"XevyrSTGSeiLZ5hj"
[store]	array[7]
  [group_id]	string	"1"
  [name]	string	"Secondary Store View"
  [code]	string	"secondary"
  [is_active]	string	"1"
  [sort_order]	string	"0"
  [is_default]	string	""
  [store_id]	string	"3"
[store_type]	string	"store"
[store_action]	string	"edit"

In case we save a new Store View the structure is the same with a difference: the store_id value is (reasonably) an empty string.

So, what our encoding function should do? It should:

  • eliminate useless parameters that can cause errors while executing back; in our example it's the form_key;
  • convert the group_id specific ID into its relative UUID. Because Store Groups don't have any unique code, we have to impose the (reasonable) assumption that we won't have different Store Groups with the same name.
  • convert the store_id specific ID into its relative UUID, if it is present. It's easy, because Store Views have a unique code and this will represent our UUID.

The decoding function should implement the steps which permit to obain the same structure with specific IDs starting from the unspecific one generated by the encoding process.

Once we implement the encode and deencode functions we are almost done; Mageploy will do the rest.

The result that the encode() method will return is an indexed array whise colums are explained below:

  • INDEX_ACTION_TIMESTAMP (index: 0): it's used to store the timestamp the result was generated; it's populated by the base abstract class and you should not need to change it.
  • INDEX_ACTION_USER (index: 1): it's used to store the user which accomplished the action; it's populated by the base abstract class and you should not need to change it. The value is taken from Mageploy's configuration and it's strictly recommended that each user sets his own username.
  • INDEX_ACTION_DESCR (index: 2): it's used to store a specific description, so the encode() method should specifically populate this field. This is the message you will see when issuing a status or history command via the Command Line tool.
  • INDEX_EXECUTOR_CLASS (index: 3): it's used to store the name of the class used to encode and decode current action. This is likely current Tracker class name.
  • INDEX_CONTROLLER_MODULE (index: 4): it's used to store the Module name of the Controller which implements the tracked action. typically it's current Controller's Module name but in some circumstances it can be different, so specifying it is upon the encode() method.
  • INDEX_CONTROLLER_NAME (index: 5): it's used to store the Controller name, which is typically current Controller name.
  • INDEX_ACTION_NAME (index: 6): it's used to store the Action name, which is typically current Action name.
  • INDEX_ACTION_PARAMS (index: 7): it is used to store the parameters needed by the action in order to be called again by the Command Line Tool. Parameters are serialized and then Base64 encoded in order to avoid problems while storing them in files. The functions to encode and decode parameters are already provided by the abstract base class.
  • INDEX_VERSION (index: 8): it is used to store a version number used in the decode() method to check decoding eligibility. The version number is obtained concatenating the first two digits of Mageploy's version number with current Tracker's version. Infact, when encoding or decoding logic changes in the base abstract class, the first and/or the second digits of Mageploy's version number should be changed too, in order to prevent deconding of rows encoded with different logics.

And here is the complete code of the Store View Tracker.

class PugMoRe_Mageploy_Model_Action_Store_StoreView extends PugMoRe_Mageploy_Model_Action_Abstract {
    const VERSION = '1';

    protected $_code = 'system_store';
    protected $_blankableParams = array('key', 'form_key');

    protected function _getVersion() {
        return Mage::helper('pugmore_mageploy')->getVersion(2).'.'.self::VERSION;
    }

    public function match() {
        if (!$this->_request) {
            return false;
        }

        if ($this->_request->getModuleName() == 'admin') {
            if ($this->_request->getControllerName() == 'system_store') {
                if (in_array($this->_request->getActionName(), array('deleteStorePost'))) {
                    return true;
                }
                if (in_array($this->_request->getActionName(), array('save'))
                    && $this->_request->getParam('store_type') == 'store') {
                        return true;
                }
            }
        }

        return false;
    }

    public function encode() {
        $result = parent::encode();

        if ($this->_request) {
            $params = $this->_request->getParams();

            // convert Group ID
            if (isset($params['store']) && ($groupId = $params['store']['group_id'])) {
                $group = Mage::getModel('core/store_group')->load($groupId);
                if ($group->getId()) {
                    $params['store']['group_id'] = $group->getName();
                }
            }

            $new = 'new';
            $actionName = $this->_request->getActionName();
            if (isset($params['store'])) {
                $storeCode = $params['store']['code'];
            }

            // Convert Store ID
            if (isset($params['store']) && $storeId = $params['store']['store_id']) {
                $new = 'existing';
                $params['store']['store_id'] = $storeCode;
            }

            // Convert Item ID (for deleteStore action)
            if ($itemId = $params['item_id']) {
                $new = 'existing';
                $store = Mage::getModel('core/store')->load($itemId);
                if ($store->getId()) {
                    $storeCode = $params['item_id'] = $store->getCode();
                    $actionName = 'delete';
                }
            }

            foreach ($this->_blankableParams as $key) {
                if (isset($params[$key])) {
                    unset($params[$key]);
                }
            }

            $result[self::INDEX_EXECUTOR_CLASS] = get_class($this);
            $result[self::INDEX_CONTROLLER_MODULE] = $this->_request->getControllerModule();
            $result[self::INDEX_CONTROLLER_NAME] = $this->_request->getControllerName();
            $result[self::INDEX_ACTION_NAME] = $this->_request->getActionName();
            $result[self::INDEX_ACTION_PARAMS] = $this->_encodeParams($params);
            $result[self::INDEX_ACTION_DESCR] = sprintf("%s %s Store View '%s'", ucfirst($actionName), $new, $storeCode);
            $result[self::INDEX_VERSION] = $this->_getVersion();
        }


        return $result;
    }

    public function decode($encodedParameters, $version) {
        // The !empty() ensures that rows without a version number can be
        // executed (not without any risk).
        if (!empty($version) && $this->_getVersion() != $version) {
            throw new Exception(sprintf("Can't decode the Action encoded with %s Tracker v %s; current Store View Tracker is v %s ", $this->_code, $version, $this->_getVersion()));
        }

        $parameters = $this->_decodeParams($encodedParameters);

        // Convert Group UUID
        if (isset($parameters['store']) && ($groupUuid = $parameters['store']['group_id'])) {
            $group = Mage::getModel('core/store_group')->load($groupUuid, 'name');
            if ($group->getId()) {
                $parameters['store']['group_id'] = $group->getId();
            }
        }

        // Convert Store UUID
        if (isset($parameters['store']) && $storeUuid = $parameters['store']['store_id']) {
            $store = Mage::getModel('core/store')->load($storeUuid, 'code');
            $parameters['store']['store_id'] = $store->getId();
        }

        // Convert Item UUID (for deleteStore action)
        if ($itemUuid = $parameters['item_id']) {
            $store = Mage::getModel('core/store')->load($itemUuid, 'code');
            if ($store->getId()) {
                $parameters['item_id'] = $store->getId();
            }
        }

        $request = new Mage_Core_Controller_Request_Http();
        $request->setPost($parameters);
        $_SERVER['REQUEST_METHOD'] = 'POST'; // checked by StoreController
        $request->setQuery($parameters);
        return $request;
    }

}

3.4 Summary

Developing a Tracker is not a difficult task as Mageploy is developed "the Magento way": adding a Tracker is a matter of writing some XML (not so much) and a class implementing tracking logic.

The difficult part is studying the Controller action that we want to track and implement the encoding and decoding methods. In some circumstances, as the one seen in the above example, we will have to impose ourselves some constraints such the one of not having two Store Groups with the same name among the system.

We hope that this documentation may have helped you understanding Mageploy's internals and that you will be a happy Mageploy's user and, why not, contributor.

3.5 Greetings

Mageploy was originally conceived by the right side of the brain using FiftyThree's Paper and is maintained by the left side of the brain using a kindly offered open source development license of JetBrains' PhpStorm Logo.