Boost Magento perfomance with Queues

WebgriffeTailored Digital Works

| webgriffe.com @webgriffe5+ Years of Experience with Magento5 Certified Developers (Zend & Magento)350+ Customers20+ Magento Extensions450+ Extensions Sold

Simple Scenario

Complex Scenario

Can we go over this limit?

Yes!Queueing several syncrounous tasks allows us to go further.

Separation of components withQueue Manager & Workers

Queue Manager: collects & assigns tasks to workersWorkers: execute tasks (with repeat-until logic)

Response time improvmentsSave 30% of the time by queuing index update on product save

Magento CE v. 1.8.1.0 with sample data. Index update on save enabled. Data collected with Xhprof profiler.

Online Hackathon Worldwide Experiencewww.mage-hackathon.de/passed/online-hackathon-worldwide-31st-jan-1st-feb.html

4 continents10+ countries100+ people

80+ open source Magento projectsgithub.com/magento-hackathon

Online Hackathon Worldwide ExperienceLilmuckers_Queue( )Beanstalk( )Your custom logic

github.com/lilmuckers/magento-lilmuckers_queue

kr.github.io/beanstalkd

Which queue manager to use?The ones we can easly integrate with Magento are:

Beanstalk ( )Amazon SQS ( )Gearman ( )

kr.github.io/beanstalkdaws.amazon.com/sqs

gearman.org

Lilmuckers_Queue ExtensionDefining a Queue Backend (for example Beanstalk)

<?xml version="1.0"?><!-- app/etc/local.xml --><config> <global> ... <queue> <backend>beanstalkd</backend> <beanstalkd> <servers> <server> <host>127.0.0.1</host> </server> </servers> </beanstalkd> </queue> ... </global></config>

Lilmuckers_Queue ExtensionDefining a Queue

<?xml version="1.0"?><!-- app/code/local/MyVendor/MyModule/etc/config.xml --><config>... <queues> <queueName> <label>The Queue Name</label> <class>module/queueHandler</class> <workers> <taskName> <class>module/worker</class> <method>methodName</method> </taskName> </workers> </queueName> </queues>...</config>

Lilmuckers_Queue ExtensionAdding tasks to Queue

$_queue = Mage::helper('lilqueue')->getQueue('queueName');$_task = Mage::helper('lilqueue')->createTask( 'taskName', array('data'=>'to provide', 'to'=>'the worker'), $storeToRunAs);

// Optionally$_task->setPriority(100) ->setDelay(60) ->setTtr(60);

$_queue->addTask($_task);

Lilmuckers_Queue ExtensionWorker

class My_Module_Model_Worker extends Lilmuckers_Queue_Model_Worker_Abstract{ public function methodName(Lilmuckers_Queue_Model_Queue_Task $task) { //get the store assigned with the task $store = $task->getStore(); //get the queue handler for this queue $queue = $task->getQueue(); //get the data assigned with the task $data = $task->getData(); //This task ended properly $task->success(); //this task needs to be repeated $task->retry(); //this task errored and we should drop it from the queue $task->hold(); //this worker is taking a long time, we should extend it $task->touch(); }}

$ php /path/to/magento/shell/queue.php --watch <queues>

Queueing Magento Index Update

Open source code on GitHub

buildbuild passingpassing

github.com/webgriffe/index-queue-extension

Webgriffe_IndexQueueExtension

Architecture

The same can be done withother components or tasks

Queues are for…Entity index updateCache cleaningStock updateRendering email templates…Any task which the user doesn't need to wait for

Any Question?

WebgriffeTailored Digital Works

| webgriffe.com @webgriffe5+ Years of Experience with Magento5 Certified Developers (Zend & Magento)350+ Customers20+ Magento Extensions450+ Extensions Sold

Thank you!