Massive Magento Attributes

Published: March 27, 2015

Magento is often criticized for being slow. I won't lie, the first time I used Magento that was my reaction as well. But after more than a year working at a Magento Gold Solution Partner I've learned that with the right hardware, and software, Magento can run rather smoothly...most of the time.

Background

The story starts with an issue reported by a client...

When I try access the manufacturer attribute I get an unresponsive script error

The screenshot looked something like this...

Screenshot of an unresponsive script error from Firefox

So I take a look at the manufacturer attribute. It's a dropdown attribute with around 3,500 options. While I'm not experiencing the unresponsive script error it's definitely sluggish. Some research is in order.

What They Say On The Interwebz

Some quick Googling brought me to this blog post. After reading through the post and quicky reviewing the proposed solution my initial reaction was go with the module created to handle this problem. However, after discussing with a colleague I decided to dig into the issue myself. Below are my findings.

What's The Problem?

Note: Below is based on investigation of the issue in 1.13.1.0. YMMV.

If you'd like to follow along, for starters, you'll need a massive attribute for testing. You can use the following MySQL script to create said attribute...

########################################
# Set Up A Masssive Dropdown Attribute
########################################
 
# User defined vars
SET @num_options = 3500;
SET @attribute_code = 'massive_attribute';
SET @attribute_label = 'Massive Attribute';
SET @store_id = 0;
 
# Create new attribute
INSERT INTO `eav_attribute` (`entity_type_id`, `attribute_code`, `backend_model`, `backend_type`, `frontend_input`, `frontend_label`, `frontend_class`, `source_model`, `is_required`, `is_user_defined`, `is_unique`) 
VALUES ('4', @attribute_code, NULL, 'int', 'select', @attribute_label, NULL, 'eav/entity_attribute_source_table', '0', '1', '0');
 
SELECT @attribute_id :=`attribute_id` FROM `eav_attribute` WHERE `attribute_code`=@attribute_code;
 
INSERT INTO `catalog_eav_attribute` (`attribute_id`, `is_global`, `is_searchable`, `is_filterable`, `is_comparable`, `is_visible_on_front`, `is_html_allowed_on_front`, `is_filterable_in_search`, `used_in_product_listing`, `used_for_sort_by`, `is_configurable`, `apply_to`, `is_visible_in_advanced_search`, `is_used_for_promo_rules`, `layered_navigation_canonical`) 
VALUES (@attribute_id, '1', '0', '0', '0', '0', '1', '0', '0', '0', '0', NULL, '0', '0', '0');
 
# Add options
DROP PROCEDURE IF EXISTS mpchadwick_add_options;
DELIMITER //
CREATE PROCEDURE mpchadwick_add_options(IN num_options INT, IN attribute_id INT, IN store_id INT)
BEGIN
    DECLARE i INT;
    SET i=0;
    WHILE i < num_options DO
      INSERT INTO `eav_attribute_option` (`attribute_id`, `sort_order`) VALUES (attribute_id, i);
      SELECT @option_id :=`option_id` FROM `eav_attribute_option` ORDER BY `option_id` DESC LIMIT 1;
      SET @value = CONCAT("value ", i);
      INSERT INTO `eav_attribute_option_value` (`option_id`, `store_id`, `value`) VALUES (@option_id, store_id, @value);
      SET i = i + 1;
    END WHILE;
END //
DELIMITER ;
 
CALL mpchadwick_add_options(@num_options, @attribute_id, @store_id);

Once you have your attribute, go there in Firefox. The issue was reported in Firefox and I actually found the Firefox developer tools (not Firebug) to be extremely useful for debugging this issue. Go head and pop open Firefox developer tools and open the "Performance" tab. Click record and then back in your browser click into "Massive Attribute" attribute. Stop the recording when the page loads. Here's what I got. Anything standing out to you here?

Dev Tools Result

One thing certainly stand out to me. 20 out of the 22 seconds required to load the page were spent on attributeOption.bindRemoveButtons. I think we're on to something!

The Culprit

To get started crack open app/design/adminhtml/default/default/template/catalog/product/attribute/options.phtml. That is the template responsible for this page:

Dev Tools Result

Spend a little to and try to grok this file.

OK now let's break it down...

Digging In

At the bottom of the file, options are added to the page through the following foreach loop

<?php foreach ($this->getOptionValues() as $_value): ?>
  attributeOption.add(<?php echo $_value->toJson(); ?>);
<?php endforeach; ?>

Here's the add method for the attributeOption object

add : function(data) {
  this.template = new Template(this.templateText, this.templateSyntax);
  var isNewOption = false;
  if(!data.id){
      data = {};
      data.id  = 'option_'+this.itemCount;
      isNewOption = true;
  }
  if (!data.intype)
      data.intype = optionDefaultInputType;
  Element.insert(this.table, {after: this.template.evaluate(data)});
  if (isNewOption && !this.isReadOnly) {
      this.enableNewOptionDeleteButton(data.id);
  }
  this.bindRemoveButtons();
  this.itemCount++;
  this.totalItems++;
  this.updateItemsCountField();
}

So bind bindRemoveButtons is getting called each time an option is added. Let's take a look a bindRemoveButtons...

bindRemoveButtons : function(){
  var buttons = $$('.delete-option');
  for(var i=0;i&lt;buttons.length;i++){
      if(!$(buttons[i]).binded){
          $(buttons[i]).binded = true;
          Event.observe(buttons[i], 'click', this.remove.bind(this));
      }
  }
}

And there's the glaring inefficiency.

bindRemoveButtons is checking every single remove button element on the page each time a new option is added rather than just checking the new option that was added. With 3,500 you could see how this could add some overhead head!

No offense Magento, but this reads like it was written by a backend developer who couldn't figure out JavaScript and called it a day once he/she got this working.

OK, now that we have a better idea what's going on let's blow this popsicle stand.

The Solution

Obviously first and foremost the JavaScript on this page needs to be refactored. From a basic standpoint, Tsvetan's solution makes sense. Let's rewrite the block and then tweak the template. However I think we can keep things a lot more simple than what Tsvetan has done in his refactored template file.

The easiest solution is to take bindRemoveButtons out of the add method and then call it once after all remove buttons have been added to the page. Let's try that and check the performance tab again in Firefox. Here's what I got...

Performance Tab After Refactoring Magento Core Code

Well would you look at that! bindRemoveButtons went from taking 20 seconds to taking less than 1 second on my machine. I'd say that's a pretty nice improvement.

Conclusion

As I said in the start, Magento can definitely whirr if tuned properly. Fortunately this is an admin only issue and won't have any customer facing impact. That being the issue impacts end users frequently enough that I'm not the only one who's written about it. Looking at the Magento 2 source code it looks like this is still an issue. Until this get's patched in the Magento core (In the process of writing this I've decided that I'd like to submit a PR) make sure to patch this template if you'd like to prevent some headaches that admins of large scale stores will run into.

Max Chadwick Hi, I'm Max!

I'm a software developer who mainly works in PHP, but loves dabbling in other languages like Go and Ruby. Technical topics that interest me are monitoring, security and performance. I'm also a stickler for good documentation and clear technical writing.

During the day I lead a team of developers and solve challenging technical problems at Rightpoint where I mainly work with the Magento platform. I've also spoken at a number of events.

In my spare time I blog about tech, work on open source and participate in bug bounty programs.

If you'd like to get in contact, you can find me on Twitter and LinkedIn.