Massive Magento Attributes
Published: March 27, 2015
Magento is often criticized for being slow. I won't lie, the first time I used Magento that was my reaction as well. But after more than a year working at a Magento Gold Solution Partner I've learned that with the right hardware, and software, Magento can run rather smoothly...most of the time.
Background
The story starts with an issue reported by a client...
When I try access the manufacturer attribute I get an unresponsive script error
The screenshot looked something like this...
So I take a look at the manufacturer attribute. It's a dropdown attribute with around 3,500 options. While I'm not experiencing the unresponsive script error it's definitely sluggish. Some research is in order.
What They Say On The Interwebz
Some quick Googling brought me to this blog post. After reading through the post and quicky reviewing the proposed solution my initial reaction was go with the module created to handle this problem. However, after discussing with a colleague I decided to dig into the issue myself. Below are my findings.
What's The Problem?
Note: Below is based on investigation of the issue in 1.13.1.0. YMMV.
If you'd like to follow along, for starters, you'll need a massive attribute for testing. You can use the following MySQL script to create said attribute...
########################################
# Set Up A Masssive Dropdown Attribute
########################################
# User defined vars
SET @num_options = 3500;
SET @attribute_code = 'massive_attribute';
SET @attribute_label = 'Massive Attribute';
SET @store_id = 0;
# Create new attribute
INSERT INTO `eav_attribute` (`entity_type_id`, `attribute_code`, `backend_model`, `backend_type`, `frontend_input`, `frontend_label`, `frontend_class`, `source_model`, `is_required`, `is_user_defined`, `is_unique`)
VALUES ('4', @attribute_code, NULL, 'int', 'select', @attribute_label, NULL, 'eav/entity_attribute_source_table', '0', '1', '0');
SELECT @attribute_id :=`attribute_id` FROM `eav_attribute` WHERE `attribute_code`=@attribute_code;
INSERT INTO `catalog_eav_attribute` (`attribute_id`, `is_global`, `is_searchable`, `is_filterable`, `is_comparable`, `is_visible_on_front`, `is_html_allowed_on_front`, `is_filterable_in_search`, `used_in_product_listing`, `used_for_sort_by`, `is_configurable`, `apply_to`, `is_visible_in_advanced_search`, `is_used_for_promo_rules`, `layered_navigation_canonical`)
VALUES (@attribute_id, '1', '0', '0', '0', '0', '1', '0', '0', '0', '0', NULL, '0', '0', '0');
# Add options
DROP PROCEDURE IF EXISTS mpchadwick_add_options;
DELIMITER //
CREATE PROCEDURE mpchadwick_add_options(IN num_options INT, IN attribute_id INT, IN store_id INT)
BEGIN
DECLARE i INT;
SET i=0;
WHILE i < num_options DO
INSERT INTO `eav_attribute_option` (`attribute_id`, `sort_order`) VALUES (attribute_id, i);
SELECT @option_id :=`option_id` FROM `eav_attribute_option` ORDER BY `option_id` DESC LIMIT 1;
SET @value = CONCAT("value ", i);
INSERT INTO `eav_attribute_option_value` (`option_id`, `store_id`, `value`) VALUES (@option_id, store_id, @value);
SET i = i + 1;
END WHILE;
END //
DELIMITER ;
CALL mpchadwick_add_options(@num_options, @attribute_id, @store_id);
Once you have your attribute, go there in Firefox. The issue was reported in Firefox and I actually found the Firefox developer tools (not Firebug) to be extremely useful for debugging this issue. Go head and pop open Firefox developer tools and open the "Performance" tab. Click record and then back in your browser click into "Massive Attribute" attribute. Stop the recording when the page loads. Here's what I got. Anything standing out to you here?
One thing certainly stand out to me. 20 out of the 22 seconds required to load the page were spent on attributeOption.bindRemoveButtons. I think we're on to something!
The Culprit
To get started crack open app/design/adminhtml/default/default/template/catalog/product/attribute/options.phtml. That is the template responsible for this page:
Spend a little to and try to grok this file.
OK now let's break it down...
Digging In
At the bottom of the file, options are added to the page through the following foreach loop
<?php foreach ($this->getOptionValues() as $_value): ?>
attributeOption.add(<?php echo $_value->toJson(); ?>);
<?php endforeach; ?>Here's the add method for the attributeOption object
add : function(data) {
this.template = new Template(this.templateText, this.templateSyntax);
var isNewOption = false;
if(!data.id){
data = {};
data.id = 'option_'+this.itemCount;
isNewOption = true;
}
if (!data.intype)
data.intype = optionDefaultInputType;
Element.insert(this.table, {after: this.template.evaluate(data)});
if (isNewOption && !this.isReadOnly) {
this.enableNewOptionDeleteButton(data.id);
}
this.bindRemoveButtons();
this.itemCount++;
this.totalItems++;
this.updateItemsCountField();
}So bind bindRemoveButtons is getting called each time an option is added. Let's take a look a bindRemoveButtons...
bindRemoveButtons : function(){
var buttons = $$('.delete-option');
for(var i=0;i<buttons.length;i++){
if(!$(buttons[i]).binded){
$(buttons[i]).binded = true;
Event.observe(buttons[i], 'click', this.remove.bind(this));
}
}
}And there's the glaring inefficiency.
bindRemoveButtons is checking every single remove button element on the page each time a new option is added rather than just checking the new option that was added. With 3,500 you could see how this could add some overhead head!
No offense Magento, but this reads like it was written by a backend developer who couldn't figure out JavaScript and called it a day once he/she got this working.
OK, now that we have a better idea what's going on let's blow this popsicle stand.
The Solution
Obviously first and foremost the JavaScript on this page needs to be refactored. From a basic standpoint, Tsvetan's solution makes sense. Let's rewrite the block and then tweak the template. However I think we can keep things a lot more simple than what Tsvetan has done in his refactored template file.
The easiest solution is to take bindRemoveButtons out of the add method and then call it once after all remove buttons have been added to the page. Let's try that and check the performance tab again in Firefox. Here's what I got...
Well would you look at that! bindRemoveButtons went from taking 20 seconds to taking less than 1 second on my machine. I'd say that's a pretty nice improvement.
Conclusion
As I said in the start, Magento can definitely whirr if tuned properly. Fortunately this is an admin only issue and won't have any customer facing impact. That being the issue impacts end users frequently enough that I'm not the only one who's written about it. Looking at the Magento 2 source code it looks like this is still an issue. Until this get's patched in the Magento core (In the process of writing this I've decided that I'd like to submit a PR) make sure to patch this template if you'd like to prevent some headaches that admins of large scale stores will run into.
Hi, I'm Max!