Blog

Profiling jekyll build with ruby‑prof

Published: October 8, 2018

Tags:

As this site has grown, the time it takes for the jekyll build command to complete has continued to increase.

Currently the site has 143 posts and it takes 9.7 seconds for jekyll build to complete with a warm cache on a beefy 2014 MacBook Pro (2.2 GHz Intel Core i7, 16 GB RAM).

Recently I became interested in exploring options for speeding up the build.

Googling “jekyll build profile” lead me to jekyll build’s --profile flag.

The output looks like this:

$ bundle exec jekyll build --profile

Filename                                                                          | Count |    Bytes |  Time
----------------------------------------------------------------------------------+-------+----------+------
_layouts/main.html                                                                |   176 | 4043.23K | 1.282
_posts/2013-12-02-non-sucky-youtube-embed.html                                    |     1 |   14.00K | 0.285
_layouts/blog-single.html                                                         |   145 | 1633.02K | 0.269
sitemap.xml                                                                       |     1 |   20.50K | 0.115
tags/index.html                                                                   |     1 |   74.14K | 0.073
_includes/bio.html                                                                |   146 |  134.56K | 0.071
feed.xml                                                                          |     1 |  115.65K | 0.054
...

I didn’t find this to be particularly useful, as it gave me virtually no information on what jekyll build was actually doing that was taking so long.

Here I’ll show you how to use ruby-prof to get a more useful profile of your jekyll build.

grep color Adding ANSI Escape Sequence

Published: September 19, 2018

Tags:

Today I was using some command line kung-fu to transform a CSV with thousands of URL redirects into a text file containing the same redirects in Apache mod_rewrite format.

The command looked roughly like this 1

$ cat redirects-with-store-ids.csv | grep ',17' | awk -F"," '{ print "redirect 301 /" $1 " https://www.example.com/" $2 }' > example-com-redirects.conf

After getting the command just right I opened generated the file in Sublime Text and saw some strange looking characters at the end of each line…

redirect 301 /page/beige.html https://www.example.com/page.html?color=51<0x1b><0x1b>[K
redirect 301 /page/black.html https://www.example.com/page.html?color=52<0x1b>[01;31m<0x1b>
redirect 301 /page/blue.html https://www.example.com/page.html?color=53<0x1b>[01;31m<0x1b>[K
redirect 301 /page/brown.html https://www.example.com/page.html?color=55<0x1b><0x1b>
redirect 301 /page/green.html https://www.example.com/page.html?color=57<0x1b><0x1b>

For some reason “<0x1b>[01;31m<0x1b>[K” had been appended at the end of every line.

Customer Grid Index Doesn't Update on Schedule

Published: September 17, 2018

Tags:

As a performance and scalability optimization, it is recommended to set all of Magento’s indexers to ‘Update by Schedule’ mode. This can be done at the command line as follows…

$ php bin/magento indexer:set-mode schedule
Index mode for Indexer Design Config Grid was changed from 'Update on Save' to 'Update by Schedule'
Index mode for Indexer Customer Grid was changed from 'Update on Save' to 'Update by Schedule'
Index mode for Indexer Category Products was changed from 'Update on Save' to 'Update by Schedule'
Index mode for Indexer Product Categories was changed from 'Update on Save' to 'Update by Schedule'
Index mode for Indexer Product Price was changed from 'Update on Save' to 'Update by Schedule'
Index mode for Indexer Product EAV was changed from 'Update on Save' to 'Update by Schedule'
Index mode for Indexer Stock was changed from 'Update on Save' to 'Update by Schedule'
Index mode for Indexer Catalog Rule Product was changed from 'Update on Save' to 'Update by Schedule'
Index mode for Indexer Catalog Product Rule was changed from 'Update on Save' to 'Update by Schedule'
Index mode for Indexer Catalog Search was changed from 'Update on Save' to 'Update by Schedule'
Index mode for Indexer Product/Target Rule was changed from 'Update on Save' to 'Update by Schedule'
Index mode for Indexer Target Rule/Product was changed from 'Update on Save' to 'Update by Schedule'
Index mode for Indexer Sales Rule was changed from 'Update on Save' to 'Update by Schedule'

However, the customer grid indexer, it turns out, doesn’t really support update on schedule.

Visualizing sar data with kSar

Published: September 13, 2018

Tags:

sar is a useful tool that is included in the sysstat package, a set of performance monitoring utilities that is pre-installed on many Linux distros.

Running the sar command prints the gathered data to the terminal in columns.

$ sar 2 10
Linux 4.14.62-v7+ (pluto) 	09/13/2018 	_armv7l_	(4 CPU)

08:48:05 PM     CPU     %user     %nice   %system   %iowait    %steal     %idle
08:48:07 PM     all      0.13      0.00      0.00      0.00      0.00     99.87
08:48:09 PM     all      0.00      0.00      0.13      0.13      0.00     99.75
08:48:11 PM     all      0.38      0.00      0.00      0.00      0.00     99.62
08:48:13 PM     all      0.13      0.00      0.13      0.00      0.00     99.75
08:48:15 PM     all      0.13      0.00      0.13      0.00      0.00     99.75
08:48:17 PM     all      0.00      0.00      0.25      0.00      0.00     99.75
08:48:19 PM     all      0.00      0.00      0.38      0.00      0.00     99.62
08:48:21 PM     all      0.00      0.00      0.13      0.00      0.00     99.87
08:48:23 PM     all      0.00      0.00      0.38      0.00      0.00     99.62
08:48:25 PM     all      0.00      0.00      0.00      0.00      0.00    100.00
Average:        all      0.08      0.00      0.15      0.01      0.00     99.76

This is useful in some scenarios such as checking CPU usage at a specific period of time. However, in other use cases such as reviewing trends over a longer period of time, this format is not particularly user friendly.

I spent some time recently reviewing the available options for visualizing sar data. While kSar doesn’t offer the most beautiful interface1, it’s easy to install and seems to be the best option available

Here I’ll cover how kSar can be used to visualize sar data.

Reindexing Customer Grid Runs out of Memory in Magento 2

Published: September 10, 2018

Tags:

The 2.1.X release line of Magento current contains an issue where, with enough customers, reindexing the customer grid will fail with an out of memory fatal error

$ bin/magento indexer:reindex customer_grid
PHP Fatal error:  Allowed memory size of 1073741824 bytes exhausted (tried to allocate 280224918230723 bytes) in /var/www/html/vendor/magento/zendframework1/library/Zend/Db/Statement/Pdo.php on line 228

The issue is fixed in the in the 2.2.X release line and can be fixed in 2.1.X by applying PATCH_MDVA-4538.

Replication Lag Race Conditions in Magento

Published: September 7, 2018

One of my favorite class of bugs are race conditions caused by replication lag (yes, I’m a masochist). These typically play out something like this…

I’ve run into a number of these both in core Magento as well as in 3rd party extensions. In this post I’ll cover a couple of my favorites…