Have you ever heard anyone say this?
We turn off logging in production for performance reasons.
Maybe, it’s even something you’ve said yourself?
Over the years, I’ve heard people say many things that effectively throw auditability out the window in the name of performance. Here are just a few…
innodb_print_all_deadlockssetting for performance reasons”
But here’s the thing, while I generally support decisions made in the name of performance, I also know from first hand experience that it’s imperative that your application leave a thorough audit trail in order to solve strange and unexpected production issues.
To demonstrate this, let’s examine the implications of the a couple of the statements listed above…
Web server access logs are one of the first places a competent engineer will look when a site goes down or becomes unresponsive. These logs are extremely informative and can answer all of the following questions (and more)…
Without access logs none of these questions can be answered.
Additionally, in the case of security breaches, forensic investigation is typically severely impaired if access logs are unavailable which could prevent an investigator from being able to identify an attacker’s entry point.
New Relic is a best in class application performance monitoring tool. In order to measure the performance of your application its agent injects itself into your code execution (known as “instrumentation”) capturing measurements to report back to the platform.
While it is true that this adds some overhead to your code execution, the cost is negligible compared to the benefits implementing such a tool provides.
Specifically, New Relic targets a maximum of 5% overhead for its agents. However it will save you much bigger than that if a developer accidentally pushes out some code that makes your site slower when it catches the slowness and alerts you about it.
The bottom line is that all too often decisions are made which have a minimal positive impact on performance, but severely negatively impact auditability. Please think twice when someone suggests disabling logs for performance reasons (and consider alternative solutions such as additional hardware).
Hi, I'm Max!
If you'd like to get in touch with me the best way is on Twitter.