Detecting Outliers

In computer performance, we’re especially concerned about latency outliers: very slow database queries, application requests, disk I/O, etc. The term “outlier” is subjective: there is no rigid mathematical definition. From [Grubbs 69]: An outlying observation, or “outlier,” is one that appears to deviate markedly from other members of the sample in which it occurs. Outliers [...]

Posted on July 1, 2013 at 10:43 am by Brendan Gregg · Permalink · 7 Comments
In: Performance · Tagged with: , , , , ,

Revealing Hidden Latency Patterns

Latency Heat Map Response time – or latency – is crucial to understand in detail, but many of the common presentations of this data hide important details and patterns. Latency heat maps are an effective way to reveal these. I often use tools that provide heat maps directly, but sometimes I have separate trace output [...]

Posted on May 19, 2013 at 2:56 pm by Brendan Gregg · Permalink · One Comment
In: Performance · Tagged with: , , ,

Off-CPU Performance Analysis

I’ve been exercising a new approach for examining application performance, which involves measuring high resolution off-CPU time that is synchronous to the workload. This reveals which code-paths are blocked and waiting while off-CPU, and for how long exactly. This differs from traditional profiling which often samples the activity of threads at a given interval, and [...]

Posted on July 8, 2011 at 9:36 am by Brendan Gregg · Permalink · 2 Comments
In: Performance · Tagged with: , , , ,