Archive for the ‘DTrace’ Category

Tuning the OpenZFS write throttle

In previous posts I discussed the problems with the legacy ZFS write throttle that cause degraded performance and wildly variable latencies. I then presented the new OpenZFS write throttle and I/O scheduler that Matt Ahrens and I designed. In addition to solving several problems in ZFS, the new approach was designed to be easy to [...]

Posted on August 31, 2014 at 9:16 am by ahl · Permalink · Comments Closed
In: DTrace, MattAhrens, OpenZFS, performance, tuning, ZFS

Understanding DTrace ustack helpers

or, everything you ever wanted to know about stack traces I promised this post over a year ago, and now that someone’s actually working on a new ustack helper, I thought it was finally time to write about what ustack helpers are, how they work, and how I went about building one for Node.js. Only [...]

Posted on November 20, 2013 at 4:50 pm by dap · Permalink · Comments Closed
In: DTrace, joyent, Node.js, smartos

Tracing Node.js add-on latency

Node.js has some built-in DTrace probes for looking at HTTP request start/done and GC start/done, and I wrote nhttpsnoop to report latency based on these probes. There’s also a future project that will make it easy to trace any asynchronous operations. But in the meantime, there’s one category of latency that’s missing, which is latency [...]

Posted on October 18, 2013 at 8:49 am by dap · Permalink · Comments Closed
In: DTrace, joyent, Node.js

Stopping a broken program in its tracks

Last week I debugged a Node issue where under some conditions, fork() failure could result in some file descriptors being closed, leading shortly to bedlam. In the best cases, the program would immediately crash, but often it would just do the wrong thing. This happens a lot, of course, and in many ways it’s worse [...]

Posted on October 14, 2013 at 7:58 am by dap · Permalink · Comments Closed
In: DTrace, smartos

The USE Method: Mac OS X Performance Checklist

This is my example USE Method-based performance checklist for the Apple Mac OS X operating system, for identifying common bottlenecks and errors. This draws upon both command line and graphical tools for coverage, focusing where possible on those that are provided with the OS by default, or by Apple (eg, Instruments). Further notes about tools [...]

Posted on September 18, 2013 at 11:37 pm by Brendan Gregg · Permalink · Comments Closed
In: DTrace, Mac OS X, macosx, performance, usemethod

Virtualization Performance: Zones, KVM, Xen

At Joyent we run a high-performance public cloud based on two different virtualization technologies: Zones and KVM. We have historically run Xen as well, but have phased it out for KVM on SmartOS. My job is to make things go fast, which often means using DTrace to analyze the kernel, applications, and those virtualization technologies. [...]

Posted on January 11, 2013 at 4:58 pm by Brendan Gregg · Permalink · Comments Closed
In: Cloud, DTrace, KVM, performance, xen, zones

Surge 2012: Real-time in the real world

In September, I attended and spoke at the Surge’12 conference in Baltimore. I highly recommend it for anyone interested in performance. The theme with most talks was problems encountered at scale – and as is often the case – you can learn more from failure than success. Bryan and I gave a talk titled “Real-time [...]

Posted on December 6, 2012 at 6:47 pm by Brendan Gregg · Permalink · Comments Closed
In: DTrace, performance, slides, video

DTracing in Anger

My Macbook has becomeso sluggish that it feels like I’m typing ove a 9600 baud modem aagn. Or 2400. It’s alo droping keystokes – which is irritatng as hll – so please forgive theapparent tyos and mistakes. It comes and goes each minute, so thiswhole post isn’t too bad. Usually I cn see what’s wrng [...]

Posted on November 14, 2012 at 11:04 pm by Brendan Gregg · Permalink · Comments Closed
In: DTrace, macosx, performance

illumos hackathon 2012: user-land types for DTrace

At the illumos hackathon last week, Robert Mustacchi and I prototyped better support for manipulating user-land structures. As anyone who’s used it knows, DTrace is currently very kernel-centric — this both reflects the reality of how operating systems and DTrace are constructed, and the origins of DTrace itself in the Solaris Kernel Group. Discussions at [...]

Posted on October 11, 2012 at 9:28 am by ahl · Permalink · Comments Closed
In: DTrace, hackathon, illumos, pid, user-land

DTrace Training

I’ve been developing a 2 day DTrace training course, which I’ll deliver in San Francisco on September 6th this year. The course materials include a copy of the DTrace book, written by myself and Jim Mauro. This course focuses on how to use DTrace: In particular, how to analyze performance issues and troubleshoot mysterious errors. [...]

Posted on August 12, 2012 at 1:34 pm by Brendan Gregg · Permalink · Comments Closed
In: DTrace, Training