Systems Performance: available now

My new book Systems Performance: Enterprise and the Cloud is shipping now from your favorite online book supplier (informIT, amazon). Thanks to those who have ordered a copy! I’ve already received feedback from people finding it useful, which is really gratifying. For an intro to the book, see my previous blog post about it. I’ve [...]

Read more...
Posted on October 28, 2013 at 3:57 pm by Brendan Gregg · Permalink · Comments Closed
In: book, performance

Open Source Systems Performance

There is an important and untold story about open source and systems performance, which I’m uniquely positioned to tell. I presented it at OSCON earlier this year, which was the perfect venue. My talk, Open Source Systems Performance, was a play in three acts: Act 1. Before Open Source Act 2. Open Source Act 3. [...]

Read more...
Posted on October 20, 2013 at 12:17 pm by Brendan Gregg · Permalink · Comments Closed
In: Open Source, performance, slides, Solaris, talk, video

Tracing Node.js add-on latency

Node.js has some built-in DTrace probes for looking at HTTP request start/done and GC start/done, and I wrote nhttpsnoop to report latency based on these probes. There’s also a future project that will make it easy to trace any asynchronous operations. But in the meantime, there’s one category of latency that’s missing, which is latency [...]

Read more...
Posted on October 18, 2013 at 8:49 am by dap · Permalink · Comments Closed
In: DTrace, joyent, Node.js

Stopping a broken program in its tracks

Last week I debugged a Node issue where under some conditions, fork() failure could result in some file descriptors being closed, leading shortly to bedlam. In the best cases, the program would immediately crash, but often it would just do the wrong thing. This happens a lot, of course, and in many ways it’s worse [...]

Read more...
Posted on October 14, 2013 at 7:58 am by dap · Permalink · Comments Closed
In: DTrace, smartos

The TSA Method

TSA Method in class (SmartOS) There are two basic performance analysis methodologies you can use for most performance issues. The first is the resource-oriented USE Method, which provides a checklist for identifying common bottlenecks and errors. The second is the thread-oriented TSA Method, for identifying issues causing poor thread performance. I summarized the TSA Method [...]

Read more...
Posted on October 11, 2013 at 12:20 pm by Brendan Gregg · Permalink · Comments Closed
In: methodology, performance, smartos, tsamethod

Control T for TENEX

I recently wrote about FreeBSD, and while on that OS I ran “sleep 5″ then hit Control-T: freebsd10$ sleep 5 load: 0.67 cmd: sleep 90628 [nanslp] 0.92r 0.00u 0.00s 0% 1464k sleep: about 4 second(s) left out of the original 5 Wow! This prints the “load:” and “sleep:” status lines, showing information about the process. [...]

Read more...
Posted on October 5, 2013 at 9:26 am by Brendan Gregg · Permalink · Comments Closed
In: FreeBSD, macosx, TENEX, TOPS-10, TOPS-20

Node.js in production: runtime log snooping

This post also appeared on the Joyeur blog. This post is the one of several about how we operate Node.js in production at Joyent. Most of my experience comes from working on our Manta Storage Service, which currently comprises a few thousand Node processes across three datacenters. Logging is one of the most primitive but [...]

Read more...
Posted on October 3, 2013 at 5:44 pm by dap · Permalink · Comments Closed
In: joyent, Node.js

The USE Method: Unix 7th Edition Performance Checklist

PDP 11/70 front panel (similar to the 11/45) Out of curiosity, I’ve developed a USE Method-based performance checklist for Unix 7th Edition on a PDP-11/45, which I’ve been running via a PDP simulator. 7th Edition is from 1979, and was the first Unix with iostat(1M) and pstat(1M), enabling more serious performance analysis from shipped tools. [...]

Read more...
Posted on September 29, 2013 at 2:37 am by Brendan Gregg · Permalink · Comments Closed
In: performance, Unix, usemethod

The USE Method: FreeBSD Performance Checklist

In this post, I’ll provide an example USE Method-based performance checklist for FreeBSD, for identifying common bottlenecks and errors. This is intended to be used early in a performance investigation, before moving onto more time consuming methodologies. This should be helpful for anyone using FreeBSD, especially system administrators. This was developed on FreeBSD 10.0 alpha, [...]

Read more...
Posted on September 25, 2013 at 11:42 am by Brendan Gregg · Permalink · Comments Closed
In: FreeBSD, performance, usemethod

The USE Method: Mac OS X Performance Checklist

This is my example USE Method-based performance checklist for the Apple Mac OS X operating system, for identifying common bottlenecks and errors. This draws upon both command line and graphical tools for coverage, focusing where possible on those that are provided with the OS by default, or by Apple (eg, Instruments). Further notes about tools [...]

Read more...
Posted on September 18, 2013 at 11:37 pm by Brendan Gregg · Permalink · Comments Closed
In: DTrace, Mac OS X, macosx, performance, usemethod