Archive for the ‘joyent’ Category

Understanding DTrace ustack helpers

or, everything you ever wanted to know about stack traces I promised this post over a year ago, and now that someone’s actually working on a new ustack helper, I thought it was finally time to write about what ustack helpers are, how they work, and how I went about building one for Node.js. Only [...]

Posted on November 20, 2013 at 4:50 pm by dap · Permalink · Comments Closed
In: DTrace, joyent, Node.js, smartos

Tracing Node.js add-on latency

Node.js has some built-in DTrace probes for looking at HTTP request start/done and GC start/done, and I wrote nhttpsnoop to report latency based on these probes. There’s also a future project that will make it easy to trace any asynchronous operations. But in the meantime, there’s one category of latency that’s missing, which is latency [...]

Posted on October 18, 2013 at 8:49 am by dap · Permalink · Comments Closed
In: DTrace, joyent, Node.js

Node.js in production: runtime log snooping

This post also appeared on the Joyeur blog. This post is the one of several about how we operate Node.js in production at Joyent. Most of my experience comes from working on our Manta Storage Service, which currently comprises a few thousand Node processes across three datacenters. Logging is one of the most primitive but [...]

Posted on October 3, 2013 at 5:44 pm by dap · Permalink · Comments Closed
In: joyent, Node.js

Kartlytics: Applying Big Data Analytics to Mario Kart 64

This post also appears on the Joyeur blog. If you missed it, Joyent recently launched Manta, a web-facing object store with compute as a first-class operation. Manta makes it easy to crunch on Big Data in the cloud, and we’ve seen it used by both ourselves and others and others to solve real business problems [...]

Posted on August 8, 2013 at 7:28 am by dap · Permalink · Comments Closed
In: joyent, manta

Fault tolerance in Manta

Since launching Manta last week, we’ve seen a lot of excitement. Thoughtful readers quickly got to burning questions about the system’s fault tolerance: what happens when backend Manta components go down? In this post, I’ll discuss fault tolerance in Manta in more detail. If anything seems left out, please ask: it’s either an oversight or [...]

Posted on July 3, 2013 at 8:04 am by dap · Permalink · Comments Closed
In: joyent, manta

Manta: Unix Meets Map Reduce

Today Joyent launched Manta: an object store built upon ZFS and Zones, the SmartOS platform, and with a familiar Unix interface as the API. This supports compute jobs such as map reduce, and provides high performance by co-locating zones with the object storage. We also have extensive DTrace instrumentation throughout the product, which we’ve been [...]

Posted on June 25, 2013 at 6:32 am by Brendan Gregg · Permalink · Comments Closed
In: filesystem, joyent, manta

Inside Manta: Distributing the Unix shell

Photo by Jackie Reid, NOAA Today, Joyent has launched Manta: our internet-facing object store with compute as a first class operation. This is the culmination of over a year’s effort on the part of the whole engineering team, and I’m personally really excited to be able to share this with the world. There’s plenty of [...]

Posted on June 25, 2013 at 6:10 am by dap · Permalink · Comments Closed
In: joyent, Node.js, smartos

OSCON Slides

Thanks to all who attended my talk at OSCON on Node.js in production: postmortem debugging and performance analysis. Just a reminder: all of the technology I described is open source, most of it part of illumos. For more info, check out the links in slide 22. For the curious, there are also some slides on [...]

Posted on July 19, 2012 at 12:32 pm by dap · Permalink · Comments Closed
In: joyent, Node.js, smartos

Debugging RangeError from a core dump

Last week, I tweeted: I had just run into this nasty Node.js error: $ node foo.js timers.js:96 if (!process.listeners(‘uncaughtException’).length) throw e; ^ RangeError: Maximum call stack size exceeded What went wrong? It was reasonably obvious from the error message that the program blew its stack, which I assumed was likely the result of some errant [...]

Posted on May 14, 2012 at 2:43 pm by dap · Permalink · Comments Closed
In: joyent, Node.js

Profiling Node.js

(For returning readers, this is basically a “tl;dr” version of my previous post on Node.js performance. The post below also appears on the Node.js blog.) It’s incredibly easy to visualize where your Node program spends its time using DTrace and node-stackvis (a Node port of Brendan Gregg’s FlameGraph tool): Run your Node.js program as usual. [...]

Posted on April 25, 2012 at 10:21 am by dap · Permalink · Comments Closed
In: DTrace, joyent, Node.js, Production, smartos