Archive for the ‘joyent’ Category

Project Bardiche: vnd Architecture

My previous entry introduced Project Bardiche, a project which revamps how we do networking for KVM guests. This entry focuses on the design and architecture of the vnd driver and how it fits into the broader networking stack. The illumos networking stack The illumos networking stack is broken into several discrete pieces which is summarized [...]

Posted on March 24, 2014 at 8:31 am by rm · Permalink · Comments Closed
In: Bardiche, illumos, joyent, smartos

Project Bardiche: Introduction

I just recently landed Project Bardiche into SmartOS. The goal of Bardiche has been to create a more streamlined data path for layer two networking in illumos. While the primary motivator for this was for KVM guests, it’s opened up a lot of room for more than just virtual machines. This bulk of this project [...]

Posted on March 20, 2014 at 4:01 pm by rm · Permalink · Comments Closed
In: Bardiche, illumos, joyent, KVM, smartos

Understanding DTrace ustack helpers

or, everything you ever wanted to know about stack traces I promised this post over a year ago, and now that someone’s actually working on a new ustack helper, I thought it was finally time to write about what ustack helpers are, how they work, and how I went about building one for Node.js. Only [...]

Posted on November 20, 2013 at 4:50 pm by dap · Permalink · Comments Closed
In: DTrace, joyent, Node.js, smartos

Tracing Node.js add-on latency

Node.js has some built-in DTrace probes for looking at HTTP request start/done and GC start/done, and I wrote nhttpsnoop to report latency based on these probes. There’s also a future project that will make it easy to trace any asynchronous operations. But in the meantime, there’s one category of latency that’s missing, which is latency [...]

Posted on October 18, 2013 at 8:49 am by dap · Permalink · Comments Closed
In: DTrace, joyent, Node.js

Node.js in production: runtime log snooping

This post also appeared on the Joyeur blog. This post is the one of several about how we operate Node.js in production at Joyent. Most of my experience comes from working on our Manta Storage Service, which currently comprises a few thousand Node processes across three datacenters. Logging is one of the most primitive but [...]

Posted on October 3, 2013 at 5:44 pm by dap · Permalink · Comments Closed
In: joyent, Node.js

Kartlytics: Applying Big Data Analytics to Mario Kart 64

This post also appears on the Joyeur blog. If you missed it, Joyent recently launched Manta, a web-facing object store with compute as a first-class operation. Manta makes it easy to crunch on Big Data in the cloud, and we’ve seen it used by both ourselves and others and others to solve real business problems [...]

Posted on August 8, 2013 at 7:28 am by dap · Permalink · Comments Closed
In: joyent, manta

Fault tolerance in Manta

Since launching Manta last week, we’ve seen a lot of excitement. Thoughtful readers quickly got to burning questions about the system’s fault tolerance: what happens when backend Manta components go down? In this post, I’ll discuss fault tolerance in Manta in more detail. If anything seems left out, please ask: it’s either an oversight or [...]

Posted on July 3, 2013 at 8:04 am by dap · Permalink · Comments Closed
In: joyent, manta

Manta: Unix Meets Map Reduce

Today Joyent launched Manta: an object store built upon ZFS and Zones, the SmartOS platform, and with a familiar Unix interface as the API. This supports compute jobs such as map reduce, and provides high performance by co-locating zones with the object storage. We also have extensive DTrace instrumentation throughout the product, which we’ve been [...]

Posted on June 25, 2013 at 6:32 am by Brendan Gregg · Permalink · Comments Closed
In: filesystem, joyent, manta

Inside Manta: Distributing the Unix shell

Photo by Jackie Reid, NOAA Today, Joyent has launched Manta: our internet-facing object store with compute as a first class operation. This is the culmination of over a year’s effort on the part of the whole engineering team, and I’m personally really excited to be able to share this with the world. There’s plenty of [...]

Posted on June 25, 2013 at 6:10 am by dap · Permalink · Comments Closed
In: joyent, Node.js, smartos

OSCON Slides

Thanks to all who attended my talk at OSCON on Node.js in production: postmortem debugging and performance analysis. Just a reminder: all of the technology I described is open source, most of it part of illumos. For more info, check out the links in slide 22. For the curious, there are also some slides on [...]

Posted on July 19, 2012 at 12:32 pm by dap · Permalink · Comments Closed
In: joyent, Node.js, smartos