Project Bardiche: Framed I/O
The series so far If you’re getting started you’ll want to see the previous entries on Project Bardiche: Project Bardiche: Introduction Project Bardiche: vnd Architecture Background Framed I/O is a new abstraction that we’re currently experimenting with through Project Bardiche. We call this framed I/O, because the core concept is what we call a frame: [...]
Read more...Project Bardiche: vnd Architecture
My previous entry introduced Project Bardiche, a project which revamps how we do networking for KVM guests. This entry focuses on the design and architecture of the vnd driver and how it fits into the broader networking stack. The illumos networking stack The illumos networking stack is broken into several discrete pieces which is summarized [...]
Read more...In: Bardiche, illumos, joyent, smartos
Project Bardiche: Introduction
I just recently landed Project Bardiche into SmartOS. The goal of Bardiche has been to create a more streamlined data path for layer two networking in illumos. While the primary motivator for this was for KVM guests, it’s opened up a lot of room for more than just virtual machines. This bulk of this project [...]
Read more...In: Bardiche, illumos, joyent, KVM, smartos
A New Challenge
About two years ago Joyent began offering Linux instances, running under KVM, stored on ZFS, and secured by Zones (“double hull virtualization”). Since then, I’ve been doing more and more work on Linux performance as customers deploy on these instances. It’s been fascinating to work on both the illumos and Linux kernels at the same [...]
Read more...The Total Cost of Unmasked Data
Data breaches make headlines at a regular cadence. Each is a surprise, but they are not, as a whole, surprising. While the extensive and sophisticated Target breach stuck in the headlines, a significant breach at three South Korean credit card companies happened around the same time. The theft of personal information for 20m subscribers didn’t [...]
Read more...Another 10 Performance Wins
Following on from my earlier 10 performance wins post, here is another group of 10 I have worked on. # Target Analyzed Key Tool Fixed Specific Improvement 11 redis System DTrace System scheduler 41% 12 rsync System DTrace Application app config 5x 13 mongoperf System DTrace System ZFS tuning up to 8x 14 backups System [...]
Read more...In: performance, wins
The OpenZFS write throttle
In my last blog post, I wrote about the ZFS write throttle, and how we saw it lead to pathological latency variability on customer systems. Matt Ahrens, the co-founder of ZFS, and I set about to fix it in OpenZFS. While the solution we came to may seem obvious, we arrived at it only through [...]
Read more...In: MattAhrens, OpenZFS, performance, ZFS
Benchmarking the Cloud
Benchmarking, and benchmarking the cloud, is incredibly error prone. I provided guidance though this minefield in the benchmarking chapter of my book (Systems Performance: Enterprise and the Cloud); that chapter can be read online on the InformIT site. I also gave a lightning talk about benchmarking gone wrong at Surge last year. In this post, [...]
Read more...In: benchmarking, Cloud, performance
ZFS fundamentals: the write throttle
It’s no small feat to build a stable, modern filesystem. The more I work with ZFS, the more impressed I am with how much it got right, and how malleable it’s proved. It has evolved to fix shortcomings and accommodate underlying technological shifts. It’s not surprising though that even while its underpinnings have withstood the [...]
Read more...In: latency, MattAhrens, OpenZFS, performance, Throughput, ZFS
Understanding DTrace ustack helpers
or, everything you ever wanted to know about stack traces I promised this post over a year ago, and now that someone’s actually working on a new ustack helper, I thought it was finally time to write about what ustack helpers are, how they work, and how I went about building one for Node.js. Only [...]
Read more...