I’m a software engineer at Joyent in San Francisco. This blog is where I write about my work, side projects outside of work, and the business and process of making software. While most of the content describes my work (first at Sun and now Joyent) and why I believe in it, my goal is to share technical ideas, not advertise for my company.
I aspire to writing solid code, especially at a system’s bedrock. So I love C, Node, and Unix and try to use the right tools for the job.
- “Industrial-Grade Node.js”. NodeSummit 2015. (slides)
- “ZFS in the Manta Storage System”. ZFS Developer Summit 2014. (slides, video)
- “Scaling the Unix Philosophy to Big Data”. Surge 2013. (slides, video)
- “Kartlytics: Applying Big Data Analytics to Mario Kart 64″. (video)
- “How Joyent Operates Node.js in Production”. 140proof Hack, Deploy, Scale. (slides, video).
- “Building a Real-Time Cloud Analytics Service with Node.js”. Surge 2011. (slides, video)
I’ve given other similar talks at NodeConfs, OSCONs, GoTo 2012, Fluent 2012, and Surge 2012.
- Postmortem debugging in dynamic environments. Communications of the ACM. December, 2011. Vol. 54, No. 12.
Work at Joyent
My biggest project at Joyent has been Marlin, the system within Manta that’s responsible for executing users’ compute jobs. This was a big learning experience about distributed systems, databases, and even principles of operating systems applied higher in the stack.
I joined Joyent in November, 2010 to work on tools for observing systems in the cloud. Our Cloud Analytics tool seeks to collect and present real-time and historical performance data for distributed systems and applications for analysis by application developers and cloud operators. As part of this project, I developed the overall distributed system architecture for CA and many of the core facilities including inter-service communication using AMQP, logging, post-mortem debugging, overall system control, data aggregation, and data collection (instrumentation). Of course, we’re standing on the giants of Solaris, DTrace, and Node.js.
Work at Fishworks (Sun/Oracle)
Before joining Joyent, I worked at Sun (later Oracle) on the Fishworks engineering team. I joined fresh out of college to work on what became the Sun Storage 7000 Unified Storage appliance, now called the Sun ZFS Storage Appliance. The project sought to deliver a low-cost but highly reliable, high-performance enterprise network storage (NAS) appliance based on Solaris, as well as an appliance kit for quickly developing other types of appliances that need common features like a web interface, streamlined configuration, fault management, alerts, remote support, and advanced monitoring tools. The product has since incorporated many additional features like direct attached storage and certifications for important enterprise applications like VMware. My work on the team spanned a variety of areas including:
- remote replication: zfs-based replication of user data and configuration to other systems for disaster recovery and data distribution
- design of Flash devices (and their firmware), specifically how they interact with the rest of the system, with particular focus on performance and fault management
- phone home features
- RAS (reliability, availability, serviceability) features, including support bundles (but not upgrade — I just blogged about that)
- alerting and general fault management
- HTTP/WebDAV protocol integration
- integration with the Solaris NDMP server for participation in remotely-managed backups
- automated testing infrastructure
I can hardly even begin to describe how much I learned from my experience at Fishworks, both technically and personally.
In 2005 I interned at Google with the AdWords Frontend Team, working on the email system and automated testing. In 2006 I interned at Sun with the Solaris kernel team, where I worked on event ports and signals, contributing the original event port implementation for libevent.
- Mocha: an “unofficial” search engine and browser for Brown’s online course catalog, still used by over 5000 students each semester. (We still maintain Mocha today.)
- Hogs: a networked multi-player hovercraft shoot-’em-up game. A CS32 project with Adam Cath, Dan Leventhal, and Bill Pijewski. Hogs was later used as the basis for a project in CS166, a class on security.
- CS17/CS18: I was first an undergraduate TA and then head TA for this two-semester intro sequence that introduces first-year CS students to programming, software engineering, algorithms, and analysis using a mix of Scheme, Ocaml, and Java.
- CS167/9: I was an undergraduate TA for the operating systems class and additional lab component, in which students write their own complete operating system called Weenix.
- Lab consultant
- Meta-TA: helped run the department’s TA program
- Weenix-on-Xen: independent study porting the CS169 Weenix operating system from running on a simulator to running as a Xen guest OS
Last updated August, 2014.