dtrace.conf 2012!
April 3rd, San Francisco: information and registration
Data Replication: Approaching the Problem
With our next Delphix release just around the corner, I wanted to spend some time discussion the engineering process behind one of the major new features: data replication between servers. The current Delphix version already has a replication solution, so how does this constitute a “new feature”? The reason is that it’s an entirely new [...]
Read more...Delphix git integration
Tonight, my Delphix colleague Zubair Khan and I presented the integration we’ve done with git at the SF Bay Area Large-Scale Production Engineering meetup. When I started at Delphix, we were using Subversion — my ire for which the margins of this blog are too narrow to contain. We switched to git, and in the [...]
Read more...In: Delphix, ZubairKhan, git
Visualizing Process Execution
In Visualizing Process Snapshots I showed processes and their parent-child hierarchy over time, using snapshots of process information. This approach misses short-lived processes that occur between the snapshots. Here I’ll fill in the gaps using system tracing, visualizing all processes that occurred. Snapshots vs Tracing 1 second snapshots (eg, top) [...]
Read more...In: Cloud, DTrace, experimental, performance, visualizations
Hardware Engineer
Back at Fishworks, my colleagues had a nickname for me: Adam Leventhal, Hardware Engineer. I wasn’t designing hardware; I wasn’t even particularly more involved with hardware specs. The name referred to my preternatural ability to fit round pegs into square holes, to know when parts would bend but not break (or if they broke how [...]
Read more...In: AdamLeventhalHardwareEngineer, Fishworks, Hardware, SSD, iwashi, riverwalk, thumper
Visualizing Process Snapshots
In Visualizing the Cloud I showed processes and their parent-child hierarchy, across a cloud environment, exploring patterns at different scales. Here I’ll take this a little further and look at processes over time. These are experimental visualizations shared to explore ideas. It’s Alive! Many who have seen this have described it as “creepy” — looking [...]
Read more...In: Cloud, experimental, performance, visualizations
Performance Analysis talk at SCALE10x
Last week I gave a talk at the Southern California Linux Expo (SCALE) titled “Performance Analysis: New Tools and Concepts from The Cloud”. There was a great turnout for my talk, which was videoed by Deirdré (video below). The conference was a lot of fun too. In my talk I presented six problems of performance [...]
Read more...In: performance, slides, video
ZFS+10: illumos meetup
ZFS recently celebrated its informal 10th anniversary; to mark the occasion, Delphix hosted a ZFS-themed meetup for the illumos community (sponsored generously by Joyent). Many thanks to Deirdre Straughan, the new illumos community manager, for helping to organize and for filming the event. Three of my colleagues at Delphix presented work they’ve been doing in [...]
Read more...Playing with Node/V8 postmortem debugging
“Post Mortem” by C. MacLaurin Several weeks ago I posted about postmortem debugging for Node.js, a critical technique for understanding fatal software failure (and thereby keeping up software quality). Now that the underlying pieces are freely available[1], you can use the documentation below to start debugging your own Node programs. With these tools you can [...]
Read more...Activity of the ZFS ARC
Disk I/O is still a common source of performance issues, despite modern cloud environments, modern file systems and huge amounts of main memory serving as file system cache. Understanding how well that cache is working is a key task while investigating disk I/O issues. In this post, I’ll show the activity of the ZFS file [...]
Read more...In: ARC, DTrace, Kernel, ZFS, performance
Where does your Node program spend its time?
Photo by Julian Lim (flickr) Performance analysis is one of the most difficult challenges in building production software. If a slow application isn’t spending much time on CPU, it could be waiting on filesystem (disk) I/O, network traffic, garbage collection, or many other things. We built the Cloud Analytics tool to help administrators and developers quickly [...]
Read more...