Archive for June, 2004

Per-process statistics and the future of Solaris observability

So a few posts ago I asked for some suggestions on improving observability in Solaris, specifically with respect to LSOF. I thought I’d summarize the responses, which fell into two basic groups: Socket and process visibility. Something along the lines of lsof -i or netstat -p on Linux. Per-process mpstat, vmstat, and iostat. I’ll defer [...]

Posted on June 30, 2004 at 10:42 pm by eschrock · Permalink · 2 Comments
In: OpenSolaris

Top I/O consumers in 5 minutes or less

In my previous blog post, it was mentioned that it would be great to have a prstat-like tool for showing the most I/O hungry processes. As the poster suggested, this is definitely possible with the new io provider for DTrace. This will be available in the next Solaris Express release; see Adam’s DTrace schedule for [...]

Posted on June 30, 2004 at 12:10 pm by eschrock · Permalink · Comments Closed
In: OpenSolaris

What can lsof do for you?

Over the years, we have been keeping an eye on the LSOF (LiSt Open Files) utility written by Vic Abell. It’s a well respected utility that provides indispensible information that cannot be found any other way. We have tried to emulate some aspects of lsof, but we still fall short in several categories. As part [...]

Posted on June 29, 2004 at 12:28 pm by eschrock · Permalink · 4 Comments
In: OpenSolaris

The (Secret) Agent LWP

One of the most powerful but least understood aspects of the Solaris /proc implementation is what’s known as the ‘agent lwp’. The agent is a special thread that can be created on-demand by external processes. There are a few limitations: only one agent thread can exist in a process, the process must be fully stopped, [...]

Posted on June 27, 2004 at 9:45 pm by eschrock · Permalink · 8 Comments
In: OpenSolaris

A brief history of /proc

Those of you who have used a UNIX system before are probably familiar with the /proc filesystem. This directory provides a view of processes running on the system. Before getting into the gory details of the Solaris implementation (see proc(4) if you’re curious), I thought I would go over some of the different variants over [...]

Posted on June 25, 2004 at 1:49 am by eschrock · Permalink · 2 Comments
In: OpenSolaris

Fun facts about corefiles

All you developers out there are probably well acquainted with corefiles. Every CS student has had a program try to dereference a NULL pointer at least once. Frequent use of assert(3c) causes your program to abort(3c) in exceptional circumstances. Unfortunately, too many developers know little else about corefiles. They are often used for nothing more [...]

Posted on June 22, 2004 at 10:54 am by eschrock · Permalink · 7 Comments
In: OpenSolaris

My kingdom for a crash dump

In the Solaris group, we take a great deal of pride in our debugging tools. Every tool we design (truss, DTrace, MDB, KMDB, ptools, libumem) helps reduce time to root cause from days to hours or minutes, in addition to solving previously unsolvable problems. Over the past week or two I’ve some conversations about the [...]

Posted on June 20, 2004 at 11:00 pm by eschrock · Permalink · 2 Comments
In: OpenSolaris

Shameless self-promotion

One of the most visible features that I have integrated into Solaris 10 is the ability to store pathnames with each open file1. This allows new avenues of observability that were previously inaccessible. First off, we simply have the files as symbolic links in /proc/<pid>/path: $ ls -l /proc/`pgrep Firebird`/path | cut -b 55- 0 [...]

Posted on June 18, 2004 at 3:48 pm by eschrock · Permalink · 3 Comments
In: OpenSolaris

Software Abstraction and the Jaws of Life

Before I start looking at some of problems we’re addressing in Solaris, I want to step back and examine one of the more fundamental problems I’ve been seeing in the industry. In order to develop more powerful software quickly, we insert layers of abstraction to distance ourselves from the actual implmentation (let someone else worry [...]

Posted on June 17, 2004 at 5:09 pm by eschrock · Permalink · 2 Comments
In: Software

Descent into the fray

So begins my first blog post ever. I have been a Solaris Kernel Engineer for 10 months now after graduating from my alma mater. Since I joined so late in the Solaris 10 development cycle, I have not had the pleasure of working on one of the larger S10 projects such as DTrace, Zones (N1 [...]

Posted on June 17, 2004 at 2:25 pm by eschrock · Permalink · Comments Closed
In: General