Reflecting on The Soul of a New Machine

Long ago as an undergraduate, I found myself back home on a break from school, bored and with eyes wandering idly across a family bookshelf. At school, I had started to find a calling in computing systems, and now in the den, an old book suddenly caught my eye: Tracy Kidder’s The Soul of a New Machine. Taking it off the shelf, the book grabbed me from its first descriptions of Tom West, captivating me with the epic tale of the development of the Eagle at Data General. I — like so many before and after me — found the book to be life changing: by telling the stories of the people behind the machine, the book showed the creative passion among engineers that might otherwise appear anodyne, inspiring me to chart a course that might one day allow me to make a similar mark.

Since reading it over two decades ago, I have recommended The Soul of a Machine at essentially every opportunity, believing that it is a part of computing’s literary foundation — that it should be considered our Odyssey. Recently, I suggested it as beach reading to Jess Frazelle, and apparently with perfect timing: when I saw the book at the top of her vacation pile, I knew a fuse had been lit. I was delighted (though not at all surprised) to see Jess livetweet her admiration of the book, starting with the compelling prose, the lucid technical explanations and the visceral anecdotes — but then moving on to the deeper technical inspiration she found in the book. And as she reached the book’s crescendo, Jess felt its full power, causing her to reflect on the nature of engineering motivation.

Excited to see the effect of the book on Jess, I experienced a kind of reflected recommendation: I was inspired to (re-)read my own recommendation! Shortly after I started reading, I began to realize that (contrary to what I had been telling myself over the years!) I had not re-read the book in full since that first reading so many years ago. Rather, over the years I had merely revisited those sections that I remembered fondly. On the one hand, these sections are singular: the saga of engineers debugging a nasty I-cache data corruption issue; the young engineer who implements the simulator in an impossibly short amount of time because no one wanted to tell him that he was being impossibly ambitious; the engineer who, frustrated with a nanosecond-scale timing problem in the ALU that he designed, moved to a commune in Vermont, claiming a desire to deal with “no unit of time shorter than a season”. But by limiting myself to these passages, I was succumbing to the selection bias of my much younger self; re-reading the book now from start to finish has given new parts depth and meaning. Aspects that were more abstract to me as an undergraduate — from the organizational rivalries and absurdities of the industry to the complexities of West’s character and the tribulations of the team down the stretch — are now deeply evocative of concrete episodes of my own career.

As a particularly visceral example: early in the book, a feud between two rival projects boils over in an argument at a Howard Johnson’s that becomes known among the Data General engineers as “the big shoot-out at HoJo’s.” Kidder has little more to say about it (the organizational civil war serves merely as a backdrop for Eagle), and I don’t recall this having any effect on me when I first read it — but reading it now, it resonates with a grim familiarity. In any engineering career of sufficient length, a beloved project will at some point be shelved or killed — and the moment will be sufficiently traumatic to be seared into collective memory and memorialized in local lore. So if you haven’t yet had your own shoot-out at HoJo’s, it is regrettably coming; may your career be blessed with few such firefights!

Another example of a passage with new relevance pertains to Tom West developing his leadership style on Eagle:

That fall West had put a new term in his vocabulary. It was trust. “Trust is risk, and risk avoidance is the name of the game in business,” West said once, in praise of trust. He would bind his team with mutual trust, he had decided. When a person signed up to do a job for him, he would in turn trust that person to accomplish it; he wouldn’t break it down into little pieces and make the task small, easy and dull.

I can’t imagine that this paragraph really affected me much as an undergraduate, but reading it now I view it as a one-paragraph crash-course in engineering leadership; those who deeply internalize it will find themselves blessed with high-functioning, innovative engineering teams. And lest it seem obvious, it is in fact more subtle than it looks: West is right that trust is risk — and risk-averse organizations can really struggle to foster mutual trust, despite its obvious advantages.

This passage also serves as a concrete example of the deeper currents that the book captures: it is not merely about the specific people or the machine they built, but about why we build things — especially things that so ardently resist being built. Kidder doesn’t offer a pat answer, though the engineers in the book repeatedly emphasize that their motivations are not so simple as ego or money. In my experience, engineers take on problems for lots of reasons: the opportunity to advance the state of the art; the potential to innovate; the promise of making a difference in everyday lives; the opportunity to learn about new technologies. Over the course of the book, each of these can be seen at times among the Eagle team. But, in my experience (and reflected in Kidder’s retelling of Eagle), when the problem is challenging and success uncertain, the endurance of engineers cannot be explained by these factors alone; regardless of why they start, engineers persist because they are bonded by a shared mission. In this regard, engineers endure not just for themselves, but for one another; tackling thorny problems is very much a team sport!

More than anything, my re-read of the book leaves me with the feeling that on teams that have a shared mission, mutual trust and ample grit, incredible things become possible. Over my career, I have had the pleasure of experiencing this several times, and they form my fondest memories: teams in which individuals banded together to build something larger than the sum of themselves. So The Soul of a New Machine serves to remind us that the soul of what we build is, above all, shared — that we do not endeavor alone but rather with a group of like-minded individuals.

Thanks to Jess for inspiring my own re-read of this classic — and may your read (or re-read!) of the book be as invigorating!

Posted on February 10, 2019 at 5:20 pm by bmc · Permalink
In: Uncategorized

11 Responses

Subscribe to comments via RSS

  1. Written by Four short links: 11 February 2019 – MasMaz
    on February 11, 2019 at 6:48 am
    Reply · Permalink

    [...] Reflecting on The Soul of a New Machine (Bryan Cantrill) — re-reading the book now from start to finish has given new parts depth and meaning. Aspects that were more abstract to me as an undergraduate—from the organizational rivalries and absurdities of the industry to the complexities of West’s character and the tribulations of the team down the stretch—are now deeply evocative of concrete episodes of my own career. [...]

  2. Written by Moishe Lettvin
    on February 11, 2019 at 7:20 am
    Reply · Permalink

    This is great and has inspired me to, also, go re-read _Soul of a New Machine_. I read it 30 years ago, when I was 17 and on a long bicycle trip with two other programmers and one philosophy major; it hit me hard, then, and I would love to re-read it with 30 years’ of industry experience behind me. Thanks for this!!!

  3. Written by Rob Jordan
    on February 11, 2019 at 10:01 am
    Reply · Permalink

    Count me as one more who read this, and whose career was defined by it. I was a teen coder, purely for fun, from the late 1970s. Went to University and studied Biochemistry, thinking that computers weren’t a ‘proper’ career option. Late in my final year, I read The Soul of a New Machine and it suddenly dawned on me, I could make a living AND have the greatest adventure. 35 years later and not one regret, I had great fun and a great career as a developer, architect and tech manager.

  4. Written by Jessamyn
    on February 11, 2019 at 11:56 am
    Reply · Permalink

    I saw this zipping by on Twitter. Heya I’m Tom West’s daughter and I always pop onto threads about SOANM to bang the drum of good work-life balance as well. My dad was an amazing man and we got along well, but he was a pretty distant dad and basically left the raising of my sister and me to my mom who might have wanted to do something professionally as well.

    The things he did at work, and his and Kidder’s ability to talk about them, were quite interesting and have lessons that can be relevant in many different timelines and lifestyles. But you know, I wound up moving to Vermont and helping people use technology to solve problems in their lives. You need all the things–technology, an understanding of people, and a life–and my dad possibly only had two of those at any one time. Thanks for writing this.

    • Written by bmc
      on February 11, 2019 at 3:36 pm
      Reply · Permalink

      Jessamyn,

      Thank you so much for your comment — and totally and completely agree. I think it’s a mistake for people to overly lionize your dad exactly because it neglects the personal consequences of some of decisions that were implicitly made. Of course, the time was different too: part of the great liberation of technology is that it has allowed us to weave our work lives into our personal lives. While this, too, has its drawbacks, speaking for myself personally, the rise of the internet has allowed me to spend orders of magnitude more time with my own three children than someone could in my station a generation ago!

      Thanks again for your thoughtful comment; I’m sure it’s somewhat peculiar to have had a facet of your own childhood made so public, and glad that you’ve taken the opportunity to remind us all of the need to balance technology with empathy, and both of them with the need for a fuller life!

  5. Written by Mark
    on February 12, 2019 at 11:28 pm
    Reply · Permalink

    This is a great post and really interesting comments! I first read this book a year or two after it was first published. At the time I was in college learning programming (on DEC’s microcomputers). A close friend of my family was an engineer who started using these Data General microcomputers because they were cost competitive with DEC. For these reasons it was an absolutely fascinating read (and very well written) but was left me with a disturbing impression of work in the computer industry. In particular the incredibly long hours and stress was depicted as heroic and justified for the cause. I had seem this almost obsessive behavior in this family friend and among others in IT (like some University IT staff who seemed to be in the office nearly 24 hours/day). I always wondered about their families and social life. And you knew once one big project ended, yet another would be started up.

  6. Written by Paul Ware
    on February 13, 2019 at 6:33 am
    Reply · Permalink

    I read the book when it first came out and it led me to work in leading edge computer and communications development. I have worked at more technology companies than I can remember. I have surfed the high tech circuit. The best part is I have gotten to be part of the development and test of the can’t be done products over and over. So I have basically spent 35 years living inside of that book and loving every minute of it.

  7. Written by Jeremy Schneider
    on February 13, 2019 at 10:17 am
    Reply · Permalink

    I interviewed at DG shortly after the book came out. When I asked if the book accurately reflected the culture and work atmosphere, the response was an embarrassed, “It’s not as bad as depicted in the book.” I ended up going to MIPS Computer Systems, which was a great choice.

  8. Written by Neal Firth
    on February 13, 2019 at 2:31 pm
    Reply · Permalink

    Wow! Someone who admits to reading my simulator chapter more than once. Thanks Bryan. Made my week.

    I have a somewhat unique perspective on how attitudes about the meaning of the book have changed. For the first years after publication students would sometimes hunt me down and ask about being part of such a team. They seemed to want to know how to find/ re-create a similar team of accomplishment. Then in the 90s they started asking how it felt to be exploited. Some wanted me to be angry. Others seemed to want to learn the best way to exploit. A few years ago I saw a copy of Soul in an office cubicle. The rest of that encounter can be summed up as “Had to read it for a class. Don’t see how something this old applies to modern development.”

    • Written by bmc
      on February 13, 2019 at 4:11 pm
      Reply · Permalink

      Neal,

      Wow — I am actually speechless, and deeply honored to be virtually visited by one of my childhood idols! Your story (at least, as told by Kidder) has been one I have retold many, many times over my career. In fact, I retold it so many times that I started retelling it incorrectly! While I got the bones of the story correct, I somehow got it in my head that the the senior engineers were too busy to find a bitesize project for you, so they gave you the simulator to keep you occupied — and when you completed it, they realized that they had forgotten to tell you that it couldn’t be done!

      While entertaining, that variant isn’t quite correct — and I suppose that Kidder’s retelling of your story became blended with the story about George Dantzig accidentally solving open problems as math homework — with the famous “you are not expected to understand this” comment in Unix tossed in for a little spice.

      Anyway, my apologies for (slightly) mistelling your story — and my heartfelt thanks for what you did at DG and on Eagle. I can assure you that you have inspired many of us (many times over!) to entrust a particularly thorny problem to someone bold and bright — and unburdened by the past. Many engineers are in your debt, even if they themselves are unaware of it, simply believing that it was surprising that they were given so interesting a problem so young! Thanks again, and thanks for the comment!

  9. Written by Neal Firth
    on February 13, 2019 at 3:11 pm
    Reply · Permalink

    A comment on the “impossible simulator” project.

    After the fact I applied the software project estimation techniques common at the time. To no surprise the estimates didn’t match reality. Even if I adjusted a work day to account for my typical 5 hours of sleep per night it didn’t come close.

    So, for years even I mostly believed the “nobody told him it couldn’t be done” theory. However, I now recognize that my personal development style combined with the team environment created by Carl, Ed, and Tom to organically operate as if the simulator was a modern Agile project. It has taken many years. But the concepts and terminology now exist to predict and describe how the simulator project would be successful as an Agile project.

    BTW… for those who can’t understand the simulator technical description in the book you’re not alone. I can’t either. Tracy and I had a long back and forth about that section. I tried to get him to correct it. But he finally convinced me that as a journalist he was bound to write that section using his best attempt to document one of my stream of consciousness descriptions. An important lesson in audience perception that has served me well over the years.

Subscribe to comments via RSS

Leave a Reply