So the DTrace team is currently at USENIX ’04, where yesterday we presented our paper on DTrace. The presentation went quite well — though it’s a bit difficult to jam so much content in 25 minutes! The reception to the work was very positive, and even the questions largely praised DTrace. The only wrinkle in the whole operation came with the last question, from an employee of IBM. To paraphrase:
DTrace seems great, I imagine many people would be interested in this, etc. etc. When are you going to port it to Linux?
I’m afraid that my answer was probably perceived as a bit politically incorrect; it was roughly:
Look: we believe in choice; we believe that people should pick the best operating system for the task at hand. We’ve been busting butt for the last three years on DTrace to make Solaris the best operating system for many different tasks. Enough said…
To be clear: this is not an attack on Linux. But there is a fundamental disagreement out there: many seem to believe that the operating system is a “commodity” — that all operating systems are basically the same. We disagree: we believe that the operating system is a nexus of innovation. And we believe that we’re proving that with Solaris 10 technologies like DTrace, Zones, ZFS, etc. etc. You certainly don’t have to agree with us; we believe that you always have the right to choose Linux and that there may be many great reasons to do so. But please stop asking if we’re going to port these features to Linux — if you want to take advantage of our OS innovation, run
Solaris.
25 Responses
Preach on, brother. I mean .. um .. I meant to say: when are you going to rewrite the Linux kernel from scratch so it is able to do some of the things the Solaris kernel can do?
Bryan, I’m a Sun stock investor, not an employee. The best way I can express my thought on software commoditization is through my experience with web browsers. I use both Internet Explorer and Mozilla and for some time I viewed them as commodities. One was just as good as the other. However, this changed when some website managed to take over the search function of my IE. Ad-aware and Spybot and SpySweeper were unable to remove this malware. Finally, I had to use a program called HijackThis which was a dangerous thing for me to get involved with because it was designed for people who know more about computer than me. Luckily, it worked.
However, this crisis elevated Mozilla over IE because it never experienced this form of web attack. Now the browsers are no longer commoditized. One is superior to the other in my mind and it took a crisis to do it. What I’m wondering is this — Is there a server OS crisis looming out there which will wake up corporate clients and cause them to see the superiority of Solaris in much the same way I eventually saw the superiority of Mozilla? You know what they say, “When the tide rolls out we find out who’s swimming naked!”
It’s called “Windows 2003” vs. Sasser, MyDoom and all the rest of ’em. Windows folk are already too far gone, as you experienced yourself with IE. After years and years of abuse as a desktop OS, Windows broke into the server market and dragged all the abuse it took with it into the datacenter.
This in itself has not been enough to signal that Microsoft and its customers can’t deal with these problems very well–are we not already in a crisis here? Windows folk are too far gone and they just don’t know it.
P.S. Solaris 10+DTrace rocks. 🙂 It can help me squeeze every last bit of performance out of my U2 as well as tell me that the crummy disk controller in my U5 is hogging most of my CPU time.. but I knew that last bit already.. 😉
That’s an excellent question about the “server OS crisis.” We believe that there is indeed a crisis, but in ways that people don’t necessarily realize. For example, we believe that there is a “crisis” in terms of the lack of observability of servers. People don’t necessarily realize that it’s a crisis, because they didn’t realize that technologies like DTrace were possible. Now that customers are seeing what is possible with DTrace, they wondered how they ever lived without it. In effect, DTrace is creating a crisis — for our competitors. This is also true of Zones, ZFS, the service management facility, etc.: these technologies address crises that our customers didn’t realize that they had, because they didn’t realize that there was an alternative…
Many of the problems with MS we’re referencing I feel can be better blamed on its utter desktop dominance than on any single technological flaw – which it pains me to say, being very much anti-MS.
Having a remote root RPC exploit is not unique to Windows by any means – having a presence on just about every user desktop in the world is, however. You can’t ignore what a huge role this plays in both virus vectors, and in what the virus creators choose to target in the first place.
I agree there is a crisis, however, and the crisis is that a superior technology (Solaris) stands a good chance of being displaced and ultimately dying, basically because Sun is losing a massive marketing war to the HP/Dell/MS/IBMs of the world. Linux is just one of the weapons being extremely effectively wielded by these marketing juggernauts, for whom “commodity” and “industry standard” are battle-cries that do not even mean what they think they mean.
The question Sun’s marketing teams should be asking is: how do you market dtrace, zones, and ZFS? To the people making the ultimate platform decisions in the Fortune 100, it’s gibberish. An ‘objective’ TCO analysis performed by a Meta, Gartner, or Giga (underwritten by Dell, of course) is worth 1000x what any dtrace cover story on sun.com is.
Some people hail Dell as proof of comoditisation of certain computer markets. I think it’s the opposite – Dell proves that customers don’t simply react to price. Why? Because if they did – if those markets truely were commodities – then HP’s and IBM’s x86 server business would be toast (pretty much) and Dell would have it all (or perhaps the plain white-box makers). Dell (and the white-box guys) have been gaining market share, but only at a few percent a year in the server space.
I really agree with the fact that Linux and open source shouldn’t mean there’s no room for innovation and that it’s not worh it investing in R&D because you can much cheaper replicate the latest features.
Some commoditization is good or otherwise companies wouldn’t be hard pressed to look for new technologies. Plus the price would be much higher.
As for Sun marketing and transmitting clear messages to the public: it’s far behind Sun engineering. Almost anyone agrees there are great engineers and products (among which Solaris gets the top spot) but the marketing is bad, far behind competitors. Although I’ve seen some recent Sun ads on yahoo.com … looks like the right direction to me.
Just a thought : if you opensource Solaris including DTrace what would stop the Linux distros to copy the model (not only for DTrace but for other technologies as well). Or the right way of thinking would be: before the catch up you’ll be ahead already ? So open sourcing Solaris brings OS popularity.
Remember: “open source” is not synonomous with the GPL; there are lots of ways to have an OSI-approved license that is not the GPL. We’re not doing this to become a technology donor to Linux; we’re doing this to give our customers the advantage of having the source code, and being able to customize and contribute to Solaris as they see fit.
Perhaps this is a discussed somewhere I haven’t seen…Are there any differences between running dtrace on a sparc machine versus an x86{64} machine? (does dtrace run on x86?)
As for a ‘port’ to Linux, I think the kernel internals between Solaris and Linux would be great enough that it would end up being a totally different product 🙂
I have worked on IRIX. Stop laughing. IRIX kernel dorks used to whistle this same tune circa 1998. “IRIX has XFS! And NUMA smarts! SpeedShop is da’ bomb! Linux will never catch up!” Reading your post, I realize how delusional we IRIX folks must have seemed. You’re right that Solaris is great. However, it is also dying, and dtrace is unlikely to save it. IBM, HP, and all those other Linux companies are investing lots of money in turning Linux into Solaris, and while it may take them years, they’ll get there eventually. In the meantime, it’s a shame that a phenomenal tool like dtrace isn’t available where it would do the most good, and where it has the greatest chance of surviving.
Dtrace certainly runs on x86. I run it on my notebook all of the time. There are a coupel of small things that are different, but that generally comes down to the processor/machine architecture. For example, the way that bcmp and bcpy are implemented on x86 (directly in assembly) means that they don’t have the prologue/epilogue that dtrace looks for when createing the fbt probe points. As such they can’t be probed. The new fpu provider is specifically for looking at what sparc floating point instructions are emulated. That’s probably not going to do much under x86 either.
With regard to the comments about Dtrace not being enough to save Solaris. The whole point is that Dtrace is not the only phenomenal technology that is going in to 10. 10 will be the most featureful release of Solaris since (and possibly including) 2.4. There is some truly amazing stuff in there. Dtrace, Zones and ZFS are only scratching the tip of the iceberg. I can wholeheartedly recommend following the “Solaris 10” link on Bryan’s page and having a good read.
Bryan, Amen.
Brian. I think a good answer to the IBM plant would have been: “Open sourcing DTrace, or directly porting it to Linux is one way to address the diagnosis needs of Linux. However, you guys at IBM have some good ideas, some perhaps better than ours. So I turn the challenge to you. Why doesn’t IBM create something better than DTrace for Linux. As Bill Joy famously said, innovation occurs elsewhere. So if you think something like DTrace would be good for Linux, build it.”
I like your answer, Mark. 😉 And it’s a relief to know that there’s an answer that would have been even more confrontational than the one I gave… 😉
Payjoecon2: “It’s called “Windows 2003″ vs. Sasser, MyDoom and all the rest of ’em. Windows folk are already too far gone, as you experienced yourself with IE. After years and years of abuse as a desktop OS, Windows broke into the server market and dragged all the abuse it took with it into the datacenter.”
Could you clarify that statement. Win2k3 wasn’t vulnerable to sasser. MyDoom was a worm that required a user to save an attachment and then execute it. I’m not sure what consumer OS out there that could have stopped that.
Well, Linux had Dprobes for ages, which basically is like Dtrace, just without all the marketing fuzz..
http://oss.software.ibm.com/developer/opensource/linux/projects/dprobes/
DProbes is “basically is like Dtrace [sic]”? Give me a break. For starters, I refer you to Section 2 of our USENIX paper — which explicitly compares DProbes to DTrace. And since you assert that DProbes and DTrace are so similar, would you mind providing the DProbes equivalents of the examples in Section 9 of the USENIX paper? (This will be quite some trick, as DProbes has neither thread-local varibles nor aggregations — two features critical to effective use of DTrace.) And as long you’re at it, how about attaching your name to future assertions so that we can have an honest and open discussion?
Some comments on porting / open sourcing Dtrace.
Given that Dtrace isn’t being backported to Solaris 9 due to the complexity, wouldn’t it be an order of magnitude more complex to port it to Linux (or whatever)? Also, to do a reasonably complete port, wouldn’t changes be need to be made all over the Linux kernal? That could be quite hard to organise, given the distributed development model.
I’m sure the majority of Solaris users would prefer more work to go into improving Dtrace on Solaris, rather than porting it. I doubt management could be persuaded to get some more resources for the porting work either. Maybe the next time someone from IBM brings it up, maybe you could ask if they have a few Linux kernal experts to spare for several months/years…? ^-^
Maybe another response you could have given the IBM guy: if we ported all the cool unique stuff Solaris has to Linux, the result would be more Solaris than Linux.
(I’ve no idea if this is true or not, but it’s interesting to consider…)
I like how some Linux folks are clearly unable and/or unwilling to realize that the elegance and utility of DTrace is two orders of magnitude greater than they could ever imagine. And people ask me why I bother with Solaris..! (I was about to say Sun hardware, but it’s in Solaris x86, too…)
…. on second thought, what we really need is an hour-long Net Talk dedicated to DTrace. Any chance of you doing another one? Your last session on Solaris 10 was as useful as it was amusing. 🙂
Hey Derek, It isn’t quite as good as a Net Talk, but did you see the Expert Exchange we did last month? A transcript is available here: http://see.sun.com/Apps/DCS/mcp?r=703SsI4297t0006304e4m32f403gpD04299i0
(And we tried to even keep it amusing…)
[Trackback] To be perfectly honest, it is with some trepidation that I have started
this blog. While I marvel at this new opportunity to be a part of
these conversations with the various Sun communities, I worry that it
will be difficult time-wise to stay invo…