Big News for ZFS on Linux

Canonical announced a few weeks ago that ZFS will be included in the next release of Ubuntu Linux, on by default and fully supported. And it’s no exaggeration when Dustin Kirkland describes ZFS as “one of the most exciting new features Linux has seen in a very long time.” In the words of our 47th Vice President, this is a big F—ing deal. Ubuntu is an extremely popular Linux distribution, particularly so for servers, and while the Linux ecosystem doesn’t want for variety when it comes to filesystem choices, there is not a clear champion when it comes to stability, functionality, and performance. By throwing their full weight behind ZFS, Canonical brings the Linux community an enterprise-class, modern filesystem, stable, but still under active development, designed to perform well for a variety of workloads.

What’s ZFS?

ZFS was originally developed at Sun Microsystem for the Solaris operating system. Some of the most demanding production environments have depended on ZFS for well over a decade. At its core are the principles of data integrity, ease of use, and performance.[1] Most notably, ZFS has first-class support for arbitrary numbers of snapshots and writable clones, serialization for replication, compression, and data repair. I’ve contributed code to ZFS at Sun, then Oracle, and to OpenZFS after Oracle abandoned the project in 2010. I’ve also built two products built around ZFS, the ZFS Storage Appliance, a NAS box, and Delphix, a copy data virtual appliance.

Why ZFS?

While the distinguishing features of ZFS are broadly useful, they have become specifically relevant in a containerizing world. Users need to save, clone, and replicate containers at will; ZFS provides key facilities for doing so. Containers and ZFS are a fantastic match, something I’ve seen my friends at Joyent demonstrate decisively for the past decade. Ubuntu has selected the most capable technology for our modern computing ecosystem.

No good deed…

So high fives and bro hugs all around, right? Not quite. Enter the licensing boogie man. The Linux kernel is licensed under the GPL v2; OpenZFS is licensed under the CDDL. Both are open source, true, but some contend that they are incompatible. Most folks in the tech world—myself among them—spend somewhere in the vicinity of no time at all considering the topic. Far from ignoring it, Canonical had their lawyers review the licenses and deemed their use of Linux and OpenZFS to be in compliance. I’m not a lawyer; I don’t have an informed opinion. But there are those who vehemently disagree with Canonical. Notably the Software Freedom Conservancy whose mission is to “promote, improve, develop, and defend Free, Libre, and Open Source Software (FLOSS) projects” has posted a lengthy wag of the finger at Canonical. Note that none of this has been specifically tested in the courts so it’s currently just a theoretical disagreement between lawyers (and in many cases, people who engage in lawyerly cosplay).

The wisdom of the crowd has proposed a couple of solutions:

“What if we ask Oracle super nicely?”

Oracle holds a copyright on most of OpenZFS since it was forked from the original ZFS code base. It would be within their rights to decide to relicense ZFS under the GPL. Problem solved! No way and not quite. Starting with the easier problem there are many other copyright holders in OpenZFS. It’s not an impossibly large list, but why would they bother? What benefits would they reap when even goodwill isn’t noticeably on offer? And it is the height of delusion to think that Oracle would grow ears to hear, a heart to care, and a brain to decide. Oracle explicitly backed away from OpenSolaris, shutting down the project in 2010. They do not want to encourage open source use of its component technologies. While open source is arguably the most significant shift in technology over the last decade (Stephen O’Grady’s The Software Paradox is a must-read), large companies and startups continue to be terrified, confused, and irrational when it comes to open source. Oracle ain’t coming to help.[2]

“Let’s re-write it! How hard could it be?”

It’s hard to dignify this with an explanation, but OpenZFS is the product of 100s of person-years. It contains some of the most sophisticated mechanisms I’ve seen designed, by some of the world’s most capable engineers. Re-writing it would probably be no easier than writing it the first time. By way of commentary, this is what makes NIH so distressing. Too often technologies are copied poorly instead of being used and improved, or understood and replaced with something truly superior.

Now what?

Now that you understand a bit of the context here’s my suggestion: consider the licenses, but focus on the technology. Canonical has (one would presume) chosen to include OpenZFS because it offers the best solution to Ubuntu users. Containers and ZFS are highly complementary with further room to grow together. As with anything, evaluate the technology, evaluate the risks, and move on. Ignore pedants who would deride your pragmatic use of technology as heretical or immoral.

I personally could not be more excited by the announcement. The Ubuntu community is going to have built-in support for a filesystem that’s better and more capable than anything they’ve had in the past. The OpenZFS community is going to have a ton more users, more interest, and more drivers for innovation. Both are going to be stronger together.

 

 


[1] “ZFS crashed on me once!” Me too, more than once. “ZFS was slow for me!” That happens. “[some other Linux filesystem] is better!” Could be, but I doubt it. I’m not denying the events, but this kind of Inhofian logic doesn’t nudge ZFS from its perch.

[2] In 2011 the source code for Oracle’s new Solaris 11 operating system appeared on the web replete with CDDL (open source) license notification. For all appearances this looked like open source code, a new version of OpenSolaris. The community asked for clarification: were these stolen goods or something given away intentionally? Was Solaris 11 free and open? Even then Oracle declined to comment.

Posted on March 7, 2016 at 4:30 pm by ahl · Permalink
In: ZFS

24 Responses

Subscribe to comments via RSS

  1. Written by Shawn Walker-Salas
    on March 7, 2016 at 6:43 pm
    Permalink

    I would just like to point out that the Image Packaging System, a key component of Solaris 11 and later is still an open source project licensed under the CDDL, where development and discussion happen in the open:

    https://java.net/projects/ips

  2. Written by Big News For ZFS On Linux | ..:: Frog In The Box ::..
    on March 7, 2016 at 7:01 pm
    Permalink

    [...] News for ZFS on Linux Comments Articolo Originale: http://dtrace.org/blogs/ahl/2016/03/07/big-news-for-zfs-on-linux/ Condividi:TweetCondividi su TumblrPocketE-mailStampaMi piace:Mi piace [...]

  3. Written by dan
    on March 7, 2016 at 7:10 pm
    Permalink

    Should I do the same with my copy of Oracle database? I could just make as many copies as I can sell and profit!

    Why hasn’t anyone thought of this before?

    You sir, are a genius. This could transform the software business as we know it.

    • Written by dan
      on March 7, 2016 at 7:13 pm
      Permalink

      And by “do the same” I mean ignore the license.

      And yes, I am trolling.

  4. Written by Paige Thompson
    on March 7, 2016 at 7:16 pm
    Permalink

    What does ZFS have going for it besides block level crypto and compression? software raid? BTRFS has this except for crypto, not that I want that to change anyway (I barely trust LUKS as it stands.) mdadm has one advantage as it stands which is that on parity raid you can adjust the stripe cash size to use more memory in favor of better performance.

    volume management, snapshots, online defrag, I’ve used all of this for well over a year with BTRFS, without any notable penalties; why are people using OpenZFS?

    https://www.reddit.com/r/linux/comments/32cu9w/zfs_vs_btrfs/

    “using it in production” where would you ever have a legitimate use for this filesystem on a “production” server..

    • Written by Nick Price
      on March 7, 2016 at 11:14 pm
      Permalink

      You would have a legitimate use for ZFS on a production server where you care about data integrity and redundancy on a platform that’s been in use for well over a decade.

      It’s nice that you have over a year of experience using BTRFS without any major issues, but it’s only been called “stable” for three years now, and plenty of people have had plenty of issues with it, along with the other house of cards you need to pile on top of it to gain complete feature parity.

      When current issues on the official site contain the term ‘massive data corruption’ it’s pretty safe to say it’s not mature enough to use in production.

      https://btrfs.wiki.kernel.org/index.php/Gotchas

      I feel like for something used in production, I should be able to mount something with a loopback driver without having to look at the docs for my particular filesystem to make sure it won’t destroy my data.

  5. Written by Dustin Kirkland
    on March 7, 2016 at 7:17 pm
    Permalink

    It’s probably also worth linking to Eben Moglen and the Software Freedom Law Center’s analysis (which discusses dtrace, too): https://www.softwarefreedom.org/resources/2016/linux-kernel-cddl.html

    Cheers,
    Dustin

    • Written by ahl
      on March 7, 2016 at 7:21 pm
      Permalink

      Thanks for the link!

  6. Written by Paige Thompson
    on March 7, 2016 at 7:22 pm
    Permalink

    also FWIW, I’m using BTRFS on LUKS on SSD …. you’ll want to do some reading to make sure you understand how to get trim working correctly but its been around.

  7. Written by darkfader
    on March 7, 2016 at 8:11 pm
    Permalink

    Funny how you managed to not refer to btrfs on the “let’s rewrite it, how hard can it be” section, especially considering Oracle has paid it’s development for years and years to ensure an open alternative was on the horizon. even to the point now where they and SUSE offered support for it.
    That it sucks and isn’t remotely something you’d wish your financial data you rest on is one of the wonders of open source.

    To those who think there’s only a marginal gap: No. Sorry. If you don’t see the difference, learn more about ZFS, consider large scale usage and maintenance and seriously, if you then still don’t see it, you haven’t learned enough yet.

  8. Written by Tim
    on March 7, 2016 at 8:14 pm
    Permalink

    The article contains some implicit contradictions.

    “It’s hard to dignify this with an explanation, but OpenZFS is the product of 100s of person-years.”

    yet:

    ““[some other Linux filesystem] is better!” Could be, but I doubt it.”

    BTRFS was designed and implemented years after ZFS, using the same basic principles, and attempting to learn from its design mistakes (and as an early ZFS user, I know them well!). There’s nothing that ZFS does that isn’t already done by BTRFS, or is on the BTRFS roadmap for the near future.

    In a sense, we’ve already put in 100s of person-years into rewriting (and improving on) ZFS under the GPL, and it’s 90% of the way there.

    The question is no longer “Can I squint just right and pretend ZFS is GPL-compatible?” or “Should we rewrite it all from scratch?” There’s a new option on the table, which is “Can we take this thing which is 90% of the way there, and improve it in the ways we need to be on par with ZFS?”

  9. Written by cmurf
    on March 7, 2016 at 8:30 pm
    Permalink

    I have to agree with the Btrfs v ZFS comments. ZFS is certainly more stable, it had a very fast and aggressive start in comparison, and has had more time in the oven. So if you need something stable, for users to pick ZFS on Linux is completely reasonable. For Canonical to dither over the legality, and include ZFS by default rather than improve their Btrfs contributions is very telling.

    RFC patch for experimental encryption was posted a week ago. This initially implements it as per subvolume encryption. The eventual idea is to also support per file encryption using the same ABI/API as f2fs and ext4.

    • Written by ahl
      on March 7, 2016 at 8:45 pm
      Permalink

      > For Canonical to dither over the legality, and include ZFS by default rather than improve their Btrfs contributions is very telling.

      Spot on.

  10. Written by Adam
    on March 7, 2016 at 9:10 pm
    Permalink

    Someone should fix the CSS on this blog. The BODY is set to a font-size of 62.5%! That makes all the text on the screen tiny! The comments section’s main text is set to a size of 8.75 *pixels*! Only by using Firefox’s Inspector tool and disabling the font-size settings in the DOM am I able to comfortably read this page.

    Please do us all a favor and google up “Let users control font size.”

    • Written by Nick Price
      on March 7, 2016 at 11:20 pm
      Permalink

      Defining font sizes in pixel is not accessible, because the user cannot change the font size from the browser. (For example, users with limited vision may wish to set the font size much larger than the size chosen by a web designer.) Therefore, avoid using pixels for font sizes if you wish to create an inclusive design.

      A popular technique to use throughout the document is to set the the font-size of the body to 62.5% (that is 62.5% of the default of 16px), which equates to 10px, or 0.625em. Now you can set the font-size for any elements using em units, with an easy-to-remember conversion, by dividing the px value by 10. This way 6px = 0.6em, 8px = 0.8em, 12px = 1.2em, 14px = 1.4em, 16px = 1.6em.

      This *is* to allow users to control font-size – instead of arbitrarily specifying pixel sizes, using em, the fonts are now directly relative to the default font size set in the user’s browser/OS.

      https://developer.mozilla.org/en-US/docs/Web/CSS/font-size

  11. [...] Leventhal’s blog » Big News for ZFS on LinuxRead More: Adam Leventhal’s blog » Big News for ZFS on Linux [...]

  12. [...] 記事詳細:http://dtrace.org/blogs/ahl/2016/03/07/big-news-for-zfs-on-linux/ 投稿日: 2016年3月8日作成者 wpmaster [...]

  13. Written by Frew Schmidt
    on March 8, 2016 at 12:54 am
    Permalink

    Here’s another link that considers the license combination in a more refreshing and charitable light than the SFC did. I found it fairly well put together: http://blog.hansenpartnership.com/are-gplv2-and-cddl-incompatible/

  14. Written by Nan Xiao
    on March 8, 2016 at 1:18 pm
    Permalink

    “large companies and startups continue to be terrified, confused, and irrational when is comes to open source.” should be “… when it comes to …”.

    • Written by ahl
      on March 8, 2016 at 8:24 pm
      Permalink

      Thanks; fixed.

  15. Written by Desmond Faulker
    on March 12, 2016 at 3:48 am
    Permalink

    AFAIK btrfs is not ready yet. It should take some time for it to get reliable and feature complete enough for professional use.

  16. Written by jelabarre59
    on March 17, 2016 at 1:15 pm
    Permalink

    But I think there are two issues at play here. One involves *distributing* ZFS, the other involves *using* it. I would think you should be able to download, then compile or directly install OpenZFS on your own system, and then run it. Certainly, it may affect your support contract if you are relying on one, and it may render your kernel “tainted” (depending on how it hooks in), but how you mix and match open-source code on your _own_ system shouldn’t matter.

    Distribution/linking of differently-licenced code is where you hit issues. Suitability of purpose discussions just aren’t relevant to the issue (other than deciding how much effort it’s worth to resolve the dispute).

  17. Written by Rich Teer
    on April 7, 2016 at 6:23 pm
    Permalink

    As a long-time ZFS (and ex-Solaris) fan, this is indeed great news. I wonder if Red Hat could be persuaded to do the same thing?

  18. Written by Daniel Letai
    on April 16, 2016 at 4:04 pm
    Permalink

    It’s almost as much telling that RedHat, after years of promoting BTRFS, decided to move to XFS instead. What do RedHat and Canonical know that the entire BTRFS fandom do not?

Subscribe to comments via RSS