APFS in Detail: Performance

This series of posts covers APFS, Apple’s new filesystem announced at WWDC 2016. See the first post for the table of contents.


Performance

APFS claims to be optimized for flash. Flash memory (NAND) is the stuff in your speedy SSD. Apple changed the computing industry when it put flash into the iPod and iPhone, volumes for which fundamentally changed the economics of flash. This consumer change impacted the enterprise (as it often does), giving rise to hybrid and all-flash arrays. Ten years ago flash cost as much as DRAM; now it’s challenging the economics of hard disks.

SSDs mimic the block interface of conventional hard drives, but the underlying technology is completely different. In particular while magnetic media can read or write sectors arbitrarily, flash erases large chunks (blocks) and reads and writes smaller chunks (pages). The management is done by what’s called the flash translation layer (FTL), software that makes blocks and pages appear more like a hard drive. An FTL is very similar to a file system, creating a virtual mapping (a translation) between block addresses and locations within the media. Apple controls the full stack including the SSD, FTL, and file system; they could have built something differentiated, optimizing this components to work together. What APFS does, however, is simply write in patterns known to be more easily handled by NAND. It’s a file system with flash-aware characteristics rather than one written explicitly for the native flash interfaces, more or less what you’d expect in 2016.

Also on the topic of flash, APFS includes TRIM support. TRIM is a command in the ATA protocol that allows a file system to indicate to an SSD (specifically, its FTL) that some space has been freed. SSDs require significant free space and perform better when there’s more of it; they include more physical space than they advertise. For example, my 1TB SSD includes 1TB (240 = 10244) bytes of flash but only reports 931GB of available space, sneakily matching the storage industry’s self-serving definition of 1TB (10004 = 1 trillion bytes). With more free space, FTLs can trade off space efficiency for performance and longevity. TRIM has become expected of file systems; it’s unsurprising that APFS supports it. The problem with TRIM though is that it’s only useful when there’s free space: it’s something of a benchmark special. Once your disk is mostly full (as mine are in my laptop and phone basically at all times) TRIM doesn’t do anything for you. I doubt that TRIM will bring any discernible benefit for APFS users beyond the placebo effect of feature parity.

APFS also focuses on latency; Apple’s number one goal is to avoid the beachball of doom. APFS addresses this with I/O QoS (quality of service) to prioritize accesses that are immediately visible to the user over background activity that doesn’t have the same time-constraints. This is inarguably a benefit to users and a sophisticated file system capability.

 

Next in this series: Data Integrity

Posted on June 19, 2016 at 7:36 pm by ahl · Permalink
In: Software · Tagged with: , , , , , ,

11 Responses

Subscribe to comments via RSS

  1. Written by UNIX admin
    on June 20, 2016 at 10:43 am
    Permalink

    What is the state of TRIM support in OpenZFS?

    • Written by scineram
      on June 20, 2016 at 12:09 pm
      Permalink

      FreeBSD has a hackish implementation. The proper thing will be spread around from Nexenta hopefully soon.

  2. Written by Tom Limoncelli
    on June 20, 2016 at 12:26 pm
    Permalink

    I think this is a great article but you blew it with the TRIM discussion.

    APFS was designed specifically for SSDs and your treating TRIM as this niche feature that doesn’t matter. A statement like “I doubt that TRIM will bring any discernible benefit for APFS users beyond the placebo effect of feature parity.” is going to spread misinformation that will be difficult to unspread.

    First, you are saying it is only useful when a volume isn’t full, and since your volumes are always full the feature is useless.

    Let’s unpack that:

    1. TRIM is only useful when there is a lot of free space. No, TRIM is even more critical when space is low. A hard drive can reuse a block any time. An SSD must erase it first, which takes a long time. An SSD erases blocks in the background so that the next time a write is required there is a pre-erased block ready to be used. Even if you have a single block free, writes will be faster if TRIM has informed the SSD what it can pre-erase.

    2. Most people run volumes that are full. Not true at all. I doubt you can cite a paper that confirms that.

    I think you should fix the TRIM section.

    You give TRIM multiple paragraphs and say it isn’t important (which it is), but then you only give 1 paragraph to QoS, and totally miss how important it is. Because writing to an SSD is so slow, being able to specify that (for example) metadata updates are higher priority than other updates, APFS will probably gain a huge amount of reliability. This may compensate for many other problems.

  3. Written by ahl
    on June 20, 2016 at 2:11 pm
    Permalink

    You’re probably right that I was too disdainful of TRIM, but I’m not convinced that typically users will see much benefit. You’re right that I can’t cite a paper that confirms that most people run their volumes full; I bet Apple has those data. Have you seen a study on the benefits of TRIM when, say, <5% of space is free?

    What else would I say about QoS? I think it’s pretty straightforward, *and* I pretty much lavished praise on APFS there…

  4. Written by nick s
    on June 21, 2016 at 2:34 am
    Permalink

    In terms of performance, modern copy-on-write filesystems with what you might call ZFS-ish feature sets tend to be more RAM-hungry than older ones. Given that Apple has a reputation for under-speccing its hardware in terms of RAM, and most of its hardware can’t be upgraded, I wonder how that’s going to affect the performance on lower-tier systems?

    Perhaps that’s the rationale behind the missing features so far such as checksumming? Or perhaps HFS+ is RAM-hungry in its own way?

  5. Written by Lanell
    on June 21, 2016 at 9:28 pm
    Permalink

    Incredible….this is a important webpage.|

  6. Written by cmurf
    on June 22, 2016 at 6:43 pm
    Permalink

    TRIM’s benefit is orthogonal to free space. What matters is the availability of already erased, ready to be written, pages at the time the file system has something to write (or even overwrite, since everything on an SSD is effectively CoW). The SSD has only two ways of knowing if a page is not being used by the file system: passively by inferring it when the fs asks for an “in-use” LBA to be overwritten, and directly when the fs uses TRIM. In the first case it gets this information at the time of a write so either there’s an already erased page free to CoW to, or there’s no erased pages and it has to do garbage collection and erase some pages (in erase block increments) before it can commit the write, and this is slow.

    There is a clear advantage of the SSD having a stateful view of in-use pages similar to that of the fs. If I copy a bunch of files to the SSD to fill it up to 99%, then delete 50% of them, the SSD thinks it’s full and will not erase those pages. The only way it knows they’re deleted is if I overwrite those LBAs with files or the fs issues TRIM.

    The problem with constantly issuing TRIM on non-queued TRIM supporting SSD’s is they’ll hiccup. The entire command queue has to be cleared before it can be used, and many SSDs hang during the ensuing garbage collect. Good ones delay garbage collection to avoid this side-effect. The work around for this is to not use TRIM as a mount time option (discard) where it results in TRIM everytime a file is deleted, but rather on a schedule such as once a week. This is also consistent with NVMe where it’s better to use delayed and scheduled TRIM than doing so concomitant with file deletions.

    Anyway, TRIM is important. But as a mount time option it’s totally overblown and has caused more problems than it solves.

  7. Written by Vinski
    on June 23, 2016 at 9:37 am
    Permalink

    TRIM is important. Without it write operations will eventually slow down considerably when SSD does not have any “fresh” blocks left, and it has to free the block first at write.

    Also, TRIM could be considered a security feature, since it makes sure that pages are physically erased when the file is deleted.

  8. Written by proxy
    on June 23, 2016 at 1:35 pm
    Permalink

    Thanks for sharing excellent informations. Your web-site is very cool. I’m impressed by the details that you have on this web site. It reveals how nicely you perceive this subject. Bookmarked this web page, will come back for extra articles. You, my pal, ROCK! I found simply the information I already searched everywhere and just could not come across. What a perfect site.

  9. Written by ฮอลิเดย์ พาเลซ
    on June 24, 2016 at 6:15 am
    Permalink

    APFS claims to be optimized for flash. Flash memory (NAND) is the stuff in your speedy SSD. Apple changed the computing industry when it put flash

  10. Written by Michael Tsai - Blog - Apple File System (APFS)
    on July 3, 2016 at 6:16 pm
    Permalink

    [...] Performance: Apple controls the full stack including the SSD, FTL, and file system; they could have built something differentiated, optimizing this components to work together. What APFS does, however, is simply write in patterns known to be more easily handled by NAND. [...]

Subscribe to comments via RSS