APFS in Detail: Overview
Apple announced a new file system that will make its way into all of its OS variants (macOS, tvOS, iOS, watchOS) in the coming years. Media coverage to this point has been mostly breathless elongations of Apple’s developer documentation. With a dearth of detail I decided to attend the presentation and Q&A with the APFS team at WWDC. Dominic Giampaolo and Eric Tamura, two members of the APFS team, gave an overview to a packed room; along with other members of the team, they patiently answered questions later in the day. With those data points and some first hand usage I wanted to provide an overview and analysis both as a user of Apple-ecosystem products and as a long-time operating system and file system developer.
I’ve divided my review into several sections that span a few posts. I’d encourage you to jump around to topics of interest or skip right to the conclusion (or to the tweet summary). Highest praise goes to encryption; ire to data integrity.
- Basics
- Paying Down Debt
- Encryption
- Snapshots and Backup
- Management and Space Sharing
- Space Efficiency and Clones
- Performance
- Data Integrity
- Conclusion
Basics
APFS, the Apple File System, was itself started in 2014 with Dominic as its lead engineer. It’s a stand-alone, from-scratch implementation (an earlier version of this post noted a dependency on Core Storage, but Dominic set me straight). I asked him about looking for inspiration in other modern file systems such as BSD’s HAMMER, Linux’s btrfs, or OpenZFS (Solaris, illumos, FreeBSD, Mac OS X, Ubuntu Linux, etc.), all of which have features similar to what APFS intends to deliver. (And note that Apple built a fairly complete port of ZFS, though Dominic was not apparently part of the group advocating for it.) Dominic explained that while, as a self-described file system guy (he built the file system in BeOS, unfairly relegated to obscurity when Apple opted to purchase NeXTSTEP instead), he was aware of them, but didn’t delve too deeply for fear, he said, of tainting himself.
Dominic praised the APFS testing team as being exemplary. This is absolutely critical. A common adage is that it takes a decade to mature a file system. And my experience with ZFS more or less confirms this. Apple will be delivering APFS broadly with 3-4 years of development so will need to accelerate quickly to maturity.
Paying Down Debt
HFS was introduced in 1985 when the Mac 512K (of memory! Holy smokes!) was Apple’s flagship. HFS+, a significant iteration, shipped in 1998 on the G3 PowerMacs with 4GB hard drives. Since then storage capacities have increased by factors of 1,000,000 and 1,000 respectively. HFS+ has been pulled in a bunch of competing directions with different forks for different devices (e.g. the iOS team created their own HFS variant, working so covertly that not even the Mac OS team knew) and different features (e.g. journaling, case insensitive). It’s old; it’s a mess; and, critically, it’s missing a bunch of features that are really considered the basic cost of doing business for most operating systems. Wikipedia lists nanosecond timestamps, checksums, snapshots, and sparse file support among those missing features. Add to that the obvious gap of large device support and you’ve got a big chunk of the APFS feature list.
APFS first and foremost pays down the unsustainable technical debt that Apple has been carrying in HFS+. (In 2001 ZFS grew from a similar need where UFS had been evolved since 1977.) It unifies the multifarious forks. It introduces the expected features. In general it first brings the derelict building up to code.
Compression is an obvious gap in the APFS feature list that is common in many file systems. It’s conceptually quite easy, I told the development team (we had it in ZFS from the outset), so why not include it? To appeal to Dominic’s BeOS nostalgia I even recalled my job interview with Be in 2000 when they talked about how compression actually improved overall performance since data I/O is far more expensive than computation (obvious now, but novel then). The Apple folks agreed, and—in typical Apple fashion—neither confirmed nor denied while strongly implying that it’s definitely a feature we can expect in APFS. I’ll be surprised if compression isn’t included in its public launch.
Next in this series: Encryption, Snapshots, and Backup
Notice: get_the_author_email is deprecated since version 2.8! Use get_the_author_meta('email') instead. in /home/knmngmprl21d/public_html/blogs/wp-includes/functions.php on line 3467
on June 20, 2016 at 4:19 am
Permalink
Let’s hope that they reconsider the user data checksum decision. Claiming it’s not required because of excellent components is kind of a strange thing to say, and makes the entire FS suspect.
Notice: get_the_author_email is deprecated since version 2.8! Use get_the_author_meta('email') instead. in /home/knmngmprl21d/public_html/blogs/wp-includes/functions.php on line 3467
on June 20, 2016 at 7:20 am
Permalink
[...] announced something inferior to what #linux had for years and media goes wild http://dtrace.org/blogs/ahl/2016/06/19/apfs-part1/ Like this:Like [...]
Notice: get_the_author_email is deprecated since version 2.8! Use get_the_author_meta('email') instead. in /home/knmngmprl21d/public_html/blogs/wp-includes/functions.php on line 3467
on June 20, 2016 at 1:30 pm
Permalink
The statement at the beginning of the article that “APFS, the Apple File System, relies on Core Storage functionality” is not accurate. Apple File System does not use any functionality or code whatsoever from CoreStorage. Apple File System was developed completely from scratch.
Notice: get_the_author_email is deprecated since version 2.8! Use get_the_author_meta('email') instead. in /home/knmngmprl21d/public_html/blogs/wp-includes/functions.php on line 3467
on June 20, 2016 at 1:59 pm
Permalink
Hey Dominic, thanks for the comment. That was the my understanding after our conversation at WWDC, my apologies for getting it wrong. I’ll make the appropriate update. Was I wide of the mark on anything else?
Notice: get_the_author_email is deprecated since version 2.8! Use get_the_author_meta('email') instead. in /home/knmngmprl21d/public_html/blogs/wp-includes/functions.php on line 3467
on June 20, 2016 at 4:10 pm
Permalink
Thanks for updating it. Everything else seems correct. Also, thanks for the even handed review.
Notice: get_the_author_email is deprecated since version 2.8! Use get_the_author_meta('email') instead. in /home/knmngmprl21d/public_html/blogs/wp-includes/functions.php on line 3467
on June 20, 2016 at 10:36 pm
Permalink
[...] APFS in Detail: Overview Apple announced a new file system that will make its way into all of its OS variants (macOS, tvOS, iOS, watchOS) in the coming years. Media coverage to this point has been mostly breathless elongations of Apple’s developer documentation. With a dearth of detail I decided to attend the presentation and Q&A with the APFS team at WWDC. Dominic Giampaolo and Eric Tamura, two members of the APFS team, gave an overview to a packed room; along with other members of the team, they patiently answered questions later in the day. With those data points and some first hand usage I wanted to provide an overview and analysis both as a user of Apple-ecosystem products and as a long-time operating system and file system developer. [...]
Notice: get_the_author_email is deprecated since version 2.8! Use get_the_author_meta('email') instead. in /home/knmngmprl21d/public_html/blogs/wp-includes/functions.php on line 3467
on June 21, 2016 at 12:53 am
Permalink
[...] http://dtrace.org/blogs/ahl/2016/06/19/apfs-part1/ [...]
Notice: get_the_author_email is deprecated since version 2.8! Use get_the_author_meta('email') instead. in /home/knmngmprl21d/public_html/blogs/wp-includes/functions.php on line 3467
on June 21, 2016 at 12:55 am
Permalink
[...] series of posts based on his talks with Apple engineers Dominic Giampaolo and Eric Tamura at WWDC. Overview (Hacker News): APFS, the Apple File System, was itself started in 2014 with Dominic as its lead [...]
Notice: get_the_author_email is deprecated since version 2.8! Use get_the_author_meta('email') instead. in /home/knmngmprl21d/public_html/blogs/wp-includes/functions.php on line 3467
on June 21, 2016 at 3:01 pm
Permalink
[...] ) kümmert, dann könnte eventuell der Mac-Finder schneller kopieren. Weiterführend: dtrace.org.Sharen [...]
Notice: get_the_author_email is deprecated since version 2.8! Use get_the_author_meta('email') instead. in /home/knmngmprl21d/public_html/blogs/wp-includes/functions.php on line 3467
on June 21, 2016 at 8:25 pm
Permalink
[...] From Leventhal’s blog: [...]
Notice: get_the_author_email is deprecated since version 2.8! Use get_the_author_meta('email') instead. in /home/knmngmprl21d/public_html/blogs/wp-includes/functions.php on line 3467
on June 21, 2016 at 8:36 pm
Permalink
[...] From Leventhal’s blog: [...]
Notice: get_the_author_email is deprecated since version 2.8! Use get_the_author_meta('email') instead. in /home/knmngmprl21d/public_html/blogs/wp-includes/functions.php on line 3467
on June 21, 2016 at 8:40 pm
Permalink
[...] Leventhal’s blog: APFS first and foremost pays down the unsustainable technical debt that Apple has been carrying in [...]
Notice: get_the_author_email is deprecated since version 2.8! Use get_the_author_meta('email') instead. in /home/knmngmprl21d/public_html/blogs/wp-includes/functions.php on line 3467
on June 21, 2016 at 9:13 pm
Permalink
[...] From Leventhal's blog: [...]
Notice: get_the_author_email is deprecated since version 2.8! Use get_the_author_meta('email') instead. in /home/knmngmprl21d/public_html/blogs/wp-includes/functions.php on line 3467
on June 22, 2016 at 12:14 am
Permalink
One bit flip corruption in a typical file isn’t a big deal. One bit flip corruption in a compressed file can be game over. And if that bit is in the core OS, do not pass go, do not collect $200.
Notice: get_the_author_email is deprecated since version 2.8! Use get_the_author_meta('email') instead. in /home/knmngmprl21d/public_html/blogs/wp-includes/functions.php on line 3467
on June 22, 2016 at 10:00 am
Permalink
Thank you Dominic, Eric, and the team. This has been on the top of my feature request list for many years. It will be a breathe of fresh air. Looking forward to it.
Notice: get_the_author_email is deprecated since version 2.8! Use get_the_author_meta('email') instead. in /home/knmngmprl21d/public_html/blogs/wp-includes/functions.php on line 3467
on June 22, 2016 at 9:21 pm
Permalink
[...] TV had where you can watch two footy games at once but that he never really used. There’s a new file system that is super frickin’ cool and could eliminate data loss forever but is not really even [...]
Notice: get_the_author_email is deprecated since version 2.8! Use get_the_author_meta('email') instead. in /home/knmngmprl21d/public_html/blogs/wp-includes/functions.php on line 3467
on June 22, 2016 at 9:22 pm
Permalink
[...] TV had where you can watch two footy games at once but that he never really used. There’s a new file system that is super frickin’ cool and could eliminate data loss forever but is not really even [...]
Notice: get_the_author_email is deprecated since version 2.8! Use get_the_author_meta('email') instead. in /home/knmngmprl21d/public_html/blogs/wp-includes/functions.php on line 3467
on June 22, 2016 at 9:25 pm
Permalink
[...] TV had where you can watch two footy games at once but that he never really used. There’s a new file system that is super frickin’ cool and could eliminate data loss forever but is not really even [...]
Notice: get_the_author_email is deprecated since version 2.8! Use get_the_author_meta('email') instead. in /home/knmngmprl21d/public_html/blogs/wp-includes/functions.php on line 3467
on June 22, 2016 at 9:37 pm
Permalink
[...] had where you can watch two footy games at once but that he never really used. There’s a new file system that is super frickin’ cool and could eliminate data loss forever but is not really even [...]
Notice: get_the_author_email is deprecated since version 2.8! Use get_the_author_meta('email') instead. in /home/knmngmprl21d/public_html/blogs/wp-includes/functions.php on line 3467
on June 22, 2016 at 9:43 pm
Permalink
[...] TV had where you can watch two footy games at once but that he never really used. There’s a new file system that is super frickin’ cool and could eliminate data loss forever but is not really even [...]
Notice: get_the_author_email is deprecated since version 2.8! Use get_the_author_meta('email') instead. in /home/knmngmprl21d/public_html/blogs/wp-includes/functions.php on line 3467
on June 22, 2016 at 9:58 pm
Permalink
[...] had where you can watch two footy games at once but that he never really used. There’s a new file system that is super frickin’ cool and could eliminate data loss forever but is not really even [...]
Notice: get_the_author_email is deprecated since version 2.8! Use get_the_author_meta('email') instead. in /home/knmngmprl21d/public_html/blogs/wp-includes/functions.php on line 3467
on June 22, 2016 at 11:12 pm
Permalink
[...] TV had where you can watch two footy games at once but that he never really used. There’s a new file system that is super frickin’ cool and could eliminate data loss forever but is not really even [...]
Notice: get_the_author_email is deprecated since version 2.8! Use get_the_author_meta('email') instead. in /home/knmngmprl21d/public_html/blogs/wp-includes/functions.php on line 3467
on June 23, 2016 at 2:21 am
Permalink
[...] TV had where you can watch two footy games at once but that he never really used. There’s a new file system that is super frickin’ cool and could eliminate data loss forever but is not really even [...]
Notice: get_the_author_email is deprecated since version 2.8! Use get_the_author_meta('email') instead. in /home/knmngmprl21d/public_html/blogs/wp-includes/functions.php on line 3467
on June 23, 2016 at 3:03 am
Permalink
[...] Adam Leventhal’s blog » APFS in Detail: Overview [...]
Notice: get_the_author_email is deprecated since version 2.8! Use get_the_author_meta('email') instead. in /home/knmngmprl21d/public_html/blogs/wp-includes/functions.php on line 3467
on June 23, 2016 at 6:48 am
Permalink
[...] TV had where you can watch two footy games at once but that he never really used. There’s a new file system that is super frickin’ cool and could eliminate data loss forever but is not really even [...]
Notice: get_the_author_email is deprecated since version 2.8! Use get_the_author_meta('email') instead. in /home/knmngmprl21d/public_html/blogs/wp-includes/functions.php on line 3467
on June 23, 2016 at 12:40 pm
Permalink
[...] APFS in Detail: Overview APFS in Detail: Encryption, Snapshots, and Backup APFS in Detail: Space Efficiency and Clones APFS [...]
Notice: get_the_author_email is deprecated since version 2.8! Use get_the_author_meta('email') instead. in /home/knmngmprl21d/public_html/blogs/wp-includes/functions.php on line 3467
on June 26, 2016 at 8:17 am
Permalink
[...] Basics [...]
Notice: get_the_author_email is deprecated since version 2.8! Use get_the_author_meta('email') instead. in /home/knmngmprl21d/public_html/blogs/wp-includes/functions.php on line 3467
on June 26, 2016 at 1:26 pm
Permalink
[...] This article was originally published on Adam Leventhal’s blog in multiple parts. [...]
Notice: get_the_author_email is deprecated since version 2.8! Use get_the_author_meta('email') instead. in /home/knmngmprl21d/public_html/blogs/wp-includes/functions.php on line 3467
on June 26, 2016 at 1:33 pm
Permalink
[...] This article was originally published on Adam Leventhal’s blog in multiple parts. [...]
Notice: get_the_author_email is deprecated since version 2.8! Use get_the_author_meta('email') instead. in /home/knmngmprl21d/public_html/blogs/wp-includes/functions.php on line 3467
on June 26, 2016 at 4:18 pm
Permalink
[...] series of posts covers APFS, Apple’s new filesystem announced at WWDC 2016. See the first post for the table of [...]
Notice: get_the_author_email is deprecated since version 2.8! Use get_the_author_meta('email') instead. in /home/knmngmprl21d/public_html/blogs/wp-includes/functions.php on line 3467
on June 26, 2016 at 6:39 pm
Permalink
[...] Two hours or so of WWDC keynoting and Tim Cook didn’t mention a new file system once? Andrew Cunningham This article was originally published on Adam Leventhal’s blog in multiple parts. [...]
Notice: get_the_author_email is deprecated since version 2.8! Use get_the_author_meta('email') instead. in /home/knmngmprl21d/public_html/blogs/wp-includes/functions.php on line 3467
on June 27, 2016 at 2:05 am
Permalink
[...] APFS in Detail: Overview [...]
Notice: get_the_author_email is deprecated since version 2.8! Use get_the_author_meta('email') instead. in /home/knmngmprl21d/public_html/blogs/wp-includes/functions.php on line 3467
on June 27, 2016 at 6:17 am
Permalink
[...] This article was originally published on Adam Leventhal’s blog in multiple parts. [...]
Notice: get_the_author_email is deprecated since version 2.8! Use get_the_author_meta('email') instead. in /home/knmngmprl21d/public_html/blogs/wp-includes/functions.php on line 3467
on June 27, 2016 at 7:02 am
Permalink
What is the maximum filesize / volume size for APFS?
Notice: get_the_author_email is deprecated since version 2.8! Use get_the_author_meta('email') instead. in /home/knmngmprl21d/public_html/blogs/wp-includes/functions.php on line 3467
on June 27, 2016 at 8:03 am
Permalink
[...] This article was originally published on Adam Leventhal’s blog in multiple parts. [...]
Notice: get_the_author_email is deprecated since version 2.8! Use get_the_author_meta('email') instead. in /home/knmngmprl21d/public_html/blogs/wp-includes/functions.php on line 3467
on June 28, 2016 at 3:58 pm
Permalink
[...] APFS, el nuevo sistema de archivos de Apple, en detalle — Parte 1 [ENG] Apple anunció hace unos días su nuevo sistema de archivos, APFS, sistema que en el futuro va a ir integrado en todos sus sistemas operativos. Después de su presentación, Dominic Giampaolo y Eric Tamura, dos miembros del equipo de diseño, junto con otros miembros del equipo respondieron pacientemente a nuestras preguntas al final del día. Con estos datos y con el uso de primera mano de este sistema de archivos, voy a hacer un análisis de APFS como usuario y com diseñador de sistemas de archivos. etiquetas: apfs, sistema de archivos, apple, adam leventhal, dominic giampaolo usuarios: 1 anónimos: 0 negativos: 0 compartir: sin comentarios SysDevs karma: 14 (adsbygoogle = window.adsbygoogle || []).push({}); (adsbygoogle = window.adsbygoogle || []).push({}); [...]
Notice: get_the_author_email is deprecated since version 2.8! Use get_the_author_meta('email') instead. in /home/knmngmprl21d/public_html/blogs/wp-includes/functions.php on line 3467
on June 29, 2016 at 3:21 am
Permalink
[...] a participé à cette session et il en a livré un récit aussi complet que possible sur son blog. Nous avons puisé dans ces observations ainsi que dans les docs d’Apple pour faire le point sur [...]
Notice: get_the_author_email is deprecated since version 2.8! Use get_the_author_meta('email') instead. in /home/knmngmprl21d/public_html/blogs/wp-includes/functions.php on line 3467
on July 8, 2016 at 6:41 pm
Permalink
[...] (an earlier version of this post noted a dependency on Core Storage, but Giampaolo set me straight in this comment). I asked him about looking for inspiration in other modern file systems such as BSD’s [...]
Notice: get_the_author_email is deprecated since version 2.8! Use get_the_author_meta('email') instead. in /home/knmngmprl21d/public_html/blogs/wp-includes/functions.php on line 3467
on July 16, 2016 at 8:22 am
Permalink
[...] Andrew Cunningham This article was originally published on Adam Leventhal’s blog in multiple parts. [...]
Notice: get_the_author_email is deprecated since version 2.8! Use get_the_author_meta('email') instead. in /home/knmngmprl21d/public_html/blogs/wp-includes/functions.php on line 3467
on July 22, 2016 at 4:02 am
Permalink
[...] TV had where you can watch two footy games at once but that he never really used. There’s a new file system that is super frickin’ cool and could eliminate data loss forever but is not really even [...]
Notice: get_the_author_email is deprecated since version 2.8! Use get_the_author_meta('email') instead. in /home/knmngmprl21d/public_html/blogs/wp-includes/functions.php on line 3467
on July 25, 2016 at 10:00 am
Permalink
That’s a nice post. I hope you will have a separate blog post for smart home and can you recommend a smart home builder.
Notice: get_the_author_email is deprecated since version 2.8! Use get_the_author_meta('email') instead. in /home/knmngmprl21d/public_html/blogs/wp-includes/functions.php on line 3467
on August 1, 2016 at 1:11 pm
Permalink
[...] APFS in Detail: Overview [...]