My Sun Storage 7410 perf limits

As part of my role in Fishworks, I push systems to their limits to investigate and solve bottlenecks. Limits can be useful to consider as a possible upper bound of performance – as it shows what the target can do. Finding and properly analyzing these limits is a useful exercise, as I can determine the kernel code or hardware that is the bottleneck, leading to product changes and better overall performance. I thought these results would make for some interesting blog posts if I could explain the setup, describe what the tests were, and include screenshots of these results in action. (Update: I’ve included results from colleagues who have tested in the same manner.)

To summarize the performance limits that I found for a single Sun Storage 7410 head node (our top performing ZFS storage appliance):

All tests are performed on Ethernet (usually 10 GbE) unless otherwise specified (“IB” == InfiniBand).

Like many products, the 7410 will undergo software and hardware updates over time. This page currently has results for:

I should make clear that these are provided as possible upper bounds – these aren’t what to expect for any given workload, unless your workload was similar to what I used for these tests. Click on the results to see details of the workloads used.

These are also the limits that were found with a given farm of clients and JBODs – it’s possible the 7410 could go faster with more clients and more JBODs.

Updated 3-Mar-2009: added CIFS results.

Updated 22-Sep-2009: added column for 7410 Istanbul. Results will be added as they are collected.

Updated 12-Nov-2009: added Cindi’s InfiniBand results.

Print Friendly
Posted on January 9, 2009 at 4:21 pm by Brendan Gregg · Permalink
In: Fishworks · Tagged with: , , , , ,

13 Responses

Subscribe to comments via RSS

  1. Written by Mika
    on January 10, 2009 at 1:18 am

    Would it be possible to have an IOPS benchmark for small reads/writes? I’m thinking mostly about IO in the size of e-mails (-> mbox format).

  2. Written by Erik
    on January 10, 2009 at 8:28 am

    Hey Brendan – love the information. Thanks!
    I’ve noticed it seems you use NFS for most of your tests. Could we expect similar performance over iSCSI? I’m doing some pre-purchase research on the 7410, and in my case, most of my clients would be connecting via iSCSI (over 10GigE).

  3. Written by eli101
    on January 10, 2009 at 8:46 pm

    could you also please post some performance numbers on the 7210.

  4. Written by Guillaume Maillard
    on January 12, 2009 at 8:55 am

    We made a complete review of the 7110, performance numbers are impressive.
    To Mika: we tested small writes too.
    It’s available at
    (in french), but we are translating it to english.

  5. Written by Guillaume Maillard
    on January 27, 2009 at 3:00 am

    If need information about the 7110, the english version of our review is available at

  6. Written by Ceri Davies
    on March 18, 2009 at 8:17 am

    Could you explain the hardware setup behind these numbers. Is this a standard 7140, and can I look this up at a webpage? What is the front end pipe, when is the bak-end pipe (SAS controllers/JBODs), a summary of the bottlenecks (front-end versus back-end..)

  7. Written by Brendan Gregg
    on March 18, 2009 at 10:47 am

    I did – click on each number for a page describing the setup and workload. It is a standard 7410.

  8. Written by John
    on April 17, 2009 at 2:50 am

    What kind of "benchmark" is that? What did you use to benchmark, what were the parameters, how many clients, etc?

  9. Written by Brendan Gregg
    on May 31, 2009 at 3:19 pm

    @John, if you click on the links, for each result I explain the workload type, I/O size, number of clients, and numbers of threads per client. The actual benchmark software is very simple, and documented here:
    The previous posts in that series will answer many other questions: did I pay attention to various things, etc:

  10. Written by Joaquim Neves
    on July 16, 2009 at 1:19 am

    Hi Brendan,
    What was the configuration used during your performance limits (7410 with SSD or without SSD)?
    Is there a big difference between the 7410 (16GB RAM + 2xJ4400 without SSD) and the 7410 (64GB RAM with Readzilla Flash + 2x J4400 1x 18GB logzilla Flash)?
    My customer transfers mainly video files in NFS/CIFS.
    Thanks in advance,

  11. Written by Sean Cochrane
    on September 11, 2009 at 11:24 am

    Will you be running these again with the new Istanbul processors?
    thanks for any pointers.

  12. Written by Brendan Gregg
    on September 23, 2009 at 3:14 pm

    @Sean: yes!

  13. [...] created a new column of numbers on my summary post, and I’ll fill out the new numbers as I get [...]

Subscribe to comments via RSS