My Sun Storage 7410 perf limits

As part of my role in Fishworks, I push systems to their limits to investigate and solve bottlenecks. Limits can be useful to consider as a possible upper bound of performance – as it shows what the target can do. Finding and properly analyzing these limits is a useful exercise, as I can determine the kernel code or hardware that is the bottleneck, leading to product changes and better overall performance. I thought these results would make for some interesting blog posts if I could explain the setup, describe what the tests were, and include screenshots of these results in action. (Update: I’ve included results from colleagues who have tested in the same manner.)

To summarize the performance limits that I found for a single Sun Storage 7410 head node (our top performing ZFS storage appliance):

All tests are performed on Ethernet (usually 10 GbE) unless otherwise specified (“IB” == InfiniBand).

Like many products, the 7410 will undergo software and hardware updates over time. This page currently has results for:

I should make clear that these are provided as possible upper bounds – these aren’t what to expect for any given workload, unless your workload was similar to what I used for these tests. Click on the results to see details of the workloads used.

These are also the limits that were found with a given farm of clients and JBODs – it’s possible the 7410 could go faster with more clients and more JBODs.

Updated 3-Mar-2009: added CIFS results.

Updated 22-Sep-2009: added column for 7410 Istanbul. Results will be added as they are collected.

Updated 12-Nov-2009: added Cindi’s InfiniBand results.

Print Friendly
Posted on January 9, 2009 at 4:21 pm by Brendan Gregg · Permalink
In: Fishworks · Tagged with: , , , , ,

13 Responses

Subscribe to comments via RSS

  1. Written by Mika
    on January 10, 2009 at 1:18 am
    Permalink

    Would it be possible to have an IOPS benchmark for small reads/writes? I’m thinking mostly about IO in the size of e-mails (-> mbox format).

  2. Written by Erik
    on January 10, 2009 at 8:28 am
    Permalink

    Hey Brendan – love the information. Thanks!
    I’ve noticed it seems you use NFS for most of your tests. Could we expect similar performance over iSCSI? I’m doing some pre-purchase research on the 7410, and in my case, most of my clients would be connecting via iSCSI (over 10GigE).
    Thanks!

  3. Written by eli101
    on January 10, 2009 at 8:46 pm
    Permalink

    could you also please post some performance numbers on the 7210.
    Thanks.

  4. Written by Guillaume Maillard
    on January 12, 2009 at 8:55 am
    Permalink

    We made a complete review of the 7110, performance numbers are impressive.
    To Mika: we tested small writes too.
    It’s available at http://www.ilm-informatique.fr/review/fishworks_sun_fr/
    (in french), but we are translating it to english.
    Regards,
    Guillaume

  5. Written by Guillaume Maillard
    on January 27, 2009 at 3:00 am
    Permalink

    If need information about the 7110, the english version of our review is available at http://www.ilm-informatique.fr/review/fishworks_sun_com/

  6. Written by Ceri Davies
    on March 18, 2009 at 8:17 am
    Permalink

    Could you explain the hardware setup behind these numbers. Is this a standard 7140, and can I look this up at a webpage? What is the front end pipe, when is the bak-end pipe (SAS controllers/JBODs), a summary of the bottlenecks (front-end versus back-end..)

  7. Written by Brendan Gregg
    on March 18, 2009 at 10:47 am
    Permalink

    Ceri,
    I did – click on each number for a page describing the setup and workload. It is a standard 7410.

  8. Written by John
    on April 17, 2009 at 2:50 am
    Permalink

    What kind of "benchmark" is that? What did you use to benchmark, what were the parameters, how many clients, etc?

  9. Written by Brendan Gregg
    on May 31, 2009 at 3:19 pm
    Permalink

    @John, if you click on the links, for each result I explain the workload type, I/O size, number of clients, and numbers of threads per client. The actual benchmark software is very simple, and documented here:
    http://blogs.sun.com/brendan/entry/performance_testing_the_7000_series3
    The previous posts in that series will answer many other questions: did I pay attention to various things, etc:
    http://blogs.sun.com/brendan/entry/performance_testing_the_7000_series1
    http://blogs.sun.com/brendan/entry/performance_testing_the_7000_series2

  10. Written by Joaquim Neves
    on July 16, 2009 at 1:19 am
    Permalink

    Hi Brendan,
    What was the configuration used during your performance limits (7410 with SSD or without SSD)?
    Is there a big difference between the 7410 (16GB RAM + 2xJ4400 without SSD) and the 7410 (64GB RAM with Readzilla Flash + 2x J4400 1x 18GB logzilla Flash)?
    My customer transfers mainly video files in NFS/CIFS.
    Thanks in advance,
    JNV

  11. Written by Sean Cochrane
    on September 11, 2009 at 11:24 am
    Permalink

    Brendan,
    Will you be running these again with the new Istanbul processors?
    thanks for any pointers.
    Sean

  12. Written by Brendan Gregg
    on September 23, 2009 at 3:14 pm
    Permalink

    @Sean: yes!

  13. [...] created a new column of numbers on my summary post, and I’ll fill out the new numbers as I get [...]

Subscribe to comments via RSS