Skip to main content

Supermicro® for Lustre on OpenZFS

Lustre is the Parallel file system of choice for High Performance Computing (HPC) and large file applications. To meet the capacity and throughput requirements of HPC workloads, Lustre has traditionally required adoption of custom proprietary storage products leading to vendor lock-in and reduced innovation. The evolution of software defined high availability platforms like OpenZFS running on Linux now make it possible to meet the performance requirements of HPC on open industry standard x86 platforms, reducing storage costs by up to 90%. Supermicro, along with some of the world's leading HPC integrators offers a reference architecture for Lustre on OpenZFS with Supermicro's industry leading hardware, software and services infrastructure, for use with Lustre Community Edition. Lustre ready hardware is a reference architecture that has been created and thoroughly tested by Supermicro's team of expert Lustre engineers. It provides customers with pre-tested and validated Lustre configurations reducing complexity and risk, controlling acquisition and operational costs and accelerating solution time to market. This solution is ideal for organizations that are able to self-support such as universities, National Labs, and others with such capabilities.

Scalable Building Block Architecture
Lustre architecture diagram

Highlights

  • Mature, fully tested and validated configurations with Lustre Community Edition for a Software Defined architecture
  • Simple building block architecture delivers predictable scaling to specific requirements
  • Supports large datasets and high performance storage environments
  • Solution building blocks includes systems, networking rack integration, and hardware support services
  • Industry leading storage density with fully redundant High Availability design
  • Highest Density storage using 90 Bay 4U design and the latest high performance SAS3 interconnect

    Open Industry standard server hardware for Software Defined architecture

    • Reduce storage costs by up to 90% with Community Lustre on OpenZFS using Supermicro’s robust industry proven hardware.
    • Software defined storage targets use cost effective high density JBOD enclosures configured with OpenZFS RAIDZ2 targets.
    • Flexible architecture providing high performance, redundancy and serviceability
    • High bandwidth utilizing 12Gb/s SAS3 and the latest Ethernet, InfiniBand and Omni-Path Interconnects up to 100Gb/s

    Validated Lustre 2.10 Hardware

      Lustre 6U Lustre 8U Lustre 12U
    Cluster Role Metadata and Mgmt Server Pod (MDS/MGS) Object Storage Server Pod OSS – 2x1 Object Storage Server Pod OSS – 2x2
    Description Single MDS Pod can support hundreds of OSS Pods and thousands of clients

    Multiple Pods can be added to meet capacity requirements

    Linear scaling of throughput and capacity

    Multiple Pods can be added to meet capacity requirements

    Linear scaling of throughput and capacity

    2X storage capacity/OSS

    Capacity/Pod 4TB raw capacity, 2TB usable 720TB raw capacity, 576TB usable 1.4PB raw capacity, 1.1PB usable
    Form factor

    6U Configuration

    Redundant Metadata servers SSG-6028R-LUSTRE-MDS1

    Shared SAS3 JBOD (6 SSD, 8 lanes per node, SSG-J2004-LUSTRE-MDT1)

    8U Configuration

    Redundant Object Storage Servers SSG-6028R-LUSTRE-OSS1

    1X 90-bay Object Storage Target (OST) (16x lanes per node) SSG-J4000-LUSTRE-OST2

    12U Configuration

    Redundant Object Storage Servers SSG-6028R-LUSTRE-OSS1

    2x 90-bay Object Storage Target (OST) (16x lanes per node) SSG-J4000-LUSTRE-OST2

    Technical Specifications
    Metadata Server (MDS/MGS)
    Server SuperStorage Server (SSG-6028R-LUSTRE-MDS1)
    CPU/Memory Dual E5-2667v4, 3.2 GHz, 8 core/ 128GB
    Fabric Customer defined Networking, options for Omnipath, Infiniband and Ethernet
    Storage Interconnect 12Gb/s (SAS 3) IT mode controller, 1x per server node
    Metadata Target (SSG-927R-E2CJB) 2U 24 2.5" Drive JBOD with Hot-swap I/O Modules
    Object Storage Server (OSS)
    Server SuperStorage Server (SSG-6028R-LUSTRE-OSS1)
    CPU/Memory Dual E5-2667v4, 3.2 GHz, 8 core/ 128GB
    Fabric Customer defined Networking, options for Omnipath, Infiniband and Ethernet
    Storage Interconnect 12Gb/s (SAS 3) redundant IT mode controllers
    Object Data Target SC946ED-R2KJBOD - 4U, 90 3.5" drive JBOD(s) with shared SAS 3 redundant expander
    Management Server
    Software Lustre Community Edition (separate download), Supermicro Server Manager
    Server Ultra (1U, 2P)
    SYS-6018U-TR4+ / SYS-6018U-TR4T+ / SYS-6018U-TRT+ / SYS-6018U-TRTP+

    Validated Software

    • Lustre community edition (currently v2.10.0) and OpenZFS on Linux (v0.7.0) and CentOS Linux (7.3.1611) (separate customer downloads)
    • Please note this hardware has also been validated on the following Intel Enterprise Edition Lustre with IML management, which is migrating to the community version: 2.2.0.2, 2.3.0.0, 3.0.0.0, and 3.1
    • Supermicro's server management (SSM)
    • Remotely manage hardware and OS configuration & health manages power consumption of nodes in cluster
    • Manages firmware provisioning through BMC/IPMI Execute commands on multiple target systems in parallel

    Rack Integration and Proof of Concept Services

    • Consulting service on solution architecture for unique designs
    • Test equipment including servers, storage, network switches, cabinets, PDUs, and power meters
    • Additional contract services available for hardware installation, configuration and custom testing
    • Remotely manage hardware and OS configuration & health

    Support

    • Flexible and customizable hardware service level agreements (SLA): 4-Hour and Next Business Day Onsite Service
    • Media Retention, Onsite deployment, Preventative Maintenance, and Remote Management Service options available
    • Live, domestic call center for hardware break/fix response