The Hyak Supercomputing Research Cluster is part of an integrated, scalable, scientific super-computing infrastructure operated by UW-IT. It includes a high-performance research network, the Hyak compute infrastructure (HPC and GPU clusters), and scientific support services available to consult on in your research workflows. The Science DMZ network supports fast data transfers among these systems and upstream to the rest of campus over the Internet.

Condo Model

The Hyak Supercomputing Research Cluster operates under a condo model common across many universities. In a condo model, the cluster consists of contributed slices from various groups across campus, and the Hyak team provides the core infrastructure (e.g., networking, storage, and support staff). Faculty, researchers, and some associates with affiliated organizations may purchase slices. Slice owners, members of their research group, and any collaborators get access to resources equivalent to your contributed slices on-demand.

As a member of the Hyak community, you also receive access to all contributed slices from other entities, subject to their availability (i.e., if the contributors or the resources aren't actively using them). This is referred to as the "checkpoint" partition due to the lack of job run-time guarantees. Checkpoint provides access to substantial resources beyond what you contribute and is the benefit of joining the Hyak ecosystem rather than buying the same hardware and operating your own server.

The total cost for Hyak compute slices can be broken down into the sum of the following two components.

1. Slice Hardware Costs
2. Annual Slot Fees

Slice Hardware Configurations

We currently offer HPC slices and GPU slices. Our current slice offering specifications are based on market price and availability. To request detailed Hyak slice pricing and information, please complete the Hyak Slice Pricing and Purchase Information Request Form. Current configurations:

Next-gen HPC slices (AMD):

• 256GB: $3,892.00 (pending delivery and installation). ORDER TODAY! Limited inventory remaining on current order.

GPU slices (Nvidia):

• L40S: 32-cores, 384GB, 2 x L40S GPUs (w/48GB per card) last quoted $17,889.75 (pending order and delivery).
• H200: 32-cores, 384GB, 2 x H200 (w/141GB per card) $58,910.25 (pending order and delivery; requires at least four slice purchases before an order can be made)

Additional RAM may NOT be added to a slice after it is purchased; however additional slices can be added to the equivalent desired RAM.

Buy Slices

Storage:

• All slices include 1 TB of storagespace and a 1 million file count limit of shared group storage (i.e., gscratch) accessible from every node. You may purchase supplemental storage at a monthly rate of $10.00 per TB per month (with a limit of 1 million inodes per TB). Supplemental storage subscripts are subject to indirect costs. Storage prices are subject to revision annually. To request supplemental storage, email help@uw.edu with "Hyak gscratch" in the subject line.
• Additional shared storage called "scrubbed" is available for short-term use, but will be automatically deleted if not accessed for several weeks.

General FAQ:

• All hardware is procured at cost (market value with substantial university negotiated bulk discounts) and no sales tax or university overhead applied.
• We reserve the 2nd Tuesday of every month for cluster maintenance.
• Hyak slices are supported for a minimum guaranteed lifetime of 4 or 5 years depending on the slice type. Beyond 4 or 5 years all slices are continued to be made available subject to hardware viability and the sponsoring entity still having capacity. Self-sponsored slices have an on-going annual cost, this means slice life is reviewed on a yearly basis subject to the lab's willingness to continue, hardware viability, and overall cluster capacity (see below for more about Sponsored vs. Self-sposored slices). Historically, slice service life has been 6 years on average. However, past performance is not a guarantee of future experiences.

• HPC Slices:

• All slices are standardized on AMD EPYC 9654 CPUs ("Genoa").
• A physical server (or node) has 192-cores and >1.5TB of memory packaged in a single box. This is in turn sub-divided into 6 equal "slices" that are resources of compute units that are sold to researchers.
• Each node has 1.5TB or more of local NVME SSD disk storage. This is non-persistent storage and is cleared after a job ends. Data must be copied to and from local SSD before and after each job to utilize this.
• Nodes are identically configured.
• Any jobs requiring multiple nodes should be prepared to be independent computations (i.e., "embarassingly parallel") or make use of message passing libraries (e.g., OpenMPI) to scale across multiple nodes simultaneously.

• GPU Slices:

• All slices are standardized on AMD EPYC 9534 CPUs ("Genoa"). We are on the NVIDIA "Ada" and "Hopper" generation of GPUs.
• 4 x GPU slices constitute a single physical server (or node). It is a single box with 128-cores, 1.5TB of memory, and 8 x GPUs of the same type. They are sold in resource slices to make this a more tractable cost for labs with more modest GPU needs.
• Any jobs requiring more than 8 x GPUs of the same type should be prepared to make use of message passing libraries (e.g., PyTorch Lightning) to scale across multiple servers. Any job up to the equivalent of 4 x GPU slices (i.e., 8 x GPU cards) can be run on the same physical machine and therefore scale easily without much further modification to the codebase.

Buy Slices

Annual Costs

In addition to the hardware purchase costs above, Hyak slices may also be subject to annual maintenance fees (called slot fees) charged to sponsoring entities (listed below). Annual slot fees could be up to $800 for a HPC slice (1 slot), up to $1,200 for a L40S GPU slice (1.5 slots), and up to $1,600 for an H200 slice (2 slots). Prospective Hyak slice owners that are not associated with a sponsor are responsible for annual slot fees, which amount to $1,750 for HPC slices (1 slot), $2,625 for L40S GPU slices (1.5 slots), and $3,500 for H200 slices (2 slots). The difference in the number of slots occupied by a slice depends on the energy consumption of the slice, and GPU slices consume more energy.

The Hyak sponsor chooses if and how to recuperate annual slot fees. Please consult your College IT director or Department Chair to determine the slot fees you can expect to pay annually.

Sponsors:

• UW Seattle
• College of Arts & Sciences
• College of Engineering
• College of the Environment
• School of Medicine
• UW Bothell
• UW Tacoma