IO500

Overview

IO500 is a benchmark suite for evaluating both bandwidth-heavy and metadata-heavy HPC storage workloads. It combines IOR, mdtest, and find workloads into a standardized 12-phase run and scoring model. This broader coverage makes it useful for MASS acceptance testing, comparison runs, and repeatable regression checks because a single run shows both data-transfer and metadata behavior.

The suite uses three benchmark tools, each covering a different part of storage performance:

• IOR measures data bandwidth by reading and writing files with parallel processes. IO500 uses ior-easy for large, sequential transfers and ior-hard for smaller, more demanding access patterns.
• mdtest measures metadata operation rates by creating, finding, and removing large numbers of files and directories. Its easy and hard phases show how the filesystem handles different directory layouts and contention levels.
• find measures parallel namespace traversal. IO500 uses the bundled pfind tool to walk the benchmark directory tree and locate entries across multiple MPI ranks.

The mass-client package includes the complete IO500 toolchain and a runner configured for MASS. You do not need to download or build the upstream source tree to run the bundled benchmark.

Official Resources

• IO500 homepage
• IO500 benchmark and submission guidance
• IO500 rules
• IO500 source repository

Prerequisites

• Install the mass-client package on every participating test host.
• Mount the target MASS volume on the host where you launch the runner. The runner uses the mount specified by DFS_PATH to identify the benchmark target; remote ranks access that same target through the bundled client runtime.
• For a multi-host run, configure passwordless SSH and passwordless sudo for the launch user on every participating host. Alternatively, set RSH_USER=root and configure passwordless root SSH.

note

For Storage resource provisioning, see Managing Storage. For mass-client installation instructions, see CLI Reference.

Running IO500 on MASS

MPI rank

MPI rank refers to a process that sends I/O requests. NPROCS specifies the number of ranks to run on each host. For a multi-host test, the total rank count is NPROCS multiplied by the number of hosts.

For example, HOSTS=client01,client02 NPROCS=16 runs 16 ranks on each host and 32 ranks in total.

1. Run a short single-host validation test with the default 30-second stonewall time:

NPROCS=16 DFS_PATH=/mnt/poc-home \
  /opt/boostx-sds/io500/scripts/io500.sh

2. To measure aggregate performance across multiple clients, provide a comma-separated host list. NPROCS remains the number of ranks per host:

HOSTS=client01,client02,client03,client04 NPROCS=16 DFS_PATH=/mnt/poc-home \
  /opt/boostx-sds/io500/scripts/io500.sh

3. After the validation run succeeds, use a stonewall time of at least 300 seconds for a run that meets the IO500 validity requirement:

HOSTS=client01,client02,client03,client04 NPROCS=16 \
  DFS_PATH=/mnt/poc-home STONEWALL=300 \
  /opt/boostx-sds/io500/scripts/io500.sh

If the script is started as a non-root user, it re-executes itself with sudo -E. By default, remote processes use the invoking user's passwordless SSH connection and passwordless sudo.

Configuration

The runner is configured through environment variables. NPROCS is the only mandatory variable; the script exits with an error if it is not set. All other variables are optional and use the defaults shown below when omitted.

Mandatory Environment Variable

Variable	Description	Default
NPROCS	Number of MPI ranks to run on each host. The total rank count is NPROCS multiplied by the number of entries in HOSTS.	None

Optional Environment Variables

Variable	Description	Default
HOSTS	Comma-separated hosts for a multi-host run. Every host must have the same MASS client installation and meet the remote access prerequisites.	localhost
DFS_PATH	Mounted MASS path used to identify the volume to benchmark.	/mnt/poc-home
RSH_USER	SSH user for launching remote processes. The user needs passwordless SSH and passwordless sudo on every host. Set it to root to use passwordless root SSH instead.	User who invoked the script, or root when already running as root
STONEWALL	Write-phase stonewall time in seconds. Values below 300 are suitable for validation but are marked invalid by the IO500 rules.	30
IOR_EASY_BLOCK	Amount of data assigned to each rank in the ior-easy phase.	64g
IOR_HARD_SEGS	Number of segments assigned to each rank in the ior-hard phase.	2000000
MDTEST_EASY_N	Number of files assigned to each rank in the mdtest-easy phase.	100000
MDTEST_HARD_N	Number of files assigned to each rank in the mdtest-hard phase.	50000
FIND_NPROC	Parallelism used by the find phase.	Total MPI rank count
RESULTDIR	Directory where IO500 writes its results. This must be a local path available on the rank-0 host.	/tmp/io500-results
CONFIG	IO500 configuration template. Copy the bundled template to customize a workload, and retain any placeholders that the runner should fill. The run fails if this file is not readable.	config.io500.ini next to the runner

For example, the following changes the workload sizes and result location:

NPROCS=32 STONEWALL=300 IOR_EASY_BLOCK=128g IOR_HARD_SEGS=4000000 \
  MDTEST_EASY_N=200000 MDTEST_HARD_N=100000 \
  RESULTDIR=/tmp/io500-production \
  /opt/boostx-sds/io500/scripts/io500.sh

Notes

• The default STONEWALL=30 is intended for a quick validation run. IO500 marks a run with a stonewall time shorter than 300 seconds as invalid, although the measured phase values remain usable for troubleshooting and tuning.
• Record the MASS client version, workload variables, participating hosts, and client topology with each result so that later comparisons remain meaningful.
• With OpenMPI 4.1.x, teardown can report MPI_ERR_COMM immediately after the [SCORE ] line. In that case, all phase results and the score have already been printed, but result.txt might be missing from RESULTDIR.