System Architecture

MASS is MangoBoost's storage appliance for shared application data and LLM inference caches. It combines NVMe storage nodes, a storage network, and centralized management through the Dashboard and CLI.

Two Storage Services

Service	Purpose	Access
File & Object Storage	Store datasets, checkpoints, and application output	Filesystem mounts or object APIs
KV Cache Store	Retain and share KV cache for reuse during LLM inference	An inference engine integration on each GPU server

File and object storage retains application data under a selected protection policy. A KV cache store holds computed data that an inference engine can regenerate when it is unavailable. These different requirements shape how each service stores and retrieves data.

Data Access

Clients communicate with storage nodes over the storage network. File and object clients access distributed data through their storage client software. GPU servers read and write KV cache pages over NVMe-oF.

Storage management is separate from these reads and writes. Application data does not pass through the Dashboard or CLI.

Management

Administrators use the Dashboard to configure storage, manage access, and view capacity, performance, and health. The mass-ctl CLI connects client machines to the storage they are allowed to use.

See Storage Management for the relationship between storage resources, client access, and monitoring.