Skip to main content
Version: 0.0.14

Mooncake

What It Is​

Mooncake is an LLM serving system. In a MASS environment, the practical question is not whether MASS replaces Mooncake's internal runtime components, but where Mooncake should place the shared files it needs to serve models reliably.

Why It Matters for MASS​

  • Shared storage for model weights, tokenizers, prompt corpora, evaluation datasets, logs, and exported traces
  • Centralized data access when several serving nodes need the same artifacts
  • Persistent storage for large serving assets that should outlive any single host

How To Use with MASS​

  1. Create a MASS volume sized for the model artifacts and other persistent files the deployment will share.
  2. Mount that volume on each serving node, or expose it through the platform layer that launches Mooncake.
  3. Store shared model repositories, configuration files, prompt datasets, and output artifacts on the MASS path.
  4. Point Mooncake configuration to those shared paths so every serving node resolves the same assets from the same location.
  5. Keep node-local hot caches or temporary scratch areas on the local media that the upstream Mooncake deployment recommends; use MASS for the shared durable layer around the serving system.

Notes​

  • MASS is best used for shared persistent assets around the deployment. If a Mooncake component expects host-local memory or local NVMe for its hottest cache tier, keep that tier local.
  • Validate model load time and steady-state throughput with the actual model sizes and client concurrency you expect in production.
  • When the deployment runs in containers or Kubernetes, mount the MASS-backed volume into the pods and keep the in-container paths consistent across replicas.