Run Muse Spark Open locally with MLX, llama.cpp, vLLM or SGLang

This guide is organized by hardware. Check your setup in the hardware calculator first, then sign in on Hugging Face and accept the Muse Model License to get download access.

Pick an approach

Hardware	Approach	Weights
Two or more 512GB Mac Studios	MLX distributed inference	MLX 4-bit
Multi-GPU server (≥700GB total VRAM)	vLLM or SGLang	NVFP4 / FP8 / BF16
One or a few GPUs plus lots of RAM	llama.cpp with CPU offload	GGUF Q2–Q4
None of the above	Hosted API	—

Multi-GPU: vLLM

The command from the official model card:

vllm serve "meta-models/Muse-Spark-Open-816B-A42B"

On a multi-GPU box you’ll usually add tensor parallelism to match your GPU count, for example eight GPUs:

vllm serve "meta-models/Muse-Spark-Open-816B-A42B" --tensor-parallel-size 8

This exposes an OpenAI-compatible API at http://localhost:8000/v1.

Multi-GPU: SGLang

python3 -m sglang.launch_server \
  --model-path "meta-models/Muse-Spark-Open-816B-A42B" \
  --host 0.0.0.0 --port 30000

Meta ships auto-tuned Helion GPU kernels for SGLang, which add kernel-level speedups on NVIDIA and AMD GPUs.

Mac Studio: MLX distributed

Meta and Apple tuned this setup together: two M5 Ultra Mac Studios, each with 512GB of unified memory and 1.2TB/s of bandwidth, pooled into roughly 1TB and run with the latest MLX distributed inference.

Steps:

Install the latest mlx and mlx-lm on both Macs: pip install -U mlx mlx-lm
Connect them directly with Thunderbolt and set up passwordless SSH
Pick an MLX 4-bit conversion on Hugging Face
Launch across both machines with mlx.launch, following Apple’s MLX distributed inference docs

macOS limits how much unified memory the GPU can wire by default. For very large models you’ll usually need to raise it:

sudo sysctl iogpu.wired_limit_mb=480000

The setting resets on reboot. Pick a value that leaves headroom for the OS.

A few GPUs: llama.cpp

Community GGUF quants are on Hugging Face and work with llama.cpp, Ollama, LM Studio and Jan. For MoE models, keep attention on the GPU and push the expert tensors to system RAM:

llama-server -m Muse-Spark-Open-Q2_K.gguf \
  -c 32768 --n-gpu-layers 99 \
  --override-tensor "exps=CPU"

Replace the filename with the quant you actually downloaded. Speed is bound by system RAM bandwidth, which on desktop platforms usually means a few tokens per second.

Docker

docker model run hf.co/meta-models/Muse-Spark-Open-816B-A42B

Inference settings

Meta recommends temperature=1.0 and top_p=1.0. Set reasoning depth (low / medium / high / xhigh) via reasoning_effort or chat_template_kwargs. Lower effort cuts latency a lot.

FAQ

Q.Can a single Mac Studio run Muse Spark Open?

A.A single 512GB Mac Studio can't hold the 4-bit weights (about 490GB) plus runtime overhead, so you'd need an aggressive 3-bit or lower quant with a noticeable quality hit. Meta's demo uses two M5 Ultra Mac Studios.

Q.Can a consumer GPU run it?

A.A single RTX 4090 or 5090 can only run it with heavy CPU offload into a large pool of system RAM. That typically means a few tokens per second at most, which is fine to try out but not for daily use.