Local / video stack
20B on 2 × 22 GB.
MiniMax H3 is a 20.11B video-and-audio transformer. It does not fit on one card, so tensor parallelism is not an accelerator here — it is the difference between a usable run and a fan that runs for four minutes.
01 / Model
What has to fit.
A video model is a memory problem before it is a speed problem. The quantisation is what makes 22 GB per card enough.
| Item | Value |
|---|---|
| Parameters / blocks | 20.11 B · 50 blocks |
| Hidden / attention | 5,376 · 56 heads × 128 |
| FFN | 14,336 |
| qkv / out projections | [21504, 5376] / [5376, 7168] |
| fc1 / fc2 | [28672, 5376] / [5376, 14336] |
| Quantisation | W4A8 · int4 (2 per byte) · group 16 · ConvRot 256 · fp8 group scale · fp32 channel scale |
| DiT on disk | 10.96 GiB (fl2va 11.68 GiB) |
| VAE | 16× spatial compression · patch (1,2,2) |
| Runtime | ComfyUI · PyTorch 2.9.1+cu130 · driver 580 · CUDA 13.0 |
02 / Throughput
Single card vs TP2.
Smoke sample: 640×352 × 124 frames × 4 steps, text-to-video with no reference image. Same workflow in both columns.
| Metric | Single card | TP2 |
|---|---|---|
| Per step | ~51 s | ~14 s |
| End to end (4 steps) | 243 s | 82 s |
| Speedup | 1.00× | 2.97× |
| GPU utilisation | GPU0 idle / GPU1 97% | GPU0 57% / GPU1 72% |
| Peak VRAM | GPU1 19,071 MiB | GPU0 11,579 / GPU1 19,681 MiB |
| Peak temperature | GPU1 78 °C (97% for 4 min) | GPU0 67 °C / GPU1 75 °C (1.4 min) |
Also measuredAt 864×480 (124 frames, ref2va): 44.6 s/step on one card, 19.8 s/step with TP2 — 2.25×.
03 / Three versions
1.31×, then 2.25×, then 2.97×.
The sharding was correct in the first version. What was missing was a thread, not a split.
| Version | s/step (864×480) | Speedup | What changed |
|---|---|---|---|
| Single card | 44.6 | 1.00× | Chunked on one card |
| TP2 v1 | 34.0 | 1.31× | Column/row split with a single launch thread |
| TP2 + one thread per rank | 19.8 | 2.25× | Each rank launches its own kernels |
CauseThe W4A8 GEMM blocks its calling host thread, so one launch thread serialised the two cards: measured overlap 5%. One thread per rank lifted it to 91%, with bit-identical output (frame MAE 0.0000).
04 / Probes
The three-row diagnosis.
Two GPUs, one GEMM, three launch strategies. Overlap is the number that separates "sharded" from "parallel".
| Test | dev0 | dev1 | Concurrent | Overlap |
|---|---|---|---|---|
| dense fp16 GEMM | 61.9 | 58.0 | 61.0 | 95% |
| W4A8 GEMM | 103.3 | 96.8 | 194.7 | 5% |
| W4A8 GEMM, one thread per card | 106.5 | 97.1 | 106.8 | 91% |
After the fixPer block: qkv + attention 211 ms, MLP 196 ms, final all-reduce 13 ms — 420 ms per block.
05 / Reproduce
Starting the two-card server.
The device order matters: exposing only one card silently collapses every cuda:0 in the graph onto it.
export CUDA_VISIBLE_DEVICES=1,0 # GPU1 first, so cuda:0 is the compute card
python main.py --listen 127.0.0.1 --port <port> --reserve-vram 1.5 --vram-headroom 0.5 --disable-pinned-memory --fast-disk
python scripts/bench.py examples/smoke_t2v.json smoke # self-contained sample
06 / Caveats
What the speedup is not.
The 2.97× comes from real renders on the workstation, and it is specific to a quantisation, a model and a pair of cards.
Not a general TP result
These numbers come from W4A8 on two modified 22 GB Turing cards at a 250 W limit. Different cards, quantisation or model shape will produce different overlaps.
NUMA is part of it
The cards sit on different NUMA nodes. The 13 ms all-reduce partially pays for that topology; another board would change it.
Weights are not here
The code is MIT, the model weights are not in the repository and follow MiniMax's own licence. The sample is a self-contained text-to-video case.