Norwin YuAI usage research
中文 AI usage research

Local / video stack

20B on 2 × 22 GB.

MiniMax H3 is a 20.11B video-and-audio transformer. It does not fit on one card, so tensor parallelism is not an accelerator here — it is the difference between a usable run and a fan that runs for four minutes.

20.11B · 50 blocksW4A8 · 10.96 GiBTP2 · 2.97×Measured / Sep–Nov 2026

01 / Model

What has to fit.

A video model is a memory problem before it is a speed problem. The quantisation is what makes 22 GB per card enough.

MiniMax H3 model parameters
ItemValue
Parameters / blocks20.11 B · 50 blocks
Hidden / attention5,376 · 56 heads × 128
FFN14,336
qkv / out projections[21504, 5376] / [5376, 7168]
fc1 / fc2[28672, 5376] / [5376, 14336]
QuantisationW4A8 · int4 (2 per byte) · group 16 · ConvRot 256 · fp8 group scale · fp32 channel scale
DiT on disk10.96 GiB (fl2va 11.68 GiB)
VAE16× spatial compression · patch (1,2,2)
RuntimeComfyUI · PyTorch 2.9.1+cu130 · driver 580 · CUDA 13.0

02 / Throughput

Single card vs TP2.

Smoke sample: 640×352 × 124 frames × 4 steps, text-to-video with no reference image. Same workflow in both columns.

Single card versus two-card tensor parallelism
MetricSingle cardTP2
Per step~51 s~14 s
End to end (4 steps)243 s82 s
Speedup1.00×2.97×
GPU utilisationGPU0 idle / GPU1 97%GPU0 57% / GPU1 72%
Peak VRAMGPU1 19,071 MiBGPU0 11,579 / GPU1 19,681 MiB
Peak temperatureGPU1 78 °C (97% for 4 min)GPU0 67 °C / GPU1 75 °C (1.4 min)

Also measuredAt 864×480 (124 frames, ref2va): 44.6 s/step on one card, 19.8 s/step with TP2 — 2.25×.

03 / Three versions

1.31×, then 2.25×, then 2.97×.

The sharding was correct in the first version. What was missing was a thread, not a split.

Tensor-parallel versions and speedups
Versions/step (864×480)SpeedupWhat changed
Single card44.61.00×Chunked on one card
TP2 v134.01.31×Column/row split with a single launch thread
TP2 + one thread per rank19.82.25×Each rank launches its own kernels

CauseThe W4A8 GEMM blocks its calling host thread, so one launch thread serialised the two cards: measured overlap 5%. One thread per rank lifted it to 91%, with bit-identical output (frame MAE 0.0000).

04 / Probes

The three-row diagnosis.

Two GPUs, one GEMM, three launch strategies. Overlap is the number that separates "sharded" from "parallel".

GEMM overlap measurements
Testdev0dev1ConcurrentOverlap
dense fp16 GEMM61.958.061.095%
W4A8 GEMM103.396.8194.75%
W4A8 GEMM, one thread per card106.597.1106.891%

After the fixPer block: qkv + attention 211 ms, MLP 196 ms, final all-reduce 13 ms — 420 ms per block.

05 / Reproduce

Starting the two-card server.

The device order matters: exposing only one card silently collapses every cuda:0 in the graph onto it.

export CUDA_VISIBLE_DEVICES=1,0          # GPU1 first, so cuda:0 is the compute card
python main.py --listen 127.0.0.1 --port <port> --reserve-vram 1.5 --vram-headroom 0.5     --disable-pinned-memory --fast-disk

python scripts/bench.py examples/smoke_t2v.json smoke   # self-contained sample

06 / Caveats

What the speedup is not.

The 2.97× comes from real renders on the workstation, and it is specific to a quantisation, a model and a pair of cards.

01

Not a general TP result

These numbers come from W4A8 on two modified 22 GB Turing cards at a 250 W limit. Different cards, quantisation or model shape will produce different overlaps.

02

NUMA is part of it

The cards sit on different NUMA nodes. The 13 ms all-reduce partially pays for that topology; another board would change it.

03

Weights are not here

The code is MIT, the model weights are not in the repository and follow MiniMax's own licence. The sample is a self-contained text-to-video case.