The new claim is easy to misread: DeepSeek V4 Flash is not suddenly a small model. It is still a 284B-parameter mixture-of-experts system with 13B active parameters. What changed is that someone has brought it up on one AMD Instinct MI300X, a card with 192GB of HBM3. That is a useful distinction. The breakthrough is less about shrinking the model than proving that the right memory layout and a stubborn ROCm engineer can turn an awkward accelerator into a workable single-card target.

MI300X memory and bandwidth make the single-card claim plausible

The fresh GitHub repository appeared on Hacker News on August 4 with the plain title "DeepSeek V4 Flash on a Single AMD MI300X." The repository is small and new, so it is not a production benchmark suite. The stronger technical evidence comes from the earlier AMD bring-up work around the same model, plus the current vLLM and SGLang deployment notes. Put together, they answer the useful question: can one MI300X load the model and produce responses? Yes. Can it replace an eight-GPU serving node for every workload? No.

The memory math is the interesting part

DeepSeek V4 Flash uses mixed precision. The routed experts are stored in FP4, while attention, normalization, routing, and other dense pieces stay in FP8. The model card describes 1M-token context, compressed sparse attention, heavily compressed attention, and a much smaller KV-cache footprint than the previous generation. The stated target is roughly 27% of V3.2's per-token inference FLOPs and 10% of its KV cache at 1M context.

That architecture changes the hardware question. A model with 284B total parameters sounds like a multi-node problem if you imagine every weight in a dense 16-bit format. The active parameter count is only one part of the story, but the mixed FP4 and FP8 checkpoint makes the stored weight footprint much smaller. The current vLLM recipe says the checkpoint occupies about 148.66 GiB in the validated MI325X single-card setup. MI300X has less memory than MI325X, so the remaining headroom is not generous, especially once context, runtime buffers, and the KV cache enter the picture.

AMD's own specifications put MI300X at 192GB HBM3 and 5.3TB/s of peak memory bandwidth. An H100 SXM has 80GB and 3.35TB/s. The raw comparison is not a complete performance test, but it explains why this particular model is a better fit for MI300X than many smaller dense models with less unusual memory behavior. DeepSeek V4 Flash is often memory-bound during decode. More capacity means fewer partitions. More bandwidth means less waiting while the card moves model state.

The chart above uses vendor specifications, not a workload result. Do not read it as proof that MI300X is faster than H100. It is a diagram of why a single card can be plausible at all.

AMD's problem was never just VRAM

The earlier bring-up report is a good warning against treating model support as a checkbox. MI300X uses the older CDNA3 generation and an AMD-specific FP8 dialect called fnuz. Later AMD parts moved to the Open Compute Project's more common FP8 behavior. The bit layout looks similar, but the exponent bias is different. Reading one dialect as the other can put values off by a factor of two.

That mismatch broke parts of the initial DeepSeek support path. The fixes had to make compression, quantization, and KV-cache writes agree with the datatype actually used by the card. This is the sort of bug that produces output that looks plausible until a difficult request exposes it. A model can load, answer a few prompts, and still be numerically wrong in a way that is hard to spot from a smoke test.

Then came attention kernels. V4's compressed attention has sparse indexer work, compressed KV state, and a sliding-window path. AMD's AITER library has tuned kernels for many shapes, but coverage on MI300X's gfx942 target is uneven. Some paths were absent. Others existed for newer AMD chips and failed on this older one. The practical fix was a mix of AITER calls, ROCm-specific helpers, and Triton fallbacks.

The fallbacks matter. Generic Triton can be several times slower than a tuned kernel for these small, repeated operations. A decoder runs this machinery for every generated token, so a slow helper compounds across a long response. The team also had to make metadata capture-safe for HIP graphs. Dynamic allocations and host-side values inside a captured decode loop can freeze the wrong state at warmup or fail outright.

This is why the single-card result is more interesting as an engineering story than as a shopping recommendation. The hardware was already capable of holding the weights. The hard part was making every unusual attention and routing path behave on a real production-oriented stack.

What the measured result does and does not say

The bring-up work reports a simple optimization benchmark moving from 2,485 to 2,699 output tokens per second per GPU, an improvement of about 8.6%. That is a useful before-and-after number, but it is not the same as 2,699 tokens per second for one interactive user. It is also not a cost-per-token comparison against H100, H200, or B200. The authors explicitly say they have not completed that calculation.

The measurement does show where performance went. After correctness was fixed, the expensive parts were sparse MLA and the MXFP4 mixture-of-experts path. Some time was also lost to bookkeeping: rebuilding ragged metadata, copying scratch outputs, materializing a BF16 projection weight repeatedly, and using one static Triton launch shape for workloads with very different batch sizes. Removing some of that overhead produced the 8.6% gain.

The current official recipes point in a different direction for general serving. vLLM recommends four-way data parallel and expert parallel layouts on supported multi-GPU nodes. Its validated AMD recipe is for one MI325X with a 4K context cap, not one MI300X with unrestricted 1M context. SGLang lists a ROCm image for MI300X and provides a V4 demo notebook, but its wider verified matrix still favors multi-GPU configurations for serious throughput.

That difference matters to anyone planning a deployment. A single MI300X can be a sensible development box, a private inference endpoint with modest concurrency, or a way to test AMD support without buying an eight-card server. It is not evidence that one card will serve a million-token workload with the same latency and concurrency as a larger node. The model can fit. The service-level target may not.

There is also a maintenance cost. You need a ROCm build that includes the relevant fixes, the right model revision, the correct tokenizer and tool-call parser, and enough spare memory for the context you actually send. A future vLLM update may improve the path, but it may also change which kernel is selected. CUDA users have their own problems, but the default path is usually less of a detective exercise.

That is the real value of the new repository. It makes AMD hardware harder to dismiss as "unsupported" and gives maintainers a concrete target to upstream. It does not erase the software tax. It puts a price on it.

For readers who are tracking the model rather than the accelerator, this is the hardware companion to DeepSeek V4 Flash's cheap coding benchmark. For the broader inference question, the memory cost of frontier inference is still the right starting point. The common thread is simple: parameter counts are a poor deployment plan when memory movement, cache state, and kernel coverage decide the bill.

Sources