Tencent Hy4 preview arrived with the kind of headline that makes local AI people reach for a calculator: 770B total parameters, 49B active per token, and a context window advertised at 1M tokens. Then the official serving command quietly answers the more useful question. It asks for 8 GPUs.

Hy4 preview benchmark comparison

That does not make Hy4 a bad release. It makes the release honest in a way that a lot of open-weight announcements are not. The weights are available, the license is Apache 2.0, and Tencent publishes working vLLM and SGLang launch recipes. But “open” and “easy to run” are two different claims. Hy4 is open enough to inspect and deploy, while still being a serious infrastructure project for most teams.

The fresh decision is simple: if you have an eight-GPU server, a reason to keep a very long context window inside your own boundary, and somebody who can operate vLLM or SGLang, Hy4 is worth testing. If you have a workstation with one or two cards, the hosted route is the practical choice. Downloading 131 weight shards to prove that a model is technically local is not a deployment plan.

The eight GPU recipe

Tencent’s own vLLM example uses the FP8 checkpoint tencent/Hy4-preview-FP8, the vllm/vllm-openai:hy4-preview image, and --tensor-parallel-size 8. It also turns on the sparse attention backend, tool and reasoning parsers, automatic tool choice, and three speculative tokens. That is a useful command because it exposes the shape of the intended system. This is not a casual Transformers demo that fits into a notebook. The supported path is a distributed inference server.

The repository reports 770B total parameters and 49B activated per token. The active count explains why the model can be competitive without doing dense 770B compute for every token, but it does not erase the memory cost of storing the model, its routing layers, KV cache, runtime buffers, and a usable context window. The Hugging Face repository contains 131 safetensors shards (131 shards) for the base checkpoint. Those files are a warning label for operations, not a performance metric. Expect a large download, a cache that needs deliberate placement, and failures that may come from collective communication or storage before they come from the model itself.

The 1M context claim is also easy to misread. It tells you the model and serving stack accept a long sequence. It does not promise that every 1M-token request will be fast, cheap, or equally accurate. KV cache pressure rises with the actual prompt and generated sequence. A team that only sends 20K-token coding tasks does not automatically benefit from paying the operational cost of a million-token ceiling.

The model uses Gated DeepSeek Sparse Attention with IndexCache, four residual streams, 256 routed experts, and one shared expert. Each token selects eight routed experts plus the shared expert. There is also one native MTP layer for speculative decoding. These details matter because the official launch flags are not decorative. The sparse attention backend and speculative settings are part of the performance story. Remove them because a generic container is easier, and you may be testing a different system from the one Tencent describes.

Before starting a server, check four things. First, confirm that the eight GPUs are visible to Docker and that their combined memory leaves room for runtime overhead, not merely checkpoint storage. Second, use the exact Hy4-compatible vLLM image or the matching SGLang image rather than assuming the newest generic build has every parser and kernel. Third, pin the model and provider during evaluation so a hosted comparison does not quietly change hardware between requests. Fourth, begin with a context limit that matches your workload. A million-token setting is a capability to validate, not a sensible default.

The launch command is a starting point, not a guarantee of production readiness:

docker run --gpus all -p 8000:8000 --ipc=host \\
  -v ~/.cache/huggingface:/root/.cache/huggingface \\
  vllm/vllm-openai:hy4-preview tencent/Hy4-preview-FP8 \\
  --tensor-parallel-size 8 \\
  --speculative-config '{"num_speculative_tokens":3,"method":"mtp"}' \\
  --attention-backend FLASHMLA_SPARSE \\
  --tool-call-parser hy_v4 --reasoning-parser hy_v4 \\
  --enable-auto-tool-choice --port 8000 \\
  --served-model-name hy4-preview

Run a small acceptance set before you move real traffic. Include a short direct answer, a tool call, a long document with a known answer near the end, and a coding task that can be checked by tests. Record time to first token, output rate, peak memory, tool-call validity, and how often the model over-verifies. Tencent lists spending too long on complex reasoning and over-checking as known limitations. Those are operational problems when an agent is paid by GPU time.

What the release numbers prove

Tencent reports a blind internal side-by-side evaluation in which 163 internal experts rated outputs on 203 engineering tasks. Hy4 preview scored 2.99 against GLM 5.3 at 2.92, with 46.8 percent wins, 12.8 percent ties, and 40.4 percent losses. Against Kimi K3, Tencent reports 2.99 versus 2.94, with 51.2 percent wins, 7.9 percent ties, and 40.9 percent losses.

Those numbers support a narrow claim: Tencent’s evaluators found Hy4 slightly better on that internal engineering set. They do not establish that Hy4 will be better for your codebase, your language mix, or your agent harness. The sample is large enough to be interesting and still too narrow to replace an acceptance test. The community reaction on Hacker News made the same point from another direction. Several users focused less on the score and more on provider speed, rate limits, cache pricing, and whether the public charts were drawn clearly. One commenter reported timing out while trying to benchmark the hosted model. That is not a model score, but it is part of the user experience.

The most interesting performance question is therefore not whether Hy4 beats GLM 5.3 by seven hundredths on Tencent’s scale. It is whether the sparse attention, long context, and speculative decoding produce a useful cost per completed task on the exact hardware you can access. A model that wins a lab comparison but takes too long to finish a code review can lose in production.

For context, the earlier GLM-5.3 local deployment math reached a similar conclusion: parameter counts turn model choice into an infrastructure decision. Hy4 changes the model and the serving recipe, but not that basic constraint. Active parameters help compute. They do not make eight GPUs disappear.

When hosted inference wins

Hosted inference wins when utilization is low, latency matters more than data locality, or the team cannot justify keeping eight GPUs busy. It also wins when you need to test Hy4 before buying hardware. Start with a pinned provider, measure cache behavior, record rate limits, and keep a fallback model for requests that fail or exceed the provider’s context policy. The HN discussion around Hy4 repeatedly returned to cache-hit percentage and provider selection. Routing can change the economics of long agent sessions, especially when a request moves between providers and loses its reusable prefix.

Self-hosting wins for a different reason. The model’s long context can stay behind your network boundary, tool calls can be audited locally, and a stable server lets you tune the exact parser, cache, and context settings your workload needs. Those benefits are real, but they only appear after the service is reliable. If the server spends its time downloading shards, rebuilding kernels, or retrying failed collectives, the privacy advantage does not help the person waiting for an answer.

There is also a naming trap. Hy4 preview is not a tiny 49B model. “49B active” describes the parameters used per token in a 770B mixture-of-experts model. It should not be compared to a dense 49B checkpoint as if the two have the same memory footprint or deployment cost. The FP8 release makes the intended server more accessible than full precision, but Tencent’s eight-way recipe is still the clearest public signal of the required class of hardware.

My recommendation is to treat Hy4 as an eight-GPU evaluation target, not a laptop download. Rent or borrow the matching hardware for a fixed test window. Put five representative tasks behind the OpenAI-compatible endpoint. Compare completed-task latency and failure rate against the hosted model, not just tokens per second. If the local run does not win on privacy, predictable latency, or total cost, delete the cache and keep the API. Open weights are valuable. Idle GPUs are not.

Sources