The weird thing about most AI video demos is that they are conversations only in the marketing copy. You type a prompt, wait, watch the clip, and discover that the character has already made every decision for you. Vidu S1 tries to remove that waiting room. Its demo is built around a voice conversation with a generated character, and the paper claims 540p output at up to 42 frames per second on an RTX 5090.

Vidu S1 streaming video generation pipeline

That claim needs a little translation. The normal service target is 540p at 25 FPS, while 42 FPS is the reported peak under the paper's three-step configuration. The result is still interesting because 25 FPS is the threshold where an avatar starts to feel like a video call instead of a slideshow. More important, Vidu S1 is designed to keep generating while the user speaks. You can interrupt the character, change the instruction, and affect what happens next.

The change is interaction, not prettier clips

Traditional video generators are batch systems. The prompt goes in, a fixed clip comes out, and any correction starts another job. That workflow is fine for a short cinematic shot. It is a poor fit for an avatar that is supposed to react to a person in front of it.

Vidu S1 uses an autoregressive diffusion setup. Instead of denoising an entire clip before showing it, the system predicts the next piece of video from the recent frames, the current visual state, the conversation context, and new speech input. That gives the user a control surface while the video is running. Say "turn around" and the instruction is meant to affect future frames rather than a second, unrelated render.

The control is also broader than lip sync. ShengShu says the model maps speech meaning and emotional intent into facial expression, eye movement, gestures, posture, and full-body action. That distinction matters. A lip-sync system can make a silent animation mouth words. It cannot decide that a sarcastic sentence deserves an eye roll and a change in posture.

The system starts from one image and a selected or cloned voice. The image can depict a person, an anime character, or a pet. There is no separate rigging step in the public workflow. That is convenient, but it also means the model has to infer anatomy, identity, and motion from a very small amount of information. I would trust it more for a stylized host than for a character whose hand movements need to be correct every time.

How the speed is assembled

The paper is clear about one thing that product pages often blur: live video is a stack, not a single model trick. Vidu S1 combines a few-step diffusion configuration with TurboDiffusion, low-bit SageAttention, sparse attention methods, optimized kernels, quantization, and TurboServe for serving and scheduling. Each piece cuts a different part of the cost.

The reported headline numbers are:

Metric Reported result
Output resolution 960 x 540
Normal interactive rate 25 FPS
Reported peak rate 42 FPS
Peak test hardware RTX 5090
Generation mode Three diffusion steps
Session duration Designed for open-ended streaming

The 42 FPS figure is not a promise that every laptop will run a talking avatar at that rate. The technical result is more specific: the authors report an average throughput of 42 FPS at 540p on RTX 5090 GPUs with a three-step setup. The public product page describes 25 FPS as the normal 540p experience and lists 42 FPS as the upper bound. Those are different measurements, and treating them as interchangeable would make the story sound better than the evidence allows.

The paper also reports 0.9192 on CSIM, 7.847 on Sync-D, and 0.5660 on DOVER in its evaluation. CSIM is used for identity consistency, Sync-D for audio-visual synchronization, and DOVER for perceptual video quality. The authors say Vidu S1 leads on the tested metrics while meeting the live constraint. That is useful evidence, but it is still an internal paper comparison, not an independent bake-off against every current video model. The paper's comparison set and serving conditions deserve more scrutiny before anyone calls this solved.

The long-session problem may be the harder part. Autoregressive video systems accumulate mistakes. A small identity shift becomes a larger one after many seconds, and a visual artifact can become part of the next frame's context. Vidu S1 says it reduces this error accumulation so the stream can continue without the usual blur, drift, or collapse. The claim is ambitious. The public materials show the intended experience, while the paper gives more detail about the data pipeline and evaluation, but neither gives a clean independent stress test of a multi-hour conversation.

The useful version is smaller than the hype

A live character is not automatically useful. Most people do not need an AI avatar that makes eye contact while answering a question. The practical cases are narrower: a game character that reacts to player speech, a virtual host that can handle follow-up requests, a tutor that changes its visual explanation as a student talks, or a branded character that can appear in a live stream without a human puppeteer.

That last example is where the economics get uncomfortable. At 25 FPS, the system must generate roughly 1,500 frames every minute for one session. Even with aggressive acceleration and batching, continuous generation costs more than rendering a ten-second clip once. TurboServe can schedule requests and keep state, but the paper does not publish a complete per-session price or a capacity table for a production deployment. "Runs on consumer GPUs" is a hardware statement, not a business model.

There are also safety and consent questions hiding in the easiest feature. A user can upload a photo and clone a voice. The same workflow that makes a personal avatar simple also makes impersonation simple. The product page says personal information is processed and asks users to agree to rules, but the public research paper is primarily about generation quality and serving speed. Anyone deploying this for real people needs stronger identity checks, consent records, and abuse handling than a demo account requires.

Vidu S1 is therefore more interesting as a systems paper than as another character generator. The model has to keep identity stable, interpret speech, generate motion, preserve temporal context, and deliver frames before the user notices the delay. A better offline clip would not solve those problems. The constraint is the point.

The paper was posted in July 2026, and the playable Vidu Stream experience is available now. I tried to keep the headline claim narrow because the distinction matters: this is not proof that AI video has become a general-purpose live world simulator. It is a credible attempt to turn a generated character from a finished asset into an interactive endpoint. If the quality holds outside controlled demos, the video model becomes less like a renderer and more like a display layer for an agent. That is a much stranger product, and probably the more useful one.

Sources