A voice product can have a good speech model and still feel broken. The usual cause is choosing the wrong transport. A live assistant needs partial transcripts quickly enough to keep a conversation moving. A call analysis job needs the cleanest possible transcript, speaker labels, and timestamps after the recording already exists.

Google's Gemini 3.5 Transcribe release makes that split unusually explicit. Gemini 3.5 Transcribe is not one endpoint with a latency toggle. Google describes two surfaces: gemini-3.5-transcribe-live through the Live API for continuous, bidirectional streaming, and gemini-3.5-transcribe through the Interactions API for recorded audio. That distinction matters more than the model name.

Gemini 3.5 Transcribe API decision guide

The short version is simple. Use Live API when the transcript is part of an interaction. Use the Interactions API when the audio is an artifact you can process after capture. Do not route a live conversation through the batch-shaped interface just because its headline WER is lower. A half-second saved in a transcript that arrives after the user has finished speaking is not a latency win.

Pick the API by workload

Workload Starting point Why Main risk
Voice assistant or live captioning gemini-3.5-transcribe-live through Live API Continuous bidirectional streaming and sub-second latency Partial text can change as the utterance develops
Recorded meetings or call logs gemini-3.5-transcribe through Interactions API Speaker attribution and word-level timestamps Speaker support is documented for up to three speakers
Post-call search and analytics Interactions API You can wait for a finalized transcript and attach metadata WER on your jargon may differ from the headline number
Voice controlled workflow Live API plus your own action gate Commands can arrive while the user is speaking Never let an early partial transcript trigger an irreversible action

That table is the deployment decision. The rest is testing.

For a live application, stream audio and treat the transcript as a proposal until the utterance closes. Render partial text in the UI, but delay actions such as sending an email, changing a calendar event, or calling a tool until your application sees a stable turn. Gemini 3.5 Transcribe is designed to handle self-corrections, so a phrase such as "meet Tuesday, no, Wednesday" may become cleaner after the speaker corrects themselves. If your action layer listens to the first partial result, your bug is in the application, not necessarily in the recognizer.

For recorded audio, make the finalized transcript your source of truth. The Interactions API path is the better fit for meetings, interviews, support calls, and media archives because speaker attribution and word-level timestamps are useful after the recording is complete. Google says the model can attribute up to three speakers, while support for more than three is experimental. That is a hard product constraint if your input is a panel discussion or a busy support queue. You need a speaker-count test before promising clean diarization to customers.

The same routing rule applies to captioning. Live captions care about arrival time and continuity. A transcript for legal review cares about stable wording, timestamps, and a review queue. Those are different products even when they start with the same microphone signal.

What Google's numbers actually tell you

Google reports average Word Error Rate of 4.0% for streaming use cases and 2.6% for non-streaming use cases, with the measurements attributed to Artificial Analysis. It also reports FLEURS results of 5.50% in streaming mode and 5.04% in non-streaming mode across a set of languages and locales. Google says time to final transcription improves by 70% over Chirp 3.

Those figures are useful, but they do not answer the question most teams actually have: will the words that matter to my product be correct? WER treats every token as a token. A missed filler word is cheap. A wrong order ID, medication name, account number, or function argument can be expensive. Google specifically calls out alphanumeric entities such as postal codes and order IDs, which is a hint about where to focus your own test set.

Build a small acceptance set before choosing an API. Record the same scripted phrases through the streaming and prerecorded paths. Include names from your customer database, product codes, acronyms, accents, interruptions, background noise, and self-corrections. Score the output twice: ordinary WER, and an error list weighted by business impact. A transcript that scores well overall but corrupts every SKU is a failure for an inventory app.

The release also says Gemini 3.5 Transcribe can automatically detect and transcribe more than 85 languages, recognize custom vocabulary, remove filler words, and format text. Those additions help, but each one needs a boundary. Automatic language detection is not a substitute for testing code-switching. Custom vocabulary is not a guarantee that a rare spelling survives noise. Filler removal is helpful for notes and captions, but dangerous if your application needs a faithful record of what was said.

Where the numbers stop helping

The cleanest mistake to avoid is comparing 2.6% and 4.0% as if they were two universal quality scores. They come from different operating modes. The streaming path has a harder timing requirement, while the non-streaming path can use the whole recording. A lower WER after the fact does not make it suitable for a turn-taking assistant.

The second mistake is assuming that formatting equals understanding. A polished transcript can still attach the wrong speaker to a sentence or normalize a technical term into a common word. Keep the original audio and store timestamps. Let a reviewer replay the audio beside any suspicious line.

The third mistake is letting transcription directly control tools. Function calling appears in Google's product description for complex tasks delegated to other Gemini models, but a transcript is still an imperfect observation. Put a confirmation step between speech and side effects. For low-risk actions, confirmation can be a short visual state. For payments, account changes, deletion, or messages sent to other people, require an explicit second turn or button press.

A sensible first implementation is a two-lane pipeline. The live lane handles audio transport, partial text, turn detection, and conservative intent extraction. The archive lane stores the audio and sends completed recordings through the prerecorded path for a stable transcript, speaker labels, and search indexing. You can use the live result to keep the interface responsive, then replace it with the finalized result when processing completes.

That approach costs more engineering than wiring one model to one microphone. It also avoids building your product around a benchmark that answers a different question. Gemini 3.5 Transcribe looks most useful when treated as two tools with a shared model family. Pick the tool by the moment at which your application needs the words, then measure the errors your users cannot afford.

Sources