Gemini 3.5 Transcribe Developer Guide: Live API, Smart Mode, and Speech-to-Text

Direct answer

Google’s Gemini 3.5 Transcribe is a speech-to-text model for developers who need more than a literal audio transcript. Google announced it on August 26, 2026, with two developer-facing paths: gemini-3.5-transcribe for recorded audio through the Interactions API, and gemini-3.5-transcribe-live for bidirectional streaming through the Live API. The model supports automatic detection across 85+ languages, custom vocabulary, smart cleanup of filler words and self-corrections, and speaker attribution for recorded audio.

The practical choice is simple: use the recorded-audio endpoint for meetings, calls, and uploaded files; use the Live endpoint for captions, voice interfaces, and applications that need partial transcripts while someone is speaking. Keep “smart” transcription separate from verbatim records: smart mode can remove disfluencies and rewrite a correction, which is useful for dictation but not always appropriate for legal, medical, or compliance archives.

What Google released

Google’s primary announcement says Gemini 3.5 Transcribe is available to developers in public preview through the Gemini API in Google AI Studio and Gemini Enterprise Agent Platform. The Gemini API model page confirms the two model IDs and their capability split. Independent reporting from Ars Technica corroborates the launch, the Gboard Rambler connection, the developer availability, and the warning that cleaned-up speech can change wording.

Google’s model documentation lists the recorded model as supporting audio input, text and word annotations, smart transcription, custom vocabulary, and speaker diarization. It also says caching, code execution, function calling, and batch processing are not supported by that model. Those constraints should shape the surrounding application rather than being hidden behind a generic transcription abstraction.

Capability map

Visual 1 — Endpoint selection and trust boundary. This original diagram summarizes the endpoint split from Google’s announcement and model documentation; the validation and review stages are implementation guidance, not a Google product claim.

Smart transcription versus verbatim records

The differentiator is context-aware cleanup. Google says the model can handle self-corrections such as “Tuesday—no, Wednesday,” remove filler words, auto-format text, and recognize specialized vocabulary. That makes it attractive for voice commands and dictation. It also creates a provenance question: the polished output is not necessarily a character-for-character record of the audio.

Use two modes when the product needs both convenience and evidence:

1. Preserve the original audio and, where required, a verbatim transcript.
2. Generate a separate smart transcript for reading, search, or downstream summarization.
3. Display which representation a user is viewing.
4. Keep timestamps and speaker labels attached to the source transcript.
5. Require human review before a smart transcript becomes a legal, clinical, financial, or customer-facing record.

Ars Technica’s hands-on reporting makes the same practical point: cleanup can be helpful for short blocks of text, but an AI system may technically change the wording. Treat “smart” as a transformation layer, not as proof that every word was spoken exactly as displayed.

Supported languages, speakers, and custom vocabulary

Google says Gemini 3.5 Transcribe automatically detects and transcribes more than 85 languages, including language switching. The model page documents custom-vocabulary biasing with up to 1,000 terms, while noting that customers typically see the best results with a smaller vocabulary of up to roughly 100 terms. This is useful for product names, people, postal codes, order IDs, and specialized jargon.

For recorded audio, the announcement describes speaker attribution with timestamps for up to three speakers, while the current model page documents support for up to eight speakers and marks attribution for three or more as experimental. That difference is important: use the current model page for integration limits, and test diarization with your actual microphone, room, overlap, and number of participants.

Do not use custom vocabulary as a security control. It can improve recognition of an internal term; it cannot prove that the term was spoken, authorize an action, or validate a customer identity. A voice-driven workflow should still require account authentication and server-side authorization.

A practical JavaScript integration through Vercel AI SDK

Vercel’s August 26, 2026 changelog documents AI SDK V7 transcription support for both Gemini 3.5 Transcribe variants through AI Gateway. The following pattern is appropriate for a server-side application using a recorded file and a live microphone stream. It keeps the provider model explicit and avoids putting credentials in browser code.

1import { experimental_transcribe as transcribe } from "ai";
2import { experimental_streamTranscribe as streamTranscribe } from "ai";
3import { gateway } from "@ai-sdk/gateway";
4import { readFile } from "node:fs/promises";
5
6const recording = await transcribe({
7  model: gateway.transcription("google/gemini-3.5-transcribe"),
8  audio: await readFile("meeting.mp3"),
9});
10
11console.log(recording.text);
12
13// microphoneStream must yield 16 kHz, 16-bit PCM chunks.
14const live = streamTranscribe({
15  model: gateway.transcription("google/gemini-3.5-transcribe-live"),
16  audio: microphoneStream,
17  inputAudioFormat: { type: "audio/pcm", rate: 16000 },
18  providerOptions: {
19    google: { mode: "SMART" }, // or "VERBATIM"
20  },
21});
22
23for await (const event of live.textStream) {
24  process.stdout.write(event);
25}

The Vercel example documents SMART and VERBATIM modes for streaming and identifies the live input as 16 kHz, 16-bit PCM. Confirm the current AI SDK version and provider adapter before deploying; experimental function names can change. If you call Google directly instead of through AI Gateway, follow Google’s current Interactions API and Live API documentation rather than translating this adapter example into an unverified REST payload.

Request flow for production voice features

Visual 2 — Original request-flow diagram. Audio is not sent directly from a model response into a business action: the application validates the result, preserves provenance, and adds review where the transcript can cause harm.

Accuracy, latency, and benchmark claims

Google’s announcement reports an average Word Error Rate of 2.6% for non-streaming use cases and 4.0% for streaming, measured by Artificial Analysis. It also reports 5.50% streaming and 5.04% non-streaming WER on FLEURS, plus a 70% improvement in time to final transcription compared with Chirp 3. Ars Technica independently reports Google’s 5.5% live-speech figure and the comparison with Chirp 3.

These numbers are useful for understanding the launch, not for promising your application’s result. WER changes with accents, code-switching, microphone quality, overlapping speakers, background noise, proper nouns, and whether smart cleanup is enabled. Build a local evaluation set containing:

• representative accents and languages;
• domain terms and alphanumeric identifiers;
• overlapping speech and realistic background noise;
• both smart and verbatim expected outputs;
• latency from audio capture to usable text;
• cases where changing wording would be unacceptable.

For live applications, measure end-to-end time to first partial transcript and time to stable final text. Network buffering, PCM conversion, browser capture, UI rendering, and retries can dominate model latency.

Cost and deployment decisions

Google’s public pricing page is the source of truth for current model rates. The dedicated model page links developers to that pricing surface rather than embedding a price in the capability table. Vercel says its AI Gateway reflects provider pricing with no markup or platform inference fee, but your application can still incur gateway, hosting, storage, bandwidth, and observability costs.

Before choosing an architecture, estimate:

Do not assume that a lower transcription error rate means a lower total cost. Smart output may reduce editing time, while diarization, retries, retention, and review add operational work.

Security and privacy checklist

• Obtain clear consent before recording or transcribing people.
• Keep API keys on the server and restrict their permissions.
• Redact secrets, payment data, and unnecessary personal information before persistence or logging.
• Keep audio, verbatim text, smart text, and derived summaries under separate retention policies.
• Treat transcript content as untrusted input; it can contain prompt injection aimed at a downstream agent.
• Never let a transcript directly execute a payment, permission change, deployment, or deletion.
• Bind any voice command to an authenticated user and confirm high-impact actions out of band.
• Test language switching, custom vocabulary, speaker overlap, and accidental recordings.
• Record model ID, mode, SDK version, timestamps, and consent state for important workflows.

The model’s ability to call functions in some Google product experiences does not mean a general transcription response should be granted arbitrary tool access. Put authorization in application code and keep the speech layer focused on producing bounded, inspectable output.

Common errors and debugging

The live transcript is empty or delayed

Check that the stream is actually 16 kHz, 16-bit PCM and that the client is sending chunks continuously. Log chunk sizes and timestamps without storing raw audio by default. Test with a known-good sample before debugging the microphone permission path.

Names and product codes are wrong

Add a minimal custom vocabulary containing the terms most likely to fail, then evaluate false substitutions as well as corrected terms. Do not send an entire dictionary: the model documentation says smaller vocabularies commonly work best.

Smart mode changed an important sentence

Switch the archival path to verbatim, preserve the audio, and present smart output only as a derived convenience view. Add a user-visible mode label.

Speaker labels are unreliable

Use a controlled test with known participants and minimal overlap. Remember that recorded diarization and live streaming have different documented support. A failure in diarization should not block the transcript from being shown as unattributed text.

A production request fails after the preview launch

Verify the model ID, account access, region, SDK/provider version, audio MIME type, duration, and current preview limits. Google’s model page and release notes are more authoritative than a copied integration snippet. Pin dependencies in CI, but schedule a compatibility review because preview APIs can evolve.

FAQ

Is Gemini 3.5 Transcribe the same as Gemini’s general audio understanding?

No. Gemini’s general audio capability can analyze audio and generate a response, while Gemini 3.5 Transcribe is a dedicated speech-to-text model with transcription-specific features such as smart cleanup, custom vocabulary, word-level timestamps, and a live transcription endpoint.

Which model should I use for a meeting recording?

Start with gemini-3.5-transcribe through the Interactions API. It is the documented path for recorded audio and supports speaker attribution and word-level timestamps, subject to the duration limits and experimental status described in the current model documentation.

Which model should I use for live captions?

Use gemini-3.5-transcribe-live through the Live API. Design the UI around incremental events and a later finalized transcript, and do not expect the same timestamp and diarization features available for recorded audio.

Can I use smart transcription for compliance records?

Not as the only record. Smart mode can remove fillers and rewrite self-corrections. Preserve the source audio and a suitable verbatim representation when the exact wording matters, then label any polished transcript as derived output.

Is the model production-ready?

Google’s announcement makes the developer endpoints available in public preview, while the release notes mark Gemini 3.5 Transcribe as generally available in the Gemini API model catalog. For a production decision, follow the current endpoint documentation, preview-specific terms, quotas, and your own evaluation results rather than treating the launch label as a guarantee.

Conclusion

Gemini 3.5 Transcribe is most interesting as an intelligent transcription layer: it can turn messy speech into useful structured text, handle multilingual input, and support both recorded and low-latency workflows. The right architecture keeps that intelligence bounded. Choose the endpoint by interaction style, preserve provenance, evaluate on real audio, separate smart text from verbatim evidence, and keep authorization outside the transcript model.

For related implementation patterns, see the site’s Google ADK for Java 1.0.0 production agent guide, Gemini 3.7 Flash API and agent guide, Gemini Robotics ER 2 safety guide, and Google Developer Knowledge API and MCP guide. They cover typed tools, multimodal model integration, physical-action safety, and grounded documentation retrieval—the controls a voice feature needs once transcription becomes part of an agent workflow.

Sources and visual credits

• Google: Introducing Gemini 3.5 Transcribe — primary announcement, capabilities, benchmark claims, availability, and first-party use cases.
• Google AI for Developers: Gemini 3.5 Transcribe model page — model IDs, input limits, feature support, and language documentation.
• Google AI for Developers: Gemini API release notes — August 26, 2026 release entry and API availability context.
• Ars Technica: Google announces Gemini 3.5 Transcribe — independent corroboration and practical caution about smart rewriting.
• Vercel: Gemini 3.5 Transcribe on AI Gateway — verified AI SDK V7 integration pattern and streaming audio format.
• Visual 1 and Visual 2 are original Mermaid diagrams by Essa Mamdani, based on the cited Google model and API documentation. No third-party screenshots or unlicensed product imagery are used.

Related reading

Continue exploring related AI engineering and developer tooling topics:

• Gemini Omni 1.1 Flash: A Practical Developer Guide to Video Generation and Editing
• Muse Spark 1.3: Meta’s Frontier Coding Agent
• Claude Fable 5.1: Anthropic’s Long-Horizon AI Upgrade