Gemini 3.1 Flash Live native-audio TTS can produce runaway/no-audio output and 1011 when explicit temperature: 0 is set

[y-lobau]

Thanks for the library. I'm reporting this here because the issue is triggered by a @google/genai Live API config value that the client accepts and forwards. The underlying failure may be a Gemini Live backend/product issue, but it would be helpful to know whether the JS SDK should validate, reject, clamp, or document this configuration for native-audio Live TTS.

Environment details

• Programming language: TypeScript / JavaScript
• OS: macOS 26.3.1, arm64
• Language runtime version: Node.js v22.22.2
• Package version: @google/genai ^1.47.0

Model / API path

• Model: models/gemini-3.1-flash-live-preview
• API: Gemini Live API via ai.live.connect
• Output modality: audio only
• Use case: native-audio TTS, text input sent to Live session
• Voice tested: puck; also compared with AI Studio-style Zephyr
• Language: Belarusian

Steps to reproduce

1. Create a Live API session with models/gemini-3.1-flash-live-preview.
2. Configure audio response modality and a normal speechConfig.
3. Add explicit temperature: 0 to the Live config.
4. Send Belarusian text to be spoken, either through sendRealtimeInput or AI Studio-style sendClientContent.
5. Collect raw audio chunks before playback and observe session close metadata.
6. Repeat a few times. The failure is intermittent but reproducible.

Minimal shape:

import { GoogleGenAI, Modality } from "@google/genai";

const ai = new GoogleGenAI({ apiKey: process.env.GEMINI_API_KEY });

const session = await ai.live.connect({
  model: "models/gemini-3.1-flash-live-preview",
  config: {
    responseModalities: [Modality.AUDIO],
    temperature: 0,
    speechConfig: {
      voiceConfig: {
        prebuiltVoiceConfig: {
          voiceName: "puck",
        },
      },
    },
    systemInstruction: {
      parts: [
        {
          text: "You are a robotic Text-To-Speech engine. Read the input text aloud exactly as provided. Language: Belarusian.",
        },
      ],
    },
  },
  callbacks: {
    onmessage(message) {
      // collect message.serverContent.modelTurn.parts[].inlineData.data
      // into raw PCM/WAV before playback
    },
    onerror(error) {
      console.error(error);
    },
    onclose(event) {
      console.log("closed", event.code, event.reason, event.wasClean);
    },
  },
});

session.sendClientContent({
  turns: [
    "Вядома, я ўсталю таймер на пяць хвілін і нагадаю вам, калі час скончыцца. Каб праверыць сінтэз маўлення, я прачытаю яшчэ некалькі беларускіх сказаў без зменаў і без адказаў на пытанні. Шчыра кажучы, гэта павінна гучаць натуральна: вучымся, груша, вугаль, ґанак, каўнер, рака, дзеці. Праігрываю гурт Naviband са spotify, але не перафармулёўваю гэты сказ і не замяняю словы сінонімамі. Калі ў тэксце ёсць пытанне, напрыклад: ці ўсё добра?, я проста агучваю пытанне як напісана.",
  ],
});

Expected behavior

The Live session should return bounded audio corresponding to the input text, then complete/close cleanly. If temperature: 0 is unsupported or unsafe for this model/output mode, the SDK or API should reject it or document the supported range.

Actual behavior

With explicit temperature: 0, the Live native-audio TTS response can become pathological before playback:

• Audio output becomes oversized/runaway, far longer than the input text requires.
• In some runs, the session closes with 1011 Resource exhausted.
• In other baseline runs, there is no usable audio.
• The problem is visible in raw PCM/WAV capture before any local playback, so this does not appear to be a speaker/playback issue.

Evidence

Using the same Belarusian prompt/text:

• Product-style Live path with explicit temperature: 0:

  • 1/3 oversized failures.
  • Failing sample: 8,911,204 raw PCM bytes, about 185.65s at 24 kHz.
  • Close code: 1011 Resource exhausted.

• Manual baseline with explicit temperature: 0:

  • 5 attempts: 3 OK, 1 no-audio, 1 oversized/error.
  • Oversized sample: 10,930,564 raw PCM bytes, about 227.7s at 24 kHz.
  • Close code: 1011 Resource exhausted.

• Literal AI Studio-style control without explicit temperature:

  • 5/5 clean.

• Temperature isolation using literal AI Studio-style session shape:

  • Without explicit temperature: 3/3 clean.
  • With temperature: 0: 1/3 oversized.
  • Failing sample: 12,616,804 raw PCM bytes, about 262.85s at 24 kHz.
  • Close code: 1011 Resource exhausted.

• After removing explicit temperature from our production TTS path:

  • 20/20 product-path attempts clean.
  • 0 oversized/no-audio/1011 failures.

We have only reproduced and validated this with Belarusian input; we have not tested whether the same temperature: 0 behavior occurs with English or other languages.

Why I'm filing here

This may ultimately be a Gemini Live product/backend bug, but from the JS client side, it is unclear whether temperature: 0 is valid for gemini-3.1-flash-live-preview native-audio output.

Could you clarify whether:

1. temperature: 0 is supported for Gemini Live native-audio TTS?
2. Should the JS SDK validate or reject unsupported temperature ranges for this mode?
3. The docs should mention a safe temperature range or recommend omitting temperature for native-audio TTS?

[dbloembe]

We're hitting the same zero-audio generation issue in production with gemini-3.1-flash-live-preview, though our trigger is different — it happens during multi-turn voice conversations with tool calls (function calling), not TTS with temperature: 0.

Our setup

• SDK: @livekit/agents-plugin-google 1.3.4 (LiveKit JS Agents SDK, wraps @google/genai)
• Use case: Voice agent for restaurant phone ordering — multi-turn conversations with 5-15 tool calls per session
• Model: gemini-3.1-flash-live-preview
• Temperature: 0.3
• Voice: Aoede

What we observe

After delivering tool call responses (function_call_output), the model occasionally enters a generation that completes with turnComplete but produces zero inlineData (audio) frames. The generation's first token timestamp is never set. The model appears to enter a thinking state and never produces output.

This is intermittent but consistent: in a test suite of ~45 automated voice calls, 2-4 calls per run hit this without client-side recovery. It tends to happen after tool call round-trips (menu lookup → add item → get cart), not during free-form speech.

Timeline from logs

18:14:20 — tool response sent (get_cart) — delivered successfully, no stale pendingToolCallIds
18:14:28 — Agent stuck in thinking for 8s (zero audio frames produced)
18:14:33 — Audio stall detected (5s with no inlineData frames mid-generation)
18:17:41 — Generation still silent, WebSocket reconnect triggered

Workaround

We detect zero-audio generations by checking _firstTokenTimestamp when markCurrentGenerationDone fires. If a generation completed with no audio after 2+ seconds, we force a WebSocket reconnect and replay context. This brings our stall rate from ~5-10% of calls to 0%, but adds a ~3-5 second pause during recovery.

The Python LiveKit SDK has a similar workaround: livekit/agents#4249

Relationship to this issue

Your report identifies temperature: 0 as a trigger. Our trigger is different (tool calls at temperature: 0.3), but the server-side symptom is identical: generation completes with zero audio output. This suggests the underlying bug may not be specific to temperature settings but a broader issue with the Live API's audio generation pipeline on gemini-3.1-flash-live-preview.

Also related: googleapis/python-genai#2117 (similar issue on Gemini 2.5)

[dbloembe]

@Varun-S10 What is the ETA for a fix? Our team is using gemini-3.1-flash-live-preview for a production voice agent and this zero-audio generation bug is blocking us from going fully live.

We've built a client-side workaround (WebSocket reconnect on zero-audio detection — details in my comment above), but the underlying server-side issue still affects ~5-10% of calls without it.

[Varun-S10]

Hi @dbloembe, Thank you for bringing this to our attention. We were able to reproduce the issue on our end. We truly appreciate you taking the time to flag this, and we will be filing an internal bug.

[y-lobau]

@dbloembe did you try without setting the temperature? Not sure if it is acceptable in your case, but in ours, it significantly improved the output.

[dbloembe]

@y-lobau Yes, still an issue!

@Varun-S10 Do you have an ETA for this fix? We're hoping to launch this week, and this is blocking us.

[dbloembe]

@Varun-S10 Just bumping this again. Still a big blocker for us.

[smehta29]

Hey folks! Thanks for your patience here.

While @Varun-S10 was able to reproduce the issue with temperature = 0, for the Gemini 3 model family, we strongly recommend keeping the temperature at its default value of 1.0.

@dbloembe Can you share a minimal reproducible code snippet using the default temperature setting that reproduces the zero-audio stall after tool responses? I wasn't able to reproduce this across 40 runs with multi-turn tool calling.