Gemini Fixed Audio Bug

I’m experimenting with using the Gemini Live API, and have (obviously) “vibe coded” (parts of) what I’m doing.

It worked surprisingly well right away, but there was this annoying audio bug. (Signal Processing is not my forte.)

Then I had an idea for something which I didn’t really think would actually work, but hey, try it anyway, right?

I recorded a short audio clip of the problem, and uploaded it to Gemini, asking it for help. And oh boy, is this impressive or what:

Listen to the attached sound file - can you hear how there is a sort of “click click” which doesn’t seem quite right? Look at the attached main.js, can you spot the bug?

Based on the audio file and the code provided, I have identified the bug. The “click click” artifact you are hearing is caused by gaps in the audio playback due to how the processAudioQueue function handles timing.

The Bug: “Stop-and-Wait” Playback

In your main.js, the playback logic waits for one audio chunk to completely finish playing before it processes and starts the next one.

Why this clicks:

The onended event fires after the audio hardware has finished playing the buffer. The time it takes for JavaScript to catch the event, process the next chunk, create a new source, and connect it creates a tiny period of silence (a gap) between every single audio packet. These periodic gaps sound like “click… click… click.”

The Fix: Precise Scheduling

To get seamless audio, you must schedule the next chunk to start exactly at the moment the previous one ends, without waiting for the onended event.

You need to introduce a nextStartTime variable to track where the audio playhead should be.

Here is the corrected version of the playback logic:

...

Key Changes:

1. Removed source.onended dependence: We no longer wait for the audio to stop to trigger the next piece.
2. nextStartTime Tracking: We calculate exactly when the current chunk ends and schedule the next one to start at that exact microsecond.
3. Recursion: We call processAudioQueue immediately (wrapped in a setTimeout or just called directly if you prefer) to drain the queue as fast as data arrives, rather than pacing it by the playback speed.