Skip to content
Seller Lab

Seller Lab 1 · The lab

Why real-time voice AI is hard

Juniper is a LiveKit voice agent that answers the phone for a made-up restaurant. First you make it your own. Then each experiment plays a recorded test caller through two settings, so you hear the difference with the same input, and you try it with your own voice. Use headphones.

At the end, put it to work on a scored practice call.

No call, or no access code? Every experiment below has a recording of both test-caller runs, made on a real call. Lock in a prediction and play them to see what happened and the question to ask a customer.

Start here

Make it your own

An agent’s instructions are plain sentences the LLM reads before every reply. Changing them is the quickest way to see what a voice agent is.

Predict
You add one rule to the instructions: “Mention tonight’s special before taking a booking.” What changes?
Do this

Rename the restaurant, add one rule and press Apply. Juniper greets you as the new restaurant. Then book a table, with your own voice or the test caller’s booking.

Lock in a prediction first.

Call Juniper to start.

Test-caller runs in the experiments switch back to the original Juniper, so everyone hears the same calls.

Notice. Your rule shows up in what the agent says. How fast it answers and how well it hears you stay the same, because instructions only steer the LLM. The timing table at the bottom shows the pause staying about the same. The experiments below cover everything around the LLM.

See the instructions the LLM reads
Call Juniper, then open this again to load them.

In the full program, each seller builds this agent in Agent Builder, LiveKit’s no-code builder, in a LiveKit Cloud project of their own. This page uses two fields instead, because Agent Builder needs a LiveKit account. See the program.

Experiment 1 · Turn detection

Knowing when you’re done

The agent can’t see you, so it has to guess when you’ve finished talking. Silence alone is a weak guess, because people pause in the middle of a thought.

Baseline
Ends your turn after a short silence.
Enhanced
Adds LiveKit’s turn detector.
  1. Step 1 · Predict

    A caller says, “I’d like a table for… four people on… Friday at seven,” pausing mid-thought. What will the baseline agent do?
  2. Step 2 · Run the test caller

    The test caller books a table and stops to think twice, for about two seconds each time: “I’d like a table for… four people on… Friday at seven.”

    Not on a call? Play a recording instead: the same test caller on a real call, recorded earlier.

    Lock in a prediction first.

    Baseline

    Not run yet.

    Enhanced

    Not run yet.

  3. Step 3 · Try it yourself

    Then say it yourself, pausing a second or two after “for” and after “on,” the way you would if you were checking your calendar. Turn on “Use my voice” in the call bar, then pick a setting:

    Listen for. Whether the agent starts talking before “seven.” The results show which words each setting heard before it replied. Results can vary between tries, so compare the test-caller runs first; they give both settings the same input.

  4. Step 4 · Take it to a call

    The answer and the question to ask open once both runs are done.

Experiment 2 · Interruptions

“Uh-huh” isn’t a stop sign

When you talk while the agent is talking, it has a split second to decide: are you taking the turn, or just showing you’re listening?

Baseline
Stops the agent for any speech that lasts half a second.
Enhanced
Adds LiveKit’s adaptive interruption handling.
  1. Step 1 · Predict

    While the agent reads the menu, the caller says “uh-huh.” What will the baseline agent do?
  2. Step 2 · Run the test caller

    The test caller asks “What’s on the menu tonight?”, says “uh-huh” twice while the agent answers, then cuts in: “Wait, do you have anything vegetarian?”

    Not on a call? Play a recording instead: the same test caller on a real call, recorded earlier.

    Lock in a prediction first.

    Baseline

    Not run yet.

    Enhanced

    Not run yet.

  3. Step 3 · Try it yourself

    Then try it yourself: say “uh-huh” while Juniper reads the menu, the way you would on a phone call. A very quick one can be too short to stop either setting. Turn on “Use my voice” in the call bar, then pick a setting:

    Listen for. Whether the agent stops at “uh-huh,” and whether it stops when the caller really cuts in. Both settings should stop for the real question; the difference is the “uh-huh.” When no words follow a stop, the agent picks its sentence back up after about two seconds, so Baseline stalls rather than stopping for good.

  4. Step 4 · Take it to a call

    The answer and the question to ask open once both runs are done.

Experiment 3 · Noise

The coffee shop

Noise does more than make the caller hard to hear. Other people’s voices end up in the transcript as if the caller had said them.

Baseline
Hears the caller’s audio as it comes in.
Enhanced
Runs ai-coustics voice isolation on it first.
  1. Step 1 · Predict

    A caller books a table from a busy café. In the baseline, what goes wrong first?
  2. Step 2 · Run the test caller

    The test caller books a table from a busy café. A barista calls out an order before the caller speaks, and someone at the next table starts talking about a game just after.

    Not on a call? Play a recording instead: the same test caller on a real call, recorded earlier.

    Lock in a prediction first.

    Baseline

    Not run yet.

    Enhanced

    Not run yet.

  3. Step 3 · Try it yourself

    Then press Café noise and book a table yourself. While you talk, the page mixes the same café into your microphone. Turn on “Use my voice” in the call bar, then pick a setting:

    Listen for. Words in the transcript that nobody said to the agent, and whether it answers them. The transcripts tell the story better than the sound does.

  4. Step 4 · Take it to a call

    The answer and the question to ask open once both runs are done.

Experiment 4 · Model

The pause

The pause before each answer is a relay of several steps, and callers feel the total.

Fast model
Answers with Gemma 4 31B, a quick LLM.
Reasoning model
Swaps in Grok 4.20 Reasoning, an LLM that thinks before it answers.
  1. Step 1 · Predict

    When you switch to the reasoning model, which part of the pause grows?
  2. Step 2 · Run the test caller

    The test caller asks, “What time do you open on Saturday?”

    Not on a call? Play a recording instead: the same test caller on a real call, recorded earlier.

    Lock in a prediction first.

    Fast model

    Not run yet.

    Reasoning model

    Not run yet.

  3. Step 3 · Try it yourself

    Then ask it yourself a few times. Turn on “Use my voice” in the call bar, then pick a setting:

    Listen for. The beat before the answer. After each turn, the timing table at the bottom shows the measured pause and its parts, so compare the numbers as well as the feel. The gap varies from turn to turn, which is why the page keeps every run.

  4. Step 4 · Take it to a call

    The answer and the question to ask open once both runs are done.

Prove it: the practice call

An 8 to 10 minute discovery call with Priya, an AI prospect. It’s scored on what this lab teaches, and a coach talks you through the result.

Go to the practice call

LiveKit’s per-turn metrics

Every turn, measured

The timing comes from LiveKit’s per-turn metrics, measured on the agent. What you hear also includes network time. The total pause runs from when the caller stopped talking to when Juniper started answering.

Turns appear here once the call starts.

Calls in this lab aren’t recorded. Each one ends on its own after 40 minutes.

Not connected