Seller Lab 1 · The lab
Why real-time voice AI is hard
Juniper is a LiveKit voice agent that answers the phone for a made-up restaurant. First you make it your own. Then each experiment plays a recorded test caller through two settings, so you hear the difference with the same input, and you try it with your own voice. Use headphones.
At the end, put it to work on a scored practice call.
Start here
Make it your own
An agent’s instructions are plain sentences the LLM reads before every reply. Changing them is the quickest way to see what a voice agent is.
Rename the restaurant, add one rule and press Apply. Juniper greets you as the new restaurant. Then book a table, with your own voice or the test caller’s booking.
Lock in a prediction first.
Call Juniper to start.
Test-caller runs in the experiments switch back to the original Juniper, so everyone hears the same calls.
Notice. Your rule shows up in what the agent says. How fast it answers and how well it hears you stay the same, because instructions only steer the LLM. The timing table at the bottom shows the pause staying about the same. The experiments below cover everything around the LLM.
See the instructions the LLM reads
Call Juniper, then open this again to load them.
In the full program, each seller builds this agent in Agent Builder, LiveKit’s no-code builder, in a LiveKit Cloud project of their own. This page uses two fields instead, because Agent Builder needs a LiveKit account. See the program.
Experiment 1 · Turn detection
Knowing when you’re done
The agent can’t see you, so it has to guess when you’ve finished talking. Silence alone is a weak guess, because people pause in the middle of a thought.
- Baseline
- Ends your turn after a short silence.
- Enhanced
- Adds LiveKit’s turn detector.
Step 1 · Predict
Step 2 · Run the test caller
The test caller books a table and stops to think twice, for about two seconds each time: “I’d like a table for… four people on… Friday at seven.”
Not on a call? Play a recording instead: the same test caller on a real call, recorded earlier.
Lock in a prediction first.
Baseline
Not run yet.
Enhanced
Not run yet.
Step 3 · Try it yourself
Then say it yourself, pausing a second or two after “for” and after “on,” the way you would if you were checking your calendar. Turn on “Use my voice” in the call bar, then pick a setting:
Listen for. Whether the agent starts talking before “seven.” The results show which words each setting heard before it replied. Results can vary between tries, so compare the test-caller runs first; they give both settings the same input.
Step 4 · Take it to a call
The answer and the question to ask open once both runs are done.
Experiment 2 · Interruptions
“Uh-huh” isn’t a stop sign
When you talk while the agent is talking, it has a split second to decide: are you taking the turn, or just showing you’re listening?
- Baseline
- Stops the agent for any speech that lasts half a second.
- Enhanced
- Adds LiveKit’s adaptive interruption handling.
Step 1 · Predict
Step 2 · Run the test caller
The test caller asks “What’s on the menu tonight?”, says “uh-huh” twice while the agent answers, then cuts in: “Wait, do you have anything vegetarian?”
Not on a call? Play a recording instead: the same test caller on a real call, recorded earlier.
Lock in a prediction first.
Baseline
Not run yet.
Enhanced
Not run yet.
Step 3 · Try it yourself
Then try it yourself: say “uh-huh” while Juniper reads the menu, the way you would on a phone call. A very quick one can be too short to stop either setting. Turn on “Use my voice” in the call bar, then pick a setting:
Listen for. Whether the agent stops at “uh-huh,” and whether it stops when the caller really cuts in. Both settings should stop for the real question; the difference is the “uh-huh.” When no words follow a stop, the agent picks its sentence back up after about two seconds, so Baseline stalls rather than stopping for good.
Step 4 · Take it to a call
The answer and the question to ask open once both runs are done.
Experiment 3 · Noise
The coffee shop
Noise does more than make the caller hard to hear. Other people’s voices end up in the transcript as if the caller had said them.
- Baseline
- Hears the caller’s audio as it comes in.
- Enhanced
- Runs ai-coustics voice isolation on it first.
Step 1 · Predict
Step 2 · Run the test caller
The test caller books a table from a busy café. A barista calls out an order before the caller speaks, and someone at the next table starts talking about a game just after.
Not on a call? Play a recording instead: the same test caller on a real call, recorded earlier.
Lock in a prediction first.
Baseline
Not run yet.
Enhanced
Not run yet.
Step 3 · Try it yourself
Then press Café noise and book a table yourself. While you talk, the page mixes the same café into your microphone. Turn on “Use my voice” in the call bar, then pick a setting:
Listen for. Words in the transcript that nobody said to the agent, and whether it answers them. The transcripts tell the story better than the sound does.
Step 4 · Take it to a call
The answer and the question to ask open once both runs are done.
Experiment 4 · Model
The pause
The pause before each answer is a relay of several steps, and callers feel the total.
- Fast model
- Answers with Gemma 4 31B, a quick LLM.
- Reasoning model
- Swaps in Grok 4.20 Reasoning, an LLM that thinks before it answers.
Step 1 · Predict
Step 2 · Run the test caller
The test caller asks, “What time do you open on Saturday?”
Not on a call? Play a recording instead: the same test caller on a real call, recorded earlier.
Lock in a prediction first.
Fast model
Not run yet.
Reasoning model
Not run yet.
Step 3 · Try it yourself
Then ask it yourself a few times. Turn on “Use my voice” in the call bar, then pick a setting:
Listen for. The beat before the answer. After each turn, the timing table at the bottom shows the measured pause and its parts, so compare the numbers as well as the feel. The gap varies from turn to turn, which is why the page keeps every run.
Step 4 · Take it to a call
The answer and the question to ask open once both runs are done.
Prove it: the practice call
An 8 to 10 minute discovery call with Priya, an AI prospect. It’s scored on what this lab teaches, and a coach talks you through the result.
LiveKit’s per-turn metrics
Every turn, measured
The timing comes from LiveKit’s per-turn metrics, measured on the agent. What you hear also includes network time. The total pause runs from when the caller stopped talking to when Juniper started answering.
Turns appear here once the call starts.
Calls in this lab aren’t recorded. Each one ends on its own after 40 minutes.