INGELLA: a voice AI agent with an avatar that drives Codex and Claude Code

An Ingello case: a live voice agent in the browser with an animated avatar, male and female voices, a pause that saves money, a physical button and Codex and Claude Code as executors. States, settings and diagrams.

INGELLA: a voice AI agent with an avatar that drives Codex and Claude Code
INGELLO / BUSINESS2948
Contact us
INGELLA: a voice AI agent with an avatar, an executor log and a live conversation chart
INGELLA at work: the task is already with the executor, the Fast profile is lit, and the button counts down to auto-pause. Service log lines and model names are blurred.

There is a colleague in our office who never drinks coffee, never goes on vacation and still knows our code very well. Her name is INGELLA. She lives in a browser window, has teal hair and does the one thing that matters: she listens to a person by voice, understands what they want done with the project, hands the job to an AI programmer and briefly reports back.

Technically, INGELLA grew out of our "Real-time AI secretary" project. There we went deep into latency, VAD, interruptions and our lab for live conversation. This article is about something else: the character and the product. What a voice agent you actually enjoy working with every day looks like, which states it goes through, what you can tune and why we turned it into a standalone app with a physical button.

You will not find prompts, decision rules or infrastructure here: that part of the project is closed. Our working combination of voice, models and profiles is closed too. What you will find are real screenshots, two short clips and diagrams.

What INGELLA does

Picture yourself deep in a task and too lazy to switch to the terminal. You say out loud: "Check why yesterday's deploy failed" or "How many markdown files are in the docs folder?". INGELLA says "Looking", hands the task to a local executor agent and, while it works, does not fill the air with small talk. When the result is ready, she keeps it short: "Done, fifteen". She will tell you the details if you ask.

  • A live voice conversation. She listens and speaks at the same time, can drop an "uh-huh" and stops talking when interrupted. No push-to-talk buttons.
  • Two executors to choose from. Tasks are run by a local Codex or a local Claude Code. These are two adapters with the same contract, switched in the interface.
  • Look or edit. The executor runs either in "just look" mode or with permission to change files. When each one is chosen is exactly the closed part.
  • Fast and Strong. Two profiles of the same agent: a nimble one for everyday tasks and a thoughtful one for heavy ones. You can see the profiles on screen as two small faces, and you can hear them too: each has its own voice for progress remarks.
  • Memory for the whole conversation. One executor thread is created per voice session, so the second task continues the first instead of starting from scratch. The thread survives a pause and even a page reload.
  • A pause that saves money. The voice model is billed by session time, and the seconds tick even when both sides are silent. So while the executor is busy with a long job, INGELLA closes the voice session herself and comes back with the result later.

States: what INGELLA looks like at work

The tricky thing about a voice interface is that the person cannot see what is going on. Silence from the speaker can mean anything: the agent is thinking, the agent froze, the agent is sulking. That is why every INGELLA state is readable at a glance: from the buttons, the executor faces, the chart and the transcript.

Eight INGELLA states: before connecting, connecting, greeting, waiting, user speaking, executor at work, paused, a reply in a male voice
1: before connecting, the avatar already blinks and looks around. 2: connecting, the Connect button dims. 3: INGELLA greets first, the green line on the chart is her voice. 4: listening. 5: the user speaks, the blue line is the microphone. 6: the task went to the executor, the Fast face lit up, the auto-pause countdown is running. 7: voice paused, the button offers to resume the conversation. 8: a reply in a male voice, in Ukrainian. The executor log and model names are blurred. The interface is in Russian.

All frames come from live sessions. To avoid sitting at a microphone, we fed a pre-recorded phrase into the browser instead of a mic, and from there INGELLA talked for real: real latency, real "uh-huh"s and a real executor.

Avatar and facial animation

INGELLA's face comes alive when the page loads, not when the session connects: she blinks, shifts her gaze, moves her eyebrows. A small thing, but without it the screen before the first word looked like a passport photo. When INGELLA speaks, her mouth follows the loudness of her voice: the avatar is attached to an analyser on the remote audio stream, so the lips move with the actual sound rather than on a timer.

INGELLA answers "who are you" in Ukrainian. The mouth and eyes are animated in the browser, the green spikes on the chart are her voice. No sound.

A fun detail: INGELLA has fourteen voices, six female and eight male, but only one face. Pick a male voice and the screen still shows a teal-haired young woman speaking in a baritone. We decided this is character, not a bug.

INGELLA speaks in a male voice in Ukrainian, a voice and microphone chart on the right, a transcript below
Male voice, Ukrainian. The live transcript at the bottom fills in as the user and INGELLA speak.

The executor at work

The fun begins when INGELLA hands off a task. The Codex log on the right shows what the executor received, which steps it takes, which commands it runs and how it finished. Progress remarks from the executor ("Checking the contents of the docs folder...") are spoken in a separate voice, so you know the work is going even without looking at the screen.

A task from voice to result, sped up about twice: the user asks to count files, INGELLA hands off the task, the executor works, the voice pauses, the answer appears in the log. Service lines are blurred.

The executor's result goes to the voice model silently. Out loud you only hear a short "done" or "had a look", and INGELLA will retell the full report if asked. Otherwise talking to her turns into reading logs out loud, which is the worst podcast in the world.

INGELLA paused: a Resume conversation button, the executor log shows a command and the answer 15
Voice paused, executor finished: the log shows its steps, the command and the answer (the folder really holds 15 files). Lines with the task wording and models are blurred.

Pause: how not to pay for silence

A real-time voice model is billed per second of session, not per word. While the executor spends a minute digging through code, the voice session dutifully burns money on silence. So a pause in INGELLA is not "mute the microphone", it closes the voice session entirely. The browser keeps everything else: the microphone, the avatar, the chart and the transcript. The log and step voicing keep going.

When the executor finishes, the server wakes the conversation up: a new session picks up the thread, the history and the deferred report, and INGELLA announces that the work is done. Auto-pause does not kick in instantly: resuming costs money too, so on a short task it is cheaper to wait. To keep the automation from looking stuck, the button shows a countdown.

In all honesty: shooting this case found us a race condition. The executor finished in exactly the second the voice was going on pause, so the answer "15" stayed in the log and was never spoken. A great excuse for the case study to double as a tester.

Settings

The top panel is short on purpose. Anything that does not change the user experience is out of sight:

  • Voice. A catalogue of fourteen voices grouped into female and male, labelled with manner and accent.
  • Language. Russian or Ukrainian. Ask her during the conversation to switch to another language and INGELLA will.
  • Executor. Codex or Claude Code.
  • Delegation backend. The model that turns a conversation into a clear task. Which one we use, we will not tell.
  • Instructions. The operator can add wishes about behaviour for today's sessions.
  • Two checkboxes. Whether to voice the executor's steps and whether to enable auto-pause.
The INGELLA extra instructions dialog with a Monday example
Extra instructions: the text is added to new sessions. The example is ours and very Monday.

A standalone app and a physical button

The first version lived in a browser tab, and that turned out to be a problem. A tab gets lost among thirty others, and a voice agent you cannot see feels like a voice from the next room. So INGELLA has a web app manifest: she can be installed as a standalone app with her own icon, no tabs and no address bar. On the work machine her window launches separately and comes up on top of all other windows.

INGELLA also has a button. A real one, on an ESP32 board that talks to the computer over Bluetooth Low Energy. One press does what the situation calls for: no conversation means connect; a conversation means pause; a pause means resume. The meaning of the press is decided by the page, because only the page knows the current state. If both the app window and a forgotten tab are open, the press goes to the window only, otherwise you could accidentally start two parallel paid sessions.

The button taught us humility. Bluetooth likes to get stuck: the board does not always resume advertising after a disconnect, and the system stack sometimes hangs on disconnect so that the service looks alive and does nothing. All of this is now covered by timeouts, a watchdog and automatic restarts, but for the first weeks the button lived a life of its own, like a cat.

INGELLA at phone width: settings, avatar, executor faces, Connect and Stop buttons, log
On a narrow screen the interface folds into a column: on the left the top of the page before connecting, on the right the middle during a conversation. The backend model is blurred.

How it works

Audio flows directly between the browser and the voice model over WebRTC; our server never touches the sound. The voice model runs the conversation and decides when help is needed. The reasoning backend words the task. The INGELLA server is attached to the same session through a second channel, catches the function call and starts the executor on the user's machine. The page only shows the transcript and the log: exactly one side executes the function.

INGELLA contour: person, browser, voice model, reasoning backend, server, executor; step voicing, log, pause, button
System contour. On top, the path of a task; below, what makes working with a voice agent convenient and affordable.
Conversation data structure: voice session, task, step, transcript line, executor thread, report
Conversation data structure. The key idea: one executor thread for the whole conversation, and it survives pauses.
INGELLA interface layout: settings, avatar, executor faces, buttons, log, chart, transcript
Screen layout.

Latency, end-of-turn detectors, interruptions and the lab where we tuned all of it are covered in detail in the "Real-time AI secretary" case.

Why this matters for business

INGELLA drives code because we are programmers and that is our daily pain. But the pattern is universal: voice on top, an executor below, an honest log in between. Instead of Codex it can be your CRM, warehouse, accounting system or internal service. A manager in the car asks "which orders have been stuck since yesterday", a warehouse worker with busy hands says "write off two boxes", a director asks "put together the report and send it to Telegram". Voice becomes a task, the task becomes an action, the action becomes a short answer.

Want a voice agent for your staff, your customers or your internal systems? Let's start with a free consultation with an architect: we will go through your scenario and calculate the price after the call. More about our approach at systems.ingello.com.

Where to see it

INGELLA has no public address: she runs on a developer's machine and drives local agents in their working directory, so we chose not to put her on the internet. We will show a live demo during the consultation.

Technology stack and approaches

SaaS architecture Microservices Cloud computing Software scalability API integrations DevOps processes High-load systems Solution cybersecurity