Building InterviewMentor AI: real-time mock interviews with the Gemini Live API

Most interview practice tools give you a list of questions. What candidates actually lack is a realistic conversation — someone who listens, follows up, watches how you code, and tells you afterwards what went well and what didn't. That was the goal for InterviewMentor AI: a mock interviewer you can talk to in real time, built entirely in the browser.

This post walks through how it works. The source is on GitHub.

The product in one paragraph

You pick a target role — Frontend, Backend, DevOps or Product Management — and an interview type. A live session starts: the AI interviewer speaks to you, you answer out loud, and a shared code editor lets it see your code and offer hints. When you finish, the app produces a structured report covering technical accuracy, communication, behavioral signals and non-verbal cues, and a dashboard tracks how you improve over time.

Why a client-only architecture

InterviewMentor AI is a single-page React app with no custom backend. The browser talks directly to Google's Gemini APIs through the @google/genai SDK.

That choice kept the project small and fast to iterate on:

• No server to deploy, scale or secure for the core experience.
• Audio and video never pass through an intermediate service of mine.
• The whole app deploys as static files to Vercel, Netlify or GitHub Pages.

The trade-off is API key handling. A key bundled into a public client can be extracted, so for anything beyond a personal prototype the right next step is a thin token-issuing endpoint or ephemeral tokens rather than a long-lived key in the bundle.

The session flow

The app has four stages:

1. Dashboard — history and quick-start actions.
2. Setup wizard — captures the role and interview type, which become the interviewer's system instructions.
3. Live session — real-time audio and video with the AI interviewer.
4. Analysis — a final structured evaluation of the whole session.

Streaming audio and video to Gemini Live

The live session uses the Gemini Live API, which keeps a WebSocket-style connection open so audio can flow in both directions with low latency. In simplified form:

const session = await ai.live.connect({
  model: LIVE_MODEL,
  config: { responseModalities: [Modality.AUDIO], systemInstruction },
  callbacks: { onmessage: handleServerMessage },
});

Microphone input is captured with the MediaStream API and converted to 16 kHz PCM, the format the Live API expects for audio input. Video is sampled as individual JPEG frames from the camera rather than streamed as a full video track — enough for the model to perceive presence and engagement without sending a continuous high-bitrate stream.

Playing the interviewer back

Responses arrive as audio chunks. These are decoded and scheduled on an AudioContext from the Web Audio API so playback is gapless. The subtle part is timing: each chunk has to be queued to start exactly when the previous one ends, otherwise the interviewer sounds choppy.

Turning a conversation into a report

A conversation is unstructured. A useful report is not. After the session ends, the app aggregates the full transcript and recording and sends a final prompt to Gemini 2.5 Flash, asking for a structured JSON evaluation.

Asking for JSON rather than prose is what makes the rest of the product possible:

• The report UI can render consistent sections every time.
• Scores can be stored and charted with Recharts on the progress dashboard.
• Different sessions become comparable.

What I'd build next

• Ephemeral tokens so the API key never ships to the browser.
• Interviewer personas for different company styles and seniority levels.
• Question banks per role, so the AI's follow-ups are grounded in a curated set rather than generated from scratch.

Privacy note

Audio and video are processed in real time and sent to Google's Gemini API. That's stated in the project README, and it's the kind of thing every AI product that handles a camera and microphone should make obvious to users up front.