AskME

A voice-first mock interviewer that actually read your resume.

Most interview prep is a list of canned questions. AskME goes the other way: you hand it your real resume (PDF) and the real job description, and Gemini 2.5 Pro builds a plan around your gaps — then an AI interviewer conducts the whole session out loud, on your phone. You talk, she talks back, and her mood shifts with the quality of your answers. Ramble nonsense and Victoria goes from Impressed to Skeptical to Furious — 14 emotional states driven by live anger/engagement meters, reflected in what she says and how her voice sounds.

The entire app is built around one constraint: a spoken conversation dies if the interviewer takes six seconds to respond. Most of the ~33k lines of TypeScript here exist to make the reply feel immediate — a fast intent-classification pass, answer scoring and speech generation running in parallel, and TTS that starts streaming on the first sentence of the reply instead of waiting for the full text.

React Native 0.81 / Expo SDK 54, New Architecture enabled. iOS and Android.

What it does

• Personalized interview plan. Upload a resume PDF (sent to Gemini as inline data — no text-extraction step) and paste a job description. Gemini 2.5 Pro extracts the role, your top matching skills, your critical gaps, standout skills, and relevant soft skills — and generates a scenario question for each ("Your React app has a memory leak in a large list. How do you debug it?"). Four session lengths: short, medium, long, freestyle.
• Two interviewers.
  • Victoria — strict but fair. Anger and engagement meters (0–100) map to 14 vibes from Impressed to Furious; each vibe changes speech speed, voice emotion, and the prompt that generates her reply.
  • Sensei — a patient mentor (Fish Audio male voice) with two sub-modes: depth (work a topic until mastered) and sprint (two attempts, move on). A progressive hint system escalates from Socratic nudges to direct reveals, and answers earn mastery ratings (expert / master / learned / needs_review).
• Full voice loop. Mic capture as PCM16 via react-native-audio-api, transcription through OpenAI Whisper, replies spoken through streaming TTS — Fish Audio (default) or Deepgram Aura, switchable in-app.
• Scored results. Per-question scores and feedback, an overall summary, favorites, and session history persisted on-device (JSON via expo-file-system), with share/copy export and audio replay of the interviewer's lines.
• A serious debug overlay. Triple-tap to open: live latency metrics, anger/engagement sliders to force any vibe, TTS diagnostics, and a Gemini-powered simulated candidate that answers questions for you so you can test the loop hands-free.

How a turn works

resume.pdf + job description
        |
        v
Gemini 2.5 Pro --> interview plan (matches, gaps, standout + soft
        |           skills -- each with a scenario question)
        v
per question:
  mic (PCM16) -> Whisper STT -> intent classifier (small, fast prompt)
                                      |
                     +----------------+----------------+
                     |            in parallel          |
                     v                                 v
        Gemini 2.5 Flash scores            interviewer's spoken reply,
        the answer                         streamed sentence-by-sentence
                                                       |
                                                       v
              Fish Audio / Deepgram WebSocket (PCM16 stream)
                -> jitter buffer -> zero-crossing alignment
                -> cross-fade at sentence boundaries -> speaker
        |
        v
final report -> history, favorites, audio replay

The audio pipeline under src/utils/audio/ (FIFO queue, jitter buffer, resampler, zero-crossing aligner, crossfade) is the hard-won part — the repo's md/ and plans/ folders contain the multi-round battle logs against clicks, truncation, and microphone deadlocks, kept as a development journal.

Project layout

App.tsx                     stack navigator: interview + two dev test screens
src/
  screens/                  VoiceInterviewScreen (main UI), audio/Gemini test pages
  hooks/interview/          useInterviewLogic orchestrator, per-phase handlers,
                            streaming voice, state, latency metrics
  services/
    gemini/                 client, intent/answer/final evaluators, prompts,
                            voice response generator, simulated candidate
    audio/                  BaseStreamingPlayer + Fish Audio / Deepgram players,
                            pre-generation LRU cache, replay player
    sensei/                 Sensei evaluation + vibe logic
    vibe-calculator.ts      anger/engagement -> vibe config
    history-storage.ts      on-device session persistence
    transcription-service.ts  Whisper STT
  components/               interview UI, DebugOverlay, ResultsModal, history
                            panel, SVG voice waveform, avatars
  utils/audio/              PCM16 pipeline: FIFO queue, jitter buffer,
                            resampler, zero-crossing aligner
  interview-planner.ts      resume + JD -> question plan (Gemini 2.5 Pro)
  types.ts                  all shared types
__tests__/                  Jest suites (services layer)

Running it

You need Node + npm, and Xcode (iOS) or Android Studio + SDK (Android). The app uses native modules that are not in Expo Go — build the dev client with expo run. Native ios/ and android/ folders are gitignored and regenerated by the build.

npm install                # .npmrc already sets legacy-peer-deps

# create .env in the repo root (see table below)

npx expo run:ios           # or: npx expo run:android
# builds the dev client, starts Metro, launches the simulator/device

npx expo start             # subsequent runs, once the dev client is installed

Environment variables

Create .env in the repo root with these names (values from each provider's dashboard):

Variable	Needed for
EXPO_PUBLIC_GEMINI_API_KEY	interview planning, answer evaluation, reply generation — the app's core
EXPO_PUBLIC_OPENAI_API_KEY	Whisper speech-to-text (your answers)
EXPO_PUBLIC_FISH_AUDIO_API_KEY	default TTS provider; required for Sensei mode
EXPO_PUBLIC_FISH_AUDIO_VOICE_ID	optional custom Fish Audio voice
EXPO_PUBLIC_DEEPGRAM_API_KEY	optional — Deepgram Aura as the alternative TTS provider

Note: EXPO_PUBLIC_* variables are inlined into the JS bundle at build time. Every AI call is made directly from the device with your keys. That is fine for a personal dev build; do not distribute a binary built with keys you care about.

Tests

There is no test script in package.json; run Jest directly:

npx jest

8 suites / 33 test cases, covering the services layer (answer evaluator, intent classifier, streaming player base, TTS service, vibe calculator, interview state, handler routing).

Status and limitations

This is a personal project under active, fast-moving development — honest edges below:

• Thin test coverage. 33 test cases against ~150 source files (~33k lines). The audio pipeline and UI layer are essentially untested; the tuned buffer thresholds in the streaming players are protected by convention (md/FISH_AUDIO_DO_NOT_MODIFY.md), not by tests.
• Client-side keys. No backend — Gemini, OpenAI, Fish Audio, and Deepgram are all called from the device, so API keys live in the app bundle. Built for personal use, not for store distribution as-is.
• iOS and Android only. An npm run web script exists, but the mic capture and streaming audio stack are native; web is not a supported target.
• Dormant code paths. Cartesia and OpenAI-TTS integrations exist in the codebase but are not wired into the in-app provider selector (live options: Fish Audio, Deepgram). A Three.js "Crystal Mic" redesign is staged (DESIGN_PLAN.md, assets/models/) but not yet rendered by the app — the three / @react-three/* / expo-gl dependencies are ahead of the UI.
• No CI. Tests run locally only.