Realtime, Multiplayer, and Voice

Navaid has three distinct real-time paths:

1. single-user voice through the chat-live Edge Function,
2. multiplayer text through Postgres, Realtime, and chat-multi,
3. multiplayer voice through LiveKit and the persistent agent/ worker.

They share session/message/maturity data but have different connection, authorization, ordering, and failure semantics.

Single-user Voice

Components

SessionSingle / useLiveVoice
  ├─ microphone capture and PCM conversion
  ├─ chat-live WebSocket
  ├─ playback queue / worklet
  ├─ reconnect and Gemini resume handle
  └─ transcript/maturity UI

chat-live
  ├─ query-token Auth validation
  ├─ session/company authorization
  ├─ budget gate and realtime usage recorder
  ├─ Gemini Live relay
  ├─ transcript persistence
  └─ text-model maturity extraction

Connection

The browser calls buildLiveVoiceWsUrl(sessionId,resumeHandle?). Because the WebSocket API cannot set Authorization headers, the URL includes the current access token.

Before upgrading, the function:

1. validates token with auth.getUser,
2. loads the session,
3. verifies active company access,
4. checks budget,
5. loads prompt/history/context.

Never log or send the complete URL to Sentry.

Audio

Browser audio is converted to the PCM format expected by Gemini Live. Returned audio is decoded/queued for playback. Audio worklets are served from frontend/public/, so CSP permits blob: workers/media where needed.

Resume

Gemini can provide a resumption handle. The hook stores it and reconnects after transient failure. Reconnect must:

• acquire a current Auth token,
• avoid overlapping sockets,
• stop old capture/playback resources,
• preserve the session ID,
• fall back to a fresh upstream session when the handle is rejected.

Maturity extraction

Spoken replies do not carry the same reliable structured blocks as text chat. After an assistant turn, chat-live uses the text model to extract maturity signals and applies them with the shared optimistic persistence helper.

Both the Live call and extraction spend must be accounted.

Feature flag

voice_enabled controls the client affordance. chat-live does not currently read the flag. It should not be described as a server kill switch.

Multiplayer Text

Data model

• sessions.is_multiplayer
• session_participants
• attributed messages.user_id
• multiplayer message kind/processing fields
• turn-claim state/RPC

Creation and join

createMultiplayerSession creates a group session and owner participant. Invited users receive participant rows/notifications. can_join_session and session_add_participant constrain self-join paths.

Every participant must also retain active membership of the session company.

Realtime subscription

useMultiplayerSession subscribes to relevant:

• messages
• participants
• presence/status as implemented

On subscription or network recovery, reload durable rows and reconcile by ID. Realtime delivery can be duplicate, delayed, or out of order; it is not a transaction log exposed directly to UI.

AI turn claim

Several connected clients can observe the same user turn. chat-multi calls claim_session_turn() so only one request generates the next assistant turn.

new user messages
  ├─ client A requests AI
  └─ client B requests AI
       └─ claim_session_turn
            ├─ winner drains pending turns and calls Gemini
            └─ loser receives {busy:true}

The winner composes a speaker-aware prompt, persists one assistant reply, updates maturity/tasks/accounting, and releases/advances turn state.

Attribution controls

Authenticated client message inserts are restricted to role='user'. messages_pin_author_user_id() forces user_id = auth.uid(). The service-role assistant pipeline remains exempt.

This prevents one participant from forging another speaker or an assistant message.

Feature flag

multiplayer_enabled is currently a client access flag. Backend operations still authorize participants and membership but do not use it as a universal server kill switch.

Multiplayer Voice

Why a separate worker

LiveKit Agents holds a room connection and a Gemini Live socket for the duration of a session. Supabase Edge Functions are request scoped and unsuitable for that lifecycle.

agent/src/agent.ts is a Node worker using:

• @livekit/agents
• Google Live plugin
• Silero VAD
• Supabase JS service client

Room naming

livekit-token accepts:

• controlled lab-* rooms for the hidden prototype
• session-<uuid> for real group sessions

Arbitrary names are rejected.

Token minting

For a real session room, livekit-token verifies:

• signed-in user
• LiveKit configuration
• group_voice_enabled unless system admin
• existing multiplayer session/company
• active company membership
• owner or participant row
• budget

It returns a short-lived room token, URL, identity, display name, and canonical room.

Agent session authorization

For session-* rooms the agent:

1. parses the session ID,
2. loads session/company/profile/prompt context through the service role,
3. refuses session audio when the owner/session authorization context is invalid,
4. rechecks a speaker's participant/membership relationship before linking and before persisting a turn.

Service-role access makes these checks mandatory.

Speaker attribution

Participant identity and metadata are minted by the token function. The worker:

• tracks active human speakers,
• links agent input to the active participant,
• binds a transcript segment to the speaker when it opens,
• falls back carefully for final-only transcripts,
• drops turns without a trustworthy speaker,
• revalidates the speaker before message insert.

Transcript persistence

Human and assistant turns are inserted into messages with:

• session ID
• correct role
• attributed user where applicable
• voice source metadata
• interruption marker for interrupted assistant output

Messages then flow to connected clients through Realtime.

Maturity context

The worker composes instructions from:

• active v2 advisor prompt
• company facts and maturity summary
• UK-readiness scope guardrail
• session identity

It persists speech; group maturity processing remains coordinated with application session flows rather than allowing arbitrary model writes.

Usage accounting

The worker estimates Live usage conservatively, inserts audit rows, and calls record_token_usage. It rechecks authorization and budget during the session so a room token is not indefinite authority.

Current dispatch caveat

The worker README notes that scoping dispatch exclusively through named/explicit session-* dispatch is future hardening. Do not claim it is already complete.

Leave and Remove

livekit-session-control is the only authoritative leave/remove path.

Action	Allowed caller
Leave self	current participant
Remove other	session owner or company admin/consultant/system admin

The function:

1. verifies session/company/action,
2. deletes participant state with service privilege,
3. calls LiveKit to remove all matching connected identities,
4. reports whether media-plane removal occurred.

A direct table delete would leave an already-connected participant in the room.

Notifications

notify-session-add sends branded participant mail only after confirming:

• caller identity,
• session existence,
• caller's owner/admin authority,
• target is owner or has a participant row,
• trusted APP_URL.

This prevents using Navaid mail as an arbitrary-recipient spam relay.

Failure and Recovery

Realtime disconnect

• show connection state,
• resubscribe once,
• reload durable rows,
• dedupe by ID,
• do not generate AI solely from an unverified duplicate event.

Gemini Live disconnect

• stop/replace the upstream socket,
• attempt one bounded resume path,
• preserve already-persisted transcripts,
• flush usage,
• avoid replaying already-committed turns.

LiveKit agent absent

Humans may connect without an AI participant. UI should show agent readiness rather than pretending the assistant is listening.

Membership revoked mid-room

Short room-token TTL limits initial authority. Agent/function checks and authoritative session control must remove/refuse the participant promptly.

Duplicate assistant replies

Inspect claim_session_turn use and whether multiple clients bypassed the request helper.

Local Development

Single voice:

npx supabase functions serve --no-verify-jwt
cd frontend
npm run dev

Group voice additionally:

cd agent
pnpm dev

Required LiveKit/Gemini/Supabase values are documented in Getting Started.

Use synthetic companies/transcripts for live acceptance tests.

Security Checklist

• no access token in logs/analytics
• current auth.getUser before WebSocket upgrade
• active company membership
• participant/owner binding
• strict room-name validation
• short token TTL
• speaker identity not taken from transcript text
• membership recheck before persistence
• leave/remove affects database and media plane
• prompt scope/tenant context validated
• usage recorder single-instance and flushed
• canonical CORS/CSP origins

Extension Guidance

When adding a real-time event:

1. define the durable row/state first,
2. make event handling idempotent,
3. tolerate duplicate/out-of-order delivery,
4. establish one-winner semantics for side effects,
5. bind actor and tenant server-side,
6. specify reconnect/replay behaviour,
7. account provider spend,
8. test two tabs and two users concurrently.