AI and Token Accounting

Navaid uses Gemini for advisor conversations, Aida, summaries, company extraction, feedback structuring, grounded research, task generation, live voice, and guided-demo TTS. Every spending path must be:

authorized,
bounded by input/rate/budget controls,
called through a canonical model configuration,
recorded in the per-call audit table, and
rolled into the atomic monthly aggregate.

Skipping accounting is a security and cost-control defect.

Canonical Models

supabase/functions/_shared/models.ts is the source of truth:

Constant	Default	Override
`GEMINI_TEXT_MODEL`	`gemini-3.1-flash-lite`	source change
`GEMINI_LIVE_MODEL`	`gemini-3.1-flash-live-preview`	`GEMINI_LIVE_MODEL`
`GEMINI_LIVE_VOICE_DEFAULT`	`Kore`	`GEMINI_LIVE_VOICE`
`GEMINI_TTS_MODEL`	`gemini-3.1-flash-tts-preview`	`GEMINI_TTS_MODEL`

Text/TTS functions use GoogleGenAI from https://esm.sh/@google/genai@2.8.0. Do not add the retired @google/generative-ai SDK or copy model IDs into each function.

Environment overrides exist for rapid provider rollback. Record the active value during incident response; source alone may not identify the hosted model.

AI Surfaces

Surface	Provider mode	Company context
`chat`	text	session company
`chat-live`	Live plus text signal extraction	session company
`chat-multi`	text	group-session company
`generate-summary`	text plus task generation	session company
`aida`	text/tool loop	selected/authorized company
`aida-live`	Live	selected/authorized company
`competitor-analysis`	grounded text/search	requested authorized company
`lead-hunt`	grounded fan-out	platform/user context
`synthesise-company-info`	text extraction	admin-authorized company
`structure-feedback`	text extraction	user/platform
`gemini-tts`	TTS	user/platform
`prompt-test`	text	system-admin test
`_shared/tasks.ts`	text generation	called within chat/summary
LiveKit `agent/`	Live	validated session company

Budget Hierarchy

The effective monthly per-user input/output budget is:

users.token_budget_* override
  else companies.default_user_token_budget_*
  else 100,000

The gate reads current-month token_usage_monthly and blocks when either input or output usage has reached its respective cap.

Master switch

system_config.company_budgets_enabled defaults on. The interpreter is fail-safe:

explicit {enabled:false} disables tenant budget checks,
missing/unreadable configuration keeps budgets enabled.

The Admin UI also stores daily_token_budget and alert_config, but current main has no runtime consumer that enforces the daily value or sends alerts from that configuration. They are configuration surfaces, not active spend controls. Do not claim platform-wide daily enforcement until a runtime path is implemented and tested.

Pre-call gate

assertWithinBudget(supabase,userId,companyId) returns a ready 429 response or null. Use companyId:null only for genuinely companyless/platform surfaces.

The current shared gate is called by:

aida, aida-live, chat, chat-live, chat-multi,
competitor-analysis, generate-summary, gemini-tts, lead-hunt,
livekit-token, structure-feedback, synthesise-company-info

prompt-test is system-admin-only and records usage, but does not use the ordinary tenant gate.

TOCTOU limitation

The pre-call check is a read, not a reservation. Several concurrent calls can pass the same remaining balance before they record usage. Fixed-window per-surface rate limits bound that amplification. Do not remove rate limits because a monthly budget exists.

Rate Limits

assertRateLimited() calls service-role-only check_rate_limit(bucket,max,window_seconds).

Examples in current source include:

Surface	Window
chat	30/minute
Aida	60/minute
competitor analysis	6/5 minutes
summary	8/5 minutes
company synthesis	12/5 minutes
lead hunt	6/5 minutes
TTS	20/minute

Public preflight/share/early-access functions use trusted cf-connecting-ip-based buckets. They do not trust client-supplied x-forwarded-for.

Rate-counter database errors currently fail open for authenticated AI helpers because the durable monthly budget remains. Log and monitor such failures.

Accounting Data Flow

Gemini response or realtime usage sample
  ├─ INSERT token_usage
  │    user, company, session/message, input, output, model/surface
  │
  └─ RPC record_token_usage(...)
       ├─ atomically upsert token_usage_daily
       └─ atomically upsert token_usage_monthly

next request
  └─ assertWithinBudget reads monthly aggregate

The audit row and aggregate serve different purposes. Writing one does not substitute for the other.

Failure handling

Provider success followed by accounting failure is dangerous. Functions should:

attempt both writes,
log aggregate failures with surface context but no sensitive content,
avoid throwing away the successful user response solely because telemetry failed unless the surface's transactional design requires it,
alert on accounting failures,
make retries idempotent where possible.

REST Usage Metadata

Text model responses expose usage metadata. Normalize provider fields into:

input tokens
output tokens

Do not count prompt text characters as tokens when provider metadata is available. Include nested task-generation/refinement calls; accounting only the outer advisor call leaves a blind spot.

Realtime Usage

The Live WebSocket paths do not provide the same reliable per-session metadata. _shared/live-usage.ts conservatively estimates:

6,000 input tokens/minute
3,000 output tokens/minute
sampled every 30 seconds

It:

inserts audit usage,
calls record_token_usage,
rechecks the budget,
invokes a budget-exceeded callback,
flushes the final partial interval on stop.

The LiveKit agent uses corresponding conservative accounting. These values are intentionally over-counting until provider metadata becomes reliable.

Task-generation Accounting

_shared/tasks.ts can make additional Gemini calls from chat or summary. When changing task generation:

determine whether usage is returned to the caller or folded into parent totals,
ensure every call appears exactly once in audit/aggregate totals,
avoid recursive unbounded generation,
rate/dedupe task generation before provider work.

Cost Display

frontend/src/utils/formatters.ts contains a display-only estimate, currently annotated for Gemini 3.1 Flash Lite March 2026 pricing:

input  $0.25 / million tokens
output $1.50 / million tokens

This estimate is not billing authority and can become stale. Date-stamp updates and distinguish:

provider bill,
budget tokens,
conservative Live estimates,
displayed estimated cost.

Do not use the display formatter to enforce budgets.

Prompt Management

Production text advisor calls load the active v2 row from prompt_versions. Prompt updates are versioned and audit logged.

Prompt Lab:

stores draft session state in prompt-lab-session-v1,
clears it on Save and Cancel,
calls prompt-test,
does not insert a production conversation,
still spends and records tokens.

Prompts should contain:

scope and refusal rules,
output schema,
current maturity/task/company context,
locale directive,
tool constraints where relevant,
instructions not to reveal system content.

Grounded Research

Competitor analysis and lead hunt use Google Search grounding. Grounding adds failure modes:

source URLs can be unavailable or malicious,
returned text can wrap JSON in prose,
fan-out multiplies cost,
results can be stale or contradictory.

Controls include:

bounded query/result/fan-out counts,
safe URL handling and server-side fetch policy,
robust JSON extraction,
retryable controlled 502 for unparseable results,
rate/budget checks before fan-out,
source attribution in stored/displayed results.

Security and Privacy

Never send more company data than needed. Provider prompts can include confidential strategy, transcripts, and extracted documents.

keep keys server-side,
do not log prompts/responses in production by default,
redact error telemetry,
restrict service-role context loads by verified tenant,
cap attachment/document text,
treat model output as untrusted,
avoid exposing hidden reasoning/system prompts,
honor account deletion/export obligations.

Adding an AI Function

Checklist:

verify user or define a narrow public capability,
bind company/session/document identifiers,
validate and bound input before provider work,
import the canonical model,
add rate limit,
add input and output budget gate,
parse/validate output defensively,
insert token_usage,
call record_token_usage,
return generic provider/internal failures,
add tests for malformed output, cross-tenant access, rate and budget,
update function and accounting matrices.

Troubleshooting

Budget indicator differs from usage report

Check whether the UI is reading the correct company/user hierarchy and whether all surfaces call the atomic RPC.

Voice reaches budget too quickly

Realtime values are deliberately conservative estimates. Verify sample timing and duplicate recorder instances before changing rates.

AI call succeeds with no audit row

Treat as a release blocker. Find early returns or nested provider calls that bypass both accounting steps.

Rate limit can be bypassed

Ensure buckets use verified user.id for authenticated surfaces and trusted gateway IP for public surfaces.

Canonical Models​

AI Surfaces​

Budget Hierarchy​

Master switch​

Pre-call gate​

TOCTOU limitation​

Rate Limits​

Accounting Data Flow​

Failure handling​

REST Usage Metadata​

Realtime Usage​

Task-generation Accounting​

Cost Display​

Prompt Management​

Grounded Research​

Security and Privacy​

Adding an AI Function​

Troubleshooting​

Budget indicator differs from usage report​

Voice reaches budget too quickly​

AI call succeeds with no audit row​

Rate limit can be bypassed​