Skip to main content

AI and Token Accounting

Navaid uses Gemini for advisor conversations, Aida, summaries, company extraction, feedback structuring, grounded research, task generation, live voice, and guided-demo TTS. Every spending path must be:

  1. authorized,
  2. bounded by input/rate/budget controls,
  3. called through a canonical model configuration,
  4. recorded in the per-call audit table, and
  5. rolled into the atomic monthly aggregate.

Skipping accounting is a security and cost-control defect.

Canonical Models​

supabase/functions/_shared/models.ts is the source of truth:

ConstantDefaultOverride
GEMINI_TEXT_MODELgemini-3.1-flash-litesource change
GEMINI_LIVE_MODELgemini-3.1-flash-live-previewGEMINI_LIVE_MODEL
GEMINI_LIVE_VOICE_DEFAULTKoreGEMINI_LIVE_VOICE
GEMINI_TTS_MODELgemini-3.1-flash-tts-previewGEMINI_TTS_MODEL

Text/TTS functions use GoogleGenAI from https://esm.sh/@google/genai@2.8.0. Do not add the retired @google/generative-ai SDK or copy model IDs into each function.

Environment overrides exist for rapid provider rollback. Record the active value during incident response; source alone may not identify the hosted model.

AI Surfaces​

SurfaceProvider modeCompany context
chattextsession company
chat-liveLive plus text signal extractionsession company
chat-multitextgroup-session company
generate-summarytext plus task generationsession company
aidatext/tool loopselected/authorized company
aida-liveLiveselected/authorized company
competitor-analysisgrounded text/searchrequested authorized company
lead-huntgrounded fan-outplatform/user context
synthesise-company-infotext extractionadmin-authorized company
structure-feedbacktext extractionuser/platform
gemini-ttsTTSuser/platform
prompt-testtextsystem-admin test
_shared/tasks.tstext generationcalled within chat/summary
LiveKit agent/Livevalidated session company

Budget Hierarchy​

The effective monthly per-user input/output budget is:

users.token_budget_* override
else companies.default_user_token_budget_*
else 100,000

The gate reads current-month token_usage_monthly and blocks when either input or output usage has reached its respective cap.

Master switch​

system_config.company_budgets_enabled defaults on. The interpreter is fail-safe:

  • explicit {enabled:false} disables tenant budget checks,
  • missing/unreadable configuration keeps budgets enabled.

The Admin UI also stores daily_token_budget and alert_config, but current main has no runtime consumer that enforces the daily value or sends alerts from that configuration. They are configuration surfaces, not active spend controls. Do not claim platform-wide daily enforcement until a runtime path is implemented and tested.

Pre-call gate​

assertWithinBudget(supabase,userId,companyId) returns a ready 429 response or null. Use companyId:null only for genuinely companyless/platform surfaces.

The current shared gate is called by:

aida, aida-live, chat, chat-live, chat-multi,
competitor-analysis, generate-summary, gemini-tts, lead-hunt,
livekit-token, structure-feedback, synthesise-company-info

prompt-test is system-admin-only and records usage, but does not use the ordinary tenant gate.

TOCTOU limitation​

The pre-call check is a read, not a reservation. Several concurrent calls can pass the same remaining balance before they record usage. Fixed-window per-surface rate limits bound that amplification. Do not remove rate limits because a monthly budget exists.

Rate Limits​

assertRateLimited() calls service-role-only check_rate_limit(bucket,max,window_seconds).

Examples in current source include:

SurfaceWindow
chat30/minute
Aida60/minute
competitor analysis6/5 minutes
summary8/5 minutes
company synthesis12/5 minutes
lead hunt6/5 minutes
TTS20/minute

Public preflight/share/early-access functions use trusted cf-connecting-ip-based buckets. They do not trust client-supplied x-forwarded-for.

Rate-counter database errors currently fail open for authenticated AI helpers because the durable monthly budget remains. Log and monitor such failures.

Accounting Data Flow​

Gemini response or realtime usage sample
├─ INSERT token_usage
│ user, company, session/message, input, output, model/surface
│
└─ RPC record_token_usage(...)
├─ atomically upsert token_usage_daily
└─ atomically upsert token_usage_monthly

next request
└─ assertWithinBudget reads monthly aggregate

The audit row and aggregate serve different purposes. Writing one does not substitute for the other.

Failure handling​

Provider success followed by accounting failure is dangerous. Functions should:

  • attempt both writes,
  • log aggregate failures with surface context but no sensitive content,
  • avoid throwing away the successful user response solely because telemetry failed unless the surface's transactional design requires it,
  • alert on accounting failures,
  • make retries idempotent where possible.

REST Usage Metadata​

Text model responses expose usage metadata. Normalize provider fields into:

  • input tokens
  • output tokens

Do not count prompt text characters as tokens when provider metadata is available. Include nested task-generation/refinement calls; accounting only the outer advisor call leaves a blind spot.

Realtime Usage​

The Live WebSocket paths do not provide the same reliable per-session metadata. _shared/live-usage.ts conservatively estimates:

  • 6,000 input tokens/minute
  • 3,000 output tokens/minute
  • sampled every 30 seconds

It:

  1. inserts audit usage,
  2. calls record_token_usage,
  3. rechecks the budget,
  4. invokes a budget-exceeded callback,
  5. flushes the final partial interval on stop.

The LiveKit agent uses corresponding conservative accounting. These values are intentionally over-counting until provider metadata becomes reliable.

Task-generation Accounting​

_shared/tasks.ts can make additional Gemini calls from chat or summary. When changing task generation:

  • determine whether usage is returned to the caller or folded into parent totals,
  • ensure every call appears exactly once in audit/aggregate totals,
  • avoid recursive unbounded generation,
  • rate/dedupe task generation before provider work.

Cost Display​

frontend/src/utils/formatters.ts contains a display-only estimate, currently annotated for Gemini 3.1 Flash Lite March 2026 pricing:

input $0.25 / million tokens
output $1.50 / million tokens

This estimate is not billing authority and can become stale. Date-stamp updates and distinguish:

  • provider bill,
  • budget tokens,
  • conservative Live estimates,
  • displayed estimated cost.

Do not use the display formatter to enforce budgets.

Prompt Management​

Production text advisor calls load the active v2 row from prompt_versions. Prompt updates are versioned and audit logged.

Prompt Lab:

  • stores draft session state in prompt-lab-session-v1,
  • clears it on Save and Cancel,
  • calls prompt-test,
  • does not insert a production conversation,
  • still spends and records tokens.

Prompts should contain:

  • scope and refusal rules,
  • output schema,
  • current maturity/task/company context,
  • locale directive,
  • tool constraints where relevant,
  • instructions not to reveal system content.

Grounded Research​

Competitor analysis and lead hunt use Google Search grounding. Grounding adds failure modes:

  • source URLs can be unavailable or malicious,
  • returned text can wrap JSON in prose,
  • fan-out multiplies cost,
  • results can be stale or contradictory.

Controls include:

  • bounded query/result/fan-out counts,
  • safe URL handling and server-side fetch policy,
  • robust JSON extraction,
  • retryable controlled 502 for unparseable results,
  • rate/budget checks before fan-out,
  • source attribution in stored/displayed results.

Security and Privacy​

Never send more company data than needed. Provider prompts can include confidential strategy, transcripts, and extracted documents.

  • keep keys server-side,
  • do not log prompts/responses in production by default,
  • redact error telemetry,
  • restrict service-role context loads by verified tenant,
  • cap attachment/document text,
  • treat model output as untrusted,
  • avoid exposing hidden reasoning/system prompts,
  • honor account deletion/export obligations.

Adding an AI Function​

Checklist:

  1. verify user or define a narrow public capability,
  2. bind company/session/document identifiers,
  3. validate and bound input before provider work,
  4. import the canonical model,
  5. add rate limit,
  6. add input and output budget gate,
  7. parse/validate output defensively,
  8. insert token_usage,
  9. call record_token_usage,
  10. return generic provider/internal failures,
  11. add tests for malformed output, cross-tenant access, rate and budget,
  12. update function and accounting matrices.

Troubleshooting​

Budget indicator differs from usage report​

Check whether the UI is reading the correct company/user hierarchy and whether all surfaces call the atomic RPC.

Voice reaches budget too quickly​

Realtime values are deliberately conservative estimates. Verify sample timing and duplicate recorder instances before changing rates.

AI call succeeds with no audit row​

Treat as a release blocker. Find early returns or nested provider calls that bypass both accounting steps.

Rate limit can be bypassed​

Ensure buckets use verified user.id for authenticated surfaces and trusted gateway IP for public surfaces.