SIYA, a governed agent system that answers a real phone line

Small businesses lose jobs to missed calls, and most "AI agent" products either do too little (a chatbot with a script) or too much (agents acting autonomously with nobody accountable). SIYA had to solve both halves: a phone that always answers, and agents that can discover and prepare work — but never act on money, contracts or deletion without human sign-off.

SIYA is a loop, not a script. Ten agents run under one orchestration layer, and every pass through the loop leaves an auditable trail:

1. Research — agents study the business, its channels, competitors and search presence
2. Evidence — findings are recorded with sources — claims without evidence don't advance
3. Memory — leads, calls, outcomes and decisions persist across 55 database tables
4. Tools — telephony, email drafting, CRM, calendars, web research — each tool-scoped per agent
5. Decision — the orchestrator proposes the next best action as a typed intent
6. Approval — intents touching money, contracts, sending or deletion queue for a human
7. Action — approved actions execute; everything lands in the audit log
8. Outcome & learning — results are tracked and feed the next recommendation

Every agent intent is classified into an authority level. Reading and researching runs freely; drafting runs freely but output is held; anything that would contact someone, spend money, sign, delete or deploy is intercepted and queued for human approval. Email is drafted but never auto-sent. Secrets are redacted from all agent output. The approvals ledger and audit log are first-class features, not afterthoughts.

Twilio's ConversationRelay hands you a raw WebSocket and no framework. The server is an RFC 6455 WebSocket implementation built on Python's standard library — zero third-party dependencies, including Twilio's HMAC-SHA1 request-signature validation. One gap needed closing: ConversationRelay's WebSocket carries no signature scheme of its own, so the TwiML response mints a single-use, 60-second relay token that the WebSocket handshake must present. On top of that: real-time barge-in (interrupt SIYA mid-sentence and she stops), DTMF handling, and latency tight enough for a live phone conversation.

SIYA runs as a Windows service behind a Cloudflare Tunnel that whitelists a handful of API routes and 404s everything else at the edge, with a watchdog process, dated database snapshots and a documented backup/rollback procedure. Operational ledgers show real usage: 1,200+ processed messages, 187 human approval decisions, and a live call log. The codebase carries 200+ test files and 300+ commits over ten weeks, including commits by a governed overnight agent whose work is reviewed under the same approval model as everything else.

I can build real-time voice AI at the protocol level, design multi-agent systems with genuine governance rather than vibes, and operate the result in production — solo, end to end.