BoondiA WhatsApp concierge that sells mithai, safely.
A WhatsApp assistant for Bombay Sweet Shop that answers every customer with live prices and stock, and logs the corporate orders the team used to miss.
- Role
- Lead AI full-stack engineer
- Timeline
- May → Jul 2026 · shipped to production
- Team
- Built solo · two engineers joined later for prompt and engineering maintenance
- Focus
- Production LLM agent stack for Bombay Sweet Shop (Hunger Inc.)
- Stack
- TypeScript·Node 24·Claude (Sonnet 5, Haiku)·Gantry·MCP·PostgreSQL 16 + pgvector·Next.js·ECS Fargate·Terraform·OpenTelemetry
- prompt-cache hit rate83.8% CHEAPER THAN LIST PRICE · 7-DAY WINDOW
- 94.6%
- concurrent turns load testedCAPACITY LADDER ON REAL ECS + RDS
- 285
- live prompt scenarios passingSIGNED-WEBHOOK E2E
- 43/43
- median cost per conversation
- $0.02
Say only true things. Miss no lead.
Bombay Sweet Shop gets a constant stream of WhatsApp messages: order status, bulk gifting briefs, store hours, whether a hamper survives a flight to Dubai, and the occasional melted box. Human 'Mithai Magicians' answered all of it, and around every big occasion the volume outran the team: slow evenings, unanswered threads, and potential customers and corporate deals lost.
The second problem hid inside the first. Corporate gifting orders and bulk festive briefs arrived as ordinary chats, and an underwater team had no time to spot them, let alone log them. So the brief gave the Sweet Concierge two jobs of equal weight: answer every message with only true things, every price, stock level and delivery claim from live Shopify data and a person one message away; and read every conversation, extract the lead and land it in the CRM on its own.
A month in, a second persona arrived: Mr. Sweetu, a co-branded concierge for Kotak Solitaire cardholders, sharing the same runtime and webhook but a different identity and lead schema.
Every reply carries its own receipt.
Four threads from the simulation console: a corporate gifting brief, a policy and store question, a question about one sweet, and a delivery that arrived crushed and ended in a handoff. Every reply shows what the runtime did to produce it: latency, new tokens, cost, and any tool the model called. Pick a scenario, or let it play through.
- queueWAITING IN QUEUE235 ms 8%
- main LLM · turn 1CLAUDE-SONNET-5 · TOOL_USE2.6 s 89%
- gapHAND-OFF / OVERHEAD1 ms 0%
- human_handoffBOONDI-CRM · TOOL CALL85 ms 3%
Boondi · Bombay Sweet Shop
SIMULATION · +000 937 289 298 · ONLINE
Architected and built, end to end.
I architected the entire system and built it end to end: the agent host on Gantry, the open-source runtime I helped build with its core team; both MCP servers; the CRM pipeline; the prompts and skills for both personas; the admin console; the Terraform; and the GitHub Actions CI/CD pipeline that ships it all to ECS. From first commit to production in twelve weeks. Two engineers joined later for maintenance, one on prompts and one on engineering.
The calls that shaped it: signed caller identity instead of trusting phone numbers in prompts, a read-only tool surface enforced at boot, DB-lease-based message ordering so concurrent workers never reply out of order, a conversation cap enforced in code rather than in the prompt, and running the load test against real AWS instead of local Docker.
Agent
- Anthropic Claude API: Sonnet 5 (chat), Haiku (extraction, digests)
- Gantry runtime, API mode
- Prompt = identity + soul + runtime + 5 skills
- Two personas on one webhook (Boondi, Mr. Sweetu)
Services
- boondi host (Node 24, TypeScript)
- mcp-shopify (read-only Admin GraphQL)
- mcp-crm (leads, handoff, outbox)
- admin-panel (Next.js 14)
Data
- PostgreSQL 16 + pgvector
- RDS Proxy
- 36 forward-only migrations
- DB-lease message ordering
- Read-only admin pool
Infra
- ECS Fargate (ARM64)
- ALB, S3, CloudFront
- Terraform (27 modules)
- GitHub Actions ordered deploys
- OpenTelemetry → CloudWatch
Four services, one database, signed everything.
Four independently deployable services around one Postgres. WhatsApp traffic enters through a signed SagePilot webhook, the runtime picks the persona per channel, tools are typed MCP servers with verified identity, and everything the admin console shows is read from the same database the runtime writes.
- 01
Channel
- SagePilot WhatsApp webhook
- HMAC-signed bodies
- Per-channel prompt variant
- 000-prefixed simulation numbers
- 02
Runtime
- Gantry host process
- Worker pool + DB leases
- Memory digests & recall
- Message cap → forced handoff
- 03
Skills
- Gifting · bulk and corporate briefs
- Order · status, tracking, changes
- Customer support · hours, serviceability, escalation
- Product care · melted boxes and hampers in transit
- Store policy · returns, delivery, payments
- 04
Tools (MCP)
- get_orders / get_products / validate_discount_code
- get_store_details / get_catalogues / get_pincode_serviceability
- human_handoff
- X-Caller-Identity HMAC, 120 s replay window
- 05
Pipelines
- Lead extraction (Haiku) from session digests
- Superleap outbox with leases and retries
- Slack cost alerts with dead-letter
- Handoff AI summary notes
- 06
Console
- Simulations & customers
- Per-message latency and cost
- Leads, comments, analytics
- Runtime tokens & roles
The four problems that earned their own runbooks.
- CHALLENGE 01
Identity you cannot prompt your way around
Any phone number inside a customer message is untrusted. The Shopify tools have no phone or email arguments at all.
The channel adapter verifies the upstream webhook signature, then signs an X-Caller-Identity header (phone, timestamp, HMAC-SHA256) for every downstream MCP call. The server rejects stale or malformed headers with a customer-safe message and keeps the verified identity in AsyncLocalStorage, so tool handlers read it without it ever appearing in tool arguments. The tool surface is checked at boot against a forbidden-write regex; a write tool cannot even start.
- CHALLENGE 02
Grounding without a round trip per turn
No price, stock or delivery claim unless a tool confirmed it. But calling Shopify on every turn would blow the Admin API rate limit.
mcp-shopify keeps one in-process catalogue snapshot: every ACTIVE and ARCHIVED product loaded with cursor pagination, refreshed every ten minutes, swapped atomically, never overlapping, with a failed refresh keeping the last good snapshot. Search runs on a token-based fuzzy index built from the snapshot's own vocabulary, so 'Lovers Hamper' resolves and 'petrol mithai' returns nothing. The prompt then forbids internal vocabulary, so the customer never hears about matches or results.
- CHALLENGE 03
Finding the real concurrency knee
Local Docker cannot exercise ECS autoscaling, ALB draining or RDS Proxy. So the load test ran against the real prod cluster with faked upstreams.
Two stand-in services, fake-anthropic and fake-shopify, sit asleep at desired_count 0. A load test is a put-parameter plus a redeploy: repoint the model gateway and Shopify base URL, verify both models show up in the fake's logs, then run a concurrency staircase of 5, 10, 20, 40, 80 conversations polling conversation_turns for completion. The first knee was not CPU or memory: it was a single global advisory lock, with a full pool as the symptom. The next ladder showed the pool itself as the ceiling at 285 concurrent turns, so it went from 40 to 100, still under the proxy's measured 167 backend connections. Event-loop lag was tracked the whole way; Boondi is I/O-bound and its CPU barely moved.
- CHALLENGE 04
A handoff that never drops a customer
When the model should not answer, a human must take over inside WhatsApp, with context, during business hours, without duplicate assignments.
Handoffs are rows with leases: the runtime claims one, checks SagePilot member availability and the 10:00–20:00 IST window, assigns the ticket and posts a Haiku-written private summary note for the agent. A deterministic 35-message cap forces a handoff before conversations degrade, enforced in code so the threshold lives in exactly one place. Release, handback and resume-context are all idempotent, because the whole pipeline assumes retries, duplicate delivery and a task being killed mid-flight.
THE REAL THING: PER-TURN LATENCY REPORT IN THE CONSOLE
Everything between the webhook and the dashboard.
- 01
Agent host
Boots the runtime with the Boondi agent, persona variants per channel, memory digests, the conversation cap and the handoff controller.
- 02
mcp-shopify
Read-only Shopify Admin MCP server: orders scoped to the verified sender, product search over a cached snapshot, discount validation. Tool names verified at boot.
- 03
mcp-crm
Lead extraction from session digests with tier rules, Superleap outbox dispatcher, handoff service, store-details scraper, catalogues on S3/CloudFront, cost alerts to Slack.
- 04
Admin console
Next.js dashboard with simulations, read-only customer threads, live SSE updates with durable cursors, per-message latency waterfalls, response-comment review threads, leads board, usage analytics and runtime token management.
- 05
Infra & delivery
Terraform for VPC, ECS, RDS + Proxy, ALB, CloudFront, SSM, CloudWatch dashboards. One GitHub Actions workflow that detects changed paths, builds, plans, migrates and deploys CRM first, then the rest.
- 06
Load-test harness
Fake Anthropic and Shopify services, a staircase generator, and the runbooks that made a load test a two-command operation instead of a weekend.
- SCROLL ↕
Analytics, 7-day window: 7,100+ model calls, 94.6% cache hit rate ANALYTICS, 7-DAY WINDOW: 7,100+ MODEL CALLS, 94.6% CACHE HIT RATE 
Sales-qualified leads auto-captured from chats, tiered by budget SALES-QUALIFIED LEADS AUTO-CAPTURED FROM CHATS, TIERED BY BUDGET 
Runtime: scoped control-API tokens and role permissions RUNTIME: SCOPED CONTROL-API TOKENS AND ROLE PERMISSIONS
EVERY SCREEN SHIPS IN LIGHT AND DARK · PICK ONE, OR LET IT SWITCH
1,200+ leads in two months. 250+ queries a day.
Boondi answers in a few seconds with prices it can prove, captures gifting briefs into the CRM without anyone typing them in, and hands over to a person with a summary already written. The console lets non-engineers tune prompts safely: every change is verified against 43 live scenarios before it ships.
- 1,200+ leads extracted from conversations in the first two months; 250+ customer queries handled a day
- 94.6% prompt-cache hit rate and 65× cache reuse, cutting model spend 83.8% against list price
- Median turn 3.5 s including tool calls; p95 ≈ 6 s at 80 concurrent conversations
- 285 concurrent turns sustained on one ARM64 task after the pool change, with both machines still idle
- Zero real messages sent from a simulation, enforced by the channel adapter rather than a flag
- 95 test files, 22 Playwright specs and 36 migrations later, still one command to run locally
“The first week it caught a bulk wedding order we would have lost in a busy evening chat.”OPERATIONS LEAD, BOMBAY SWEET SHOP (PARAPHRASED)
NOTE · THE UAT CONSOLE SITS BEHIND AUTH, SO THESE SCREENSHOTS WERE CAPTURED FROM AN AUTHENTICATED SESSION; NAMES AND NUMBERS SHOWN ARE SIMULATION DATA.