§ 00 — LOADING STACK
◆
LangGraphLangGraph
Azure OpenAIAzure OpenAI
QdrantQdrant
Arize PhoenixArize Phoenix
LLangfuse
MMastra
Next.jsNext.js
SSupabase
LANGGRAPH ◆ AZURE ◆ QDRANT ◆ ARIZE ◆ LANGFUSE ◆ MASTRA ◆ NEXT.JS ◆ SUPABASE ◆ LANGGRAPH ◆ AZURE ◆ QDRANT ◆ ARIZE ◆ LANGFUSE ◆ MASTRA ◆ NEXT.JS ◆ SUPABASE
KKomal Vardhan.
HomeWorkAboutWritingResourcesContact
HomeWorkWritingResourcesAboutContact
Build like an engineer. Teach like a friend.

© 2026 Komal Vardhan Lolugu

Sitemap
  • Home
  • Work
  • About
  • Writing
  • Contact
  • Resources
Elsewhere
  • LinkedIn · 3.5K
  • Medium · Writing
  • Instagram
  • GitHub
  • Topmate
Newsletter

A field note every other Sunday. No hype, no AI spam. Unsubscribe anytime.

Designed & built by Komal. Made in India.
← All work
2025 · Agentic AILiveLead Engineer

Memory Agent

A dual-agent conversational platform that turns expert institutional knowledge into structured, searchable guidance — captured through voice and delivered on demand to frontline teams.

70%Knowledge domain coverage at launch
2×Agent types — one that captures, one that retrieves
24/7Expert guidance available to every associate, regardless of shift
0Documents written manually — all knowledge captured through conversation
§ 01

The Problem

Organizations with deep operational expertise carry most of it in people's heads. When experienced specialists leave, that knowledge walks out with them. New team members spend weeks interrupting colleagues for guidance on edge cases, regulatory nuances, and process judgment calls — slowing productivity, increasing error rates, and burning out the senior people they rely on. There was no scalable way to capture what veterans knew and make it continuously available to everyone else.

§ 02

The Solution

Built a dual-agent AI system: one agent listens to subject matter experts and captures their knowledge, the other surfaces it to associates on demand. Experts speak naturally in a real-time voice session — the capture agent extracts structured insights, categorizes them against a domain taxonomy, and builds a knowledge graph linking related concepts. Associates call the assistant agent with questions and receive cited, expert-backed answers drawn from the living knowledge base. Every session is monitored end-to-end for quality and trace analysis.

§ 02b

How it works

01
Expert speaks

A subject matter expert starts a voice session and describes processes, edge cases, and decision patterns in natural language — no forms, no documentation burden.

02
Capture agent extracts

The agent listens in real time, identifies discrete knowledge units, and categorizes each one against a structured taxonomy: process, experiential, decision, regulatory, or product knowledge.

03
Knowledge graph builds

Extracted insights are connected into a graph — concepts, relationships, and dependencies become navigable. Coverage gaps surface automatically for managers to act on.

04
Associate asks

A frontline associate starts a voice session with a question. The assistant agent retrieves relevant nodes from the knowledge graph and responds with grounded, source-attributed guidance.

§ 03

What I Learnt

  • 01

    Defining the taxonomy before building the agent made extraction significantly more reliable — the model needs a schema to extract into, not just a prompt to extract from.

  • 02

    Real-time audio changes the UX contract entirely. Users expect the agent to feel present, not transactional. Streaming partial transcripts rather than waiting for final output was critical for perceived responsiveness.

  • 03

    Observability traces revealed extraction failures that testing alone never caught — specifically on multi-topic utterances. Instrumenting before launch is not optional on voice agents.

  • 04

    The knowledge graph visualisation became the most-used feature by managers — not by design, but because it answered the question everyone actually had: what do we know, and what are we still missing?

§ 04

Technologies Used

Next.js + React 19Next.js + React 19

App Router with server components for dashboard, session views, and knowledge graph UI

OpenAI Realtime API (WebRTC)OpenAI Realtime API (WebRTC)

Browser-to-model audio — speech recognition and synthesis handled natively by the model

tRPC v11tRPC v11

End-to-end type-safe API layer between Next.js frontend and Express backend

MongoDBMongoDB

Persistent store for sessions, knowledge chunks, utterances, and graph nodes and edges

LLangfuse

Full observability — prompt versioning, token tracking, trace analysis, and eval scoring across both agents

Reagraph + Three.jsReagraph + Three.js

Interactive 3D knowledge graph visualisation for domain coverage and concept relationships

Turborepo + BunTurborepo + Bun

Monorepo orchestration with unified dev and build pipeline across frontend, backend, and shared packages

Next.js + React 19Next.js + React 19
OpenAI Realtime API (WebRTC)OpenAI Realtime API (WebRTC)
tRPC v11tRPC v11
MongoDBMongoDB
LLangfuse
Reagraph + Three.jsReagraph + Three.js
Turborepo + BunTurborepo + Bun
← All workWork together ↗
Next →Visit Agent