Evaluating multi-agent systems when ground truth is incomplete
Three eval modes — output, trajectory, and side effect — and the practical pattern we use when there's no single right answer.
#agents
#eval
#production-ml
Imran Ahamed
I build applied AI/ML systems. Currently co-founding VEZRAN — agentic AI for security operations. Previously: Senior Data Scientist at Starbucks and FedEx. Half my career was pre-LLM ML.
Focused on multi-agent systems and RAG pipelines that survive contact with production. I write about the unglamorous parts — eval design, infrastructure, what actually ships.
Writing
all →Projects
all →Zyberpol — agentic SOC platform
Co-founder · Head of AI/ML · VEZRANCo-founder and Head of AI/ML. Multi-agent autonomy on top of an existing security stack, with audit-ready evidence for every action.
GPT-4oClaudeMulti-agentRAGAzurePython
zyberpol.com ↗
Contact-center NLP at Starbucks
Senior Data Scientist · Starbucks · People AnalyticsTransformer-based topic modeling, summarization, and semantic search over partner contact-center data.
BERTT5TransformersPythonAzure Databricks
Package-at-risk and loss-intensity models at FedEx
Senior Data Scientist · FedExProduction ML across the logistics network. Identified at-risk shipments before they failed; quantified customer-level loss exposure.
PythonAzure MLEnsemble ModelsReal-time Inference
Newsletter
Field notes on applied AI — agent eval, RAG in production, what the demos leave out. No threads. No hot takes.
TODO: wire form action to Substack / Buttondown