Skip to main content

Ruby Jha

Engineering Manager · Applied AI · Cloud The next decade belongs to engineering leaders who build with AI.

I spent the last two decades leading engineering teams at State Street, Centene, and EY. Building products that handle real money, real patient data, and real regulatory scrutiny. The kind of work where downtime means someone's claim doesn't get processed and a bad deployment means financial loss.

Now I am applying that same engineering discipline to AI. I am building 9 AI systems covering RAG pipelines, embedding fine-tuning, and multi-agent orchestration. Each one has evaluation frameworks, architecture decision records, and metrics I would trust in a code review. The same standards I would hold any production system to.

The Full Stack

Leadership

People Management Hiring & Team Building Performance & Promotions Executive Communication Technical Strategy

Technical

Python Java TypeScript OpenAI API LangChain CrewAI FastAPI ChromaDB Azure Docker Kubernetes React Spring Boot Astro

Featured Projects

What I'm building

Project 01
Demo: streamlit Completed

Synthetic Data Generation Pipeline

I built a pipeline that generates synthetic training data, validates it with an LLM judge, and self-corrects until every record passes. Started with a 20% failure rate, ended at zero.

Python Pydantic OpenAI API GPT-4o-mini GPT-4o +1
Project 02
Demo: streamlit Completed

RAG Evaluation Pipeline

I tested 16 RAG configurations and found that semantic chunking + OpenAI embeddings + Cohere reranking gets 0.747 Recall@5 on structured Markdown docs. This is how I got there.

Python LangChain RAGAS Sentence-Transformers Braintrust +4
Project 03
Completed

Contrastive Embedding Fine-Tuning

I fine-tuned all-MiniLM-L6-v2 on 1,475 dating profile pairs and flipped Spearman from -0.22 to +0.85. LoRA got 96.9% of that using 0.32% of the parameters.

Python Sentence-Transformers PEFT/LoRA PyTorch UMAP +2
Project 05
Completed

ShopTalk Knowledge Management Agent

I built a RAG system from scratch with no LangChain, tested 46 configurations across 5 chunking strategies, 4 embedding models, and 3 retrieval methods, and found that heading-aware chunking + OpenAI embeddings hits NDCG@5 = 0.896 and Recall@5 = 1.0.

Python PyMuPDF FAISS SentenceTransformers OpenAI +6
Project 06
Completed

Torvalds Digital Clone

Multi-agent system that answers CS questions in the voice of Linus Torvalds or Greg Kroah-Hartman, with a deterministic gate that declines rather than hallucinates. 78.6% and 81.0% in-domain deliver rates, 11 of 12 out-of-domain queries correctly refused, zero hallucinations.

Python CrewAI OpenAI Cohere FAISS +5

Latest Blog Posts

llm-evaluation Jul 6, 2026

No Model Passed the Bake-Off. I Shipped One Anyway.

Choosing a groundedness metric when no leaderboard covers your corpus, every candidate fails your gates, and the bias you ship has to be documented instead of hidden.

10 min read

llm-evaluation Jun 22, 2026

The Clone Was Fine. My Groundedness Metric Was Measuring Copying.

I was two days from rewriting a working system to fix a deficit that did not exist. Cosine similarity was scoring vocabulary reuse, not truth.

7 min read

multi-agent Jun 8, 2026

I Put an LLM Where a Comparison Operator Belonged

My ship gate came back 0 for 14, three passes, zero variance. The thresholds existed only as text inside a prompt, and the LLM reading them could not do arithmetic.

9 min read

More about my background