Choosing a groundedness metric when no leaderboard covers your corpus, every candidate fails your gates, and the bias you ship has to be documented instead of hidden.
I was two days from rewriting a working system to fix a deficit that did not exist. Cosine similarity was scoring vocabulary reuse, not truth.
My ship gate came back 0 for 14, three passes, zero variance. The thresholds existed only as text inside a prompt, and the LLM reading them could not do arithmetic.
A mocked unit test passed on every commit while the live Cohere call failed on every run. What that gap cost, and the three-layer methodology that closes it.
One day of evaluation produced three null results. Every one was a measurement artifact, and each nearly became a documented fact.
How Instructor, flat schemas, and two-phase validation got me to 100% structured output success across 580 LLM-generated records.
I fine-tuned all-MiniLM-L6-v2 on dating profiles, flipped Spearman from -0.22 to +0.85, and found LoRA hit 96.2% of that with 0.32% of parameters.
Grid search across 16 RAG configurations reveals embedding model selection drives 26% more retrieval quality than chunk tuning.
My first LLM judge had a 0% failure rate. That meant it was useless. This is the story of calibrating it to actually catch failures, and building a correction loop that took synthetic data failures from 36 to zero.