Micheal Lanham·Oct 27Reflective LLM Judges: Checklists, Verifiers and Dispute-Only ReviewEvals Are Your IP, Day 7 of 10: a judge’s second look improves the grade when it reconciles against the record, not when it rereads its own…A response icon1A response icon1
PUNETI HEMANTH KUMAR REDDY·Oct 11LLM-as-a-Judge?Over the last few years, AI systems have risen rapidly — from coding agents and workflow automation to voice agents and autonomous systems…A response icon1A response icon1
Sigmoid·Sep 30What your LLM judge is measuring and what it is notKey takeawaysA response icon1A response icon1
InDatabricks for StartupsbyYuval Danino·Sep 21See it, Grade it, Query it: Running an AI Agent like a Real Production System on DatabricksObservability, evaluation, and governance for AI agents, without ever moving your data off the lakehouse
Sravanth·Sep 11LLM-as-Judge for Knowledge Graph Triplet VerificationBuild an independent supported/contradicted/insufficient-evidence verifier, add deterministic gates, and evaluate whether LLM-as-judge…
Arun Balaraman·Aug 14LLM-as-Judge: Using a Model to Grade a ModelUnit tests and schema checks can tell you whether an agent’s output is well-formed, but not whether it’s good. For things like plan…
Chris Hughes·Aug 1Can You Trust the Model Grading Your Model?An LLM judge is a fallible instrument with its own error rate. You don’t have to trust it blindly — or re-grade everything by hand. You…
Srinivas Bommena·Jul 23OpenRouter Fusion Is Not Model Routing But It Is Deliberation Infrastructure.I did not start testing OpenRouter Fusion because I wanted another model router. I already had enough ways to choose models. Some tasks…
QuarkAndCode·Jul 10Human Evaluation vs Automated Evaluation: Reliability in AI TestingEvaluation is not just about assigning a score to a system. It is about deciding whether that score can be trusted.
Alan Hsu·Jun 9Using LLM-as-a-Judge for AI Systems EvaluationThe Hardest Part of Building AI Systems Isn’t Generation. It’s Evaluation!
