Tag

Evaluation

Posts tagged Evaluation, newest first.

TrackerUpdated Oct 29, 2026

AI agent reliability benchmarks, tracked

How reliably the best AI agents succeed when it counts, in plain language: success on every try, the length of task they can handle, and realistic office work. From the official leaderboards, updated when they change.

Ivan Bardak

Benchmarks

Oct 29, 202613 min read

Agentic AI from demo to production: what the evidence says about the architecture

Most agent pilots stall on the architecture around the model, not the model. What surveys, published incidents and reliability benchmarks show about tools, identity, retrieval, reversibility and measurement, and the six questions I ask before anything ships.

Ivan Bardak

Oct 29, 20266 min read

GraphRAG vs RAG: what the evidence actually says

GraphRAG wins on multi-hop and whole-corpus questions and loses on plain lookups, at a higher cost. What three studies show, how much weight each deserves, and a decision rule you can use.

Ivan Bardak

Oct 29, 20266 min read

A GraphRAG business case, done the evidence-based way

Most GraphRAG business cases are a vendor benchmark and a hopeful percentage. A structure that survives a CFO's questions: six steps of evidence-based management, a cost model from measured token counts, and a value range instead of one number.

Ivan Bardak

Oct 29, 20266 min read

Does AI make developers faster? What the controlled studies say

55% faster, 21% faster, 19% slower, roughly even: the controlled studies disagree for good reasons. What each one measured, how much weight it deserves, and what to put in an AI transformation plan instead of a single number.

Ivan Bardak

Oct 29, 20266 min read

MCP tool design: fewer, safer, testable tools

Accuracy falls as the tool list grows, and every major vendor now says keep it small. What the MCP specification, the vendors and independent benchmarks say about tool design, and the checklist I use to review an MCP server.

Ivan Bardak

Oct 29, 20265 min read

"95% of AI pilots fail": what the MIT report actually measured

The most quoted number in AI strategy decks comes from a preliminary report built on 52 interviews, and it means four different things. What the MIT NANDA report measured, what it gets right, and what larger surveys say instead.

Ivan Bardak

Enterprise