feature · r2-32-eval-harnesses

Eval harnesses

Taxonomy

M

5 tags · 8 out · 8 in · 3 artifacts · 1 gaps · 1 corpus docs

Catalog tags

group
r2
ev:inferred
maps.dm
absent
status:absent · ev:inferred
maps.technologies
8
ev:primary
slug
r2-32-eval-harnesses
ev:primary
title
Eval harnesses
ev:inferred

What it is

How should agent quality be measured, gated, and promoted — combining deterministic checks, calibrated LLM-as-judge, and statistical regression CI — without blocking every PR or trusting uncalibrated vibes?

Why it matters

This topic was frozen in the corporate agentic OS research corpus (wave 10) to answer architecture and measurement questions that sit under the locked loop (observe→understand→predict→intervene→resolve→verify→learn). Stage-1 criticality: M. Research/Spec/Platform readiness: R5/S4/P0.

Related features

Testing, Debugging & Optimization, Cost, Latency & Operational Economics, Test, Debug & Evaluate

Catalog links

Graph edges for this feature — technologies, category lens, companies, DM.

Technologies

features/r2-32-eval-harnesses/README.md

Artifacts

  • dm_mapabsent
  • readmepresent · features/r2-32-eval-harnesses/README.md
  • technologies_mappresent · features/r2-32-eval-harnesses/technologies.md

Gaps

  • G_DM_CROSSWALKsev 3

    feature missing document-management.md

Out · maps_to (8)

  • abm-agent-behavior-miningtechnology · primary · features/_source/feature-technology-map.json
  • agent-evaluationtechnology · primary · features/_source/feature-technology-map.json
  • deepevaltechnology · primary · features/_source/feature-technology-map.json
  • hal-the-holistic-agent-leaderboardtechnology · primary · features/_source/feature-technology-map.json
  • pm4pytechnology · primary · features/_source/feature-technology-map.json
  • promptfootechnology · primary · features/_source/feature-technology-map.json
  • ragastechnology · primary · features/_source/feature-technology-map.json
  • trulenstechnology · primary · features/_source/feature-technology-map.json

In · maps_to (8)

  • abm-agent-behavior-miningtechnology · primary · features/_source/feature-technology-map.json
  • agent-evaluationtechnology · primary · features/_source/feature-technology-map.json
  • deepevaltechnology · primary · features/_source/feature-technology-map.json
  • hal-the-holistic-agent-leaderboardtechnology · primary · features/_source/feature-technology-map.json
  • pm4pytechnology · primary · features/_source/feature-technology-map.json
  • promptfootechnology · primary · features/_source/feature-technology-map.json
  • ragastechnology · primary · features/_source/feature-technology-map.json
  • trulenstechnology · primary · features/_source/feature-technology-map.json

Corpus tags (document_tags)

feature
r2-32-eval-harnesses
kind
feature_readme
quality
ok
slug
r2-32-eval-harnesses
technology
abm-agent-behavior-mining
agent-evaluation
deepeval
hal-the-holistic-agent-leaderboard
pm4py
promptfoo
ragas
trulens

Corpus documents (1)

feature_readme · 1

  • T32 — Eval Harnesses & CI Gates for Agents · ok

    features/r2-32-eval-harnesses/README.md

Research path: features/r2-32-eval-harnesses