A linter for
agent behavior.

Every eval platform tests your agent.
GAMA tests your tests.

pip install gama-ai

harness run

$ harness run Generated: 6/6 passed (100%)  threshold 80%Holdout:   2/2 passed (100%)  threshold 70% ┏━━━━━━━━━┳━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━┓┃ Case    ┃ Result ┃ Scorer                ┃ Latency ┃┡━━━━━━━━━╇━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━┩│ seed-1  │ PASS   │ exact_match           │   95 ms ││ gen-001 │ PASS   │ llm_judge             │   88 ms ││ gen-002 │ PASS   │ exact_match→llm_judge │  104 ms ││ gen-004 │ PASS   │ llm_judge             │  100 ms │└─────────┴────────┴───────────────────────┴─────────┘All thresholds met.

How it works

Grounded checks first. The judge only when they can't decide.

A pass is final

Deterministic and free.

A failure escalates

The judge is told why, and can overturn it.

A crash is not negotiable

But declining bad input is scored, not punished.

The part nobody ships

A holdout set you can't tune against.

Generated evals refresh as your agent improves. The holdout is created once, and the suggestion engine never sees it.

# evals/
generated.json   refreshes freely
holdout.json     created once

thresholds:
  pass_rate: 0.8
  holdout_pass_rate: 0.7

Receipts

It found a bug in its own author's code.

50%before

100%after

“Those error messages are all reasonable, well-formed refusals for the inputs given. That pattern points at the eval harness rather than at the agent's instructions.”

harness suggest, unedited

Get started

$ pip install gama-ai
$ harness init
$ harness gen-evals
$ harness run

No API key for init, run, report
Plain JSON in your repo. No database.
Exits 1 on regression, so CI fails the build
Any language, over stdin or HTTP

Read the docs

A linter foragent behavior.