Target GPUs
H200
1× GPU per bench
Submissions
105
75 passing · 27 rejected
Scoring rule
median e2e speedup
H200
p99 latency
2000 / 50 ms
TTFT / ITL ceiling
Priority metric
Win condition
>=10% GPU-hour reduction at SLA
Scoring rule
Correctness
Precision
Context window
Baseline repo
Baseline commit
Patch visibility
Manifest hash
Base image
Baseline engine
Workload
32 prompts per round from nebius/SWE-agent-trajectories
default/train · 80,036 rows · 5,120 max output tokens
Allowed paths
Denied paths