Entries
5
4 scored · 1 disqualified
Settled in
58m 47s
seated Sep 09, 17:34 UTC
Top score
0.8188
0 is baseline speed
Leader
Held
incumbent kept the lead
5
| Role | Miner | Patch | Status | Score | Duration | Open |
|---|---|---|---|---|---|---|
| baseline | — | — | scored | 0 score breakdown | — | |
| leader |
| 0e4e1b0…61104 |
| scored |
| 0.8393 score breakdown |
| — |
| challenger | 2f68f1b…11475 | scored | 0.8188 score breakdown | — |
| challenger | 35cc8ae…1f20e | scored | 0.8092 score breakdown | — |
| challenger | 38c9879…9b2eb | disqualifiedengine died before becoming healthy (last error: [Errno 111] Connection refused); container log tail: WARNING 09-09 18:20:47 [cuda.py:45] Failed to import from vllm. qutlass C: ModuleNotFoundError("No module named 'vllm. qutlass C'") Traceback (most recent call last): File "<frozen runpy>", line 198, in run module as main File "<frozen runpy>", line 88, in run code File "/src/vllm/entrypoints/openai/api server.py", line 34, in <module> from vllm.entrypoints.openai.serving warmup import warm up generation paths ModuleNotFoundError: No module named 'vllm.entrypoints.openai.serving warmup' | — score breakdown | — |
19/19 steps
Setup
4/4Waiting on a provider to hand over a machine.
Installing the harness and shipping the repo.
Fetching the baseline and candidate engine images.
Staging the campaign model onto the pod.
Baseline
2/2Loading weights and waiting for the server to answer.
Incumbent
2/2Loading weights and waiting for the server to answer.
Candidate 1 of 3
2/2Loading weights and waiting for the server to answer.
Candidate 2 of 3
2/2Loading weights and waiting for the server to answer.
Candidate 3 of 3
2/2Loading weights and waiting for the server to answer.
Scorer
2/2Loading weights and waiting for the server to answer.
Drift baseline
2/2Loading weights and waiting for the server to answer.
Teardown
1/1Destroying the pod and its volume.
Full workload replay against the latency gates.
Full workload replay against the latency gates.
Full workload replay against the latency gates.
Full workload replay against the latency gates.
Full workload replay against the latency gates.
The pinned baseline image teacher-forces captured outputs against the campaign thresholds.
Full workload replay against the latency gates.