Round score
0.6975
median speedup minus reliability deduction
Prompts scored
27 / 30
median_e2e_speedup
Below tolerance
3
answered too little to score
Zeroed total
3
scored 0.0 without measuring
Temperature range
Request interval
Thinking
Context limit
The same inputs and output limits apply to every engine. Actual concurrency depends on response duration and engine scheduling.
Median speedup
Failed requests
Deduction
Final score
The round score is the scoring rule applied to the prompts below. Both engines are compared at the same output token count, so a candidate that stops early cannot buy speed by answering less: a prompt under the tolerance bar scores nothing and shows its gate reason.
30 prompts
| Prompt | Speedup | Baseline | Candidate | Aligned output | Gate |
|---|---|---|---|---|---|
| hf-025 | +73.27% | 85.425s | 22.838s | 4360 | — |
| hf-017 | +72.30% | 85.835s | 23.779s | 4368 | — |
| hf-009 | +70.70% | 85.134s | 24.946s |
32 requests
| Request | TTFT | Total | Tokens | Slowest gap | Stream |
|---|---|---|---|---|---|
| hf-000 | 229.67ms | 27.978s | 5120 | 30.14ms | — |
| hf-001 | 251.05ms | 27.760s | 4775 | 36.47ms | — |
| hf-002 | 252.07ms | 29.021s |
Each latency-median response is compared with the lowest valid opening-baseline ratio for the same prompt. Drops above the recorded limit fail; absolute checks also apply. These checks detect repetition, not overall writing quality.
| Prompt | Candidate char-16 | Baseline char-16 | Drop | Limit | Result |
|---|---|---|---|---|---|
| hf-000 | Not recorded | Not recorded | Not recorded | Not recorded | Excluded baseline-drift: longest repeated span ratio 0.351 at or above harness ceiling over 15662 chars |
| hf-001 | 0.9722 | 0.9789 | 0.0067 | 0.1000 | Passed |
| hf-002 | 0.9766 | 0.9624 | -0.0142 | 0.1000 | Passed |
| 4343 |
| — |
| hf-027 | — | — | — | 4156 | candidate output below tolerance |
| hf-023 | +70.28% | 89.915s | 26.727s | 4592 | — |
| hf-031 | +73.17% | 88.159s | 23.650s | 4515 | — |
| hf-014 | +72.16% | 77.782s | 21.656s | 3972 | — |
| hf-026 | +71.47% | 80.000s | 22.827s | 4078 | — |
| hf-019 | +71.36% | 87.873s | 25.170s | 4473 | — |
| hf-011 | +71.79% | 86.806s | 24.492s | 4431 | — |
| hf-028 | — | — | — | 4220 | candidate output below tolerance |
| hf-012 | +70.79% | 97.031s | 28.339s | 5120 | — |
| hf-006 | +70.62% | 96.567s | 28.372s | 5082 | — |
| hf-022 | +71.80% | 91.881s | 25.913s | 4712 | — |
| hf-018 | +70.65% | 95.617s | 28.062s | 5007 | — |
| hf-007 | +70.32% | 87.444s | 25.953s | 4581 | — |
| hf-001 | +69.95% | 92.381s | 27.760s | 4775 | — |
| hf-024 | +69.75% | 93.429s | 28.264s | 4848 | — |
| hf-010 | +71.92% | 76.329s | 21.432s | 3884 | — |
| hf-015 | +70.91% | 92.400s | 26.882s | 4962 | — |
| hf-003 | +71.94% | 97.081s | 27.242s | 5120 | — |
| hf-020 | +68.63% | 94.148s | 29.534s | 5107 | — |
| hf-021 | +71.07% | 97.046s | 28.080s | 5120 | — |
| hf-030 | — | — | — | 4072 | candidate output below tolerance |
| hf-029 | +68.87% | 94.308s | 29.354s | 5120 | — |
| hf-008 | +72.03% | 89.926s | 25.152s | 4591 | — |
| hf-005 | +69.52% | 94.351s | 28.761s | 5120 | — |
| hf-002 | +69.25% | 94.363s | 29.021s | 5120 | — |
| hf-013 | +68.96% | 94.336s | 29.281s | 5120 | — |
| hf-004 | +71.20% | 97.060s | 27.948s | 5120 | — |
| 5120 |
| 32.32ms |
| — |
| hf-003 | 239.41ms | 27.242s | 5120 | 28.42ms | — |
| hf-004 | 245.78ms | 27.948s | 5120 | 30.06ms | — |
| hf-005 | 246.72ms | 28.761s | 5120 | 29.77ms | — |
| hf-006 | 244.62ms | 28.568s | 5120 | 32.51ms | — |
| hf-007 | 242.15ms | 28.526s | 5120 | 28.87ms | — |
| hf-008 | 229.92ms | 25.152s | 4591 | 28.64ms | — |
| hf-009 | 236.59ms | 28.690s | 5120 | 29.11ms | — |
| hf-010 | 226.26ms | 24.073s | 4388 | 28.49ms | — |
| hf-011 | 232.67ms | 27.842s | 5120 | 29.41ms | — |
| hf-012 | 232.39ms | 28.339s | 5120 | 30.01ms | — |
| hf-013 | 219.77ms | 29.281s | 5120 | 28.52ms | — |
| hf-014 | 227.96ms | 25.721s | 4766 | 32.48ms | — |
| hf-015 | 225.07ms | 27.515s | 5120 | 28.89ms | — |
| hf-016 | 214.65ms | 22.424s | 4006 | 29.96ms | — |
| hf-017 | 226.67ms | 27.570s | 5120 | 30.36ms | — |
| hf-018 | 214.60ms | 28.527s | 5120 | 35.12ms | — |
| hf-019 | 220.76ms | 26.709s | 4782 | 31.77ms | — |
| hf-020 | 218.47ms | 29.534s | 5107 | 34.52ms | — |
| hf-021 | 218.96ms | 28.080s | 5120 | 33.03ms | — |
| hf-022 | 215.13ms | 27.683s | 5120 | 31.93ms | — |
| hf-023 | 212.69ms | 28.478s | 4982 | 32.78ms | — |
| hf-024 | 212.12ms | 28.264s | 4848 | 31.33ms | — |
| hf-025 | 207.92ms | 22.838s | 4360 | 31.27ms | — |
| hf-026 | 197.75ms | 27.835s | 5120 | 34.29ms | — |
| hf-027 | 196.80ms | 23.651s | 4156 | 30.15ms | — |
| hf-028 | 193.44ms | 23.716s | 4220 | 33.19ms | — |
| hf-029 | 192.68ms | 29.354s | 5120 | 33.14ms | — |
| hf-030 | 198.03ms | 23.708s | 4072 | 33.10ms | — |
| hf-031 | 187.86ms | 26.217s | 5120 | 31.39ms | — |
| hf-003 |
| 0.8038 |
| 0.9015 |
| 0.0977 |
| 0.1000 |
| Passed |
| hf-004 | 0.9634 | 0.9450 | -0.0184 | 0.1000 | Passed |
| hf-005 | 0.9768 | 0.9423 | -0.0345 | 0.1000 | Passed |
| hf-006 | 0.9500 | 0.9666 | 0.0166 | 0.1000 | Passed |
| hf-007 | 0.9676 | 0.9712 | 0.0035 | 0.1000 | Passed |
| hf-008 | 0.9303 | 0.9503 | 0.0200 | 0.1000 | Passed |
| hf-009 | 0.9659 | 0.9245 | -0.0414 | 0.1000 | Passed |
| hf-010 | 0.9503 | 0.9236 | -0.0267 | 0.1000 | Passed |
| hf-011 | 0.9379 | 0.8646 | -0.0734 | 0.1000 | Failed rep 2: longest repeated span ratio 0.394 at or above harness ceiling over 20592 chars |
| hf-012 | 0.9428 | 0.8951 | -0.0477 | 0.1000 | Passed |
| hf-013 | 0.9760 | 0.9597 | -0.0163 | 0.1000 | Passed |
| hf-014 | 0.9089 | 0.9435 | 0.0345 | 0.1000 | Passed |
| hf-015 | 0.8320 | 0.9588 | 0.1268 | 0.1000 | Failed distinct 16-gram ratio 0.832 is more than 0.10 below baseline 0.959 over 21393 chars |
| hf-016 | Not recorded | Not recorded | Not recorded | Not recorded | Excluded baseline: longest repeated span ratio 0.437 at or above harness ceiling over 21186 chars |
| hf-017 | 0.9190 | 0.9479 | 0.0289 | 0.1000 | Passed |
| hf-018 | 0.9770 | 0.9473 | -0.0296 | 0.1000 | Passed |
| hf-019 | 0.9685 | 0.9289 | -0.0396 | 0.1000 | Passed |
| hf-020 | 0.9869 | 0.9505 | -0.0364 | 0.1000 | Passed |
| hf-021 | 0.9383 | 0.8082 | -0.1301 | 0.1000 | Passed |
| hf-022 | 0.9171 | 0.9013 | -0.0158 | 0.1000 | Passed |
| hf-023 | 0.9701 | 0.9606 | -0.0094 | 0.1000 | Passed |
| hf-024 | 0.9818 | 0.9624 | -0.0194 | 0.1000 | Passed |
| hf-025 | 0.8901 | 0.5903 | -0.2998 | 0.1000 | Passed |
| hf-026 | 0.9315 | 0.8562 | -0.0753 | 0.1000 | Passed |
| hf-027 | 0.9498 | 0.9214 | -0.0284 | 0.1000 | Passed |
| hf-028 | 0.9586 | 0.9060 | -0.0526 | 0.1000 | Passed |
| hf-029 | 0.9781 | 0.9761 | -0.0020 | 0.1000 | Passed |
| hf-030 | 0.9689 | 0.9462 | -0.0228 | 0.1000 | Passed |
| hf-031 | 0.7519 | 0.8823 | 0.1304 | 0.1000 | Failed distinct 16-gram ratio 0.752 is more than 0.10 below baseline 0.882 over 21922 chars |
Actual settings for each measured request. Each prompt keeps its temperature across repetitions and engines. Generation uses seed 0 throughout qualification, warmups and measured repetitions; fixed settings do not guarantee identical output text.
| Prompt | Repetition | Temperature | Top p | Seed |
|---|---|---|---|---|
| hf-016 | 1 | 0.156148 | 1 | 0 |
| hf-030 | 1 | 0.788195 | 1 | 0 |
| hf-025 | 1 | 0.184925 | 1 | 0 |
| hf-027 | 1 | 0.698585 | 1 | 0 |
| hf-028 | 1 | 0.23282 | 1 | 0 |
| hf-010 | 1 | 0.173137 | 1 | 0 |
| hf-008 | 1 | 0.187967 | 1 | 0 |
| hf-019 | 1 | 0.541533 | 1 | 0 |
| hf-031 | 1 | 0.108348 | 1 | 0 |
| hf-021 | 1 | 0.487367 | 1 | 0 |
| hf-009 | 1 | 0.739765 | 1 | 0 |
| hf-001 | 1 | 0.977129 | 1 | 0 |
| hf-014 | 1 | 0.621609 | 1 | 0 |
| hf-024 | 1 | 0.844961 | 1 | 0 |
| hf-023 | 1 | 0.914697 | 1 | 0 |
| hf-003 | 1 | 0.176141 | 1 | 0 |
| hf-026 | 1 | 0.253126 | 1 | 0 |
| hf-000 | 1 | 0.434562 | 1 | 0 |
| hf-022 | 1 | 0.20785 | 1 | 0 |
| hf-007 | 1 | 0.660116 | 1 | 0 |
| hf-015 | 1 | 0.3788 | 1 | 0 |
| hf-018 | 1 | 0.77017 | 1 | 0 |
| hf-017 | 1 | 0.527309 | 1 | 0 |
| hf-011 | 1 | 0.146846 | 1 | 0 |
| hf-012 | 1 | 0.318269 | 1 | 0 |
| hf-004 | 1 | 0.574894 | 1 | 0 |
| hf-005 | 1 | 0.582113 | 1 | 0 |
| hf-006 | 1 | 0.684593 | 1 | 0 |
| hf-002 | 1 | 0.820779 | 1 | 0 |
| hf-013 | 1 | 0.905653 | 1 | 0 |
| hf-020 | 1 | 0.994442 | 1 | 0 |
| hf-029 | 1 | 0.956853 | 1 | 0 |
| hf-026 | 2 | 0.253126 | 1 | 0 |
| hf-028 | 2 | 0.23282 | 1 | 0 |
| hf-008 | 2 | 0.187967 | 1 | 0 |
| hf-000 | 2 | 0.434562 | 1 | 0 |
| hf-010 | 2 | 0.173137 | 1 | 0 |
| hf-016 | 2 | 0.156148 | 1 | 0 |
| hf-014 | 2 | 0.621609 | 1 | 0 |
| hf-011 | 2 | 0.146846 | 1 | 0 |
| hf-022 | 2 | 0.20785 | 1 | 0 |
| hf-031 | 2 | 0.108348 | 1 | 0 |
| hf-025 | 2 | 0.184925 | 1 | 0 |
| hf-003 | 2 | 0.176141 | 1 | 0 |
| hf-019 | 2 | 0.541533 | 1 | 0 |
| hf-015 | 2 | 0.3788 | 1 | 0 |
| hf-017 | 2 | 0.527309 | 1 | 0 |
| hf-004 | 2 | 0.574894 | 1 | 0 |
| hf-021 | 2 | 0.487367 | 1 | 0 |
| hf-024 | 2 | 0.844961 | 1 | 0 |
| hf-012 | 2 | 0.318269 | 1 | 0 |
| hf-007 | 2 | 0.660116 | 1 | 0 |
| hf-006 | 2 | 0.684593 | 1 | 0 |
| hf-027 | 2 | 0.698585 | 1 | 0 |
| hf-009 | 2 | 0.739765 | 1 | 0 |
| hf-005 | 2 | 0.582113 | 1 | 0 |
| hf-030 | 2 | 0.788195 | 1 | 0 |
| hf-018 | 2 | 0.77017 | 1 | 0 |
| hf-023 | 2 | 0.914697 | 1 | 0 |
| hf-029 | 2 | 0.956853 | 1 | 0 |
| hf-002 | 2 | 0.820779 | 1 | 0 |
| hf-013 | 2 | 0.905653 | 1 | 0 |
| hf-001 | 2 | 0.977129 | 1 | 0 |
| hf-020 | 2 | 0.994442 | 1 | 0 |
| hf-027 | 3 | 0.698585 | 1 | 0 |
| hf-031 | 3 | 0.108348 | 1 | 0 |
| hf-016 | 3 | 0.156148 | 1 | 0 |
| hf-025 | 3 | 0.184925 | 1 | 0 |
| hf-030 | 3 | 0.788195 | 1 | 0 |
| hf-017 | 3 | 0.527309 | 1 | 0 |
| hf-014 | 3 | 0.621609 | 1 | 0 |
| hf-012 | 3 | 0.318269 | 1 | 0 |
| hf-010 | 3 | 0.173137 | 1 | 0 |
| hf-015 | 3 | 0.3788 | 1 | 0 |
| hf-019 | 3 | 0.541533 | 1 | 0 |
| hf-018 | 3 | 0.77017 | 1 | 0 |
| hf-028 | 3 | 0.23282 | 1 | 0 |
| hf-022 | 3 | 0.20785 | 1 | 0 |
| hf-001 | 3 | 0.977129 | 1 | 0 |
| hf-011 | 3 | 0.146846 | 1 | 0 |
| hf-004 | 3 | 0.574894 | 1 | 0 |
| hf-026 | 3 | 0.253126 | 1 | 0 |
| hf-003 | 3 | 0.176141 | 1 | 0 |
| hf-008 | 3 | 0.187967 | 1 | 0 |
| hf-005 | 3 | 0.582113 | 1 | 0 |
| hf-006 | 3 | 0.684593 | 1 | 0 |
| hf-023 | 3 | 0.914697 | 1 | 0 |
| hf-009 | 3 | 0.739765 | 1 | 0 |
| hf-007 | 3 | 0.660116 | 1 | 0 |
| hf-000 | 3 | 0.434562 | 1 | 0 |
| hf-013 | 3 | 0.905653 | 1 | 0 |
| hf-021 | 3 | 0.487367 | 1 | 0 |
| hf-002 | 3 | 0.820779 | 1 | 0 |
| hf-024 | 3 | 0.844961 | 1 | 0 |
| hf-020 | 3 | 0.994442 | 1 | 0 |
| hf-029 | 3 | 0.956853 | 1 | 0 |
verdict
evidence
min logprob
num prompts
mean logprob
coverage ratio
quantile logprob
num positions scored
role
evidence
requests per s
sla goodput ratio
output tokens per s
p99 e2e ms rel range
p99 itl ms rel range
p99 ttft ms rel range
Image digest
Patch hash