Round score
—
engine died before becoming healthy (last error: [Errno 111] Connection refused); container log tail: m_tokens) File "/src/python/sglang/srt/model_executor/runner/prefill_cuda_graph_runner.py", line 1546, in capture_one_shape self.backend.capture_one( File "/src/python/sglang/srt/model_executor/runner_backend/breakable_cuda_graph_backend.py", line 123, in capture_one warmup_out = forward_fn() ^^^^^^^^^^^^ File "/src/python/sglang/srt/model_executor/runner/prefill_cuda_graph_runner.py", line 1530, in run_once return self._run_forward(forward_batch, num_tokens) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/opt/venv/lib/python3.12/site-packages/torch/utils/_contextlib.py", line 124, in decorate_context return func(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^ File "/src/python/sglang/srt/model_executor/runner/prefill_cuda_graph_runner.py", line 756, in _run_forward return self.layer_model.forward( ^^^^^^^^^^^^^^^^^^^^^^^^^ File "/opt/venv/lib/python3.12/site-packages/torch/utils/_contextlib.py", line 124, in decorate_context return func(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^ File "/src/python/sglang/srt/models/qwen3_5.py", line 1723, in forward hidden_states, residual = layer( ^^^^^^ File "/opt/venv/lib/python3.12/site-packages/torch/nn/modules/module.py", line 1778, in _wrapped_call_impl return self._call_impl(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/opt/venv/lib/python3.12/site-packages/torch/nn/modules/module.py", line 1789, in _call_impl return forward_call(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/src/python/sglang/srt/models/qwen3_5.py", line 1457, in forward hidden_states = self.mlp(hidden_states) ^^^^^^^^^^^^^^^^^^^^^^^ File "/opt/venv/lib/python3.12/site-packages/torch/nn/modules/module.py", line 1778, in _wrapped_call_impl return self._call_impl(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/opt/venv/lib/python3.12/site-packages/torch/nn/modules/module.py", line 1789, in _call_impl return forward_call(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/src/python/sglang/srt/models/qwen2_moe.py", line 254, in forward gate_up, _ = self.gate_up_proj(x) ^^^^^^^^^^^^^^^^^^^^ File "/opt/venv/lib/python3.12/site-packages/torch/nn/modules/module.py", line 1778, in _wrapped_call_impl return self._call_impl(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/opt/venv/lib/python3.12/site-packages/torch/nn/modules/module.py", line 1789, in _call_impl return forward_call(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/src/python/sglang/srt/layers/linear.py", line 505, in forward output_parallel = self.quant_method.apply(self, input_, bias) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/src/python/sglang/srt/layers/quantization/unquant.py", line 433, in apply return F.linear(x, layer.weight, bias) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ torch.OutOfMemoryError: CUDA out of memory. Tried to allocate 136.00 MiB. GPU 3 has a total capacity of 31.36 GiB of which 97.06 MiB is free. Including non-PyTorch memory, this process has 31.11 GiB memory in use. Of the allocated memory 28.19 GiB is allocated by PyTorch, with 959.50 MiB allocated in private pools (e.g., CUDA Graphs), and 354.27 MiB is reserved by PyTorch but unallocated. If reserved but unallocated memory is large try setting PYTORCH_CUDA_ALLOC_CONF=expandable_segments:True to avoid fragmentation. See documentation for Memory Management (https://docs.pytorch.org/docs/stable/notes/cuda.html#optimizing-memory-usage-with-pytorch-cuda-alloc-conf) [2026-09-17 14:11:47] Received sigquit from a child process. It usually means the child failed. [2026-09-17 14:11:47] kill_process_tree called: parent_pid=1, include_parent=True, pid=1 [2026-09-17 14:11:47] kill_process_tree called: parent_pid=1, include_parent=False, pid=1
Prompts scored
0 / 0
median_e2e_speedup
Below tolerance
0
answered too little to score
Zeroed total
0
scored 0.0 without measuring
Request interval
Thinking
Context limit
The same inputs and output limits apply to every engine. Actual concurrency depends on response duration and engine scheduling.
The round score is the scoring rule applied to the prompts below. Both engines are compared at the same output token count, so a candidate that stops early cannot buy speed by answering less: a prompt under the tolerance bar scores nothing and shows its gate reason.
No prompt scores
engine died before becoming healthy (last error: [Errno 111] Connection refused); container log tail: m_tokens) File "/src/python/sglang/srt/model_executor/runner/prefill_cuda_graph_runner.py", line 1546, in capture_one_shape self.backend.capture_one( File "/src/python/sglang/srt/model_executor/runner_backend/breakable_cuda_graph_backend.py", line 123, in capture_one warmup_out = forward_fn() ^^^^^^^^^^^^ File "/src/python/sglang/srt/model_executor/runner/prefill_cuda_graph_runner.py", line 1530, in run_once return self._run_forward(forward_batch, num_tokens) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/opt/venv/lib/python3.12/site-packages/torch/utils/_contextlib.py", line 124, in decorate_context return func(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^ File "/src/python/sglang/srt/model_executor/runner/prefill_cuda_graph_runner.py", line 756, in _run_forward return self.layer_model.forward( ^^^^^^^^^^^^^^^^^^^^^^^^^ File "/opt/venv/lib/python3.12/site-packages/torch/utils/_contextlib.py", line 124, in decorate_context return func(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^ File "/src/python/sglang/srt/models/qwen3_5.py", line 1723, in forward hidden_states, residual = layer( ^^^^^^ File "/opt/venv/lib/python3.12/site-packages/torch/nn/modules/module.py", line 1778, in _wrapped_call_impl return self._call_impl(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/opt/venv/lib/python3.12/site-packages/torch/nn/modules/module.py", line 1789, in _call_impl return forward_call(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/src/python/sglang/srt/models/qwen3_5.py", line 1457, in forward hidden_states = self.mlp(hidden_states) ^^^^^^^^^^^^^^^^^^^^^^^ File "/opt/venv/lib/python3.12/site-packages/torch/nn/modules/module.py", line 1778, in _wrapped_call_impl return self._call_impl(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/opt/venv/lib/python3.12/site-packages/torch/nn/modules/module.py", line 1789, in _call_impl return forward_call(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/src/python/sglang/srt/models/qwen2_moe.py", line 254, in forward gate_up, _ = self.gate_up_proj(x) ^^^^^^^^^^^^^^^^^^^^ File "/opt/venv/lib/python3.12/site-packages/torch/nn/modules/module.py", line 1778, in _wrapped_call_impl return self._call_impl(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/opt/venv/lib/python3.12/site-packages/torch/nn/modules/module.py", line 1789, in _call_impl return forward_call(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/src/python/sglang/srt/layers/linear.py", line 505, in forward output_parallel = self.quant_method.apply(self, input_, bias) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/src/python/sglang/srt/layers/quantization/unquant.py", line 433, in apply return F.linear(x, layer.weight, bias) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ torch.OutOfMemoryError: CUDA out of memory. Tried to allocate 136.00 MiB. GPU 3 has a total capacity of 31.36 GiB of which 97.06 MiB is free. Including non-PyTorch memory, this process has 31.11 GiB memory in use. Of the allocated memory 28.19 GiB is allocated by PyTorch, with 959.50 MiB allocated in private pools (e.g., CUDA Graphs), and 354.27 MiB is reserved by PyTorch but unallocated. If reserved but unallocated memory is large try setting PYTORCH_CUDA_ALLOC_CONF=expandable_segments:True to avoid fragmentation. See documentation for Memory Management (https://docs.pytorch.org/docs/stable/notes/cuda.html#optimizing-memory-usage-with-pytorch-cuda-alloc-conf) [2026-09-17 14:11:47] Received sigquit from a child process. It usually means the child failed. [2026-09-17 14:11:47] kill_process_tree called: parent_pid=1, include_parent=True, pid=1 [2026-09-17 14:11:47] kill_process_tree called: parent_pid=1, include_parent=False, pid=1
Image digest
Patch hash
Engine