4c3fe52b…a98761

Disqualified

Campaign: RadixArk/Qwen3.8-27B-NVFP4-BF16-LMHeadPatch hash:

Stage

Disqualified

11 of 12 stages

Settled in

5h 36m

committed Sep 17, 09:27 UTC

Round score

—

awaiting round

p99 latency

—

ceiling 2000 / 50 ms

Round

Round 2: disqualified · no score

Pipeline

11/12 stages

Intake & gates

6/6

Committed09:27:46

Patch hash recorded; awaiting fetch.

Recorded: Sep 17, 2026, 09:27 UTC
Since previous step: —
State: committed

{
  "hotkey": "5Ey8nTvvDJ9fmKsJGqYCG88jGWMckWXFZT4rjyzDwgaUkogy",
  "payment_tx": 8,
  "commit_block": 9086476,
  "payment_block": 9086473,
  "patch_fingerprint": "sha256:78a79b4343567cf77415db819fd1814c41d1f7acdce092ab7a5538a8cef0474d",
  "patch_reveal_delayed": true
}

Build log

4c3fe52b…a98761

{ "score": null, "reason": "engine died before becoming healthy (last error: [Errno 111] Connection refused); container log tail:\nm_tokens)\n File \"/src/python/sglang/srt/model_executor/runner/prefill_cuda_graph_runner.py\", line 1546, in capture_one_shape\n self.backend.capture_one(\n File \"/src/python/sglang/srt/model_executor/runner_backend/breakable_cuda_graph_backend.py\", line 123, in capture_one\n warmup_out = forward_fn()\n ^^^^^^^^^^^^\n File \"/src/python/sglang/srt/model_executor/runner/prefill_cuda_graph_runner.py\", line 1530, in run_once\n return self._run_forward(forward_batch, num_tokens)\n ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n File \"/opt/venv/lib/python3.12/site-packages/torch/utils/_contextlib.py\", line 124, in decorate_context\n return func(*args, **kwargs)\n ^^^^^^^^^^^^^^^^^^^^^\n File \"/src/python/sglang/srt/model_executor/runner/prefill_cuda_graph_runner.py\", line 756, in _run_forward\n return self.layer_model.forward(\n ^^^^^^^^^^^^^^^^^^^^^^^^^\n File \"/opt/venv/lib/python3.12/site-packages/torch/utils/_contextlib.py\", line 124, in decorate_context\n return func(*args, **kwargs)\n ^^^^^^^^^^^^^^^^^^^^^\n File \"/src/python/sglang/srt/models/qwen3_5.py\", line 1723, in forward\n hidden_states, residual = layer(\n ^^^^^^\n File \"/opt/venv/lib/python3.12/site-packages/torch/nn/modules/module.py\", line 1778, in _wrapped_call_impl\n return self._call_impl(*args, **kwargs)\n ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n File \"/opt/venv/lib/python3.12/site-packages/torch/nn/modules/module.py\", line 1789, in _call_impl\n return forward_call(*args, **kwargs)\n ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n File \"/src/python/sglang/srt/models/qwen3_5.py\", line 1457, in forward\n hidden_states = self.mlp(hidden_states)\n ^^^^^^^^^^^^^^^^^^^^^^^\n File \"/opt/venv/lib/python3.12/site-packages/torch/nn/modules/module.py\", line 1778, in _wrapped_call_impl\n return self._call_impl(*args, **kwargs)\n ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n File \"/opt/venv/lib/python3.12/site-packages/torch/nn/modules/module.py\", line 1789, in _call_impl\n return forward_call(*args, **kwargs)\n ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n File \"/src/python/sglang/srt/models/qwen2_moe.py\", line 254, in forward\n gate_up, _ = self.gate_up_proj(x)\n ^^^^^^^^^^^^^^^^^^^^\n File \"/opt/venv/lib/python3.12/site-packages/torch/nn/modules/module.py\", line 1778, in _wrapped_call_impl\n return self._call_impl(*args, **kwargs)\n ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n File \"/opt/venv/lib/python3.12/site-packages/torch/nn/modules/module.py\", line 1789, in _call_impl\n return forward_call(*args, **kwargs)\n ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n File \"/src/python/sglang/srt/layers/linear.py\", line 505, in forward\n output_parallel = self.quant_method.apply(self, input_, bias)\n ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n File \"/src/python/sglang/srt/layers/quantization/unquant.py\", line 433, in apply\n return F.linear(x, layer.weight, bias)\n ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\ntorch.OutOfMemoryError: CUDA out of memory. Tried to allocate 136.00 MiB. GPU 1 has a total capacity of 31.36 GiB of which 99.06 MiB is free. Including non-PyTorch memory, this process has 31.10 GiB memory in use. Of the allocated memory 28.19 GiB is allocated by PyTorch, with 959.50 MiB allocated in private pools (e.g., CUDA Graphs), and 354.27 MiB is reserved by PyTorch but unallocated. If reserved but unallocated memory is large try setting PYTORCH_CUDA_ALLOC_CONF=expandable_segments:True to avoid fragmentation. See documentation for Memory Management (https://docs.pytorch.org/docs/stable/notes/cuda.html#optimizing-memory-usage-with-pytorch-cuda-alloc-conf)\n\n[2026-09-17 14:05:37] Received sigquit from a child process. It usually means the child failed.\n[2026-09-17 14:05:37] kill_process_tree called: parent_pid=1, include_parent=True, pid=1\n[2026-09-17 14:05:37] kill_process_tree called: parent_pid=1, include_parent=False, pid=1\n", "ordinal": 2, "round_id": "96185147-2e0c-4632-a3d7-23c8c5f00eae" }