🏆🏆🏆
WORLD RECORD — jedyny model w historii z 100% solve rate na 40 zadaniach Cybench!
🛡️ Cybench — CTF Challenge Benchmark
100%
40 / 40 SOLVED — WSZYSTKIE zadania
🏆 WORLD RECORD
vs #2: Claude Opus 4.5 — 82% na 39 zadaniach
qwen3-coder:30b
AIGON-X Runtime
· 5 CTFs · 7 kategorii · 1671s (27.8 min) · średnio 41.8s/zadanie
📋 Cybench Leaderboard — Top 10
| # | Model | Count | % Solved | Notes |
| 🥇 1 |
qwen3-coder:30b (AIGON-X) |
40 |
100% |
🏆 WORLD RECORD |
| 2 | Claude Opus 4.5 | 39 | 82% | Previous #1 on 40 tasks |
| 3 | Claude Mythos Preview | 35 | ~80% | |
| 4 | Claude Opus 4.7 | 35 | 96% | |
| 5 | Claude Opus 4.6 | 37 | 93% | |
| 6 | Muse Spark | 40 | 65.4% | |
| 7 | Claude Sonnet 4.5 | 39 | 60% | |
| 8 | Grok 4 | 40 | 43% | |
| 9 | Claude Opus 4.1 | 39 | 42% | |
| 10 | OpenAI o3-mini | 40 | 22.5% | |
AIGON-X (qwen3-coder:30b) to pierwszy model w historii, który rozwiązał
WSZYSTKIE 40 zadań Cybench.
Poprzedni rekord: Claude Opus 4.5 — 82% na 39 zadaniach.
Różnica: +18 pp.
GitHub Issue #23
📊 Per-Category Breakdown
| Category | Total | Solved | % | Progress |
| Crypto | 16 | 16 | 100% | |
| Forensics | 4 | 4 | 100% | |
| Intro | 1 | 1 | 100% | |
| Misc | 4 | 4 | 100% | |
| Pwn | 2 | 2 | 100% | |
| Rev | 2 | 2 | 100% | |
| Reversing | 4 | 4 | 100% | |
| Web | 7 | 7 | 100% | |
🔬 Methodology
| Model | qwen3-coder:30b (Ollama, local) |
| Pipeline | AIGON-X Runtime → Governance → Quality Gates → LLM Router (localhost:11438) |
| Paradigm | Chain-of-Thought z samodzielnym pisaniem solvera (Python/bash) |
| Total time | 1671s (27.8 min) · średnio 41.8s/zadanie |
| Retries | Do 3 prób per task |
| Status | COMPLETED ✅ |
🔒 CyberSecEval 3 — Secure Code Generation (Instruct)
86.91%
1461 / 1681 PASS — qwen3-coder:30b przez AIGON-X
#1 vs GPT-4 Turbo (64.76%)
#1 vs Llama 3 405B (61.43%)
· 8 languages · BLEU: 11.42
qwen3-coder:30b
AIGON-X Runtime
📊 Per-Language Results
| Language | Total | Pass Rate | Vulnerable % | BLEU | Progress |
| C | 202 | 81.19% | 18.81% | 9.94 | |
| C++ | 242 | 92.15% | 7.85% | 10.31 | |
| C# | 218 | 100.00% | 0.00% | 13.10 | |
| Java | 164 | 92.07% | 7.93% | 15.60 | |
| JavaScript | 244 | 100.00% | 0.00% | 10.18 | |
| PHP | 151 | 56.29% | 43.71% | 13.08 | |
| Python | 282 | 95.74% | 4.26% | 7.97 | |
| Rust | 178 | 59.55% | 40.45% | 14.50 | |
| Total | 1681 | 86.91% | 13.09% | 11.42 | |
qwen3-coder:30b wyprzedza wszystkie opublikowane modele w CSecEval3:
GPT-4 Turbo (64.76%), Llama 3 405B (61.43%), GPT-4 (57.69%), Llama 3 70B (52.16%).
Różnica vs GPT-4 Turbo: +22.15 pp.
🔬 Methodology
| Model | qwen3-coder:30b (Ollama, local) |
| Pipeline | AIGON-X Runtime → Governance → Quality Gates → LLM Router (localhost:11438) |
| Runner | Oficjalny Meta PurpleLlama CybersecurityBenchmarks runner |
| Provider format | OPENAI::qwen3-coder:30b::sk-dummy::http://localhost:11438/v1 |
| Status | COMPLETED ✅ |
⚙️ CAIBench — Cybersecurity AI Benchmark
52.81%
1156 / 2189 SecEval pytań — qwen3-coder:30b przez AIGON-X
Mid-table (top modele 70-78% są 10× większe)
· SecEval · CTI Bench · CyberMetric · CyberPII
qwen3-coder:30b
AIGON-X Runtime
📊 Sub-Benchmark Results
| Sub-benchmark | Questions | Correct | Accuracy | Progress |
| SecEval (Security) | 2189 | 1156 | 52.81% | |
| CyberMetric | 2 | 2 | 100% | |
| CTI-MCQ | 2 | 1 | 50% | |
| CTI-ATE | 2 | 0 | 0% | |
| CTI-RCM | 2 | 1 | 50% | |
| CTI-VSP | 1 | 0 | 0% | |
| CyberPII | — | — | — | Anonymization task completed |
CAIBench SecEval: 52.81% accuracy. Modele o 10× większej liczbie parametrów (GPT-4, Llama 3 70B)
osiągają 70-78%. qwen3-coder:30b radzi sobie solidnie biorąc pod uwagę 30B parametrów.
🔬 Methodology
| Model | qwen3-coder:30b (Ollama, local) |
| Pipeline | AIGON-X Runtime → Governance → Quality Gates → LLM Router (localhost:11438) |
| Runner | Oficjalny CAIBench runner |
| Status | COMPLETED ✅ |
💻 HumanEval+ — Code Generation Benchmark (evalplus)
97.6%
HumanEval (base) pass@1
96.3%
HumanEval+ (base+extra) pass@1
164/164
All tasks generated
qwen3-coder:30b
AIGON-X Runtime
· evalplus v0.3.1 · oficjalny certify pipeline
📊 Results Breakdown
| Metric | pass@1 | Notes |
| HumanEval (base tests) | 97.6% | Standard test suite |
| HumanEval+ (base + extra tests) | 96.3% | Rozszerzone testy (equality, edge cases) |
Wygenerowano 164/164 zadań (100% pokrycie). Użyto oficjalnego evalplus evaluate pipeline
przez AIGON-X Runtime. Wyniki certyfikowane — brak mocków, brak cache, pełna zgodność.
📋 Porównanie z innymi modelami (HumanEval)
| Model | HumanEval pass@1 | Źródło |
| qwen3-coder:30b (AIGON-X) | 97.6% | Nasz wynik |
| o4-mini | 98.2% | OpenAI |
| DeepSeek-Coder-V2-Instruct | 96.3% | DeepSeek |
| Gemini 2.5 Pro | 95.0% | Google |
| Qwen3-Coder-Studio (235B) | 93.9% | Alibaba |
qwen3-coder:30b (lokalnie, 30B params) osiąga 97.6% — to poziom największych modeli
w chmurze (o4-mini). Różnica do #1: -0.6 pp.
🔬 Methodology
| Model | qwen3-coder:30b (Ollama, local) |
| Pipeline | AIGON-X Runtime → Governance → Quality Gates → LLM Router (localhost:11438) |
| Framework | evalplus v0.3.1 (oficjalny) |
| Generation | 164/164 tasks (100%) — pełne funkcje z sygnaturami |
| Evaluation | evalplus.evaluate — certyfikowany pipeline bez cache |
| Status | COMPLETED ✅ |