CyberSecQwen-4B: A 4B‑Parameter Defensive Cybersecurity Model Trained on a Single AMD MI300X
Deploy CyberSecQwen‑4B on a single 12 GB GPU to classify CWEs and answer CTI queries, achieving 97 % of the 8B baseline accuracy with half the parameters.
Deploy the model on your on‑prem GPU, run inference with the provided code, and use the 12 GB GPU for low‑cost defensive triage.
Summary
CyberSecQwen‑4B is a 4‑billion‑parameter instruction‑tuned model fine‑tuned on defensive cybersecurity data, released on 8 May 2026. Trained on a single AMD Instinct MI300X 192 GB GPU, the model uses full bf16 precision, FlashAttention‑2, a 4096‑token sequence length, and a batch size of four. Compared to the 8‑billion‑parameter Foundation‑Sec‑Instruct‑8B baseline, CyberSecQwen‑4B retains 97.3 % of the CTI‑RCM accuracy (0.6664 vs 0.6850) while exceeding the MCQ score by +8.7 pp (0.5868 vs 0.4996). The training recipe employs LoRA with r = 64, alpha = 64, dropout = 0.05, learning rate = 5e‑5, 10 epochs, and paged_adamw_8bit optimizer. Inference can run on any 12 GB+ GPU using the vLLM v0.10.1 Docker image, with the provided code snippet enabling quick deployment. The model is Apache‑2.0 licensed, making it suitable for on‑prem security teams that cannot rely on hosted APIs.
Key changes
- Fine‑tuned Qwen3‑4B‑Instruct‑2507 on defensive corpora for CWE classification and CTI Q&A
- CTI‑RCM accuracy 0.6664 vs 0.6850 baseline, retaining 97.3 % of 8B performance
- CTI‑MCQ score 0.5868, +8.7 pp over baseline
- Runs on AMD Instinct MI300X 192 GB with full bf16 and FlashAttention‑2
- Inference on any 12 GB+ GPU using vLLM v0.10.1 Docker image
- Training recipe: LoRA r = 64, alpha = 64, dropout = 0.05, LR = 5e‑5, 10 epochs, paged_adamw_8bit
- Supports 4096‑token sequences, batch size 4, no quantization needed
- Apache‑2.0 license for on‑prem deployment