syv-ai/qwen38-27b-rtx3090
@syv-aiQwen3.8-27B on a single RTX 3090 with vLLM: ~1,000 tok/s at 64 concurrent (int8 tensor-core GEMMs, fp16 DeltaNet state), ~114 tok/s single-user at default sampling / ~124 greedy (MTP drafts, own-output draft vocab, calibrated int4 lm_head, split-KV verify attention), 150k-262k context; patches, requant scripts, benchmarks
Stars
1,266
Forks
177
Language
Python
License
Apache-2.0
Last push
24 hours ago
Related intel (0)
No related intel yet
This repo has not appeared in any of the sources the radar tracks. The collector runs on a schedule — check back once it covers this repo.