carloslfu/slotstream

Run Qwen3.8-Flash-Next (125B MoE, 104 GB at 4-bit) on Macs with a fraction of that RAM by streaming experts from SSD. MLX + Swift, Ollama-compatible API.

↗ Open on GitHub

Stars

287

Forks

16

Language

Swift

License

MIT

Last push

6 days ago

Swift macos llm swift ollama llm-inference local-llm qwen apple-silicon mlx mixture-of-experts

Hacker News Front Page1 week ago

Show HN: Running 104GB Qwen3.8-Flash-Next on 48GB Mac with at ~12 tok/s

by carloslfu220 points101 commentsDiscussion → GitHub repo: carloslfu/slotstream