vllm-project/vllm

A high-throughput and memory-efficient inference and serving engine for LLMs

↗ Open on GitHub

Stars

90,817

Forks

21,619

Language

Python

License

Apache-2.0

Last push

5 days ago

Python openai llm cuda gpt transformer pytorch deepseek qwen llama inference kimi qwen3 tpu llm-serving model-serving moe deepseek-v3 amd gpt-oss blackwell

Hacker News Best1 week ago

vLLM v0.28.0

by mrrrcs107 points38 commentsDiscussion → GitHub repo: vllm-project/vllm