jmaczan/tiny-vllm

Build your own high performance LLM inference engine in C++ and CUDA - a smaller version of vLLM

↗ Open on GitHub

Stars

1,094

Forks

85

Language

C++

License

Apache-2.0

Last push

2 weeks ago

C++cpp ai llm cuda llm-inference vllm hpc inference course attention batching pagedattention tiny-vllm

No related intel yet

This repo has not appeared in any of the sources the radar tracks. The collector runs on a schedule — check back once it covers this repo.