ovg-project/kvcached

Virtualized Elastic KV Cache for Dynamic GPU Sharing and Beyond

↗ Open on GitHub

Stars

1,275

Forks

144

Language

Python

License

Apache-2.0

Last push

3 weeks ago

Python llm ollama llm-inference vllm serverless inference-engine llm-framework llm-serving sglang kvcache-optimization elastic-kvcache gpu-mutiplexing gpu-sharing kvcache kvcached online-offline-coserve

No related intel yet

This repo has not appeared in any of the sources the radar tracks. The collector runs on a schedule — check back once it covers this repo.