FMInference/H2O

[NeurIPS'23] H2O: Heavy-Hitter Oracle for Efficient Generative Inference of Large Language Models.

↗ Open on GitHub

Stars

532

Forks

89

Language

Python

License

—

Last push

2 years ago

Python large-language-models high-throughput gpt-3 heavy-hitters kv-cache sparsity

No related intel yet

This repo has not appeared in any of the sources the radar tracks. The collector runs on a schedule — check back once it covers this repo.