feifeibear/long-context-attention

USP: Unified (a.k.a. Hybrid, 2D) Sequence Parallel Attention for Long Context Transformers Model Training and Inference

↗ Open on GitHub

Stars

691

Forks

84

Language

Python

License

Apache-2.0

Last push

3 days ago

Python llm-inference pytorch llm-training attention-is-all-you-need deepspeed-ulysses ring-attention

No related intel yet

This repo has not appeared in any of the sources the radar tracks. The collector runs on a schedule — check back once it covers this repo.