attention

🧑‍🏫 60+ Implementations/tutorials of deep learning papers with side-by-side notes 📝; including transformers (original, xl, switch, feedback, vit, ...), optimizers (adam, adabelief, sophia, ...), gans(cyclegan, stylegan2, ...), 🎮 reinforcement learning (ppo, dqn), capsnet, distillation, ... 🧠

SGLang is a high-performance serving framework for large language models and multimodal models.

Natural Language Processing Tutorial for Deep Learning Researchers

A theoretical reconstruction of the Claude Mythos architecture, built from first principles using the available research literature.

🍀 Pytorch implementation of various Attention Mechanisms, MLP, Re-parameter, Convolution, which is helpful to further understand papers.⭐⭐⭐

A PyTorch implementation of the Transformer model in "Attention is All You Need".

FlashInfer: Kernel Library for LLM Serving

Tutorials on implementing a few sequence-to-sequence (seq2seq) models with PyTorch and TorchText.

Transformer: PyTorch Implementation of "Attention Is All You Need"

Scenic: A Jax Library for Computer Vision Research and Beyond

[ICLR2025, ICML2025, NeurIPS2025 Spotlight] Quantized Attention achieves speedup of 2-5x compared to FlashAttention, without losing end-to-end metrics across language, image, and video models.

This is a pytorch repository of YOLOv4, attentive YOLOv4 and mobilenet YOLOv4 with PASCAL VOC and COCO

A compilation of the best multi-agent papers

Pytorch implementation of Transfusion, "Predict the Next Token and Diffuse Images with One Multi-Modal Model", from MetaAI

Replication of simple CV Projects including attention, classification, detection, keypoint detection, etc.

Build your own high performance LLM inference engine in C++ and CUDA - a smaller version of vLLM

[ICML2025] SpargeAttention: A training-free sparse attention that accelerates any model inference.

cuDNN Frontend is NVIDIA's modern, open-source entry point to the cuDNN library and a growing collection of high-performance open-source kernels.

A PyTorch library for all things Reinforcement Learning (RL) for Combinatorial Optimization (CO)

Implementation of the sparse attention pattern proposed by the Deepseek team in their "Native Sparse Attention" paper

多标签文本分类，多标签分类，文本分类, multi-label, classifier, text classification, BERT, seq2seq，attention, multi-label-classification

[TNSRE 23] EEG Transformer 2.0. i. Convolutional Transformer for EEG Decoding. ii. Novel visualization - Class Activation Topography.

Implementation of plug in and play Attention from "LongNet: Scaling Transformers to 1,000,000,000 Tokens"

Achieve the llama3 inference step-by-step, grasp the core concepts, master the process derivation, implement the code.

Keras beit,caformer,CMT,CoAtNet,convnext,davit,dino,efficientdet,edgenext,efficientformer,efficientnet,eva,fasternet,fastervit,fastvit,flexivit,gcvit,ghostnet,gpvit,hornet,hiera,iformer,inceptionnext,lcnet,levit,maxvit,mobilevit,moganet,nat,nfnets,pvt,swin,tinynet,tinyvit,uniformer,volo,vanillanet,yolor,yolov7,yolov8,yolox,gpt2,llama2, alias kecam

Julia Implementation of Transformer models

The official PyTorch implementation of the paper "SAITS: Self-Attention-based Imputation for Time Series". A fast and state-of-the-art (SOTA) deep-learning neural network model for efficient time-series imputation (impute multivariate incomplete time series containing NaN missing data/values with machine learning). https://arxiv.org/abs/2202.08516

Design hardware-friendly model architectures and migrate existing LLMs with minimal performance loss