attention-mechanism

Implement a ChatGPT-like LLM in PyTorch from scratch, step by step

Implementation of Vision Transformer, a simple way to achieve SOTA in vision classification with only a single transformer encoder, in Pytorch

邱锡鹏《神经网络与深度学习》（蒲公英书）理论书 v2 与通识版

RWKV (pronounced RwaKuv) is an RNN with great LLM performance, which can also be directly trained like a GPT transformer (parallelizable). We are at RWKV-7 "Goose". So it's combining the best of RNN and transformer - great performance, linear time, constant space (no kv-cache), fast training, infinite ctx_len, and free sentence embedding.

A concise but complete full-attention transformer with a set of promising experimental features from various papers

Implementation / replication of DALL-E, OpenAI's Text to Image Transformer, in Pytorch

An ultimately comprehensive paper list of Vision Transformer/Attention, including papers, codes, and related websites

RuVector is a High Performance, Real-Time, Self-Learning Ai, Vector GNN, Memory DB built in Rust.

Build a modern LLM from scratch. Every line commented. Explained like we are five.

Multilingual Automatic Speech Recognition with word-level timestamps and confidence

Keras Attention Layer (Luong and Bahdanau scores).

Implementation of SoundStorm, Efficient Parallel Audio Generation from Google Deepmind, in Pytorch

Learn LLM internals step by step - from tokenization to attention to inference optimization.

Implementation of Perceiver, General Perception with Iterative Attention, in Pytorch

Sequence-to-sequence framework with a focus on Neural Machine Translation based on PyTorch

A Deep Learning library for EEG Tasks (Signals) Classification, based on TensorFlow.

A curated list of NLP resources focused on Transformer networks, attention mechanism, GPT, BERT, ChatGPT, LLMs, and transfer learning.

OpenSTL: A Comprehensive Benchmark of Spatio-Temporal Predictive Learning

Implementation of TabTransformer, attention network for tabular data, in Pytorch

Transformer based on a variant of attention that is linear complexity in respect to sequence length

[ECCV 2020] Learning Enriched Features for Real Image Restoration and Enhancement. SOTA results for image denoising, super-resolution, and image enhancement.

based on yolo-high-level project (detect\pose\classify\segment\):include yolov5\yolov7\yolov8\ core ,improvement research ,SwintransformV2 and Attention Series. training skills, business customization, engineering deployment C

MORAN: A Multi-Object Rectified Attention Network for Scene Text Recognition

Achieve the llama3 inference step-by-step, grasp the core concepts, master the process derivation, implement the code.

[IGARSS'22]: A Transformer-Based Siamese Network for Change Detection

FlashAttention (Metal Port)

[CVPR 2021, Oral] PREDATOR: Registration of 3D Point Clouds with Low Overlap.

Build high-performance AI models with modular building blocks

[ICLR2025 Spotlight🔥] Official Implementation of TokenFormer: Rethinking Transformer Scaling with Tokenized Model Parameters

Democratization of RT-2 "RT-2: New model translates vision and language into action"