cuda-kernels

Modern CUDA Learn Notes with PyTorch for Beginners, 200+ CUDA Kernels, Tensor Cores, HGEMM, FA-2 MMA.

Samples for CUDA Developers which demonstrates features in CUDA Toolkit

LMDeploy is a toolkit for compressing, deploying, and serving LLMs.

Ecosystem of libraries and tools for writing and executing fast GPU code fully in Rust.

LLM speculative inference server for heterogeneous hardware & consumer GPUs

CUDA Core Compute Libraries

Deep learning in Rust, with shape checked tensors and neural networks

Safe rust wrapper around CUDA toolkit

A machine learning library with a TypeScript API and Rust backend. CUDA and WebGPU compatibility. Built to understand how ML frameworks and models work internally.

CUDA Kernel Benchmarking Library

cuDNN Frontend is NVIDIA's modern, open-source entry point to the cuDNN library and a growing collection of high-performance open-source kernels.

Pure Rust + CUDA LLM inference engine — no PyTorch, OpenAI-compatible, serves Qwen3 to Kimi-K2

FlashRT is a high-performance realtime inference engine for small-batch, latency-sensitive AI workloads. The flagship integration is production VLA control for Pi0, Pi0.5, GROOT N1.6, and Pi0-FAST. Also support llm e.g, qwen3.6-27B