cuda

:metal: TT-NN operator library, and TT-Metalium low level kernel programming model.

🔥 Real-time NVIDIA GPU dashboard

ThunderSVM: A Fast SVM Library on GPUs and CPUs

llm deploy project based mnn. This project has merged into MNN.

Open Source Continuous Inference Benchmark Research Platform — Kimi K3 2.8T, MiniMax M3, DeepSeekv4, GLM5 - GB200 NVL72 vs MI355X vs B200 vs GB300 NVL72 & soon™ TPUv6e/v7/Trainium2/3 | 开源持续推理基准研究平台 — Kimi K2.7-Code、MiniMax M3、DeepSeekv4、GLM5 - GB200 NVL72 vs MI355X vs B200 vs GB300 NVL72，即将推出™ TPUv6e/v7/Trainium2/3

an implementation of 3D Ken Burns Effect from a Single Image using PyTorch

Autoresearch for GPU kernels. Give it any PyTorch model, go to sleep, wake up to optimized Triton kernels.

UCCL is an efficient communication library for GPUs, covering collectives, P2P (e.g., KV cache transfer, RL weight transfer), and EP (e.g., GPU-driven)

Write Java. Run on GPUs. Fast.

🔥🔥🔥TensorRT for YOLOv8、YOLOv8-Pose、YOLOv8-Seg、YOLOv8-Cls、YOLOv7、YOLOv6、YOLOv5、YOLONAS......🚀🚀🚀CUDA IS ALL YOU NEED.🍎🍎🍎

An efficient C++20 GPU numerical computing library with Python-like syntax

Self-host the powerful Chatterbox TTS model. This server offers a user-friendly Web UI, flexible API endpoints (incl. OpenAI compatible), predefined voices, voice cloning, and large audiobook-scale text processing. Runs accelerated on NVIDIA (CUDA), AMD (ROCm), and CPU.

CUDA programming in Julia.

A fast, ergonomic and portable tensor library in Nim with a deep learning focus for CPU, GPU and embedded devices via OpenMP, Cuda and OpenCL backends

A fast communication-overlapping library for tensor/expert parallelism on GPUs.

LuxCore source repository

https://wavespeed.ai/ Best inference performance optimization framework for HuggingFace Diffusers on NVIDIA GPUs.

stdgpu: Efficient STL-like Data Structures on the GPU

GraphVite: A General and High-performance Graph Embedding System

The FFmpeg build script provides an easy way to build a static FFmpeg on OSX and Linux with non-free codecs included.

Safe rust wrapper around CUDA toolkit

Samples for Intel® oneAPI Toolkits

OpenCL integration for Python, plus shiny features

The Hacker's Machine Learning Engine

Fast Clojure Matrix Library

SGLang-Omni is a high-performance serving framework for audio models (TTS, ASR) and unified multimodal models.

Build your own high performance LLM inference engine in C++ and CUDA - a smaller version of vLLM

Programmable CUDA/C++ GPU Graph Analytics

Robotics with GPU computing

High-Performance Rendering Framework on Stream Architectures