knowledge-distillation

Distilly — Distill how they think into reusable Skills for any Agent or Bot. Formerly Colleague Skill（原同事 Skill）.

把书、长视频、播客等高价值内容蒸馏成可执行的 Agent Skills（Distill high-value content from books, long-form videos, podcasts, and more into executable Agent Skills）

A treasure chest for visual classification and recognition powered by PaddlePaddle

Awesome Knowledge Distillation

Pretrained language model and its related optimization techniques developed by Huawei Noah's Ark Lab.

Collection of AWESOME vision-language models for vision tasks

AI Powered Knowledge Graph Generator

SOTA low-bit LLM quantization (INT8/FP8/MXFP8/INT4/MXFP4/NVFP4) & sparsity; leading model compression techniques on PyTorch, TensorFlow, and ONNX Runtime

EasyNLP: A Comprehensive and Easy-to-use NLP Toolkit

A curated list for Efficient Large Language Models

This is a collection of our NAS and Vision Transformer work.

OpenMMLab Model Compression Toolbox and Benchmark.

A coding-free framework built on PyTorch for reproducible deep learning studies. PyTorch Ecosystem. 🏆26 knowledge distillation methods presented at TPAMI, CVPR, ICLR, ECCV, NeurIPS, ICCV, AAAI, etc are implemented so far. 🎁 Trained models, training logs and configurations are available for ensuring the reproducibiliy and benchmark.

Efficient computing methods developed by Huawei Noah's Ark Lab

This repository collects papers for "A Survey on Knowledge Distillation of Large Language Models". We break down KD into Knowledge Elicitation and Distillation Algorithms, and explore the Skill & Vertical Distillation of LLMs.

Collection of recent methods on (deep) neural network compression and acceleration.

An Extendible (General) Continual Learning Framework based on Pytorch - official codebase of Dark Experience for General Continual Learning

A curated collection of papers, technical reports, frameworks, and tools for on-policy distillation (OPD) of large language models

[ICCV 2023] MI-GAN: A Simple Baseline for Image Inpainting on Mobile Devices

From teacher to tiles — a from-scratch LLM distillation & serving engine: custom Triton/CUDA kernels, FSDP distillation, paged-KV continuous batching, speculative decoding, a Rust gateway, a JAX oracle, and interpretability tooling.

A curated collection of papers and resources on On-Policy Distillation for Large Language Models.