vision-language-model

The implementation of "Prismer: A Vision-Language Model with Multi-Task Experts".

Fully Open Framework for Democratized Multimodal Training

This series will take you on a journey from the fundamentals of NLP and Computer Vision to the cutting edge of Vision-Language Models.

Turn documents into AI-ready Markdown with visual understanding

为纯文本模型"看图“设计更好的视觉工具箱和技能，支持多图理解，图片问答，前端UI还原、GUI 自动化等，并可选无缝接入多个主流agent，直接识别粘贴图片｜ A vision toolkit and skill designed for text-only llms — image Q&A, long-screenshot OCR, frontend UI restoration, and GUI automation, with optional seamless integration for Codex, Claude Code, Pi, Oh My Pi, and OpenCode

Parsing-free RAG supported by VLMs

[CVPR 2024 🔥] Grounding Large Multimodal Model (GLaMM), the first-of-its-kind model capable of generating natural language responses that are seamlessly integrated with object segmentation masks.

[CVPR 2024 Highlight🔥] Chat-UniVi: Unified Visual Representation Empowers Large Language Models with Image and Video Understanding

About This repository is a curated collection of the most exciting and influential CVPR 2025 papers. 🔥 [Paper + Code + Demo]

[CVPR 2024] Alpha-CLIP: A CLIP Model Focusing on Wherever You Want

[dsh]为纯文本模型设计更强大的视觉工具箱：一行安装使用、粘贴图片直接识别、多张图片问答、截图到前端UI 还原等｜DeepSeek Harness-native integration for agent-vision-toolkit: image Q&A, long-screenshot OCR, UI restoration, grounding, pixel diff, Artifacts, and Web UI.

VoxPoser: Composable 3D Value Maps for Robotic Manipulation with Language Models

[NeurIPS 2025 Spotlight] OpenCUA: Open Foundations for Computer-Use Agents

🍃 MINT-1T: A one trillion token multimodal interleaved dataset.

A curated list of 3D Vision papers relating to Robotics domain in the era of large models i.e. LLMs/VLMs, inspired by awesome-computer-vision, including papers, codes, and related websites

A curated list of awesome prompt/adapter learning methods for vision-language models like CLIP.

[ICLR 2026] The offical Implementation of "Soft-Prompted Transformer as Scalable Cross-Embodiment Vision-Language-Action Model"

OmniVinci is an omni-modal LLM for joint understanding of vision, audio, and language.

MiMo-VL

[CVPR 2026] SpatialVID: A Large-Scale Video Dataset with Spatial Annotations

🎉 PILOT: A Pre-trained Model-Based Continual Learning Toolbox

Evaluating text-to-image/video/3D models with VQAScore

An open-weight 11B model series for long-form and real-time video understanding

[ECCV2024] Grounded Multimodal Large Language Model with Localized Visual Tokenization

LLaVA-Mini is a unified large multimodal model (LMM) that can support the understanding of images, high-resolution images, and videos in an efficient manner.

Cambrian-S: Towards Spatial Supersensing in Video

Chatbot Arena meets multi-modality! Multi-Modality Arena allows you to benchmark vision-language models side-by-side while providing images as inputs. Supports MiniGPT-4, LLaMA-Adapter V2, LLaVA, BLIP-2, and many more!

Flame is an open-source multimodal AI system designed to translate UI design mockups into high-quality React code. It leverages vision-language modeling, automated data synthesis, and structured training workflows to bridge the gap between design and front-end development.

Code and implementation guidelines for the paper ✨Counting Anything. Project Page: https://mengqi-lei.github.io/count-anything-projectpage/