multi-modal

Build and run agents you can see, understand and trust.

A Pocket-Sized MLLM for Ultra-Efficient Image and Video Understanding on Your Phone

Open-source framework for conversational voice AI agents

[CVPR 2024 Oral] InternVL Family: A Pioneering Open-Source Alternative to GPT-4o. 接近GPT-4o表现的开源多模态对话模型

ModelScope: bring the notion of Model-as-a-Service to life.

AI suite powered by state-of-the-art models and providing advanced AI/AGI functions. Includes AI personas, AGI functions, world-class Beam multi-model chats, text-to-image, voice, response streaming, code highlighting and execution, PDF import, presets for developers, much more. Deploy on-prem or in the cloud.

Data processing for and with foundation models! 🍎 🍋 🌽 ➡️ ➡️🍸 🍹 🍷

a state-of-the-art-level open visual language model | 多模态预训练模型

Open Source Routing Engine for OpenStreetMap

Chinese version of CLIP which achieves Chinese cross-modal retrieval and representation generation.

Implementation / replication of DALL-E, OpenAI's Text to Image Transformer, in Pytorch

Ecommerce Search and Discovery - marqo.ai

[EMNLP 2022] An Open Toolkit for Knowledge Graph Extraction and Construction

Open-source evaluation toolkit of large multi-modality models (LMMs), support 220+ LMMs, 80+ benchmarks

OmniGen: Unified Image Generation. https://arxiv.org/pdf/2409.11340

Chinese and English multimodal conversational language model | 多模态中英双语对话语言模型

A C#/.NET library to run LLM (🦙LLaMA/LLaVA) on your local device efficiently.

【EMNLP 2024🔥】Video-LLaVA: Learning United Visual Representation by Alignment Before Projection

Open-source AI assistant ecosystem with MCP integrations, multimodal workflows, IoT support, and cross-platform voice interaction.

Represent, send, store and search multimodal data

Project Page for "LISA: Reasoning Segmentation via Large Language Model"

GPT4V-level open-source multi-modal model based on Llama3-8B

【TMM 2025🔥】 Mixture-of-Experts for Large Vision-Language Models

推荐/广告/搜索领域工业界经典以及最前沿论文集合。A collection of industry classics and cutting-edge papers in the field of recommendation/advertising/search.

[NeurIPS 2023] MotionGPT: Human Motion as a Foreign Language, a unified motion-language generation model using LLMs

A robust, all-in-one GPT interface for Discord. ChatGPT-style conversations, image generation, AI-moderation, custom indexes/knowledgebase, youtube summarizer, and more!

SALMONN family: A suite of advanced multi-modal LLMs

[pip install medmnist] 18x Standardized Datasets for 2D and 3D Biomedical Image Classification

Pytorch implementation of Transfusion, "Predict the Next Token and Diffuse Images with One Multi-Modal Model", from MetaAI

This repository collects papers for "A Survey on Knowledge Distillation of Large Language Models". We break down KD into Knowledge Elicitation and Distillation Algorithms, and explore the Skill & Vertical Distillation of LLMs.