multimodal

SDK for interacting with stability.ai APIs (e.g. stable diffusion inference)

mPLUG-DocOwl: Modularized Multimodal Large Language Model for Document Understanding

[ECCV2024] Video Foundation Models & Data for Multimodal Understanding

🎨 NeMo Data Designer: Generate high-quality synthetic data from scratch or from seed data.

GenAI Processors is a lightweight Python library that enables efficient, parallel content processing.

Let Claude (or any LLM) actually watch a video — scene-aware, deduplicated frames + transcript, from a URL or local file. Runs locally, MIT.

On-device, real-time multimodal AI with features similar to GPT-Live

[ICLR & NeurIPS 2025] Repository for Show-o series, One Single Transformer to Unify Multimodal Understanding and Generation.

An open-source implementaion for fine-tuning Qwen-VL series by Alibaba Cloud.

Implementation of "BitNet: Scaling 1-bit Transformers for Large Language Models" in pytorch

A collection of original, innovative ideas and algorithms towards Advanced Literate Machinery. This project is maintained by the OCR Team in the Language Technology Lab, Tongyi Lab, Alibaba Group.

Synthesizing Graphics Programs for Scientific Figures and Sketches with TikZ.

The Self-Coding System for Your App — Alan AI SDK for Android

The Self-Coding System for Your App — Alan AI SDK for Flutter

The Self-Coding System for Your App — Alan AI SDK for Ionic

Unified multimodal backend for AI data apps

Fast Multimodal LLM on Mobile Devices

Get clean data from tricky documents, powered by vision-language models ⚡

A novel Multimodal Large Language Model (MLLM) architecture, designed to structurally align visual and textual embeddings.

Visual intelligence for your home.

日本語LLMまとめ - Overview of Japanese LLMs

WeMM-Embedding is a family of universal multimodal embedding models by the WeChat Vision Team at Tencent, supporting multimodal understanding and retrieval.

This repository is the official implementation of Disentangling Writer and Character Styles for Handwriting Generation (CVPR 2023)

Offline inference engine for art, real-time voice conversations, LLM powered chatbots and automated workflows

Curated visual catalog of 155+ vision-language model (VLM/MLLM) architectures: papers, diagrams, training recipes, datasets, and a release timeline for multimodal AI agents.

Give Claude the ability to watch and understand videos — Claude Code plugin with frame extraction and multimodal audio analysis

Pocket-Sized Multimodal AI for content understanding and generation across multilingual texts, images, and 🔜 video, up to 5x faster than OpenAI CLIP and LLaVA 🖼️ & 🖋️

Xtreme1 is an all-in-one data labeling and annotation platform for multimodal data training and supports 3D LiDAR point cloud, image, and LLM.

Unifying 3D Mesh Generation with Language Models

Turn documents into AI-ready Markdown with visual understanding