multimodal

Stop renting your intelligence. Own it with AnythingLLM. Everything you need for a powerful local-first agent experience

《深入理解 AI Agent：设计原理与工程实践》（李博杰 著）开源主仓库：全书正文、编译版 PDF 与按章配套代码

The Open-Source Multimodal AI Agent Stack: Connecting Cutting-Edge AI Models and Agent Infra

[NeurIPS'23 Oral] Visual Instruction Tuning (LLaVA) built towards GPT-4V level capabilities and beyond.

Large-scale Self-supervised Pre-training Across Tasks, Languages, and Modalities

☁️ Build multimodal AI applications with cloud-native stack

YC (S26) | Open Computer History | Record your screen continuously locally and provide context to your agents (Claude, Codex, Openclaw, Hermes, Runner...)

Janus-Series: Unified Multimodal Understanding and Generation Models

Use PEFT or Full-parameter to CPT/SFT/DPO/GRPO 600+ LLMs (Qwen3.6, DeepSeek-V4, GLM-5.1, InternLM3, Llama4, ...) and 300+ MLLMs (Qwen3-VL, Qwen3-Omni, InternVL3.5, Ovis2.5, GLM4.5v, Gemma4, Llava, Phi4, ...) (AAAI 2025).

Visualize, query, and stream to train on multimodal robotics data.

🔍大模型应用开发实战一：RAG 技术全栈指南，在线阅读地址：https://datawhalechina.github.io/all-in-rag/

Production ready toolkit to run AI locally

A Python library for anomaly detection across tabular, time series, graph, text, image, and audio data. 60+ detectors, benchmark-backed ADEngine orchestration, and an agentic workflow for AI agents.

https://arxiv.org/abs/2606.28344. The end of web parsing. The beginning of scalable pixel-native search. link: https://pixelrag.ai/

SeaTunnel is a multimodal, high-performance, distributed, massive data integration tool.

Deeplake is AI Data Runtime for Agents. It provides serverless postgres with a multimodal datalake, enabling scalable retrieval and training.

Mobile-Agent: The Powerful GUI Agent Family

The easiest way to serve AI apps and models - Build Model Inference APIs, Job queues, LLM apps, Multi-model pipelines, and more!

A text-to-speech (TTS), speech-to-text (STT) and speech-to-speech (STS) library built on Apple's MLX framework, providing efficient speech analysis on Apple Silicon.

A framework for efficient model inference with omni-modality models

This repository is a curated collection of links to various courses and resources about Artificial Intelligence (AI)

Open-source framework for building agentic apps in JavaScript, Go, Dart, and Python, built and used in production by Google

notes for software engineers getting up to speed on new AI developments. Serves as datastore for https://latent.space writing, and product brainstorming, but has cleaned up canonical references under the /Resources folder.

Solve Visual Understanding with Reinforced VLMs

A visual playground for agentic workflows: Iterate over your agents 10x faster

High-performance data engine for AI and multimodal workloads. Process images, audio, video, and structured data at any scale

A Low-Code MCP Framework for Building Complex and Innovative RAG Pipelines

A modular framework for vision & language multimodal research from Facebook AI Research (FAIR)

A Next-Generation Training Engine Built for Ultra-Large MoE Models

Align Anything: Training All-modality Model with Feedback