multimodal

Plug in and Play Implementation of Tree of Thoughts: Deliberate Problem Solving with Large Language Models that Elevates Model Reasoning by atleast 70%

Curated tutorials and resources for Large Language Models, AI Painting, and more.

Easily turn large sets of image urls to an image dataset. Can download, resize and package 100M urls in 20h on one machine.

One-for-All Multimodal Evaluation Toolkit Across Text, Image, Video, and Audio Tasks

Fengshenbang-LM(封神榜大模型)是IDEA研究院认知计算与自然语言研究中心主导的大模型开源体系，成为中文AIGC和认知智能的基础设施。

An open-source model family for long-form speech, dialogue synthesis, voice design, sound effects, and real-time streaming TTS

OpenMMLab Pre-training Toolbox and Benchmark

The first vision plugin for DeepSeek Harness, and the vision bridge for every text-only coding agent. Paste an image, get structured JSON evidence (OCR, layout, semantics). | 全网最强 DeepSeek Harness 外挂视觉插件，为 DeepSeek、GLM 等纯文本模型外挂视觉能力，粘贴图片即得结构化 JSON 证据（OCR、版面、语义）。

SimpleMem: Efficient Lifelong Memory for LLM Agents — Text & Multimodal

Open-source multimodal retrieval engine (Morphik Core). By Morphik — AI back office for skilled nursing & senior living (morphik.ai).

From Chain-of-Thought prompting to OpenAI o1 and DeepSeek-R1 🍓

Code and models for ICML 2024 paper, NExT-GPT: Any-to-Any Multimodal Large Language Model

MTEB: State-of-the-art evaluation of embeddings across languages and modalities

InternGPT (iGPT) is an open source demo platform where you can easily showcase your AI models. Now it supports DragGAN, ChatGPT, ImageBind, multimodal chat like GPT-4, SAM, interactive image editing, etc. Try it at igpt.opengvlab.com (支持DragGAN、ChatGPT、ImageBind、SAM的在线Demo系统)

An extensible, state-of-the-art framework for columnar compression, and the fastest FOSS columnar file format. Formerly at @spiraldb, now an Incubation Stage project at LFAI&Data, part of the Linux Foundation.

Foundation Architecture for (M)LLMs

Represent, send, store and search multimodal data

[NeurIPS 2024] OSWorld: Benchmarking Multimodal Agents for Open-Ended Tasks in Real Computer Environments

🤖 Type-safe, provider-agnostic TypeScript AI SDK for streaming chat, tool calling, agents, and multimodal apps across OpenAI, Anthropic, Gemini, React, Vue, Svelte, and Solid.

InternLM-XComposer2.5-OmniLive: A Comprehensive Multimodal System for Long-term Streaming Video and Audio Interactions

The Context Layer for unstructured data: typed, versioned datasets over S3, GCS, Azure

Easily compute clip embeddings and build a clip retrieval system with them

Images to inference with no labeling (use foundation models to train supervised models).

streamline the fine-tuning process for multimodal models: PaliGemma 2, Florence-2, and Qwen2.5-VL

[EMNLP-2024] Build multimodal language agents for fast prototype and production

Comprehensive resources on Generative AI, including a detailed roadmap, projects, use cases, interview preparation, and coding preparation.

Official repository of OFA (ICML 2022). Paper: OFA: Unifying Architectures, Tasks, and Modalities Through a Simple Sequence-to-Sequence Learning Framework

mPLUG-Owl: The Powerful Multi-modal Large Language Model Family

HuixiangDou: Overcoming Group Chat Scenarios with LLM-based Technical Assistance

(ෆ`꒳´ෆ) A Survey on Text-to-Image Generation/Synthesis.