pipeline

Python library for creating data pipelines with chain functional programming

cubestudio开源云原生一站式机器学习/深度学习/大模型AI平台/MaaS/mlops/人工智能平台/训推平台，算法全链路流程，多租户，算力租赁平台，token中转，拖拉拽任务流pipeline编排，多机多卡分布式训练，超参搜索，推理服务，VGPU虚拟化，云边端协同，边缘计算，自动化标注平台，deepseek等大模型sft微调/奖励模型/强化学习训练，vllm/ollama/mindie大模型多机推理，私有知识库llmops智能体，AI模型市场，支持国产异构算力调度,昇腾/寒武纪/海光/摩尔/沐曦等，支持ib/roce/RDMA，信创支持

🔮 Instill Core is a full-stack AI infrastructure tool for data, model and pipeline orchestration, designed to streamline every aspect of building versatile AI-first applications

【A common used C++ & Python DAG framework】 一个通用的、无三方依赖的、跨平台的、收录于awesome-cpp的、基于流图的并行计算框架。欢迎star & fork & 交流

The Fastest Unified Streaming SQL Engine in a Single C++ Binary. ⚡ Millisecond latency. 100+ GB/s throughput. Continuously compute real-time context from streams, logs, metrics, events, and CDC.

A lightweight stream processing library for Go

A Julia machine learning framework

XLS: Accelerated HW Synthesis

Go toolkit for clean, composable, channel-based concurrency

Data intensive science for everyone.

PyTorch extensions for fast R&D prototyping and Kaggle farming

Hop Orchestration Platform

a realtime plotting utility for terminal/console with data input from stdin

RNA sequencing analysis pipeline using STAR, RSEM, HISAT2 or Salmon with gene/isoform counts and extensive quality control.

An R-focused pipeline toolkit for reproducibility and high-performance computing

Highly Performant, Modular, Memory Safe and Production-ready Inference, Ingestion and Indexing built in Rust 🦀

蓝鲸智云标准运维(SOPS)

A micro-library of stream components for building custom JSON and JSONC processing pipelines with a minimal memory footprint — parse, filter, and transform JSON far larger than available memory with a SAX-inspired token API, on Node.js or Web Streams.

A cross-platform command-line tool for executing jobs in parallel

Robust, flexible and resource-efficient pipelines using Go and the commandline

Function-oriented Make-like declarative workflows for R

An open-source ML pipeline development platform

Tiny automation pipelines. Bring CI/CD to the smallest projects. Self-hosted, Lightweight, CLI only.

A dashboard for Tekton!

A WDL, CWL and Python API supporting easy-to-use workflow engine. It is scalable, efficient and cross-platform (Linux/macOS).

A flexible, high-performance carrier for machine learning models（『飞桨』服务化部署框架）

Neum AI is a best-in-class framework to manage the creation and synchronization of vector embeddings at large scale.

Declarative language for composable Al workflows. Devtool for agents and mere humans.

🪿 LinGoose is a Go framework for building awesome AI/LLM applications.

Plumber detects CI/CD security issues in your GitHub workflows and gives you a score