distributed-computing

MapReduce, Spark, Java, and Scala for Data Algorithms Book

:rocket: R package: future: Unified Parallel and Distributed Processing in R for Everyone

LizardFS is an Open Source Distributed File System licensed under GPLv3.

Unified Interface for Constructing and Managing Workflows on different workflow engines, such as Argo Workflows, Tekton Pipelines, and Apache Airflow.

Distributed Stockfish analysis for lichess.org

Bagua Speeds up PyTorch

Awesome list of distributed systems resources

Unleash the power of cloud

A curated list of awesome parallel computing resources

GPU worker client for the Talos network. Pairs with your Talos account, serves open-model inference jobs over a WebSocket, and reports uptime for payouts.

A library to train, evaluate, interpret, and productionize decision forest models such as Random Forest and Gradient Boosted Decision Trees.

Backend.AI is a streamlined, container-based computing cluster platform that hosts popular computing/ML frameworks and diverse programming languages, with pluggable heterogeneous accelerator support including CUDA GPU, ROCm GPU, Gaudi NPU, Google TPU, GraphCore IPU and other NPUs.

An experimental platform for federated learning.

A source-grounded course and architectural reference for engineers designing systems that must survive real traffic, partial failure, security review, and changing requirements, spanning enterprise system design, distributed systems, AI systems, cybersecurity, reliability, cloud, HPC, edge, and mission-critical infrastructure.

A simplified, lightweight ETL Framework based on Apache Spark

User-friendly Scheduler for sub-node tasks for HPC systems

Kubernetes Operator for MPI-based applications (distributed training, HPC, etc.)

Self-hosted AI agent OS. Your memory, chat, agents, and files stay on hardware you own, offline by default, cloud by choice. Offline AI memory (taOSmd), self-hosted multi-framework group chat, a full web desktop + app store, and auto-clustering across the consumer hardware you already have (Orange/Raspberry Pi, Mac mini, gaming PC).