big-data

High-performance data engine for AI and multimodal workloads. Process images, audio, video, and structured data at any scale

Simple and Distributed Machine Learning Python Library porting ML algorithms for Spark

Apache Calcite

Apache Ignite

Stream Framework is a Python library, which allows you to build news feed, activity streams and notification systems using Cassandra and/or Redis. The authors of Stream-Framework also provide a cloud service for feed technology:

⚡️A vue component support big amount data list with high render performance and efficient.

Easily turn large sets of image urls to an image dataset. Can download, resize and package 100M urls in 20h on one machine.

CrateDB is a distributed and scalable SQL database for storing and analyzing massive amounts of data in near real-time, even with complex queries. It is PostgreSQL-compatible, and based on Lucene.

🚄 FASTJSON2 is a Java JSON library with excellent performance.

Data Science Roadmap from A to Z

🔨 🍇 💻 🚀 GraphScope: A One-Stop Large-Scale Graph Computing System from Alibaba | 一站式图计算系统

Extensible SQL Lexer and Parser for Rust

Apache Paimon is a lake format that enables building a Realtime Lakehouse Architecture with Flink and Spark for both streaming and batch operations.

Koalas: pandas API on Apache Spark

Drop-in Apache Spark replacement written in Rust, unifying batch processing, stream processing, and compute-intensive AI workloads.

A graph database that supports more than 100+ billion data, high performance and scalability (Include OLTP Engine & REST-API & Backends)

An easy to use, self-service open BI reporting and BI dashboard platform.

Open-Source Web UI for managing Apache Kafka clusters

ArcticDB is a high performance, serverless DataFrame database built for the Python Data Science ecosystem.

Open-source BI for engineers

Apache Ambari simplifies provisioning, managing, and monitoring of Apache Hadoop clusters.

YTsaurus is a scalable and fault-tolerant open-source big data platform.

Apache Fluss is a streaming storage built for real-time analytics.

Apache DataFusion Ballista Distributed Query Engine

Apache Drill is a distributed MPP query layer for self describing data

Apache BookKeeper - a scalable, fault tolerant and low latency storage service optimized for append-only workloads

Fluid, elastic data abstraction and acceleration for BigData/AI applications in cloud. (Project under CNCF)

Mirror of Apache Kudu

🚀 500+ curated resources for Data Analysis & Data Science: Python, SQL, Statistics, ML, AI, Visualization, Cheatsheets, Roadmaps, Interview Prep. For beginners and experts.

Apache Auron(Incubating) is an accelerator for big data engines, leveraging native vectorized execution to accelerate query processing.