Apache Spark is an open source distributed general-purpose cluster-computing framework. It provides an interface for programming entire clusters with implicit data parallelism and fault tolerance.
The open source AI engineering platform for agents, LLMs, and ML models. MLflow enables teams of all sizes to debug, evaluate, monitor, and optimize production-quality AI applications while controlling costs and managing access to models and data.
lakeFS - Data version control for your data lake | Git for data
Simple and Distributed Machine Learning
酷玩 Spark: Spark 源代码解析、Spark 类库等
Drop-in Apache Spark replacement written in Rust, unifying batch processing, stream processing, and compute-intensive AI workloads.
Kubernetes operator for managing the lifecycle of Apache Spark applications on Kubernetes.
Interactive and Reactive Data Science using Scala and Spark.
BigDL: Distributed TensorFlow, Keras and PyTorch on Apache Spark/Flink & Ray
.NET for Apache® Spark™ makes Apache Spark™ easily accessible to .NET developers.
Apache Spark docker image
Feathr – A scalable, unified data and AI engineering platform for enterprise
A curated list of awesome Apache Spark packages and resources.
SQL data analysis & visualization projects using MySQL, PostgreSQL, SQLite, Tableau, Apache Spark and pySpark.
Oryx 2: Lambda architecture on Apache Spark, Apache Kafka for real-time large scale machine learning
An end-to-end GoodReads Data Pipeline for Building Data Lake, Data Warehouse and Analytics Platform.
This is the github repo for Learning Spark: Lightning-Fast Data Analytics [2nd Edition]
GraphFrames is a package for Apache Spark which provides DataFrame-based Graphs
PySpark + Scikit-learn = Sparkit-learn
MapReduce, Spark, Java, and Scala for Data Algorithms Book