Here are 4,229 public repositories matching this topic...

apache / airflow

Apache Airflow - A platform to programmatically author, schedule, and monitor workflows

Updated May 6, 2026
Python

argoproj / argo-workflows

Workflow Engine for Kubernetes

kubernetes workflow machine-learning airflow workflow-engine cncf pipelines data-engineering argo k8s cloud-native batch-processing hacktoberfest dag gitops mlops knative argo-workflows

Updated May 4, 2026
Go

apache / dolphinscheduler

Apache DolphinScheduler is the modern data orchestration platform. Agile to create high performance workflow with low-code

workflow airflow job-scheduler orchestration cloud-native task-scheduler data-pipelines azkaban workflow-orchestration workflow-schedule powerful-data-pipelines

Updated May 4, 2026
Java

orchest / orchest

Build data pipelines, the easy way 🛠️

python docker kubernetes data-science machine-learning airflow cloud deployment jupyter etl ide pipelines self-hosted jupyterlab notebooks data-pipelines dag etl-pipeline orchest

Updated Jun 6, 2023
TypeScript

jghoman / awesome-apache-airflow

Curated list of resources about Apache Airflow

airflow awesome awesome-list workflow-management apache-airflow

Updated May 4, 2026
Shell

puckel / docker-airflow

Docker Apache Airflow

docker workflow task airflow scheduler management docker-airflow

Updated Mar 1, 2023
Shell

WeBankFinTech / DataSphereStudio

DataSphereStudio is a one stop data application development& management portal, covering scenarios including data exchange, desensitization/cleansing, analysis/mining, quality measurement, visualization, and task scheduling.

Updated Nov 4, 2025
Java

elyra-ai / elyra

Elyra extends JupyterLab with an AI centric approach.

Updated Apr 30, 2026
Python

san089 / Udacity-Data-Engineering-Projects

Few projects related to Data Engineering including Data Modeling, Infrastructure setup on cloud, Data Warehousing and Data Lake development.

Updated Aug 26, 2022
Python

teamclairvoyant / airflow-maintenance-dags

A series of DAGs/Workflows to help maintain the operation of Airflow

workflow airflow maintenance cleanup dag airflow-maintenance-dags apache-airflow

Updated Jun 18, 2024
Python

OBenner / data-engineering-interview-questions

More than 2000+ Data engineer interview questions.

Updated Jan 13, 2026
Python

san089 / goodreads_etl_pipeline

An end-to-end GoodReads Data Pipeline for Building Data Lake, Data Warehouse and Analytics Platform.

Updated Mar 9, 2020
Python

astronomer / dag-factory

Construct Apache Airflow DAGs Declaratively via YAML configuration files

python airflow apache-airflow dags

Updated May 5, 2026
Python

damklis / DataEngineeringProject

Example end to end data engineering project.

python redis elasticsearch airflow kafka big-data mongodb scraping django-rest-framework s3 data-engineering minio kafka-connect hacktoberfest data-pipeline debezium

Updated Dec 8, 2022
Python

astronomer / astronomer-cosmos

Run your dbt Core or dbt Fusion projects as Apache Airflow DAGs and Task Groups with a few lines of code

python workflow airflow dbt apache-airflow airflow-operators

Updated May 5, 2026
Python

abhishek-ch / around-dataengineering

A Data Engineering & Machine Learning Knowledge Hub

infrastructure devops machine-learning airflow spark datascience data-engineering mlops

Updated Feb 2, 2024
Python

alanchn31 / Data-Engineering-Projects

Personal Data Engineering Projects

postgres airflow spark cassandra mongodb data-warehouse data-engineering data-lake scrapy data-modeling aws-redshift star-schema ingest-data data-engineering-nanodegree

Updated Feb 8, 2023
Jupyter Notebook

iusztinpaul / energy-forecasting

🌀 𝗧𝗵𝗲 𝗙𝘂𝗹𝗹 𝗦𝘁𝗮𝗰𝗸 𝟳-𝗦𝘁𝗲𝗽𝘀 𝗠𝗟𝗢𝗽𝘀 𝗙𝗿𝗮𝗺𝗲𝘄𝗼𝗿𝗸 | 𝗟𝗲𝗮𝗿𝗻 𝗠𝗟𝗘 & 𝗠𝗟𝗢𝗽𝘀 for free by designing, building and deploying an end-to-end ML batch system ~ 𝘴𝘰𝘶𝘳𝘤𝘦 𝘤𝘰𝘥𝘦 + 2.5 𝘩𝘰𝘶𝘳𝘴 𝘰𝘧 𝘳𝘦𝘢𝘥𝘪𝘯𝘨 & 𝘷𝘪𝘥𝘦𝘰 𝘮𝘢𝘵𝘦𝘳𝘪𝘢𝘭𝘴

Updated Apr 3, 2024
Python

ankurchavda / streamify

A data engineering project with Kafka, Spark Streaming, dbt, Docker, Airflow, Terraform, GCP and much more!

python airflow kafka spark gcp data-engineering dbt

Updated Apr 16, 2022
Python

raystack / optimus

Optimus is an easy-to-use, reliable, and performant workflow orchestrator for data transformation, data modeling, pipelines, and data quality management.

golang bigquery airflow automation etl analytics data-transformation data-warehouse business-intelligence dataops elt workflows data-pipelines data-modelling analytics-engineering

Updated Jun 8, 2024
Go