◑ applying · kind tool · level 3 → 4 via project · 40h
- Requiere: Distributed Systems
- Aplica: dbt
DAG-based scheduling and orchestration with retries, backfills, and lineage. Runs the batch and dbt jobs of the platform on a dependable schedule.
Apache Airflow is an open-source workflow management platform for data engineering pipelines. It started at Airbnb in October 2014 as a solution to manage the company’s increasingly complex workflows. Creating Airflow allowed Airbnb to programmatically author and schedule their workflows and monitor them via the built-in Airflow user interface. From the beginning, the project was made open source, becoming an Apache Incubator project in March 2016 and a top-level Apache Software Foundation project in January 2019.
Enlaces
- Requiere: Distributed Systems
- Aplica: dbt