Apache Airflow in Production: Orchestrating Data Harvesting Pipelines

• Joao Ramiro

Room: 31/3-004 - IT Amphitheatre Apache Airflow ([https://airflow.apache.org/][1]) is a key tool we use to orchestrate our data harvesting processes and manage production workflows. This talk will show how Airflow helps us design, schedule, and monitor harvesting pipelines, with concrete examples from daily operations. We will also highlight the main pros and cons we have encountered, including operational and scaling considerations. The session is aimed at engineers and technical users interested in practical workflow orchestration. [1]: https://airflow.apache.org/