Learning Apache Airflow may seem daunting for those adventuring in the data world. This workshop aims to save engineers and scientists time. By the end of this session, attendees will have written two workflows which solve practical problems, running them locally and deploying them in a production-like environment.
Have you wondered about what to do when cronjobs are not enough? Have you heard about Apache Airflow but don't know where to start? Come and join us!
This session aims to smoothly introduce the open-source data orchestration platform, teaching critical concepts through practical examples. We'll be encouraging best practices in writing workflows with Python, including using the Task Flow API, dynamic tasks and the open-source library Astro Python SDK.
Some of the topics covered:
* Building attractive DAGs (Direct Acyclic Graphs)
* How to not miss the schedule
* Running your DAGs locally
* Deploying Airflow
* Troubleshooting
The examples used in the session are based on open datasets.
