Airflow Tutorial - Variables and Connections
Hello Data Pros, In our last blog, we covered the fundamental concepts of Apache Airflow, including dags , tasks and operators! In addition, we demonstrated the importance of the airflow configuration file, and the provider packages! Today, we’ll learn about the power of Airflow Variables and Connections and how to use them effectively within your Dags . Let's start with Apache Airflow Variables! Variables are like little storage containers for values, that you can reuse across your dags . Instead of hardcoding values, you can store them as variables, and reference them with its name whenever needed. Technically each variable is a key and a value pair. You can think of the key as the variable name and the value as the data it holds. There are two types of Airflow variables! One, Regular variables, Where the value can be any string. Two, JSON variables, where the value is a JSON string. Let’s consider I have a dag , that in...