SleekData

Hello Data Pros, and welcome back to another exciting episode of our Snowflake learning series! In our previous video, we’ve covered Snowflake Streams; and demonstrated their role in change data capture. Today, we will explore Snowflake Tasks, which are essential for automating a variety of data processes within Snowflake. ------------------- Follow the link in the description for the SQLs used in this lab ------------------- --------------------------------------------------------------------------- ---------------- Setting Up Foundational Objects for the Lab--------------- --------------------------------------------------------------------------- --Create Database and Schema CREATE OR REPLACE DATABASE order_db ; CREATE OR REPLACE SCHEMA order_db . order_schema ; -- Create Tables USE SCHEMA order_db . order_schema ; CREATE or replace TABLE order_raw (     order_id INT ,     order_date DATE ,     cust_fname VARCHAR( 50 ),     cust_lna...

Hello Data Pros,  In our last blog, we demonstrated step-by-step installation of Apache Airflow on a Windows PC, and successfully executed our very first Airflow dag! Now, it's time to dive deeper! In this video, we'll learn about the airflow configuration file! Explore each section inside a dag! Understand various Operator types! Experience the power of provider packages! Let's begin right away!   As we already know, airflow dags are coded in Python language.   Every Airflow setup has a ‘dags folder’. You can set this folder path in the Airflow configuration file, named airflow dot cfg.   In addition to the dags folder, this configuration file has many other settings that you can customize to meet your needs.   For example, to enable my Airflow instance to send email notifications, I added another Docker container in my docker compose. This new container will locally host a simple SMTP server. I then updated the Airflow configuration file to use the correspondi...