Udemy
    •  
    •  
    •  
    •  
    •  
    •  
    •  
    •  
Turn what you know into an opportunity and reach millions around the world.
Learn More
Your cart is empty.
Keep shopping
Learn Airflow v2 in an hour
Rating: 4.4 out of 5(25 ratings)
124 students

Learn Airflow v2 in an hour

Presented by And Ozbay
Created byAnd Ozbay
Last updated 3/2021
English
English [Auto],

What you'll learn

  • Understand the overall Airflow system
  • Learn about DAGs, operators, sensors, connections, hooks, etc.
  • Get an introduction into advanced topics, like XCOMs, conditional execution, etc.
  • Do a high level review of the Apache Airflow code base
  • Get the ability to unblock yourself if you run into any issues running Airflow workflows

Course content

3 sections • 13 lectures • 1h 13m total length

  • Introduction0:39

    Learn to run workflows with Airflow v2, set up with Docker containers, and master operators, hooks, connections, and tags to build practical workflows.

  • Course prerequisites - please do not skip2:29

    Follow the os-specific cheat sheet to install prerequisites, including docker engine and docker compose on Linux, and Mac/Windows desktop versions, with optional Atem editor and terminal.

  • Set up6:10

    Set up your Airflow v2 environment by creating a project folder, unzipping resources, and fixing permissions on macOS or Linux; then run docker compose up to launch Postgres and Airflow.

  • Interacting with the containers3:34

    Interact with the PostgreSQL and Airflow containers from the provided interface or by entering the containers with docker exec, inspect the DAG folder, and run database commands from within.

Requirements

  • At least 8 gigabytes of memory
  • Familiarity with Python

Description

Airflow is an Apache project, used to programmatically author, schedule and monitor workflows. It was originally developed by Airbnb and then later open-sourced. Today, many technology companies use Airflow to orchestrate their workflows.


This course is designed to get you up and running with Airflow v2 in an hour. The specific focus of the course is on teaching you the fundamentals of using Airflow rather than setting up a production environment. We will set up a sandbox environment with Airflow and PostgreSQL Servers running for you to code along and practice what you have learned. The skills you learn in this course will be transferable to your work, where a production environment is already available and maintained. Alternatively, if you are interested in using Airflow for hobby projects, the sandbox environment that you set up on your machine will likely be sufficient.


When you are taking this course, you will:

  • Learn the core Airflow concepts

  • Write an end to end workflow that takes data, cleans it up and writes into a database

  • Learn a number of different operators and get the hand of using operators so that you can easily incorporate new ones into your workflow

  • Understand how you can unblock yourself if you run into issues running your code



Who this course is for:

  • Data scientists and data engineers who would like to start writing workflows in Airflow v2 quickly

Report abuse