Loading...

Course Description

Apache Airflow is a configuration-as-code OSS solution for workflow automation that is positioned as a replacement of cron-like scheduling systems. Written in Python, Airflow enables developers to programmatically author, schedule for execution, and monitor highly configurable complex workflows.

Course Outline

Chapter 1. Apache Airflow Introduction

  • A Traditional ETL Approach
  • Apache Airflow Defined
  • Airflow Core Components
  • The Component Collaboration Diagram
  • Workflow Building Blocks and Concepts
  • Airflow CLI
  • Main Configuration File
  • Extending Airflow
  • Jinja Templates
  • Variables and Macros
  • Summary

Chapter 2. Apache Airflow Web UI

  • Web UI - the Landing (DAGs) Page
  • Web UI - the DAG Graph View
  • Run Status Legends
  • The Pause Button (Trigger Latch)
  • The DAG Triggering/Job Checking Sequence
  • The Control Panel for a Task
  • Sample Log File Messages (Abridged for Space)
  • Summary

Chapter 3. Anatomy of a DAG and Scheduling

  • What is a DAG?
  • Scheduled and Manually Triggered DAG Runs
  • The DAG Object
  • Tasks
  • Task Lifecycle
  • Operators
  • Idempotent Operators
  • Operator Types
  • Airflow Common Operators
  • Specifying Dependencies
  • Associating Operators with a DAG
  • Associating Operators Using the "With DAG" Statement Example
  • Associating Operators with DAG Using the Operator's Constructor
  • The default_args Parameter
  • Passing DAG Parameters Through Web UI
  • DAG Run Scheduling
  • Examples of the schedule_interval Parameter
  • DAG Scheduling Nuances
  • Understanding The Backfill Process
  • Killing/Stopping DAG Runs
  • An XCom Messaging Example
  • Summary

Prerequisites

Participants should be familiar with Python syntax (or have a background in programming).

Applies Towards the Following Certificates

Loading...

Thank you for your interest in this course. Unfortunately, the course you have selected is currently not open for enrollment. Please complete a Course Inquiry or call 314-977-3226 so that we may promptly notify you when enrollment opens.

Required fields are indicated by .
*Academic Unit eligibility to be determined by college/university in which you are enrolled in a degree seeking program.