Apache Airflow Cheat Sheet

CLI Commands

DAG Definition Pattern

from airflow import DAG
from airflow.operators.python import PythonOperator
from airflow.providers.snowflake.operators.snowflake import SnowflakeOperator
from datetime import datetime, timedelta

default_args = {
    'owner': 'data-engineering',
    'depends_on_past': False,
    'email_on_failure': True,
    'email': ['[email protected]'],
    'retries': 2,
    'retry_delay': timedelta(minutes=5),
}

with DAG(
    dag_id='daily_etl_pipeline',
    default_args=default_args,
    description='Daily ETL from sources to warehouse',
    schedule_interval='0 6 * * *',       # 6 AM daily
    start_date=datetime(2026, 1, 1),
    catchup=False,
    tags=['etl', 'daily', 'production'],
    max_active_runs=1,
) as dag:

    extract = PythonOperator(
        task_id='extract_data',
        python_callable=extract_from_api,
    )

    transform = SnowflakeOperator(
        task_id='transform_data',
        snowflake_conn_id='snowflake_prod',
        sql='sql/transform.sql',
    )

    validate = PythonOperator(
        task_id='validate_output',
        python_callable=run_data_quality_checks,
    )

    extract >> transform >> validate

Common Operators

Schedule Interval Quick Reference

Best Practices

• ✓Set catchup=False unless you specifically need historical backfills
• ✓Use max_active_runs=1 for ETL DAGs to prevent parallel run conflicts
• ✓Store SQL in separate files (sql/ directory) instead of inline strings
• ✓Use Airflow Variables and Connections — never hardcode credentials in DAGs
• ✓Use TaskGroups to visually organize complex DAGs instead of SubDAGs (deprecated)
• ✓Set meaningful retries and retry_delay — most transient failures resolve in 5 minutes
• ✓Use depends_on_past=False unless tasks truly depend on their own previous run
• ✓Tag DAGs (tags=['etl','daily']) for filtering in the Airflow UI

Frequently Asked Questions

Related Cheat Sheets