This post is for teams that already run Airflow and want their dbt transformations to run inside Snowflake instead of on a separate server. It shows how I wired Snowflake’s native dbt Projects into Airflow: the permissions, deploying the project, the DAG, versioning, monitoring, and the gotchas I hit along the way.
I’ll be honest with you: for a long time I ran dbt the way most people run it. dbt Core installed on a server, a profiles.yml file I kept updating by hand, a cron job (yes, a cron job) doing the scheduling, and Airflow somewhere nearby doing the “real” orchestration while dbt lived in its own separate corner of the infrastructure.
It worked. It was fine. It was also quietly annoying in ways I’d stopped noticing. Managing the dbt server separately. Keeping the Snowflake credentials synced in two places. Debugging failures by jumping between the Airflow UI, SSH logs on the dbt server and Snowsight, all at once.
Then Snowflake went GA with dbt Projects in November 2025, and I spent a weekend rebuilding the whole thing. This article is what I learned.
What we’re building is a genuine end-to-end pipeline: raw data lands in Snowflake, Airflow orchestrates the entire flow, and the dbt transformations run as a native DBT PROJECT object inside Snowflake. Not on an external box, not in a container, but inside Snowflake itself. The monitoring, the scheduling trigger and the execution logs are all in one place.
What a dbt Project on Snowflake actually is
The terminology can trip you up, and I don’t want you 45 minutes into setup before the confusion hits.
dbt Projects on Snowflake let you create, edit, test, run and manage dbt Core projects with Snowflake’s own tools. You work on the project files in Snowsight Workspaces, then deploy the project as a schema-level dbt PROJECT object.
The key word is object. DBT PROJECT is a first-class, schema-level object: a file container that holds one or more dbt Core projects. It is versioned, so every change made through ALTER adds a new version.
That means your dbt project (the models, the sources YAML, dbt_project.yml) lives inside Snowflake as a versioned, native object. Not on a VM. Not in an S3 bucket somewhere.
Out of the box you get a file-based IDE in Workspaces that integrates with Git, a DAG view for inspecting lineage and dependencies in the UI, deployment and scheduling with native Snowflake Tasks, and the ability to run dbt commands such as compile, test and run from Workspaces.
So yes, you can schedule and run it purely with Snowflake Tasks and never touch Airflow. But if your organization already runs Airflow, or your dbt pipeline is one piece of a larger flow that includes ingestion, validation, downstream alerts and reporting, you want Airflow in charge, calling into Snowflake to execute the DBT PROJECT object. That hybrid approach is what this article covers.
The architecture we’re building
Before I show you any code, here is the full picture, because I think this is where most blog posts let you down: they show you a piece without the whole.
JSON document — structured configuration or API payload.
[Source System / S3 / API] ↓[Airflow DAG starts] ↓ Task 1: Load raw data → Snowflake staging table (via COPY INTO or S3 stage) ↓ Task 2: Run data quality checks on raw data (SQLExecuteQueryOperator) ↓ Task 3: EXECUTE DBT PROJECT → runs dbt build on your native Snowflake dbt project ↓ Task 4: Post-run row count validation (SQLExecuteQueryOperator) ↓ Task 5: Trigger downstream alert / Slack notification / refresh BI layer ↓[Pipeline complete]

Airflow owns the orchestration. Snowflake owns the execution of the dbt transformations. The DBT PROJECT object bridges them, because you can trigger it with a SQL command, and Airflow’s SQLExecuteQueryOperator can fire that SQL command.
That SQL command is simple:
Snowflake native dbt command — executes the versioned DBT PROJECT object in the warehouse.
EXECUTE DBT PROJECT my_database.my_schema.my_dbt_project ARGS = 'dbt build' VERSION = 'LAST';
EXECUTE DBT PROJECT runs a dbt project object (or a project in a workspace) with the dbt command and options you pass in ARGS (Snowflake documentation).
One SQL statement is all Airflow needs to fire. Here is the full setup to make that work.
Step 1: Snowflake setup (roles, warehouse and permissions)
I always start here, because bad permissions cause the most confusing failures, and they surface late in the process when you’re tired and frustrated.
Snowflake native dbt command — executes the versioned DBT PROJECT object in the warehouse.
USE ROLE ACCOUNTADMIN; -- Create a dedicated role for dbt executionCREATE OR REPLACE ROLE dbt_executor_role;GRANT ROLE dbt_executor_role TO ROLE SYSADMIN; -- Create the service user Airflow will useCREATE OR REPLACE USER airflow_svc_user PASSWORD = 'YourStrongPassword123!' DEFAULT_ROLE = dbt_executor_role DEFAULT_WAREHOUSE = dbt_transform_wh COMMENT = 'Airflow service user for dbt orchestration'; GRANT ROLE dbt_executor_role TO USER airflow_svc_user; -- Create a dedicated warehouse for dbt runsUSE ROLE SYSADMIN;CREATE OR REPLACE WAREHOUSE dbt_transform_wh WITH WAREHOUSE_SIZE = 'SMALL' AUTO_SUSPEND = 120 AUTO_RESUME = TRUE INITIALLY_SUSPENDED = TRUE; GRANT ALL ON WAREHOUSE dbt_transform_wh TO ROLE dbt_executor_role; -- Grant database and schema privilegesGRANT USAGE ON DATABASE analytics_db TO ROLE dbt_executor_role;GRANT USAGE, CREATE TABLE, CREATE VIEW ON SCHEMA analytics_db.staging TO ROLE dbt_executor_role;GRANT USAGE, CREATE TABLE, CREATE VIEW ON SCHEMA analytics_db.marts TO ROLE dbt_executor_role; -- Grant the ability to execute dbt project objectsGRANT EXECUTE DBT PROJECT ON SCHEMA analytics_db.transforms TO ROLE dbt_executor_role;
This creates a dedicated role, a service user for Airflow, a Small warehouse that suspends after two minutes, and the grants dbt needs to build into the staging and marts schemas. The dbt project object itself lives in a third schema, transforms. Replace the password and the database and schema names with your own.
I made a mistake my first time through: I granted object-level access but forgot the schema-level EXECUTE DBT PROJECT privilege, which is separate. The error message wasn’t obvious. Save yourself that 20-minute debugging session.
Step 2: Deploy your dbt project as a native Snowflake object
This is the step that feels most different from a traditional dbt Core setup. You’re not installing dbt on a server; you’re registering your project inside Snowflake.
Option A: Snowsight Workspaces (recommended the first time)
Log into Snowsight, go to Workspaces and connect it to your Git repository:
SQL statement — review schema and warehouse context before executing.
-- First, create an API integration for GitHubCREATE OR REPLACE API INTEGRATION github_integration API_PROVIDER = git_https_api API_ALLOWED_PREFIXES = ('https://github.com/yourorg/') ENABLED = TRUE; -- Create the Git repository object in SnowflakeCREATE OR REPLACE GIT REPOSITORY dbt_project_repo API_INTEGRATION = github_integration GIT_CREDENTIALS = my_github_secret ORIGIN = 'https://github.com/yourorg/your-dbt-project.git';
The API integration allows Snowflake to reach your GitHub organization, and the Git repository object points at the repo itself. my_github_secret is a Snowflake secret holding your Git credentials, which you need to create first.
Option B: Deploy with SQL (good for CI/CD)
SQL statement — review schema and warehouse context before executing.
-- Create the DBT PROJECT object from your connected Git repoCREATE OR REPLACE DBT PROJECT analytics_db.transforms.sales_dbt_project FROM GIT REPOSITORY dbt_project_repo REF = 'main' TARGET_PATH = 'models/' WAREHOUSE = dbt_transform_wh;
Install dbt dependencies
You populate the dbt_packages folder by running dbt deps, either in a Snowflake workspace, on your local machine or in your Git-based CI:
Snowflake native dbt command — executes the versioned DBT PROJECT object in the warehouse.
-- Run this once after creating the project, or include in CI/CDEXECUTE DBT PROJECT analytics_db.transforms.sales_dbt_project ARGS = 'dbt deps' VERSION = 'LAST';
A heads-up: running dbt deps inside Snowflake Workspaces needs an external access integration, because the runtime has to reach external package repositories. The alternative is to run dbt deps locally or in CI and ship the populated dbt_packages folder with your deployment.
I found it cleaner to run dbt deps in my GitHub Actions pipeline and commit the dbt_packages folder, rather than configuring external access integrations for every environment. Your call; both approaches work.
Verify the deployment
Snowflake native dbt command — executes the versioned DBT PROJECT object in the warehouse.
-- Check your dbt project versionsSHOW DBT PROJECTS IN SCHEMA analytics_db.transforms; -- Test execute manually before wiring AirflowEXECUTE DBT PROJECT analytics_db.transforms.sales_dbt_project ARGS = 'dbt compile' VERSION = 'LAST';
If dbt compile completes without error, your project is live and ready for Airflow to call.
Step 3: Set up a real dbt project structure
Here is what the project actually looks like. I’m using a sales pipeline as the example: raw orders come in, we stage them, and we build a daily summary fact table.
dbt_project.yml
YAML configuration — sets name, version, config-version.
name: 'sales_pipeline'version: '1.0.0'config-version: 2 profile: 'snowflake_prod' model-paths: ["models"]test-paths: ["tests"]seed-paths: ["seeds"] models: sales_pipeline: staging: +schema: staging +materialized: view marts: +schema: marts +materialized: table
Staging models build as views in the staging schema and marts build as tables in the marts schema, which matches the grants from Step 1.
models/staging/stg_orders.sql
SQL statement — review schema and warehouse context before executing.
-- Staging model: clean and type-cast raw ordersWITH raw AS ( SELECT * FROM {{ source('raw', 'orders_raw') }}), cleaned AS ( SELECT order_id::VARCHAR AS order_id, customer_id::VARCHAR AS customer_id, order_date::DATE AS order_date, UPPER(TRIM(status)) AS order_status, amount::DECIMAL(18, 2) AS order_amount, region::VARCHAR AS region, CURRENT_TIMESTAMP() AS _loaded_at FROM raw WHERE order_id IS NOT NULL AND order_date >= '2023-01-01') SELECT * FROM cleaned
The staging model casts each column to its proper type, normalizes the status text, drops rows without an order ID and keeps orders from 2023 onwards.
models/marts/fct_daily_orders.sql
SQL statement — review schema and warehouse context before executing.
-- Fact table: daily order summary by regionWITH staged AS ( SELECT * FROM {{ ref('stg_orders') }}) SELECT order_date, region, order_status, COUNT(DISTINCT order_id) AS total_orders, COUNT(DISTINCT customer_id) AS unique_customers, SUM(order_amount) AS total_revenue, AVG(order_amount) AS avg_order_value, SUM(CASE WHEN order_status = 'RETURNED' THEN order_amount ELSE 0 END) AS returned_amount, CURRENT_TIMESTAMP() AS _refreshed_atFROM stagedGROUP BY order_date, region, order_statusORDER BY order_date DESC, region
The fact table aggregates staged orders by day, region and status, including the value of returned orders.
models/staging/sources.yml
YAML configuration — sets version, sources.
version: 2 sources: - name: raw database: analytics_db schema: raw_landing tables: - name: orders_raw description: "Raw orders from the source system" columns: - name: order_id tests: - not_null - unique - name: customer_id tests: - not_null - name: order_date tests: - not_null - name: amount tests: - not_null
models/marts/schema.yml
YAML configuration — sets version, models.
version: 2 models: - name: fct_daily_orders description: "Daily order summary by region and status" columns: - name: order_date tests: - not_null - name: total_orders tests: - not_null - name: total_revenue tests: - not_null
This gives us a clean, testable project with source tests and column-level tests. When Airflow executes dbt build, all of it runs (models and tests) in dependency order.
Step 4: Wire it together in Airflow
Now the fun part: a complete Airflow DAG that:
- Validates that raw data arrived in Snowflake
- Fires the native dbt project execution
- Validates row counts on the output marts
- Marks the pipeline complete, with email alerts on failure
First, install the Snowflake provider if you haven’t:
CLI command, run it in a terminal with your own credentials and environment.
pip install apache-airflow-providers-snowflake
Set up your Snowflake connection in the Airflow UI (Admin → Connections):
Configuration snippet — adapt parameters to your environment.
Connection ID : snowflake_analyticsConnection Type : SnowflakeAccount : yourorg.us-east-1Login : airflow_svc_userPassword : YourStrongPassword123!Schema : transformsDatabase : analytics_dbWarehouse: dbt_transform_whRole : dbt_executor_role
Now the DAG, dags/sales_pipeline_dag.py:
Python orchestration DAG (SQLExecuteQueryOperator) — verify connection IDs and retry arguments.
from airflow import DAGfrom airflow.providers.snowflake.operators.snowflake import SQLExecuteQueryOperatorfrom airflow.operators.python import PythonOperator, BranchPythonOperatorfrom airflow.operators.empty import EmptyOperatorfrom airflow.utils.dates import days_agofrom datetime import datetime, timedeltaimport logging # ── Default args ────────────────────────────────────────────────default_args = { 'owner': 'data-engineering', 'depends_on_past': False, 'retries': 1, 'retry_delay': timedelta(minutes=5), 'email_on_failure': True, 'email': ['[email protected]'],} SNOWFLAKE_CONN = 'snowflake_analytics' # ── SQL snippets ─────────────────────────────────────────────────RAW_DATA_CHECK_SQL = """SELECT COUNT(*) AS raw_row_countFROM analytics_db.raw_landing.orders_rawWHERE order_date = CURRENT_DATE() - 1;""" EXECUTE_DBT_SQL = """EXECUTE DBT PROJECT analytics_db.transforms.sales_dbt_project ARGS = 'dbt build --select staging.stg_orders+ --vars "{\\"run_date\\": \\"{{ ds }}\\"}"' VERSION = 'LAST';""" MART_VALIDATION_SQL = """SELECT COUNT(*) AS mart_row_count, MAX(order_date) AS latest_date, SUM(total_revenue) AS total_revenueFROM analytics_db.marts.fct_daily_ordersWHERE order_date = CURRENT_DATE() - 1;""" ROW_COUNT_GUARD_SQL = """SELECT CASE WHEN COUNT(*) = 0 THEN 'FAIL: No rows found in mart for yesterday' ELSE 'PASS: ' || COUNT(*) || ' rows present' END AS validation_resultFROM analytics_db.marts.fct_daily_ordersWHERE order_date = CURRENT_DATE() - 1;""" # ── DAG definition ───────────────────────────────────────────────with DAG( dag_id='sales_pipeline_end_to_end', default_args=default_args, description='End-to-end sales pipeline: raw → dbt native project → marts', schedule_interval='0 6 * * *', # 6 AM UTC daily start_date=days_ago(1), catchup=False, tags=['snowflake', 'dbt', 'sales'],) as dag: # Task 1: Check raw data arrived check_raw_data = SQLExecuteQueryOperator( task_id='check_raw_data_arrived', conn_id=SNOWFLAKE_CONN, sql=RAW_DATA_CHECK_SQL, handler=lambda cursor: logging.info( f"Raw row count: {cursor.fetchone()[0]}" ), ) # Task 2: Execute the native dbt project on Snowflake run_dbt_project = SQLExecuteQueryOperator( task_id='execute_dbt_project_snowflake', conn_id=SNOWFLAKE_CONN, sql=EXECUTE_DBT_SQL, # Give dbt build enough time for large projects execution_timeout=timedelta(hours=2), ) # Task 3: Post-run mart validation validate_mart_output = SQLExecuteQueryOperator( task_id='validate_mart_output', conn_id=SNOWFLAKE_CONN, sql=ROW_COUNT_GUARD_SQL, handler=lambda cursor: logging.info( f"Validation result: {cursor.fetchone()[0]}" ), ) # Task 4: Run broader stats query (logged for observability) log_mart_stats = SQLExecuteQueryOperator( task_id='log_mart_statistics', conn_id=SNOWFLAKE_CONN, sql=MART_VALIDATION_SQL, ) # Task 5: Success marker pipeline_complete = EmptyOperator(task_id='pipeline_complete') # ── Dependencies ───────────────────────────────────────────── ( check_raw_data >> run_dbt_project >> validate_mart_output >> log_mart_stats >> pipeline_complete )
The DAG runs every day at 6 AM UTC. It logs how many raw orders arrived for yesterday, runs dbt build on stg_orders and everything downstream of it (passing Airflow’s run date {{ ds }} as a dbt variable), checks that the mart has rows for yesterday, logs summary statistics and finishes on an empty marker task. It starts at the raw-data check, so loading the raw data (Task 1 in the diagram) and the Slack step happen outside this DAG. Note that the raw-data check and the row-count guard only log their results; they won’t stop the run on their own.
Step 5: Run specific dbt selectors from Airflow
One of the things I really like about this approach is that dbt’s full selector syntax passes straight through the ARGS parameter. You don’t have to run the entire project every time.
Run only staging models:
Snowflake native dbt command — executes the versioned DBT PROJECT object in the warehouse.
EXECUTE_STAGING_ONLY = """EXECUTE DBT PROJECT analytics_db.transforms.sales_dbt_project ARGS = 'dbt run --select staging.*' VERSION = 'LAST';"""
Run a specific model and everything downstream of it:
Snowflake native dbt command — executes the versioned DBT PROJECT object in the warehouse.
EXECUTE_ORDERS_DOWNSTREAM = """EXECUTE DBT PROJECT analytics_db.transforms.sales_dbt_project ARGS = 'dbt build --select stg_orders+' VERSION = 'LAST';"""
Run tests only, separately from the model run:
Snowflake native dbt command — executes the versioned DBT PROJECT object in the warehouse.
RUN_DBT_TESTS = """EXECUTE DBT PROJECT analytics_db.transforms.sales_dbt_project ARGS = 'dbt test --select staging.*' VERSION = 'LAST';"""
This means you can split a single DAG into multiple tasks (one for staging, one for marts, one for tests) and get granular retry behavior in Airflow. If something fails mid-pipeline, Airflow retries only the failed task instead of rerunning everything.
Here’s that pattern as a DAG:
Snowflake native dbt command — executes the versioned DBT PROJECT object in the warehouse.
run_staging = SQLExecuteQueryOperator( task_id='run_dbt_staging', conn_id=SNOWFLAKE_CONN, sql=""" EXECUTE DBT PROJECT analytics_db.transforms.sales_dbt_project ARGS = 'dbt run --select staging.*' VERSION = 'LAST'; """,) test_staging = SQLExecuteQueryOperator( task_id='test_dbt_staging', conn_id=SNOWFLAKE_CONN, sql=""" EXECUTE DBT PROJECT analytics_db.transforms.sales_dbt_project ARGS = 'dbt test --select staging.*' VERSION = 'LAST'; """,) run_marts = SQLExecuteQueryOperator( task_id='run_dbt_marts', conn_id=SNOWFLAKE_CONN, sql=""" EXECUTE DBT PROJECT analytics_db.transforms.sales_dbt_project ARGS = 'dbt run --select marts.*' VERSION = 'LAST'; """,) run_staging >> test_staging >> run_marts
This is how I actually run it in practice. If staging tests fail, marts never execute. If marts fail, I retry marts without re-running staging. Clean dependency management with very little code.
Job-level vs model-level orchestration
Splitting the project by layer, as above, sits between the two approaches the Airflow and dbt community has argued about for years: run all of dbt as one Airflow task (job-level), or give every model its own task (model-level).

Job-level: one task for the whole run
One Airflow task runs something like dbt run --select tag:daily. It’s simple: one task, one line of configuration, fast deploys. Most teams picked it for that reason. The downside is that if one model fails in the middle of a 400-model run, the whole job fails, and you have to dig through dbt logs to find which model actually broke.
Model-level: one task per model
Astronomer’s open-source Cosmos library changed that. You point Cosmos at your dbt project directory and it generates the Airflow DAG, one task per model, without hand-written task definitions. Parsing 400 models still costs something, roughly 1 to 2 seconds added to DAG parsing, but that’s now acceptable. A failed model shows up as a red task in the Airflow UI with its compile error, instead of “dbt run exited with code 1” in an SSH log.
Model-level orchestration gives every model its own Airflow task, so failures are scoped to a model and Airflow’s UI shows exactly where the pipeline broke. It used to carry a penalty: rendering 400 models as 400 tasks loaded the Airflow scheduler, DAG parsing time doubled, and the dynamic task generation you had to write was brittle.
A 400-model example
One team running Airflow on AWS (3x m5.2xlarge), dbt Core on a separate EC2 box and a MEDIUM Snowflake warehouse compared the two:
- Job-level with external dbt:
dbt run --select tag:dailyas a single daily task took 18 minutes wall-clock. A failed model was buried in the dbt log, and debugging took 45+ minutes of correlating dbt logs, Snowflake QUERY_HISTORY and Airflow task logs. - Model-level with Cosmos and Snowflake-native dbt: the same 400 models as 400 Cosmos-generated tasks, up to 8 running in parallel on the MEDIUM warehouse, took 12 minutes (33% faster). The failed model, its compile error and the exact line were visible from the red task. Debugging took 8 minutes.
The speed gain comes from running independent models in parallel; the debugging gain comes from per-model visibility.
Which one to use
Job-level still makes sense if your project is small (under 50 models), it fits in a single logical unit, you don’t need per-model visibility, or you’re on dbt Cloud’s own scheduler rather than Airflow.
Model-level makes sense if you have 100+ models, need per-model failure isolation, care about debugging speed, or want Airflow to be the single source of truth for the whole pipeline.
The hybrid: some teams run both in the same DAG: one job-level task for hourly incremental loads (dbt run --select tag:hourly) and a Cosmos task group for the daily mart builds where visibility matters.
The principle I’d use to decide: observability at execution time beats simplicity at configuration time. Job-level is simpler to configure; model-level is simpler to debug. In production you spend more time debugging than configuring, so pick the granularity that lets you see failures where they happen.
Step 6: Handle new versions of your dbt project
I didn’t think about this until I pushed a breaking change to main and my 6 AM pipeline executed the wrong version.
The DBT PROJECT object is versioned: each change made through ALTER adds a new version, named VERSION$<num>.
In practice, your CI/CD pipeline (GitHub Actions or similar) should update the DBT PROJECT object after every merge to main:
CLI file (.github/workflows/deploy_dbt.yml) — review parameters before deploying.
# .github/workflows/deploy_dbt.ymlname: Deploy dbt Project to Snowflake on: push: branches: [main] jobs: deploy: runs-on: ubuntu-latest steps: - uses: actions/checkout@v3 - name: Install Snowflake CLI run: pip install snowflake-cli-labs - name: Deploy new dbt project version env: SNOWFLAKE_ACCOUNT: ${{ secrets.SNOWFLAKE_ACCOUNT }} SNOWFLAKE_USER: ${{ secrets.SNOWFLAKE_USER }} SNOWFLAKE_PASSWORD: ${{ secrets.SNOWFLAKE_PASSWORD }} run: | snow dbt deploy \ --project-name analytics_db.transforms.sales_dbt_project \ --from-git \ --ref main
On every push to main, the workflow installs the Snowflake CLI and deploys a new version of the project, reading the account, user and password from GitHub secrets.
In your Airflow SQL, VERSION = 'LAST' always picks up the most recently deployed version. Once CI/CD deploys a new version, the next DAG run uses it with no Airflow changes.
Step 7: Monitoring (what to watch and where)
Before this setup, I was watching three screens at once when something went wrong. Now it’s mostly one.
In Snowsight:
Snowflake native dbt command — executes the versioned DBT PROJECT object in the warehouse.
-- Check recent dbt project execution historySELECT query_id, query_text, execution_status, start_time, end_time, DATEDIFF('second', start_time, end_time) AS duration_seconds, error_messageFROM TABLE( INFORMATION_SCHEMA.QUERY_HISTORY( END_TIME_RANGE_START => DATEADD('day', -1, CURRENT_TIMESTAMP()), RESULT_LIMIT => 50 ))WHERE query_text ILIKE '%EXECUTE DBT PROJECT%'ORDER BY start_time DESC;
This lists the last day’s EXECUTE DBT PROJECT statements with their status, duration in seconds and any error message, newest first.
Row count drift detection, added as an Airflow task:
SQL statement — review schema and warehouse context before executing.
-- Compare today's mart row count to yesterday's-- Flag if it drops more than 20%WITH today AS ( SELECT COUNT(*) AS cnt FROM analytics_db.marts.fct_daily_orders WHERE order_date = CURRENT_DATE() - 1),yesterday AS ( SELECT COUNT(*) AS cnt FROM analytics_db.marts.fct_daily_orders WHERE order_date = CURRENT_DATE() - 2)SELECT today.cnt AS today_rows, yesterday.cnt AS yesterday_rows, ROUND((today.cnt - yesterday.cnt) / NULLIF(yesterday.cnt, 0) * 100, 2) AS pct_change, CASE WHEN today.cnt < yesterday.cnt * 0.80 THEN 'ALERT: Row count dropped over 20%' ELSE 'OK' END AS statusFROM today, yesterday;
I added this query as a SQLExecuteQueryOperator task right after the mart validation step. If the row count drops by more than 20% compared with the previous day, the task raises a warning in the Airflow logs and the email alert fires.
Not every data quality problem shows up as a dbt test failure. Sometimes the data just quietly shrinks because an upstream feed stopped delivering. This catches that.
Gotchas I hit along the way
1. EXECUTE DBT PROJECT is synchronous by default. Airflow waits for it to complete before marking the task done. For large projects that’s the behavior you want; just make sure the task’s execution_timeout is generous enough.
2. Cross-project references don’t work the way you might expect. The DBT PROJECT object can’t reference external file paths, so cross-project dependencies have to be copied into the root of the main project. If you have multiple dbt projects, plan your consolidation before deploying.
3. VERSION = 'LAST' always runs the most recently deployed version. To pin a specific version in production, use VERSION = 'VERSION$3' (or whichever number). I run LAST in dev and a pinned version in prod, deployed via CI/CD.
4. Warehouse auto-resume and the first task. The first EXECUTE DBT PROJECT of the day can have a few seconds of latency while dbt_transform_wh resumes. I added a lightweight warm-up query as the very first task in my DAG, so the warehouse is already running when dbt build starts:
SQL statement — review schema and warehouse context before executing.
warm_up_warehouse = SQLExecuteQueryOperator( task_id='warm_up_warehouse', conn_id=SNOWFLAKE_CONN, sql="SELECT CURRENT_TIMESTAMP();",) warm_up_warehouse >> check_raw_data >> run_dbt_project >> ...
It costs almost nothing and removes 5 to 10 seconds of variability at the start of every run.
5. dbt artifacts stay in Snowflake. When dbt runs inside Snowflake, manifest.json and the other run artifacts stay there too. If anything downstream uses them (dbt Cloud webhooks, dbt Mesh coordination, lineage tools), add a post-run task that exports them explicitly after each run.
What this setup actually changed for me
I want to be concrete here, because the “benefits” sections in most blog posts are too abstract.
Before: my pipeline had six moving parts. An Airflow DAG on one server. dbt installed on a separate instance. A profiles.yml with credentials that needed updating every time we rotated passwords. Separate monitoring in CloudWatch for the dbt server. Debugging a failure meant SSH to the dbt server, find the log file, then cross-reference it with the Airflow logs.
After: the pipeline has three moving parts: Airflow, Snowflake and GitHub. The dbt credentials are managed by Airflow’s Snowflake connection, which I was already maintaining. Debugging a failure means clicking into the Airflow task logs (which capture the SQL response from Snowflake) and, if I need more detail, running the QUERY_HISTORY query above in Snowsight.
Performance has also improved since preview: result upload, which took roughly 6 to 6.5 minutes during preview, now completes in around 40 to 45 seconds, about 8 to 10x faster.
The startup time improvement alone was worth it for me. My morning pipeline used to take 28 to 32 minutes. It now consistently runs in 18 to 22 minutes. That’s not from faster models; it’s from removing environment spin-up overhead.
Why I think this is the right direction
I started exploring this because nobody told me to. My team’s existing setup worked, and a reasonable person would have left it alone.
But the more I looked at it, the more I thought about the overhead we carry when tools don’t talk to each other natively. Every boundary between systems is a place where credentials leak, latency is added and debugging gets harder. The native dbt project in Snowflake closes one of those boundaries. Airflow still owns orchestration, which is where it belongs, but the transformation runs where the data lives.
One line I read about this stuck with me:
For the growing number of organizations that have standardized on Snowflake, the native integration offers something genuinely compelling: one fewer system to run, one fewer vendor to manage, and one fewer boundary between your data and the logic that transforms it.
That’s exactly what this is.
If you’ve been running dbt Core on a server with Airflow alongside it, and you’ve tolerated the overhead for so long that you’ve stopped noticing it, try this weekend rebuild. You might be surprised how much lighter the pipeline feels on the other side.
And if you try it and hit something weird, drop it in the comments. I’m still learning this myself.
Related reading: dbt State on Snowflake: skip unchanged models, cut runtime 60% · dbt Fusion: 30x faster parsing with the Rust engine · Data contracts: stop schema breakage before it happens · Official dbt and Airflow integration guide
Questions this article answers
Short answers first. Open a question to read the working note.
What a dbt Project on Snowflake actually is?
The terminology can trip you up, and I don't want you 45 minutes into setup before the confusion hits. dbt Projects on Snowflake let you create, edit, test, run and manage dbt Core projects with Snowflake's own tools. You work on the project files in Snowsight Workspaces, then deploy the project as a schema-level dbt PROJECT object.
Which one to use?
Job-level still makes sense if your project is small (under 50 models), it fits in a single logical unit, you don't need per-model visibility, or you're on dbt Cloud's own scheduler rather than Airflow. Model-level makes sense if you have 100+ models, need per-model failure isolation, care about debugging speed, or want Airflow to be the single source of truth for the whole pipeline.
What this setup actually changed for me?
I want to be concrete here, because the "benefits" sections in most blog posts are too abstract. Before: my pipeline had six moving parts. An Airflow DAG on one server. dbt installed on a separate instance. A profiles.yml with credentials that needed updating every time we rotated passwords. Separate monitoring in CloudWatch for the dbt server. Debugging a failure meant SSH to the dbt server, find the log file, then cross-reference it with the Airflow logs.
