dbt (data build tool) is the standard framework for data transformation inside the warehouse. It enables data teams to build, test, document, and deploy SQL-based transformation pipelines using software engineering best practices. Master dbt models, custom testing, documentation generation, and native data warehouse integrations (like Snowflake and BigQuery). Explore our collection of 17 in-depth articles about dbt on DataEngineer Hub. Each tutorial provides practical, hands-on guidance with real-world examples to help you master dbt concepts and best practices.
Every hour, on a schedule, your dbt job rebuilds every model in your project. The staging models run because they always run. The intermediate joins run because they always run….
Two dbt models can be byte-for-byte identical in output, pass every test, and read cleanly in review — and one of them costs $50 a month while the other costs…
The 2 a.m. page said the pipeline “succeeded.” The dashboard was green. And the finance team was still staring at yesterday’s numbers, because one task in a forty-task DAG had…
Our CI got slower and more expensive at exactly the same rate our test suite got better, and for a while nobody connected the two. We’d done everything the best-practice…
I once inherited an Airflow repo with 214 DAG files that were, functionally, the same DAG. Each one extracted a table from a source system, loaded it into Snowflake, and…
For years, the pattern was: Airflow sits in one corner of your infrastructure, dbt runs on a server somewhere else, they pass data between each other via manual credential handoffs…
You read about dbt State. You understood the pitch — skip unchanged models, cut compute, stop paying to rebuild what didn’t change. Then you opened your project and stared at…
You have a 400-model dbt project. A junior analyst tweaks one source definition. Every. Single. Model. Rebuilds. Ninety minutes later, you’ve burned $1,200 in warehouse compute for a change that…
You’ve probably heard the buzz: dbt’s new Fusion engine is 30x faster. But what nobody says clearly is faster at what, for whom, and does it break your project? The answer is…
TL;DR→ Delta Lake is easier to start with, especially if you’re already on Databricks→ Iceberg wins on engine flexibility — works natively with Spark, Flink, Trino, Snowflake, and more without…
I passed the SnowPro Gen AI certification not too long ago. Within the same week I was back at my desk staring at a broken pipeline that no multiple-choice question…
Three practical methods to query Snowflake data in DuckDB — via Iceberg tables, ADBC, or a hybrid architecture — with real cost breakdowns showing 70–90% savings on BI and dev workloads.
I’ve been running dbt in production for a while now. And I’ll be honest — there was a phase where I genuinely believed that if my dbt tests were green,…
How I Wired Snowflake’s Native dbt Projects to Airflow — And Finally Got True End-to-End Orchestration I’ll be honest with you — for a long time I was running dbt…
The Moment Everything Changed It was a Tuesday morning when I finally snapped. My dbt project had grown to 147 models, and the daily run was taking 2 hours and…
Run dbt Core Directly in Snowflake Without Infrastructure Snowflake native dbt integration announced at Summit 2025 eliminates the need for separate containers or VMs to run dbt Core. Data teams…
If you’ve ever inherited a dbt project, you know there are two kinds: the clean, logical, and easy-to-navigate project, and the other kind—a tangled mess of models that makes you…