Explore our collection of 56 in-depth articles tagged with snowflake on DataEngineer Hub. Each tutorial provides practical, hands-on guidance with real-world examples to help you master snowflake concepts and best practices in data engineering.
The question came up in a Slack channel for a platform team I was advising: “We want to give our AI coding agent access to the pipeline metadata so it…
Your data engineer says the customer data already lives in Snowflake — all of it, clean, modeled, production-ready. Your architect wants to copy it into Salesforce Data Cloud. You’re running…
The 2 a.m. page said the pipeline “succeeded.” The dashboard was green. And the finance team was still staring at yesterday’s numbers, because one task in a forty-task DAG had…
The pitch is intoxicating and mostly true: keep your data in open Apache Iceberg format on your own object storage, let external engines read it, and let Snowflake’s Dynamic Tables…
In March 2026, two days after Snowflake shipped Cortex Code, security researchers at PromptArmor published something that should have changed how every data team thinks about AI agents. They didn’t…
Our CI got slower and more expensive at exactly the same rate our test suite got better, and for a while nobody connected the two. We’d done everything the best-practice…
The Snowflake bill for our CI account had roughly tripled over a quarter, and nobody could point to why. We hadn’t loaded meaningfully more data. Compute was flat. But storage…
The first thing I tried to do in CoCo Desktop was ask it to pull the open tickets for a data pipeline I was debugging. It couldn’t. Not because it…
The first time a product manager asked me why our “real-time” Snowflake dashboard took four seconds to load on a Monday morning, I didn’t have a good answer. The data…
For years, the pattern was: Airflow sits in one corner of your infrastructure, dbt runs on a server somewhere else, they pass data between each other via manual credential handoffs…
You read about dbt State. You understood the pitch — skip unchanged models, cut compute, stop paying to rebuild what didn’t change. Then you opened your project and stared at…
Your team has a 200GB Parquet file on S3. Someone suggests running the analysis in Spark. You spin up a four-node cluster, configure executors, tune shuffle partitions, wait three minutes…
Most data engineers I talk to still store everything in Snowflake native format. It’s simple: load data, query data, done. But here’s what nobody’s talking about: if you’re querying that…
Ask ten data engineers what happens when you run a query in Snowflake and most of them will tell you the same thing: the warehouse runs it. SQL goes in,…
A dashboard that ran in four seconds on Monday took nineteen seconds on Tuesday. Same query, same data, same warehouse size. I spent the better part of an hour convinced…
In 2018, a wave of Medium posts declared SQL obsolete. NoSQL was the future. Python would handle everything. Data lakes would make relational thinking irrelevant. The hot take had a…
TL;DR → Time Travel is not a backup — it’s a versioned metadata pointer to immutable micro-partitions you already paid to store→ Snowflake never overwrites data in place. Every UPDATE…
Snowflake just announced a lot at Summit 2026. Most of it was the usual conference noise. CoCo Desktop isn’t. I’ve been following Cortex Code — now officially rebranded as CoCo…
I passed the SnowPro Gen AI certification not too long ago. Within the same week I was back at my desk staring at a broken pipeline that no multiple-choice question…
Every time I demo Snowflake to someone new, zero-copy cloning gets the biggest reaction. You type one line. You get an instant copy of a table — or an entire…
Three practical methods to query Snowflake data in DuckDB — via Iceberg tables, ADBC, or a hybrid architecture — with real cost breakdowns showing 70–90% savings on BI and dev workloads.
I want to be clear about something before I say anything critical: Snowflake Tasks are genuinely good. I used them for months. I recommended them to people. I wrote internal…
When I first started using Cortex Code, cost was the last thing on my mind. It’s right there in the Snowsight UI, it feels like a built-in feature, and nothing…
After all of this, the real tell at the senior level isn’t whether you know all these answers. It’s whether you can connect them. The best signal a senior candidate…
Nobody told me to do this. No manager pinged me. No sprint ticket had “explore Cortex Code” written on it. I stumbled across it one evening while clicking around Snowsight…
The Moment Everything Changed It was a Tuesday morning when I finally snapped. My dbt project had grown to 147 models, and the daily run was taking 2 hours and…
⚡ TL;DR (Too Long; Didn’t Read) What it is: Snowflake Managed Iceberg Tables store data in your cloud storage (S3, GCS, Azure) instead of Snowflake’s storage, while Snowflake manages the…
Why Document Processing Matters in 2026 Enterprises store approximately 80-90% of their business data in unstructured formats—PDFs, Word documents, scanned images, contracts, invoices, and reports.
Snowflake Cortex AI matured significantly between 2023-2026, expanding from simple LLM functions to a comprehensive AI platform with AISQL, Cortex Search, Cortex Analyst, Document AI, and Agents.
The Night Everything Broke (And How Streams Saved Me) It was 2 AM on a Tuesday. My phone was buzzing non-stop. Our nightly ETL job had failed—again. This time, it…
Why Snowflake Costs Spiral Out of Control If your Snowflake bill jumped 200% last quarter while data volume only grew 30%, you’re not alone. I’ve audited dozens of Snowflake environments…
Last year, I interviewed for a Senior Data Engineer role at three different companies. All three used Snowflake heavily. All three asked completely different questions. The first interview?
Why I Started Exploring Snowflake Cortex AI Three months ago, I was sitting in a meeting where someone asked, “Can we analyze sentiment in these 50,000 customer reviews?” My immediate…
Prepare for the Snowflake GES-C01 exam with realistic practice questions. Master Snowflake Cortex, LLMs, and RAG pipelines to get certified in 2025.
I’ll be honest – when I first saw the OpenFlow announcement at Snowflake BUILD, my initial reaction was “Great, another data pipeline tool.” We already have dbt, Airflow, Fivetran, and…
When I first heard about building Retrieval-Augmented Generation (RAG) systems directly in Snowflake, I’ll admit I was skeptical. Could a data warehouse really handle AI workloads this seamlessly?
Modern data architectures are evolving rapidly, and Snowflake Cortex AISQL is at the forefront of this change.
I’ve spent the last few days working with Snowflake Intelligence, and I want to share what actually works—not just the marketing pitch. If you’re tired of being the bottleneck for…
Run dbt Core Directly in Snowflake Without Infrastructure Snowflake native dbt integration announced at Summit 2025 eliminates the need for separate containers or VMs to run dbt Core. Data teams…
When you think of aggregation functions in SQL, SUM(), COUNT(), and AVG() likely come to mind first. These are the workhorses of data analysis, undoubtedly. However, Snowflake, a titan in…
Revolutionary Declarative Data Pipelines That Transform ETL In 2025, Snowflake Dynamic Tables have become the most powerful way to build automated data pipelines.
Revolutionary SQL Features That Transform data engineering In 2025, Snowflake has introduced groundbreaking improvements that fundamentally change how data engineers write queries.
Revolutionary Performance Without Lifting a Finger On October 8, 2025, Snowflake unveiled Snowflake Optima—a groundbreaking optimization engine that fundamentally changes how data warehouses handle…
Breaking: Tech Giants Unite to Solve AI’s Biggest Bottleneck The Open Semantic Interchange was announced by Snowflake in their official blog On September 23, 2025, something unprecedented happened in…
The world of data analytics is changing. For years, accessing insights required writing complex SQL queries. However, the industry is now shifting towards a more intuitive, conversational approach.
In the realm of data warehousing, choosing the right schema design is crucial for efficient data management, querying, and analysis. Two of the most popular multidimensional schemas are the star…
Introduction to Data Pipelines in Python In today’s data-driven world, creating robust data pipelines solutions is essential for businesses to handle large volumes of information efficiently.
The financial services industry is in the midst of a technological revolution. At the heart of this change lies Artificial Intelligence.
The 60–80% Problem Killing Data Science Productivity Data science productivity is being crushed by the 60–80% problem.
Introduction: The Dawn of Context-Aware AI in Enterprise Data Enterprise AI is experiencing a fundamental shift in October 2025.
Snowflake is renowned for its incredible performance, but as data scales into terabytes and petabytes, no platform is immune to a slow-running query. For a data engineer, mastering Snowflake query…
Snowflake MERGE statements are powerful tools for upserting data, but poor optimization can lead to massive performance bottlenecks.
In Part 1 of our guide, we explored Snowflake’s unique architecture, and in Part 2, we learned how to load data. Now comes the most important part: turning that raw…
In Part 1 of our guide, we covered the revolutionary architecture of Snowflake. Now, it’s time to get hands-on. A data platform is only as good as the data within it, so understanding…
Stop memorizing the difference between a VARCHAR and a TEXT field. If you’re an experienced data engineer, you know that real Snowflake interviews go much deeper.
If you’ve ever inherited a dbt project, you know there are two kinds: the clean, logical, and easy-to-navigate project, and the other kind—a tangled mess of models that makes you…