How Snowflake Stores Data Internally: Micro-Partitions, FDN & Pruning

A team I worked with ran a nightly job that did something completely reasonable: it updated a single status flag on rows in a 2 TB orders table. One...

A team I worked with ran a nightly job that did something completely reasonable: it updated a single status flag on rows in a 2 TB orders table. One column. A few million rows a night. Harmless. Three months later their storage bill had nearly quadrupled, and no one had loaded any new data. The culprit wasn’t the update itself — it was a fact about Snowflake’s storage engine that almost nobody internalizes until it bites them: you cannot change a row in place. That one-column update was silently rewriting entire micro-partitions and stockpiling the old versions in Time Travel and Fail-safe, and they were paying to store every generation.

There’s a popular assumption that Snowflake just stores compressed Parquet files behind the scenes. It doesn’t. And the real design isn’t trivia — it’s the thing that explains why your queries prune well or scan everything, why a tiny update can be expensive, and why your storage bill has line items you never created directly. This is the storage layer, specifically — not the “cloud services” box everyone waves at in architecture diagrams. If you want the compute-and-query side, I’ve covered what really happens when you run a query separately; this article is about what’s actually sitting on disk, and why it dictates so much of your day.

TL;DR

→ Snowflake does not store Parquet — it stores its own proprietary columnar format (widely called FDN) in files called micro-partitions on cloud object storage.
→ Each micro-partition holds 50–500 MB of uncompressed data, stored column-by-column and compressed per column, with a metadata header of per-column min/max, distinct counts, and null counts.
→ That metadata — not the data — lives in FoundationDB and drives pruning: the optimizer skips whole micro-partitions by reading statistics, never touching the files.
→ Micro-partitions are immutable; every UPDATE, DELETE, or MERGE rewrites whole partitions rather than editing rows in place.
→ Immutability is why Time Travel, zero-copy cloning, and Fail-safe exist — and why churny small DML quietly inflates storage.
→ Clustering (natural by load order, or a defined key) controls how well pruning works; you can measure it with SYSTEM$CLUSTERING_INFORMATION.
→ You don’t tune Snowflake storage directly — you influence it by how you load, update, and cluster data.

The three layers, and why storage is the interesting one

Snowflake separates into three layers: cloud services (the brain — metadata, security, the optimizer), compute (virtual warehouses that run queries), and storage (the data itself). The famous selling point is that compute and storage are decoupled, which is why you can resize a warehouse without touching data and why the warehouse cache behaves the way it does. But the layer that quietly determines your costs and query speed is the bottom one — and it’s the one people understand the least.

What’s actually on disk: micro-partitions, not Parquet

When you load data into a table, Snowflake automatically slices it into micro-partitions. Per Snowflake’s own documentation, each one holds between 50 MB and 500 MB of uncompressed data (smaller on disk, since it’s always compressed), and the rows in it are stored in a columnar layout — each column laid out and compressed independently, with Snowflake picking the best compression scheme per column. A large table isn’t a handful of big partitions; it can be millions of these small, uniform files.

The file format is proprietary and closed — commonly referred to as FDN (“Flocon de Neige,” French for snowflake). It is emphatically not Parquet, even though both are columnar and compressed. Why build a custom format instead of reusing an open one? Because the format is co-designed with the metadata and pruning system, and that tight coupling is where Snowflake’s speed comes from. The anatomy diagram above is the mental model to hold: a header of statistics, then column chunks.

The metadata is the magic (and it lives in FoundationDB)

Here’s the part that reframes everything. For every micro-partition, Snowflake records metadata — the range of values (min and max) for each column, the number of distinct values, null counts, and more. That metadata is not stored in the file with the data; it lives in Snowflake’s metadata store, which Snowflake has publicly described as being built on FoundationDB, a distributed key-value store. The logical table you query is really a set of pointers, held in that metadata store, mapping to the physical FDN files in object storage.

This separation is what makes pruning possible. When you filter on a column, the optimizer consults the min/max metadata for each micro-partition and skips any whose range can’t possibly contain a match — without reading the file at all.

Pruning reads statistics, not data. Only the partition whose min/max range straddles July 4th is opened; the other four are eliminated before any I/O.

This is also why two SQL habits quietly kill performance. Wrapping a filter column in a function — WHERE DATE(order_ts) = '2026-07-04' — means the min/max stats on the raw order_ts column can’t be used, so pruning is defeated and every partition gets scanned. And SELECT * forces every column chunk to be read even though the columnar layout was designed to let Snowflake read only the columns you asked for. The half-open range order_ts >= '2026-07-04' AND order_ts < '2026-07-05' on the bare column, selecting only needed columns, is what lets both optimizations fire.

Immutability changes how you think about DML

Micro-partitions are immutable. Snowflake never edits a row in place — a truth that has bigger consequences than it first appears. When you run an UPDATE, DELETE, or MERGE, Snowflake reads the affected micro-partitions, produces new micro-partitions with the changes applied, and re-points the table’s metadata at the new files. The old files don’t vanish; they’re retained for Time Travel, then Fail-safe.

That’s the mechanism behind the quadrupled bill from the intro. A one-column update touching rows spread across thousands of micro-partitions rewrites all of those partitions in full — and keeps the previous versions around for the retention window. This same copy-on-write design is exactly why Time Travel isn’t really a backup feature — it’s just versioned pointers to immutable partitions you already paid to store — and why zero-copy clones are nearly free at creation: a clone is a new set of pointers to the same existing files, and it only costs storage as the two copies diverge and new partitions get written.

Seeing it for yourself

None of this has to be taken on faith. You can inspect the physical reality directly. To see how well a table is clustered — how much its micro-partitions overlap on a column, which is what determines pruning quality — use the built-in function:

SELECT SYSTEM$CLUSTERING_INFORMATION('sales', '(order_date)');

{
  "cluster_by_keys" : "LINEAR(order_date)",
  "total_partition_count" : 12483,
  "average_overlaps" : 3.11,
  "average_depth" : 2.94,
  "partition_depth_histogram" : {
    "00000" : 0,
    "00001" : 4821,
    "00002" : 5102,
    "00004" : 2560
  }
}

A low average_depth means a filter on order_date hits few overlapping partitions — good pruning. A high depth means the values are smeared across many partitions and queries scan more than they should. To see the storage consequences of immutability, query the account usage view that breaks storage into its real components:

SELECT
    table_name,
    active_bytes      / POW(1024, 3) AS active_gb,
    time_travel_bytes / POW(1024, 3) AS time_travel_gb,
    failsafe_bytes    / POW(1024, 3) AS failsafe_gb
FROM snowflake.account_usage.table_storage_metrics
WHERE table_name = 'ORDERS'
ORDER BY active_bytes DESC;

+------------+-----------+----------------+-------------+
| TABLE_NAME | ACTIVE_GB | TIME_TRAVEL_GB | FAILSAFE_GB |
+------------+-----------+----------------+-------------+
| ORDERS     |    198.4  |         912.7  |      301.5  |
+------------+-----------+----------------+-------------+

That’s the intro’s bug, made visible: 198 GB of live data, but ~1.2 TB of retained old versions you’re billed for, generated by churny updates rewriting partitions night after night.

Clustering: natural, defined, and not free

By default, data is naturally clustered by the order it was loaded — load July’s data in date order and date-filtered queries prune beautifully; load it shuffled and they don’t. For large tables with a consistent filter pattern, you can define a clustering key and Snowflake’s Automatic Clustering service will reorganize micro-partitions in the background to keep them well-sorted. It genuinely helps pruning, and it’s central to keeping interactive dashboards fast.

But reclustering rewrites micro-partitions (immutability again), which consumes credits and generates yet more retained versions. Clustering keys earn their keep only on large tables that are frequently filtered on the key. Adding one to a small or write-heavy table often costs more than it saves — this is the same “don’t pay to reprocess what didn’t change” discipline behind dbt state-based selection.

The cost math

Put rough numbers on the immutability tax. Say you have a 200 GB table and a job that rewrites 20% of its partitions daily — a modest MERGE. With a 7-day Time Travel window, you’re retaining roughly seven days of old versions of that churned 20%: on the order of 200 GB of Time Travel bytes on top of the 200 GB active, plus a further ~7 days of Fail-safe Snowflake keeps regardless. You can easily end up billed for 3–4x your live data size. The fix isn’t to stop updating — it’s to batch changes so you rewrite partitions once instead of trickling small updates, to right-size Time Travel retention per table, and to be deliberate about which tables actually need churn. Storage is cheap per GB, but “cheap × 4 × forever” is a real line on the invoice.

FDN vs open formats

The proprietary FDN format is what gives Snowflake its pruning speed and features — but it also means your data is locked inside Snowflake’s engine. That trade-off is exactly what the industry’s shift toward open table formats is reacting to: Iceberg tables let you keep data as open Parquet on your own object storage, readable by other engines, at some cost in Snowflake-native performance and features. If you’re weighing it, I’ve written on the native-to-Iceberg migration trap and when it’s actually worth migrating. The short version: FDN for Snowflake-first workloads, Iceberg when open access matters more than raw speed.

The gotchas nobody warns you about

Small, frequent updates are a storage trap. Single-row updates to a big table rewrite whole micro-partitions and pile up retained versions. Batch DML; don’t trickle it.

Functions on filter columns silently disable pruning. DATE(ts) = ..., UPPER(name) = ..., or a cast on the filtered column all prevent Snowflake from using the raw column’s min/max stats. Filter on the bare column with ranges.

Load order is a performance decision. Because natural clustering follows insertion order, loading data shuffled destroys pruning for range queries. Sort on load, or accept the cost of a clustering key.

DELETE doesn’t free storage immediately. Deleted rows live on as old partition versions through Time Travel and Fail-safe. If you need space back fast, that retention window matters.

Overlapping ranges beat pruning even with a clustering key. A high average_depth from SYSTEM$CLUSTERING_INFORMATION means your “clustered” table still scans widely. Measure it; don’t assume the key is working.

The one principle

Snowflake doesn’t store rows — it stores immutable, self-describing micro-partitions, and every cost and speed characteristic you experience is a downstream consequence of that one fact. Pruning is the metadata header doing its job. Expensive updates are immutability doing its job. Time Travel, cloning, and your storage bill are all the same design viewed from different angles. Once you see storage as immutable statistics-wrapped column chunks, Snowflake stops being magic and starts being predictable — which is exactly when you can make it cheap and fast.