Terraform is the YAML/HCL standard for infrastructure. Pulumi is the 'Infrastructure as Code' tool that lets you use real programming languages (Python, Go, JS).
I've stood up data platforms with both Terraform and Pulumi, and the tool choice matters less than most IaC religious wars would suggest — until the day it really, really matters. Both end up provisioning the same S3 buckets, the same Snowflake warehouses, the same VPCs. The difference shows up when you need dynamic behavior, when you need to test your infrastructure logic, and when you need to onboard someone who has never touched HCL before.
for_each and dynamic blocks start to hurt, and real Python/TypeScript loops start to shine.terratest, but it's heavier (spins up real resources) and more ceremonial.On a recent platform build, we had to provision per-tenant Snowflake databases, roles, warehouses, and resource monitors for about 200 tenants, with metadata stored in a Postgres config table. In Terraform, the for_each + dynamic dance worked but became a 400-line module that no one could read. We rewrote it in Pulumi (Python) in two days: a for-loop over a Postgres query, a function per resource, maybe 90 lines total, with actual pytest unit tests validating that a suspended tenant wouldn't provision a warehouse. That was the moment I stopped defaulting to Terraform for data infra.
That said, when we later needed to onboard a contractor who had five years of Terraform and zero Python, Pulumi's learning curve bit us. There's still 10× more Terraform talent in the market.
.tf files in HCL, a declarative DSL. terraform plan diffs your desired state against a state file (stored locally, in S3, or in Terraform Cloud) and shows what will change. terraform apply executes it against provider APIs (AWS, Snowflake, GCP, 3,000+ others).pulumi SDK. The program describes resources. pulumi up does the same plan-and-apply cycle, with state stored in Pulumi Cloud (free tier) or self-hosted. Under the hood, Pulumi often uses the same Terraform providers via a bridge — so the resource coverage is effectively the same.Choose Terraform if:
for_each+dynamic blocks for the occasional dynamic scenarioFor a new data team whose engineers are Python-fluent and whose infra involves lots of dynamic provisioning (per-tenant Snowflake, per-client S3 buckets, etc.), start with Pulumi. For a team embedded in a large org with existing Terraform tooling, Terraform Cloud, and a DevOps group that wants to own the platform pattern, stay on Terraform — you'll spend more time fighting organizational gravity than you'll save in ergonomics. And whichever you pick, write down the decision and the reasons in your README. Future you will want that note when someone proposes migrating for the third time.
Yes — and this is one of the reasons Pulumi's resource coverage is comparable to Terraform's out of the box. Pulumi's Terraform bridge auto-generates Pulumi packages from Terraform providers, so most AWS/GCP/Azure/Snowflake/Kubernetes resources work identically. The edge case is brand-new provider features that haven't yet been rebridged; typically there's a 1–2 week lag behind the upstream Terraform provider.
HCL is simpler for the basics — declaring a single resource in HCL is more concise than the Python equivalent. But HCL has no loops, no real functions, and a limited type system. Once you need dynamic behavior (looping over a map of configs, conditional resource creation, data transforms from external sources), HCL's meta-blocks (`for_each`, `dynamic`) become harder than Python would be. Rule of thumb: if your engineers already know Python, the Pulumi learning curve is a week; if they don't, HCL is easier.
Yes. Use `pulumi import` to adopt existing resources into a new Pulumi stack without recreating them — Pulumi reads the cloud state and writes it to the Pulumi state file. For bulk migrations, the `tf2pulumi` converter translates HCL to Pulumi code in your target language. The risk zone is shared resources: if both Terraform and Pulumi think they own a resource, whoever runs `apply` last wins. Do one module at a time, fully remove it from Terraform state (`terraform state rm`) before importing to Pulumi.
Both integrate with standard secret backends (AWS Secrets Manager, Vault, 1Password). Pulumi has first-class secret support in state: mark a config value as `--secret` and it's encrypted in the state file automatically, with decryption keys managed by Pulumi Cloud or your own KMS. Terraform can do the same but requires more setup (SOPS, separate secret resources). For teams without a dedicated secrets-management pattern yet, Pulumi's out-of-box experience is noticeably smoother.
Pulumi tests are just Python/jest/Go tests. You mock the Pulumi runtime, instantiate your program, then assert on the resource graph (`assert bucket.versioning == 'enabled'`). Very fast, no cloud calls. Terraform unit testing is harder — `terratest` spins up real resources in a sandbox account and asserts on them, which is closer to integration testing and costs real money and minutes. HashiCorp's newer `terraform test` (added in v1.6) is closer to Pulumi's model but still less mature. For teams that want fast infra tests in CI, Pulumi has the advantage today.
Data platforms tend to have lots of repeated patterns — per-environment Snowflake warehouses, per-pipeline IAM roles, per-tenant S3 buckets. That dynamism plays to Pulumi's strengths. Also, data engineers usually already know Python, so the language choice aligns. Terraform still wins when your data infra is a thin layer inside a larger company-wide Terraform monorepo owned by a platform team — fighting that org gravity isn't worth it. Decide by your team's skills and the surrounding tooling, not by tech-merit alone.
Yes. Both `terraform plan` and `pulumi preview` show drift between state and reality. Pulumi Cloud has a built-in scheduled drift-detection feature that will run previews on a cron and alert you; Terraform Cloud has an equivalent paid feature. If you're on the open-source tier, both require you to wire up your own CI job to run plan/preview on a schedule and surface non-empty diffs. Drift happens in every real environment — someone always clicks in the console eventually — so building this muscle early saves pain later.