All posts Vision

Why You Should Choose SQLMesh Over dbt in 2026

Scotty Pate 10 min read

dbt came first. It was created at Fishtown Analytics in 2016 and open-sourced the same year. It took a folder of SQL files, figured out the dependency graph from {{ ref() }} calls, and ran them in order against a warehouse. Version control, modularity, and tests for SQL. It was a genuine step forward, and it defined how a generation of analytics engineers work.

It also treated SQL as text and left state to the warehouse. Those two decisions were reasonable in 2016 and they are the source of nearly every operational problem dbt teams live with today. SQLMesh, open-sourced by Tobiko Data in 2023 and built on SQLGlot, was designed by people who had built internal data platforms at places like Airbnb and Netflix and knew which problems dbt never considered. SQLMesh parses your SQL into a syntax tree, and it keeps a state database that records what it has done. Everything in this post is a consequence of those two facts.

I have been a data engineer for more than fifteen years. If I am honest about where the time went, about 80% of it was spent answering three questions:

  1. What is the state of this pipeline? Is this table current? Did last night's run finish? What is missing?
  2. Would this change work in prod? Not in my dev schema with a sample of the data. In prod.
  3. If I make this change, what all breaks downstream, and does anything need to be rebuilt?

None of those questions are data engineering. They are bookkeeping about data engineering. SQLMesh was built specifically to address the bookeeping that most data engineers do. dbt makes you answer them by hand, every time, forever. Let me take them in order.


"What is the state of this pipeline?"

In dbt Core, the answer lives in your head, in Slack, or in whatever orchestrator you wrapped around it. dbt itself does not know. dbt run runs what you select. It does not know whether yesterday's run finished, whether Tuesday's data was ever loaded, or whether the incremental model that failed at 3am got rerun. You tell incremental models where to pick up by writing a max(updated_at) filter inside is_incremental() and hoping the target table is the truth. If a run was skipped, or source data arrived late, or a backfill overlapped with a scheduled run, the table is quietly wrong and nothing tells you.

The standard mitigation is a lookback window. Reprocess the last three days on every run, just in case. That is paying warehouse compute every day to compensate for the tool not knowing what it did.

SQLMesh knows what it did. Every model declares its own cron, and the state database records which time intervals of each model have been successfully processed. When sqlmesh run starts, it compares the intervals that should exist against the intervals that do, and evaluates exactly the difference. Nothing is due, nothing runs. A day was missed, that day gets filled. A run failed halfway, the completed intervals are recorded and the next run picks up the rest.

Here is what that looks like on a small project with a daily incremental model and two full-refresh models downstream of it. Prod was last applied two days ago, so two daily intervals are missing from the incremental model, and that is exactly what the run inserts. The two downstream models refresh because their input changed. Run it again and there is nothing to do, so nothing runs:

$ sqlmesh run
[1/1] demo.raw_orders         [insert 2026-09-28 - 2026-09-29]
[1/1] demo.customer_revenue   [full refresh]
[1/1] demo.orders_daily       [full refresh]

Run finished for environment 'prod'

$ sqlmesh run
No models are ready to run. Please wait until a model `cron` interval has
elapsed.

Next run will be ready at 2026-09-30 07:00PM CDT (2026-10-01 12:00AM UTC).

That output is the answer to question one. The state of the pipeline is a query against the state database, and the tool consults it before doing anything. I do not have to.

dbt has a version of this now. It is called dbt State, it went GA in September 2026, and it is a metered feature billed per daily active target table. The thing that makes the tool know what it already did is the thing you pay dbt Labs for. In SQLMesh, it is the engine.


"Would this change work in prod?"

This is the question that eats afternoons. You have a change. You want to know if it works against production data before you merge it. In dbt your options are:

Every one of those is a bookkeeping system. Which schema has which version of which model, built from which data, and when. Multiply by the number of engineers on the team and the number of branches in flight. I have watched teams build internal tooling just to answer "is my dev schema current" and I have built some of it myself.

SQLMesh gives you ephemeral dev environments. sqlmesh plan dev creates an environment that contains exactly the changes on your branch. The models you changed get new physical tables, backfilled with production data. Every model you did not change is resolved against the same physical table prod is using. Not a copy. A pointer. Creating the environment is instant for everything you did not touch, and there is nothing to track because there is only one version of every unchanged table in existence.

Here is a branch that changes one model, customer_revenue, to exclude cancelled orders. The plan builds one table:

$ sqlmesh plan dev --auto-apply --no-prompts

**New environment `dev` will be created from `prod`**

**Directly Modified:**
* `demo__dev.customer_revenue` (Breaking)
   customer_id,
   SUM(amount) AS revenue
 FROM demo.raw_orders
+WHERE
+  status <> 'cancelled'
 GROUP BY
   customer_id

**Models needing backfill:**
* `demo__dev.customer_revenue`: [full refresh]
[1/1] demo__dev.customer_revenue   [full refresh]

Virtual layer updated

The warehouse confirms it. Prod's views and the dev views for the two models I did not touch resolve to the same snapshot tables. Only the model I changed points somewhere new. The dev views for unchanged models are opt-in with --include-unmodified, and adding them is a virtual update: no physical tables are built, only views are written.

$ sqlmesh plan dev --auto-apply --no-prompts --include-unmodified
SKIP: No physical layer updates to perform
SKIP: No model batches to execute
Virtual layer updated

# information_schema.views: which snapshot table each view resolves to
-- schema: demo (prod)
customer_revenue: SELECT * FROM sqlmesh__demo.demo__customer_revenue__2069255949
orders_daily:     SELECT * FROM sqlmesh__demo.demo__orders_daily__2423542018
raw_orders:       SELECT * FROM sqlmesh__demo.demo__raw_orders__2824761531

-- schema: demo__dev
customer_revenue: SELECT * FROM sqlmesh__demo.demo__customer_revenue__167319619
orders_daily:     SELECT * FROM sqlmesh__demo.demo__orders_daily__2423542018
raw_orders:       SELECT * FROM sqlmesh__demo.demo__raw_orders__2824761531

$ sqlmesh environments
prod - No Expiry
dev - 2026-10-07 00:00:00

Environments expire on a TTL, seven days by default, and the janitor cleans them up. You do not maintain a dev schema. You create one when you need it, it is a full-fidelity view of production plus your change, and it goes away.

The part that took me longest to appreciate: when you promote that plan to prod, the backfill you already ran in dev is the data prod points at. Promotion is a view swap. The compute happened once, in dev, on production data, and the answer to "would this work in prod" was already yes because it already ran against prod.


"If I make this change, what breaks?"

dbt can tell you a file changed. state:modified compares manifests and selects models whose definition differs. What it cannot tell you is what the change means. Did you add a column, which downstream models can ignore? Did you change a filter, which means every downstream table now holds the wrong rows? dbt Core treats SQL as text, so those two changes look identical to it: a modified file. dbt shipped a Rust engine that parses SQL in September 2026, and the SQL comprehension lives in the proprietary distribution, not in dbt OSS. Even there, parsing the SQL is not the same as classifying the change and deciding what downstream needs a backfill. Your choices are to rebuild everything downstream, which is expensive, or rebuild nothing, which is wrong, or read every downstream model yourself and decide. That last option is what most teams actually do, and it is the third question.

SQLMesh parses the SQL, so it knows the difference. When you plan a change, it classifies it. Adding a column is non-breaking: downstream models keep their data and nothing is rebuilt. Changing a filter is breaking: SQLMesh names every downstream model that is affected and schedules the backfill for exactly those models over exactly the intervals that need it. It tells you this before anything runs, and you approve it.

Both plans below start from the same prod. First, adding a column to the incremental model. SQLMesh classifies the change as non-breaking, marks the two downstream models as indirectly non-breaking, and the backfill list contains only the model I edited:

$ sqlmesh plan dev --no-prompts

**Directly Modified:**
* `demo__dev.raw_orders` (Non-breaking)
  Indirectly Modified Children:
    - `demo__dev.customer_revenue` (Indirect Non-breaking)
    - `demo__dev.orders_daily` (Indirect Non-breaking)

   r.n * 10.5 AS amount,
+  r.n * 1.05 AS amount_with_tax,
   CASE WHEN r.n = 5 THEN 'cancelled' ELSE 'complete' END AS status,

**Models needing backfill:**
* `demo__dev.raw_orders`: [2026-09-20 - 2026-09-29]
Apply - Backfill Tables [y/n]:

Now a filter on the same model. Same command, different answer. The change is breaking, both downstream models are named as indirectly breaking, and both are scheduled for a rebuild, before anything executes:

$ sqlmesh plan dev --no-prompts

**Directly Modified:**
* `demo__dev.raw_orders` (Breaking)
  Indirectly Modified Children:
    - `demo__dev.customer_revenue` (Indirect Breaking)
    - `demo__dev.orders_daily` (Indirect Breaking)

 CROSS JOIN RANGE(1, 6) AS r(n)
+WHERE
+  r.n <> 4

**Models needing backfill:**
* `demo__dev.customer_revenue`: [full refresh]
* `demo__dev.orders_daily`: [full refresh]
* `demo__dev.raw_orders`: [2026-09-20 - 2026-09-29]
Apply - Backfill Tables [y/n]:

That is question three answered by the tool, from the syntax tree, with the blast radius and the rebuild plan in the output. I did not read a single downstream model.


Fivetran kept the one with the install base

In September 2025, Fivetran acquired Tobiko Data. A month later it announced a merger with dbt Labs. With that merger pending, in March 2026 it contributed SQLMesh to the Linux Foundation, under vendor-neutral governance, Apache 2.0, with member companies whose production workloads depend on it. The merger closed in June. dbt stayed a Fivetran product.

Fivetran has not said why it made that split, so here is my read of it. They gave away the superior technical product and kept the one with the install base. Fivetran itself says the overwhelming majority of its customers already use dbt. That is a rational commercial decision. It is not a reason for you to choose dbt in 2026. Install base is a fact about the past. The three questions above are facts about every day of your working life for as long as you run the tool.

I wrote about what the Linux Foundation move means for SQLMesh's future in SQLMesh Is Not Going Anywhere. The short version: it is community-governed, Apache 2.0, and forever available, and no product decision at any one company can change that.


Choose the tool that knows what it did

SQLMesh keeps state and understands SQL. Those two design decisions turn the three questions that ate most of my career into output from a command. dbt made the opposite decisions in 2016, and every mitigation since, lookback windows, clones, defer, manifest diffs, and now a metered state add-on, is a patch over a tool that does not know what it did or what your SQL means.

If you are picking a transformation tool in 2026, pick the one where the hard questions are no-ops.