Go Even Further with dbt

Let’s take a look at a few features that can take our dbt game to the next level!

State comparison

dbt provides a powerful feature called state-based selection. This feature allows us to run only the models that have changed compared to a previous version.

manifest.json

To identify changes, dbt compares our current manifest.json file to a previous version. manifest.json is a single file, stored in the target directory, that contains information about our entire dbt project. This file is used both for state comparison and generating documentation.

Every time a user runs dbt run, dbt ls, or dbt compile, the manifest.json file is updated.

The DBT_STATE environment

dbt must compare our current project to a previous version. To do so, we need to give dbt a previous version of the manifest.json file. A common way to do so is to copy that file when a pull request is made to the main branch. This way, users can compare their changes to the main branch.

Once we’ve stored a copy of the previous manifest.json file, we need to tell dbt where that file is located. We can do so by setting the DBT_STATE environment variable and pass it the folder that contains the previous manifest.json file: