Skip to content

[1.x Bug] [dbt State] loaded_at_field freshness override renders current_timestamp() as max_loaded_at — no max(<field>), no FROM #16362

Description

@justinfarnish

Is this a new bug in dbt-core?

  • I believe this is a new bug in dbt-core
  • I have searched the existing issues, and I could not find an existing issue for this bug

Current Behavior

When dbt State does consult a source's loaded_at_field (the source is a direct parent of the node being evaluated), the freshness statement it executes on Snowflake is:

select

  convert_timezone('UTC', current_timestamp())  as max_loaded_at,
  convert_timezone('UTC', current_timestamp()) as snapshotted_at

There is no max(<loaded_at_field>) and no FROM <source>. The override therefore always returns "now", the source is considered modified on every run, and the node (plus everything downstream) is rebuilt whenever it falls outside lag_tolerance — even when the source data has not changed for hours.

default__collect_freshness in dbt-adapters renders max({{ loaded_at_field }}) as max_loaded_at, ... from {{ source }} unconditionally, so this statement is not coming from that macro with the configured field. In dbt-state 2.46.4 (run_cache.py ~L1997) the override calls adapter.calculate_freshness(relation, defn.loaded_at_field, filter, manifest); the manifest for the affected run shows loaded_at_field == "ingestedat" on the source. Something between that call and the rendered SQL drops the field and the relation.

Expected Behavior

The override issues the same query dbt source freshness issues for that source:

select max(ingestedat) as max_loaded_at, convert_timezone('UTC', current_timestamp()) as snapshotted_at
from HUBSPOT_SHARE.V2_DAILY.OWNERS

and the returned max_loaded_at is used as the source's last-modified timestamp, so a node whose source has not changed is reused.

Steps To Reproduce

  1. Snowflake. A source with loaded_at_field whose warehouse metadata is unavailable (a secure view in a share reproduces it; the same rendering presumably happens for any source, but for normal tables last_altered masks it):
sources:
  - name: hubspot_daily
    database: HUBSPOT_SHARE
    schema: V2_DAILY
    config:
      loaded_at_field: ingestedat
    tables:
      - name: owners
  1. A table-materialized model that reads the source directly, so the source is a direct parent:
-- models/src_hubspot_daily_owners.sql
{{ config(materialized='table') }}
select * from {{ source('hubspot_daily', 'owners') }}
  1. dbt run --log-format json --debug with dbt State enabled, twice, more than lag_tolerance apart, with no change to the source data.
  2. In the debug log, find the statement issued right after State adapter: Fetching last_modified for tables: "HUBSPOT_SHARE"."V2_DAILY"."OWNERS". It is the current_timestamp()-only statement above. The model is rebuilt on the second run with reason model was executed because either its query didn't match or its upstream data is out of date.

Relevant log output

22:31:44  State adapter: Fetching last_modified for tables: "HUBSPOT_SHARE"."V2_DAILY"."ASSOCIATIONS_EVENTS_TO_CONTACTS"
22:31:44  Using snowflake connection "run_cache_prewarm_5"
22:31:44  On run_cache_prewarm_5: select

      convert_timezone('UTC', current_timestamp())  as max_loaded_at,
      convert_timezone('UTC', current_timestamp()) as snapshotted_at
22:31:44  SQL status: SUCCESS 1 in 0.08 seconds
...
22:31:45  State adapter: Explained state decision for node 'src_hubspot_daily_owners': model was executed because either its query didn't match or its upstream data is out of date

Same run, warehouse side: select max(ingestedat) from HUBSPOT_SHARE.V2_DAILY.OWNERS2026-09-05 08:15:30; ASSOCIATIONS_EVENTS_TO_CONTACTS had no rows ingested that day. Both tables were nonetheless rebuilt at 22:31 and again at 23:38. Across four consecutive 30-minute runs the pattern was strictly reused-within-lag / built / reused-within-lag / built. Four such statements per run, all current_timestamp(), none referencing ingestedat.

Environment

- OS: dbt platform runner (Linux)
- dbt: 2026.9.1+9adbe3e (Python engine, dbt platform "Latest"), dbt-state 2.46.4 (override code unchanged in dbt-state 2.48.0)
- dbt-snowflake: 1.12.0
- lag_tolerance: default 45m

Which database adapter are you using with dbt?

snowflake

Additional Context

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions