[v1/v2 Bug] `KeyError: 'macro.dbt_snowflake.snowflake__date_spine'` error when using v2 parser

[jeremyyeo]

Is this a new bug in dbt v2.x compared to the latest version of dbt 1.x?

• I believe this is a new bug in dbt v2.x
• I have searched the existing issues and could not find a duplicate

Current Behavior

When using the v2 parser with state:modified and modified macros, we run into an exception.

Expected Behavior

No exception.

Steps To Reproduce

Project setup

# dbt_project.yml
name: dbt_basic_5
profile: sf

models:
  dbt_basic_5:
    +materialized: table

flags:
  use_v2_parser: true

# packages.yml
packages:
  - package: dbt-labs/dbt_utils
    version: 1.4.1

-- macros/my_macro.sql
{% macro my_macro() %}
    {{ 'hello' }}    
{% endmacro %}

-- models/date_spine.sql
{{ config(materialized='view') }}

with date_spine as (
    {{ 
        dbt_utils.date_spine(
            datepart="day", 
            start_date="cast('2019-01-01' as date)", 
            end_date="cast('2020-01-02' as date)"
        )
    }}
)
select * from date_spine

-- models/foo.sql
select * from {{ ref('date_spine') }}

-- models/bar.sql
select '{{ my_macro() }}' as c

Invoke dbt

Parse the project for the first time:

$ rm -rf target target_old && dbt parse --no-partial-parse && mv target target_old
20:04:53  Running with dbt=1.12.5
20:04:53  Delegating parse to v2 parser: dbt-core-experimental-parser parse
20:04:53  Running with dbt-oss=2.0.5
20:04:53  dbt-oss 2.0.5
20:04:53  Loading profiles.yml
20:04:53  Loading packages.yml
20:04:54  Artifact written to ../../../../private/var/folders/wx/1kz1jzp92xg__rcvpgnd_ckh0000gp/T/dbt-fusion-homc08oe/semantic_manifest.json
20:04:54  Artifact written to ../../../../private/var/folders/wx/1kz1jzp92xg__rcvpgnd_ckh0000gp/T/dbt-fusion-homc08oe/manifest.json
20:04:54  
==================== Execution Summary =====================
Finished 'parse' successfully for target 'dev' [343ms]
20:04:54  v2 parser completed in 0.72s
20:04:54  Registered adapter: snowflake=1.12.1
20:04:54  Registered adapter: snowflake=1.12.1

Modify our macro:

-- macros/my_macro.sql
{% macro my_macro() %}
    {{ 'hello world' }}    
{% endmacro %}

Then invoke dbt deferring to the previously stored state:

$ dbt ls -s state:modified --defer --state target_old

20:07:11  Running with dbt=1.12.5
20:07:12  Delegating parse to v2 parser: dbt-core-experimental-parser parse
20:07:12  Running with dbt-oss=2.0.5
20:07:12  dbt-oss 2.0.5
20:07:12  Loading profiles.yml
20:07:12  Loading packages.yml
20:07:12  Artifact written to ../../../../private/var/folders/wx/1kz1jzp92xg__rcvpgnd_ckh0000gp/T/dbt-fusion-cdeolugt/semantic_manifest.json
20:07:12  Artifact written to ../../../../private/var/folders/wx/1kz1jzp92xg__rcvpgnd_ckh0000gp/T/dbt-fusion-cdeolugt/manifest.json
20:07:12  
==================== Execution Summary =====================
Finished 'parse' successfully for target 'dev' [328ms]
20:07:12  v2 parser completed in 0.63s
20:07:12  Registered adapter: snowflake=1.12.1
20:07:13  Registered adapter: snowflake=1.12.1
20:07:13  Found 3 models, 679 macros
20:07:13  [ERROR]: Encountered an error:
'macro.dbt_snowflake.snowflake__date_spine'
20:07:13  Traceback (most recent call last):
  File "/Users/jeremy/git/dbt-basic/venv_dbt_1.12/lib/python3.12/site-packages/dbt/cli/requires.py", line 198, in wrapper
    result, success = func(*args, **kwargs)
                      ^^^^^^^^^^^^^^^^^^^^^
  File "/Users/jeremy/git/dbt-basic/venv_dbt_1.12/lib/python3.12/site-packages/dbt/cli/requires.py", line 144, in wrapper
    return func(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^
  File "/Users/jeremy/git/dbt-basic/venv_dbt_1.12/lib/python3.12/site-packages/dbt/cli/requires.py", line 295, in wrapper
    return func(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^
  File "/Users/jeremy/git/dbt-basic/venv_dbt_1.12/lib/python3.12/site-packages/dbt/cli/requires.py", line 340, in wrapper
    return func(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^
  File "/Users/jeremy/git/dbt-basic/venv_dbt_1.12/lib/python3.12/site-packages/dbt/cli/requires.py", line 387, in wrapper
    return func(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^
  File "/Users/jeremy/git/dbt-basic/venv_dbt_1.12/lib/python3.12/site-packages/dbt/cli/requires.py", line 404, in wrapper
    return func(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^
  File "/Users/jeremy/git/dbt-basic/venv_dbt_1.12/lib/python3.12/site-packages/dbt/cli/main.py", line 542, in list
    results = task.run()
              ^^^^^^^^^^
  File "/Users/jeremy/git/dbt-basic/venv_dbt_1.12/lib/python3.12/site-packages/dbt/task/list.py", line 209, in run
    return self.output_results(generator())
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/jeremy/git/dbt-basic/venv_dbt_1.12/lib/python3.12/site-packages/dbt/task/list.py", line 213, in output_results
    for result in results:
                  ^^^^^^^
  File "/Users/jeremy/git/dbt-basic/venv_dbt_1.12/lib/python3.12/site-packages/dbt/task/list.py", line 111, in generate_selectors
    for node in self._iterate_selected_nodes():
                ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/jeremy/git/dbt-basic/venv_dbt_1.12/lib/python3.12/site-packages/dbt/task/list.py", line 81, in _iterate_selected_nodes
    unique_ids = sorted(selector.get_selected(spec))
                        ^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/jeremy/git/dbt-basic/venv_dbt_1.12/lib/python3.12/site-packages/dbt/graph/selector.py", line 371, in get_selected
    selected_nodes, indirect_only = self.select_nodes(
                                    ^^^^^^^^^^^^^^^^^^
  File "/Users/jeremy/git/dbt-basic/venv_dbt_1.12/lib/python3.12/site-packages/dbt/graph/selector.py", line 191, in select_nodes
    direct_nodes, indirect_nodes = self.select_nodes_recursively(
                                   ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/jeremy/git/dbt-basic/venv_dbt_1.12/lib/python3.12/site-packages/dbt/graph/selector.py", line 156, in select_nodes_recursively
    self.select_nodes_recursively(spec=component, warn_on_no_nodes=warn_on_no_nodes)
  File "/Users/jeremy/git/dbt-basic/venv_dbt_1.12/lib/python3.12/site-packages/dbt/graph/selector.py", line 156, in select_nodes_recursively
    self.select_nodes_recursively(spec=component, warn_on_no_nodes=warn_on_no_nodes)
  File "/Users/jeremy/git/dbt-basic/venv_dbt_1.12/lib/python3.12/site-packages/dbt/graph/selector.py", line 156, in select_nodes_recursively
    self.select_nodes_recursively(spec=component, warn_on_no_nodes=warn_on_no_nodes)
  File "/Users/jeremy/git/dbt-basic/venv_dbt_1.12/lib/python3.12/site-packages/dbt/graph/selector.py", line 153, in select_nodes_recursively
    direct_nodes, indirect_nodes = self.get_nodes_from_criteria(spec)
                                   ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/jeremy/git/dbt-basic/venv_dbt_1.12/lib/python3.12/site-packages/dbt/graph/selector.py", line 95, in get_nodes_from_criteria
    collected = self.select_included(nodes, spec)
                ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/jeremy/git/dbt-basic/venv_dbt_1.12/lib/python3.12/site-packages/dbt/graph/selector.py", line 81, in select_included
    return set(method.search(included_nodes, spec.value))
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/jeremy/git/dbt-basic/venv_dbt_1.12/lib/python3.12/site-packages/dbt/graph/selector_methods.py", line 834, in search
    if checker(previous_node, node, **keyword_args):  # type: ignore
       ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/jeremy/git/dbt-basic/venv_dbt_1.12/lib/python3.12/site-packages/dbt/graph/selector_methods.py", line 727, in check_modified_content
    upstream_macro_change = self.check_macros_modified(new)
                            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/jeremy/git/dbt-basic/venv_dbt_1.12/lib/python3.12/site-packages/dbt/graph/selector_methods.py", line 711, in check_macros_modified
    return self.recursively_check_macros_modified(node, visited_macros)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/jeremy/git/dbt-basic/venv_dbt_1.12/lib/python3.12/site-packages/dbt/graph/selector_methods.py", line 687, in recursively_check_macros_modified
    upstream_macros_changed = self.recursively_check_macros_modified(
                              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/jeremy/git/dbt-basic/venv_dbt_1.12/lib/python3.12/site-packages/dbt/graph/selector_methods.py", line 685, in recursively_check_macros_modified
    macro_node = self.manifest.macros[macro_uid]
                 ~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^
KeyError: 'macro.dbt_snowflake.snowflake__date_spine'

Relevant log output

Environment

- OS: macOS
- CPU: ARM
- dbt distribution and version: (`dbt --version`)
Core:
  - installed: 1.12.5
  - latest:    1.12.5 - Up to date!

Plugins:
  - snowflake: 1.12.1 - Up to date!

Which database adapter are you using?

snowflake (not specific to snowflake, same is observed with bigquery, etc).

Is this a discrepancy vs. dbt 1.x?

• Yes — this works in dbt 1.x but not in dbt v2.x

Additional Context

No response