Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Bug] Unit test compilation error 'var is undefined' when combining a variable with math operators #10353

Open
2 tasks done
adamribaudo-velir opened this issue Jun 22, 2024 · 5 comments · May be fixed by #10849
Open
2 tasks done
Labels
bug Something isn't working unit tests Issues related to built-in dbt unit testing functionality

Comments

@adamribaudo-velir
Copy link

adamribaudo-velir commented Jun 22, 2024

Is this a new bug in dbt-core?

  • I believe this is a new bug in dbt-core
  • I have searched the existing issues, and I could not find an existing issue for this bug

Current Behavior

When running a unit test for a model that contains a Jinja math operation on a variable, a compilation error occurs

Example operation

select {{ var('foo', 10 ) * 1}} as result

Error:

Compilation Error in unit_test test_unit_test (models\staging\test.yml)
  'var' is undefined. This can happen when calling a macro that does not exist. Check for typos and/or install package dependencies with "dbt deps".

Running the unit test on a Jinja variable without the math operation works successfully. Ex: select {{ var('foo', 10 ) }} as result

Expected Behavior

Expected behavior would be to compile unit test and executing the math operation on the variable.

Steps To Reproduce

  1. In any dbt project, create a model called test with the following SQL:
select {{ var('foo', 10 ) * 1}} as result
  1. In that model's .yml file, create a unit test as follows (the 'input' is not used, but required):
version: 2

unit_tests:
  - name: test_unit_test
    model: test
    given:
      - input: ref('stg_ga4__events')
        format: csv
        rows: |
          event_key
          A
    expect:
      format: csv
      rows: |
        result
        10
  1. Run the unit test dbt test -s test_unit_test

Relevant log output

(python312env) C:\GitHub\velir-ga4-test-project>dbt test -s test_unit_test
22:04:52  Running with dbt=1.8.2
22:04:53  Registered adapter: bigquery=1.8.1
22:04:54  Found 59 models, 28 data tests, 1 seed, 1 source, 604 macros, 11 unit tests
22:04:54  
22:04:55  Concurrency: 4 threads (target='dev')
22:04:55  
22:04:55  1 of 1 START unit_test test::test_unit_test .................................... [RUN]
22:04:55  1 of 1 ERROR test::test_unit_test .............................................. [ERROR in 0.01s]
22:04:55  
22:04:55  Finished running 1 unit test in 0 hours 0 minutes and 1.22 seconds (1.22s).
22:04:55  
22:04:55  Completed with 1 error and 0 warnings:
22:04:55
22:04:55    Compilation Error in unit_test test_unit_test (models\staging\test.yml)
  'var' is undefined. This can happen when calling a macro that does not exist. Check for typos and/or install package dependencies with "dbt deps".
22:04:55
22:04:55  Done. PASS=0 WARN=0 ERROR=1 SKIP=0 TOTAL=1

Environment

- OS: Windows
- Python: 3.12
- dbt-core: 1.8.3, bigquery: 1.8.3

Which database adapter are you using with dbt?

bigquery

Additional Context

No response

@adamribaudo-velir adamribaudo-velir added bug Something isn't working triage labels Jun 22, 2024
@dbeatty10 dbeatty10 added the unit tests Issues related to built-in dbt unit testing functionality label Jun 25, 2024
@dkultasev
Copy link

I can also add that it doesn't work even if you add this variable to overrides->vars

@dbeatty10
Copy link
Contributor

Thanks for reporting this @adamribaudo-velir and @dkultasev for confirming.

I also observed the same thing. Simplified reprex below.

models/_unit_tests.yml

version: 2

unit_tests:
  - name: test_unit_test
    model: test
    given: []
    expect:
      format: csv
      rows: |
        result
        10

This worked for models/test.sql:

select {{ var('foo', 10) }} as result

But it didn't when I changed it to this:

select {{ var('foo', 10) * 1 }} as result
$ dbt build -s test

00:11:04  Running with dbt=1.8.3
00:11:04  Registered adapter: duckdb=1.8.1
00:11:05  Found 1 model, 406 macros, 1 unit test
00:11:05  
00:11:05  Concurrency: 1 threads (target='dev')
00:11:05  
00:11:05  1 of 2 START unit_test test::test_unit_test .................................... [RUN]
00:11:05  1 of 2 ERROR test::test_unit_test .............................................. [ERROR in 0.01s]
00:11:05  2 of 2 SKIP relation feature_456.test .......................................... [SKIP]
00:11:05  
00:11:05  Finished running 1 unit test, 1 view model in 0 hours 0 minutes and 0.19 seconds (0.19s).
00:11:05  
00:11:05  Completed with 1 error and 0 warnings:
00:11:05  
00:11:05    Compilation Error in unit_test test_unit_test (models/_unit_tests.yml)
  'var' is undefined. This can happen when calling a macro that does not exist. Check for typos and/or install package dependencies with "dbt deps".
00:11:05  
00:11:05  Done. PASS=0 WARN=0 ERROR=1 SKIP=1 TOTAL=2

@dbeatty10 dbeatty10 removed the triage label Jul 19, 2024
@dbeatty10
Copy link
Contributor

This error message looks similar to #10410, so I suspect they may have the same underlying root cause(s).

@Aviss
Copy link

Aviss commented Sep 9, 2024

To perhaps help narrow down the problem: We ran into the same problem with env_var

{{
    config(
        materialized='incremental',
        grants = {
            'select': env_var('DBT_ENV_CUSTOM_ENV_ROLE_PREFIX') + '_some_role'
        }
    )
}}

would run into the error while

[...]
        grants = {
            'select': env_var('DBT_ENV_CUSTOM_ENV_ROLE_PREFIX')
        }
[...]

would not.

Interestingly, replacing the plus with the Jinja string concatenation operator also avoided this problem:

[...]
        grants = {
            'select': env_var('DBT_ENV_CUSTOM_ENV_ROLE_PREFIX') ~ '_some_role'
        }
[...]

@devmessias devmessias linked a pull request Oct 14, 2024 that will close this issue
5 tasks
@devmessias
Copy link
Contributor

I found that var('foo', 10 ) * 1} dosen't work but var('foo', 10 ) works because the template is rendered with no var object specified. So var("foo", 10) will not be rendered you will get select as result discussion at #10849

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working unit tests Issues related to built-in dbt unit testing functionality
Projects
None yet
Development

Successfully merging a pull request may close this issue.

5 participants