Skip to content

Commit

Permalink
Refactored dbt utility for modularity
Browse files Browse the repository at this point in the history
  • Loading branch information
austinweisgrau committed Nov 4, 2024
1 parent a9b45c2 commit 099113a
Show file tree
Hide file tree
Showing 7 changed files with 552 additions and 333 deletions.
327 changes: 0 additions & 327 deletions parsons/utilities/dbt.py

This file was deleted.

60 changes: 60 additions & 0 deletions parsons/utilities/dbt/__init__.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,60 @@
"""Utility for running and logging output from dbt commands
Enable this utility by installing parsons with a dbt extra:
`pip install parsons[dbt-redshift]`
or `pip install parsons[dbt-postgres]`
or `pip install parsons[dbt-snowflake]`
or `pip install parsons[dbt-bigquery]`
To run dbt commands, you will need to have a dbt project directory
somewhere on the local filesystem.
The dbt command will inherit environment variables from the python
process shell, so if your dbt profiles.yml file uses environment
variables, ensure those are set in python or the parent shell before
running this dbt utility.
Logging is handled separately from the dbt run itself. The
dbtRunner.run method returns a dbtCommandResult object which can be
passed to a child class of dbtLogger for logging to stdout, slack,
etc.
Parsons provides a few example dbtLogger child classes, but for
best results, design your own!
Example usage:
```
from parsons.utilities.dbt import (
run_dbt_commands,
dbtLoggerSlack,
dbtLoggerPython
)
run_dbt_commands(
commands=['run', 'test'],
dbt_project_directory='/home/ubuntu/code/dbt_project/',
loggers=[
dbtLoggerPython,
dbtLoggerSlack(slack_webhook=os.environ['SLACK_WEBHOOK'])
]
)
```
"""

from parsons.utilities.dbt.dbt import run_dbt_commands
from parsons.utilities.dbt.logging import (
dbtLoggerMarkdown,
dbtLoggerSlack,
dbtLoggerStdout,
dbtLoggerPython,
)

__all__ = [
"run_dbt_commands",
"dbtLoggerMarkdown",
"dbtLoggerSlack",
"dbtLoggerStdout",
"dbtLoggerPython",
]
Loading

0 comments on commit 099113a

Please sign in to comment.