Skip to content

Commit

Permalink
Refactor dbt utility for modular, separable logging
Browse files Browse the repository at this point in the history
  • Loading branch information
austinweisgrau committed Jul 17, 2024
1 parent 2a7a1d6 commit c066e86
Show file tree
Hide file tree
Showing 7 changed files with 435 additions and 287 deletions.
283 changes: 0 additions & 283 deletions parsons/utilities/dbt.py

This file was deleted.

60 changes: 60 additions & 0 deletions parsons/utilities/dbt/__init__.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,60 @@
"""Utility for running and logging output from dbt commands
Enable this utility by installing parsons with a dbt extra:
`pip install parsons[dbt-redshift]`
or `pip install parsons[dbt-postgres]`
or `pip install parsons[dbt-snowflake]`
or `pip install parsons[dbt-bigquery]`
To run dbt commands, you will need to have a dbt project directory
somewhere on the local filesystem.
The dbt command will inherit environment variables from the python
process shell, so if your dbt profiles.yml file uses environment
variables, ensure those are set in python or the parent shell before
running this dbt utility.
Logging is handled separately from the dbt run itself. The
dbtRunner.run method returns a dbtCommandResult object which can be
passed to a child class of dbtLogger for logging to stdout, slack,
etc.
Parsons provides a few example dbtLogger child classes, but for
best results, design your own!
Example usage:
```
from parsons.utilities.dbt import (
run_dbt_commands,
dbtLoggerSlack,
dbtLoggerPython
)
run_dbt_commands(
commands=['run', 'test'],
dbt_project_directory='/home/ubuntu/code/dbt_project/',
loggers=[
dbtLoggerPython,
dbtLoggerSlack(slack_webhook=os.environ['SLACK_WEBHOOK'])
]
)
```
"""

from parsons.utilities.dbt.dbt import run_dbt_commands
from parsons.utilities.dbt.logging import (
dbtLoggerMarkdown,
dbtLoggerSlack,
dbtLoggerStdout,
dbtLoggerPython,
)

__all__ = [
"run_dbt_commands",
"dbtLoggerMarkdown",
"dbtLoggerSlack",
"dbtLoggerStdout",
"dbtLoggerPython",
]
Loading

0 comments on commit c066e86

Please sign in to comment.