Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Refactor dbt utility for modularity #1095

Open
wants to merge 1 commit into
base: main
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
327 changes: 0 additions & 327 deletions parsons/utilities/dbt.py

This file was deleted.

60 changes: 60 additions & 0 deletions parsons/utilities/dbt/__init__.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,60 @@
"""Utility for running and logging output from dbt commands

Enable this utility by installing parsons with a dbt extra:
`pip install parsons[dbt-redshift]`
or `pip install parsons[dbt-postgres]`
or `pip install parsons[dbt-snowflake]`
or `pip install parsons[dbt-bigquery]`

To run dbt commands, you will need to have a dbt project directory
somewhere on the local filesystem.

The dbt command will inherit environment variables from the python
process shell, so if your dbt profiles.yml file uses environment
variables, ensure those are set in python or the parent shell before
running this dbt utility.

Logging is handled separately from the dbt run itself. The
dbtRunner.run method returns a dbtCommandResult object which can be
passed to a child class of dbtLogger for logging to stdout, slack,
etc.

Parsons provides a few example dbtLogger child classes, but for
best results, design your own!

Example usage:
```
from parsons.utilities.dbt import (
run_dbt_commands,
dbtLoggerSlack,
dbtLoggerPython
)

run_dbt_commands(
commands=['run', 'test'],
dbt_project_directory='/home/ubuntu/code/dbt_project/',
loggers=[
dbtLoggerPython,
dbtLoggerSlack(slack_webhook=os.environ['SLACK_WEBHOOK'])
]
)

```

"""

from parsons.utilities.dbt.dbt import run_dbt_commands
from parsons.utilities.dbt.logging import (
dbtLoggerMarkdown,
dbtLoggerSlack,
dbtLoggerStdout,
dbtLoggerPython,
)

__all__ = [
"run_dbt_commands",
"dbtLoggerMarkdown",
"dbtLoggerSlack",
"dbtLoggerStdout",
"dbtLoggerPython",
]
Loading
Loading