Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Minor: add flags for temporary ddl #12561

Merged
merged 2 commits into from
Oct 14, 2024

Conversation

hailelagi
Copy link
Contributor

@hailelagi hailelagi commented Sep 20, 2024

Which issue does this PR close?

Closes #12463

Rationale for this change

This introduces a new flag option temporary on the DdlStatement of:

  1. CreateMemoryTable,
  2. CreateExternalTable

These flags enable an implementation built on DF to define the semantics of a temporary table.

What changes are included in this PR?

Are these changes tested?

yes

Are there any user-facing changes?

@github-actions github-actions bot added sql SQL Planner logical-expr Logical plan and expressions labels Sep 20, 2024
@hailelagi hailelagi closed this Sep 24, 2024
@hailelagi hailelagi reopened this Sep 27, 2024
@hailelagi hailelagi force-pushed the haile/add-flag-for-temporary-ddl branch from 0e03cba to a023c22 Compare September 27, 2024 22:58
@hailelagi
Copy link
Contributor Author

sorry, i took a bit longer than i was hoping with this one as i tried to get it to require minimal changes in the diff but as it is an api change i ended up spending quite a lot of time reading code. I have some questions and i'm super open to feedback, i'll self-review a few places where i was especially confused

@github-actions github-actions bot added core Core DataFusion crate proto Related to proto crate labels Sep 27, 2024
@hailelagi hailelagi force-pushed the haile/add-flag-for-temporary-ddl branch from a023c22 to 2cb1e93 Compare September 27, 2024 23:00
@hailelagi hailelagi marked this pull request as ready for review September 27, 2024 23:12
@hailelagi hailelagi changed the title add flags for temporary ddl Minor: add flags for temporary ddl Sep 27, 2024
Copy link
Member

@jonahgao jonahgao left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks @hailelagi . I think we should add some sqllogictest tests for this change. Perhaps in ddl.slt. In order to ensure the generation of correct logical plans, we can add tests similar to the following.

set datafusion.explain.logical_plan_only=true;
explain create temp table t(a int);

datafusion/core/src/execution/context/mod.rs Outdated Show resolved Hide resolved
datafusion/core/src/execution/context/mod.rs Outdated Show resolved Hide resolved
datafusion/proto/src/logical_plan/mod.rs Outdated Show resolved Hide resolved
datafusion/sql/src/parser.rs Outdated Show resolved Hide resolved
@hailelagi hailelagi force-pushed the haile/add-flag-for-temporary-ddl branch from 2cb1e93 to 7ac494c Compare September 30, 2024 20:42
@github-actions github-actions bot added the sqllogictest SQL Logic Tests (.slt) label Sep 30, 2024
@alamb alamb marked this pull request as draft October 6, 2024 11:32
@alamb
Copy link
Contributor

alamb commented Oct 6, 2024

This PR seems to be failing CI and needs some more work, so marking it as a draft. When it is ready for another look, please mark it as ready for review

@hailelagi hailelagi force-pushed the haile/add-flag-for-temporary-ddl branch from 7ac494c to c909da2 Compare October 12, 2024 03:11
@hailelagi hailelagi marked this pull request as ready for review October 12, 2024 03:12
@hailelagi hailelagi force-pushed the haile/add-flag-for-temporary-ddl branch from c909da2 to 5a82ff4 Compare October 12, 2024 03:22
@hailelagi hailelagi force-pushed the haile/add-flag-for-temporary-ddl branch from 5a82ff4 to d955734 Compare October 12, 2024 10:32
Copy link
Member

@jonahgao jonahgao left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM👍, thanks @hailelagi

@jonahgao jonahgao merged commit 636f433 into apache:main Oct 14, 2024
24 checks passed
@jonahgao jonahgao added the api change Changes the API exposed to users of the crate label Oct 14, 2024
@hailelagi hailelagi deleted the haile/add-flag-for-temporary-ddl branch October 14, 2024 05:14
hailelagi added a commit to hailelagi/datafusion that referenced this pull request Oct 14, 2024
* Minor: add flags for temporary ddl

Signed-off-by: Haile Lagi <[email protected]>

* Update datafusion/proto/src/logical_plan/mod.rs

Co-authored-by: Jonah Gao <[email protected]>

---------

Signed-off-by: Haile Lagi <[email protected]>
Co-authored-by: Jonah Gao <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
api change Changes the API exposed to users of the crate core Core DataFusion crate logical-expr Logical plan and expressions proto Related to proto crate sql SQL Planner sqllogictest SQL Logic Tests (.slt)
Projects
None yet
Development

Successfully merging this pull request may close these issues.

hooks for temporary tables in the catalog
3 participants