Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add method + implementation for retrieving table metadata #56

Open
wants to merge 2 commits into
base: main
Choose a base branch
from

Conversation

tdikland
Copy link
Contributor

Which issue does this PR close?

Closes #.

Rationale for this change

Changes are included in this PR

Are there any user-facing changes?

@tdikland
Copy link
Contributor Author

@roeap I'm not an expert at protobuf. Could you check if these definitions make sense? If validated, I'd be happy to implement the remaining methods of the protocol using the same approach.

Copy link
Collaborator

@roeap roeap left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@tdikland - looking great. left a few comments, but noting major.

I'd be curious as to your opinion on using protobuf to generate our types? Is the convenience of a structured interface definition and prospect of in principle being able to also quickly build a gPRC based sharing service worth the additional effort?

optional string name = 2;
optional string description = 3;
optional ParquetFormat format = 4;
string schemaString = 5;
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

to be consistent, we should use snake_case for the message fields. The generated serde code will handle conversion for us, as this is luckily part of the json spec for proto messages :).

Comment on lines +181 to +192
delta_protocol: Some(t::DeltaProtocol {
min_reader_version: protocol.min_reader_version,
min_writer_version: protocol.min_writer_version,
reader_features: protocol
.reader_features
.clone()
.unwrap_or_default(),
writer_features: protocol
.writer_features
.clone()
.unwrap_or_default(),
}),
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should we define From implementations for the Protocol and Metadata types we get from kernel in the core crate?.

@@ -174,6 +174,110 @@ message GetTableVersionResponse {
int64 version = 1;
}

message GetTableMetadataRequest {
// The share name to query. It's case-insensitive.
string share = 1 [(buf.validate.field).string.min_len = 1];
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

just FYI - the buf.validate... annotations are neither required nor used right now :). But since there are constraints on these fields, and I wanted to test the protovalidate framework, I added some and just kept them, since they also do no harm.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants