Question about type conversion between parquet and arrow. #4731
Answered
by
tustvold
RinChanNOWWW
asked this question in
Q&A
-
|
However, Why? |
Beta Was this translation helpful? Give feedback.
Answered by
tustvold
Aug 24, 2023
Replies: 1 comment
-
|
Int96 is deprecated - https://github.com/apache/parquet-format/blob/master/Encodings.md#plain-plain--0 Timestamps should always be encoded as int64 now - https://github.com/apache/parquet-format/blob/master/LogicalTypes.md#timestamp Int96 is only still supported for compatibility with spark, which still has issues supporting nanosecond timestamps stored as int64 |
Beta Was this translation helpful? Give feedback.
0 replies
Answer selected by
RinChanNOWWW
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Int96 is deprecated - https://github.com/apache/parquet-format/blob/master/Encodings.md#plain-plain--0
Timestamps should always be encoded as int64 now - https://github.com/apache/parquet-format/blob/master/LogicalTypes.md#timestamp
Int96 is only still supported for compatibility with spark, which still has issues supporting nanosecond timestamps stored as int64