how to build python for HDFS usage

[momhiar]

i have tried to build python with hdfs feature
features = ["azure", "gcs", "python", "datafusion", "unity-experimental", "hdfs"]
it does install hdfs3 and more but after make build it creates a libjvm.so file in deltalake.libs directory in venv site packages.
i have copied it manually to a parent directory as i needed because otherwise it will therow an error: libjava.so not found.

but another problem occurred that libzip.so not found,

Error occurred during initialization of VM
Unable to load ZIP library: /path/delta/python/venv/lib/python3.10/site-packages/libzip.so
/arrow/cpp/src/arrow/filesystem/s3fs.cc:2598

i have tried to cope system libzip.so file from root but after that i got

Error occurred during initialization of VM
Corrupted ZIP library: /path/delta/python/venv/lib/python3.10/site-packages/libzip.so
/arrow/cpp/src/arrow/filesystem/s3fs.cc:2598:  arrow::fs::FinalizeS3 was not called even though
S3 was initialized.  This could lead to a segmentation fault at exit

platform: ubuntu22.04
deltalake: 0.10.1
python: 3.8 and 3.10
java: 8 and 11
rust: default

[roeap]

Do you have JAVA_HOME set in your environment? the HDFS driver requires java.

The issues with the zip library seem to stem from pyarrow and not from within delta-rs.

[momhiar]

thanks @roeap , yes JAVA_HOME environment is set on my machine , after a research and development i've found out zip library error came from datafusion-objectstore-hdfs library which is using zip in a automation test. i tried to rebuild this package and removed those tests, but after that i got new error: "Segmentation Error".