-
Notifications
You must be signed in to change notification settings - Fork 23
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Bug: 2 install_hooks bugs #548
Conversation
dagshub/streaming/filesystem.py
Outdated
return DagshubPath(self, None, None, Path(file)) | ||
if file == "": | ||
return DagshubPath(self, None, None, orig_path) | ||
abspath = Path(os.path.abspath(file)) | ||
return DagshubPath(self, None, None, Path(file)) | ||
|
||
expanded = os.path.expanduser(file) | ||
orig_path = Path(expanded) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
- You intentionally don't expand the
orig_path
before passing it into theDagshubPath
? Just so I understand why. Maybe a comment explaining it? - I think you introduced a bug here. The default
open
doesn'texpanduser
on the supplied path, from what I can gather. But now, with this change, every call toopen
will pass through us and get parsed here and then you use the returned absolute path in the call to__open
. So I think instead, to be really safe, you should make sure to always call the underlying patched functions with exactly the same args as you were called with, making sure they're not modified in any way
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I guess comment 2 is also relevant for __chdir
and __listdir
etc.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
[duplicating in-person chat]
You intentionally don't expand the orig_path before passing it into the DagshubPath? Just so I understand why. Maybe a comment explaining it?
Yes, this is because this original path is later reused in functions that have different behavior depending on the accessed paths (notable example: listdir
, that returns different paths depending on whether passed an absolute or relative path)
Yea, you're right, I actually shouldn't be expanding it at all probably.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Added explaining comment to each of the field of DagsHubPath and how it's being used
This reverts commit 56ba8b3.
Reverted the fix for "bug" 1, because after discussion we realised that it might just be a bug of the calling library that it's trying to access |
Bug 1: Paths with
~
in them not getting expanded to the user home.Example:
The tilde wasn't being expanded and the wrong path was being read as a result (it was reading
$CWD/~/some/file.txt
)Bug 2: Recursive call in
DagsHubFilesystem.listdir()
for passthrough paths:If accessing a passthrough directory inside a mounted fs (e.g. something with
site-packages
), the listdir of the fs kept calling itself instead of passing through to the OS filesystem.Example: