Support package trees containing absolute symlinks

[aspiers]

[This feature request was originally confused with #11 and reported there; moving here to stop the confusion.]

Example use case given by @dmarcoux is a Stow package containing dotfiles for $HOME, in which .mysql_history, .bash_history, and .less_history are symlinks to /dev/null.

Currently Stow refuses to stow these absolute symlinks:

WARNING! stowing home would cause conflicts:
  * source is an absolute symlink dotfiles/home/.bash_history => /dev/null
  * source is an absolute symlink dotfiles/home/.lesshst => /dev/null
  * source is an absolute symlink dotfiles/home/.mysql_history => /dev/null
All operations aborted.

To clarify:

• This issue is about package trees containing absolute symlinks.
• Should we add support stow directory is symbolic link. #11 is about the Stow directory being a symlink.
• Create symlinks with absolute path names #3 is about the symlinks which Stow manages in the target directory being absolute symlinks.

They're all separate issues despite appearing similar on the surface.

[z1atk0]

Hi!

Strangely enough, I only experienced this issue starting with 2.4.0. Up to & including 2.3.1, both stowing and unstowing of absolute symlinks has never been a problem before (and I've been using stow since the early 1.3.x days, almost as long as I've been using Linux itself), despite the comment in https://github.com/aspiers/stow/blob/master/lib/Stow.pm.in#L503 (which I never knew about, as it just worked as expected all those years 🤷‍♂️).

I have not unrolled the whole history of this issue yet, but I don't understand what would actually be the problem with stowing and unstowing an absolute symlink. As far as I understand, unstowing means the deletion of relative symlinks. What difference does it make where this relative symlink is pointing to (I understand of course the difference between pointing to a directory vs. pointing to a file with regards to folding/unfolding, but why is it any different depending on what file type the symlink points to?

To break it down, manually doing ...

ln -s ../stow/some_package/bin/actual_file /usr/local/bin/actual_file
ln -s ../stow/some_package/share/doc/package/absolute_symlink_to_somewhere /usr/local/share/doc/package/absolute_symlink_to_somewhere
ln -s ../stow/some_package/dev/package_device_node /usr/local/dev/package_device_node
ln -s ../stow/some/package/var/run/package_socket /usr/local/var/run/package_socket

... would work just fine, as would ...

rm /usr/local/bin/actual_file
rm /usr/local/share/doc/package/absolute_symlink_to_somewhere
rm /usr/local/dev/package_device_node
rm /usr/local/var/run/package_socket

... to remove the symlinks again, without breaking anything. So what is the reason for the strange special-casing of absolute symlinks? Does it have to do with stow's complex folding/unfolding logic (which I admittedly didn't consider at all here 😇)? Are there some tricky non-obvious corner cases that I'm missing here?

And, if this change in behaviour from 2.3.1 to 2.4.0 was intentional, could you please point me to the commit that is responsible for it, so that I can keep a local diff around to revert it back to the old behaviour? 🤓

Thanks for listening! 🙂

[aspiers]

@z1atk0 There were quite a lot of changes between these releases so I can't answer that offhand - please could you try a git bisect to find out?

[z1atk0]

Okay, I'll try to give it a shot when I find some spare time to fool around with it. 🙂

[aspiers]

The easiest way to do this would be to create a script test.sh containing something like

#!/bin/sh -e
make maintainer-clean
autoreconf -iv
eval `perl -V:siteprefix`
automake --add-missing
./configure --prefix=$siteprefix

# insert your commands here to test the use case regarding absolute symlinks

then:

git bisect start $BAD_COMMIT $GOOD_COMMIT
git bisect run ./test.sh

[Profpatsch]

I was hit by this as well, my use-case is that I have a bunch of /nix/store/ paths that I want to link to from my user directory.

[aspiers]

Just realised my hint above was a bit misleading; now edited to be more accurate.

If you have problems running git bisect, if someone could provide a minimal test case that would be very helpful.

[z1atk0]

Okay, finally found some time to toy around with git bisect today ... here's the result:

[...]
[zlatko@disclosure:~/usrlocal/src/0-GIT/stow]$ git bisect good
afa50077c98b6fc0b13530912b2c1ea35603ee32 is the first bad commit
commit afa50077c98b6fc0b13530912b2c1ea35603ee32
Author: Adam Spiers <[email protected]>
Date:   Mon Apr 1 22:50:58 2024 +0100

    dotfiles: switch {un,}stow_{contents,node}() recursion parameters
    
    Stow walks the package and target tree hierarchies by using mutually
    recursive pairs of functions:
    
    - `stow_contents()` and `stow_node()`
    - `unstow_contents()` and `unstow_node()`
    
    As Stow runs its planning from the target directory (`plan_*()` both
    call `within_target_do()`), previously the parameters for these
    included:
    
    - `$target_subpath` (or `$target_subdir` in the `*_node()` functions):
      the relative path from the target top-level directory to the target
      subdirectory (initially `.` at the beginning of recursion).  For
      example, this could be `dir1/subdir1/file1`.
    
    - `$source`: the relative path from the target _subdirectory_ (N.B. _not_
      top-level directory) to the package subdirectory.  For example, if
      the relative path to the Stow directory is `../stow`, this could be
      `../../../stow/pkg1/dir1/subdir1/file1`.  This is used when stowing
      to construct a new link, or when unstowing to detect whether the
      link can be unstowed.
    
    Each time it descends into a further subdirectory of the target and
    package, it appends the new path segment onto both of these, and also
    prefixes `$source` with another `..`.  When the `--dotfiles` parameter
    is enabled, it adjusts `$target_subdir`, performing the `dot-foo` =>
    `.foo` adjustment on all segments of the path in one go.  In this
    case, `$target_subpath` could be something like `.dir1/subdir1/file1`,
    and the corresponding `$source` could be something like
    `../../../stow/pkg1/dot-dir1/subdir1/file1`.
    
    However this doesn't leave an easy way to obtain the relative path
    from the target _top-level_ directory to the package subdirectory
    (i.e. `../stow/pkg1/dot-dir1/subdir1/file1`), which is needed for
    checking its existence and if necessary iterating over its contents.
    
    The current implementation solves this by including an extra `$level`
    parameter which tracks the recursion depth, and uses that to strip the
    right number of leading path segments off the front of `$source`.
    (In the above example, it would remove `../..`.)
    
    This implementation isn't the most elegant because:
    
    - It involves adding things to `$source` and then removing them again.
    
    - It performs the `dot-` => `.` adjustment on every path segment
      at each level, which is overkill, since when recursing down a level,
      only adjustment on the final subdirectory is required since the higher
      segments have already had any required adjustment.
    
      This in turn requires `adjust_dotfile` to be more complex than it
      needs to be.
    
      It also prevents a potential future where we might want Stow to
      optionally start iterating from within a subdirectory of the whole
      package install image / target tree, avoiding adjustment at higher
      levels and only doing it at the levels below the starting point.
    
    - It requires passing an extra `$level` parameter which can be
      automatically calculated simply by counting the number of slashes
      in `$target_subpath`.
    
    So change the `$source` recursion parameter to instead track the
    relative path from the top-level package directory to the package
    subdirectory or file being considered for (un)stowing, and rename it
    to avoid the ambiguity caused by the word "source".
    
    Also automatically calculate the depth simply by counting the number
    of slashes, and reconstruct `$source` when needed by combining the
    relative path to the Stow directory with the package name and
    `$target_subpath`.
    
    Closes #33.

 lib/Stow.pm.in      | 251 ++++++++++++++++++++++++++++++++--------------------
 lib/Stow/Util.pm.in |  14 +--
 t/dotfiles.t        |  13 +--
 3 files changed, 160 insertions(+), 118 deletions(-)

I hope this helps in narrowing down the exact change that "broke" (although it was never intended to work, or at least not officially supported) things for me. My test case was simply trying to stow the following "package", and the above commit was where it broke:

[zlatko@disclosure:~/usrlocal/src/0-GIT/stow]$ tree /usr/local/stow/stow-test/
/usr/local/stow/stow-test/
└── bin
    └── test -> /usr/bin/test

1 directory, 1 file

Ie. /usr/local/stow/stow-test/bin/test is a simple absolute symlink to /usr/bin/test, and I tried to stow -v stow-test using the stow binary from the git bisect dir. If you need some more information and/or testing please let me know. Thanks! 🙂

EDIT: just in case someone else wants to test this, here's the test.sh I used for git bisect run ./test.sh:

#!/bin/sh -e
make maintainer-clean || true
autoreconf -iv
eval `perl -V:siteprefix`
automake --add-missing
./configure --prefix=$siteprefix
make
(cd /usr/local/stow/; sudo PERL5LIB=/usr/local/src/0-GIT/stow/lib /usr/local/src/0-GIT/stow/bin/stow -v stow-test && sudo PERL5LIB=/usr/local/src/0-GIT/stow/lib /usr/local/src/0-GIT/stow/bin/stow -v -D stow-test)