coreutils binary contains build path / build is not reproducible

[Ecordonnier]

At the moment, the coreutils binary contains several occurrences of the absolute path under which coreutils was built, which makes the build not reproducible (the binaries will be different depending on under which path they are built). Reproducibility is important in some industries ( see also https://github.com/rust-lang/rust/labels/A-reproducibility and rust-lang/rust#111540 ) and also triggers a "buildpath" build error which I disabled in the yocto integration of coreutils as workaround.

For instance, in the Ubuntu package rust-coreutils 0.0.24-2, the binary /usr/bin/coreutils contains occurences of absolute paths starting with "/usr/share/cargo/registry/coreutils-0.0.24-2/src/uucore/..."

I can also reproduce in current master branch:

~/dev/coreutils$ cargo build --release --features unix
~/dev/coreutils$ cat target/release/coreutils | grep -a "$(pwd)"
/home/ecordonnier/dev/coreutils/src/uu/truncate/src/truncate.rsSIZE is an integer with an optional prefix and optional unit.
    '%'  => round up to multiple ofError writing clap::Error/home/ecordonnier/dev/coreutils/src/uu/cp/src/cp.rs/home/ecordonnier/dev/coreutils/src/uu/pr/src/pr.rs/home/ecordonnier/dev/coreutils/src/uu/fmt/src/fmt.rs--adjustment/home/ecordonnier/dev/coreutils/src/uu/nice/src/nice.rsA command must be given with an adjustment.: warning: setpriority: " is not a valid number: getpriority: cannot stat attributes of  (multi-call binary)
�
 �
���(�)�*�+�,�-�.�/�(�)�*�+�,�-�.�/�h�i�j�k�l�m�n�o�h�i�j�k�l�m�n�o��������B�B����������B�B����BB��BB�������B�B���FFFIFLFFIFFLSTSTDFD5D;NFD=&&&&�R�R�RS>S/home/ecordonnier/.cargo/registry/src/index.crates.io-6f17d22bba15001f/num-prime-0.4.4/src/buffer.rs/home/ecordonnier/dev/coreutils/src/uu/unexpand/src/unexpand.rsSpecifierNotAtStartOfNumberSpecifierOnlyAllowedWithLastValueTabSizeCannotBeZeroTabSizeTooLargeTabSizesMustBeAscending/home/ecordonnier/dev/coreutils/src/uu/mv/src/mv.rsThe argument '<>...' requires at least 2 values, but only 1 was providedoptions --backup and --no-clobber are mutually exclusive/home/ecordonnier/dev/coreutils/src/uu/ptx/src/ptx.rsLineOutOfRangeLineOutOfRangeOnRepetitionMatchNotFoundMatchNotFoundOnRepetitionLineNumberSmallerThanPreviousInvalidPatternSuffixFormatIncorrectSuffixFormatTooManyPercentsNotRegularFile/home/ecordonnier/dev/coreutils/src/uu/fold/src/fold.rsillegal width value (elvishpowershellzsh/home/ecordonnier/dev/coreutils/src/uu/chgrp/src/chgrp.rs/home/ecordonnier/dev/coreutils/src/uu/touch/src/touch.rsmissing file operand
...

The string "SIZE is an integer with an optional prefix and optional unit." right after "/home/ecordonnier/dev/coreutils/src/uu/truncate/src/truncate.rs" comes from https://github.com/uutils/coreutils/blob/d1f253481172786b819044f3717de9517506e910/src/uu/truncate/truncate.md

[tertsdiepraam]

There's this option that might fix your problem: https://doc.rust-lang.org/rustc/command-line-arguments.html#--remap-path-prefix-remap-source-names-in-output

I'm not sure whether this is something that we should fix necessarily if it can be configured. Let me know if this option works for you!

[Ecordonnier]

I tested this in .cargo/config.toml and it works for the X86 release build:

[build]
rustflags = ["--remap-path-prefix=/home/ecordonnier=foobar"]

However there are some instances left in the debug build., and it does not work completely for the yocto cross-build (there is one occurence left), so I'll try to debug this:

__gmon_start__^@_ITM_deregisterTMCloneTable^@_ITM_registerTMCloneTable^@__cxa_finalize^@__stdbuf^@_Unwind_Resume^@memcmp^@setvbuf^@fileno^@stdin^@stdout^@stderr^@memcpy^@_Unwind_GetTextRelBase^@_Unwind_GetDataRelBase^@close^@syscall^@munmap^@memmove^@bcmp^@__errno_location^@open64^@__tls_get_addr^@abort^@_Unwind_GetIP^@_Unwind_Backtrace^@getcwd^@strlen^@fstat64^@write^@lseek64^@writev^@read^@__xpg_strerror_r^@memset^@malloc^@posix_memalign^@free^@realpath^@pthread_setspecific^@pthread_getspecific^@realloc^@calloc^@readlink^@getenv^@exit^@getauxval^@sigaltstack^@mmap64^@__cxa_thread_atexit_impl^@statx^@_Unwind_GetLanguageSpecificData^@_Unwind_GetIPInfo^@_Unwind_GetRegionStart^@_Unwind_SetGR^@_Unwind_SetIP^@pthread_key_create^@pthread_key_delete^@dl_iterate_phdr^@_Unwind_RaiseException^@libgcc_s.so.1^@libc.so.6^@ld-linux-x86-64.so.2^@GLIBC_2.3^@GCC_4.2.0^@GCC_3.0^@GCC_3.3^@GLIBC_2.28^@GLIBC_2.16^@GLIBC_2.14^@GLIBC_2.33^@GLIBC_2.34^@GLIBC_2.3.4^@GLIBC_2.18^@GLIBC_2.2.5^@/home/ecordonnier/dev/poky/build/tmp/work/cortexa57-poky-linux/uutils-coreutils/0.0.27/recipe-sysroot-native/usr/lib:/home/ecordonnier/dev/poky/build/tmp/work/cortexa57-poky-linux/uutils-coreutils/0.0.27/recipe-sysroot-native/lib

I also tested "trim-paths" in .cargo/config.toml which is available only with cargo nightly at the moment, with the same result:

[unstable]
trim-paths = true

[profile.release]
trim-paths = ["diagnostics", "object"]

[profile.debug]
trim-paths = ["diagnostics", "object"]

Thanks for the hint. It looks like this is a generic rust issue not specific to uutils-coreutils.

[tertsdiepraam]

I'm glad it works! We could enable this by default for release builds. You mention debug builds, why would you need the reproducibility there? And I wonder where the trouble with the cross-build is!

[Ecordonnier]

I mentioned debug builds because I was wondering if debug symbols are the reason there is one instance of the buildpath left when compiling with yocto (yocto generates debug symbols for all binaries and then split them to some extra debug symbol file). You are correct that in principle for the normal cargo debug binary, reproducibility is not important.

[Ecordonnier]

The remaining occurence of the buildpath in the yocto build is caused by libstdbuf.rlib (from src/uu/stdbuf/src/libstdbuf). However I have not yet found a way to reproduce outside of yocto. When compiling uu_stdbuf directly in the uutils-coreutils repository this buildpath is not there. I'll debug further when I find some time. I also want to double-check "remap-debuginfo = true" as simpler alternative to "remap-path-prefix=XXX".