rerun_py/build.rs leads to error in nixpkgs packaging

[RobWalt]

I'm currently updating the nixpkgs package of the rerun app and we run the python sdk tests on every build. However I had to patch the rerun_py/build.rs file to get everything to run like this:

NixOS/nixpkgs@ffa3876#diff-16a717928b24120d28cdc8a3638f7383e24438a652fa6c5c9289652ab1ac6669

I'm not sure it that is an nix specific issue or if it is a bug generally present in the repo. I just wanted to report it here.

If I don't patch the path, then I get an error that the directory specified in the build.rs doesn't exist, which is true. Not sure if something within your internal build system fixes this automatically or not.

Even though it's irrelevant, here's the error

   Compiling serde_bytes v0.11.9
   Compiling re_smart_channel v0.18.2 (/build/source/crates/utils/re_smart_channel)
   Compiling rmp-serde v1.1.1
   Compiling cargo-platform v0.1.2
   Compiling rustc_version v0.4.0
   Compiling cfb v0.7.3
   Compiling re_arrow2 v0.17.5
   Compiling cargo_metadata v0.14.2
   Compiling cargo_metadata v0.18.1
   Compiling infer v0.15.0
   Compiling skeptic v0.13.7
   Compiling re_build_tools v0.18.2 (/build/source/crates/build/re_build_tools)
   Compiling re_sdk v0.18.2 (/build/source/crates/top/re_sdk)
   Compiling rerun_py v0.18.2 (/build/source/rerun_py)
   Compiling ply-rs v0.1.3
error: failed to run custom build command for `rerun_py v0.18.2 (/build/source/rerun_py)`

Caused by:
  process didn't exit successfully: `/build/source/target/release/build/rerun_py-26ffb1f388ac94c1/build-script-build` (exit status: 1)
  --- stdout
  cargo:rerun-if-env-changed=IS_IN_RERUN_WORKSPACE
  cargo:rerun-if-env-changed=RERUN_IS_PUBLISHING
  cargo:rustc-env=RE_BUILD_DATETIME=
  cargo:rustc-env=RE_BUILD_GIT_HASH=
  cargo:rustc-env=RE_BUILD_GIT_BRANCH=
  cargo:rustc-env=RE_BUILD_TARGET_TRIPLE=x86_64-unknown-linux-gnu
  cargo:rustc-env=RE_BUILD_RUSTC_VERSION=1.80.1 (3f5fd8dd4 2024-08-06) (built from a source tarball)
  cargo:rustc-env=RE_BUILD_LLVM_VERSION=18.1.8
  cargo:rustc-env=RE_BUILD_IS_IN_RERUN_WORKSPACE=yes
  cargo:rerun-if-env-changed=RERUN_BUILDING_WHEEL
  cargo:rerun-if-env-changed=RERUN_ALLOW_MISSING_BIN

  --- stderr
  Environment: env-var IS_IN_RERUN_WORKSPACE is set
  ERROR: Expected to find `rerun` at `"/build/source/rerun_py/rerun_sdk/rerun_cli/rerun"`.
warning: build failed, waiting for other jobs to finish...
💥 maturin failed
  Caused by: Failed to build a native library through cargo
  Caused by: Cargo build finished with "exit status: 101": `env -u CARGO "cargo" "rustc" "--jobs" "32" "--target" "x86_64-unknown-linux-gnu" "--message-format" "json-render-diagnostics" "--locked" "--offline" "--config" "env.RERUN_BUILDING_WHEEL=\"1\"" "--manifest-path" "/build/source/rerun_py/Cargo.toml" "--release" "--lib" "--" "-C" "strip=symbols"`

[jleibs]

@RobWalt sorry for the confusion on that -- sounds like we definitely need to write a PACKAGING.md doc to help folks maintaining downstream packages like this. Also, thanks for helping out with the nixpkgs -- this has been a surprisingly popular distribution channel for us.

Our rerun-sdk python package actually inlines a build of the Rerun CLI executable into the python package so that when you pip install it you end up with a runnable standalone viewer.

If you want to replicate this functionality, the nix build script should likely be doing something similar to what we do in the conda package:

• https://github.com/conda-forge/rerun-sdk-feedstock/blob/main/recipe/build.sh#L38-L47

That said, if nix is packaging the Rerun-CLI package separately, we should probably just leave it out of the python package all together. I think you can can achieve this by simply setting RERUN_ALLOW_MISSING_BIN=1, which I believe should be easier than the patching the build.rs file as you have done.

[jleibs]

I created an issue for the meta-problem of documenting and tracking this stuff here:

• Document the details of our packaging systems in PACKAGING.md #7413

[RobWalt]

It looks like the patch isn't needed anymore in the latest release

NixOS/nixpkgs#358093