Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

allow extensions to trigger from packages in [deps] #54009

Merged
merged 4 commits into from
Apr 10, 2024

Conversation

KristofferC
Copy link
Member

@KristofferC KristofferC commented Apr 9, 2024

There is a use case where you have a weak dependency (for one of your extensions) that is misbehaving and you quickly want to try debug that issue. A workflow that feels reasonable for this could be:

pkg> dev WeakDependency

# do some fixes in this dependency

julia> using Package, WeakDependency
# this loads the extension of Package triggered by loading WeakDependency

# check that things work ok now

This doesn't work right now for two reasons:

  1. Doing the dev WeakDependency will add the dependency to [deps] but not remove it from [weakdeps] which means you all of a sudden are in the scenario described in https://pkgdocs.julialang.org/v1/creating-packages/#Transition-from-normal-dependency-to-extension which is not what is desired.
  2. The extension will not actually load because you can right now only trigger extensions from weak deps getting loaded, not from deps getting loaded.

Point 1. is fixed by JuliaLang/Pkg.jl#3865
Point 2. is fixed by this PR.

@KristofferC KristofferC added packages Package management and loading package extensions labels Apr 9, 2024
@KristofferC KristofferC requested a review from Keno April 9, 2024 14:55
@KristofferC
Copy link
Member Author

The cycle detector for parallel precompile (once improved, see #53972) might be a bit upset with this workflow since it will assume that Package actually load WeakDependency but at least it would make this type of workflow possible.

@Keno
Copy link
Member

Keno commented Apr 9, 2024

I think this is a good compromise. Not what you want for the long term of course, but it's frustrating not to be able to dev a particular version of a weakdep to fix a bug in it.

@KristofferC KristofferC merged commit f46cb4c into master Apr 10, 2024
5 of 7 checks passed
@KristofferC KristofferC deleted the kc/triggers_deps branch April 10, 2024 18:08
@maleadt maleadt added the backport 1.11 Change should be backported to release-1.11 label Sep 26, 2024
@maleadt
Copy link
Member

maleadt commented Sep 26, 2024

Adding backport label as suggested in JuliaLang/Pkg.jl#4028 (comment).

@KristofferC KristofferC mentioned this pull request Sep 30, 2024
39 tasks
KristofferC added a commit that referenced this pull request Oct 1, 2024
Backported PRs:
- [x] #55849 <!-- Mmap: fix grow! for non file IOs -->
- [x] #55863 <!-- Update TaskLocalRNG docstring according to #49110 -->
- [x] #54433 <!-- Root globals in toplevel exprs -->
- [x] #55828 <!-- Fix some corner cases of `isapprox` with unsigned
integers -->
- [x] #55890 <!-- Profile: fix order of fields in heapsnapshot & improve
formatting -->
- [x] #55884 <!-- inference: add missing `TypeVar` handling for
`instanceof_tfunc` -->
- [x] #55881 <!-- Install terminfo data under /usr/share/julia -->
- [x] #55909 <!-- do not intentionally suppress errors in precompile
script from being reported or failing the result -->
- [x] #55355 <!-- relocation: account for trailing path separator in
depot paths -->
- [x] #55917 <!-- fix rawbigints OOB issues -->
- [x] #55892 <!-- TOML: Avoid type-pirating `Base.TOML.Parser` -->
- [x] #55798 <!-- Broadcast binary ops involving strided triangular -->
- [x] #55919 <!-- Limit `@inbounds` to indexing in the dual-iterator
branch in `copyto_unaliased!` -->

Contains multiple commits, manual intervention needed:
- [ ] #54009 <!-- allow extensions to trigger from packages in [deps]
-->
- [ ] #55509 <!-- Fix cong implementation to be properly random and not
just cycling. -->
- [ ] #55569 <!-- Add a docs section about loading/precomp/ttfx time
tuning -->
- [ ] #55824 <!-- Replace regex package module checks with actual code
checks -->

Non-merged PRs with backport label:
- [ ] #55932 <!-- REPL: make UndefVarError aware of imported modules -->
- [ ] #55910 <!-- Prevent extensions from blocking parallel
pre-compilation -->
- [ ] #55908 <!-- add logic to prefer loading modules that are already
loaded -->
- [ ] #55886 <!-- irrationals: restrict assume effects annotations to
known types -->
- [ ] #55871 <!-- lowering: don't reverse handler order in
`(pop-handler-list ...)` -->
- [ ] #55870 <!-- fix infinite recursion in `promote_type` for
`Irrational` -->
- [ ] #55867 <!-- update `hash` doc string: `widen` not required any
more -->
- [ ] #55851 <!-- [REPL] Fix #55850 by using `safe_realpath` instead of
`abspath` in `projname` -->
- [ ] #55813 <!-- Check for conflicting `@ccallable` name before JIT
registration -->
- [ ] #55743 <!-- doc: heap snapshot viewing -->
- [ ] #55741 <!-- Change annotations to use a NamedTuple -->
- [ ] #55534 <!-- Set stdlib sources as read-only during installation
-->
- [ ] #55499 <!-- propagate the terminal's `displaysize` to the
`IOContext` used by the REPL -->
- [ ] #55458 <!-- Allow for generically extracting unannotated string
-->
- [ ] #55457 <!-- Make AnnotateChar equality consider annotations -->
- [ ] #55220 <!-- `isfile_casesensitive` fixes on Windows -->
- [ ] #55169 <!-- `propertynames` for SVD respects private argument -->
- [ ] #54457 <!-- Make `String(::Memory)` copy -->
- [ ] #53957 <!-- tweak how filtering is done for what packages should
be precompiled -->
- [ ] #51479 <!-- prevent code loading from lookin in the versioned
environment when building Julia -->
- [ ] #50813 <!-- More doctests for Sockets and capitalization fix -->
- [ ] #50157 <!-- improve docs for `@inbounds` and
`Base.@propagate_inbounds` -->
@KristofferC KristofferC mentioned this pull request Oct 7, 2024
44 tasks
KristofferC added a commit that referenced this pull request Oct 9, 2024
There is a use case where you have a weak dependency (for one of your
extensions) that is misbehaving and you quickly want to try debug that
issue. A workflow that feels reasonable for this could be:

```
pkg> dev WeakDependency

julia> using Package, WeakDependency

```

This doesn't work right now for two reasons:

1. Doing the `dev WeakDependency` will add the dependency to `[deps]`
but not remove it from `[weakdeps]` which means you all of a sudden are
in the scenario described in
https://pkgdocs.julialang.org/v1/creating-packages/#Transition-from-normal-dependency-to-extension
which is not what is desired.
2. The extension will not actually load because you can right now only
trigger extensions from weak deps getting loaded, not from deps getting
loaded.

Point 1. is fixed by JuliaLang/Pkg.jl#3865
Point 2. is fixed by this PR.

(cherry picked from commit f46cb4c)
KristofferC added a commit that referenced this pull request Oct 15, 2024
Backported PRs:
- [x] #55945 <!-- Fix logic in `?` docstring example -->
- [x] #55932 <!-- REPL: make UndefVarError aware of imported modules -->
- [x] #55968 <!-- [build] avoid libedit linkage and align libccalllazy*
SONAMEs -->
- [x] #55977 <!-- fix comma logic in time_print -->
- [x] #55982 <!-- `@time` actually fix time report commas & add tests
-->
- [x] #55743 <!-- doc: heap snapshot viewing -->
- [x] #55851 <!-- [REPL] Fix #55850 by using `safe_realpath` instead of
`abspath` in `projname` -->
- [x] #55992 <!-- Avoid `stat`-ing stdlib path if it's unreadable -->
- [x] #55589 <!-- prevent loading other extensions when precompiling an
extension -->
- [x] #54009 <!-- allow extensions to trigger from packages in [deps]
-->
- [x] #56019 <!-- Fix no-arg `ScopedValues.@with` within a scope -->
- [x] #56023 <!-- Sockets: Warn when local network access not granted.
-->
- [x] #55569 <!-- Add a docs section about loading/precomp/ttfx time
tuning -->
- [x] #55824 <!-- Replace regex package module checks with actual code
checks -->
- [x] #56041 <!-- Don't show keymap `@error` for hints -->
- [x] #53469
- [x] #56029 <!-- fix `_growbeg!` unncessary resizing -->
- [x] #56103
- [x] #55941 <!-- Fix zero elements for block-matrix kron involving
Diagonal -->
@KristofferC KristofferC removed the backport 1.11 Change should be backported to release-1.11 label Oct 18, 2024
@topolarity topolarity added the backport 1.10 Change should be backported to the 1.10 release label Oct 29, 2024
@topolarity
Copy link
Member

Added 1.10 backport label as suggested here: #56368 (comment)

@topolarity topolarity mentioned this pull request Oct 29, 2024
47 tasks
topolarity pushed a commit to topolarity/julia that referenced this pull request Oct 29, 2024
… [deps] (JuliaLang#54009)

There is a use case where you have a weak dependency (for one of your
extensions) that is misbehaving and you quickly want to try debug that
issue. A workflow that feels reasonable for this could be:

```
pkg> dev WeakDependency

julia> using Package, WeakDependency

```

This doesn't work right now for two reasons:

1. Doing the `dev WeakDependency` will add the dependency to `[deps]`
but not remove it from `[weakdeps]` which means you all of a sudden are
in the scenario described in
https://pkgdocs.julialang.org/v1/creating-packages/#Transition-from-normal-dependency-to-extension
which is not what is desired.
2. The extension will not actually load because you can right now only
trigger extensions from weak deps getting loaded, not from deps getting
loaded.

Point 1. is fixed by JuliaLang/Pkg.jl#3865
Point 2. is fixed by this PR.

(cherry picked from commit f46cb4c)
topolarity added a commit to JuliaLang/Pkg.jl that referenced this pull request Oct 29, 2024
This is a partial backport of JuliaLang/julia#54009, with just the
pre-compilation piece that lives in Pkg on 1.10
topolarity pushed a commit to topolarity/julia that referenced this pull request Oct 30, 2024
… [deps] (JuliaLang#54009)

There is a use case where you have a weak dependency (for one of your
extensions) that is misbehaving and you quickly want to try debug that
issue. A workflow that feels reasonable for this could be:

```
pkg> dev WeakDependency

julia> using Package, WeakDependency

```

This doesn't work right now for two reasons:

1. Doing the `dev WeakDependency` will add the dependency to `[deps]`
but not remove it from `[weakdeps]` which means you all of a sudden are
in the scenario described in
https://pkgdocs.julialang.org/v1/creating-packages/#Transition-from-normal-dependency-to-extension
which is not what is desired.
2. The extension will not actually load because you can right now only
trigger extensions from weak deps getting loaded, not from deps getting
loaded.

Point 1. is fixed by JuliaLang/Pkg.jl#3865
Point 2. is fixed by this PR.

(cherry picked from commit f46cb4c)
topolarity pushed a commit to topolarity/julia that referenced this pull request Oct 30, 2024
… [deps] (JuliaLang#54009)

There is a use case where you have a weak dependency (for one of your
extensions) that is misbehaving and you quickly want to try debug that
issue. A workflow that feels reasonable for this could be:

```
pkg> dev WeakDependency

julia> using Package, WeakDependency

```

This doesn't work right now for two reasons:

1. Doing the `dev WeakDependency` will add the dependency to `[deps]`
but not remove it from `[weakdeps]` which means you all of a sudden are
in the scenario described in
https://pkgdocs.julialang.org/v1/creating-packages/#Transition-from-normal-dependency-to-extension
which is not what is desired.
2. The extension will not actually load because you can right now only
trigger extensions from weak deps getting loaded, not from deps getting
loaded.

Point 1. is fixed by JuliaLang/Pkg.jl#3865
Point 2. is fixed by this PR.

(cherry picked from commit f46cb4c)
topolarity pushed a commit to topolarity/julia that referenced this pull request Oct 30, 2024
… [deps] (JuliaLang#54009)

There is a use case where you have a weak dependency (for one of your
extensions) that is misbehaving and you quickly want to try debug that
issue. A workflow that feels reasonable for this could be:

```
pkg> dev WeakDependency

julia> using Package, WeakDependency

```

This doesn't work right now for two reasons:

1. Doing the `dev WeakDependency` will add the dependency to `[deps]`
but not remove it from `[weakdeps]` which means you all of a sudden are
in the scenario described in
https://pkgdocs.julialang.org/v1/creating-packages/#Transition-from-normal-dependency-to-extension
which is not what is desired.
2. The extension will not actually load because you can right now only
trigger extensions from weak deps getting loaded, not from deps getting
loaded.

Point 1. is fixed by JuliaLang/Pkg.jl#3865
Point 2. is fixed by this PR.

(cherry picked from commit f46cb4c)
topolarity pushed a commit to topolarity/julia that referenced this pull request Oct 30, 2024
… [deps] (JuliaLang#54009)

There is a use case where you have a weak dependency (for one of your
extensions) that is misbehaving and you quickly want to try debug that
issue. A workflow that feels reasonable for this could be:

```
pkg> dev WeakDependency

julia> using Package, WeakDependency

```

This doesn't work right now for two reasons:

1. Doing the `dev WeakDependency` will add the dependency to `[deps]`
but not remove it from `[weakdeps]` which means you all of a sudden are
in the scenario described in
https://pkgdocs.julialang.org/v1/creating-packages/#Transition-from-normal-dependency-to-extension
which is not what is desired.
2. The extension will not actually load because you can right now only
trigger extensions from weak deps getting loaded, not from deps getting
loaded.

Point 1. is fixed by JuliaLang/Pkg.jl#3865
Point 2. is fixed by this PR.

(cherry picked from commit f46cb4c)
topolarity added a commit that referenced this pull request Oct 31, 2024
… [deps] (#54009) (#56383)

This is a backport of #54009

---------

Co-authored-by: Kristoffer Carlsson <[email protected]>
Co-authored-by: KristofferC <[email protected]>
@KristofferC KristofferC mentioned this pull request Nov 22, 2024
45 tasks
@topolarity topolarity removed the backport 1.10 Change should be backported to the 1.10 release label Nov 23, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
package extensions packages Package management and loading
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants