Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

V3<> main sync #2330

Closed
wants to merge 131 commits into from
Closed

V3<> main sync #2330

wants to merge 131 commits into from

Conversation

jhamman
Copy link
Member

@jhamman jhamman commented Oct 10, 2024

This PR merges the history from the main branch into the v3 branch. The merge was done with --strategy ours, favoring changes that occurred in the v3 branch over those on main.

Toward: #2308

TODO:

  • Add unit tests and/or doctests in docstrings
  • Add docstrings and API docs for any new/modified user-facing classes and functions
  • New/modified features documented in docs/tutorial.rst
  • Changes documented in docs/release.rst
  • GitHub Actions have all passed
  • Test coverage is 100% (Codecov passes)

dependabot bot and others added 30 commits December 4, 2023 09:13
…rs#1586)

Bumps [pypa/gh-action-pypi-publish](https://github.com/pypa/gh-action-pypi-publish) from 1.8.10 to 1.8.11.
- [Release notes](https://github.com/pypa/gh-action-pypi-publish/releases)
- [Commits](pypa/gh-action-pypi-publish@v1.8.10...v1.8.11)

---
updated-dependencies:
- dependency-name: pypa/gh-action-pypi-publish
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
…pers#1587)

Bumps [conda-incubator/setup-miniconda](https://github.com/conda-incubator/setup-miniconda) from 2.3.0 to 3.0.1.
- [Release notes](https://github.com/conda-incubator/setup-miniconda/releases)
- [Changelog](https://github.com/conda-incubator/setup-miniconda/blob/main/CHANGELOG.md)
- [Commits](conda-incubator/setup-miniconda@v2.3.0...v3.0.1)

---
updated-dependencies:
- dependency-name: conda-incubator/setup-miniconda
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Prevent runtime-overhead in doing this check multiple times
Even if this is only a test, list comprehensions are faster than
repeatedly call append().

Also use tuple instead of list when possible.

Co-authored-by: Davis Bennett <[email protected]>
Bumps [numcodecs](https://github.com/zarr-developers/numcodecs) from 0.11.0 to 0.12.1.
- [Release notes](https://github.com/zarr-developers/numcodecs/releases)
- [Changelog](https://github.com/zarr-developers/numcodecs/blob/main/docs/release.rst)
- [Commits](zarr-developers/numcodecs@v0.11.0...v0.12.1)

---
updated-dependencies:
- dependency-name: numcodecs
  dependency-type: direct:development
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Joe Hamman <[email protected]>
It's faster and probably more readable.

Co-authored-by: Davis Bennett <[email protected]>
Co-authored-by: Joe Hamman <[email protected]>
Bumps [pymongo](https://github.com/mongodb/mongo-python-driver) from 4.5.0 to 4.6.1.
- [Release notes](https://github.com/mongodb/mongo-python-driver/releases)
- [Changelog](https://github.com/mongodb/mongo-python-driver/blob/master/doc/changelog.rst)
- [Commits](mongodb/mongo-python-driver@4.5.0...4.6.1)

---
updated-dependencies:
- dependency-name: pymongo
  dependency-type: direct:development
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Joe Hamman <[email protected]>
Starting with codespell 2.2.2, options can be specified in
`pyrpoject.toml` in addition to `setup.cfg`:
https://github.com/codespell-project/codespell#using-a-config-file

Specifying options in a config file instead of command line options in
`.pre-commit-config.yaml` ensures codespell uses the same options when
run as pre-commit hook or from the command line in the repository root
directory.
* Bump fsspec from 2023.10.0 to 2023.12.1

Bumps [fsspec](https://github.com/fsspec/filesystem_spec) from 2023.10.0 to 2023.12.1.
- [Commits](fsspec/filesystem_spec@2023.10.0...2023.12.1)

---
updated-dependencies:
- dependency-name: fsspec
  dependency-type: direct:development
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <[email protected]>

* Update s3fs as well

* Fix s3fs

---------

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Josh Moore <[email protected]>
* Add type hints to zarr.create

* Use protocol for MetaArray

* Use protocol for Synchronizer

* Fix Path typing

* Add release note

* Fix dim separator typing

* Ignore ... in coverage reporting

* Fix chunk typing

---------

Co-authored-by: Davis Bennett <[email protected]>
Bumps [actions/setup-python](https://github.com/actions/setup-python) from 4.7.1 to 5.0.0.
- [Release notes](https://github.com/actions/setup-python/releases)
- [Commits](actions/setup-python@v4.7.1...v5.0.0)

---
updated-dependencies:
- dependency-name: actions/setup-python
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Joe Hamman <[email protected]>
Bumps [github/codeql-action](https://github.com/github/codeql-action) from 2 to 3.
- [Release notes](https://github.com/github/codeql-action/releases)
- [Changelog](https://github.com/github/codeql-action/blob/main/CHANGELOG.md)
- [Commits](github/codeql-action@v2...v3)

---
updated-dependencies:
- dependency-name: github/codeql-action
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
* chore: update pre-commit hooks

updates:
- https://github.com/charliermarsh/ruff-pre-commithttps://github.com/astral-sh/ruff-pre-commit
- [github.com/astral-sh/ruff-pre-commit: v0.0.224 → v0.1.8](astral-sh/ruff-pre-commit@v0.0.224...v0.1.8)
- [github.com/psf/black: 23.10.1 → 23.12.0](psf/black@23.10.1...23.12.0)
- [github.com/codespell-project/codespell: v2.2.5 → v2.2.6](codespell-project/codespell@v2.2.5...v2.2.6)
- [github.com/pre-commit/pre-commit-hooks: v4.4.0 → v4.5.0](pre-commit/pre-commit-hooks@v4.4.0...v4.5.0)
- [github.com/pre-commit/mirrors-mypy: v1.3.0 → v1.7.1](pre-commit/mirrors-mypy@v1.3.0...v1.7.1)

* Attempt to fix ruff

* Use isinstance

---------

Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: Josh Moore <[email protected]>
updates:
- [github.com/astral-sh/ruff-pre-commit: v0.1.8 → v0.1.9](astral-sh/ruff-pre-commit@v0.1.8...v0.1.9)
- [github.com/psf/black: 23.12.0 → 23.12.1](psf/black@23.12.0...23.12.1)
- [github.com/pre-commit/mirrors-mypy: v1.7.1 → v1.8.0](pre-commit/mirrors-mypy@v1.7.1...v1.8.0)

Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
* Bump fsspec from 2023.12.1 to 2023.12.2

Bumps [fsspec](https://github.com/fsspec/filesystem_spec) from 2023.12.1 to 2023.12.2.
- [Commits](fsspec/filesystem_spec@2023.12.1...2023.12.2)

---
updated-dependencies:
- dependency-name: fsspec
  dependency-type: direct:development
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <[email protected]>

* Update requirements_dev_optional.txt

---------

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Joe Hamman <[email protected]>
Bumps [pytest-doctestplus](https://github.com/scientific-python/pytest-doctestplus) from 1.0.0 to 1.1.0.
- [Release notes](https://github.com/scientific-python/pytest-doctestplus/releases)
- [Changelog](https://github.com/scientific-python/pytest-doctestplus/blob/main/CHANGES.rst)
- [Commits](scientific-python/pytest-doctestplus@v1.0.0...v1.1.0)

---
updated-dependencies:
- dependency-name: pytest-doctestplus
  dependency-type: direct:development
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
updates:
- [github.com/astral-sh/ruff-pre-commit: v0.1.9 → v0.1.11](astral-sh/ruff-pre-commit@v0.1.9...v0.1.11)

Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
* Drop 3.8 and add 3.12

* Try removing line_profiler

* Also bump the minimal numpy to 1.21

* Drop 3.12 again

* Revert "Try removing line_profiler"

This reverts commit 837854b.

* Update release.rst

---------

Co-authored-by: Joe Hamman <[email protected]>
Co-authored-by: jakirkham <[email protected]>
updates:
- [github.com/astral-sh/ruff-pre-commit: v0.1.11 → v0.1.13](astral-sh/ruff-pre-commit@v0.1.11...v0.1.13)

Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
* Bump actions/download-artifact from 3 to 4

Bumps [actions/download-artifact](https://github.com/actions/download-artifact) from 3 to 4.
- [Release notes](https://github.com/actions/download-artifact/releases)
- [Commits](actions/download-artifact@v3...v4)

---
updated-dependencies:
- dependency-name: actions/download-artifact
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <[email protected]>

* Also bump upload-artifact

see https://github.com/actions/download-artifact?tab=readme-ov-file#breaking-changes

> Downloading artifacts that were created from action/upload-artifact@v3 and below are not supported.

---------

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Joe Hamman <[email protected]>
Co-authored-by: Josh Moore <[email protected]>
zarr-developers#1615)

* Update tutorial.rst to include section about accessing Zip Files on S3

Per discussion here, add information about about accessing zip files on s3:
zarr-developers#1613

* Update release.rst

* Implement d-v-b's suggestions

---------

Co-authored-by: Davis Bennett <[email protected]>
Co-authored-by: Josh Moore <[email protected]>
* doc(v3): add v3 roadmap and design document

* Update v3-roadmap-and-design.md

* updates after latest round of reviews

* Update v3-roadmap-and-design.md

Co-authored-by: Norman Rzepka <[email protected]>

* Update v3-roadmap-and-design.md

Co-authored-by: Sanket Verma <[email protected]>

---------

Co-authored-by: Norman Rzepka <[email protected]>
Co-authored-by: Sanket Verma <[email protected]>
updates:
- [github.com/astral-sh/ruff-pre-commit: v0.1.13 → v0.1.14](astral-sh/ruff-pre-commit@v0.1.13...v0.1.14)

Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
This patch removes fasteners and disables zarr.sync which uses process and thread

Co-authored-by: Wei Ouyang <[email protected]>
jhamman and others added 21 commits August 26, 2024 12:56
* deprecate(n5): add deprecation warning to N5Store

* also deprecate N5FSStore

* docs

* fix doc
…evelopers#2129)

Bumps the requirements group with 3 updates in the / directory: [numpy](https://github.com/numpy/numpy), [ipywidgets](https://github.com/jupyter-widgets/ipywidgets) and [azure-storage-blob](https://github.com/Azure/azure-sdk-for-python).


Updates `numpy` from 2.0.1 to 2.1.0
- [Release notes](https://github.com/numpy/numpy/releases)
- [Changelog](https://github.com/numpy/numpy/blob/main/doc/RELEASE_WALKTHROUGH.rst)
- [Commits](numpy/numpy@v2.0.1...v2.1.0)

Updates `ipywidgets` from 8.1.3 to 8.1.5
- [Release notes](https://github.com/jupyter-widgets/ipywidgets/releases)
- [Commits](jupyter-widgets/ipywidgets@8.1.3...8.1.5)

Updates `azure-storage-blob` from 12.16.0 to 12.21.0
- [Release notes](https://github.com/Azure/azure-sdk-for-python/releases)
- [Changelog](https://github.com/Azure/azure-sdk-for-python/blob/main/doc/esrp_release.md)
- [Commits](Azure/azure-sdk-for-python@azure-storage-blob_12.16.0...azure-storage-blob_12.21.0)

---
updated-dependencies:
- dependency-name: numpy
  dependency-type: direct:development
  update-type: version-update:semver-minor
  dependency-group: requirements
- dependency-name: ipywidgets
  dependency-type: direct:development
  update-type: version-update:semver-patch
  dependency-group: requirements
- dependency-name: azure-storage-blob
  dependency-type: direct:development
  update-type: version-update:semver-minor
  dependency-group: requirements
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
* chore: bump minimum version to 1.24

* Update .github/workflows/python-package.yml
Bumps the actions group with 2 updates: [actions/setup-python](https://github.com/actions/setup-python) and [pypa/gh-action-pypi-publish](https://github.com/pypa/gh-action-pypi-publish).


Updates `actions/setup-python` from 5.1.1 to 5.2.0
- [Release notes](https://github.com/actions/setup-python/releases)
- [Commits](actions/setup-python@v5.1.1...v5.2.0)

Updates `pypa/gh-action-pypi-publish` from 1.9.0 to 1.10.0
- [Release notes](https://github.com/pypa/gh-action-pypi-publish/releases)
- [Commits](pypa/gh-action-pypi-publish@v1.9.0...v1.10.0)

---
updated-dependencies:
- dependency-name: actions/setup-python
  dependency-type: direct:production
  update-type: version-update:semver-minor
  dependency-group: actions
- dependency-name: pypa/gh-action-pypi-publish
  dependency-type: direct:production
  update-type: version-update:semver-minor
  dependency-group: actions
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
…pers#2151)

Bumps the requirements group with 1 update: [numpy](https://github.com/numpy/numpy).


Updates `numpy` from 2.1.0 to 2.1.1
- [Release notes](https://github.com/numpy/numpy/releases)
- [Changelog](https://github.com/numpy/numpy/blob/main/doc/RELEASE_WALKTHROUGH.rst)
- [Commits](numpy/numpy@v2.1.0...v2.1.1)

---
updated-dependencies:
- dependency-name: numpy
  dependency-type: direct:development
  update-type: version-update:semver-patch
  dependency-group: requirements
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
… group (zarr-developers#2161)

Bumps the actions group with 1 update: [pypa/gh-action-pypi-publish](https://github.com/pypa/gh-action-pypi-publish).


Updates `pypa/gh-action-pypi-publish` from 1.10.0 to 1.10.1
- [Release notes](https://github.com/pypa/gh-action-pypi-publish/releases)
- [Commits](pypa/gh-action-pypi-publish@v1.10.0...v1.10.1)

---
updated-dependencies:
- dependency-name: pypa/gh-action-pypi-publish
  dependency-type: direct:production
  update-type: version-update:semver-patch
  dependency-group: actions
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
* remove primary sidebar from tutorial

* lint
…opers#2172)

Bumps the requirements group with 1 update: [pytest](https://github.com/pytest-dev/pytest).


Updates `pytest` from 8.3.2 to 8.3.3
- [Release notes](https://github.com/pytest-dev/pytest/releases)
- [Changelog](https://github.com/pytest-dev/pytest/blob/main/CHANGELOG.rst)
- [Commits](pytest-dev/pytest@8.3.2...8.3.3)

---
updated-dependencies:
- dependency-name: pytest
  dependency-type: direct:development
  update-type: version-update:semver-patch
  dependency-group: requirements
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
…s#2220)

Bumps the actions group with 1 update: [pypa/gh-action-pypi-publish](https://github.com/pypa/gh-action-pypi-publish).


Updates `pypa/gh-action-pypi-publish` from 1.10.1 to 1.10.2
- [Release notes](https://github.com/pypa/gh-action-pypi-publish/releases)
- [Commits](pypa/gh-action-pypi-publish@v1.10.1...v1.10.2)

---
updated-dependencies:
- dependency-name: pypa/gh-action-pypi-publish
  dependency-type: direct:production
  update-type: version-update:semver-patch
  dependency-group: actions
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
…evelopers#2284)

Bumps the requirements group with 3 updates in the / directory: [redis](https://github.com/redis/redis-py), [pymongo](https://github.com/mongodb/mongo-python-driver) and [h5py](https://github.com/h5py/h5py).


Updates `redis` from 5.0.8 to 5.1.0
- [Release notes](https://github.com/redis/redis-py/releases)
- [Changelog](https://github.com/redis/redis-py/blob/master/CHANGES)
- [Commits](redis/redis-py@v5.0.8...v5.1.0)

Updates `pymongo` from 4.8.0 to 4.10.0
- [Release notes](https://github.com/mongodb/mongo-python-driver/releases)
- [Changelog](https://github.com/mongodb/mongo-python-driver/blob/master/doc/changelog.rst)
- [Commits](mongodb/mongo-python-driver@4.8.0...4.10.0)

Updates `h5py` from 3.11.0 to 3.12.1
- [Release notes](https://github.com/h5py/h5py/releases)
- [Changelog](https://github.com/h5py/h5py/blob/master/docs/release_guide.rst)
- [Commits](h5py/h5py@3.11.0...3.12.1)

---
updated-dependencies:
- dependency-name: redis
  dependency-type: direct:development
  update-type: version-update:semver-minor
  dependency-group: requirements
- dependency-name: pymongo
  dependency-type: direct:development
  update-type: version-update:semver-minor
  dependency-group: requirements
- dependency-name: h5py
  dependency-type: direct:development
  update-type: version-update:semver-minor
  dependency-group: requirements
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
…velopers#2288)

Bumps the requirements group with 1 update: [pymongo](https://github.com/mongodb/mongo-python-driver).


Updates `pymongo` from 4.10.0 to 4.10.1
- [Release notes](https://github.com/mongodb/mongo-python-driver/releases)
- [Changelog](https://github.com/mongodb/mongo-python-driver/blob/master/doc/changelog.rst)
- [Commits](mongodb/mongo-python-driver@4.10.0...4.10.1)

---
updated-dependencies:
- dependency-name: pymongo
  dependency-type: direct:development
  update-type: version-update:semver-patch
  dependency-group: requirements
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
…s#2304)

Bumps the actions group with 1 update: [pypa/gh-action-pypi-publish](https://github.com/pypa/gh-action-pypi-publish).


Updates `pypa/gh-action-pypi-publish` from 1.10.2 to 1.10.3
- [Release notes](https://github.com/pypa/gh-action-pypi-publish/releases)
- [Commits](pypa/gh-action-pypi-publish@v1.10.2...v1.10.3)

---
updated-dependencies:
- dependency-name: pypa/gh-action-pypi-publish
  dependency-type: direct:production
  update-type: version-update:semver-patch
  dependency-group: actions
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Bumps the requirements group with 2 updates: [numpy](https://github.com/numpy/numpy) and [redis](https://github.com/redis/redis-py).


Updates `numpy` from 2.1.1 to 2.1.2
- [Release notes](https://github.com/numpy/numpy/releases)
- [Changelog](https://github.com/numpy/numpy/blob/main/doc/RELEASE_WALKTHROUGH.rst)
- [Commits](numpy/numpy@v2.1.1...v2.1.2)

Updates `redis` from 5.1.0 to 5.1.1
- [Release notes](https://github.com/redis/redis-py/releases)
- [Changelog](https://github.com/redis/redis-py/blob/master/CHANGES)
- [Commits](redis/redis-py@v5.1.0...v5.1.1)

---
updated-dependencies:
- dependency-name: numpy
  dependency-type: direct:development
  update-type: version-update:semver-patch
  dependency-group: requirements
- dependency-name: redis
  dependency-type: direct:development
  update-type: version-update:semver-patch
  dependency-group: requirements
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
…developers#2326)

Bumps the requirements group with 1 update: [numcodecs](https://github.com/zarr-developers/numcodecs).


Updates `numcodecs` from 0.13.0 to 0.13.1
- [Release notes](https://github.com/zarr-developers/numcodecs/releases)
- [Changelog](https://github.com/zarr-developers/numcodecs/blob/main/docs/release.rst)
- [Commits](zarr-developers/numcodecs@v0.13.0...v0.13.1)

---
updated-dependencies:
- dependency-name: numcodecs
  dependency-type: direct:development
  update-type: version-update:semver-patch
  dependency-group: requirements
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
@jhamman jhamman changed the title V3 main sync V3<> main sync Oct 10, 2024
@jhamman jhamman mentioned this pull request Oct 10, 2024
3 tasks
@dstansby
Copy link
Contributor

dstansby commented Oct 11, 2024

Since it doesn't look like this changes any files, is there a benefit to doing this that I'm missing?

@joshmoore
Copy link
Member

I was also a bit confused by that. It might be that there were enough changes that -s ours is simply too big a hammer. It might require finding changes that should be preserved and hand-merging them.

@jhamman jhamman mentioned this pull request Oct 11, 2024
@jhamman
Copy link
Member Author

jhamman commented Oct 11, 2024

Closing in favor of #2335

@jhamman jhamman closed this Oct 11, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.