Move V3 development to main #2341

jhamman · 2024-10-11T22:52:34Z

This PR moves the current v3 branch to main. Following this merge, the following actions should be taken:

reset the default branch in this repo to main
remove the v3 branch (this is an important signal to downstream testers that are currently using @v3
update our github actions to trigger off main instead of v3
update the target branch for PRs pointing at the support/2.x branch

@zarr-developers/python-core-devs -- looking for two 👍 here.

Co-authored-by: Davis Bennett <[email protected]>

* Enable two more strict mypy options * Disallow untyped decorators

* LocalStore__eq__ #1744 Fix equality comparison in LocalStore class by implementing __eq__ method * Update src/zarr/v3/store/local.py Changed 'LocalStore' to object in __eq__ Co-authored-by: Davis Bennett <[email protected]> --------- Co-authored-by: Davis Bennett <[email protected]>

* chore: add deprecation warnings to v3 classes / functions * Resolve Mypy erorrs in `v3` branch (#1692) * refactor(v3): Using appropriate types * fix(v3): Typing fixes + minor code fixes * fix(v3): _sync_iter works with coroutines * docs(v3/store/core.py): clearer comment * fix(metadata.py): Use Any outside TYPE_CHECKING for Pydantic * fix(zarr/v3): correct zarr format + remove unused method * fix(v3/store/core.py): Potential suggestion on handling str store_like * refactor(zarr/v3): Add more typing * ci(.pre-commit-config.yaml): zarr v3 mypy checks turned on in pre-commit * Specify hatch envs using GitHub actions matrix for v3 tests (#1728) * Specify v3 hatch envs using GitHub actions matrix * Update .github/workflows/test-v3.yml Co-authored-by: Joe Hamman <[email protected]> * Update .github/workflows/test-v3.yml Co-authored-by: Joe Hamman <[email protected]> * test on 3.12 too * no 3.12 --------- Co-authored-by: Joe Hamman <[email protected]> Co-authored-by: Joe Hamman <[email protected]> * black -> ruff format + cleanup (#1639) * black -> ruff + cleanup * format * Preserve git blame * pre-commit fix * Remove outdated dev install docs from installation.rst and link to contributing.rst (#1643) Co-authored-by: Joe Hamman <[email protected]> * chore: remove old v3 implementation * chore: remove more version-conditional logic * chore: remove v3_storage_transformers.py again --------- Co-authored-by: Daniel Jahn (dahn) <[email protected]> Co-authored-by: Max Jones <[email protected]> Co-authored-by: Joe Hamman <[email protected]> Co-authored-by: Joe Hamman <[email protected]> Co-authored-by: Saransh Chopra <[email protected]> Co-authored-by: Alden Keefe Sampson <[email protected]>

Co-authored-by: Davis Bennett <[email protected]>

* move v2 code/tests to v2 directories * move zarr v3 code to root * add extra doc deps * import __version__ in root __init__

* fix: return Array from resize and update_attributes instead of AsyncArray * test(sync): add tests for sync module * clear up wait usage * _get_loop must return a loop * chore: clean up type hints * feat: add timeout to sync and sync config class, and add a test * chore: reword docstring * chore: adjust line length for the linter * update after v3 reorg merge * improve tests using asyncmock --------- Co-authored-by: Joseph Hamman <[email protected]>

* feat: functional .children method for groups * changes necessary for correctly generating list of children * add stand-alone test for group.children * give type hints a glow-up * test: use separate assert statements to avoid platform-dependent ordering issues * test: put fixtures in conftest, add MemoryStore fixture * docs: release notes * test: remove prematurely-added mock s3 fixture * fix: Rename children to members; AsyncGroup.members yields tuples of (name, AsyncArray / AsyncGroup) pairs; Group.members repackages these into a dict. * fix: make Group.members return a tuple of str, Array | Group pairs * fix: revert changes to synchronization code; this is churn that we need to deal with * make mypy happy * feat: implement member-specific iteration methods in asyncgroup * chore: clean up some post-merge issues * chore: remove extra directory added by test code --------- Co-authored-by: Joseph Hamman <[email protected]>

Co-authored-by: Joe Hamman <[email protected]>

* Simplify mypy override lists * Disallow untyped calls

* chore(ci): add numpy 2 release candidate to test matrix * also add to pyproject.toml * list env in ci * specify numpy in matrix deps * add list-env * use np.inf * more inf fixes

…py (#1848) section

* wip: more group members work * remove implicit groups refactor Group.getitem and Group.open to better handle loading members for v2/v3 * tidy

* feature(store): list_* -> AsyncGenerators - add zarr.testing.store module to support downstream use cases - set pytest-asyncio mode to auto - * revert changes to v2/test_storage.py * fix v2 import

* Use donfig for sync configuration * Consolidate concurrency config * Remove unused parameter * finish removing runtime config -- a few todos remain * fix order constructor * add basic tests for config state * add order property to Array * update comment in sharding codec --------- Co-authored-by: Joseph Hamman <[email protected]>

Co-authored-by: Joe Hamman <[email protected]>

* chore: update project settings per scientific python repo-review suggestions * newline * udpate maintainers

* Enable some ruff rules (RUF) * Apply ruff rule RUF010 RUF010 Use explicit conversion flag * Apply ruff rule RUF019 RUF019 Unnecessary key check before dictionary access * Apply ruff rule RUF100 RUF100 Unused `noqa` directive

…nto v3-main-sync-merge

V3 main sync merge

d-v-b

🚀

dstansby · 2024-10-12T15:54:16Z

I'd like to advocate for branch named v2 instead of support/2.x - v2 is shorter to type, and mirrors the v3 branch we've had going on for a while.

jhamman · 2024-10-12T16:23:49Z

@dstansby - the name of the support branch is somewhat independent of this PR but the reason I like having the support name in there is that it clearly defines the purpose of the branch -- supporting v2.

dstansby · 2024-10-12T17:33:26Z

I don't think it's orthogonal, because this PR proposes to switch main from a development branch for v2 to v3, and I think we should have consensus on the details of what is happening to v2 development, which includes branch naming, before this gets merged. So please can we hold off this until #2345 is resolved?

* fix: zarr v2 compatability fixes - port normalize_chunks from v2 - add array.store property - default to append in create * move zarr.store to zarr.storage also fix failing ci * make chunks a tuple * Apply suggestions from code review * more merge conflict resolution * fixups * fixup zipstore * Apply suggestions from code review * Apply suggestions from code review * add test * extend test * clean up parents * debug race condition * more debug * document storage classes and some developer apis * Update src/zarr/core/array.py * inherit docstrings from baseclass * fix sphinx warning * # docstring inherited * add storage guide * add missing file * update links --------- Co-authored-by: David Stansby <[email protected]>

…nto fix/logging_store_improvements

Fix/logging store improvements

fix RemoteStore.empty

fix(consolidated metadata): skip .zmetadata key in members search

…ry (#2360) Bumps the actions group with 1 update in the / directory: [sphinx](https://github.com/sphinx-doc/sphinx). Updates `sphinx` from 8.0.2 to 8.1.3 - [Release notes](https://github.com/sphinx-doc/sphinx/releases) - [Changelog](https://github.com/sphinx-doc/sphinx/blob/master/CHANGES.rst) - [Commits](sphinx-doc/sphinx@v8.0.2...v8.1.3) --- updated-dependencies: - dependency-name: sphinx dependency-type: direct:production update-type: version-update:semver-minor dependency-group: actions ... Signed-off-by: dependabot[bot] <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

dstansby and others added 30 commits April 11, 2024 20:48

Allow dmypy to be run (#1780)

3a73950

Remove unused typing ignore comments (#1781)

3a9d968

Co-authored-by: Davis Bennett <[email protected]>

Check untyped defs (#1784)

ce6fcbb

Co-authored-by: Davis Bennett <[email protected]>

[v3] Enable some more strict mypy options (#1793)

a33ebf2

* Enable two more strict mypy options * Disallow untyped decorators

moves v3 tests

472ab13

Disallow generic Any typing (#1794)

f046322

Co-authored-by: Davis Bennett <[email protected]>

V3 update pre commit (#1808)

f7993db

V3 reorg (#1809)

d1a0d99

* move v2 code/tests to v2 directories * move zarr v3 code to root * add extra doc deps * import __version__ in root __init__

remove windows testing on v3 branch (#1817)

096c900

Disallow incomplete type definitions (#1814)

5889e96

fix: add mypy to test dependencies (#1789)

c1323c4

Co-authored-by: Joe Hamman <[email protected]>

Disallow untyped calls (#1811)

f44fd09

* Simplify mypy override lists * Disallow untyped calls

chore(ci): add numpy 2 release candidate to test matrix (#1828)

19a28df

* chore(ci): add numpy 2 release candidate to test matrix * also add to pyproject.toml * list env in ci * specify numpy in matrix deps * add list-env * use np.inf * more inf fixes

fix dependencies (#1840)

dd0ea50

Add pytest to mypy dependencies (#1846)

7e944f0

chore(pre-commit): update pre-commit versions and remove attrs dep my…

4f5ca4b

…py (#1848) section

Remove implicit groups (#1827)

8d483c9

* wip: more group members work * remove implicit groups refactor Group.getitem and Group.open to better handle loading members for v2/v3 * tidy

feature(store): list_* -> AsyncGenerators (#1844)

3499acb

* feature(store): list_* -> AsyncGenerators - add zarr.testing.store module to support downstream use cases - set pytest-asyncio mode to auto - * revert changes to v2/test_storage.py * fix v2 import

adds test for codec entrypoint (#1835)

fcab650

Remove extra v3 sync module (#1856)

666a8b9

Disallow any generics in zarr.array (#1861)

f632771

Co-authored-by: Joe Hamman <[email protected]>

Fix some untyped calls (#1865)

8da3df6

chore: update project settings per scientific python repo-review (#1863)

0bb2be2

* chore: update project settings per scientific python repo-review suggestions * newline * udpate maintainers

Disallow untyped defs (#1834)

5a39ff6

Enable some ruff rules (RUF) and fix issues (#1869)

c31a785

* Enable some ruff rules (RUF) * Apply ruff rule RUF010 RUF010 Use explicit conversion flag * Apply ruff rule RUF019 RUF019 Unnecessary key check before dictionary access * Apply ruff rule RUF100 RUF100 Unused `noqa` directive

jhamman added 2 commits October 11, 2024 15:14

Merge branch 'v3' of https://github.com/zarr-developers/zarr-python i…

437234a

…nto v3-main-sync-merge

Merge pull request #2340 from jhamman/v3-main-sync-merge

3b787a4

V3 main sync merge

jhamman added the V3 Affects the v3 branch label Oct 11, 2024

jhamman requested a review from a team October 12, 2024 14:34

fix RemoteStore.empty

d901734

d-v-b approved these changes Oct 12, 2024

View reviewed changes

TomAugspurger mentioned this pull request Oct 12, 2024

Compatibility for zarr-python 3.x pydata/xarray#9552

Merged

4 tasks

This was referenced Oct 12, 2024

Rename support/2.x branch to v2? #2345

Closed

Moving v3 development to main #2308

Closed

jhamman and others added 15 commits October 12, 2024 19:27

test

5ec909a

feature(logging): include keys in logging store logs

76f835c

Merge branch 'v3' of https://github.com/zarr-developers/zarr-python i…

04b12d3

…nto fix/logging_store_improvements

fix bad merge

20d8392

Merge pull request #2355 from jhamman/fix/logging_store_improvements

ad68a33

Fix/logging store improvements

Merge pull request #2343 from rabernat/ryan/fix-remote-store-exists

9bbfd88

fix RemoteStore.empty

fix(consolidated metadata): skip .zmetadata key in members search

d78d177

test + userwarning

b6e8935

fixup tests

2b1e90b

Merge pull request #2356 from jhamman/fix/consolidated-metadata-members

b26cf98

fix(consolidated metadata): skip .zmetadata key in members search

Update deprecated stage names (#2362)

016ec25

Multiple imports for an import name (#2361)

243ee1b

Fix logo height in doc navbar (#2352)

c646028

jhamman merged commit 840a3f7 into main Oct 14, 2024
26 of 27 checks passed

jhamman mentioned this pull request Oct 15, 2024

test(ci): change branch name in v3 workflows #2368

Merged

jhamman deleted the v3 branch October 15, 2024 12:29

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Move V3 development to main #2341

Move V3 development to main #2341

jhamman commented Oct 11, 2024 •

edited

Loading

d-v-b left a comment

dstansby commented Oct 12, 2024

jhamman commented Oct 12, 2024

dstansby commented Oct 12, 2024 •

edited

Loading

Move V3 development to main #2341

Move V3 development to main #2341

Conversation

jhamman commented Oct 11, 2024 • edited Loading

d-v-b left a comment

Choose a reason for hiding this comment

dstansby commented Oct 12, 2024

jhamman commented Oct 12, 2024

dstansby commented Oct 12, 2024 • edited Loading

jhamman commented Oct 11, 2024 •

edited

Loading

dstansby commented Oct 12, 2024 •

edited

Loading