Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[DO NOT MERGE] Backport some fixes and compatibility commits from master to ozone-1.4 #7218

Merged
merged 16 commits into from
Sep 29, 2024

Conversation

xichen01
Copy link
Contributor

@xichen01 xichen01 commented Sep 19, 2024

What changes were proposed in this pull request?

Backport the following commit from master to ozone-1.4 for 1.4.1. Including some bug fixes and necessary dependencies.

HDDS-8900. Mark TestSecretKeysApi#testSecretKeyApiSuccess as flaky
HDDS-10985. EC Reconstruction failed because the size of currentChunks was not equal to checksumBlockDataChunks. (#7009)
HDDS-11375. DN startup fails due to illegal configuration of raft.grpc.message.size.max (#7128)
HDDS-11350. NullPointerException thrown on checking container balancer status (#7134)
HDDS-11120. Rich rebalancing status info (#6911)
HDDS-9889. Refactor tests related to dynamical adaptation for datanode limits in ContainerBalancer (#5758)
HDDS-10612. Add Robot test to verify Container Balancer for RATIS containers (#6457)
HDDS-11331. Fix Datanode unable to report for a long time (#7090)
HDDS-11320. Update OM, SCM, Datanode conf for RATIS-2135. (#7080)
HDDS-10773. Simplify OM RaftProperties formatting (#6605)
HDDS-10761. Add raft close threshold config to OM RaftProperties (#6594)
HDDS-11309. Increase CONTAINER_STATE Column Length in UNHEALTHY_CONTAINERS to Avoid Truncation (#7071)
HDDS-10293. IllegalArgumentException: containerSize Negative (#6178)
HDDS-9819. Recon - Potential memory overflow in Container Health Task. (#5841)
HDDS-11209. Avoid insufficient EC pipelines in the container pipeline cache (#6974)
HDDS-11152. OMDoubleBuffer error when handling snapshot's background operations (#7112)
HDDS-9198. Maintain local cache in OMSnapshotPurgeRequest to get updated snapshotInfo and pass the same to OMSnapshotPurgeResponse (#7045)
HDDS-11137. Removed locks from SnapshotPurge and SnapshotSetProperty APIs. (#7018)

How was this patch tested?

https://github.com/xichen01/ozone/actions/runs/10927685549

hemantk-12 and others added 13 commits September 16, 2024 01:37
…ted snapshotInfo and pass the same to OMSnapshotPurgeResponse (apache#7045)

(cherry picked from commit be34303)
…INERS to Avoid Truncation (apache#7071)

(cherry picked from commit 8f8d809)
…e limits in ContainerBalancer (apache#5758)

(cherry picked from commit 78a7e7a)
@xichen01
Copy link
Contributor Author

@adoroszlai @hemantk-12 @whbing @devmadhuu @ArafatKhan2198 @szetszwo @jianghuazhu @afilpp @Montura @juncevich @siddhantsangwan @jojochuang @slfan1989 please check that your commits are correctly cherry-picked.

@juncevich
Copy link
Contributor

@xichen01, LGTM.

@devmadhuu
Copy link
Contributor

@xichen01 LGTM.

2 similar comments
@slfan1989
Copy link
Contributor

@xichen01 LGTM.

@jianghuazhu
Copy link
Contributor

@xichen01 LGTM.

@jojochuang
Copy link
Contributor

HDDS-11375 LGTM

Copy link
Contributor

@siddhantsangwan siddhantsangwan left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

My commits look OK. Thanks @xichen01.

@whbing
Copy link
Contributor

whbing commented Sep 24, 2024

@xichen01 LGTM for my commits

@adoroszlai
Copy link
Contributor

HDDS-11120. Rich rebalancing status info (#6911)

I'm not sure this is appropriate for a patch release.

@ArafatKhan2198
Copy link
Contributor

@xichen01 LGTM!

@hemantk-12
Copy link
Contributor

Thanks, @xichen01 for backporting these changes. Snapshot changes look good to me +1.

@xichen01
Copy link
Contributor Author

HDDS-11120. Rich rebalancing status info (#6911)

I'm not sure this is appropriate for a patch release.

@adoroszlai
HDDS-11120 makes "cherry-pick" HDDS-11350 less conflicting, and it's also a useful ticket, so it's backported too.

@adoroszlai
Copy link
Contributor

HDDS-11120. Rich rebalancing status info (#6911)

I'm not sure this is appropriate for a patch release.

HDDS-11120 makes "cherry-pick" HDDS-11350 less conflicting, and it's also a useful ticket, so it's backported too.

Patch releases are not for "useful" tickets (improvements), rather for bugfixes and security issues. Does HDDS-11350 affect 1.4.0 anyway?

Please also see HDDS-11385, which is a follow-up to HDDS-11120.

jojochuang and others added 2 commits September 29, 2024 00:36
…s was not equal to checksumBlockDataChunks. (apache#7009)

(cherry picked from commit 0915f0b)
@xichen01
Copy link
Contributor Author

HDDS-11350 is caused by HDDS-11120, not a fix for the previous bug. So if we don't cherry-pick HDDS-11120, the HDDS-11350 doesn't need to be backported.
So remove the patches HDDS-11350 and HDDS-11120 in the release.
cc: @adoroszlai

@xichen01 xichen01 merged commit 0b22d8f into apache:ozone-1.4 Sep 29, 2024
36 of 38 checks passed
@xichen01
Copy link
Contributor Author

Merged using:

git checkout ozone-1.4
git pull
git merge --ff-only ozone-1.4-backport
git push

@xichen01 xichen01 deleted the ozone-1.4-backport branch October 1, 2024 11:55
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging this pull request may close these issues.