Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

OSS Sync | Mon Sep 11 11:20:30 UTC 2023 #569

Open
wants to merge 55 commits into
base: master
Choose a base branch
from

Commits on Sep 11, 2023

  1. Ds scalafmt (#32485)

    * format: data streams/
    
    * format: iptf/
    
    GitOrigin-RevId: a31eb0dc0eaff014511665e52497dea67a3f9ff1
    leonbur authored and wix-oss committed Sep 11, 2023
    Configuration menu
    Copy the full SHA
    1c7b3ce View commit details
    Browse the repository at this point in the history
  2. Remove unused test deps (#33323)

    * Remove unused test deps
    
    * Remove unused imports
    
    * Add runtime deps
    
    * Initial cross-repo check
    
    * Updating cross-repo check
    
    * Updating cross-repo check
    
    * Updating cross-repo check
    
    ---------
    
    Co-authored-by: wixapiregistry <[email protected]>
    GitOrigin-RevId: bd83b6b6f4ca9706f6b9e0f247a8cac8bd3f1292
    2 people authored and wix-oss committed Sep 11, 2023
    Configuration menu
    Copy the full SHA
    3d3a2c0 View commit details
    Browse the repository at this point in the history
  3. Blocking retries attempts tracking fix (#32757)

    * Blocking retries attempts tracking fix #pr
    
    * Factored out RetryAttempt logic, TODO tests
    
    * Added RetryAttemptTest
    
    * Reverted RetryHeader back
    
    * Fixed retry test
    
    GitOrigin-RevId: 7e0e8cbccdd4c5fe6446775262b6695697785b2b
    atamurius authored and wix-oss committed Sep 11, 2023
    Configuration menu
    Copy the full SHA
    e276ca6 View commit details
    Browse the repository at this point in the history
  4. Revert "Blocking retries attempts tracking fix" (#33818)

    Revert "Blocking retries attempts tracking fix (#32757)"
    
    This reverts commit 7e0e8cbccdd4c5fe6446775262b6695697785b2b.
    
    GitOrigin-RevId: eb859fcf70860c06a0bd0492b746ed3ff00bc8ac
    natansil authored and wix-oss committed Sep 11, 2023
    Configuration menu
    Copy the full SHA
    eb184b2 View commit details
    Browse the repository at this point in the history
  5. [greyhound] remove internal topic creation - wix adapter (#33820)

    not create retry topics inside RecordConsumer builder, from wix adapter (it's already created). Removing last reference to an actual AdminClient in wix-adapter when asking to use proxy.
    
    GitOrigin-RevId: fb1416141b6b05559b1b606576cc03556b6a78f4
    berman7 authored and wix-oss committed Sep 11, 2023
    Configuration menu
    Copy the full SHA
    52de8e7 View commit details
    Browse the repository at this point in the history
  6. [greyhound] parallel consumer OffsetsAndGaps (#33605)

    GitOrigin-RevId: f178c94663c7cbcb22bd7266c3e15919d8997d8c
    ben-wattelman authored and wix-oss committed Sep 11, 2023
    Configuration menu
    Copy the full SHA
    271302e View commit details
    Browse the repository at this point in the history
  7. Blocking retries attempts tracking fix + fix transitive header bug (#…

    …33827)
    
    * Revert "Revert "Blocking retries attempts tracking fix" (#33818)"
    
    This reverts commit eb859fcf70860c06a0bd0492b746ed3ff00bc8ac.
    
    * fix bug on retryAttempt resolution - ignore propagated headers
    
    * CR change
    
    * fix test. change retry name to correctly structured one
    
    GitOrigin-RevId: 53fb7cf3efa315f496a00ade574397e30f20eeaf
    natansil authored and wix-oss committed Sep 11, 2023
    Configuration menu
    Copy the full SHA
    b90bbfa View commit details
    Browse the repository at this point in the history
  8. [greyhound] expose internal kafka producer metrics (#34160)

    GitOrigin-RevId: ab792e2227f389d5642802944a982f3a8d3587d8
    berman7 authored and wix-oss committed Sep 11, 2023
    Configuration menu
    Copy the full SHA
    305c732 View commit details
    Browse the repository at this point in the history
  9. [greyhound] Cooperative Rebalance fix (#34153)

    * [greyhound] code cleanup #pr
    
    * fix for paused partitions unrevoked in cooperative
    
    GitOrigin-RevId: 7cf8fe80c2ac9108b3ebe72472355fe73158df9a
    leonbur authored and wix-oss committed Sep 11, 2023
    Configuration menu
    Copy the full SHA
    a06705c View commit details
    Browse the repository at this point in the history
  10. code cleanup (#34171)

    GitOrigin-RevId: c2a4d2c2de669eff3c98b2277be8502d834de3e4
    leonbur authored and wix-oss committed Sep 11, 2023
    Configuration menu
    Copy the full SHA
    d81a795 View commit details
    Browse the repository at this point in the history
  11. [greyhound] parallel consumer implementation (#34061)

    GitOrigin-RevId: 5c9cc1ea4fcc5935c6f905a5269ac17f6b82c294
    ben-wattelman authored and wix-oss committed Sep 11, 2023
    Configuration menu
    Copy the full SHA
    0dc2bdb View commit details
    Browse the repository at this point in the history
  12. [gh-consumers-proxy] s3 bridge (#34839)

    * [gh-consumers-proxy] s3 bridge #pr #skipreview
    
    * .
    
    * .
    
    * .
    
    * .
    
    * .
    
    * .
    
    * .
    
    * .
    
    GitOrigin-RevId: fed83b505e4772a09ae189c6a01ac15368e60600
    berman7 authored and wix-oss committed Sep 11, 2023
    Configuration menu
    Copy the full SHA
    0abd3fc View commit details
    Browse the repository at this point in the history
  13. [greyhound] parallel consumer - add visibility (#34908)

    GitOrigin-RevId: 611f9b35285657e84dfaf69cf2af1744b54672e8
    ben-wattelman authored and wix-oss committed Sep 11, 2023
    Configuration menu
    Copy the full SHA
    86d8e73 View commit details
    Browse the repository at this point in the history
  14. [greyhound] parallel consumer visibility (#34926)

    GitOrigin-RevId: 3cbafbea85e93a678fc7528f1b3f4bb75514b9f2
    ben-wattelman authored and wix-oss committed Sep 11, 2023
    Configuration menu
    Copy the full SHA
    fb833c6 View commit details
    Browse the repository at this point in the history
  15. [greyhound] parallel consumer - add offsets and gaps init (#35027)

    GitOrigin-RevId: fe53459fd02e8e627356cbe68d71e4d4bc6b3d24
    ben-wattelman authored and wix-oss committed Sep 11, 2023
    Configuration menu
    Copy the full SHA
    d73a3bd View commit details
    Browse the repository at this point in the history
  16. [greyhound] parallel consumer - add grouping for no-key records (#35071)

    GitOrigin-RevId: 9e6dc2bb61a9b56aa55640567702ecb66c67563a
    ben-wattelman authored and wix-oss committed Sep 11, 2023
    Configuration menu
    Copy the full SHA
    1d2929d View commit details
    Browse the repository at this point in the history
  17. make GreyhoundNG RetryConfig serializable (so can be sent over gRpc a…

    …nd persisted) (#35207)
    
    * make GreyhoundNG RetryConfig serializable (so can be sent over gRpc and persisted) #pr
    
    * fix lp build
    
    * full cross-repo
    
    * Initial cross-repo check
    
    * Updating cross-repo check
    
    * Updating cross-repo check
    
    * Updating cross-repo check
    
    * Updating cross-repo check
    
    * CR changes
    
    ---------
    
    Co-authored-by: wixapiregistry <[email protected]>
    GitOrigin-RevId: feaf45b3c77f9074a3445bffdd6f748d3df77056
    2 people authored and wix-oss committed Sep 11, 2023
    Configuration menu
    Copy the full SHA
    f4cad57 View commit details
    Browse the repository at this point in the history
  18. multi-tenant consumer proxy redesign - initial commit (#35244)

    GitOrigin-RevId: e7a2532372964be70c34350951ca99bfc52f7684
    berman7 authored and wix-oss committed Sep 11, 2023
    Configuration menu
    Copy the full SHA
    f740257 View commit details
    Browse the repository at this point in the history
  19. [greyhound] parallel consumer - add gaps limit (#35313)

    GitOrigin-RevId: af1fd4bede9f00146454b2f28657bc1c845ae682
    ben-wattelman authored and wix-oss committed Sep 11, 2023
    Configuration menu
    Copy the full SHA
    dcffb99 View commit details
    Browse the repository at this point in the history
  20. Fixes in preparation for ZIO 2.0.15 (#35320)

    * batch event loop fix
    
    * fix worker timeout interruption. cleanup code
    
    GitOrigin-RevId: 31023194ccc982dfe8720620bfbfe26dc274918c
    leonbur authored and wix-oss committed Sep 11, 2023
    Configuration menu
    Copy the full SHA
    eaf82f8 View commit details
    Browse the repository at this point in the history
  21. [greyhound-consumer-proxy] start from latest offset when group doesn'…

    …t exist (temporary until full config is done) (#35428)
    
    * [greyhound-consumer-proxy] start from latest offset when group doesn't exist (temporary until full config is done) #automerge #skipreview
    
    * .
    
    GitOrigin-RevId: 34b953997eee3d167bb2beb23d50045daf03460f
    berman7 authored and wix-oss committed Sep 11, 2023
    Configuration menu
    Copy the full SHA
    7bf6ccf View commit details
    Browse the repository at this point in the history
  22. [greyhound] parallel consumer - compression and encoding for gaps lim…

    …it (#35423)
    
    GitOrigin-RevId: 10b761cb8fcd926178aebf586e99ed1a8829c734
    ben-wattelman authored and wix-oss committed Sep 11, 2023
    Configuration menu
    Copy the full SHA
    ab23221 View commit details
    Browse the repository at this point in the history
  23. greyhound proxy non blocking retries (#35456)

    * greyhound proxy retry: API + failing test + gh side
    
    * extract and unit test current state
    
    * remainder
    
    * fix UT
    
    * fix ITs
    
    * try with increasing eventually
    
    GitOrigin-RevId: e60cbb04d6d6b603e22954bbe4a8658ef594712a
    natansil authored and wix-oss committed Sep 11, 2023
    Configuration menu
    Copy the full SHA
    18fab35 View commit details
    Browse the repository at this point in the history
  24. new kafka monitor server for proxy (currently) (#35598)

    * new kafka monitor server for proxy (currently) #automerge #skipreview
    
    * .
    
    * .
    
    * .
    
    GitOrigin-RevId: 156ff1cbde2b11cfb35259600de00b47e1012367
    berman7 authored and wix-oss committed Sep 11, 2023
    Configuration menu
    Copy the full SHA
    b02c6fe View commit details
    Browse the repository at this point in the history
  25. [greyhound] fix OffsetsInitializer metadata bug (#35684)

    GitOrigin-RevId: d9758275c502278ed9fd845b3781cd92b7152f8a
    ben-wattelman authored and wix-oss committed Sep 11, 2023
    Configuration menu
    Copy the full SHA
    1b0b972 View commit details
    Browse the repository at this point in the history
  26. ConsumerIT: add a test: allow to override offsetReset with autoResetO…

    …ffset from extra properties (#35845)
    
    ConsumerIT: add a test: allow to override offsetReset with autoResetOffset from extra properties #automerge
    GitOrigin-RevId: a9159b9c15445daa243e84a0b4bcaeae8db8b29d
    natansil authored and wix-oss committed Sep 11, 2023
    Configuration menu
    Copy the full SHA
    90ad440 View commit details
    Browse the repository at this point in the history
  27. gh RecordConsumer - add visibility to extra properties setup with aut…

    …o.offset.reset (#35848)
    
    GitOrigin-RevId: 7b878399acd1e25003ccae071e817ed22b2fba21
    natansil authored and wix-oss committed Sep 11, 2023
    Configuration menu
    Copy the full SHA
    650fb79 View commit details
    Browse the repository at this point in the history
  28. gh RecordConsumer - allow to override offsetReset with autoResetOffse…

    …t from extra properties taking into account a non-zero rewindUncommittedOffsetsBy (#35863)
    
    * gh RecordConsumer - allow to override offsetReset with autoResetOffset from extra properties taking into account a non-zero rewindUncommittedOffsetsBy #automerge
    
    * fix tests depending on com.wixpress.greyhound.GreyhoundTestingSupport
    
    * try another tactic to fix tests
    
    * fix build file #automerge
    
    GitOrigin-RevId: dcf84b5740a3f99e1a637fca84c0ef6059b0e1a9
    natansil authored and wix-oss committed Sep 11, 2023
    Configuration menu
    Copy the full SHA
    dc37dfc View commit details
    Browse the repository at this point in the history
  29. [greyhound] gaps limit calculation script (#35898)

    [greyhound] gaps limit calculation script #automerge
    
    GitOrigin-RevId: 75a060f20589036570bad2077d3e906fa8db9bcd
    ben-wattelman authored and wix-oss committed Sep 11, 2023
    Configuration menu
    Copy the full SHA
    219f82e View commit details
    Browse the repository at this point in the history
  30. [greyhound] parallel consumer - update gaps limit (#35902)

    [greyhound] parallel consumer - update gaps limit #automerge
    
    GitOrigin-RevId: bebbfb8b314cb96c4f7ce09d79686f2163bc7a6d
    ben-wattelman authored and wix-oss committed Sep 11, 2023
    Configuration menu
    Copy the full SHA
    a56e65c View commit details
    Browse the repository at this point in the history
  31. [gh-consumer-proxy] fix: if position is lower than beginning offset -…

    … move forward to first offset (#35924)
    
    GitOrigin-RevId: 881fe31e29499ec28bfbd5cd6f526feac6b4d363
    berman7 authored and wix-oss committed Sep 11, 2023
    Configuration menu
    Copy the full SHA
    dfdbe0e View commit details
    Browse the repository at this point in the history
  32. fix build for greyhound publish to maven (#35951)

    fix build for greyhound publish to maven #automerge
    
    GitOrigin-RevId: 65676f4cc39274c75267220d7b90e904c9d1d5d8
    natansil authored and wix-oss committed Sep 11, 2023
    Configuration menu
    Copy the full SHA
    f1b1624 View commit details
    Browse the repository at this point in the history
  33. [greyhound] parallel consumer - improve logging (#36009)

    * [greyhound] parallel consumer - improve logging
    
    * dummy commit
    
    GitOrigin-RevId: 60e02e82d457076a06b0c0f2a575130051db8c69
    ben-wattelman authored and wix-oss committed Sep 11, 2023
    Configuration menu
    Copy the full SHA
    c688fe2 View commit details
    Browse the repository at this point in the history
  34. [greyhound] parallel consumer - add logs (#36082)

    [greyhound] parllel consumer - add logs
    
    GitOrigin-RevId: 2d3421c1d110e20b68222e195cd82227b772dc93
    ben-wattelman authored and wix-oss committed Sep 11, 2023
    Configuration menu
    Copy the full SHA
    510f31a View commit details
    Browse the repository at this point in the history
  35. [greyhound] parallel consumer - fix update bug (#36161)

    GitOrigin-RevId: 5b75a27782dec7e534f72f57d3b3ba38c4194d1e
    ben-wattelman authored and wix-oss committed Sep 11, 2023
    Configuration menu
    Copy the full SHA
    e1bd65f View commit details
    Browse the repository at this point in the history
  36. [greyhound] change onPartitionsAssigned result to DelayedRebalanceEff…

    …ect (#36206)
    
    GitOrigin-RevId: 43388b3fc6c16782373261553029c86e5111286d
    ben-wattelman authored and wix-oss committed Sep 11, 2023
    Configuration menu
    Copy the full SHA
    a15f4a0 View commit details
    Browse the repository at this point in the history
  37. fix batch consumer test timeout interruption (#36246)

    GitOrigin-RevId: 3332f5e1405eba5397bfcc93174b0415f2220703
    leonbur authored and wix-oss committed Sep 11, 2023
    Configuration menu
    Copy the full SHA
    a665919 View commit details
    Browse the repository at this point in the history
  38. [greyhound] parallel consumer - fix skip logic (#36281)

    GitOrigin-RevId: 8fe84370c72e4bf864708f42ed0e6fe435cddc92
    ben-wattelman authored and wix-oss committed Sep 11, 2023
    Configuration menu
    Copy the full SHA
    c3f1545 View commit details
    Browse the repository at this point in the history
  39. [greyhound] parallel consumer - improve visibility (#36346)

    GitOrigin-RevId: 8f75ec7018fc2b8723e22f70b42c1bf7f8404b99
    ben-wattelman authored and wix-oss committed Sep 11, 2023
    Configuration menu
    Copy the full SHA
    d050887 View commit details
    Browse the repository at this point in the history
  40. [greyhound] parallel consumer - init gaps on every rebalance (#36375)

    GitOrigin-RevId: 5b55ff311413a0ca90f0c66b7b5be0f12c8d3582
    ben-wattelman authored and wix-oss committed Sep 11, 2023
    Configuration menu
    Copy the full SHA
    eb6b1b8 View commit details
    Browse the repository at this point in the history
  41. [greyhound] parallel cosnumer - add init log (#36421)

    [greyhound] parallel cosnumer - add init log #automerge
    
    GitOrigin-RevId: 8e6b0c2f3a877fe4b9805f60a98a814a144b2ae6
    ben-wattelman authored and wix-oss committed Sep 11, 2023
    Configuration menu
    Copy the full SHA
    9287458 View commit details
    Browse the repository at this point in the history
  42. [greyhound] parallel consumer - fix shutdown (#36489)

    GitOrigin-RevId: eb0dc817fa50494422fba10994dd4382f1a58585
    ben-wattelman authored and wix-oss committed Sep 11, 2023
    Configuration menu
    Copy the full SHA
    7c836d9 View commit details
    Browse the repository at this point in the history
  43. [greyhound] add shutdown visibility (#36571)

    [greyhound] add shutdown visibility #automerge
    
    GitOrigin-RevId: a65aaecff072d8a19e9e70f68d4da713971bd538
    ben-wattelman authored and wix-oss committed Sep 11, 2023
    Configuration menu
    Copy the full SHA
    248de42 View commit details
    Browse the repository at this point in the history
  44. [greyhound] parallel consumer - interrupt fiber on shutdown timeout (…

    …#36642)
    
    GitOrigin-RevId: aaccb23bdc18584d42b2240e5b5e63c7d9432838
    ben-wattelman authored and wix-oss committed Sep 11, 2023
    Configuration menu
    Copy the full SHA
    3983f05 View commit details
    Browse the repository at this point in the history
  45. [greyhound] parallel consumer - add disconnect on shutdown (#36660)

    GitOrigin-RevId: 84bff3d0d5156c9cd3315423becf3811fe374776
    ben-wattelman authored and wix-oss committed Sep 11, 2023
    Configuration menu
    Copy the full SHA
    dc554df View commit details
    Browse the repository at this point in the history
  46. [greyhound] parallel consumer - add OffsetsAndGaps visibility (#36685)

    GitOrigin-RevId: cfb918350d137d551a9ca88d2e88a207e4b00fc6
    ben-wattelman authored and wix-oss committed Sep 11, 2023
    Configuration menu
    Copy the full SHA
    c113286 View commit details
    Browse the repository at this point in the history
  47. enrich commit metadata with pod and timestamp (#36679)

    * enrich commit metadata with pod and timestamp
    
    GitOrigin-RevId: 2669ad3b6fdb31749768eda02cf4cf0872933e74
    leonbur authored and wix-oss committed Sep 11, 2023
    Configuration menu
    Copy the full SHA
    8863bc0 View commit details
    Browse the repository at this point in the history
  48. Metadata to commit log (#36749)

    * enrich commit metadata with pod and timestamp
    
    * fix for metrics layer
    
    GitOrigin-RevId: 8f4f026e95f2008cfe9e958a518381f31630e6c1
    leonbur authored and wix-oss committed Sep 11, 2023
    Configuration menu
    Copy the full SHA
    bc847eb View commit details
    Browse the repository at this point in the history
  49. GH commit metadata ts is in UTC (#36774)

    GH commit metadata ts is in UTC #pr #skipreview
    
    GitOrigin-RevId: 2efd9f9a307f625f4112ca172cd53e256bcd24f7
    leonbur authored and wix-oss committed Sep 11, 2023
    Configuration menu
    Copy the full SHA
    f7ee844 View commit details
    Browse the repository at this point in the history
  50. moved enrich metadata out of GH core. add it to commit on rebalance (…

    …#36783)
    
    moved enrich metadata out of GH core. add it to commit on rebalance #pr #skipreview
    
    GitOrigin-RevId: bfdcccbc4c5210870118a1d049063db157d9f771
    leonbur authored and wix-oss committed Sep 11, 2023
    Configuration menu
    Copy the full SHA
    b7922ce View commit details
    Browse the repository at this point in the history
  51. [greyhound] parallel consumer - remove noisy logs (#36777)

    GitOrigin-RevId: 67e32052558deb48f4373262a724d1ccd1751b82
    ben-wattelman authored and wix-oss committed Sep 11, 2023
    Configuration menu
    Copy the full SHA
    54aa335 View commit details
    Browse the repository at this point in the history
  52. [consumer-proxy] fix retry bug (#36902)

    fix retry bug - was taking all records in batch once the first record was finished backing off, resulting in too-soon retries for the rest of the batch (up to 2000 records)
    
    GitOrigin-RevId: 0a0abfd8c905ddb7f89696655162a92021a6fcd6
    berman7 authored and wix-oss committed Sep 11, 2023
    Configuration menu
    Copy the full SHA
    08695e9 View commit details
    Browse the repository at this point in the history
  53. [greyhound] consumer: produceWithoutShutdown under flag (#36895)

    * [greyhound] consumer: produceWithoutShutdown under flag #pr
    
    Signed-off-by: Asaf Jaffi <[email protected]>
    
    * [greyhound] consumer: produceWithoutShutdown under flag: with the actual flag condition
    
    Signed-off-by: Asaf Jaffi <[email protected]>
    
    ---------
    
    Signed-off-by: Asaf Jaffi <[email protected]>
    GitOrigin-RevId: 1985d9423d595a449839d0ff340aa8eaba8bd053
    assafj-wix authored and wix-oss committed Sep 11, 2023
    Configuration menu
    Copy the full SHA
    9a3603e View commit details
    Browse the repository at this point in the history
  54. gh-consumer-proxy - init offsets 6 minutes back for non-committed par…

    …titions (#37048)
    
    * failing test + refactor OffsetsInitializer
    
    * gh-consumer-proxy - init offsets 6 minutes back for non-committed partitions #pr
    
    GitOrigin-RevId: 30bc3e1eedecf3518233a4b509e85989a80998ca
    natansil authored and wix-oss committed Sep 11, 2023
    Configuration menu
    Copy the full SHA
    4b36171 View commit details
    Browse the repository at this point in the history
  55. [greyhound] use no-interruption-retry toggle in blocking retries as w…

    …ell (#37095)
    
    GitOrigin-RevId: f9b12e8030fb67924cdaaa27a57462ed256dd238
    berman7 authored and wix-oss committed Sep 11, 2023
    Configuration menu
    Copy the full SHA
    d83b43b View commit details
    Browse the repository at this point in the history