You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
We have many use cases where we consume a record from topic A and forward it unchanged to topic B (in some cases even to separate clusters), essentially relaying it.
This means in our handlers we need to set the consumed record topic to the actual target topic first, can't just blindly forward, e.g.
func(ctx context.Context, r kgo.Record) error {
r.Topic=produceTopicreturnproducer.Produce(ctx, r)
}
It would be handy to be able to configure the producer client to always produce to a given topic. Making r.Topic = produceTopic redundant.
Also, while the docs are clear about this, for newcomers like us, preferring the record topic over the value provided by kgo.DefaultProduceTopic resulted in forwarding the records back to the original topic producing lots of duplicates : ) when switching to this pkg. Therefore in-house we're considering adding an option like AlwaysUseDefaultProduceTopic for our kgo.Client wrappers to guard us against this in the future.
Has anyone had similar issues? And if yes, how did you handle it? Did you build anything specific?
Is there any appetite for adding a similar option to this pkg to tweak the final target topic?
The text was updated successfully, but these errors were encountered:
I would love to use this library in a POC for a new service which relays records from the consumer assigned topic to a destination topic which's name is build dynamically.
Therefore, this is unfortunately also blocking me and I'll closely monitor this issue :)
We have many use cases where we consume a record from topic
A
and forward it unchanged to topicB
(in some cases even to separate clusters), essentially relaying it.This means in our handlers we need to set the consumed record topic to the actual target topic first, can't just blindly forward, e.g.
It would be handy to be able to configure the producer client to always produce to a given topic. Making
r.Topic = produceTopic
redundant.Also, while the docs are clear about this, for newcomers like us, preferring the record topic over the value provided by
kgo.DefaultProduceTopic
resulted in forwarding the records back to the original topic producing lots of duplicates : ) when switching to this pkg. Therefore in-house we're considering adding an option likeAlwaysUseDefaultProduceTopic
for ourkgo.Client
wrappers to guard us against this in the future.Has anyone had similar issues? And if yes, how did you handle it? Did you build anything specific?
Is there any appetite for adding a similar option to this pkg to tweak the final target topic?
The text was updated successfully, but these errors were encountered: