Feature Request: Add producer UID of event to event Metadata so consumers can identify the source

[conorclifford]

With the very welcome addition of security/authentication features, it is now possible to control who can read, and (more importantly from the perspective of this feature request) write to an event_type.

This allows Nakadi to be used as an asynchronous data ingestion API - allowing (controlled) clients send data to an application via Nakadi.

However, there is no (automatic/guaranteed) way to identify originating client data in this scenario - it would need to be done via a property of the data itself, which is unfortunately prone to error/omission, etc.

This feature request is to extend Nakadi to automatically add the producer's authentication UID to the metadata of the event as a simple enrichment.

[zrobgar]

Thank you for the request, We will discuss it internally.

[whiskeysierra]

@conorclifford Can you outline why you need this?

[conorclifford]

@whiskeysierra Our particular use cases around this are for both (1) audit trail and also (2) context-specific authorisation:

Audit trail

To allow Nakadi to be used as a GPP data ingestion API, where data providers can simply stream data to our services, it is important to be able to track the origin of the various pieces of data back to the originating identity that was used when producing to Nakadi.

Context Specific Authorization

We have several applications that have extra authorization beyond simple "scope" based limitations - whereby, only certain clients are allowed provide certain types of data (where many different clients will be allowed provide data for these services in general, different subsets of the clients will have different abilities depending on the context of the data they are providing.
To be able to use Nakadi as an asynchronous ingestion mechanism for these applications, the originating identity of the producer is important.

[hjacobs]

We have another use case for this where we want to have a generic infrastructure for "application" events (e.g. deployments) --- the producer OAuth token already contains the necessary information and mapping it to an application ID in the event metadata would ensure that the source of the application event is always correctly identified.