NFR - Log all security relevant events in an Audit Log

[DanielaWuensch]

Feature Request

Non-functional requirement - Log all security relevant events in an Audit Log
As a CX partner I want to see all administrative activities as well as potential malicious end user activities in a dedicated audit log.
Information to be logged:

• Unique Identifier for the event
• UserId
• timestamp
• IP (in case of external access)
• Description to understand the event
• Access to the audit log should be restricted to by authorizations. Records should be immutable. Do not log secrets.

Based on this discussion: #1381 Audit log for API Access is required, Contract Negotiation and Transfer Process (missing: pre-actions; post-actions are covered by event framework already)

Which Areas Would Be Affected?

all, including DPF, CI, build, transfer, etc._

Why Is the Feature Desired?

Security Requirement

Solution Proposal

Implement a dedicated audit log

Type of Issue

non-functional requirement

Checklist

Audit Log available

[lholthof]

My current understanding is that the event framework was not designed for audit logging and hence it offers just post-action methods to implement some log. In addition it is not available on Data Mgt API so far and I do not know the plans. I am wondering if this is a fit at all.

Alternative approach to using the event framework would be to have another service similar to the Monitor which is explicitly meant to write audit logs and put these log statements just in the code where it is required.

In addition I could also think of reusing the Monitor and introduce a special log level "audit" for enabling audit logging in edc.

Was there so far any discussion around this besides #1381 in the EDC community?

[jimmarino]

Can you please outline the use cases and semantics of what "audit logging" is? It can mean many different things.

[lholthof]

As already stated, the expectation to an audit log would be that it contains logs of all administrative activities as well as potential malicious end user activities.

To make it more specific I currently see the following activities to be logged with the details stated in the initial description (acknowledging ISO/IEC 27002):

• Changing requests (Create/Update/Delete) to APIs /assets, /policies, /contractdefinition
• Contract Negotiation results
• Transfer Process (intial state and complete state)

[jimmarino]

Thanks. I read the original discussion, which does not outline use cases, requirements, or semantics at the level needed. That's fine because it's a discussion, but to arrive at actionable issues, let's set aside the question of how potential features should be implemented and agree on how to approach this.

The three areas you identified above (Data Management, Contract Negotiation, and Data Transfer) are very different. For example, creating an Asset can be initiated by a user that may potentially be represented as a subject (a concept we don't currently have in EDC). Transitioning a transfer process to the PROVISIONING state is done by the Connector, not an "end-user" subject.

I recommend starting with requirements for auditing data management operations and picking one or two operations to drill down in detail. For example:

As a data security officer, I need to determine who created every asset in the system in a guaranteed way.

Or, a different version:

As a data security officer, I want to determine who created an asset.

One version has reliability guarantees. Both versions assume the notion of a Subject that EDC tracks. That would seem to imply we first need to examine introducing the notion of a security context and Subject (and potentially permissions) to some of the EDC subsystems.

Expanding on this, the EDC core would likely not supply all the pieces required to fulfill the above requirement. For example, the EDC may only ensure that an event is dispatched to an EventSubscriber registered with the EventRouter. The subscriber implementation (provided by the EDC deployer) would be responsible for sending the event to a grid, durable storage, or another system that the data security officer can access.

Once we have a clear picture of the requirements, a committer will write one or more Decision Records to address any architectural impact and specify the overall technical direction. My initial guess is that we will need to address the notion of a security context first.

FWIW, I think the event system with a few minor enhancements will play a role in this but let's not discuss implementation details until we have a clear picture of the requirements.

Does that make sense as an approach?

[domreuter]

I think we don't need to outline any specific use-cases or requirements at this point. It is not about defining and implementing any specific audit trails for special security officer needs rather than starting with a simple audit log in generell. The different trails can be extracted from the audit log on-demand. (API-Trail, Asset-Trail, Connector2Connector-Communication-Trail)

In my opinion we should work out a generell instrument to implement this, like it is already there for the monitoring component. It should be extensible to different implementations, but to get started a filesystem or std logging is a good thing to ramp it up.

I would suggest to start with the datamanagement-api as the audit log relevant parts can be identified easily there. As soon as we have some samples by hand it will be hopefully clearer to everyone.

[jimmarino]

Implementing without clear requirements or a technical plan is exactly the approach we want to avoid. Please review how our project contribution process works and, in particular, Decision Records.

We should not introduce another interface similar to Monitor. As I mentioned previously, "audting" can likely be handled by the existing EventRouter. I would suggest you take a closer look at it and the EDC architecture in general. However, before discussing implementation details, we need to have a clear definition of the requirements at hand.