Visibility of changes to targets created from EGLImages is underspecified

[lynxeye-dev]

EGL specifies that modifications to one sibling of a EGLImage are visible to all other siblings:

If an application specifies an EGLImage sibling as the destination for rendering and/or pixel download operations (e.g., as an OpenGL or OpenGL ES framebuffer object, glTexSubImage2D, etc.), the modified image results will be observed by all EGLImage siblings in all client API contexts.

However, I can't find any language on when a modification to one target must become visible
to other targets.

E.g. a texture object shared between two GL contexts that aren't in the same share group via an EGL image export/import. While we can serialize access to the resource via the known EGL synchronization primitives, we can still effectively create the situation that is solved by glTextureBarrier by having a target bound for rendering in one context and have another target bound for sampling in the other context. Do we need to issue a texture barrier after the rendering command in one context and another texture barrier before sampling in the other context to make sure that render/sampler caches are flushed/invalidated so the changes become visible?

OES_EGL_image_external explicitly includes language to specify that modifications to the texture object must become visible at (re-)bind time:
Sampling an external texture which has been modified since it was bound will return samples which may correspond to image values either before, during, or after the modification. Binding (or re-binding if already bound) an external texture by calling BindTexture after all modifications are complete guarantees that sampling done in future draw calls will return values corresponding to the values in the buffer at or after the time that BindTexture is called.

There is no such language for non-external textures created via OES_EGL_image. So there is some ambiguity on when changes must be visible. Are gltextureBarrier and/or glMemoryBarrier the right primitives to enforce visibility of modifications, or is there any other way? What about GL/GLES contexts that don't implement those primitives?

[pdaniell-nv]

@stonesthrow Any thoughts on this?

[stonesthrow]

I don't think some of the language got into the EGL specification from the original extensions. Check the original extension specs:
https://registry.khronos.org/EGL/extensions/KHR/EGL_KHR_gl_image.txt
https://registry.khronos.org/OpenGL/extensions/OES/OES_EGL_image.txt
https://registry.khronos.org/OpenGL/extensions/OES/OES_EGL_image_external.txt

With images, no copies of the buffer are made (unless re-specification) so its all sharing the same buffer. So anything that draws to it is semi-immediate, however, a synchronization/fence should be used by application/user to ensure the draw is complete, there is no inherent flush, before underlying buffer is used somewhere else. Although Piers can confirm on rebinding that a flush is complete. In your scenario, a fence should be inserted after rendering and waited by the sampling context.

[cubanismo]

@stonesthrow, by fence to you mean an EGLSync as specified in EGL_KHR_fence_sync/EGL 1.5? From the KHR spec, speaking in regards to fence sync objects:

The sync object will not be signaled until all effects from these commands on the client API's internal and framebuffer state are fully realized.

Which I believe is trying to say the effects are visible, so I would agree with your interpretation.

Texture and memory barrier commands do not seem relevant from my interpretation of their definition. They merely order specific types of memory accesses relevant to each other within a given context. That may have the desired side effects on an EGLImage content's visibility external to that context, but it doesn't seem guaranteed. Fences are the only thing that explicitly operate across context boundaries from what I can tell.

Side note: There still doesn't seem to be a correct way to express support for binding an EGLImage to a texture in OpenGL contexts. The OES extension technically only applies to GLES contexts. EGL_KHR_fence_sync used to have the same gap, but it was resolved when it was promoted to EGL 1.5.

[lynxeye-dev]

The sync object will not be signaled until all effects from these commands on the client API's internal and framebuffer state are fully realized.

Okay, so this seems to take care about the side doing the modifications to the image. Not sure if all drivers implement this properly in all cases, but at least that's a interpretation of the sync object semantics that one can easily get behind and treat any missing cache flushes as driver bugs.

But what about the side consuming the modified image? Does waiting for a sync object also imply that changes to resources currently bound in the waiting context are visible at that point? E.g. does a successful eglClientWaitSync guarantee that the modified data in the image is visible to the sampler of a currently bound texture specified from the EGL image?
I guess the answer would be yes. But then this should be made clear in the specification, as at least the Mesa drivers currently implement the waits as simple waits for the signalling of the sync object. There is no cache invalidation or any other driver action hooked up, which seems to be a valid interpretation of the current spec language about waiting for sync objects.

Also what about use-cases where the EGL images are created from resources where no EGL sync object is available to be waited on, e.g. the EGL image being constructed from a Linux dma-buf? How should changes to the underlying data be communicated to the consuming context in that case?

[stonesthrow]

Yes, use eglClientWitSync for consuming side. I've never seen cache operations with this, but it would have to be a known property of the buffer that is used to create the EGLImage. EGL will have to know that and if the buffer has any associated reference counting/wonership mechanism.

dmabufs are used to create EGLImages. the EGL fence sync will work with that.

[lynxeye-dev]

I've never seen cache operations with this, but it would have to be a known property of the buffer that is used to create the EGLImage. EGL will have to know that and if the buffer has any associated reference counting/wonership mechanism.

Not sure if we are talking past each other, so just to make it clear: I'm not talking about CPU side caches, but specifically GPU caches. In the case where producer and consumer are in the same context glTextureBarrier is roughly what I'm trying to get at: it's flushing the render cache after the draw and invalidates sampler caches before the next draw.

I see that flushing the render cache before signalling the sync object created with eglCreateSync and invalidating the sampler caches on eglClientWaitSync or eglWaitSync would achieve the same thing across contexts. While that behaviour makes sense to me, as it allows the required data coherency to be achieved cross context with only the primitives available in the base EGL spec, it's not how many drivers interpret the spec today, so I think it warrants clarification.

dmabufs are used to create EGLImages. the EGL fence sync will work with that.

That will work fine when the producer is able to provide a EGL sync object. However dma-bufs may be produced/modified by APIs that aren't able to provide a EGL sync object, like V4L2 capture devices or even the wayland winsys with zwp_linux_dmabuf, which by default only provides dma-bufs, but no sync objects.

[stonesthrow]

The fence is to ensure that rendering is complete and that all pixels are flushed to the buffer.
glTextureBarrier is GL function that I am not that familiar with, I can't speak to its internal implementation.

If your producer is non-EGL, such as platform API, then use a platform synchronization primitive that you signal when the software or other platform rendering is complete. On the consumer side, if the platform sync is not a type that is importable to a EGLSync object, then your application, that is driving the consumer, must wait on the platform sync before proceeding with consuming/sampling the buffer.
If I am misleading on anything, hopefully Cubanismo will correct me.

[lynxeye-dev]

Right, there are other means than a EGL sync object to wait for the platform sync to be complete.

However this only covers the producer side: waiting for the platform sync only ensures that the modification to the image is complete. It doesn't cover the consumer side, where the the application needs some means to tell the consuming context that the data in the EGL image might have changed and it needs to invalidate caches. Waiting for a EGL sync object might be those means when the spec language is clarified that way, but that would still not cover the case where the platform sync can't be imported into a EGL sync object, as then there is nothing to eglWaitSync on in the consuming context.

[stonesthrow]

There is a little bit of text in the GLES and GL specifications about cache and barriers. I'll direct you there for those specifics.

If EGLSync is not usable, then you will nee to use something like a semaphore between producer and consumer. Or I should say that software that drives the producer and that software that drives the consumer - application or middleware or whatever. Since its not importable to the EGL/GL driver.

[cubanismo]

If you're not using EGLSync, presumably with a dma_fence sync via EGL_ANDROID_native_fence_sync, at least on Linux you're relying on implicit sync. That's not great IMHO, but in that case it's up to implicit sync to perform equivalent synchronization to EGLSync implicitly based on, I would assume, submission order.

I wouldn't think eglClientWaitSync() is sufficient to make changes visible on the GPU. It performs a CPU-side wait and does not require a current context, so I wouldn't expect it to invalidate any GPU/GL server-side caches, as those caches may be managed relative to a context. My reading is you'd have to use eglWaitSync().

[lynxeye-dev]

Oh right, eglClientWaitSync() not requiring any active context and thus being unable to invalidate any caches is a little detail I missed to consider. This seems to be quite crucial, as with the current spec language one could easily come to the conclusion that eglWaitSync() is strictly an optimization over eglClientWaitSync() allowing more parallelism, while it seems to be required for correctness with resources shared via EGL images.

Also it seems that many drivers implement eglWaitSync() as a server side CPU wait or simply insert a dependency in the commands, but don't do any cache handling. I think this warrants some clarification in the EGL spec.

[stonesthrow]

@pdaniell-nv , I will follow up with any suggestions to make the spec text more clear for users. So this issue, which I think should be EGL, just needs to discuss any new or improved text to merge and close.
Lucas and James, after discussion, suggestions - seems the primary point is
"language on when a modification to one target must become visible to other targets."

[stonesthrow]

Section 3.9.1 "lidetime and Usage of EGLImages". Need to add paragraph about timing of updates/writes such that changes are visible.