Make PopupMenu/Panel shadows properly visible again

[YeldhamDev]

This PR does the following:

• Make PopupMenu/Panel nodes have spacing around them to show the shadows of their theming, as well as repositioning them accordingly with their shadows' size when popping up.
• Make PopupMenu/Panel nodes be transparent by default. Not only this is necessary for the shadows, but also for rounded corners. It slightly breaks compatibility, as a default value is changed. But I think it's non-intrusive enough to be acceptable.
• Fix some bugs related to content scaling, and submenu positioning.

Closes godotengine/godot-proposals#7532.
Closes godotengine/godot-proposals#8738.

Test Project: shadow-of-the-popups.zip

[Calinou]

I'd say this also closes godotengine/godot-proposals#8738 (by superseding it).

[passivestar]

Doesn't look right with the default theme on mac 🙃

[YeldhamDev]

@passivestar Could you enable display/window/per_pixel_transparency/allowed in the project settings?

[passivestar]

@passivestar Could you enable display/window/per_pixel_transparency/allowed in the project settings?

yeah that works

[YeldhamDev]

I've added a warning that appears when a PopupMenu/Panel has a theme that needs per pixel transparency but it isn't enabled.

Note that currently it always appears when running the game because the default project theme has rounded corners. If this is merged, it will need to be modified.

[bruvzg]

We probably can enable transparency for the editor by default, it may increase VRAM usage a bit, but should not have any significant performance impact on modern systems.

Also, I'll take a look if we can test if transparency is really supported and working (checks if compositor is active and surface/front buffer have correct format), and will add a method to the display server.

[YeldhamDev]

@bruvzg Nice! But I think having the warning is a good thing nonetheless.

[passivestar]

Themed the menu to take advantage of this PR, makes a huge difference imo

As I understand transparency needs to be an editor setting because we probably don't want a project setting to affect editor theming. Even better if enabled by default like @bruvzg suggested

P.S. Clicking on the popup menu shadow doesn't close the popup which is a bit annoying with a large shadow

[YeldhamDev]

P.S. Clicking on the popup menu shadow doesn't close the popup which is a bit annoying with a large shadow

Got you covered. 😉

[YeldhamDev]

Updated to use the new is_window_transparency_available() function.

[passivestar]

Still needs transparency to be enabled in the project settings to work but also it looks like things broke since the last time I tested this PR:

• Popup menus now have scrollbars
• Submenu is drawn at a weird distance
• When shadow is enabled it's only drawn in corners inside of the bounding box of the menu

No shadow	Shadow

Same with the default theme:

Got you covered. 😉

Can't test if clicking on shadows to close the menu is fixed because there's no shadow to click :)

[YeldhamDev]

@passivestar Indeed, it seems that this PR rotted after a while. Sadly, I don't have any time to work on this at all currently.

[YeldhamDev]

@passivestar Welp, I'm a masochist, so I worked on it anyways. Please test again.

[passivestar]

Works great on my end, can confirm that clicking on shadows closes popups now! Can't wait for this to be merged

[Geometror]

Unfortunately, I can't get this to work for me. (tested on Windows 11 and Linux[Wayland])

Doesn't matter whether the popup is embedded or native, in the 2D editor or running project.

Test project:
mrp1_popup_shadows.zip

[YeldhamDev]

@Geometror Enable display/window/per_pixel_transparency/allowed in the Project Settings, the shadows shall appear in the running game and in the editor when you restart. A warning about it appears when you run without the setting being enabled.

[Geometror]

I actually tested with that setting enabled and disabled. Transparency wasn't the problem, it's that the shadow is cropped.

[YeldhamDev]

How does the shadows look in your end in with my test project?

[Geometror]

Well, they are not there :)

[YeldhamDev]

Are you sure you didn't change anything? I'm on GNU/Linux (Wayland) and the shadows work fine. Maybe try rebasing it to master in your end?

[passivestar]

The test project on my end (transparency is not enabled in the project so I had to enable it):

[Geometror]

Ok, I finally got it to work. Sorry, I assumed this project setting was turned on in the test project :)
However, this only works for me when using the compatibility backend (both Linux (Wayland) and Windows 11), despite per pixel transparency being supported (#91333 (comment))
So it looks like if DisplayServer::get_singleton()->is_window_transparency_available() doesn't determine correctly whether it is available.

[YeldhamDev]

@Geometror Huh, all backends work for me.

Anyways, CC @bruvzg, since you were the one who implemented it.

[bruvzg]

Should also work on macOS (with any renderer) and on Windows with D3D 12, but Vulkan depend on the driver and NVIDIA broke it some time ago.

[Geometror]

Alright, haven't looked at the code yet, but functionality-wise this it's almost ready - just one thing left:
This should work regardless of the backend when subwindows are embedded (plus, it should also work in the 2D editor, but I guess this comes for free when it works for embedded subwindows)

Related to this PR, but not required/just for reference:

• DisplayServer::get_singleton()->is_window_transparency_available() doesn't report per-pixel transparency for Wayland, although it is supported (in my case) as you can see in my screenshot
• Per-pixel transparency doesn't work for me on Windows 11 both with DX12 and Vulkan (single GPU system, 3060 Ti), which is kind of strange (at least when using DX12)

[YeldhamDev]

@Geometror Changes made.