KeyboardEvent properties don't match real browsers

[Amphitryon0]

Operating System Info

Windows 10

Other OS

No response

OBS Studio Version

27.2.4

OBS Studio Version (Other)

No response

OBS Studio Log URL

https://obsproject.com/logs/1RjIPQfX0BqFKJ98

OBS Studio Crash Log URL

No response

Expected Behavior

The KeyboardEvent generated should be similar to that originating in a regular browser. For example, using a standard US layout, typing a colon with the right shift key should generate two events with (among others) the following properties:

.code	.key	.location
"ShiftRight"	"Shift"	2
"Semicolon"	":"	0

Current Behavior

Using the same example of typing a colon, I see the following properties:

.code	.key	.location
""	"Shift"	0
""	":"	0

From this and other tests, it seems that code is always blank, and location is always 0.

Steps to Reproduce

1. Create a browser source with the URL set to https://domeventviewer.com/key-event-viewer.html. Leave the other settings unchanged.
2. Interact with the source and press a key.
3. Observe incorrect properties.

Anything else we should know?

I saw that there have been similar issues closed, but I'm not sure if they were considered fixed or just as close as we could get. If it's the latter case, I'm sorry for being yet another person to bring this up. The webpage I'm actually using can be modified to use OBS's key values when it detects that it's running in OBS, but then it's still locked to one keyboard layout.

[ElectronicWar]

We are more or less at the mercy of CEF here and how it handles these things. Please mind that on Windows, our CEF is quite old (equivalent of Chrome 75) and this alone might make the difference.

You can track progress in updating to a newer CEF release here: obsproject/obs-studio#3853

[WizardCM]

This is something I've been trying my best to try and fix over time. a14ed2fe attempts to get it relatively functional on macOS, ae25c36a attempted to fix it on Linux, and I've always tested it on Windows.

I also use keycode.info as a smoke test, but I can confirm modifiers are a bit of a nightmare. I admit that I hadn't been testing colon vs semicolon.

It's definitely an area I want to improve further - the challenge is that we go through 3 layers of conversion: first Qt reads it, it gets converted into an obs_key_t, and then we convert that to something CEF approves of.

Additionally, the OBS Interact window doesn't handle held keys or related events - that's the same reason drag & drop cursor events don't work properly.

I'll continue experimenting with this when I have the motivation. Honestly this particular OBS API is a bit of a nightmare to work with.

[WizardCM]

Edit: Oh god did my recent changes break #236 again? I hope not.

[WizardCM]

OK, after further testing, recent code did not break this. What I've actually learned is that the event->text from OBS to CEF is correct, but by the time it reaches the webpage it's wrong. I'm not quite sure why.

Additionally, the good news is that the "modifiers" match on OBS and CEF, so that particular aspect we don't need to worry about.

Top: Browser Dock
Bottom: Browser Source

[Amphitryon0]

I updated the report to indicate what occurs on 27.2.4 (which is almost the same), and I updated the steps to reproduce since keycode.info is no longer accurate. One other thing I noticed (though you probably were aware of it) is that interacting with a browser source through remote debugging gives mostly correct property values.

[Amphitryon0]

Also, the modifiers do not match exactly. The modifier states for Shift, Ctrl, and Alt (and their specific properties) are false on keydown and true on keyup of their respective keys. The wording of the spec isn't super clear about that imo, but its examples show them as true on keydown and false on keyup, which is also what happens in remote debugging. The Meta modifier and its property, as well as Caps Lock, Scroll Lock, and Num Lock, are always false. (Meta and its property can be true for events from remote debugging, but the other modifiers will not.)

[pkviet]

This seems related to this CEF bug:
chromiumembedded/cef#2597