Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Integrate with Window Segmentation API #35

Closed
kenchris opened this issue Sep 15, 2020 · 5 comments
Closed

Integrate with Window Segmentation API #35

kenchris opened this issue Sep 15, 2020 · 5 comments
Assignees
Labels
documentation Improvements or additions to documentation enhancement New feature or request question Further information is requested

Comments

@kenchris
Copy link

kenchris commented Sep 15, 2020

Some internal displays may be foldable displays or made up of multiple physical displays - though be considered one display by the OS, like on the Surface Duo. The Window Segments API (should really be called Screen Seg.) allows getting the segments for these displays.

I think it would be nice with a property like hasSegments and a getter to get the segments for a given display.

As on macOS you can use an iPad as external display, you could imagine using some of these devices as external displays as well in the near future, so this doesn't just cover internal/primary displays

https://github.com/webscreens/window-segments

@darktears @zouhir

@michaelwasserman
Copy link
Member

michaelwasserman commented Sep 16, 2020

Thanks for the feedback; see also #21. Let's seek compatibility between proposals and support forthcoming screen paradigms!
Q: Do we know how dual/foldable/etc. screen hardware will be exposed by OS/WMs? That may inform some API shapes.
Q: Should we plan to discuss plans/compatibility as a TPAC Second Screen CG topic?

IIUC, the Window Segments Enumeration API exposes bounds for each content region within a single window that spans multiple displays/Screens. If a single Screen may contain one or more folds, their bounds or the bounds of the 'screen segments' delineated by those folds should be exposed on Screen (or on Fold?), beyond the proposed window.getWindowSegments() interface. I filed webscreens/window-segments#7

IIUC, the Screen Fold API exposes the angle and orientation of a Fold in a Screen. This seems to indicate that a foldable screen is exposed as a single Screen interface object, but also notes relevance of "devices with a single flexible screen (seamless), and devices with two screens (with seam)". I'd like to better understand how this API might handle (1) folds on a seam between two Screens/displays, (2) off-center folds, and (3) multiple folds in a single Screen. I filed w3c/screen-fold#38

Here are initial high-level thoughts on how these APIs might coordinate with Screen interface extensions and related proposals:

  1. If new information/interfaces are exposed through the Screen interface, a multi-screen API should support that too.
    a. If Screen gains fold or HDR/WCG attributes, multi-screen API objects should include those attributes.
    b. If Screen gains fold(s), they should include bounds, or Screen should expose fold-delineated screen segment bounds.
  2. If the web should expose folds between multiple separate Screens, a multi-screen API should support that too.
    a. Perhaps the Fold interface should expose its relationship to a single Screen or multiple Screens?
    b. Perhaps the Screen interface should expose multiple Fold interface objects?
    c. Perhaps the Fold interface objects should be vended outside the Screen interface?

I hope proposal owners will consider my questions/feedback, prompts for coordination, and suggested TPAC discussion!

@michaelwasserman michaelwasserman self-assigned this Sep 16, 2020
@kenchris
Copy link
Author

@anssiko can we get this on the agenda?

@anssiko
Copy link
Member

anssiko commented Sep 16, 2020

Good discussion topic. Add a comment to w3c/secondscreen-wg#1

@michaelwasserman
Copy link
Member

One potential concern for this proposal's shape is the singular Screens.currentScreen. If Windows will regularly span multiple Screens, it should be easy to determine which Screens (plural) the Window is shown on.

@michaelwasserman michaelwasserman added documentation Improvements or additions to documentation enhancement New feature or request question Further information is requested labels Oct 29, 2021
@michaelwasserman
Copy link
Member

The relationship between these APIs has been explored to some degree with a Form Factors Explained CG report. The Device Posture API picks up where the Screen Fold API left off and Window Segments Enumeration is now a segments attribute of the Visual Viewport API.

It remains to be seen how the OS and WM semantics of emerging and future form factors relate to the concept of a Screen in the web platform. I think there are viable backwards-compatible approaches to supply information about multiple-current-screens, the segments within individual Screens, the global coordinates in such a space where a window should be placed, or even the desired set of screens over which a content fullscreen session might span.

See our API Design Risks doc for more thoughts in this vein, but I believe the current API shape is a responsible incremental step towards supporting web applications making opinionated use of generalized multi-screen environments on the web.

I'm going to close this issue until further specific issues are identified.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
documentation Improvements or additions to documentation enhancement New feature or request question Further information is requested
Projects
None yet
Development

No branches or pull requests

3 participants