There may be various reasons that an application could need access to the
underlying NSWindow that is being used to display the GdkMacosSurface
contents. This provides a minimal API to do that without exposing our
implementation details through public API.
As our rendering system is likely to change over time, we very much want
to keep GdkMacosView, GdkMacosLayer, GdkMacosTile, and GdkMacosWindow all
private implementation details which are subject to change.
As this is public API, we are a bit long-winded with the name so it is
clear what is being accessed without polluting symbol names with things
like "ns" as we used to.
When using server-side-decorations, we need to avoid potential cycles with
compute-size as it may not have the new sizing information yet. We can
just short circuit during "live resize" to get that effect.
Fixes poor window resizing from top-left on titled windows.
This doesn't give us appropriate results if we use the window delegate.
Instead, we need to adjust the frame at the same time we change the
style mask so that we end up in the same location.
Previously we had issues on macos where the overshoot would keep showing.
To fix this we need to actually use discrete events instead of the
generated deltas from macOS in the scroll wheel case. Additionally, we need
to drop the kinetic momentum events from macOS and rely on the gtk kinetic
events which are already happening anyway. We also need to submit the
is_stop event for the GDK_SCROLL_SMOOTH case when we detect it.
To keep the discrete scroll events correct, we need to alter the hack in
gtkscrolledwindow.c to use the same path as other platforms except for
when a smooth scroll event is in place. In the future, I would imagine that
this falls into the boundary of high-precision scrolling and would share
the same code paths as other platforms.
With all of these in place, kinetic scrolling with overshoot appears the
same on macOS as other platforms.
When creating new windows, it is better if we create them with a slight
offset to where they were created before so that they are visible to the
user separately from what they might be overshadowing.
This broke with the previous fixes for initial window positioning. We need
the initial positioning so that tails will be displayed correctly when the
popover surface is displayed.
If the size changes, we need to relayout the tiles. Otherwise we can keep
using what we had before. Generally, that shouldn't happen, but the
previous check was failing in a number of ways.
It looks like, particularly on the M1, we might need to double buffer the
contents of the IOSurface<->OpenGL texture bindings. This doesn't appear
to show up on the Intel macbooks I've tried, but I've seen it in the wild
on an M1.
If we have a 2x scale laptop with a 1x scale external display, we would
need to create a new IOSurface for the external display once it crosses
a boundary, otherwise we won't have something capable of displaying
correctly on the second monitor.
This provides a major shift in how we draw both when accelerated OpenGL
as well as software rendering with Cairo. In short, it uses tiles of Core
Animation's CALayer to display contents from an OpenGL or Cairo rendering
so that the window can provide partial damage updates. Partial damage is
not generally available when using OpenGL as the whole buffer is flipped
even if you only submitted a small change using a scissor rect.
Thankfully, this speeds up Cairo rendering a bit too by using IOSurface to
upload contents to the display server. We use the tiling system we do for
OpenGL which reduces overall complexity and differences between them.
A New Buffer
============
GdkMacosBuffer is a wrapper around an IOSurfaceRef. The term buffer was
used because 1) surface is already used and 2) it loosely maps to a
front/back buffer semantic.
However, it appears that IOSurfaceRef contents are being retained in
some fashion (likely in the compositor result) so we can update the same
IOSurfaceRef without flipping as long as we're fast. This appears to be
what Chromium does as well, but Firefox uses two IOSurfaceRef and flips
between them. We would like to avoid two surfaces because it doubles the
GPU VRAM requirements of the application.
Changes to Windows
==================
Previously, the NSWindow would dynamically change between different
types of NSView based on the renderer being used. This is no longer
necessary as we just have a single NSView type, GdkMacosView, which
inherits from GdkMacosBaseView just to keep the tedius stuff separate
from the machinery of GdkMacosView. We can merge those someday if we
are okay with that.
Changes to Views
================
GdkMacosCairoView, GdkMacosCairoSubView, GdkMacosGLView have all been
removed and replaced with GdkMacosView. This new view has a single
CALayer (GdkMacosLayer) attached to it which itself has sublayers.
The contents of the CALayer is populated with an IOSurfaceRef which
we allocated with the GdkMacosSurface. The surface is replaced when
the NSWindow resizes.
Changes to Layers
=================
We now have a dedicated GdkMacosLayer which contains sublayers of
GdkMacosTile. The tile has a maximum size of 128x128 pixels in device
units.
The GdkMacosTile is partitioned by splitting both the transparent
region (window bounds minus opaque area) and then by splitting the
opaque area.
A tile has either translucent contents (and therefore is not opaque) or
has opaque contents (and therefore is opaque). An opaque tile never
contains transparent contents. As such, the opaque tiles contain a black
background so that Core Animation will consider the tile's bounds as
opaque. This can be verified with "Quartz Debug -> Show opaque regions".
Changes to Cairo
================
GTK 4 cannot currently use cairo-quartz because of how CSS borders are
rendered. It simply causes errors in the cairo_quartz_surface_t backend.
Since we are restricted to using cairo_image_surface_t (which happens to
be faster anyway) we can use the IOSurfaceBaseAddress() to obtain a
mapping of the IOSurfaceRef in user-space. It always uses BGRA 32-bit
with alpha channel even if we will discard the alpha channel as that is
necessary to hit the fast paths in other parts of the platform. Note
that while Cairo says CAIRO_FORMAT_ARGB32, it is really 32-bit BGRA on
little-endian as we expect.
OpenGL will render flipped (Quartz Native Co-ordinates) while Cairo
renders with 0,O in the top-left. We could use cairo_translate() and
cairo_scale() to reverse this, but it looks like some cairo things may
not look quite as right if we do so. To reduce the chances of one-off
bugs this continues to draw as Cairo would normally, but instead uses
an CGAffineTransform in the tiles and some CGRect translation when
swapping buffers to get the same effect.
Changes to OpenGL
=================
To simplify things, removal of all NSOpenGL* related components have
been removed and we strictly use the Core GL (CGL*) API. This probably
should have been done long ago anyay.
Most examples found in the browsers to use IOSurfaceRef with OpenGL are
using Legacy GL and there is still work underway to make this fit in
with the rest of how the GSK GL renderer works.
Since IOSurfaceRef bound to a texture/framebuffer will not have a
default framebuffer ID of 0, we needed to add a default framebuffer id
to the GdkGLContext. GskGLRenderer can use this to setup the command
queue in such a way that our IOSurface destination has been
glBindFramebuffer() as if it were the default drawable.
This stuff is pretty slight-of-hand, so where things are and what needs
flushing when and where has been a bit of an experiment to see what
actually works to get synchronization across subsystems.
Efficient Damages
=================
After we draw with Cairo, we unlock the IOSurfaceRef and the contents
are uploaded to the GPU. To make the contents visible to the app,
we must clear the tiles contents with `layer.contents=nil;` and then
re-apply the IOSurfaceRef. Since the buffer has likely not changed, we
only do this if the tile overlaps the damage region.
This gives the effect of having more tightly controlled damage regions
even though updating the layer would damage be the whole window (as it
is with OpenGL/Metal today with the exception of scissor-rect).
This too can be verified usign "Quartz Debug -> Flash screen udpates".
Frame Synchronized Resize
=========================
In GTK 4, we have the ability to perform sizing changes from compute-size
during the layout phase. Since the macOS backend already tracks window
resizes manually, we can avoid doing the setFrame: immediately and instead
do it within the frame clock's layout phase.
Doing so gives us vastly better resize experience as we're more likely to
get the size-change and updated-contents in the same frame on screen. It
makes things feel "connected" in a way they weren't before.
Some additional effort to tweak gravity during the process is also
necessary but we were already doing that in the GTK 4 backend.
Backporting
===========
The design here has made an attempt to make it possible to backport by
keeping GdkMacosBuffer, GdkMacosLayer, and GdkMacosTile fairly
independent. There may be an opportunity to integrate this into GTK 3's
quartz backend with a fair bit of work. Doing so could improve the
situation for applications which are damage-rich such as The GIMP.
Previously, the popover would cause the window to go into the :backdrop
state which is not what we want for consistency with other platforms. This
fixes that by walking up the surface chain when we get notified of
loosing or acquiring "key" input from the display server.
We might have panels with controls in them where the window is running in
another process. The control could have a wrapper window which we would
see from this process. This can happen with the GtkFileChooserNative, but
any NSSavePanel in macOS 10.15+ is out of process (not just sandboxed
applications).
This significantly cleans up how we handle various move-resize, compute-
size, and configure (notification of changes) in the macOS GDK backend.
Originally when prototyping this backend, there were some bits that came
over from the quartz backend and some bits which did not. It got confusing
and so this makes an attempt to knock down all that technical debt.
It is much simpler now in that the GdkMacosSurface makes requests of the
GdkMacosWindow, and the GdkMacosWindow notifies the GdkMacosSurface of
changes that happen.
User resizes are delayed until the next compute-size so that we are much
closer to the layout phase, reducing chances for in-between frames.
This also improves the situation of leaving maximized state so that a
grab and drag feels like you'd expect on other platforms.
I removed the opacity hack we had in before, because that is all coming
out anyway and it's a bit obnoxious to maintain through the async flows
here.
This fixes GTK's NSWindow for toplevels so that they are allowed to enter
fullscreen. We were already handlign the state transitions from the
setStyleMask: halper, but we didn't previously tell the window we are
allowed to transition into that.
There is a bit of a mismatch here in that GTK doesn't have any such flag
that determines if a window is "allowed" by policy to enter fullscreen
since window managers on Linux are free to do that at will.
We want our tracking area to be limited to the input region so that we
don't pass along events outside of them for the window. This improves the
chances we click-out of a popover with a large shadow.
This still doesn't let us pass-through clicks for large shadows on top-
level windows though.
We only should be asserting in static functions. Furthermore, this function
did not need to have GDK_BEGIN_MACOS_ALLOC_POOL as nothing is being
allocated there which would cause pooling to get used.
This needs to handle the boundary case where the value is exactly equal
to the edge of a rectangle (which gdk_rectangle_contains_point() does not
consider to be containing). However, if there is a monitor in the list
that is a better match, we still want to prefer it.
When using an external mouse on MacOS, the scrolling behavior is
reversed from the user's scrolling preference. Additionally, it is
noticeably sluggish.
This commit fixes both issues by negating the deltas and multiplying
them by 32 before constructing a new scroll event. 32 seems to be the
"traditional" scaling factor according to [Druid], but I'm not sure
where that value actually comes from. Regardless, scaling the deltas by
this amount makes scrolling feel a lot more responsive in the GTK demos.
Scrolling with a trackpad is not affected by either issue because it
triggers a different code path that uses more precise deltas, and
already negates them.
[Druid]: https://linebender.gitbook.io/linebender-graphics-wiki/mouse-wheel#external-mouse-wheel-vs-trackpad
We don't want to risk having something really weird come out if we have a
WCG colorspace, so instead only do the performance hack on systems where
the output is likely reasonable.
We will want to eventually just be drawing in the appropriate colorspace,
but that is not available yet.
When using software rendering w/ cairo, assume we're drawing in
the best-monitor's colorspace rather than RGB to avoid colorspace
conversions on every frame.
Instead of relying on cairo_t to perform drawing from our backing
image surface to the Core Graphics context, we can convert the
cairo_image_surface_t into a CGImageRef without having to copy
data if we are certain of the alignment of the image up front.
Without this, there are many situations, based on the size of the
window that could cause cairo to take a slow path and malloc/copy
the data to ensure that alignment.
The previous commit titled "macos: align image surface rowstride to
16-bytes" ensures that this invariant is true so that our drawing
code can assume we can reference the framebuffer from the
cairo_image_surface_t using a CGDataProvider.
Since GdkMacosCairoContext and GdkMacosCairoSubview are coordinating,
we can also setup the transformation/scale early when drawing the
cairo_image_surface_t instead of when copying it to Core Graphics.
Furthermore, the CGImageRef is created with an RGB colorspace so
that we are not performing colorspace conversion to the output
device. We don't get color matching between displays, but we don't
expect that anyway, particularly with the software renderer.
When creating a cairo_image_surface_t we want both the framebuffer pointer
and each row to be aligned to 16-bytes so that Core Graphics will use more
optimal paths.
However, cairo_image_surface_create() will not guarantee that the rowstride
is aligned to 16-bytes so we must do that ourselves.
We need to avoid conflating the managing of frame callbacks from
the freeze/thaw mechanics and ensure we don't perform extra thaw
requests at the wrong time.
It makes sense to connect the begin/update/end events
for touchpad swipes and pinches in a sequence. This
commit adds the plumbing for it, but not backends
are setting sequences yet.
This is a port of the fix in the quartz backend to the new macOS backend.
From the original commit:
In macOS-12.sdk CGContextConverSizeToDeviceSpace returns a negative
height and passing that to CGContextScaleCTM in turn causes the cairo
surface to draw outside the window where it can't be seen. Passing the
absolute values of the scale factors fixes the display on macOS 12 without
affecting earlier macOS versions.
Add gdk_gl_context_is_api_allowed() for backends and make them use it.
Finally, have them return the final API as the return value (or 0 on
error).
And then use that api instead of a use_es boolean flag.
Fixes#4221
The term "hdr" is so overloaded, we shouldn't use them anywhere, except
from maybe describing all of this work in blog posts and other marketing
materials.
So do renames:
* hdr => high_depth
* request_hdr => prefers_high_depth
This more accurately describes what is going on.
Creative people managed to create an X11 display and a Wayland display
at once, thereby getting EGL and GLX involved in a fight to the death
over the ownership of the glFoo() symbolspace.
A way to force such a fight with available tools here is (on Wayland)
running something like:
GTK_INSPECTOR_DISPLAY=:1 GTK_DEBUG=interactive gtk4-demo
Related: xdg-desktop-portal-gnome#5
_gdk_macos_event_source_new() calls g_source_set_static_name(), which
for GLib versions before 2.69.1 is a macro defined in gdk-private.h.
Fixes#4195
modified: gdk/macos/gdkmacoseventsource.c
Usually the "dnd-finished" signal will be used to unref the GdkDrag. In
those cases, we would lose the object, so that when we do the final
drag_drop_done() afterwards, we wouldn't have a remaining reference.
With the reference guard, this now works.
Since UCKeyTranslate() converts these keys to Space key unexpectedly,
applications can't distinguish these keys by keysyms.
To solve it, this fix translates these keys by the same way with
function keys & keypad keys.
This patch is equivalent to the patch proposed in:
https://bugzilla.gnome.org/show_bug.cgi?id=702841Closes#4117
Now that we have the display's context to hook into, we can use it to
construct other GL contexts and don't need a GdkSurface vfunc anymore.
This has the added benefit that backends can have different GdkGLContext
classes on the display and get new GLContexts generated from them, so
we get multiple GL backend support per GDK backend for free.
I originally wanted to make this a vfunc on GdkGLContextClass, but
it turns out all the abckends would just call g_object_new() anyway.
Instead of
Display::make_gl_context_current()
we now have
GLContext::clear_current()
GLContext::make_current()
This fits better with the backends (we can actually implement
clearCurrent on macOS now) and makes it easier to implement different GL
backends for backends (like EGL/GLX on X11).
We also pass a surfaceless boolean to make_current() so the calling code
can decide if a surface needs to be bound or not, because the backends
were all doing whatever, which was very counterproductive.
... or more exactly: Only use paint contexts with
gdk_cairo_draw_from_gl().
Instead of paint contexts being the only contexts who call swapBuffer(),
any context can be used for this, when it's used with
begin_frame()/end_frame().
This removes 2 features:
1. We no longer need a big sharing hierarchy. All contexts are now
shared with gdk_display_get_gl_context().
2. There is no longer a difference between attached and non-attached
contexts. All contexts work the same way.
The vfunc is called to initialize GL and it returns a "base" context
that GDK then uses as the context all others are shared with. So the GL
context share tree now looks like:
+ context from init_gl
- context1
- context2
...
So this is a flat tree now, the complexity is gone.
The only caveat is that backends now need to create a GL context when
initializing GL so some refactoring was needed.
Two new functions have been added:
* gdk_display_prepare_gl()
This is public API and can be used to ensure that GL has been
initialized or if not, retrieve an error to display (or debug-print).
* gdk_display_get_gl_context()
This is a private function to retrieve the base context from
init_gl(). It replaces gdk_surface_get_shared_data_context().
This gets the basic mechanics of the drop portion of DnD working on the
macOS backend. You can drag, for example, from TextEdit into GNOME
Text Editor when using the macOS backend.
Other content formats are supported, and match what is currently
supported by the clipboard backend as the implementation to read
from the pasteboard is shared.
Currently, we look up the GdkDrag for the new GdkDrop. However,
nothing is stashing the drag away for further lookup. More work is
needed on GdkMacosDrag for that to be doable.
We will want to be able to reuse the pasteboard reading code from
the macOS DnD drop backend. This just removes the pasteboard
bits from the implementation and allows that to be passed in as in
both clipboard and DnD cases we'll have a specific NSPasteboard
to read from.
If our opaque region is the entire surface, then we can make the OpenGL
context opaque like we do for decorated windows. This improves performance
as the compositor does not need to blend the surface with the contents
behind the window.
To support Sierra, we need to have access to pasteboard types as a
NSString. Constants are provided in later versions of macOS, but we
can emulate that with an array which is initialized on first access.
On older systems, the availability of some methods seem to be incorrect
based on Apple documentation. This works around the issue by using
the rect conversion on older systems.
These functions were not implemented when the sizing changes
landed before GTK 4 was released. This fixes an issue with non-
resizeable windows not reacting to layout changes.
Fixes#3532
When being fullscreen, and wanting to unfullscreen but not caring about
whether to go unmaximized or maximized (as this information is lost), if
the GdkToplevelLayout represents the full intended state, we won't be
able to do the right thing.
To avoid this issue, make the GdkToplevelLayout API intend based, where
if one e.g. doesn't call gdk_toplevel_set_maximized() with anything, the
backend will not attempt to change the maximized state.
This means we can also remove the old 'initially_maximized' and
'initially_fullscreen' fields from the private GtkWindow struct, as we
only deal with intents now.
We don't need to go through the NSOpenGLContext for these.
We can just use the C API directly. It's also clearer what is using
CGLEnable() vs CGLSetParameter().
It was used by all surfaces to track 'is-mapped', but still part of the
GdkToplevelState, and is now replaced with a separate boolean in the
GdkSurface structure.
It also caused issues when a widget was unmapped, and due to that
unmapped a popover which hid its corresponding surface. When this
surface was hidden, it emitted a state change event, which would then go
back into GTK and queue a resize on popover widget, which would travel
back down to the widget that was originally unmapped, causing confusino
when doing future allocations.
To summarize, one should not hide widgets during allocation, and to
avoid this, make this new is-mapped boolean asynchronous when hiding a
surface, meaning the notification event for the changed mapped state
will be emitted in an idle callback. This avoids the above described
reentry issue.
This removes the GDK_CONFIGURE event and all related functions and data
types; it includes untested changes to the MacOSX, Win32 and Broadway
backends.
This removes the gdk_surface_set_shadow_width() function and related
vfuncs. The point here is that the shadow width and surface size can now
be communicated to GDK atomically, meaning it's possible to avoid
intermediate stages where the surface size includes the shadow, but
without the shadow width set, or the other way around.
We don't need the OpenGL view to be transparent if the window itself
is not transparent. This has the potential to speed up the compositing
of the GL view onto the NSWindow.
This fixes an issue where we would ignore events with Y delta
and no X delta while scrolling due to a typo when checking for
any delta.
This fixes deceleration of kinetic scrolling on the macOS backend.
Fixes#3418
We need to keep this consistent so that we can look things up
faster in other places. Therefore, just take the hit here and clear
the entire list ensuring prev/next poniters are cleared.
This isn't done automatically for us, so we need to synthesize it in
our hide helper.
With this commit, we properly re-focus the new main/key window after
we have closed a transient-for window.
We need to re-attach to the transient-for window whenever we present or
we risk getting placed behind the window by the display server. Apparently
that setting does not persist across a hide of the NSWindow.
Once we figure out what is going on with textures, changes are we'll be
able to let this stay a zero rect. But that is still a bit up in the air right now.
The Cairo implementation for the Macos backend uses a toplevel
window with full transparency and a series of NSView to create opaque
regions. This improves compositor performance because it allows the
display server to avoid costly blends.
However, we want to ensure we clip better when exposing the
transparent region so that we only expose the shadows/corners as
necessary.
This typedef was not used in any public APIs, and is
only used in the MacOS backend. It is not worth preserving
as public API, move it to the only user.
This implements the basics for a GdkGLContext on macOS. Currently, rendering
only is fully working for the GskCairoRenderer case where we read back pixels
into a cairo surface for rendering. More work on synchronization is required for
the GL on GskGLRenderer case.
When we attempt to render a surface itself with GL, the context will ensure that
the new GdkMacosGLView is placed within the NSWindow. In other cases, we
use a dummy NSView and NSWindow for backing the NSOpenGLContext to
ensure that we can get accelerated drawing.
This gets GtkGLArea working when running with GSK_RENDERER=cairo.
This helps a situation where the window contents has not changed
in time for a drawing. Setting the texture gravity helps that side or
corner to be less jittery while moving.
Ideally, we can get to a point where we are synchronized and keeping
up with drawing fast enough to not need this. That may require some
work to drive frame clocks from drawRect: though.
This was incorrectly reporting the toplevel surface instead of the
popup surface that was placed above it. This fixes event delivery
to popups for selecting menu items and more.
When querying a device, we need to ensure we are providing coordinates
in the coordinate system of the surface. Further, we need to actually
provide the button and keyboard state.
This fixes some issues related to dragging scrollbars and selecting list
box rows more reliably.
GTK will not up front know how to correctly calculate a size, since it
will not be able to reliably predict the constraints that may exist
where it will be mapped.
Thus, to handle this, calculate the size of the toplevel by having GDK
emitting a signal called 'compute-size' that will contain information
needed for computing a toplevel window size.
This signal may be emitted at any time, e.g. during
gdk_toplevel_present(), or spontaneously if constraints change.
This also drops the max size from the toplevel layout, while moving the
min size from the toplevel layout struct to the struct passed via the
signal,
This needs changes to a test case where we make sure we process
GDK_CONFIGURE etc, which means we also needs to show the window and
process all pending events in the test-focus-chain test case.
It's not a portable API, so remove it. The corresponding backend
specific functions are still available, if they were implemented, e.g.
gdk_macos_monitor_get_workarea() and gdk_x11_monitor_get_workarea().
When converting DisplayLink frame presentation times, we need to take into
account the arch-specific types. This tracks changes in GNOME/GLib!1566 so
that precision is not lost.
This is fairly substantial rewrite of the GDK backend for quartz and
renamed to macOS to allow for a greenfield implementation.
Many things have come across from the quartz implementation fairly
intact such as the eventloop integration design and discovery of
event windows from the NSEvent.
However much has been changed to fit in with the new GDK design and
how removal of child GdkWindow have been completely eliminated.
Furthermore, the new GdkPopup allows for regular NSWindow to be used
to provide popovers unlike the previous implementation.
The object design more closely follows the ideal for a GDK backend.
Views have been broken out into subclasses so that we can support
multiple GSK renderer paths such as GL and Cairo (and Metal in the
future). However mixed mode GL and Cairo will not be supported. Currently
only the Cairo renderer has been implemented.
A new frame clock implementation using CVDisplayLink provides more
accurate information about when to draw drawing the next frame. Some
testing will need to be done here to understand the power implications
of this.
This implementation has also gained edge snapping for CSD windows. Some
work was also done to ensure that CSD windows have opaque regions
registered with the display server.
** This is still very much a work-in-progress **
Some outstanding work that needs to be done:
- Finish a GL context for macOS and alternate NSView for GL rendering
(possibly using speciailized CALayer for OpenGL).
- Input rework to ensure that we don't loose remapping of keys that was
dropped from GDK during GTK 4 development.
- Make sure input methods continue to work.
- Drag-n-Drop is still very much a work in progress
- High resolution input scrolling needs various work in GDK to land
first before we can plumb that to NSEvent.
- gtk/ has a number of things based on GDK_WINDOWING_QUARTZ that need
to be updated to use the macOS backend.
But this is good enough to start playing with and breaking things which
is what I'd like to see.
