AuroraMiddleware/gtk
1
0
Fork 1
mirror of https://gitlab.gnome.org/GNOME/gtk.git synced 2024-09-19 21:40:22 +00:00
Code Issues Projects Releases Wiki Activity
main
gtk/gdk/macos/gdkmacoscairocontext.c

291 lines
8.7 KiB
C
Raw Permalink Normal View History

macos: prototype new GDK backend for macOS This is fairly substantial rewrite of the GDK backend for quartz and renamed to macOS to allow for a greenfield implementation. Many things have come across from the quartz implementation fairly intact such as the eventloop integration design and discovery of event windows from the NSEvent. However much has been changed to fit in with the new GDK design and how removal of child GdkWindow have been completely eliminated. Furthermore, the new GdkPopup allows for regular NSWindow to be used to provide popovers unlike the previous implementation. The object design more closely follows the ideal for a GDK backend. Views have been broken out into subclasses so that we can support multiple GSK renderer paths such as GL and Cairo (and Metal in the future). However mixed mode GL and Cairo will not be supported. Currently only the Cairo renderer has been implemented. A new frame clock implementation using CVDisplayLink provides more accurate information about when to draw drawing the next frame. Some testing will need to be done here to understand the power implications of this. This implementation has also gained edge snapping for CSD windows. Some work was also done to ensure that CSD windows have opaque regions registered with the display server. ** This is still very much a work-in-progress ** Some outstanding work that needs to be done: - Finish a GL context for macOS and alternate NSView for GL rendering (possibly using speciailized CALayer for OpenGL). - Input rework to ensure that we don't loose remapping of keys that was dropped from GDK during GTK 4 development. - Make sure input methods continue to work. - Drag-n-Drop is still very much a work in progress - High resolution input scrolling needs various work in GDK to land first before we can plumb that to NSEvent. - gtk/ has a number of things based on GDK_WINDOWING_QUARTZ that need to be updated to use the macOS backend. But this is good enough to start playing with and breaking things which is what I'd like to see.
2020-04-23 23:36:46 +00:00
/*
* Copyright © 2016 Benjamin Otte
* Copyright © 2020 Red Hat, Inc.
*
* This library is free software; you can redistribute it and/or
* modify it under the terms of the GNU Lesser General Public
* License as published by the Free Software Foundation; either
* version 2.1 of the License, or (at your option) any later version.
*
* This library is distributed in the hope that it will be useful,
* but WITHOUT ANY WARRANTY; without even the implied warranty of
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
* Lesser General Public License for more details.
*
* You should have received a copy of the GNU Lesser General Public
* License along with this library. If not, see <http://www.gnu.org/licenses/>.
*
* SPDX-License-Identifier: LGPL-2.1-or-later
*/
#include "config.h"
#include "gdkconfig.h"
macos: use CALayer and IOSurface for rendering This provides a major shift in how we draw both when accelerated OpenGL as well as software rendering with Cairo. In short, it uses tiles of Core Animation's CALayer to display contents from an OpenGL or Cairo rendering so that the window can provide partial damage updates. Partial damage is not generally available when using OpenGL as the whole buffer is flipped even if you only submitted a small change using a scissor rect. Thankfully, this speeds up Cairo rendering a bit too by using IOSurface to upload contents to the display server. We use the tiling system we do for OpenGL which reduces overall complexity and differences between them. A New Buffer ============ GdkMacosBuffer is a wrapper around an IOSurfaceRef. The term buffer was used because 1) surface is already used and 2) it loosely maps to a front/back buffer semantic. However, it appears that IOSurfaceRef contents are being retained in some fashion (likely in the compositor result) so we can update the same IOSurfaceRef without flipping as long as we're fast. This appears to be what Chromium does as well, but Firefox uses two IOSurfaceRef and flips between them. We would like to avoid two surfaces because it doubles the GPU VRAM requirements of the application. Changes to Windows ================== Previously, the NSWindow would dynamically change between different types of NSView based on the renderer being used. This is no longer necessary as we just have a single NSView type, GdkMacosView, which inherits from GdkMacosBaseView just to keep the tedius stuff separate from the machinery of GdkMacosView. We can merge those someday if we are okay with that. Changes to Views ================ GdkMacosCairoView, GdkMacosCairoSubView, GdkMacosGLView have all been removed and replaced with GdkMacosView. This new view has a single CALayer (GdkMacosLayer) attached to it which itself has sublayers. The contents of the CALayer is populated with an IOSurfaceRef which we allocated with the GdkMacosSurface. The surface is replaced when the NSWindow resizes. Changes to Layers ================= We now have a dedicated GdkMacosLayer which contains sublayers of GdkMacosTile. The tile has a maximum size of 128x128 pixels in device units. The GdkMacosTile is partitioned by splitting both the transparent region (window bounds minus opaque area) and then by splitting the opaque area. A tile has either translucent contents (and therefore is not opaque) or has opaque contents (and therefore is opaque). An opaque tile never contains transparent contents. As such, the opaque tiles contain a black background so that Core Animation will consider the tile's bounds as opaque. This can be verified with "Quartz Debug -> Show opaque regions". Changes to Cairo ================ GTK 4 cannot currently use cairo-quartz because of how CSS borders are rendered. It simply causes errors in the cairo_quartz_surface_t backend. Since we are restricted to using cairo_image_surface_t (which happens to be faster anyway) we can use the IOSurfaceBaseAddress() to obtain a mapping of the IOSurfaceRef in user-space. It always uses BGRA 32-bit with alpha channel even if we will discard the alpha channel as that is necessary to hit the fast paths in other parts of the platform. Note that while Cairo says CAIRO_FORMAT_ARGB32, it is really 32-bit BGRA on little-endian as we expect. OpenGL will render flipped (Quartz Native Co-ordinates) while Cairo renders with 0,O in the top-left. We could use cairo_translate() and cairo_scale() to reverse this, but it looks like some cairo things may not look quite as right if we do so. To reduce the chances of one-off bugs this continues to draw as Cairo would normally, but instead uses an CGAffineTransform in the tiles and some CGRect translation when swapping buffers to get the same effect. Changes to OpenGL ================= To simplify things, removal of all NSOpenGL* related components have been removed and we strictly use the Core GL (CGL*) API. This probably should have been done long ago anyay. Most examples found in the browsers to use IOSurfaceRef with OpenGL are using Legacy GL and there is still work underway to make this fit in with the rest of how the GSK GL renderer works. Since IOSurfaceRef bound to a texture/framebuffer will not have a default framebuffer ID of 0, we needed to add a default framebuffer id to the GdkGLContext. GskGLRenderer can use this to setup the command queue in such a way that our IOSurface destination has been glBindFramebuffer() as if it were the default drawable. This stuff is pretty slight-of-hand, so where things are and what needs flushing when and where has been a bit of an experiment to see what actually works to get synchronization across subsystems. Efficient Damages ================= After we draw with Cairo, we unlock the IOSurfaceRef and the contents are uploaded to the GPU. To make the contents visible to the app, we must clear the tiles contents with `layer.contents=nil;` and then re-apply the IOSurfaceRef. Since the buffer has likely not changed, we only do this if the tile overlaps the damage region. This gives the effect of having more tightly controlled damage regions even though updating the layer would damage be the whole window (as it is with OpenGL/Metal today with the exception of scissor-rect). This too can be verified usign "Quartz Debug -> Flash screen udpates". Frame Synchronized Resize ========================= In GTK 4, we have the ability to perform sizing changes from compute-size during the layout phase. Since the macOS backend already tracks window resizes manually, we can avoid doing the setFrame: immediately and instead do it within the frame clock's layout phase. Doing so gives us vastly better resize experience as we're more likely to get the size-change and updated-contents in the same frame on screen. It makes things feel "connected" in a way they weren't before. Some additional effort to tweak gravity during the process is also necessary but we were already doing that in the GTK 4 backend. Backporting =========== The design here has made an attempt to make it possible to backport by keeping GdkMacosBuffer, GdkMacosLayer, and GdkMacosTile fairly independent. There may be an opportunity to integrate this into GTK 3's quartz backend with a fair bit of work. Doing so could improve the situation for applications which are damage-rich such as The GIMP.
2022-02-14 10:20:19 +00:00
#include <cairo.h>
#include <QuartzCore/QuartzCore.h>
macos: prototype new GDK backend for macOS This is fairly substantial rewrite of the GDK backend for quartz and renamed to macOS to allow for a greenfield implementation. Many things have come across from the quartz implementation fairly intact such as the eventloop integration design and discovery of event windows from the NSEvent. However much has been changed to fit in with the new GDK design and how removal of child GdkWindow have been completely eliminated. Furthermore, the new GdkPopup allows for regular NSWindow to be used to provide popovers unlike the previous implementation. The object design more closely follows the ideal for a GDK backend. Views have been broken out into subclasses so that we can support multiple GSK renderer paths such as GL and Cairo (and Metal in the future). However mixed mode GL and Cairo will not be supported. Currently only the Cairo renderer has been implemented. A new frame clock implementation using CVDisplayLink provides more accurate information about when to draw drawing the next frame. Some testing will need to be done here to understand the power implications of this. This implementation has also gained edge snapping for CSD windows. Some work was also done to ensure that CSD windows have opaque regions registered with the display server. ** This is still very much a work-in-progress ** Some outstanding work that needs to be done: - Finish a GL context for macOS and alternate NSView for GL rendering (possibly using speciailized CALayer for OpenGL). - Input rework to ensure that we don't loose remapping of keys that was dropped from GDK during GTK 4 development. - Make sure input methods continue to work. - Drag-n-Drop is still very much a work in progress - High resolution input scrolling needs various work in GDK to land first before we can plumb that to NSEvent. - gtk/ has a number of things based on GDK_WINDOWING_QUARTZ that need to be updated to use the macOS backend. But this is good enough to start playing with and breaking things which is what I'd like to see.
2020-04-23 23:36:46 +00:00
#include <CoreGraphics/CoreGraphics.h>
macos: use CALayer and IOSurface for rendering This provides a major shift in how we draw both when accelerated OpenGL as well as software rendering with Cairo. In short, it uses tiles of Core Animation's CALayer to display contents from an OpenGL or Cairo rendering so that the window can provide partial damage updates. Partial damage is not generally available when using OpenGL as the whole buffer is flipped even if you only submitted a small change using a scissor rect. Thankfully, this speeds up Cairo rendering a bit too by using IOSurface to upload contents to the display server. We use the tiling system we do for OpenGL which reduces overall complexity and differences between them. A New Buffer ============ GdkMacosBuffer is a wrapper around an IOSurfaceRef. The term buffer was used because 1) surface is already used and 2) it loosely maps to a front/back buffer semantic. However, it appears that IOSurfaceRef contents are being retained in some fashion (likely in the compositor result) so we can update the same IOSurfaceRef without flipping as long as we're fast. This appears to be what Chromium does as well, but Firefox uses two IOSurfaceRef and flips between them. We would like to avoid two surfaces because it doubles the GPU VRAM requirements of the application. Changes to Windows ================== Previously, the NSWindow would dynamically change between different types of NSView based on the renderer being used. This is no longer necessary as we just have a single NSView type, GdkMacosView, which inherits from GdkMacosBaseView just to keep the tedius stuff separate from the machinery of GdkMacosView. We can merge those someday if we are okay with that. Changes to Views ================ GdkMacosCairoView, GdkMacosCairoSubView, GdkMacosGLView have all been removed and replaced with GdkMacosView. This new view has a single CALayer (GdkMacosLayer) attached to it which itself has sublayers. The contents of the CALayer is populated with an IOSurfaceRef which we allocated with the GdkMacosSurface. The surface is replaced when the NSWindow resizes. Changes to Layers ================= We now have a dedicated GdkMacosLayer which contains sublayers of GdkMacosTile. The tile has a maximum size of 128x128 pixels in device units. The GdkMacosTile is partitioned by splitting both the transparent region (window bounds minus opaque area) and then by splitting the opaque area. A tile has either translucent contents (and therefore is not opaque) or has opaque contents (and therefore is opaque). An opaque tile never contains transparent contents. As such, the opaque tiles contain a black background so that Core Animation will consider the tile's bounds as opaque. This can be verified with "Quartz Debug -> Show opaque regions". Changes to Cairo ================ GTK 4 cannot currently use cairo-quartz because of how CSS borders are rendered. It simply causes errors in the cairo_quartz_surface_t backend. Since we are restricted to using cairo_image_surface_t (which happens to be faster anyway) we can use the IOSurfaceBaseAddress() to obtain a mapping of the IOSurfaceRef in user-space. It always uses BGRA 32-bit with alpha channel even if we will discard the alpha channel as that is necessary to hit the fast paths in other parts of the platform. Note that while Cairo says CAIRO_FORMAT_ARGB32, it is really 32-bit BGRA on little-endian as we expect. OpenGL will render flipped (Quartz Native Co-ordinates) while Cairo renders with 0,O in the top-left. We could use cairo_translate() and cairo_scale() to reverse this, but it looks like some cairo things may not look quite as right if we do so. To reduce the chances of one-off bugs this continues to draw as Cairo would normally, but instead uses an CGAffineTransform in the tiles and some CGRect translation when swapping buffers to get the same effect. Changes to OpenGL ================= To simplify things, removal of all NSOpenGL* related components have been removed and we strictly use the Core GL (CGL*) API. This probably should have been done long ago anyay. Most examples found in the browsers to use IOSurfaceRef with OpenGL are using Legacy GL and there is still work underway to make this fit in with the rest of how the GSK GL renderer works. Since IOSurfaceRef bound to a texture/framebuffer will not have a default framebuffer ID of 0, we needed to add a default framebuffer id to the GdkGLContext. GskGLRenderer can use this to setup the command queue in such a way that our IOSurface destination has been glBindFramebuffer() as if it were the default drawable. This stuff is pretty slight-of-hand, so where things are and what needs flushing when and where has been a bit of an experiment to see what actually works to get synchronization across subsystems. Efficient Damages ================= After we draw with Cairo, we unlock the IOSurfaceRef and the contents are uploaded to the GPU. To make the contents visible to the app, we must clear the tiles contents with `layer.contents=nil;` and then re-apply the IOSurfaceRef. Since the buffer has likely not changed, we only do this if the tile overlaps the damage region. This gives the effect of having more tightly controlled damage regions even though updating the layer would damage be the whole window (as it is with OpenGL/Metal today with the exception of scissor-rect). This too can be verified usign "Quartz Debug -> Flash screen udpates". Frame Synchronized Resize ========================= In GTK 4, we have the ability to perform sizing changes from compute-size during the layout phase. Since the macOS backend already tracks window resizes manually, we can avoid doing the setFrame: immediately and instead do it within the frame clock's layout phase. Doing so gives us vastly better resize experience as we're more likely to get the size-change and updated-contents in the same frame on screen. It makes things feel "connected" in a way they weren't before. Some additional effort to tweak gravity during the process is also necessary but we were already doing that in the GTK 4 backend. Backporting =========== The design here has made an attempt to make it possible to backport by keeping GdkMacosBuffer, GdkMacosLayer, and GdkMacosTile fairly independent. There may be an opportunity to integrate this into GTK 3's quartz backend with a fair bit of work. Doing so could improve the situation for applications which are damage-rich such as The GIMP.
2022-02-14 10:20:19 +00:00
#include "gdkmacosbuffer-private.h"
macos: prototype new GDK backend for macOS This is fairly substantial rewrite of the GDK backend for quartz and renamed to macOS to allow for a greenfield implementation. Many things have come across from the quartz implementation fairly intact such as the eventloop integration design and discovery of event windows from the NSEvent. However much has been changed to fit in with the new GDK design and how removal of child GdkWindow have been completely eliminated. Furthermore, the new GdkPopup allows for regular NSWindow to be used to provide popovers unlike the previous implementation. The object design more closely follows the ideal for a GDK backend. Views have been broken out into subclasses so that we can support multiple GSK renderer paths such as GL and Cairo (and Metal in the future). However mixed mode GL and Cairo will not be supported. Currently only the Cairo renderer has been implemented. A new frame clock implementation using CVDisplayLink provides more accurate information about when to draw drawing the next frame. Some testing will need to be done here to understand the power implications of this. This implementation has also gained edge snapping for CSD windows. Some work was also done to ensure that CSD windows have opaque regions registered with the display server. ** This is still very much a work-in-progress ** Some outstanding work that needs to be done: - Finish a GL context for macOS and alternate NSView for GL rendering (possibly using speciailized CALayer for OpenGL). - Input rework to ensure that we don't loose remapping of keys that was dropped from GDK during GTK 4 development. - Make sure input methods continue to work. - Drag-n-Drop is still very much a work in progress - High resolution input scrolling needs various work in GDK to land first before we can plumb that to NSEvent. - gtk/ has a number of things based on GDK_WINDOWING_QUARTZ that need to be updated to use the macOS backend. But this is good enough to start playing with and breaking things which is what I'd like to see.
2020-04-23 23:36:46 +00:00
#include "gdkmacoscairocontext-private.h"
#include "gdkmacossurface-private.h"
struct _GdkMacosCairoContext
{
macos: use CALayer and IOSurface for rendering This provides a major shift in how we draw both when accelerated OpenGL as well as software rendering with Cairo. In short, it uses tiles of Core Animation's CALayer to display contents from an OpenGL or Cairo rendering so that the window can provide partial damage updates. Partial damage is not generally available when using OpenGL as the whole buffer is flipped even if you only submitted a small change using a scissor rect. Thankfully, this speeds up Cairo rendering a bit too by using IOSurface to upload contents to the display server. We use the tiling system we do for OpenGL which reduces overall complexity and differences between them. A New Buffer ============ GdkMacosBuffer is a wrapper around an IOSurfaceRef. The term buffer was used because 1) surface is already used and 2) it loosely maps to a front/back buffer semantic. However, it appears that IOSurfaceRef contents are being retained in some fashion (likely in the compositor result) so we can update the same IOSurfaceRef without flipping as long as we're fast. This appears to be what Chromium does as well, but Firefox uses two IOSurfaceRef and flips between them. We would like to avoid two surfaces because it doubles the GPU VRAM requirements of the application. Changes to Windows ================== Previously, the NSWindow would dynamically change between different types of NSView based on the renderer being used. This is no longer necessary as we just have a single NSView type, GdkMacosView, which inherits from GdkMacosBaseView just to keep the tedius stuff separate from the machinery of GdkMacosView. We can merge those someday if we are okay with that. Changes to Views ================ GdkMacosCairoView, GdkMacosCairoSubView, GdkMacosGLView have all been removed and replaced with GdkMacosView. This new view has a single CALayer (GdkMacosLayer) attached to it which itself has sublayers. The contents of the CALayer is populated with an IOSurfaceRef which we allocated with the GdkMacosSurface. The surface is replaced when the NSWindow resizes. Changes to Layers ================= We now have a dedicated GdkMacosLayer which contains sublayers of GdkMacosTile. The tile has a maximum size of 128x128 pixels in device units. The GdkMacosTile is partitioned by splitting both the transparent region (window bounds minus opaque area) and then by splitting the opaque area. A tile has either translucent contents (and therefore is not opaque) or has opaque contents (and therefore is opaque). An opaque tile never contains transparent contents. As such, the opaque tiles contain a black background so that Core Animation will consider the tile's bounds as opaque. This can be verified with "Quartz Debug -> Show opaque regions". Changes to Cairo ================ GTK 4 cannot currently use cairo-quartz because of how CSS borders are rendered. It simply causes errors in the cairo_quartz_surface_t backend. Since we are restricted to using cairo_image_surface_t (which happens to be faster anyway) we can use the IOSurfaceBaseAddress() to obtain a mapping of the IOSurfaceRef in user-space. It always uses BGRA 32-bit with alpha channel even if we will discard the alpha channel as that is necessary to hit the fast paths in other parts of the platform. Note that while Cairo says CAIRO_FORMAT_ARGB32, it is really 32-bit BGRA on little-endian as we expect. OpenGL will render flipped (Quartz Native Co-ordinates) while Cairo renders with 0,O in the top-left. We could use cairo_translate() and cairo_scale() to reverse this, but it looks like some cairo things may not look quite as right if we do so. To reduce the chances of one-off bugs this continues to draw as Cairo would normally, but instead uses an CGAffineTransform in the tiles and some CGRect translation when swapping buffers to get the same effect. Changes to OpenGL ================= To simplify things, removal of all NSOpenGL* related components have been removed and we strictly use the Core GL (CGL*) API. This probably should have been done long ago anyay. Most examples found in the browsers to use IOSurfaceRef with OpenGL are using Legacy GL and there is still work underway to make this fit in with the rest of how the GSK GL renderer works. Since IOSurfaceRef bound to a texture/framebuffer will not have a default framebuffer ID of 0, we needed to add a default framebuffer id to the GdkGLContext. GskGLRenderer can use this to setup the command queue in such a way that our IOSurface destination has been glBindFramebuffer() as if it were the default drawable. This stuff is pretty slight-of-hand, so where things are and what needs flushing when and where has been a bit of an experiment to see what actually works to get synchronization across subsystems. Efficient Damages ================= After we draw with Cairo, we unlock the IOSurfaceRef and the contents are uploaded to the GPU. To make the contents visible to the app, we must clear the tiles contents with `layer.contents=nil;` and then re-apply the IOSurfaceRef. Since the buffer has likely not changed, we only do this if the tile overlaps the damage region. This gives the effect of having more tightly controlled damage regions even though updating the layer would damage be the whole window (as it is with OpenGL/Metal today with the exception of scissor-rect). This too can be verified usign "Quartz Debug -> Flash screen udpates". Frame Synchronized Resize ========================= In GTK 4, we have the ability to perform sizing changes from compute-size during the layout phase. Since the macOS backend already tracks window resizes manually, we can avoid doing the setFrame: immediately and instead do it within the frame clock's layout phase. Doing so gives us vastly better resize experience as we're more likely to get the size-change and updated-contents in the same frame on screen. It makes things feel "connected" in a way they weren't before. Some additional effort to tweak gravity during the process is also necessary but we were already doing that in the GTK 4 backend. Backporting =========== The design here has made an attempt to make it possible to backport by keeping GdkMacosBuffer, GdkMacosLayer, and GdkMacosTile fairly independent. There may be an opportunity to integrate this into GTK 3's quartz backend with a fair bit of work. Doing so could improve the situation for applications which are damage-rich such as The GIMP.
2022-02-14 10:20:19 +00:00
GdkCairoContext parent_instance;
macos: prototype new GDK backend for macOS This is fairly substantial rewrite of the GDK backend for quartz and renamed to macOS to allow for a greenfield implementation. Many things have come across from the quartz implementation fairly intact such as the eventloop integration design and discovery of event windows from the NSEvent. However much has been changed to fit in with the new GDK design and how removal of child GdkWindow have been completely eliminated. Furthermore, the new GdkPopup allows for regular NSWindow to be used to provide popovers unlike the previous implementation. The object design more closely follows the ideal for a GDK backend. Views have been broken out into subclasses so that we can support multiple GSK renderer paths such as GL and Cairo (and Metal in the future). However mixed mode GL and Cairo will not be supported. Currently only the Cairo renderer has been implemented. A new frame clock implementation using CVDisplayLink provides more accurate information about when to draw drawing the next frame. Some testing will need to be done here to understand the power implications of this. This implementation has also gained edge snapping for CSD windows. Some work was also done to ensure that CSD windows have opaque regions registered with the display server. ** This is still very much a work-in-progress ** Some outstanding work that needs to be done: - Finish a GL context for macOS and alternate NSView for GL rendering (possibly using speciailized CALayer for OpenGL). - Input rework to ensure that we don't loose remapping of keys that was dropped from GDK during GTK 4 development. - Make sure input methods continue to work. - Drag-n-Drop is still very much a work in progress - High resolution input scrolling needs various work in GDK to land first before we can plumb that to NSEvent. - gtk/ has a number of things based on GDK_WINDOWING_QUARTZ that need to be updated to use the macOS backend. But this is good enough to start playing with and breaking things which is what I'd like to see.
2020-04-23 23:36:46 +00:00
};
struct _GdkMacosCairoContextClass
{
GdkCairoContextClass parent_class;
};
G_DEFINE_TYPE (GdkMacosCairoContext, _gdk_macos_cairo_context, GDK_TYPE_CAIRO_CONTEXT)
macos: use CGContext to draw cairo surfaces Instead of relying on cairo_t to perform drawing from our backing image surface to the Core Graphics context, we can convert the cairo_image_surface_t into a CGImageRef without having to copy data if we are certain of the alignment of the image up front. Without this, there are many situations, based on the size of the window that could cause cairo to take a slow path and malloc/copy the data to ensure that alignment. The previous commit titled "macos: align image surface rowstride to 16-bytes" ensures that this invariant is true so that our drawing code can assume we can reference the framebuffer from the cairo_image_surface_t using a CGDataProvider. Since GdkMacosCairoContext and GdkMacosCairoSubview are coordinating, we can also setup the transformation/scale early when drawing the cairo_image_surface_t instead of when copying it to Core Graphics. Furthermore, the CGImageRef is created with an RGB colorspace so that we are not performing colorspace conversion to the output device. We don't get color matching between displays, but we don't expect that anyway, particularly with the software renderer.
2022-02-04 02:31:09 +00:00
static cairo_t *
macos: use CALayer and IOSurface for rendering This provides a major shift in how we draw both when accelerated OpenGL as well as software rendering with Cairo. In short, it uses tiles of Core Animation's CALayer to display contents from an OpenGL or Cairo rendering so that the window can provide partial damage updates. Partial damage is not generally available when using OpenGL as the whole buffer is flipped even if you only submitted a small change using a scissor rect. Thankfully, this speeds up Cairo rendering a bit too by using IOSurface to upload contents to the display server. We use the tiling system we do for OpenGL which reduces overall complexity and differences between them. A New Buffer ============ GdkMacosBuffer is a wrapper around an IOSurfaceRef. The term buffer was used because 1) surface is already used and 2) it loosely maps to a front/back buffer semantic. However, it appears that IOSurfaceRef contents are being retained in some fashion (likely in the compositor result) so we can update the same IOSurfaceRef without flipping as long as we're fast. This appears to be what Chromium does as well, but Firefox uses two IOSurfaceRef and flips between them. We would like to avoid two surfaces because it doubles the GPU VRAM requirements of the application. Changes to Windows ================== Previously, the NSWindow would dynamically change between different types of NSView based on the renderer being used. This is no longer necessary as we just have a single NSView type, GdkMacosView, which inherits from GdkMacosBaseView just to keep the tedius stuff separate from the machinery of GdkMacosView. We can merge those someday if we are okay with that. Changes to Views ================ GdkMacosCairoView, GdkMacosCairoSubView, GdkMacosGLView have all been removed and replaced with GdkMacosView. This new view has a single CALayer (GdkMacosLayer) attached to it which itself has sublayers. The contents of the CALayer is populated with an IOSurfaceRef which we allocated with the GdkMacosSurface. The surface is replaced when the NSWindow resizes. Changes to Layers ================= We now have a dedicated GdkMacosLayer which contains sublayers of GdkMacosTile. The tile has a maximum size of 128x128 pixels in device units. The GdkMacosTile is partitioned by splitting both the transparent region (window bounds minus opaque area) and then by splitting the opaque area. A tile has either translucent contents (and therefore is not opaque) or has opaque contents (and therefore is opaque). An opaque tile never contains transparent contents. As such, the opaque tiles contain a black background so that Core Animation will consider the tile's bounds as opaque. This can be verified with "Quartz Debug -> Show opaque regions". Changes to Cairo ================ GTK 4 cannot currently use cairo-quartz because of how CSS borders are rendered. It simply causes errors in the cairo_quartz_surface_t backend. Since we are restricted to using cairo_image_surface_t (which happens to be faster anyway) we can use the IOSurfaceBaseAddress() to obtain a mapping of the IOSurfaceRef in user-space. It always uses BGRA 32-bit with alpha channel even if we will discard the alpha channel as that is necessary to hit the fast paths in other parts of the platform. Note that while Cairo says CAIRO_FORMAT_ARGB32, it is really 32-bit BGRA on little-endian as we expect. OpenGL will render flipped (Quartz Native Co-ordinates) while Cairo renders with 0,O in the top-left. We could use cairo_translate() and cairo_scale() to reverse this, but it looks like some cairo things may not look quite as right if we do so. To reduce the chances of one-off bugs this continues to draw as Cairo would normally, but instead uses an CGAffineTransform in the tiles and some CGRect translation when swapping buffers to get the same effect. Changes to OpenGL ================= To simplify things, removal of all NSOpenGL* related components have been removed and we strictly use the Core GL (CGL*) API. This probably should have been done long ago anyay. Most examples found in the browsers to use IOSurfaceRef with OpenGL are using Legacy GL and there is still work underway to make this fit in with the rest of how the GSK GL renderer works. Since IOSurfaceRef bound to a texture/framebuffer will not have a default framebuffer ID of 0, we needed to add a default framebuffer id to the GdkGLContext. GskGLRenderer can use this to setup the command queue in such a way that our IOSurface destination has been glBindFramebuffer() as if it were the default drawable. This stuff is pretty slight-of-hand, so where things are and what needs flushing when and where has been a bit of an experiment to see what actually works to get synchronization across subsystems. Efficient Damages ================= After we draw with Cairo, we unlock the IOSurfaceRef and the contents are uploaded to the GPU. To make the contents visible to the app, we must clear the tiles contents with `layer.contents=nil;` and then re-apply the IOSurfaceRef. Since the buffer has likely not changed, we only do this if the tile overlaps the damage region. This gives the effect of having more tightly controlled damage regions even though updating the layer would damage be the whole window (as it is with OpenGL/Metal today with the exception of scissor-rect). This too can be verified usign "Quartz Debug -> Flash screen udpates". Frame Synchronized Resize ========================= In GTK 4, we have the ability to perform sizing changes from compute-size during the layout phase. Since the macOS backend already tracks window resizes manually, we can avoid doing the setFrame: immediately and instead do it within the frame clock's layout phase. Doing so gives us vastly better resize experience as we're more likely to get the size-change and updated-contents in the same frame on screen. It makes things feel "connected" in a way they weren't before. Some additional effort to tweak gravity during the process is also necessary but we were already doing that in the GTK 4 backend. Backporting =========== The design here has made an attempt to make it possible to backport by keeping GdkMacosBuffer, GdkMacosLayer, and GdkMacosTile fairly independent. There may be an opportunity to integrate this into GTK 3's quartz backend with a fair bit of work. Doing so could improve the situation for applications which are damage-rich such as The GIMP.
2022-02-14 10:20:19 +00:00
_gdk_macos_cairo_context_cairo_create (GdkCairoContext *cairo_context)
macos: use CGContext to draw cairo surfaces Instead of relying on cairo_t to perform drawing from our backing image surface to the Core Graphics context, we can convert the cairo_image_surface_t into a CGImageRef without having to copy data if we are certain of the alignment of the image up front. Without this, there are many situations, based on the size of the window that could cause cairo to take a slow path and malloc/copy the data to ensure that alignment. The previous commit titled "macos: align image surface rowstride to 16-bytes" ensures that this invariant is true so that our drawing code can assume we can reference the framebuffer from the cairo_image_surface_t using a CGDataProvider. Since GdkMacosCairoContext and GdkMacosCairoSubview are coordinating, we can also setup the transformation/scale early when drawing the cairo_image_surface_t instead of when copying it to Core Graphics. Furthermore, the CGImageRef is created with an RGB colorspace so that we are not performing colorspace conversion to the output device. We don't get color matching between displays, but we don't expect that anyway, particularly with the software renderer.
2022-02-04 02:31:09 +00:00
{
macos: use CALayer and IOSurface for rendering This provides a major shift in how we draw both when accelerated OpenGL as well as software rendering with Cairo. In short, it uses tiles of Core Animation's CALayer to display contents from an OpenGL or Cairo rendering so that the window can provide partial damage updates. Partial damage is not generally available when using OpenGL as the whole buffer is flipped even if you only submitted a small change using a scissor rect. Thankfully, this speeds up Cairo rendering a bit too by using IOSurface to upload contents to the display server. We use the tiling system we do for OpenGL which reduces overall complexity and differences between them. A New Buffer ============ GdkMacosBuffer is a wrapper around an IOSurfaceRef. The term buffer was used because 1) surface is already used and 2) it loosely maps to a front/back buffer semantic. However, it appears that IOSurfaceRef contents are being retained in some fashion (likely in the compositor result) so we can update the same IOSurfaceRef without flipping as long as we're fast. This appears to be what Chromium does as well, but Firefox uses two IOSurfaceRef and flips between them. We would like to avoid two surfaces because it doubles the GPU VRAM requirements of the application. Changes to Windows ================== Previously, the NSWindow would dynamically change between different types of NSView based on the renderer being used. This is no longer necessary as we just have a single NSView type, GdkMacosView, which inherits from GdkMacosBaseView just to keep the tedius stuff separate from the machinery of GdkMacosView. We can merge those someday if we are okay with that. Changes to Views ================ GdkMacosCairoView, GdkMacosCairoSubView, GdkMacosGLView have all been removed and replaced with GdkMacosView. This new view has a single CALayer (GdkMacosLayer) attached to it which itself has sublayers. The contents of the CALayer is populated with an IOSurfaceRef which we allocated with the GdkMacosSurface. The surface is replaced when the NSWindow resizes. Changes to Layers ================= We now have a dedicated GdkMacosLayer which contains sublayers of GdkMacosTile. The tile has a maximum size of 128x128 pixels in device units. The GdkMacosTile is partitioned by splitting both the transparent region (window bounds minus opaque area) and then by splitting the opaque area. A tile has either translucent contents (and therefore is not opaque) or has opaque contents (and therefore is opaque). An opaque tile never contains transparent contents. As such, the opaque tiles contain a black background so that Core Animation will consider the tile's bounds as opaque. This can be verified with "Quartz Debug -> Show opaque regions". Changes to Cairo ================ GTK 4 cannot currently use cairo-quartz because of how CSS borders are rendered. It simply causes errors in the cairo_quartz_surface_t backend. Since we are restricted to using cairo_image_surface_t (which happens to be faster anyway) we can use the IOSurfaceBaseAddress() to obtain a mapping of the IOSurfaceRef in user-space. It always uses BGRA 32-bit with alpha channel even if we will discard the alpha channel as that is necessary to hit the fast paths in other parts of the platform. Note that while Cairo says CAIRO_FORMAT_ARGB32, it is really 32-bit BGRA on little-endian as we expect. OpenGL will render flipped (Quartz Native Co-ordinates) while Cairo renders with 0,O in the top-left. We could use cairo_translate() and cairo_scale() to reverse this, but it looks like some cairo things may not look quite as right if we do so. To reduce the chances of one-off bugs this continues to draw as Cairo would normally, but instead uses an CGAffineTransform in the tiles and some CGRect translation when swapping buffers to get the same effect. Changes to OpenGL ================= To simplify things, removal of all NSOpenGL* related components have been removed and we strictly use the Core GL (CGL*) API. This probably should have been done long ago anyay. Most examples found in the browsers to use IOSurfaceRef with OpenGL are using Legacy GL and there is still work underway to make this fit in with the rest of how the GSK GL renderer works. Since IOSurfaceRef bound to a texture/framebuffer will not have a default framebuffer ID of 0, we needed to add a default framebuffer id to the GdkGLContext. GskGLRenderer can use this to setup the command queue in such a way that our IOSurface destination has been glBindFramebuffer() as if it were the default drawable. This stuff is pretty slight-of-hand, so where things are and what needs flushing when and where has been a bit of an experiment to see what actually works to get synchronization across subsystems. Efficient Damages ================= After we draw with Cairo, we unlock the IOSurfaceRef and the contents are uploaded to the GPU. To make the contents visible to the app, we must clear the tiles contents with `layer.contents=nil;` and then re-apply the IOSurfaceRef. Since the buffer has likely not changed, we only do this if the tile overlaps the damage region. This gives the effect of having more tightly controlled damage regions even though updating the layer would damage be the whole window (as it is with OpenGL/Metal today with the exception of scissor-rect). This too can be verified usign "Quartz Debug -> Flash screen udpates". Frame Synchronized Resize ========================= In GTK 4, we have the ability to perform sizing changes from compute-size during the layout phase. Since the macOS backend already tracks window resizes manually, we can avoid doing the setFrame: immediately and instead do it within the frame clock's layout phase. Doing so gives us vastly better resize experience as we're more likely to get the size-change and updated-contents in the same frame on screen. It makes things feel "connected" in a way they weren't before. Some additional effort to tweak gravity during the process is also necessary but we were already doing that in the GTK 4 backend. Backporting =========== The design here has made an attempt to make it possible to backport by keeping GdkMacosBuffer, GdkMacosLayer, and GdkMacosTile fairly independent. There may be an opportunity to integrate this into GTK 3's quartz backend with a fair bit of work. Doing so could improve the situation for applications which are damage-rich such as The GIMP.
2022-02-14 10:20:19 +00:00
GdkMacosCairoContext *self = (GdkMacosCairoContext *)cairo_context;
const cairo_region_t *damage;
cairo_surface_t *image_surface;
GdkMacosBuffer *buffer;
macos: use CGContext to draw cairo surfaces Instead of relying on cairo_t to perform drawing from our backing image surface to the Core Graphics context, we can convert the cairo_image_surface_t into a CGImageRef without having to copy data if we are certain of the alignment of the image up front. Without this, there are many situations, based on the size of the window that could cause cairo to take a slow path and malloc/copy the data to ensure that alignment. The previous commit titled "macos: align image surface rowstride to 16-bytes" ensures that this invariant is true so that our drawing code can assume we can reference the framebuffer from the cairo_image_surface_t using a CGDataProvider. Since GdkMacosCairoContext and GdkMacosCairoSubview are coordinating, we can also setup the transformation/scale early when drawing the cairo_image_surface_t instead of when copying it to Core Graphics. Furthermore, the CGImageRef is created with an RGB colorspace so that we are not performing colorspace conversion to the output device. We don't get color matching between displays, but we don't expect that anyway, particularly with the software renderer.
2022-02-04 02:31:09 +00:00
GdkSurface *surface;
macos: use CALayer and IOSurface for rendering This provides a major shift in how we draw both when accelerated OpenGL as well as software rendering with Cairo. In short, it uses tiles of Core Animation's CALayer to display contents from an OpenGL or Cairo rendering so that the window can provide partial damage updates. Partial damage is not generally available when using OpenGL as the whole buffer is flipped even if you only submitted a small change using a scissor rect. Thankfully, this speeds up Cairo rendering a bit too by using IOSurface to upload contents to the display server. We use the tiling system we do for OpenGL which reduces overall complexity and differences between them. A New Buffer ============ GdkMacosBuffer is a wrapper around an IOSurfaceRef. The term buffer was used because 1) surface is already used and 2) it loosely maps to a front/back buffer semantic. However, it appears that IOSurfaceRef contents are being retained in some fashion (likely in the compositor result) so we can update the same IOSurfaceRef without flipping as long as we're fast. This appears to be what Chromium does as well, but Firefox uses two IOSurfaceRef and flips between them. We would like to avoid two surfaces because it doubles the GPU VRAM requirements of the application. Changes to Windows ================== Previously, the NSWindow would dynamically change between different types of NSView based on the renderer being used. This is no longer necessary as we just have a single NSView type, GdkMacosView, which inherits from GdkMacosBaseView just to keep the tedius stuff separate from the machinery of GdkMacosView. We can merge those someday if we are okay with that. Changes to Views ================ GdkMacosCairoView, GdkMacosCairoSubView, GdkMacosGLView have all been removed and replaced with GdkMacosView. This new view has a single CALayer (GdkMacosLayer) attached to it which itself has sublayers. The contents of the CALayer is populated with an IOSurfaceRef which we allocated with the GdkMacosSurface. The surface is replaced when the NSWindow resizes. Changes to Layers ================= We now have a dedicated GdkMacosLayer which contains sublayers of GdkMacosTile. The tile has a maximum size of 128x128 pixels in device units. The GdkMacosTile is partitioned by splitting both the transparent region (window bounds minus opaque area) and then by splitting the opaque area. A tile has either translucent contents (and therefore is not opaque) or has opaque contents (and therefore is opaque). An opaque tile never contains transparent contents. As such, the opaque tiles contain a black background so that Core Animation will consider the tile's bounds as opaque. This can be verified with "Quartz Debug -> Show opaque regions". Changes to Cairo ================ GTK 4 cannot currently use cairo-quartz because of how CSS borders are rendered. It simply causes errors in the cairo_quartz_surface_t backend. Since we are restricted to using cairo_image_surface_t (which happens to be faster anyway) we can use the IOSurfaceBaseAddress() to obtain a mapping of the IOSurfaceRef in user-space. It always uses BGRA 32-bit with alpha channel even if we will discard the alpha channel as that is necessary to hit the fast paths in other parts of the platform. Note that while Cairo says CAIRO_FORMAT_ARGB32, it is really 32-bit BGRA on little-endian as we expect. OpenGL will render flipped (Quartz Native Co-ordinates) while Cairo renders with 0,O in the top-left. We could use cairo_translate() and cairo_scale() to reverse this, but it looks like some cairo things may not look quite as right if we do so. To reduce the chances of one-off bugs this continues to draw as Cairo would normally, but instead uses an CGAffineTransform in the tiles and some CGRect translation when swapping buffers to get the same effect. Changes to OpenGL ================= To simplify things, removal of all NSOpenGL* related components have been removed and we strictly use the Core GL (CGL*) API. This probably should have been done long ago anyay. Most examples found in the browsers to use IOSurfaceRef with OpenGL are using Legacy GL and there is still work underway to make this fit in with the rest of how the GSK GL renderer works. Since IOSurfaceRef bound to a texture/framebuffer will not have a default framebuffer ID of 0, we needed to add a default framebuffer id to the GdkGLContext. GskGLRenderer can use this to setup the command queue in such a way that our IOSurface destination has been glBindFramebuffer() as if it were the default drawable. This stuff is pretty slight-of-hand, so where things are and what needs flushing when and where has been a bit of an experiment to see what actually works to get synchronization across subsystems. Efficient Damages ================= After we draw with Cairo, we unlock the IOSurfaceRef and the contents are uploaded to the GPU. To make the contents visible to the app, we must clear the tiles contents with `layer.contents=nil;` and then re-apply the IOSurfaceRef. Since the buffer has likely not changed, we only do this if the tile overlaps the damage region. This gives the effect of having more tightly controlled damage regions even though updating the layer would damage be the whole window (as it is with OpenGL/Metal today with the exception of scissor-rect). This too can be verified usign "Quartz Debug -> Flash screen udpates". Frame Synchronized Resize ========================= In GTK 4, we have the ability to perform sizing changes from compute-size during the layout phase. Since the macOS backend already tracks window resizes manually, we can avoid doing the setFrame: immediately and instead do it within the frame clock's layout phase. Doing so gives us vastly better resize experience as we're more likely to get the size-change and updated-contents in the same frame on screen. It makes things feel "connected" in a way they weren't before. Some additional effort to tweak gravity during the process is also necessary but we were already doing that in the GTK 4 backend. Backporting =========== The design here has made an attempt to make it possible to backport by keeping GdkMacosBuffer, GdkMacosLayer, and GdkMacosTile fairly independent. There may be an opportunity to integrate this into GTK 3's quartz backend with a fair bit of work. Doing so could improve the situation for applications which are damage-rich such as The GIMP.
2022-02-14 10:20:19 +00:00
NSWindow *nswindow;
macos: use CGContext to draw cairo surfaces Instead of relying on cairo_t to perform drawing from our backing image surface to the Core Graphics context, we can convert the cairo_image_surface_t into a CGImageRef without having to copy data if we are certain of the alignment of the image up front. Without this, there are many situations, based on the size of the window that could cause cairo to take a slow path and malloc/copy the data to ensure that alignment. The previous commit titled "macos: align image surface rowstride to 16-bytes" ensures that this invariant is true so that our drawing code can assume we can reference the framebuffer from the cairo_image_surface_t using a CGDataProvider. Since GdkMacosCairoContext and GdkMacosCairoSubview are coordinating, we can also setup the transformation/scale early when drawing the cairo_image_surface_t instead of when copying it to Core Graphics. Furthermore, the CGImageRef is created with an RGB colorspace so that we are not performing colorspace conversion to the output device. We don't get color matching between displays, but we don't expect that anyway, particularly with the software renderer.
2022-02-04 02:31:09 +00:00
cairo_t *cr;
macos: use CALayer and IOSurface for rendering This provides a major shift in how we draw both when accelerated OpenGL as well as software rendering with Cairo. In short, it uses tiles of Core Animation's CALayer to display contents from an OpenGL or Cairo rendering so that the window can provide partial damage updates. Partial damage is not generally available when using OpenGL as the whole buffer is flipped even if you only submitted a small change using a scissor rect. Thankfully, this speeds up Cairo rendering a bit too by using IOSurface to upload contents to the display server. We use the tiling system we do for OpenGL which reduces overall complexity and differences between them. A New Buffer ============ GdkMacosBuffer is a wrapper around an IOSurfaceRef. The term buffer was used because 1) surface is already used and 2) it loosely maps to a front/back buffer semantic. However, it appears that IOSurfaceRef contents are being retained in some fashion (likely in the compositor result) so we can update the same IOSurfaceRef without flipping as long as we're fast. This appears to be what Chromium does as well, but Firefox uses two IOSurfaceRef and flips between them. We would like to avoid two surfaces because it doubles the GPU VRAM requirements of the application. Changes to Windows ================== Previously, the NSWindow would dynamically change between different types of NSView based on the renderer being used. This is no longer necessary as we just have a single NSView type, GdkMacosView, which inherits from GdkMacosBaseView just to keep the tedius stuff separate from the machinery of GdkMacosView. We can merge those someday if we are okay with that. Changes to Views ================ GdkMacosCairoView, GdkMacosCairoSubView, GdkMacosGLView have all been removed and replaced with GdkMacosView. This new view has a single CALayer (GdkMacosLayer) attached to it which itself has sublayers. The contents of the CALayer is populated with an IOSurfaceRef which we allocated with the GdkMacosSurface. The surface is replaced when the NSWindow resizes. Changes to Layers ================= We now have a dedicated GdkMacosLayer which contains sublayers of GdkMacosTile. The tile has a maximum size of 128x128 pixels in device units. The GdkMacosTile is partitioned by splitting both the transparent region (window bounds minus opaque area) and then by splitting the opaque area. A tile has either translucent contents (and therefore is not opaque) or has opaque contents (and therefore is opaque). An opaque tile never contains transparent contents. As such, the opaque tiles contain a black background so that Core Animation will consider the tile's bounds as opaque. This can be verified with "Quartz Debug -> Show opaque regions". Changes to Cairo ================ GTK 4 cannot currently use cairo-quartz because of how CSS borders are rendered. It simply causes errors in the cairo_quartz_surface_t backend. Since we are restricted to using cairo_image_surface_t (which happens to be faster anyway) we can use the IOSurfaceBaseAddress() to obtain a mapping of the IOSurfaceRef in user-space. It always uses BGRA 32-bit with alpha channel even if we will discard the alpha channel as that is necessary to hit the fast paths in other parts of the platform. Note that while Cairo says CAIRO_FORMAT_ARGB32, it is really 32-bit BGRA on little-endian as we expect. OpenGL will render flipped (Quartz Native Co-ordinates) while Cairo renders with 0,O in the top-left. We could use cairo_translate() and cairo_scale() to reverse this, but it looks like some cairo things may not look quite as right if we do so. To reduce the chances of one-off bugs this continues to draw as Cairo would normally, but instead uses an CGAffineTransform in the tiles and some CGRect translation when swapping buffers to get the same effect. Changes to OpenGL ================= To simplify things, removal of all NSOpenGL* related components have been removed and we strictly use the Core GL (CGL*) API. This probably should have been done long ago anyay. Most examples found in the browsers to use IOSurfaceRef with OpenGL are using Legacy GL and there is still work underway to make this fit in with the rest of how the GSK GL renderer works. Since IOSurfaceRef bound to a texture/framebuffer will not have a default framebuffer ID of 0, we needed to add a default framebuffer id to the GdkGLContext. GskGLRenderer can use this to setup the command queue in such a way that our IOSurface destination has been glBindFramebuffer() as if it were the default drawable. This stuff is pretty slight-of-hand, so where things are and what needs flushing when and where has been a bit of an experiment to see what actually works to get synchronization across subsystems. Efficient Damages ================= After we draw with Cairo, we unlock the IOSurfaceRef and the contents are uploaded to the GPU. To make the contents visible to the app, we must clear the tiles contents with `layer.contents=nil;` and then re-apply the IOSurfaceRef. Since the buffer has likely not changed, we only do this if the tile overlaps the damage region. This gives the effect of having more tightly controlled damage regions even though updating the layer would damage be the whole window (as it is with OpenGL/Metal today with the exception of scissor-rect). This too can be verified usign "Quartz Debug -> Flash screen udpates". Frame Synchronized Resize ========================= In GTK 4, we have the ability to perform sizing changes from compute-size during the layout phase. Since the macOS backend already tracks window resizes manually, we can avoid doing the setFrame: immediately and instead do it within the frame clock's layout phase. Doing so gives us vastly better resize experience as we're more likely to get the size-change and updated-contents in the same frame on screen. It makes things feel "connected" in a way they weren't before. Some additional effort to tweak gravity during the process is also necessary but we were already doing that in the GTK 4 backend. Backporting =========== The design here has made an attempt to make it possible to backport by keeping GdkMacosBuffer, GdkMacosLayer, and GdkMacosTile fairly independent. There may be an opportunity to integrate this into GTK 3's quartz backend with a fair bit of work. Doing so could improve the situation for applications which are damage-rich such as The GIMP.
2022-02-14 10:20:19 +00:00
gpointer data;
double scale;
guint width;
guint height;
guint stride;
gboolean opaque;
macos: prototype new GDK backend for macOS This is fairly substantial rewrite of the GDK backend for quartz and renamed to macOS to allow for a greenfield implementation. Many things have come across from the quartz implementation fairly intact such as the eventloop integration design and discovery of event windows from the NSEvent. However much has been changed to fit in with the new GDK design and how removal of child GdkWindow have been completely eliminated. Furthermore, the new GdkPopup allows for regular NSWindow to be used to provide popovers unlike the previous implementation. The object design more closely follows the ideal for a GDK backend. Views have been broken out into subclasses so that we can support multiple GSK renderer paths such as GL and Cairo (and Metal in the future). However mixed mode GL and Cairo will not be supported. Currently only the Cairo renderer has been implemented. A new frame clock implementation using CVDisplayLink provides more accurate information about when to draw drawing the next frame. Some testing will need to be done here to understand the power implications of this. This implementation has also gained edge snapping for CSD windows. Some work was also done to ensure that CSD windows have opaque regions registered with the display server. ** This is still very much a work-in-progress ** Some outstanding work that needs to be done: - Finish a GL context for macOS and alternate NSView for GL rendering (possibly using speciailized CALayer for OpenGL). - Input rework to ensure that we don't loose remapping of keys that was dropped from GDK during GTK 4 development. - Make sure input methods continue to work. - Drag-n-Drop is still very much a work in progress - High resolution input scrolling needs various work in GDK to land first before we can plumb that to NSEvent. - gtk/ has a number of things based on GDK_WINDOWING_QUARTZ that need to be updated to use the macOS backend. But this is good enough to start playing with and breaking things which is what I'd like to see.
2020-04-23 23:36:46 +00:00
macos: use CGContext to draw cairo surfaces Instead of relying on cairo_t to perform drawing from our backing image surface to the Core Graphics context, we can convert the cairo_image_surface_t into a CGImageRef without having to copy data if we are certain of the alignment of the image up front. Without this, there are many situations, based on the size of the window that could cause cairo to take a slow path and malloc/copy the data to ensure that alignment. The previous commit titled "macos: align image surface rowstride to 16-bytes" ensures that this invariant is true so that our drawing code can assume we can reference the framebuffer from the cairo_image_surface_t using a CGDataProvider. Since GdkMacosCairoContext and GdkMacosCairoSubview are coordinating, we can also setup the transformation/scale early when drawing the cairo_image_surface_t instead of when copying it to Core Graphics. Furthermore, the CGImageRef is created with an RGB colorspace so that we are not performing colorspace conversion to the output device. We don't get color matching between displays, but we don't expect that anyway, particularly with the software renderer.
2022-02-04 02:31:09 +00:00
g_assert (GDK_IS_MACOS_CAIRO_CONTEXT (self));
macos: prototype new GDK backend for macOS This is fairly substantial rewrite of the GDK backend for quartz and renamed to macOS to allow for a greenfield implementation. Many things have come across from the quartz implementation fairly intact such as the eventloop integration design and discovery of event windows from the NSEvent. However much has been changed to fit in with the new GDK design and how removal of child GdkWindow have been completely eliminated. Furthermore, the new GdkPopup allows for regular NSWindow to be used to provide popovers unlike the previous implementation. The object design more closely follows the ideal for a GDK backend. Views have been broken out into subclasses so that we can support multiple GSK renderer paths such as GL and Cairo (and Metal in the future). However mixed mode GL and Cairo will not be supported. Currently only the Cairo renderer has been implemented. A new frame clock implementation using CVDisplayLink provides more accurate information about when to draw drawing the next frame. Some testing will need to be done here to understand the power implications of this. This implementation has also gained edge snapping for CSD windows. Some work was also done to ensure that CSD windows have opaque regions registered with the display server. ** This is still very much a work-in-progress ** Some outstanding work that needs to be done: - Finish a GL context for macOS and alternate NSView for GL rendering (possibly using speciailized CALayer for OpenGL). - Input rework to ensure that we don't loose remapping of keys that was dropped from GDK during GTK 4 development. - Make sure input methods continue to work. - Drag-n-Drop is still very much a work in progress - High resolution input scrolling needs various work in GDK to land first before we can plumb that to NSEvent. - gtk/ has a number of things based on GDK_WINDOWING_QUARTZ that need to be updated to use the macOS backend. But this is good enough to start playing with and breaking things which is what I'd like to see.
2020-04-23 23:36:46 +00:00
macos: use CGContext to draw cairo surfaces Instead of relying on cairo_t to perform drawing from our backing image surface to the Core Graphics context, we can convert the cairo_image_surface_t into a CGImageRef without having to copy data if we are certain of the alignment of the image up front. Without this, there are many situations, based on the size of the window that could cause cairo to take a slow path and malloc/copy the data to ensure that alignment. The previous commit titled "macos: align image surface rowstride to 16-bytes" ensures that this invariant is true so that our drawing code can assume we can reference the framebuffer from the cairo_image_surface_t using a CGDataProvider. Since GdkMacosCairoContext and GdkMacosCairoSubview are coordinating, we can also setup the transformation/scale early when drawing the cairo_image_surface_t instead of when copying it to Core Graphics. Furthermore, the CGImageRef is created with an RGB colorspace so that we are not performing colorspace conversion to the output device. We don't get color matching between displays, but we don't expect that anyway, particularly with the software renderer.
2022-02-04 02:31:09 +00:00
surface = gdk_draw_context_get_surface (GDK_DRAW_CONTEXT (self));
macos: use CALayer and IOSurface for rendering This provides a major shift in how we draw both when accelerated OpenGL as well as software rendering with Cairo. In short, it uses tiles of Core Animation's CALayer to display contents from an OpenGL or Cairo rendering so that the window can provide partial damage updates. Partial damage is not generally available when using OpenGL as the whole buffer is flipped even if you only submitted a small change using a scissor rect. Thankfully, this speeds up Cairo rendering a bit too by using IOSurface to upload contents to the display server. We use the tiling system we do for OpenGL which reduces overall complexity and differences between them. A New Buffer ============ GdkMacosBuffer is a wrapper around an IOSurfaceRef. The term buffer was used because 1) surface is already used and 2) it loosely maps to a front/back buffer semantic. However, it appears that IOSurfaceRef contents are being retained in some fashion (likely in the compositor result) so we can update the same IOSurfaceRef without flipping as long as we're fast. This appears to be what Chromium does as well, but Firefox uses two IOSurfaceRef and flips between them. We would like to avoid two surfaces because it doubles the GPU VRAM requirements of the application. Changes to Windows ================== Previously, the NSWindow would dynamically change between different types of NSView based on the renderer being used. This is no longer necessary as we just have a single NSView type, GdkMacosView, which inherits from GdkMacosBaseView just to keep the tedius stuff separate from the machinery of GdkMacosView. We can merge those someday if we are okay with that. Changes to Views ================ GdkMacosCairoView, GdkMacosCairoSubView, GdkMacosGLView have all been removed and replaced with GdkMacosView. This new view has a single CALayer (GdkMacosLayer) attached to it which itself has sublayers. The contents of the CALayer is populated with an IOSurfaceRef which we allocated with the GdkMacosSurface. The surface is replaced when the NSWindow resizes. Changes to Layers ================= We now have a dedicated GdkMacosLayer which contains sublayers of GdkMacosTile. The tile has a maximum size of 128x128 pixels in device units. The GdkMacosTile is partitioned by splitting both the transparent region (window bounds minus opaque area) and then by splitting the opaque area. A tile has either translucent contents (and therefore is not opaque) or has opaque contents (and therefore is opaque). An opaque tile never contains transparent contents. As such, the opaque tiles contain a black background so that Core Animation will consider the tile's bounds as opaque. This can be verified with "Quartz Debug -> Show opaque regions". Changes to Cairo ================ GTK 4 cannot currently use cairo-quartz because of how CSS borders are rendered. It simply causes errors in the cairo_quartz_surface_t backend. Since we are restricted to using cairo_image_surface_t (which happens to be faster anyway) we can use the IOSurfaceBaseAddress() to obtain a mapping of the IOSurfaceRef in user-space. It always uses BGRA 32-bit with alpha channel even if we will discard the alpha channel as that is necessary to hit the fast paths in other parts of the platform. Note that while Cairo says CAIRO_FORMAT_ARGB32, it is really 32-bit BGRA on little-endian as we expect. OpenGL will render flipped (Quartz Native Co-ordinates) while Cairo renders with 0,O in the top-left. We could use cairo_translate() and cairo_scale() to reverse this, but it looks like some cairo things may not look quite as right if we do so. To reduce the chances of one-off bugs this continues to draw as Cairo would normally, but instead uses an CGAffineTransform in the tiles and some CGRect translation when swapping buffers to get the same effect. Changes to OpenGL ================= To simplify things, removal of all NSOpenGL* related components have been removed and we strictly use the Core GL (CGL*) API. This probably should have been done long ago anyay. Most examples found in the browsers to use IOSurfaceRef with OpenGL are using Legacy GL and there is still work underway to make this fit in with the rest of how the GSK GL renderer works. Since IOSurfaceRef bound to a texture/framebuffer will not have a default framebuffer ID of 0, we needed to add a default framebuffer id to the GdkGLContext. GskGLRenderer can use this to setup the command queue in such a way that our IOSurface destination has been glBindFramebuffer() as if it were the default drawable. This stuff is pretty slight-of-hand, so where things are and what needs flushing when and where has been a bit of an experiment to see what actually works to get synchronization across subsystems. Efficient Damages ================= After we draw with Cairo, we unlock the IOSurfaceRef and the contents are uploaded to the GPU. To make the contents visible to the app, we must clear the tiles contents with `layer.contents=nil;` and then re-apply the IOSurfaceRef. Since the buffer has likely not changed, we only do this if the tile overlaps the damage region. This gives the effect of having more tightly controlled damage regions even though updating the layer would damage be the whole window (as it is with OpenGL/Metal today with the exception of scissor-rect). This too can be verified usign "Quartz Debug -> Flash screen udpates". Frame Synchronized Resize ========================= In GTK 4, we have the ability to perform sizing changes from compute-size during the layout phase. Since the macOS backend already tracks window resizes manually, we can avoid doing the setFrame: immediately and instead do it within the frame clock's layout phase. Doing so gives us vastly better resize experience as we're more likely to get the size-change and updated-contents in the same frame on screen. It makes things feel "connected" in a way they weren't before. Some additional effort to tweak gravity during the process is also necessary but we were already doing that in the GTK 4 backend. Backporting =========== The design here has made an attempt to make it possible to backport by keeping GdkMacosBuffer, GdkMacosLayer, and GdkMacosTile fairly independent. There may be an opportunity to integrate this into GTK 3's quartz backend with a fair bit of work. Doing so could improve the situation for applications which are damage-rich such as The GIMP.
2022-02-14 10:20:19 +00:00
nswindow = _gdk_macos_surface_get_native (GDK_MACOS_SURFACE (surface));
opaque = [nswindow isOpaque];
buffer = _gdk_macos_surface_get_buffer (GDK_MACOS_SURFACE (surface));
damage = _gdk_macos_buffer_get_damage (buffer);
width = _gdk_macos_buffer_get_width (buffer);
height = _gdk_macos_buffer_get_height (buffer);
scale = _gdk_macos_buffer_get_device_scale (buffer);
stride = _gdk_macos_buffer_get_stride (buffer);
data = _gdk_macos_buffer_get_data (buffer);
/* Instead of forcing cairo to do everything through a CGContext,
* we just use an image surface backed by an IOSurfaceRef mapped
* into user-space. We can then use pixman which is quite fast as
* far as software rendering goes.
*
* Additionally, cairo_quartz_surface_t can't handle a number of
* tricks that the GSK cairo renderer does with border nodes and
* shadows, so an image surface is necessary for that.
*
* Since our IOSurfaceRef is width*scale-by-height*scale, we undo
* the scaling using cairo_surface_set_device_scale() so the renderer
* just thinks it's on a 2x scale surface for HiDPI.
*/
image_surface = cairo_image_surface_create_for_data (data,
CAIRO_FORMAT_ARGB32,
width,
height,
stride);
cairo_surface_set_device_scale (image_surface, scale, scale);
macos: fix cairo renderer with double buffering If we are double buffering surfaces with IOSurface then we need to copy the area that was damaged in the previous frame to the back buffer. This can be done with IOSurface but we need to hold the read-only lock so that we don't cause the underlying IOSurface contents to be invalidated. Additionally, since this is only used in the context of rendering to a GdkMacosSurface, we know the life-time of the cairo_surface_t and can simply lock/unlock the IOSurface buffer from begin_frame/end_frame to have the buffer flushing semantics we want. To ensure that we don't over damage, we store the damage in begin_frame (and copy it) and then subtract it from the next frames damage to determine the smallest amount we need to copy (taking scale factor into account). We don't care to modify the damage region to swapBuffers because they already have the right contents and could potentially fall into another tile anyway and we'd like to avoid damaging that. Fixes #4735
2022-02-28 09:42:48 +00:00
/* The buffer should already be locked at this point, and will
* be unlocked as part of end_frame.
*/
macos: use CALayer and IOSurface for rendering This provides a major shift in how we draw both when accelerated OpenGL as well as software rendering with Cairo. In short, it uses tiles of Core Animation's CALayer to display contents from an OpenGL or Cairo rendering so that the window can provide partial damage updates. Partial damage is not generally available when using OpenGL as the whole buffer is flipped even if you only submitted a small change using a scissor rect. Thankfully, this speeds up Cairo rendering a bit too by using IOSurface to upload contents to the display server. We use the tiling system we do for OpenGL which reduces overall complexity and differences between them. A New Buffer ============ GdkMacosBuffer is a wrapper around an IOSurfaceRef. The term buffer was used because 1) surface is already used and 2) it loosely maps to a front/back buffer semantic. However, it appears that IOSurfaceRef contents are being retained in some fashion (likely in the compositor result) so we can update the same IOSurfaceRef without flipping as long as we're fast. This appears to be what Chromium does as well, but Firefox uses two IOSurfaceRef and flips between them. We would like to avoid two surfaces because it doubles the GPU VRAM requirements of the application. Changes to Windows ================== Previously, the NSWindow would dynamically change between different types of NSView based on the renderer being used. This is no longer necessary as we just have a single NSView type, GdkMacosView, which inherits from GdkMacosBaseView just to keep the tedius stuff separate from the machinery of GdkMacosView. We can merge those someday if we are okay with that. Changes to Views ================ GdkMacosCairoView, GdkMacosCairoSubView, GdkMacosGLView have all been removed and replaced with GdkMacosView. This new view has a single CALayer (GdkMacosLayer) attached to it which itself has sublayers. The contents of the CALayer is populated with an IOSurfaceRef which we allocated with the GdkMacosSurface. The surface is replaced when the NSWindow resizes. Changes to Layers ================= We now have a dedicated GdkMacosLayer which contains sublayers of GdkMacosTile. The tile has a maximum size of 128x128 pixels in device units. The GdkMacosTile is partitioned by splitting both the transparent region (window bounds minus opaque area) and then by splitting the opaque area. A tile has either translucent contents (and therefore is not opaque) or has opaque contents (and therefore is opaque). An opaque tile never contains transparent contents. As such, the opaque tiles contain a black background so that Core Animation will consider the tile's bounds as opaque. This can be verified with "Quartz Debug -> Show opaque regions". Changes to Cairo ================ GTK 4 cannot currently use cairo-quartz because of how CSS borders are rendered. It simply causes errors in the cairo_quartz_surface_t backend. Since we are restricted to using cairo_image_surface_t (which happens to be faster anyway) we can use the IOSurfaceBaseAddress() to obtain a mapping of the IOSurfaceRef in user-space. It always uses BGRA 32-bit with alpha channel even if we will discard the alpha channel as that is necessary to hit the fast paths in other parts of the platform. Note that while Cairo says CAIRO_FORMAT_ARGB32, it is really 32-bit BGRA on little-endian as we expect. OpenGL will render flipped (Quartz Native Co-ordinates) while Cairo renders with 0,O in the top-left. We could use cairo_translate() and cairo_scale() to reverse this, but it looks like some cairo things may not look quite as right if we do so. To reduce the chances of one-off bugs this continues to draw as Cairo would normally, but instead uses an CGAffineTransform in the tiles and some CGRect translation when swapping buffers to get the same effect. Changes to OpenGL ================= To simplify things, removal of all NSOpenGL* related components have been removed and we strictly use the Core GL (CGL*) API. This probably should have been done long ago anyay. Most examples found in the browsers to use IOSurfaceRef with OpenGL are using Legacy GL and there is still work underway to make this fit in with the rest of how the GSK GL renderer works. Since IOSurfaceRef bound to a texture/framebuffer will not have a default framebuffer ID of 0, we needed to add a default framebuffer id to the GdkGLContext. GskGLRenderer can use this to setup the command queue in such a way that our IOSurface destination has been glBindFramebuffer() as if it were the default drawable. This stuff is pretty slight-of-hand, so where things are and what needs flushing when and where has been a bit of an experiment to see what actually works to get synchronization across subsystems. Efficient Damages ================= After we draw with Cairo, we unlock the IOSurfaceRef and the contents are uploaded to the GPU. To make the contents visible to the app, we must clear the tiles contents with `layer.contents=nil;` and then re-apply the IOSurfaceRef. Since the buffer has likely not changed, we only do this if the tile overlaps the damage region. This gives the effect of having more tightly controlled damage regions even though updating the layer would damage be the whole window (as it is with OpenGL/Metal today with the exception of scissor-rect). This too can be verified usign "Quartz Debug -> Flash screen udpates". Frame Synchronized Resize ========================= In GTK 4, we have the ability to perform sizing changes from compute-size during the layout phase. Since the macOS backend already tracks window resizes manually, we can avoid doing the setFrame: immediately and instead do it within the frame clock's layout phase. Doing so gives us vastly better resize experience as we're more likely to get the size-change and updated-contents in the same frame on screen. It makes things feel "connected" in a way they weren't before. Some additional effort to tweak gravity during the process is also necessary but we were already doing that in the GTK 4 backend. Backporting =========== The design here has made an attempt to make it possible to backport by keeping GdkMacosBuffer, GdkMacosLayer, and GdkMacosTile fairly independent. There may be an opportunity to integrate this into GTK 3's quartz backend with a fair bit of work. Doing so could improve the situation for applications which are damage-rich such as The GIMP.
2022-02-14 10:20:19 +00:00
if (!(cr = cairo_create (image_surface)))
goto failure;
/* Clip to the current damage region */
if (damage != NULL)
{
gdk_cairo_region (cr, damage);
cairo_clip (cr);
}
macos: prototype new GDK backend for macOS This is fairly substantial rewrite of the GDK backend for quartz and renamed to macOS to allow for a greenfield implementation. Many things have come across from the quartz implementation fairly intact such as the eventloop integration design and discovery of event windows from the NSEvent. However much has been changed to fit in with the new GDK design and how removal of child GdkWindow have been completely eliminated. Furthermore, the new GdkPopup allows for regular NSWindow to be used to provide popovers unlike the previous implementation. The object design more closely follows the ideal for a GDK backend. Views have been broken out into subclasses so that we can support multiple GSK renderer paths such as GL and Cairo (and Metal in the future). However mixed mode GL and Cairo will not be supported. Currently only the Cairo renderer has been implemented. A new frame clock implementation using CVDisplayLink provides more accurate information about when to draw drawing the next frame. Some testing will need to be done here to understand the power implications of this. This implementation has also gained edge snapping for CSD windows. Some work was also done to ensure that CSD windows have opaque regions registered with the display server. ** This is still very much a work-in-progress ** Some outstanding work that needs to be done: - Finish a GL context for macOS and alternate NSView for GL rendering (possibly using speciailized CALayer for OpenGL). - Input rework to ensure that we don't loose remapping of keys that was dropped from GDK during GTK 4 development. - Make sure input methods continue to work. - Drag-n-Drop is still very much a work in progress - High resolution input scrolling needs various work in GDK to land first before we can plumb that to NSEvent. - gtk/ has a number of things based on GDK_WINDOWING_QUARTZ that need to be updated to use the macOS backend. But this is good enough to start playing with and breaking things which is what I'd like to see.
2020-04-23 23:36:46 +00:00
macos: use CALayer and IOSurface for rendering This provides a major shift in how we draw both when accelerated OpenGL as well as software rendering with Cairo. In short, it uses tiles of Core Animation's CALayer to display contents from an OpenGL or Cairo rendering so that the window can provide partial damage updates. Partial damage is not generally available when using OpenGL as the whole buffer is flipped even if you only submitted a small change using a scissor rect. Thankfully, this speeds up Cairo rendering a bit too by using IOSurface to upload contents to the display server. We use the tiling system we do for OpenGL which reduces overall complexity and differences between them. A New Buffer ============ GdkMacosBuffer is a wrapper around an IOSurfaceRef. The term buffer was used because 1) surface is already used and 2) it loosely maps to a front/back buffer semantic. However, it appears that IOSurfaceRef contents are being retained in some fashion (likely in the compositor result) so we can update the same IOSurfaceRef without flipping as long as we're fast. This appears to be what Chromium does as well, but Firefox uses two IOSurfaceRef and flips between them. We would like to avoid two surfaces because it doubles the GPU VRAM requirements of the application. Changes to Windows ================== Previously, the NSWindow would dynamically change between different types of NSView based on the renderer being used. This is no longer necessary as we just have a single NSView type, GdkMacosView, which inherits from GdkMacosBaseView just to keep the tedius stuff separate from the machinery of GdkMacosView. We can merge those someday if we are okay with that. Changes to Views ================ GdkMacosCairoView, GdkMacosCairoSubView, GdkMacosGLView have all been removed and replaced with GdkMacosView. This new view has a single CALayer (GdkMacosLayer) attached to it which itself has sublayers. The contents of the CALayer is populated with an IOSurfaceRef which we allocated with the GdkMacosSurface. The surface is replaced when the NSWindow resizes. Changes to Layers ================= We now have a dedicated GdkMacosLayer which contains sublayers of GdkMacosTile. The tile has a maximum size of 128x128 pixels in device units. The GdkMacosTile is partitioned by splitting both the transparent region (window bounds minus opaque area) and then by splitting the opaque area. A tile has either translucent contents (and therefore is not opaque) or has opaque contents (and therefore is opaque). An opaque tile never contains transparent contents. As such, the opaque tiles contain a black background so that Core Animation will consider the tile's bounds as opaque. This can be verified with "Quartz Debug -> Show opaque regions". Changes to Cairo ================ GTK 4 cannot currently use cairo-quartz because of how CSS borders are rendered. It simply causes errors in the cairo_quartz_surface_t backend. Since we are restricted to using cairo_image_surface_t (which happens to be faster anyway) we can use the IOSurfaceBaseAddress() to obtain a mapping of the IOSurfaceRef in user-space. It always uses BGRA 32-bit with alpha channel even if we will discard the alpha channel as that is necessary to hit the fast paths in other parts of the platform. Note that while Cairo says CAIRO_FORMAT_ARGB32, it is really 32-bit BGRA on little-endian as we expect. OpenGL will render flipped (Quartz Native Co-ordinates) while Cairo renders with 0,O in the top-left. We could use cairo_translate() and cairo_scale() to reverse this, but it looks like some cairo things may not look quite as right if we do so. To reduce the chances of one-off bugs this continues to draw as Cairo would normally, but instead uses an CGAffineTransform in the tiles and some CGRect translation when swapping buffers to get the same effect. Changes to OpenGL ================= To simplify things, removal of all NSOpenGL* related components have been removed and we strictly use the Core GL (CGL*) API. This probably should have been done long ago anyay. Most examples found in the browsers to use IOSurfaceRef with OpenGL are using Legacy GL and there is still work underway to make this fit in with the rest of how the GSK GL renderer works. Since IOSurfaceRef bound to a texture/framebuffer will not have a default framebuffer ID of 0, we needed to add a default framebuffer id to the GdkGLContext. GskGLRenderer can use this to setup the command queue in such a way that our IOSurface destination has been glBindFramebuffer() as if it were the default drawable. This stuff is pretty slight-of-hand, so where things are and what needs flushing when and where has been a bit of an experiment to see what actually works to get synchronization across subsystems. Efficient Damages ================= After we draw with Cairo, we unlock the IOSurfaceRef and the contents are uploaded to the GPU. To make the contents visible to the app, we must clear the tiles contents with `layer.contents=nil;` and then re-apply the IOSurfaceRef. Since the buffer has likely not changed, we only do this if the tile overlaps the damage region. This gives the effect of having more tightly controlled damage regions even though updating the layer would damage be the whole window (as it is with OpenGL/Metal today with the exception of scissor-rect). This too can be verified usign "Quartz Debug -> Flash screen udpates". Frame Synchronized Resize ========================= In GTK 4, we have the ability to perform sizing changes from compute-size during the layout phase. Since the macOS backend already tracks window resizes manually, we can avoid doing the setFrame: immediately and instead do it within the frame clock's layout phase. Doing so gives us vastly better resize experience as we're more likely to get the size-change and updated-contents in the same frame on screen. It makes things feel "connected" in a way they weren't before. Some additional effort to tweak gravity during the process is also necessary but we were already doing that in the GTK 4 backend. Backporting =========== The design here has made an attempt to make it possible to backport by keeping GdkMacosBuffer, GdkMacosLayer, and GdkMacosTile fairly independent. There may be an opportunity to integrate this into GTK 3's quartz backend with a fair bit of work. Doing so could improve the situation for applications which are damage-rich such as The GIMP.
2022-02-14 10:20:19 +00:00
if (!opaque)
{
cairo_save (cr);
mac: Remove clear optimization 1. We want to get rid of exposing the opaque region 2. Cairo optimizes clearing the whole surface
2024-08-09 05:46:24 +00:00
cairo_set_operator (cr, CAIRO_OPERATOR_CLEAR);
cairo_paint (cr);
macos: use CALayer and IOSurface for rendering This provides a major shift in how we draw both when accelerated OpenGL as well as software rendering with Cairo. In short, it uses tiles of Core Animation's CALayer to display contents from an OpenGL or Cairo rendering so that the window can provide partial damage updates. Partial damage is not generally available when using OpenGL as the whole buffer is flipped even if you only submitted a small change using a scissor rect. Thankfully, this speeds up Cairo rendering a bit too by using IOSurface to upload contents to the display server. We use the tiling system we do for OpenGL which reduces overall complexity and differences between them. A New Buffer ============ GdkMacosBuffer is a wrapper around an IOSurfaceRef. The term buffer was used because 1) surface is already used and 2) it loosely maps to a front/back buffer semantic. However, it appears that IOSurfaceRef contents are being retained in some fashion (likely in the compositor result) so we can update the same IOSurfaceRef without flipping as long as we're fast. This appears to be what Chromium does as well, but Firefox uses two IOSurfaceRef and flips between them. We would like to avoid two surfaces because it doubles the GPU VRAM requirements of the application. Changes to Windows ================== Previously, the NSWindow would dynamically change between different types of NSView based on the renderer being used. This is no longer necessary as we just have a single NSView type, GdkMacosView, which inherits from GdkMacosBaseView just to keep the tedius stuff separate from the machinery of GdkMacosView. We can merge those someday if we are okay with that. Changes to Views ================ GdkMacosCairoView, GdkMacosCairoSubView, GdkMacosGLView have all been removed and replaced with GdkMacosView. This new view has a single CALayer (GdkMacosLayer) attached to it which itself has sublayers. The contents of the CALayer is populated with an IOSurfaceRef which we allocated with the GdkMacosSurface. The surface is replaced when the NSWindow resizes. Changes to Layers ================= We now have a dedicated GdkMacosLayer which contains sublayers of GdkMacosTile. The tile has a maximum size of 128x128 pixels in device units. The GdkMacosTile is partitioned by splitting both the transparent region (window bounds minus opaque area) and then by splitting the opaque area. A tile has either translucent contents (and therefore is not opaque) or has opaque contents (and therefore is opaque). An opaque tile never contains transparent contents. As such, the opaque tiles contain a black background so that Core Animation will consider the tile's bounds as opaque. This can be verified with "Quartz Debug -> Show opaque regions". Changes to Cairo ================ GTK 4 cannot currently use cairo-quartz because of how CSS borders are rendered. It simply causes errors in the cairo_quartz_surface_t backend. Since we are restricted to using cairo_image_surface_t (which happens to be faster anyway) we can use the IOSurfaceBaseAddress() to obtain a mapping of the IOSurfaceRef in user-space. It always uses BGRA 32-bit with alpha channel even if we will discard the alpha channel as that is necessary to hit the fast paths in other parts of the platform. Note that while Cairo says CAIRO_FORMAT_ARGB32, it is really 32-bit BGRA on little-endian as we expect. OpenGL will render flipped (Quartz Native Co-ordinates) while Cairo renders with 0,O in the top-left. We could use cairo_translate() and cairo_scale() to reverse this, but it looks like some cairo things may not look quite as right if we do so. To reduce the chances of one-off bugs this continues to draw as Cairo would normally, but instead uses an CGAffineTransform in the tiles and some CGRect translation when swapping buffers to get the same effect. Changes to OpenGL ================= To simplify things, removal of all NSOpenGL* related components have been removed and we strictly use the Core GL (CGL*) API. This probably should have been done long ago anyay. Most examples found in the browsers to use IOSurfaceRef with OpenGL are using Legacy GL and there is still work underway to make this fit in with the rest of how the GSK GL renderer works. Since IOSurfaceRef bound to a texture/framebuffer will not have a default framebuffer ID of 0, we needed to add a default framebuffer id to the GdkGLContext. GskGLRenderer can use this to setup the command queue in such a way that our IOSurface destination has been glBindFramebuffer() as if it were the default drawable. This stuff is pretty slight-of-hand, so where things are and what needs flushing when and where has been a bit of an experiment to see what actually works to get synchronization across subsystems. Efficient Damages ================= After we draw with Cairo, we unlock the IOSurfaceRef and the contents are uploaded to the GPU. To make the contents visible to the app, we must clear the tiles contents with `layer.contents=nil;` and then re-apply the IOSurfaceRef. Since the buffer has likely not changed, we only do this if the tile overlaps the damage region. This gives the effect of having more tightly controlled damage regions even though updating the layer would damage be the whole window (as it is with OpenGL/Metal today with the exception of scissor-rect). This too can be verified usign "Quartz Debug -> Flash screen udpates". Frame Synchronized Resize ========================= In GTK 4, we have the ability to perform sizing changes from compute-size during the layout phase. Since the macOS backend already tracks window resizes manually, we can avoid doing the setFrame: immediately and instead do it within the frame clock's layout phase. Doing so gives us vastly better resize experience as we're more likely to get the size-change and updated-contents in the same frame on screen. It makes things feel "connected" in a way they weren't before. Some additional effort to tweak gravity during the process is also necessary but we were already doing that in the GTK 4 backend. Backporting =========== The design here has made an attempt to make it possible to backport by keeping GdkMacosBuffer, GdkMacosLayer, and GdkMacosTile fairly independent. There may be an opportunity to integrate this into GTK 3's quartz backend with a fair bit of work. Doing so could improve the situation for applications which are damage-rich such as The GIMP.
2022-02-14 10:20:19 +00:00
cairo_restore (cr);
}
macos: prototype new GDK backend for macOS This is fairly substantial rewrite of the GDK backend for quartz and renamed to macOS to allow for a greenfield implementation. Many things have come across from the quartz implementation fairly intact such as the eventloop integration design and discovery of event windows from the NSEvent. However much has been changed to fit in with the new GDK design and how removal of child GdkWindow have been completely eliminated. Furthermore, the new GdkPopup allows for regular NSWindow to be used to provide popovers unlike the previous implementation. The object design more closely follows the ideal for a GDK backend. Views have been broken out into subclasses so that we can support multiple GSK renderer paths such as GL and Cairo (and Metal in the future). However mixed mode GL and Cairo will not be supported. Currently only the Cairo renderer has been implemented. A new frame clock implementation using CVDisplayLink provides more accurate information about when to draw drawing the next frame. Some testing will need to be done here to understand the power implications of this. This implementation has also gained edge snapping for CSD windows. Some work was also done to ensure that CSD windows have opaque regions registered with the display server. ** This is still very much a work-in-progress ** Some outstanding work that needs to be done: - Finish a GL context for macOS and alternate NSView for GL rendering (possibly using speciailized CALayer for OpenGL). - Input rework to ensure that we don't loose remapping of keys that was dropped from GDK during GTK 4 development. - Make sure input methods continue to work. - Drag-n-Drop is still very much a work in progress - High resolution input scrolling needs various work in GDK to land first before we can plumb that to NSEvent. - gtk/ has a number of things based on GDK_WINDOWING_QUARTZ that need to be updated to use the macOS backend. But this is good enough to start playing with and breaking things which is what I'd like to see.
2020-04-23 23:36:46 +00:00
macos: use CALayer and IOSurface for rendering This provides a major shift in how we draw both when accelerated OpenGL as well as software rendering with Cairo. In short, it uses tiles of Core Animation's CALayer to display contents from an OpenGL or Cairo rendering so that the window can provide partial damage updates. Partial damage is not generally available when using OpenGL as the whole buffer is flipped even if you only submitted a small change using a scissor rect. Thankfully, this speeds up Cairo rendering a bit too by using IOSurface to upload contents to the display server. We use the tiling system we do for OpenGL which reduces overall complexity and differences between them. A New Buffer ============ GdkMacosBuffer is a wrapper around an IOSurfaceRef. The term buffer was used because 1) surface is already used and 2) it loosely maps to a front/back buffer semantic. However, it appears that IOSurfaceRef contents are being retained in some fashion (likely in the compositor result) so we can update the same IOSurfaceRef without flipping as long as we're fast. This appears to be what Chromium does as well, but Firefox uses two IOSurfaceRef and flips between them. We would like to avoid two surfaces because it doubles the GPU VRAM requirements of the application. Changes to Windows ================== Previously, the NSWindow would dynamically change between different types of NSView based on the renderer being used. This is no longer necessary as we just have a single NSView type, GdkMacosView, which inherits from GdkMacosBaseView just to keep the tedius stuff separate from the machinery of GdkMacosView. We can merge those someday if we are okay with that. Changes to Views ================ GdkMacosCairoView, GdkMacosCairoSubView, GdkMacosGLView have all been removed and replaced with GdkMacosView. This new view has a single CALayer (GdkMacosLayer) attached to it which itself has sublayers. The contents of the CALayer is populated with an IOSurfaceRef which we allocated with the GdkMacosSurface. The surface is replaced when the NSWindow resizes. Changes to Layers ================= We now have a dedicated GdkMacosLayer which contains sublayers of GdkMacosTile. The tile has a maximum size of 128x128 pixels in device units. The GdkMacosTile is partitioned by splitting both the transparent region (window bounds minus opaque area) and then by splitting the opaque area. A tile has either translucent contents (and therefore is not opaque) or has opaque contents (and therefore is opaque). An opaque tile never contains transparent contents. As such, the opaque tiles contain a black background so that Core Animation will consider the tile's bounds as opaque. This can be verified with "Quartz Debug -> Show opaque regions". Changes to Cairo ================ GTK 4 cannot currently use cairo-quartz because of how CSS borders are rendered. It simply causes errors in the cairo_quartz_surface_t backend. Since we are restricted to using cairo_image_surface_t (which happens to be faster anyway) we can use the IOSurfaceBaseAddress() to obtain a mapping of the IOSurfaceRef in user-space. It always uses BGRA 32-bit with alpha channel even if we will discard the alpha channel as that is necessary to hit the fast paths in other parts of the platform. Note that while Cairo says CAIRO_FORMAT_ARGB32, it is really 32-bit BGRA on little-endian as we expect. OpenGL will render flipped (Quartz Native Co-ordinates) while Cairo renders with 0,O in the top-left. We could use cairo_translate() and cairo_scale() to reverse this, but it looks like some cairo things may not look quite as right if we do so. To reduce the chances of one-off bugs this continues to draw as Cairo would normally, but instead uses an CGAffineTransform in the tiles and some CGRect translation when swapping buffers to get the same effect. Changes to OpenGL ================= To simplify things, removal of all NSOpenGL* related components have been removed and we strictly use the Core GL (CGL*) API. This probably should have been done long ago anyay. Most examples found in the browsers to use IOSurfaceRef with OpenGL are using Legacy GL and there is still work underway to make this fit in with the rest of how the GSK GL renderer works. Since IOSurfaceRef bound to a texture/framebuffer will not have a default framebuffer ID of 0, we needed to add a default framebuffer id to the GdkGLContext. GskGLRenderer can use this to setup the command queue in such a way that our IOSurface destination has been glBindFramebuffer() as if it were the default drawable. This stuff is pretty slight-of-hand, so where things are and what needs flushing when and where has been a bit of an experiment to see what actually works to get synchronization across subsystems. Efficient Damages ================= After we draw with Cairo, we unlock the IOSurfaceRef and the contents are uploaded to the GPU. To make the contents visible to the app, we must clear the tiles contents with `layer.contents=nil;` and then re-apply the IOSurfaceRef. Since the buffer has likely not changed, we only do this if the tile overlaps the damage region. This gives the effect of having more tightly controlled damage regions even though updating the layer would damage be the whole window (as it is with OpenGL/Metal today with the exception of scissor-rect). This too can be verified usign "Quartz Debug -> Flash screen udpates". Frame Synchronized Resize ========================= In GTK 4, we have the ability to perform sizing changes from compute-size during the layout phase. Since the macOS backend already tracks window resizes manually, we can avoid doing the setFrame: immediately and instead do it within the frame clock's layout phase. Doing so gives us vastly better resize experience as we're more likely to get the size-change and updated-contents in the same frame on screen. It makes things feel "connected" in a way they weren't before. Some additional effort to tweak gravity during the process is also necessary but we were already doing that in the GTK 4 backend. Backporting =========== The design here has made an attempt to make it possible to backport by keeping GdkMacosBuffer, GdkMacosLayer, and GdkMacosTile fairly independent. There may be an opportunity to integrate this into GTK 3's quartz backend with a fair bit of work. Doing so could improve the situation for applications which are damage-rich such as The GIMP.
2022-02-14 10:20:19 +00:00
failure:
cairo_surface_destroy (image_surface);
macos: avoid unnecessarily destroying cairo_t This avoids destroying a cairo_t when we are within the frame as it otherwise causes the surface to unnecessarily flush.
2022-02-03 21:01:46 +00:00
macos: use CALayer and IOSurface for rendering This provides a major shift in how we draw both when accelerated OpenGL as well as software rendering with Cairo. In short, it uses tiles of Core Animation's CALayer to display contents from an OpenGL or Cairo rendering so that the window can provide partial damage updates. Partial damage is not generally available when using OpenGL as the whole buffer is flipped even if you only submitted a small change using a scissor rect. Thankfully, this speeds up Cairo rendering a bit too by using IOSurface to upload contents to the display server. We use the tiling system we do for OpenGL which reduces overall complexity and differences between them. A New Buffer ============ GdkMacosBuffer is a wrapper around an IOSurfaceRef. The term buffer was used because 1) surface is already used and 2) it loosely maps to a front/back buffer semantic. However, it appears that IOSurfaceRef contents are being retained in some fashion (likely in the compositor result) so we can update the same IOSurfaceRef without flipping as long as we're fast. This appears to be what Chromium does as well, but Firefox uses two IOSurfaceRef and flips between them. We would like to avoid two surfaces because it doubles the GPU VRAM requirements of the application. Changes to Windows ================== Previously, the NSWindow would dynamically change between different types of NSView based on the renderer being used. This is no longer necessary as we just have a single NSView type, GdkMacosView, which inherits from GdkMacosBaseView just to keep the tedius stuff separate from the machinery of GdkMacosView. We can merge those someday if we are okay with that. Changes to Views ================ GdkMacosCairoView, GdkMacosCairoSubView, GdkMacosGLView have all been removed and replaced with GdkMacosView. This new view has a single CALayer (GdkMacosLayer) attached to it which itself has sublayers. The contents of the CALayer is populated with an IOSurfaceRef which we allocated with the GdkMacosSurface. The surface is replaced when the NSWindow resizes. Changes to Layers ================= We now have a dedicated GdkMacosLayer which contains sublayers of GdkMacosTile. The tile has a maximum size of 128x128 pixels in device units. The GdkMacosTile is partitioned by splitting both the transparent region (window bounds minus opaque area) and then by splitting the opaque area. A tile has either translucent contents (and therefore is not opaque) or has opaque contents (and therefore is opaque). An opaque tile never contains transparent contents. As such, the opaque tiles contain a black background so that Core Animation will consider the tile's bounds as opaque. This can be verified with "Quartz Debug -> Show opaque regions". Changes to Cairo ================ GTK 4 cannot currently use cairo-quartz because of how CSS borders are rendered. It simply causes errors in the cairo_quartz_surface_t backend. Since we are restricted to using cairo_image_surface_t (which happens to be faster anyway) we can use the IOSurfaceBaseAddress() to obtain a mapping of the IOSurfaceRef in user-space. It always uses BGRA 32-bit with alpha channel even if we will discard the alpha channel as that is necessary to hit the fast paths in other parts of the platform. Note that while Cairo says CAIRO_FORMAT_ARGB32, it is really 32-bit BGRA on little-endian as we expect. OpenGL will render flipped (Quartz Native Co-ordinates) while Cairo renders with 0,O in the top-left. We could use cairo_translate() and cairo_scale() to reverse this, but it looks like some cairo things may not look quite as right if we do so. To reduce the chances of one-off bugs this continues to draw as Cairo would normally, but instead uses an CGAffineTransform in the tiles and some CGRect translation when swapping buffers to get the same effect. Changes to OpenGL ================= To simplify things, removal of all NSOpenGL* related components have been removed and we strictly use the Core GL (CGL*) API. This probably should have been done long ago anyay. Most examples found in the browsers to use IOSurfaceRef with OpenGL are using Legacy GL and there is still work underway to make this fit in with the rest of how the GSK GL renderer works. Since IOSurfaceRef bound to a texture/framebuffer will not have a default framebuffer ID of 0, we needed to add a default framebuffer id to the GdkGLContext. GskGLRenderer can use this to setup the command queue in such a way that our IOSurface destination has been glBindFramebuffer() as if it were the default drawable. This stuff is pretty slight-of-hand, so where things are and what needs flushing when and where has been a bit of an experiment to see what actually works to get synchronization across subsystems. Efficient Damages ================= After we draw with Cairo, we unlock the IOSurfaceRef and the contents are uploaded to the GPU. To make the contents visible to the app, we must clear the tiles contents with `layer.contents=nil;` and then re-apply the IOSurfaceRef. Since the buffer has likely not changed, we only do this if the tile overlaps the damage region. This gives the effect of having more tightly controlled damage regions even though updating the layer would damage be the whole window (as it is with OpenGL/Metal today with the exception of scissor-rect). This too can be verified usign "Quartz Debug -> Flash screen udpates". Frame Synchronized Resize ========================= In GTK 4, we have the ability to perform sizing changes from compute-size during the layout phase. Since the macOS backend already tracks window resizes manually, we can avoid doing the setFrame: immediately and instead do it within the frame clock's layout phase. Doing so gives us vastly better resize experience as we're more likely to get the size-change and updated-contents in the same frame on screen. It makes things feel "connected" in a way they weren't before. Some additional effort to tweak gravity during the process is also necessary but we were already doing that in the GTK 4 backend. Backporting =========== The design here has made an attempt to make it possible to backport by keeping GdkMacosBuffer, GdkMacosLayer, and GdkMacosTile fairly independent. There may be an opportunity to integrate this into GTK 3's quartz backend with a fair bit of work. Doing so could improve the situation for applications which are damage-rich such as The GIMP.
2022-02-14 10:20:19 +00:00
return cr;
macos: prototype new GDK backend for macOS This is fairly substantial rewrite of the GDK backend for quartz and renamed to macOS to allow for a greenfield implementation. Many things have come across from the quartz implementation fairly intact such as the eventloop integration design and discovery of event windows from the NSEvent. However much has been changed to fit in with the new GDK design and how removal of child GdkWindow have been completely eliminated. Furthermore, the new GdkPopup allows for regular NSWindow to be used to provide popovers unlike the previous implementation. The object design more closely follows the ideal for a GDK backend. Views have been broken out into subclasses so that we can support multiple GSK renderer paths such as GL and Cairo (and Metal in the future). However mixed mode GL and Cairo will not be supported. Currently only the Cairo renderer has been implemented. A new frame clock implementation using CVDisplayLink provides more accurate information about when to draw drawing the next frame. Some testing will need to be done here to understand the power implications of this. This implementation has also gained edge snapping for CSD windows. Some work was also done to ensure that CSD windows have opaque regions registered with the display server. ** This is still very much a work-in-progress ** Some outstanding work that needs to be done: - Finish a GL context for macOS and alternate NSView for GL rendering (possibly using speciailized CALayer for OpenGL). - Input rework to ensure that we don't loose remapping of keys that was dropped from GDK during GTK 4 development. - Make sure input methods continue to work. - Drag-n-Drop is still very much a work in progress - High resolution input scrolling needs various work in GDK to land first before we can plumb that to NSEvent. - gtk/ has a number of things based on GDK_WINDOWING_QUARTZ that need to be updated to use the macOS backend. But this is good enough to start playing with and breaking things which is what I'd like to see.
2020-04-23 23:36:46 +00:00
}
macos: fix cairo renderer with double buffering If we are double buffering surfaces with IOSurface then we need to copy the area that was damaged in the previous frame to the back buffer. This can be done with IOSurface but we need to hold the read-only lock so that we don't cause the underlying IOSurface contents to be invalidated. Additionally, since this is only used in the context of rendering to a GdkMacosSurface, we know the life-time of the cairo_surface_t and can simply lock/unlock the IOSurface buffer from begin_frame/end_frame to have the buffer flushing semantics we want. To ensure that we don't over damage, we store the damage in begin_frame (and copy it) and then subtract it from the next frames damage to determine the smallest amount we need to copy (taking scale factor into account). We don't care to modify the damage region to swapBuffers because they already have the right contents and could potentially fall into another tile anyway and we'd like to avoid damaging that. Fixes #4735
2022-02-28 09:42:48 +00:00
static void
copy_surface_data (GdkMacosBuffer *from,
GdkMacosBuffer *to,
const cairo_region_t *region,
int scale)
{
const guint8 *from_base;
guint8 *to_base;
guint from_stride;
guint to_stride;
guint n_rects;
g_assert (GDK_IS_MACOS_BUFFER (from));
g_assert (GDK_IS_MACOS_BUFFER (to));
g_assert (region != NULL);
g_assert (!cairo_region_is_empty (region));
from_base = _gdk_macos_buffer_get_data (from);
from_stride = _gdk_macos_buffer_get_stride (from);
to_base = _gdk_macos_buffer_get_data (to);
to_stride = _gdk_macos_buffer_get_stride (to);
n_rects = cairo_region_num_rectangles (region);
for (guint i = 0; i < n_rects; i++)
{
cairo_rectangle_int_t rect;
int y2;
cairo_region_get_rectangle (region, i, &rect);
rect.y *= scale;
rect.height *= scale;
rect.x *= scale;
rect.width *= scale;
y2 = rect.y + rect.height;
for (int y = rect.y; y < y2; y++)
memcpy (&to_base[y * to_stride + rect.x * 4],
&from_base[y * from_stride + rect.x * 4],
rect.width * 4);
}
}
macOS: Clamp damage region to surface size ...in _gdk_macos_cairo_context_begin_frame () GdkMacosCairoContext needs regions that are clamped to the actual surface size. Fixes https://gitlab.gnome.org/GNOME/gtk/-/issues/5812
2023-09-13 14:50:08 +00:00
static void
clamp_region_to_surface (cairo_region_t *region,
GdkSurface *surface)
{
cairo_rectangle_int_t rectangle = {0, 0, surface->width, surface->height};
cairo_region_intersect_rectangle (region, &rectangle);
}
macos: prototype new GDK backend for macOS This is fairly substantial rewrite of the GDK backend for quartz and renamed to macOS to allow for a greenfield implementation. Many things have come across from the quartz implementation fairly intact such as the eventloop integration design and discovery of event windows from the NSEvent. However much has been changed to fit in with the new GDK design and how removal of child GdkWindow have been completely eliminated. Furthermore, the new GdkPopup allows for regular NSWindow to be used to provide popovers unlike the previous implementation. The object design more closely follows the ideal for a GDK backend. Views have been broken out into subclasses so that we can support multiple GSK renderer paths such as GL and Cairo (and Metal in the future). However mixed mode GL and Cairo will not be supported. Currently only the Cairo renderer has been implemented. A new frame clock implementation using CVDisplayLink provides more accurate information about when to draw drawing the next frame. Some testing will need to be done here to understand the power implications of this. This implementation has also gained edge snapping for CSD windows. Some work was also done to ensure that CSD windows have opaque regions registered with the display server. ** This is still very much a work-in-progress ** Some outstanding work that needs to be done: - Finish a GL context for macOS and alternate NSView for GL rendering (possibly using speciailized CALayer for OpenGL). - Input rework to ensure that we don't loose remapping of keys that was dropped from GDK during GTK 4 development. - Make sure input methods continue to work. - Drag-n-Drop is still very much a work in progress - High resolution input scrolling needs various work in GDK to land first before we can plumb that to NSEvent. - gtk/ has a number of things based on GDK_WINDOWING_QUARTZ that need to be updated to use the macOS backend. But this is good enough to start playing with and breaking things which is what I'd like to see.
2020-04-23 23:36:46 +00:00
static void
drawcontext: Make begin_frame() set colorstate + depth Make begin_frame() set a rendering colorstate and depth, and provide it to the renderers via gdk_draw_context_get_depth() and gdk_draw_context_get_color_state(). This allows the draw contexts to define their own values, so that ie the Cairo and GL renderer can choose different settings for rendering (in particular, GL can choose GL_SRGB and do the srgb conversion; while Cairo relies on the renderer).
2024-06-30 02:06:48 +00:00
_gdk_macos_cairo_context_begin_frame (GdkDrawContext *draw_context,
GdkMemoryDepth depth,
cairo_region_t *region,
GdkColorState **out_color_state,
GdkMemoryDepth *out_depth)
macos: prototype new GDK backend for macOS This is fairly substantial rewrite of the GDK backend for quartz and renamed to macOS to allow for a greenfield implementation. Many things have come across from the quartz implementation fairly intact such as the eventloop integration design and discovery of event windows from the NSEvent. However much has been changed to fit in with the new GDK design and how removal of child GdkWindow have been completely eliminated. Furthermore, the new GdkPopup allows for regular NSWindow to be used to provide popovers unlike the previous implementation. The object design more closely follows the ideal for a GDK backend. Views have been broken out into subclasses so that we can support multiple GSK renderer paths such as GL and Cairo (and Metal in the future). However mixed mode GL and Cairo will not be supported. Currently only the Cairo renderer has been implemented. A new frame clock implementation using CVDisplayLink provides more accurate information about when to draw drawing the next frame. Some testing will need to be done here to understand the power implications of this. This implementation has also gained edge snapping for CSD windows. Some work was also done to ensure that CSD windows have opaque regions registered with the display server. ** This is still very much a work-in-progress ** Some outstanding work that needs to be done: - Finish a GL context for macOS and alternate NSView for GL rendering (possibly using speciailized CALayer for OpenGL). - Input rework to ensure that we don't loose remapping of keys that was dropped from GDK during GTK 4 development. - Make sure input methods continue to work. - Drag-n-Drop is still very much a work in progress - High resolution input scrolling needs various work in GDK to land first before we can plumb that to NSEvent. - gtk/ has a number of things based on GDK_WINDOWING_QUARTZ that need to be updated to use the macOS backend. But this is good enough to start playing with and breaking things which is what I'd like to see.
2020-04-23 23:36:46 +00:00
{
GdkMacosCairoContext *self = (GdkMacosCairoContext *)draw_context;
macos: use CALayer and IOSurface for rendering This provides a major shift in how we draw both when accelerated OpenGL as well as software rendering with Cairo. In short, it uses tiles of Core Animation's CALayer to display contents from an OpenGL or Cairo rendering so that the window can provide partial damage updates. Partial damage is not generally available when using OpenGL as the whole buffer is flipped even if you only submitted a small change using a scissor rect. Thankfully, this speeds up Cairo rendering a bit too by using IOSurface to upload contents to the display server. We use the tiling system we do for OpenGL which reduces overall complexity and differences between them. A New Buffer ============ GdkMacosBuffer is a wrapper around an IOSurfaceRef. The term buffer was used because 1) surface is already used and 2) it loosely maps to a front/back buffer semantic. However, it appears that IOSurfaceRef contents are being retained in some fashion (likely in the compositor result) so we can update the same IOSurfaceRef without flipping as long as we're fast. This appears to be what Chromium does as well, but Firefox uses two IOSurfaceRef and flips between them. We would like to avoid two surfaces because it doubles the GPU VRAM requirements of the application. Changes to Windows ================== Previously, the NSWindow would dynamically change between different types of NSView based on the renderer being used. This is no longer necessary as we just have a single NSView type, GdkMacosView, which inherits from GdkMacosBaseView just to keep the tedius stuff separate from the machinery of GdkMacosView. We can merge those someday if we are okay with that. Changes to Views ================ GdkMacosCairoView, GdkMacosCairoSubView, GdkMacosGLView have all been removed and replaced with GdkMacosView. This new view has a single CALayer (GdkMacosLayer) attached to it which itself has sublayers. The contents of the CALayer is populated with an IOSurfaceRef which we allocated with the GdkMacosSurface. The surface is replaced when the NSWindow resizes. Changes to Layers ================= We now have a dedicated GdkMacosLayer which contains sublayers of GdkMacosTile. The tile has a maximum size of 128x128 pixels in device units. The GdkMacosTile is partitioned by splitting both the transparent region (window bounds minus opaque area) and then by splitting the opaque area. A tile has either translucent contents (and therefore is not opaque) or has opaque contents (and therefore is opaque). An opaque tile never contains transparent contents. As such, the opaque tiles contain a black background so that Core Animation will consider the tile's bounds as opaque. This can be verified with "Quartz Debug -> Show opaque regions". Changes to Cairo ================ GTK 4 cannot currently use cairo-quartz because of how CSS borders are rendered. It simply causes errors in the cairo_quartz_surface_t backend. Since we are restricted to using cairo_image_surface_t (which happens to be faster anyway) we can use the IOSurfaceBaseAddress() to obtain a mapping of the IOSurfaceRef in user-space. It always uses BGRA 32-bit with alpha channel even if we will discard the alpha channel as that is necessary to hit the fast paths in other parts of the platform. Note that while Cairo says CAIRO_FORMAT_ARGB32, it is really 32-bit BGRA on little-endian as we expect. OpenGL will render flipped (Quartz Native Co-ordinates) while Cairo renders with 0,O in the top-left. We could use cairo_translate() and cairo_scale() to reverse this, but it looks like some cairo things may not look quite as right if we do so. To reduce the chances of one-off bugs this continues to draw as Cairo would normally, but instead uses an CGAffineTransform in the tiles and some CGRect translation when swapping buffers to get the same effect. Changes to OpenGL ================= To simplify things, removal of all NSOpenGL* related components have been removed and we strictly use the Core GL (CGL*) API. This probably should have been done long ago anyay. Most examples found in the browsers to use IOSurfaceRef with OpenGL are using Legacy GL and there is still work underway to make this fit in with the rest of how the GSK GL renderer works. Since IOSurfaceRef bound to a texture/framebuffer will not have a default framebuffer ID of 0, we needed to add a default framebuffer id to the GdkGLContext. GskGLRenderer can use this to setup the command queue in such a way that our IOSurface destination has been glBindFramebuffer() as if it were the default drawable. This stuff is pretty slight-of-hand, so where things are and what needs flushing when and where has been a bit of an experiment to see what actually works to get synchronization across subsystems. Efficient Damages ================= After we draw with Cairo, we unlock the IOSurfaceRef and the contents are uploaded to the GPU. To make the contents visible to the app, we must clear the tiles contents with `layer.contents=nil;` and then re-apply the IOSurfaceRef. Since the buffer has likely not changed, we only do this if the tile overlaps the damage region. This gives the effect of having more tightly controlled damage regions even though updating the layer would damage be the whole window (as it is with OpenGL/Metal today with the exception of scissor-rect). This too can be verified usign "Quartz Debug -> Flash screen udpates". Frame Synchronized Resize ========================= In GTK 4, we have the ability to perform sizing changes from compute-size during the layout phase. Since the macOS backend already tracks window resizes manually, we can avoid doing the setFrame: immediately and instead do it within the frame clock's layout phase. Doing so gives us vastly better resize experience as we're more likely to get the size-change and updated-contents in the same frame on screen. It makes things feel "connected" in a way they weren't before. Some additional effort to tweak gravity during the process is also necessary but we were already doing that in the GTK 4 backend. Backporting =========== The design here has made an attempt to make it possible to backport by keeping GdkMacosBuffer, GdkMacosLayer, and GdkMacosTile fairly independent. There may be an opportunity to integrate this into GTK 3's quartz backend with a fair bit of work. Doing so could improve the situation for applications which are damage-rich such as The GIMP.
2022-02-14 10:20:19 +00:00
GdkMacosBuffer *buffer;
macos: fix cairo renderer with double buffering If we are double buffering surfaces with IOSurface then we need to copy the area that was damaged in the previous frame to the back buffer. This can be done with IOSurface but we need to hold the read-only lock so that we don't cause the underlying IOSurface contents to be invalidated. Additionally, since this is only used in the context of rendering to a GdkMacosSurface, we know the life-time of the cairo_surface_t and can simply lock/unlock the IOSurface buffer from begin_frame/end_frame to have the buffer flushing semantics we want. To ensure that we don't over damage, we store the damage in begin_frame (and copy it) and then subtract it from the next frames damage to determine the smallest amount we need to copy (taking scale factor into account). We don't care to modify the damage region to swapBuffers because they already have the right contents and could potentially fall into another tile anyway and we'd like to avoid damaging that. Fixes #4735
2022-02-28 09:42:48 +00:00
GdkMacosSurface *surface;
macos: prototype new GDK backend for macOS This is fairly substantial rewrite of the GDK backend for quartz and renamed to macOS to allow for a greenfield implementation. Many things have come across from the quartz implementation fairly intact such as the eventloop integration design and discovery of event windows from the NSEvent. However much has been changed to fit in with the new GDK design and how removal of child GdkWindow have been completely eliminated. Furthermore, the new GdkPopup allows for regular NSWindow to be used to provide popovers unlike the previous implementation. The object design more closely follows the ideal for a GDK backend. Views have been broken out into subclasses so that we can support multiple GSK renderer paths such as GL and Cairo (and Metal in the future). However mixed mode GL and Cairo will not be supported. Currently only the Cairo renderer has been implemented. A new frame clock implementation using CVDisplayLink provides more accurate information about when to draw drawing the next frame. Some testing will need to be done here to understand the power implications of this. This implementation has also gained edge snapping for CSD windows. Some work was also done to ensure that CSD windows have opaque regions registered with the display server. ** This is still very much a work-in-progress ** Some outstanding work that needs to be done: - Finish a GL context for macOS and alternate NSView for GL rendering (possibly using speciailized CALayer for OpenGL). - Input rework to ensure that we don't loose remapping of keys that was dropped from GDK during GTK 4 development. - Make sure input methods continue to work. - Drag-n-Drop is still very much a work in progress - High resolution input scrolling needs various work in GDK to land first before we can plumb that to NSEvent. - gtk/ has a number of things based on GDK_WINDOWING_QUARTZ that need to be updated to use the macOS backend. But this is good enough to start playing with and breaking things which is what I'd like to see.
2020-04-23 23:36:46 +00:00
g_assert (GDK_IS_MACOS_CAIRO_CONTEXT (self));
macos: use CALayer and IOSurface for rendering This provides a major shift in how we draw both when accelerated OpenGL as well as software rendering with Cairo. In short, it uses tiles of Core Animation's CALayer to display contents from an OpenGL or Cairo rendering so that the window can provide partial damage updates. Partial damage is not generally available when using OpenGL as the whole buffer is flipped even if you only submitted a small change using a scissor rect. Thankfully, this speeds up Cairo rendering a bit too by using IOSurface to upload contents to the display server. We use the tiling system we do for OpenGL which reduces overall complexity and differences between them. A New Buffer ============ GdkMacosBuffer is a wrapper around an IOSurfaceRef. The term buffer was used because 1) surface is already used and 2) it loosely maps to a front/back buffer semantic. However, it appears that IOSurfaceRef contents are being retained in some fashion (likely in the compositor result) so we can update the same IOSurfaceRef without flipping as long as we're fast. This appears to be what Chromium does as well, but Firefox uses two IOSurfaceRef and flips between them. We would like to avoid two surfaces because it doubles the GPU VRAM requirements of the application. Changes to Windows ================== Previously, the NSWindow would dynamically change between different types of NSView based on the renderer being used. This is no longer necessary as we just have a single NSView type, GdkMacosView, which inherits from GdkMacosBaseView just to keep the tedius stuff separate from the machinery of GdkMacosView. We can merge those someday if we are okay with that. Changes to Views ================ GdkMacosCairoView, GdkMacosCairoSubView, GdkMacosGLView have all been removed and replaced with GdkMacosView. This new view has a single CALayer (GdkMacosLayer) attached to it which itself has sublayers. The contents of the CALayer is populated with an IOSurfaceRef which we allocated with the GdkMacosSurface. The surface is replaced when the NSWindow resizes. Changes to Layers ================= We now have a dedicated GdkMacosLayer which contains sublayers of GdkMacosTile. The tile has a maximum size of 128x128 pixels in device units. The GdkMacosTile is partitioned by splitting both the transparent region (window bounds minus opaque area) and then by splitting the opaque area. A tile has either translucent contents (and therefore is not opaque) or has opaque contents (and therefore is opaque). An opaque tile never contains transparent contents. As such, the opaque tiles contain a black background so that Core Animation will consider the tile's bounds as opaque. This can be verified with "Quartz Debug -> Show opaque regions". Changes to Cairo ================ GTK 4 cannot currently use cairo-quartz because of how CSS borders are rendered. It simply causes errors in the cairo_quartz_surface_t backend. Since we are restricted to using cairo_image_surface_t (which happens to be faster anyway) we can use the IOSurfaceBaseAddress() to obtain a mapping of the IOSurfaceRef in user-space. It always uses BGRA 32-bit with alpha channel even if we will discard the alpha channel as that is necessary to hit the fast paths in other parts of the platform. Note that while Cairo says CAIRO_FORMAT_ARGB32, it is really 32-bit BGRA on little-endian as we expect. OpenGL will render flipped (Quartz Native Co-ordinates) while Cairo renders with 0,O in the top-left. We could use cairo_translate() and cairo_scale() to reverse this, but it looks like some cairo things may not look quite as right if we do so. To reduce the chances of one-off bugs this continues to draw as Cairo would normally, but instead uses an CGAffineTransform in the tiles and some CGRect translation when swapping buffers to get the same effect. Changes to OpenGL ================= To simplify things, removal of all NSOpenGL* related components have been removed and we strictly use the Core GL (CGL*) API. This probably should have been done long ago anyay. Most examples found in the browsers to use IOSurfaceRef with OpenGL are using Legacy GL and there is still work underway to make this fit in with the rest of how the GSK GL renderer works. Since IOSurfaceRef bound to a texture/framebuffer will not have a default framebuffer ID of 0, we needed to add a default framebuffer id to the GdkGLContext. GskGLRenderer can use this to setup the command queue in such a way that our IOSurface destination has been glBindFramebuffer() as if it were the default drawable. This stuff is pretty slight-of-hand, so where things are and what needs flushing when and where has been a bit of an experiment to see what actually works to get synchronization across subsystems. Efficient Damages ================= After we draw with Cairo, we unlock the IOSurfaceRef and the contents are uploaded to the GPU. To make the contents visible to the app, we must clear the tiles contents with `layer.contents=nil;` and then re-apply the IOSurfaceRef. Since the buffer has likely not changed, we only do this if the tile overlaps the damage region. This gives the effect of having more tightly controlled damage regions even though updating the layer would damage be the whole window (as it is with OpenGL/Metal today with the exception of scissor-rect). This too can be verified usign "Quartz Debug -> Flash screen udpates". Frame Synchronized Resize ========================= In GTK 4, we have the ability to perform sizing changes from compute-size during the layout phase. Since the macOS backend already tracks window resizes manually, we can avoid doing the setFrame: immediately and instead do it within the frame clock's layout phase. Doing so gives us vastly better resize experience as we're more likely to get the size-change and updated-contents in the same frame on screen. It makes things feel "connected" in a way they weren't before. Some additional effort to tweak gravity during the process is also necessary but we were already doing that in the GTK 4 backend. Backporting =========== The design here has made an attempt to make it possible to backport by keeping GdkMacosBuffer, GdkMacosLayer, and GdkMacosTile fairly independent. There may be an opportunity to integrate this into GTK 3's quartz backend with a fair bit of work. Doing so could improve the situation for applications which are damage-rich such as The GIMP.
2022-02-14 10:20:19 +00:00
[CATransaction begin];
[CATransaction setDisableActions:YES];
macos: avoid unnecessarily destroying cairo_t This avoids destroying a cairo_t when we are within the frame as it otherwise causes the surface to unnecessarily flush.
2022-02-03 21:01:46 +00:00
macos: fix cairo renderer with double buffering If we are double buffering surfaces with IOSurface then we need to copy the area that was damaged in the previous frame to the back buffer. This can be done with IOSurface but we need to hold the read-only lock so that we don't cause the underlying IOSurface contents to be invalidated. Additionally, since this is only used in the context of rendering to a GdkMacosSurface, we know the life-time of the cairo_surface_t and can simply lock/unlock the IOSurface buffer from begin_frame/end_frame to have the buffer flushing semantics we want. To ensure that we don't over damage, we store the damage in begin_frame (and copy it) and then subtract it from the next frames damage to determine the smallest amount we need to copy (taking scale factor into account). We don't care to modify the damage region to swapBuffers because they already have the right contents and could potentially fall into another tile anyway and we'd like to avoid damaging that. Fixes #4735
2022-02-28 09:42:48 +00:00
surface = GDK_MACOS_SURFACE (gdk_draw_context_get_surface (draw_context));
buffer = _gdk_macos_surface_get_buffer (surface);
macos: avoid unnecessarily destroying cairo_t This avoids destroying a cairo_t when we are within the frame as it otherwise causes the surface to unnecessarily flush.
2022-02-03 21:01:46 +00:00
macOS: Clamp damage region to surface size ...in _gdk_macos_cairo_context_begin_frame () GdkMacosCairoContext needs regions that are clamped to the actual surface size. Fixes https://gitlab.gnome.org/GNOME/gtk/-/issues/5812
2023-09-13 14:50:08 +00:00
clamp_region_to_surface (region, GDK_SURFACE (surface));
macos: use CALayer and IOSurface for rendering This provides a major shift in how we draw both when accelerated OpenGL as well as software rendering with Cairo. In short, it uses tiles of Core Animation's CALayer to display contents from an OpenGL or Cairo rendering so that the window can provide partial damage updates. Partial damage is not generally available when using OpenGL as the whole buffer is flipped even if you only submitted a small change using a scissor rect. Thankfully, this speeds up Cairo rendering a bit too by using IOSurface to upload contents to the display server. We use the tiling system we do for OpenGL which reduces overall complexity and differences between them. A New Buffer ============ GdkMacosBuffer is a wrapper around an IOSurfaceRef. The term buffer was used because 1) surface is already used and 2) it loosely maps to a front/back buffer semantic. However, it appears that IOSurfaceRef contents are being retained in some fashion (likely in the compositor result) so we can update the same IOSurfaceRef without flipping as long as we're fast. This appears to be what Chromium does as well, but Firefox uses two IOSurfaceRef and flips between them. We would like to avoid two surfaces because it doubles the GPU VRAM requirements of the application. Changes to Windows ================== Previously, the NSWindow would dynamically change between different types of NSView based on the renderer being used. This is no longer necessary as we just have a single NSView type, GdkMacosView, which inherits from GdkMacosBaseView just to keep the tedius stuff separate from the machinery of GdkMacosView. We can merge those someday if we are okay with that. Changes to Views ================ GdkMacosCairoView, GdkMacosCairoSubView, GdkMacosGLView have all been removed and replaced with GdkMacosView. This new view has a single CALayer (GdkMacosLayer) attached to it which itself has sublayers. The contents of the CALayer is populated with an IOSurfaceRef which we allocated with the GdkMacosSurface. The surface is replaced when the NSWindow resizes. Changes to Layers ================= We now have a dedicated GdkMacosLayer which contains sublayers of GdkMacosTile. The tile has a maximum size of 128x128 pixels in device units. The GdkMacosTile is partitioned by splitting both the transparent region (window bounds minus opaque area) and then by splitting the opaque area. A tile has either translucent contents (and therefore is not opaque) or has opaque contents (and therefore is opaque). An opaque tile never contains transparent contents. As such, the opaque tiles contain a black background so that Core Animation will consider the tile's bounds as opaque. This can be verified with "Quartz Debug -> Show opaque regions". Changes to Cairo ================ GTK 4 cannot currently use cairo-quartz because of how CSS borders are rendered. It simply causes errors in the cairo_quartz_surface_t backend. Since we are restricted to using cairo_image_surface_t (which happens to be faster anyway) we can use the IOSurfaceBaseAddress() to obtain a mapping of the IOSurfaceRef in user-space. It always uses BGRA 32-bit with alpha channel even if we will discard the alpha channel as that is necessary to hit the fast paths in other parts of the platform. Note that while Cairo says CAIRO_FORMAT_ARGB32, it is really 32-bit BGRA on little-endian as we expect. OpenGL will render flipped (Quartz Native Co-ordinates) while Cairo renders with 0,O in the top-left. We could use cairo_translate() and cairo_scale() to reverse this, but it looks like some cairo things may not look quite as right if we do so. To reduce the chances of one-off bugs this continues to draw as Cairo would normally, but instead uses an CGAffineTransform in the tiles and some CGRect translation when swapping buffers to get the same effect. Changes to OpenGL ================= To simplify things, removal of all NSOpenGL* related components have been removed and we strictly use the Core GL (CGL*) API. This probably should have been done long ago anyay. Most examples found in the browsers to use IOSurfaceRef with OpenGL are using Legacy GL and there is still work underway to make this fit in with the rest of how the GSK GL renderer works. Since IOSurfaceRef bound to a texture/framebuffer will not have a default framebuffer ID of 0, we needed to add a default framebuffer id to the GdkGLContext. GskGLRenderer can use this to setup the command queue in such a way that our IOSurface destination has been glBindFramebuffer() as if it were the default drawable. This stuff is pretty slight-of-hand, so where things are and what needs flushing when and where has been a bit of an experiment to see what actually works to get synchronization across subsystems. Efficient Damages ================= After we draw with Cairo, we unlock the IOSurfaceRef and the contents are uploaded to the GPU. To make the contents visible to the app, we must clear the tiles contents with `layer.contents=nil;` and then re-apply the IOSurfaceRef. Since the buffer has likely not changed, we only do this if the tile overlaps the damage region. This gives the effect of having more tightly controlled damage regions even though updating the layer would damage be the whole window (as it is with OpenGL/Metal today with the exception of scissor-rect). This too can be verified usign "Quartz Debug -> Flash screen udpates". Frame Synchronized Resize ========================= In GTK 4, we have the ability to perform sizing changes from compute-size during the layout phase. Since the macOS backend already tracks window resizes manually, we can avoid doing the setFrame: immediately and instead do it within the frame clock's layout phase. Doing so gives us vastly better resize experience as we're more likely to get the size-change and updated-contents in the same frame on screen. It makes things feel "connected" in a way they weren't before. Some additional effort to tweak gravity during the process is also necessary but we were already doing that in the GTK 4 backend. Backporting =========== The design here has made an attempt to make it possible to backport by keeping GdkMacosBuffer, GdkMacosLayer, and GdkMacosTile fairly independent. There may be an opportunity to integrate this into GTK 3's quartz backend with a fair bit of work. Doing so could improve the situation for applications which are damage-rich such as The GIMP.
2022-02-14 10:20:19 +00:00
_gdk_macos_buffer_set_damage (buffer, region);
_gdk_macos_buffer_set_flipped (buffer, FALSE);
macos: fix cairo renderer with double buffering If we are double buffering surfaces with IOSurface then we need to copy the area that was damaged in the previous frame to the back buffer. This can be done with IOSurface but we need to hold the read-only lock so that we don't cause the underlying IOSurface contents to be invalidated. Additionally, since this is only used in the context of rendering to a GdkMacosSurface, we know the life-time of the cairo_surface_t and can simply lock/unlock the IOSurface buffer from begin_frame/end_frame to have the buffer flushing semantics we want. To ensure that we don't over damage, we store the damage in begin_frame (and copy it) and then subtract it from the next frames damage to determine the smallest amount we need to copy (taking scale factor into account). We don't care to modify the damage region to swapBuffers because they already have the right contents and could potentially fall into another tile anyway and we'd like to avoid damaging that. Fixes #4735
2022-02-28 09:42:48 +00:00
_gdk_macos_buffer_lock (buffer);
/* If there is damage that was on the previous frame that is not on
* this frame, we need to copy that rendered region over to the back
* buffer so that when swapping buffers, we still have that content.
* This is done with a read-only lock on the IOSurface to avoid
* invalidating the buffer contents.
*/
if (surface->front != NULL)
{
const cairo_region_t *previous = _gdk_macos_buffer_get_damage (surface->front);
if (previous != NULL)
{
cairo_region_t *copy;
copy = cairo_region_copy (previous);
cairo_region_subtract (copy, region);
if (!cairo_region_is_empty (copy))
{
int scale = gdk_surface_get_scale_factor (GDK_SURFACE (surface));
_gdk_macos_buffer_read_lock (surface->front);
copy_surface_data (surface->front, buffer, copy, scale);
_gdk_macos_buffer_read_unlock (surface->front);
}
cairo_region_destroy (copy);
}
}
drawcontext: Make begin_frame() set colorstate + depth Make begin_frame() set a rendering colorstate and depth, and provide it to the renderers via gdk_draw_context_get_depth() and gdk_draw_context_get_color_state(). This allows the draw contexts to define their own values, so that ie the Cairo and GL renderer can choose different settings for rendering (in particular, GL can choose GL_SRGB and do the srgb conversion; while Cairo relies on the renderer).
2024-06-30 02:06:48 +00:00
*out_color_state = GDK_COLOR_STATE_SRGB;
*out_depth = gdk_color_state_get_depth (GDK_COLOR_STATE_SRGB);
macos: prototype new GDK backend for macOS This is fairly substantial rewrite of the GDK backend for quartz and renamed to macOS to allow for a greenfield implementation. Many things have come across from the quartz implementation fairly intact such as the eventloop integration design and discovery of event windows from the NSEvent. However much has been changed to fit in with the new GDK design and how removal of child GdkWindow have been completely eliminated. Furthermore, the new GdkPopup allows for regular NSWindow to be used to provide popovers unlike the previous implementation. The object design more closely follows the ideal for a GDK backend. Views have been broken out into subclasses so that we can support multiple GSK renderer paths such as GL and Cairo (and Metal in the future). However mixed mode GL and Cairo will not be supported. Currently only the Cairo renderer has been implemented. A new frame clock implementation using CVDisplayLink provides more accurate information about when to draw drawing the next frame. Some testing will need to be done here to understand the power implications of this. This implementation has also gained edge snapping for CSD windows. Some work was also done to ensure that CSD windows have opaque regions registered with the display server. ** This is still very much a work-in-progress ** Some outstanding work that needs to be done: - Finish a GL context for macOS and alternate NSView for GL rendering (possibly using speciailized CALayer for OpenGL). - Input rework to ensure that we don't loose remapping of keys that was dropped from GDK during GTK 4 development. - Make sure input methods continue to work. - Drag-n-Drop is still very much a work in progress - High resolution input scrolling needs various work in GDK to land first before we can plumb that to NSEvent. - gtk/ has a number of things based on GDK_WINDOWING_QUARTZ that need to be updated to use the macOS backend. But this is good enough to start playing with and breaking things which is what I'd like to see.
2020-04-23 23:36:46 +00:00
}
static void
_gdk_macos_cairo_context_end_frame (GdkDrawContext *draw_context,
cairo_region_t *painted)
{
macos: fix cairo renderer with double buffering If we are double buffering surfaces with IOSurface then we need to copy the area that was damaged in the previous frame to the back buffer. This can be done with IOSurface but we need to hold the read-only lock so that we don't cause the underlying IOSurface contents to be invalidated. Additionally, since this is only used in the context of rendering to a GdkMacosSurface, we know the life-time of the cairo_surface_t and can simply lock/unlock the IOSurface buffer from begin_frame/end_frame to have the buffer flushing semantics we want. To ensure that we don't over damage, we store the damage in begin_frame (and copy it) and then subtract it from the next frames damage to determine the smallest amount we need to copy (taking scale factor into account). We don't care to modify the damage region to swapBuffers because they already have the right contents and could potentially fall into another tile anyway and we'd like to avoid damaging that. Fixes #4735
2022-02-28 09:42:48 +00:00
GdkMacosCairoContext *self = (GdkMacosCairoContext *)draw_context;
GdkMacosSurface *surface;
macos: use CALayer and IOSurface for rendering This provides a major shift in how we draw both when accelerated OpenGL as well as software rendering with Cairo. In short, it uses tiles of Core Animation's CALayer to display contents from an OpenGL or Cairo rendering so that the window can provide partial damage updates. Partial damage is not generally available when using OpenGL as the whole buffer is flipped even if you only submitted a small change using a scissor rect. Thankfully, this speeds up Cairo rendering a bit too by using IOSurface to upload contents to the display server. We use the tiling system we do for OpenGL which reduces overall complexity and differences between them. A New Buffer ============ GdkMacosBuffer is a wrapper around an IOSurfaceRef. The term buffer was used because 1) surface is already used and 2) it loosely maps to a front/back buffer semantic. However, it appears that IOSurfaceRef contents are being retained in some fashion (likely in the compositor result) so we can update the same IOSurfaceRef without flipping as long as we're fast. This appears to be what Chromium does as well, but Firefox uses two IOSurfaceRef and flips between them. We would like to avoid two surfaces because it doubles the GPU VRAM requirements of the application. Changes to Windows ================== Previously, the NSWindow would dynamically change between different types of NSView based on the renderer being used. This is no longer necessary as we just have a single NSView type, GdkMacosView, which inherits from GdkMacosBaseView just to keep the tedius stuff separate from the machinery of GdkMacosView. We can merge those someday if we are okay with that. Changes to Views ================ GdkMacosCairoView, GdkMacosCairoSubView, GdkMacosGLView have all been removed and replaced with GdkMacosView. This new view has a single CALayer (GdkMacosLayer) attached to it which itself has sublayers. The contents of the CALayer is populated with an IOSurfaceRef which we allocated with the GdkMacosSurface. The surface is replaced when the NSWindow resizes. Changes to Layers ================= We now have a dedicated GdkMacosLayer which contains sublayers of GdkMacosTile. The tile has a maximum size of 128x128 pixels in device units. The GdkMacosTile is partitioned by splitting both the transparent region (window bounds minus opaque area) and then by splitting the opaque area. A tile has either translucent contents (and therefore is not opaque) or has opaque contents (and therefore is opaque). An opaque tile never contains transparent contents. As such, the opaque tiles contain a black background so that Core Animation will consider the tile's bounds as opaque. This can be verified with "Quartz Debug -> Show opaque regions". Changes to Cairo ================ GTK 4 cannot currently use cairo-quartz because of how CSS borders are rendered. It simply causes errors in the cairo_quartz_surface_t backend. Since we are restricted to using cairo_image_surface_t (which happens to be faster anyway) we can use the IOSurfaceBaseAddress() to obtain a mapping of the IOSurfaceRef in user-space. It always uses BGRA 32-bit with alpha channel even if we will discard the alpha channel as that is necessary to hit the fast paths in other parts of the platform. Note that while Cairo says CAIRO_FORMAT_ARGB32, it is really 32-bit BGRA on little-endian as we expect. OpenGL will render flipped (Quartz Native Co-ordinates) while Cairo renders with 0,O in the top-left. We could use cairo_translate() and cairo_scale() to reverse this, but it looks like some cairo things may not look quite as right if we do so. To reduce the chances of one-off bugs this continues to draw as Cairo would normally, but instead uses an CGAffineTransform in the tiles and some CGRect translation when swapping buffers to get the same effect. Changes to OpenGL ================= To simplify things, removal of all NSOpenGL* related components have been removed and we strictly use the Core GL (CGL*) API. This probably should have been done long ago anyay. Most examples found in the browsers to use IOSurfaceRef with OpenGL are using Legacy GL and there is still work underway to make this fit in with the rest of how the GSK GL renderer works. Since IOSurfaceRef bound to a texture/framebuffer will not have a default framebuffer ID of 0, we needed to add a default framebuffer id to the GdkGLContext. GskGLRenderer can use this to setup the command queue in such a way that our IOSurface destination has been glBindFramebuffer() as if it were the default drawable. This stuff is pretty slight-of-hand, so where things are and what needs flushing when and where has been a bit of an experiment to see what actually works to get synchronization across subsystems. Efficient Damages ================= After we draw with Cairo, we unlock the IOSurfaceRef and the contents are uploaded to the GPU. To make the contents visible to the app, we must clear the tiles contents with `layer.contents=nil;` and then re-apply the IOSurfaceRef. Since the buffer has likely not changed, we only do this if the tile overlaps the damage region. This gives the effect of having more tightly controlled damage regions even though updating the layer would damage be the whole window (as it is with OpenGL/Metal today with the exception of scissor-rect). This too can be verified usign "Quartz Debug -> Flash screen udpates". Frame Synchronized Resize ========================= In GTK 4, we have the ability to perform sizing changes from compute-size during the layout phase. Since the macOS backend already tracks window resizes manually, we can avoid doing the setFrame: immediately and instead do it within the frame clock's layout phase. Doing so gives us vastly better resize experience as we're more likely to get the size-change and updated-contents in the same frame on screen. It makes things feel "connected" in a way they weren't before. Some additional effort to tweak gravity during the process is also necessary but we were already doing that in the GTK 4 backend. Backporting =========== The design here has made an attempt to make it possible to backport by keeping GdkMacosBuffer, GdkMacosLayer, and GdkMacosTile fairly independent. There may be an opportunity to integrate this into GTK 3's quartz backend with a fair bit of work. Doing so could improve the situation for applications which are damage-rich such as The GIMP.
2022-02-14 10:20:19 +00:00
GdkMacosBuffer *buffer;
macos: prototype new GDK backend for macOS This is fairly substantial rewrite of the GDK backend for quartz and renamed to macOS to allow for a greenfield implementation. Many things have come across from the quartz implementation fairly intact such as the eventloop integration design and discovery of event windows from the NSEvent. However much has been changed to fit in with the new GDK design and how removal of child GdkWindow have been completely eliminated. Furthermore, the new GdkPopup allows for regular NSWindow to be used to provide popovers unlike the previous implementation. The object design more closely follows the ideal for a GDK backend. Views have been broken out into subclasses so that we can support multiple GSK renderer paths such as GL and Cairo (and Metal in the future). However mixed mode GL and Cairo will not be supported. Currently only the Cairo renderer has been implemented. A new frame clock implementation using CVDisplayLink provides more accurate information about when to draw drawing the next frame. Some testing will need to be done here to understand the power implications of this. This implementation has also gained edge snapping for CSD windows. Some work was also done to ensure that CSD windows have opaque regions registered with the display server. ** This is still very much a work-in-progress ** Some outstanding work that needs to be done: - Finish a GL context for macOS and alternate NSView for GL rendering (possibly using speciailized CALayer for OpenGL). - Input rework to ensure that we don't loose remapping of keys that was dropped from GDK during GTK 4 development. - Make sure input methods continue to work. - Drag-n-Drop is still very much a work in progress - High resolution input scrolling needs various work in GDK to land first before we can plumb that to NSEvent. - gtk/ has a number of things based on GDK_WINDOWING_QUARTZ that need to be updated to use the macOS backend. But this is good enough to start playing with and breaking things which is what I'd like to see.
2020-04-23 23:36:46 +00:00
macos: fix cairo renderer with double buffering If we are double buffering surfaces with IOSurface then we need to copy the area that was damaged in the previous frame to the back buffer. This can be done with IOSurface but we need to hold the read-only lock so that we don't cause the underlying IOSurface contents to be invalidated. Additionally, since this is only used in the context of rendering to a GdkMacosSurface, we know the life-time of the cairo_surface_t and can simply lock/unlock the IOSurface buffer from begin_frame/end_frame to have the buffer flushing semantics we want. To ensure that we don't over damage, we store the damage in begin_frame (and copy it) and then subtract it from the next frames damage to determine the smallest amount we need to copy (taking scale factor into account). We don't care to modify the damage region to swapBuffers because they already have the right contents and could potentially fall into another tile anyway and we'd like to avoid damaging that. Fixes #4735
2022-02-28 09:42:48 +00:00
g_assert (GDK_IS_MACOS_CAIRO_CONTEXT (self));
macos: prototype new GDK backend for macOS This is fairly substantial rewrite of the GDK backend for quartz and renamed to macOS to allow for a greenfield implementation. Many things have come across from the quartz implementation fairly intact such as the eventloop integration design and discovery of event windows from the NSEvent. However much has been changed to fit in with the new GDK design and how removal of child GdkWindow have been completely eliminated. Furthermore, the new GdkPopup allows for regular NSWindow to be used to provide popovers unlike the previous implementation. The object design more closely follows the ideal for a GDK backend. Views have been broken out into subclasses so that we can support multiple GSK renderer paths such as GL and Cairo (and Metal in the future). However mixed mode GL and Cairo will not be supported. Currently only the Cairo renderer has been implemented. A new frame clock implementation using CVDisplayLink provides more accurate information about when to draw drawing the next frame. Some testing will need to be done here to understand the power implications of this. This implementation has also gained edge snapping for CSD windows. Some work was also done to ensure that CSD windows have opaque regions registered with the display server. ** This is still very much a work-in-progress ** Some outstanding work that needs to be done: - Finish a GL context for macOS and alternate NSView for GL rendering (possibly using speciailized CALayer for OpenGL). - Input rework to ensure that we don't loose remapping of keys that was dropped from GDK during GTK 4 development. - Make sure input methods continue to work. - Drag-n-Drop is still very much a work in progress - High resolution input scrolling needs various work in GDK to land first before we can plumb that to NSEvent. - gtk/ has a number of things based on GDK_WINDOWING_QUARTZ that need to be updated to use the macOS backend. But this is good enough to start playing with and breaking things which is what I'd like to see.
2020-04-23 23:36:46 +00:00
macos: fix cairo renderer with double buffering If we are double buffering surfaces with IOSurface then we need to copy the area that was damaged in the previous frame to the back buffer. This can be done with IOSurface but we need to hold the read-only lock so that we don't cause the underlying IOSurface contents to be invalidated. Additionally, since this is only used in the context of rendering to a GdkMacosSurface, we know the life-time of the cairo_surface_t and can simply lock/unlock the IOSurface buffer from begin_frame/end_frame to have the buffer flushing semantics we want. To ensure that we don't over damage, we store the damage in begin_frame (and copy it) and then subtract it from the next frames damage to determine the smallest amount we need to copy (taking scale factor into account). We don't care to modify the damage region to swapBuffers because they already have the right contents and could potentially fall into another tile anyway and we'd like to avoid damaging that. Fixes #4735
2022-02-28 09:42:48 +00:00
surface = GDK_MACOS_SURFACE (gdk_draw_context_get_surface (draw_context));
buffer = _gdk_macos_surface_get_buffer (surface);
_gdk_macos_buffer_unlock (buffer);
macos: prototype new GDK backend for macOS This is fairly substantial rewrite of the GDK backend for quartz and renamed to macOS to allow for a greenfield implementation. Many things have come across from the quartz implementation fairly intact such as the eventloop integration design and discovery of event windows from the NSEvent. However much has been changed to fit in with the new GDK design and how removal of child GdkWindow have been completely eliminated. Furthermore, the new GdkPopup allows for regular NSWindow to be used to provide popovers unlike the previous implementation. The object design more closely follows the ideal for a GDK backend. Views have been broken out into subclasses so that we can support multiple GSK renderer paths such as GL and Cairo (and Metal in the future). However mixed mode GL and Cairo will not be supported. Currently only the Cairo renderer has been implemented. A new frame clock implementation using CVDisplayLink provides more accurate information about when to draw drawing the next frame. Some testing will need to be done here to understand the power implications of this. This implementation has also gained edge snapping for CSD windows. Some work was also done to ensure that CSD windows have opaque regions registered with the display server. ** This is still very much a work-in-progress ** Some outstanding work that needs to be done: - Finish a GL context for macOS and alternate NSView for GL rendering (possibly using speciailized CALayer for OpenGL). - Input rework to ensure that we don't loose remapping of keys that was dropped from GDK during GTK 4 development. - Make sure input methods continue to work. - Drag-n-Drop is still very much a work in progress - High resolution input scrolling needs various work in GDK to land first before we can plumb that to NSEvent. - gtk/ has a number of things based on GDK_WINDOWING_QUARTZ that need to be updated to use the macOS backend. But this is good enough to start playing with and breaking things which is what I'd like to see.
2020-04-23 23:36:46 +00:00
macos: fix cairo renderer with double buffering If we are double buffering surfaces with IOSurface then we need to copy the area that was damaged in the previous frame to the back buffer. This can be done with IOSurface but we need to hold the read-only lock so that we don't cause the underlying IOSurface contents to be invalidated. Additionally, since this is only used in the context of rendering to a GdkMacosSurface, we know the life-time of the cairo_surface_t and can simply lock/unlock the IOSurface buffer from begin_frame/end_frame to have the buffer flushing semantics we want. To ensure that we don't over damage, we store the damage in begin_frame (and copy it) and then subtract it from the next frames damage to determine the smallest amount we need to copy (taking scale factor into account). We don't care to modify the damage region to swapBuffers because they already have the right contents and could potentially fall into another tile anyway and we'd like to avoid damaging that. Fixes #4735
2022-02-28 09:42:48 +00:00
_gdk_macos_surface_swap_buffers (surface, painted);
macos: avoid unnecessarily destroying cairo_t This avoids destroying a cairo_t when we are within the frame as it otherwise causes the surface to unnecessarily flush.
2022-02-03 21:01:46 +00:00
macos: use CALayer and IOSurface for rendering This provides a major shift in how we draw both when accelerated OpenGL as well as software rendering with Cairo. In short, it uses tiles of Core Animation's CALayer to display contents from an OpenGL or Cairo rendering so that the window can provide partial damage updates. Partial damage is not generally available when using OpenGL as the whole buffer is flipped even if you only submitted a small change using a scissor rect. Thankfully, this speeds up Cairo rendering a bit too by using IOSurface to upload contents to the display server. We use the tiling system we do for OpenGL which reduces overall complexity and differences between them. A New Buffer ============ GdkMacosBuffer is a wrapper around an IOSurfaceRef. The term buffer was used because 1) surface is already used and 2) it loosely maps to a front/back buffer semantic. However, it appears that IOSurfaceRef contents are being retained in some fashion (likely in the compositor result) so we can update the same IOSurfaceRef without flipping as long as we're fast. This appears to be what Chromium does as well, but Firefox uses two IOSurfaceRef and flips between them. We would like to avoid two surfaces because it doubles the GPU VRAM requirements of the application. Changes to Windows ================== Previously, the NSWindow would dynamically change between different types of NSView based on the renderer being used. This is no longer necessary as we just have a single NSView type, GdkMacosView, which inherits from GdkMacosBaseView just to keep the tedius stuff separate from the machinery of GdkMacosView. We can merge those someday if we are okay with that. Changes to Views ================ GdkMacosCairoView, GdkMacosCairoSubView, GdkMacosGLView have all been removed and replaced with GdkMacosView. This new view has a single CALayer (GdkMacosLayer) attached to it which itself has sublayers. The contents of the CALayer is populated with an IOSurfaceRef which we allocated with the GdkMacosSurface. The surface is replaced when the NSWindow resizes. Changes to Layers ================= We now have a dedicated GdkMacosLayer which contains sublayers of GdkMacosTile. The tile has a maximum size of 128x128 pixels in device units. The GdkMacosTile is partitioned by splitting both the transparent region (window bounds minus opaque area) and then by splitting the opaque area. A tile has either translucent contents (and therefore is not opaque) or has opaque contents (and therefore is opaque). An opaque tile never contains transparent contents. As such, the opaque tiles contain a black background so that Core Animation will consider the tile's bounds as opaque. This can be verified with "Quartz Debug -> Show opaque regions". Changes to Cairo ================ GTK 4 cannot currently use cairo-quartz because of how CSS borders are rendered. It simply causes errors in the cairo_quartz_surface_t backend. Since we are restricted to using cairo_image_surface_t (which happens to be faster anyway) we can use the IOSurfaceBaseAddress() to obtain a mapping of the IOSurfaceRef in user-space. It always uses BGRA 32-bit with alpha channel even if we will discard the alpha channel as that is necessary to hit the fast paths in other parts of the platform. Note that while Cairo says CAIRO_FORMAT_ARGB32, it is really 32-bit BGRA on little-endian as we expect. OpenGL will render flipped (Quartz Native Co-ordinates) while Cairo renders with 0,O in the top-left. We could use cairo_translate() and cairo_scale() to reverse this, but it looks like some cairo things may not look quite as right if we do so. To reduce the chances of one-off bugs this continues to draw as Cairo would normally, but instead uses an CGAffineTransform in the tiles and some CGRect translation when swapping buffers to get the same effect. Changes to OpenGL ================= To simplify things, removal of all NSOpenGL* related components have been removed and we strictly use the Core GL (CGL*) API. This probably should have been done long ago anyay. Most examples found in the browsers to use IOSurfaceRef with OpenGL are using Legacy GL and there is still work underway to make this fit in with the rest of how the GSK GL renderer works. Since IOSurfaceRef bound to a texture/framebuffer will not have a default framebuffer ID of 0, we needed to add a default framebuffer id to the GdkGLContext. GskGLRenderer can use this to setup the command queue in such a way that our IOSurface destination has been glBindFramebuffer() as if it were the default drawable. This stuff is pretty slight-of-hand, so where things are and what needs flushing when and where has been a bit of an experiment to see what actually works to get synchronization across subsystems. Efficient Damages ================= After we draw with Cairo, we unlock the IOSurfaceRef and the contents are uploaded to the GPU. To make the contents visible to the app, we must clear the tiles contents with `layer.contents=nil;` and then re-apply the IOSurfaceRef. Since the buffer has likely not changed, we only do this if the tile overlaps the damage region. This gives the effect of having more tightly controlled damage regions even though updating the layer would damage be the whole window (as it is with OpenGL/Metal today with the exception of scissor-rect). This too can be verified usign "Quartz Debug -> Flash screen udpates". Frame Synchronized Resize ========================= In GTK 4, we have the ability to perform sizing changes from compute-size during the layout phase. Since the macOS backend already tracks window resizes manually, we can avoid doing the setFrame: immediately and instead do it within the frame clock's layout phase. Doing so gives us vastly better resize experience as we're more likely to get the size-change and updated-contents in the same frame on screen. It makes things feel "connected" in a way they weren't before. Some additional effort to tweak gravity during the process is also necessary but we were already doing that in the GTK 4 backend. Backporting =========== The design here has made an attempt to make it possible to backport by keeping GdkMacosBuffer, GdkMacosLayer, and GdkMacosTile fairly independent. There may be an opportunity to integrate this into GTK 3's quartz backend with a fair bit of work. Doing so could improve the situation for applications which are damage-rich such as The GIMP.
2022-02-14 10:20:19 +00:00
[CATransaction commit];
macos: prototype new GDK backend for macOS This is fairly substantial rewrite of the GDK backend for quartz and renamed to macOS to allow for a greenfield implementation. Many things have come across from the quartz implementation fairly intact such as the eventloop integration design and discovery of event windows from the NSEvent. However much has been changed to fit in with the new GDK design and how removal of child GdkWindow have been completely eliminated. Furthermore, the new GdkPopup allows for regular NSWindow to be used to provide popovers unlike the previous implementation. The object design more closely follows the ideal for a GDK backend. Views have been broken out into subclasses so that we can support multiple GSK renderer paths such as GL and Cairo (and Metal in the future). However mixed mode GL and Cairo will not be supported. Currently only the Cairo renderer has been implemented. A new frame clock implementation using CVDisplayLink provides more accurate information about when to draw drawing the next frame. Some testing will need to be done here to understand the power implications of this. This implementation has also gained edge snapping for CSD windows. Some work was also done to ensure that CSD windows have opaque regions registered with the display server. ** This is still very much a work-in-progress ** Some outstanding work that needs to be done: - Finish a GL context for macOS and alternate NSView for GL rendering (possibly using speciailized CALayer for OpenGL). - Input rework to ensure that we don't loose remapping of keys that was dropped from GDK during GTK 4 development. - Make sure input methods continue to work. - Drag-n-Drop is still very much a work in progress - High resolution input scrolling needs various work in GDK to land first before we can plumb that to NSEvent. - gtk/ has a number of things based on GDK_WINDOWING_QUARTZ that need to be updated to use the macOS backend. But this is good enough to start playing with and breaking things which is what I'd like to see.
2020-04-23 23:36:46 +00:00
}
Add empty callbacks for Gdk.DrawContext.empty_frame on macOS
2024-01-27 14:34:59 +00:00
static void
_gdk_macos_cairo_context_empty_frame (GdkDrawContext *draw_context)
{
}
macos: prototype new GDK backend for macOS This is fairly substantial rewrite of the GDK backend for quartz and renamed to macOS to allow for a greenfield implementation. Many things have come across from the quartz implementation fairly intact such as the eventloop integration design and discovery of event windows from the NSEvent. However much has been changed to fit in with the new GDK design and how removal of child GdkWindow have been completely eliminated. Furthermore, the new GdkPopup allows for regular NSWindow to be used to provide popovers unlike the previous implementation. The object design more closely follows the ideal for a GDK backend. Views have been broken out into subclasses so that we can support multiple GSK renderer paths such as GL and Cairo (and Metal in the future). However mixed mode GL and Cairo will not be supported. Currently only the Cairo renderer has been implemented. A new frame clock implementation using CVDisplayLink provides more accurate information about when to draw drawing the next frame. Some testing will need to be done here to understand the power implications of this. This implementation has also gained edge snapping for CSD windows. Some work was also done to ensure that CSD windows have opaque regions registered with the display server. ** This is still very much a work-in-progress ** Some outstanding work that needs to be done: - Finish a GL context for macOS and alternate NSView for GL rendering (possibly using speciailized CALayer for OpenGL). - Input rework to ensure that we don't loose remapping of keys that was dropped from GDK during GTK 4 development. - Make sure input methods continue to work. - Drag-n-Drop is still very much a work in progress - High resolution input scrolling needs various work in GDK to land first before we can plumb that to NSEvent. - gtk/ has a number of things based on GDK_WINDOWING_QUARTZ that need to be updated to use the macOS backend. But this is good enough to start playing with and breaking things which is what I'd like to see.
2020-04-23 23:36:46 +00:00
static void
_gdk_macos_cairo_context_surface_resized (GdkDrawContext *draw_context)
{
macos: use CALayer and IOSurface for rendering This provides a major shift in how we draw both when accelerated OpenGL as well as software rendering with Cairo. In short, it uses tiles of Core Animation's CALayer to display contents from an OpenGL or Cairo rendering so that the window can provide partial damage updates. Partial damage is not generally available when using OpenGL as the whole buffer is flipped even if you only submitted a small change using a scissor rect. Thankfully, this speeds up Cairo rendering a bit too by using IOSurface to upload contents to the display server. We use the tiling system we do for OpenGL which reduces overall complexity and differences between them. A New Buffer ============ GdkMacosBuffer is a wrapper around an IOSurfaceRef. The term buffer was used because 1) surface is already used and 2) it loosely maps to a front/back buffer semantic. However, it appears that IOSurfaceRef contents are being retained in some fashion (likely in the compositor result) so we can update the same IOSurfaceRef without flipping as long as we're fast. This appears to be what Chromium does as well, but Firefox uses two IOSurfaceRef and flips between them. We would like to avoid two surfaces because it doubles the GPU VRAM requirements of the application. Changes to Windows ================== Previously, the NSWindow would dynamically change between different types of NSView based on the renderer being used. This is no longer necessary as we just have a single NSView type, GdkMacosView, which inherits from GdkMacosBaseView just to keep the tedius stuff separate from the machinery of GdkMacosView. We can merge those someday if we are okay with that. Changes to Views ================ GdkMacosCairoView, GdkMacosCairoSubView, GdkMacosGLView have all been removed and replaced with GdkMacosView. This new view has a single CALayer (GdkMacosLayer) attached to it which itself has sublayers. The contents of the CALayer is populated with an IOSurfaceRef which we allocated with the GdkMacosSurface. The surface is replaced when the NSWindow resizes. Changes to Layers ================= We now have a dedicated GdkMacosLayer which contains sublayers of GdkMacosTile. The tile has a maximum size of 128x128 pixels in device units. The GdkMacosTile is partitioned by splitting both the transparent region (window bounds minus opaque area) and then by splitting the opaque area. A tile has either translucent contents (and therefore is not opaque) or has opaque contents (and therefore is opaque). An opaque tile never contains transparent contents. As such, the opaque tiles contain a black background so that Core Animation will consider the tile's bounds as opaque. This can be verified with "Quartz Debug -> Show opaque regions". Changes to Cairo ================ GTK 4 cannot currently use cairo-quartz because of how CSS borders are rendered. It simply causes errors in the cairo_quartz_surface_t backend. Since we are restricted to using cairo_image_surface_t (which happens to be faster anyway) we can use the IOSurfaceBaseAddress() to obtain a mapping of the IOSurfaceRef in user-space. It always uses BGRA 32-bit with alpha channel even if we will discard the alpha channel as that is necessary to hit the fast paths in other parts of the platform. Note that while Cairo says CAIRO_FORMAT_ARGB32, it is really 32-bit BGRA on little-endian as we expect. OpenGL will render flipped (Quartz Native Co-ordinates) while Cairo renders with 0,O in the top-left. We could use cairo_translate() and cairo_scale() to reverse this, but it looks like some cairo things may not look quite as right if we do so. To reduce the chances of one-off bugs this continues to draw as Cairo would normally, but instead uses an CGAffineTransform in the tiles and some CGRect translation when swapping buffers to get the same effect. Changes to OpenGL ================= To simplify things, removal of all NSOpenGL* related components have been removed and we strictly use the Core GL (CGL*) API. This probably should have been done long ago anyay. Most examples found in the browsers to use IOSurfaceRef with OpenGL are using Legacy GL and there is still work underway to make this fit in with the rest of how the GSK GL renderer works. Since IOSurfaceRef bound to a texture/framebuffer will not have a default framebuffer ID of 0, we needed to add a default framebuffer id to the GdkGLContext. GskGLRenderer can use this to setup the command queue in such a way that our IOSurface destination has been glBindFramebuffer() as if it were the default drawable. This stuff is pretty slight-of-hand, so where things are and what needs flushing when and where has been a bit of an experiment to see what actually works to get synchronization across subsystems. Efficient Damages ================= After we draw with Cairo, we unlock the IOSurfaceRef and the contents are uploaded to the GPU. To make the contents visible to the app, we must clear the tiles contents with `layer.contents=nil;` and then re-apply the IOSurfaceRef. Since the buffer has likely not changed, we only do this if the tile overlaps the damage region. This gives the effect of having more tightly controlled damage regions even though updating the layer would damage be the whole window (as it is with OpenGL/Metal today with the exception of scissor-rect). This too can be verified usign "Quartz Debug -> Flash screen udpates". Frame Synchronized Resize ========================= In GTK 4, we have the ability to perform sizing changes from compute-size during the layout phase. Since the macOS backend already tracks window resizes manually, we can avoid doing the setFrame: immediately and instead do it within the frame clock's layout phase. Doing so gives us vastly better resize experience as we're more likely to get the size-change and updated-contents in the same frame on screen. It makes things feel "connected" in a way they weren't before. Some additional effort to tweak gravity during the process is also necessary but we were already doing that in the GTK 4 backend. Backporting =========== The design here has made an attempt to make it possible to backport by keeping GdkMacosBuffer, GdkMacosLayer, and GdkMacosTile fairly independent. There may be an opportunity to integrate this into GTK 3's quartz backend with a fair bit of work. Doing so could improve the situation for applications which are damage-rich such as The GIMP.
2022-02-14 10:20:19 +00:00
g_assert (GDK_IS_MACOS_CAIRO_CONTEXT (draw_context));
macos: cleanup window surface when destroying context Make sure we release the cairo_surface_t for the window when disposing the GdkMacosCairoContext.
2022-02-03 21:28:56 +00:00
macos: use CALayer and IOSurface for rendering This provides a major shift in how we draw both when accelerated OpenGL as well as software rendering with Cairo. In short, it uses tiles of Core Animation's CALayer to display contents from an OpenGL or Cairo rendering so that the window can provide partial damage updates. Partial damage is not generally available when using OpenGL as the whole buffer is flipped even if you only submitted a small change using a scissor rect. Thankfully, this speeds up Cairo rendering a bit too by using IOSurface to upload contents to the display server. We use the tiling system we do for OpenGL which reduces overall complexity and differences between them. A New Buffer ============ GdkMacosBuffer is a wrapper around an IOSurfaceRef. The term buffer was used because 1) surface is already used and 2) it loosely maps to a front/back buffer semantic. However, it appears that IOSurfaceRef contents are being retained in some fashion (likely in the compositor result) so we can update the same IOSurfaceRef without flipping as long as we're fast. This appears to be what Chromium does as well, but Firefox uses two IOSurfaceRef and flips between them. We would like to avoid two surfaces because it doubles the GPU VRAM requirements of the application. Changes to Windows ================== Previously, the NSWindow would dynamically change between different types of NSView based on the renderer being used. This is no longer necessary as we just have a single NSView type, GdkMacosView, which inherits from GdkMacosBaseView just to keep the tedius stuff separate from the machinery of GdkMacosView. We can merge those someday if we are okay with that. Changes to Views ================ GdkMacosCairoView, GdkMacosCairoSubView, GdkMacosGLView have all been removed and replaced with GdkMacosView. This new view has a single CALayer (GdkMacosLayer) attached to it which itself has sublayers. The contents of the CALayer is populated with an IOSurfaceRef which we allocated with the GdkMacosSurface. The surface is replaced when the NSWindow resizes. Changes to Layers ================= We now have a dedicated GdkMacosLayer which contains sublayers of GdkMacosTile. The tile has a maximum size of 128x128 pixels in device units. The GdkMacosTile is partitioned by splitting both the transparent region (window bounds minus opaque area) and then by splitting the opaque area. A tile has either translucent contents (and therefore is not opaque) or has opaque contents (and therefore is opaque). An opaque tile never contains transparent contents. As such, the opaque tiles contain a black background so that Core Animation will consider the tile's bounds as opaque. This can be verified with "Quartz Debug -> Show opaque regions". Changes to Cairo ================ GTK 4 cannot currently use cairo-quartz because of how CSS borders are rendered. It simply causes errors in the cairo_quartz_surface_t backend. Since we are restricted to using cairo_image_surface_t (which happens to be faster anyway) we can use the IOSurfaceBaseAddress() to obtain a mapping of the IOSurfaceRef in user-space. It always uses BGRA 32-bit with alpha channel even if we will discard the alpha channel as that is necessary to hit the fast paths in other parts of the platform. Note that while Cairo says CAIRO_FORMAT_ARGB32, it is really 32-bit BGRA on little-endian as we expect. OpenGL will render flipped (Quartz Native Co-ordinates) while Cairo renders with 0,O in the top-left. We could use cairo_translate() and cairo_scale() to reverse this, but it looks like some cairo things may not look quite as right if we do so. To reduce the chances of one-off bugs this continues to draw as Cairo would normally, but instead uses an CGAffineTransform in the tiles and some CGRect translation when swapping buffers to get the same effect. Changes to OpenGL ================= To simplify things, removal of all NSOpenGL* related components have been removed and we strictly use the Core GL (CGL*) API. This probably should have been done long ago anyay. Most examples found in the browsers to use IOSurfaceRef with OpenGL are using Legacy GL and there is still work underway to make this fit in with the rest of how the GSK GL renderer works. Since IOSurfaceRef bound to a texture/framebuffer will not have a default framebuffer ID of 0, we needed to add a default framebuffer id to the GdkGLContext. GskGLRenderer can use this to setup the command queue in such a way that our IOSurface destination has been glBindFramebuffer() as if it were the default drawable. This stuff is pretty slight-of-hand, so where things are and what needs flushing when and where has been a bit of an experiment to see what actually works to get synchronization across subsystems. Efficient Damages ================= After we draw with Cairo, we unlock the IOSurfaceRef and the contents are uploaded to the GPU. To make the contents visible to the app, we must clear the tiles contents with `layer.contents=nil;` and then re-apply the IOSurfaceRef. Since the buffer has likely not changed, we only do this if the tile overlaps the damage region. This gives the effect of having more tightly controlled damage regions even though updating the layer would damage be the whole window (as it is with OpenGL/Metal today with the exception of scissor-rect). This too can be verified usign "Quartz Debug -> Flash screen udpates". Frame Synchronized Resize ========================= In GTK 4, we have the ability to perform sizing changes from compute-size during the layout phase. Since the macOS backend already tracks window resizes manually, we can avoid doing the setFrame: immediately and instead do it within the frame clock's layout phase. Doing so gives us vastly better resize experience as we're more likely to get the size-change and updated-contents in the same frame on screen. It makes things feel "connected" in a way they weren't before. Some additional effort to tweak gravity during the process is also necessary but we were already doing that in the GTK 4 backend. Backporting =========== The design here has made an attempt to make it possible to backport by keeping GdkMacosBuffer, GdkMacosLayer, and GdkMacosTile fairly independent. There may be an opportunity to integrate this into GTK 3's quartz backend with a fair bit of work. Doing so could improve the situation for applications which are damage-rich such as The GIMP.
2022-02-14 10:20:19 +00:00
/* Do nothing, next begin_frame will get new buffer */
macos: cleanup window surface when destroying context Make sure we release the cairo_surface_t for the window when disposing the GdkMacosCairoContext.
2022-02-03 21:28:56 +00:00
}
macos: prototype new GDK backend for macOS This is fairly substantial rewrite of the GDK backend for quartz and renamed to macOS to allow for a greenfield implementation. Many things have come across from the quartz implementation fairly intact such as the eventloop integration design and discovery of event windows from the NSEvent. However much has been changed to fit in with the new GDK design and how removal of child GdkWindow have been completely eliminated. Furthermore, the new GdkPopup allows for regular NSWindow to be used to provide popovers unlike the previous implementation. The object design more closely follows the ideal for a GDK backend. Views have been broken out into subclasses so that we can support multiple GSK renderer paths such as GL and Cairo (and Metal in the future). However mixed mode GL and Cairo will not be supported. Currently only the Cairo renderer has been implemented. A new frame clock implementation using CVDisplayLink provides more accurate information about when to draw drawing the next frame. Some testing will need to be done here to understand the power implications of this. This implementation has also gained edge snapping for CSD windows. Some work was also done to ensure that CSD windows have opaque regions registered with the display server. ** This is still very much a work-in-progress ** Some outstanding work that needs to be done: - Finish a GL context for macOS and alternate NSView for GL rendering (possibly using speciailized CALayer for OpenGL). - Input rework to ensure that we don't loose remapping of keys that was dropped from GDK during GTK 4 development. - Make sure input methods continue to work. - Drag-n-Drop is still very much a work in progress - High resolution input scrolling needs various work in GDK to land first before we can plumb that to NSEvent. - gtk/ has a number of things based on GDK_WINDOWING_QUARTZ that need to be updated to use the macOS backend. But this is good enough to start playing with and breaking things which is what I'd like to see.
2020-04-23 23:36:46 +00:00
static void
_gdk_macos_cairo_context_class_init (GdkMacosCairoContextClass *klass)
{
GdkCairoContextClass *cairo_context_class = GDK_CAIRO_CONTEXT_CLASS (klass);
GdkDrawContextClass *draw_context_class = GDK_DRAW_CONTEXT_CLASS (klass);
draw_context_class->begin_frame = _gdk_macos_cairo_context_begin_frame;
draw_context_class->end_frame = _gdk_macos_cairo_context_end_frame;
Add empty callbacks for Gdk.DrawContext.empty_frame on macOS
2024-01-27 14:34:59 +00:00
draw_context_class->empty_frame = _gdk_macos_cairo_context_empty_frame;
macos: prototype new GDK backend for macOS This is fairly substantial rewrite of the GDK backend for quartz and renamed to macOS to allow for a greenfield implementation. Many things have come across from the quartz implementation fairly intact such as the eventloop integration design and discovery of event windows from the NSEvent. However much has been changed to fit in with the new GDK design and how removal of child GdkWindow have been completely eliminated. Furthermore, the new GdkPopup allows for regular NSWindow to be used to provide popovers unlike the previous implementation. The object design more closely follows the ideal for a GDK backend. Views have been broken out into subclasses so that we can support multiple GSK renderer paths such as GL and Cairo (and Metal in the future). However mixed mode GL and Cairo will not be supported. Currently only the Cairo renderer has been implemented. A new frame clock implementation using CVDisplayLink provides more accurate information about when to draw drawing the next frame. Some testing will need to be done here to understand the power implications of this. This implementation has also gained edge snapping for CSD windows. Some work was also done to ensure that CSD windows have opaque regions registered with the display server. ** This is still very much a work-in-progress ** Some outstanding work that needs to be done: - Finish a GL context for macOS and alternate NSView for GL rendering (possibly using speciailized CALayer for OpenGL). - Input rework to ensure that we don't loose remapping of keys that was dropped from GDK during GTK 4 development. - Make sure input methods continue to work. - Drag-n-Drop is still very much a work in progress - High resolution input scrolling needs various work in GDK to land first before we can plumb that to NSEvent. - gtk/ has a number of things based on GDK_WINDOWING_QUARTZ that need to be updated to use the macOS backend. But this is good enough to start playing with and breaking things which is what I'd like to see.
2020-04-23 23:36:46 +00:00
draw_context_class->surface_resized = _gdk_macos_cairo_context_surface_resized;
cairo_context_class->cairo_create = _gdk_macos_cairo_context_cairo_create;
}
static void
_gdk_macos_cairo_context_init (GdkMacosCairoContext *self)
{
}
Reference in New Issue Copy Permalink