AuroraMiddleware/gtk
1
0
Fork 1
mirror of https://gitlab.gnome.org/GNOME/gtk.git synced 2024-12-27 14:10:30 +00:00
Code Issues Projects Releases Wiki Activity
df8e2bc0a0
gtk/gdk/macos/gdkmacosglcontext.c

651 lines
20 KiB
C
Raw Normal View History

macos: prototype new GDK backend for macOS This is fairly substantial rewrite of the GDK backend for quartz and renamed to macOS to allow for a greenfield implementation. Many things have come across from the quartz implementation fairly intact such as the eventloop integration design and discovery of event windows from the NSEvent. However much has been changed to fit in with the new GDK design and how removal of child GdkWindow have been completely eliminated. Furthermore, the new GdkPopup allows for regular NSWindow to be used to provide popovers unlike the previous implementation. The object design more closely follows the ideal for a GDK backend. Views have been broken out into subclasses so that we can support multiple GSK renderer paths such as GL and Cairo (and Metal in the future). However mixed mode GL and Cairo will not be supported. Currently only the Cairo renderer has been implemented. A new frame clock implementation using CVDisplayLink provides more accurate information about when to draw drawing the next frame. Some testing will need to be done here to understand the power implications of this. This implementation has also gained edge snapping for CSD windows. Some work was also done to ensure that CSD windows have opaque regions registered with the display server. ** This is still very much a work-in-progress ** Some outstanding work that needs to be done: - Finish a GL context for macOS and alternate NSView for GL rendering (possibly using speciailized CALayer for OpenGL). - Input rework to ensure that we don't loose remapping of keys that was dropped from GDK during GTK 4 development. - Make sure input methods continue to work. - Drag-n-Drop is still very much a work in progress - High resolution input scrolling needs various work in GDK to land first before we can plumb that to NSEvent. - gtk/ has a number of things based on GDK_WINDOWING_QUARTZ that need to be updated to use the macOS backend. But this is good enough to start playing with and breaking things which is what I'd like to see.
2020-04-23 23:36:46 +00:00
/*
* Copyright © 2020 Red Hat, Inc.
*
* This library is free software; you can redistribute it and/or
* modify it under the terms of the GNU Lesser General Public
* License as published by the Free Software Foundation; either
* version 2.1 of the License, or (at your option) any later version.
*
* This library is distributed in the hope that it will be useful,
* but WITHOUT ANY WARRANTY; without even the implied warranty of
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
* Lesser General Public License for more details.
*
* You should have received a copy of the GNU Lesser General Public
* License along with this library. If not, see <http://www.gnu.org/licenses/>.
*
* SPDX-License-Identifier: LGPL-2.1-or-later
*/
#include "config.h"
macos: use CALayer and IOSurface for rendering This provides a major shift in how we draw both when accelerated OpenGL as well as software rendering with Cairo. In short, it uses tiles of Core Animation's CALayer to display contents from an OpenGL or Cairo rendering so that the window can provide partial damage updates. Partial damage is not generally available when using OpenGL as the whole buffer is flipped even if you only submitted a small change using a scissor rect. Thankfully, this speeds up Cairo rendering a bit too by using IOSurface to upload contents to the display server. We use the tiling system we do for OpenGL which reduces overall complexity and differences between them. A New Buffer ============ GdkMacosBuffer is a wrapper around an IOSurfaceRef. The term buffer was used because 1) surface is already used and 2) it loosely maps to a front/back buffer semantic. However, it appears that IOSurfaceRef contents are being retained in some fashion (likely in the compositor result) so we can update the same IOSurfaceRef without flipping as long as we're fast. This appears to be what Chromium does as well, but Firefox uses two IOSurfaceRef and flips between them. We would like to avoid two surfaces because it doubles the GPU VRAM requirements of the application. Changes to Windows ================== Previously, the NSWindow would dynamically change between different types of NSView based on the renderer being used. This is no longer necessary as we just have a single NSView type, GdkMacosView, which inherits from GdkMacosBaseView just to keep the tedius stuff separate from the machinery of GdkMacosView. We can merge those someday if we are okay with that. Changes to Views ================ GdkMacosCairoView, GdkMacosCairoSubView, GdkMacosGLView have all been removed and replaced with GdkMacosView. This new view has a single CALayer (GdkMacosLayer) attached to it which itself has sublayers. The contents of the CALayer is populated with an IOSurfaceRef which we allocated with the GdkMacosSurface. The surface is replaced when the NSWindow resizes. Changes to Layers ================= We now have a dedicated GdkMacosLayer which contains sublayers of GdkMacosTile. The tile has a maximum size of 128x128 pixels in device units. The GdkMacosTile is partitioned by splitting both the transparent region (window bounds minus opaque area) and then by splitting the opaque area. A tile has either translucent contents (and therefore is not opaque) or has opaque contents (and therefore is opaque). An opaque tile never contains transparent contents. As such, the opaque tiles contain a black background so that Core Animation will consider the tile's bounds as opaque. This can be verified with "Quartz Debug -> Show opaque regions". Changes to Cairo ================ GTK 4 cannot currently use cairo-quartz because of how CSS borders are rendered. It simply causes errors in the cairo_quartz_surface_t backend. Since we are restricted to using cairo_image_surface_t (which happens to be faster anyway) we can use the IOSurfaceBaseAddress() to obtain a mapping of the IOSurfaceRef in user-space. It always uses BGRA 32-bit with alpha channel even if we will discard the alpha channel as that is necessary to hit the fast paths in other parts of the platform. Note that while Cairo says CAIRO_FORMAT_ARGB32, it is really 32-bit BGRA on little-endian as we expect. OpenGL will render flipped (Quartz Native Co-ordinates) while Cairo renders with 0,O in the top-left. We could use cairo_translate() and cairo_scale() to reverse this, but it looks like some cairo things may not look quite as right if we do so. To reduce the chances of one-off bugs this continues to draw as Cairo would normally, but instead uses an CGAffineTransform in the tiles and some CGRect translation when swapping buffers to get the same effect. Changes to OpenGL ================= To simplify things, removal of all NSOpenGL* related components have been removed and we strictly use the Core GL (CGL*) API. This probably should have been done long ago anyay. Most examples found in the browsers to use IOSurfaceRef with OpenGL are using Legacy GL and there is still work underway to make this fit in with the rest of how the GSK GL renderer works. Since IOSurfaceRef bound to a texture/framebuffer will not have a default framebuffer ID of 0, we needed to add a default framebuffer id to the GdkGLContext. GskGLRenderer can use this to setup the command queue in such a way that our IOSurface destination has been glBindFramebuffer() as if it were the default drawable. This stuff is pretty slight-of-hand, so where things are and what needs flushing when and where has been a bit of an experiment to see what actually works to get synchronization across subsystems. Efficient Damages ================= After we draw with Cairo, we unlock the IOSurfaceRef and the contents are uploaded to the GPU. To make the contents visible to the app, we must clear the tiles contents with `layer.contents=nil;` and then re-apply the IOSurfaceRef. Since the buffer has likely not changed, we only do this if the tile overlaps the damage region. This gives the effect of having more tightly controlled damage regions even though updating the layer would damage be the whole window (as it is with OpenGL/Metal today with the exception of scissor-rect). This too can be verified usign "Quartz Debug -> Flash screen udpates". Frame Synchronized Resize ========================= In GTK 4, we have the ability to perform sizing changes from compute-size during the layout phase. Since the macOS backend already tracks window resizes manually, we can avoid doing the setFrame: immediately and instead do it within the frame clock's layout phase. Doing so gives us vastly better resize experience as we're more likely to get the size-change and updated-contents in the same frame on screen. It makes things feel "connected" in a way they weren't before. Some additional effort to tweak gravity during the process is also necessary but we were already doing that in the GTK 4 backend. Backporting =========== The design here has made an attempt to make it possible to backport by keeping GdkMacosBuffer, GdkMacosLayer, and GdkMacosTile fairly independent. There may be an opportunity to integrate this into GTK 3's quartz backend with a fair bit of work. Doing so could improve the situation for applications which are damage-rich such as The GIMP.
2022-02-14 10:20:19 +00:00
#include "gdkconfig.h"
#include <OpenGL/gl3.h>
#include <OpenGL/CGLIOSurface.h>
#include <QuartzCore/QuartzCore.h>
#include "gdkmacosbuffer-private.h"
macos: prototype new GDK backend for macOS This is fairly substantial rewrite of the GDK backend for quartz and renamed to macOS to allow for a greenfield implementation. Many things have come across from the quartz implementation fairly intact such as the eventloop integration design and discovery of event windows from the NSEvent. However much has been changed to fit in with the new GDK design and how removal of child GdkWindow have been completely eliminated. Furthermore, the new GdkPopup allows for regular NSWindow to be used to provide popovers unlike the previous implementation. The object design more closely follows the ideal for a GDK backend. Views have been broken out into subclasses so that we can support multiple GSK renderer paths such as GL and Cairo (and Metal in the future). However mixed mode GL and Cairo will not be supported. Currently only the Cairo renderer has been implemented. A new frame clock implementation using CVDisplayLink provides more accurate information about when to draw drawing the next frame. Some testing will need to be done here to understand the power implications of this. This implementation has also gained edge snapping for CSD windows. Some work was also done to ensure that CSD windows have opaque regions registered with the display server. ** This is still very much a work-in-progress ** Some outstanding work that needs to be done: - Finish a GL context for macOS and alternate NSView for GL rendering (possibly using speciailized CALayer for OpenGL). - Input rework to ensure that we don't loose remapping of keys that was dropped from GDK during GTK 4 development. - Make sure input methods continue to work. - Drag-n-Drop is still very much a work in progress - High resolution input scrolling needs various work in GDK to land first before we can plumb that to NSEvent. - gtk/ has a number of things based on GDK_WINDOWING_QUARTZ that need to be updated to use the macOS backend. But this is good enough to start playing with and breaking things which is what I'd like to see.
2020-04-23 23:36:46 +00:00
#include "gdkmacosglcontext-private.h"
#include "gdkmacossurface-private.h"
macos: use CALayer and IOSurface for rendering This provides a major shift in how we draw both when accelerated OpenGL as well as software rendering with Cairo. In short, it uses tiles of Core Animation's CALayer to display contents from an OpenGL or Cairo rendering so that the window can provide partial damage updates. Partial damage is not generally available when using OpenGL as the whole buffer is flipped even if you only submitted a small change using a scissor rect. Thankfully, this speeds up Cairo rendering a bit too by using IOSurface to upload contents to the display server. We use the tiling system we do for OpenGL which reduces overall complexity and differences between them. A New Buffer ============ GdkMacosBuffer is a wrapper around an IOSurfaceRef. The term buffer was used because 1) surface is already used and 2) it loosely maps to a front/back buffer semantic. However, it appears that IOSurfaceRef contents are being retained in some fashion (likely in the compositor result) so we can update the same IOSurfaceRef without flipping as long as we're fast. This appears to be what Chromium does as well, but Firefox uses two IOSurfaceRef and flips between them. We would like to avoid two surfaces because it doubles the GPU VRAM requirements of the application. Changes to Windows ================== Previously, the NSWindow would dynamically change between different types of NSView based on the renderer being used. This is no longer necessary as we just have a single NSView type, GdkMacosView, which inherits from GdkMacosBaseView just to keep the tedius stuff separate from the machinery of GdkMacosView. We can merge those someday if we are okay with that. Changes to Views ================ GdkMacosCairoView, GdkMacosCairoSubView, GdkMacosGLView have all been removed and replaced with GdkMacosView. This new view has a single CALayer (GdkMacosLayer) attached to it which itself has sublayers. The contents of the CALayer is populated with an IOSurfaceRef which we allocated with the GdkMacosSurface. The surface is replaced when the NSWindow resizes. Changes to Layers ================= We now have a dedicated GdkMacosLayer which contains sublayers of GdkMacosTile. The tile has a maximum size of 128x128 pixels in device units. The GdkMacosTile is partitioned by splitting both the transparent region (window bounds minus opaque area) and then by splitting the opaque area. A tile has either translucent contents (and therefore is not opaque) or has opaque contents (and therefore is opaque). An opaque tile never contains transparent contents. As such, the opaque tiles contain a black background so that Core Animation will consider the tile's bounds as opaque. This can be verified with "Quartz Debug -> Show opaque regions". Changes to Cairo ================ GTK 4 cannot currently use cairo-quartz because of how CSS borders are rendered. It simply causes errors in the cairo_quartz_surface_t backend. Since we are restricted to using cairo_image_surface_t (which happens to be faster anyway) we can use the IOSurfaceBaseAddress() to obtain a mapping of the IOSurfaceRef in user-space. It always uses BGRA 32-bit with alpha channel even if we will discard the alpha channel as that is necessary to hit the fast paths in other parts of the platform. Note that while Cairo says CAIRO_FORMAT_ARGB32, it is really 32-bit BGRA on little-endian as we expect. OpenGL will render flipped (Quartz Native Co-ordinates) while Cairo renders with 0,O in the top-left. We could use cairo_translate() and cairo_scale() to reverse this, but it looks like some cairo things may not look quite as right if we do so. To reduce the chances of one-off bugs this continues to draw as Cairo would normally, but instead uses an CGAffineTransform in the tiles and some CGRect translation when swapping buffers to get the same effect. Changes to OpenGL ================= To simplify things, removal of all NSOpenGL* related components have been removed and we strictly use the Core GL (CGL*) API. This probably should have been done long ago anyay. Most examples found in the browsers to use IOSurfaceRef with OpenGL are using Legacy GL and there is still work underway to make this fit in with the rest of how the GSK GL renderer works. Since IOSurfaceRef bound to a texture/framebuffer will not have a default framebuffer ID of 0, we needed to add a default framebuffer id to the GdkGLContext. GskGLRenderer can use this to setup the command queue in such a way that our IOSurface destination has been glBindFramebuffer() as if it were the default drawable. This stuff is pretty slight-of-hand, so where things are and what needs flushing when and where has been a bit of an experiment to see what actually works to get synchronization across subsystems. Efficient Damages ================= After we draw with Cairo, we unlock the IOSurfaceRef and the contents are uploaded to the GPU. To make the contents visible to the app, we must clear the tiles contents with `layer.contents=nil;` and then re-apply the IOSurfaceRef. Since the buffer has likely not changed, we only do this if the tile overlaps the damage region. This gives the effect of having more tightly controlled damage regions even though updating the layer would damage be the whole window (as it is with OpenGL/Metal today with the exception of scissor-rect). This too can be verified usign "Quartz Debug -> Flash screen udpates". Frame Synchronized Resize ========================= In GTK 4, we have the ability to perform sizing changes from compute-size during the layout phase. Since the macOS backend already tracks window resizes manually, we can avoid doing the setFrame: immediately and instead do it within the frame clock's layout phase. Doing so gives us vastly better resize experience as we're more likely to get the size-change and updated-contents in the same frame on screen. It makes things feel "connected" in a way they weren't before. Some additional effort to tweak gravity during the process is also necessary but we were already doing that in the GTK 4 backend. Backporting =========== The design here has made an attempt to make it possible to backport by keeping GdkMacosBuffer, GdkMacosLayer, and GdkMacosTile fairly independent. There may be an opportunity to integrate this into GTK 3's quartz backend with a fair bit of work. Doing so could improve the situation for applications which are damage-rich such as The GIMP.
2022-02-14 10:20:19 +00:00
G_GNUC_BEGIN_IGNORE_DEPRECATIONS
macos: prototype new GDK backend for macOS This is fairly substantial rewrite of the GDK backend for quartz and renamed to macOS to allow for a greenfield implementation. Many things have come across from the quartz implementation fairly intact such as the eventloop integration design and discovery of event windows from the NSEvent. However much has been changed to fit in with the new GDK design and how removal of child GdkWindow have been completely eliminated. Furthermore, the new GdkPopup allows for regular NSWindow to be used to provide popovers unlike the previous implementation. The object design more closely follows the ideal for a GDK backend. Views have been broken out into subclasses so that we can support multiple GSK renderer paths such as GL and Cairo (and Metal in the future). However mixed mode GL and Cairo will not be supported. Currently only the Cairo renderer has been implemented. A new frame clock implementation using CVDisplayLink provides more accurate information about when to draw drawing the next frame. Some testing will need to be done here to understand the power implications of this. This implementation has also gained edge snapping for CSD windows. Some work was also done to ensure that CSD windows have opaque regions registered with the display server. ** This is still very much a work-in-progress ** Some outstanding work that needs to be done: - Finish a GL context for macOS and alternate NSView for GL rendering (possibly using speciailized CALayer for OpenGL). - Input rework to ensure that we don't loose remapping of keys that was dropped from GDK during GTK 4 development. - Make sure input methods continue to work. - Drag-n-Drop is still very much a work in progress - High resolution input scrolling needs various work in GDK to land first before we can plumb that to NSEvent. - gtk/ has a number of things based on GDK_WINDOWING_QUARTZ that need to be updated to use the macOS backend. But this is good enough to start playing with and breaking things which is what I'd like to see.
2020-04-23 23:36:46 +00:00
macos: use CALayer and IOSurface for rendering This provides a major shift in how we draw both when accelerated OpenGL as well as software rendering with Cairo. In short, it uses tiles of Core Animation's CALayer to display contents from an OpenGL or Cairo rendering so that the window can provide partial damage updates. Partial damage is not generally available when using OpenGL as the whole buffer is flipped even if you only submitted a small change using a scissor rect. Thankfully, this speeds up Cairo rendering a bit too by using IOSurface to upload contents to the display server. We use the tiling system we do for OpenGL which reduces overall complexity and differences between them. A New Buffer ============ GdkMacosBuffer is a wrapper around an IOSurfaceRef. The term buffer was used because 1) surface is already used and 2) it loosely maps to a front/back buffer semantic. However, it appears that IOSurfaceRef contents are being retained in some fashion (likely in the compositor result) so we can update the same IOSurfaceRef without flipping as long as we're fast. This appears to be what Chromium does as well, but Firefox uses two IOSurfaceRef and flips between them. We would like to avoid two surfaces because it doubles the GPU VRAM requirements of the application. Changes to Windows ================== Previously, the NSWindow would dynamically change between different types of NSView based on the renderer being used. This is no longer necessary as we just have a single NSView type, GdkMacosView, which inherits from GdkMacosBaseView just to keep the tedius stuff separate from the machinery of GdkMacosView. We can merge those someday if we are okay with that. Changes to Views ================ GdkMacosCairoView, GdkMacosCairoSubView, GdkMacosGLView have all been removed and replaced with GdkMacosView. This new view has a single CALayer (GdkMacosLayer) attached to it which itself has sublayers. The contents of the CALayer is populated with an IOSurfaceRef which we allocated with the GdkMacosSurface. The surface is replaced when the NSWindow resizes. Changes to Layers ================= We now have a dedicated GdkMacosLayer which contains sublayers of GdkMacosTile. The tile has a maximum size of 128x128 pixels in device units. The GdkMacosTile is partitioned by splitting both the transparent region (window bounds minus opaque area) and then by splitting the opaque area. A tile has either translucent contents (and therefore is not opaque) or has opaque contents (and therefore is opaque). An opaque tile never contains transparent contents. As such, the opaque tiles contain a black background so that Core Animation will consider the tile's bounds as opaque. This can be verified with "Quartz Debug -> Show opaque regions". Changes to Cairo ================ GTK 4 cannot currently use cairo-quartz because of how CSS borders are rendered. It simply causes errors in the cairo_quartz_surface_t backend. Since we are restricted to using cairo_image_surface_t (which happens to be faster anyway) we can use the IOSurfaceBaseAddress() to obtain a mapping of the IOSurfaceRef in user-space. It always uses BGRA 32-bit with alpha channel even if we will discard the alpha channel as that is necessary to hit the fast paths in other parts of the platform. Note that while Cairo says CAIRO_FORMAT_ARGB32, it is really 32-bit BGRA on little-endian as we expect. OpenGL will render flipped (Quartz Native Co-ordinates) while Cairo renders with 0,O in the top-left. We could use cairo_translate() and cairo_scale() to reverse this, but it looks like some cairo things may not look quite as right if we do so. To reduce the chances of one-off bugs this continues to draw as Cairo would normally, but instead uses an CGAffineTransform in the tiles and some CGRect translation when swapping buffers to get the same effect. Changes to OpenGL ================= To simplify things, removal of all NSOpenGL* related components have been removed and we strictly use the Core GL (CGL*) API. This probably should have been done long ago anyay. Most examples found in the browsers to use IOSurfaceRef with OpenGL are using Legacy GL and there is still work underway to make this fit in with the rest of how the GSK GL renderer works. Since IOSurfaceRef bound to a texture/framebuffer will not have a default framebuffer ID of 0, we needed to add a default framebuffer id to the GdkGLContext. GskGLRenderer can use this to setup the command queue in such a way that our IOSurface destination has been glBindFramebuffer() as if it were the default drawable. This stuff is pretty slight-of-hand, so where things are and what needs flushing when and where has been a bit of an experiment to see what actually works to get synchronization across subsystems. Efficient Damages ================= After we draw with Cairo, we unlock the IOSurfaceRef and the contents are uploaded to the GPU. To make the contents visible to the app, we must clear the tiles contents with `layer.contents=nil;` and then re-apply the IOSurfaceRef. Since the buffer has likely not changed, we only do this if the tile overlaps the damage region. This gives the effect of having more tightly controlled damage regions even though updating the layer would damage be the whole window (as it is with OpenGL/Metal today with the exception of scissor-rect). This too can be verified usign "Quartz Debug -> Flash screen udpates". Frame Synchronized Resize ========================= In GTK 4, we have the ability to perform sizing changes from compute-size during the layout phase. Since the macOS backend already tracks window resizes manually, we can avoid doing the setFrame: immediately and instead do it within the frame clock's layout phase. Doing so gives us vastly better resize experience as we're more likely to get the size-change and updated-contents in the same frame on screen. It makes things feel "connected" in a way they weren't before. Some additional effort to tweak gravity during the process is also necessary but we were already doing that in the GTK 4 backend. Backporting =========== The design here has made an attempt to make it possible to backport by keeping GdkMacosBuffer, GdkMacosLayer, and GdkMacosTile fairly independent. There may be an opportunity to integrate this into GTK 3's quartz backend with a fair bit of work. Doing so could improve the situation for applications which are damage-rich such as The GIMP.
2022-02-14 10:20:19 +00:00
G_DEFINE_TYPE (GdkMacosGLContext, gdk_macos_gl_context, GDK_TYPE_GL_CONTEXT)
macos: implement GL context This implements the basics for a GdkGLContext on macOS. Currently, rendering only is fully working for the GskCairoRenderer case where we read back pixels into a cairo surface for rendering. More work on synchronization is required for the GL on GskGLRenderer case. When we attempt to render a surface itself with GL, the context will ensure that the new GdkMacosGLView is placed within the NSWindow. In other cases, we use a dummy NSView and NSWindow for backing the NSOpenGLContext to ensure that we can get accelerated drawing. This gets GtkGLArea working when running with GSK_RENDERER=cairo.
2020-10-29 17:33:22 +00:00
macos: use CALayer and IOSurface for rendering This provides a major shift in how we draw both when accelerated OpenGL as well as software rendering with Cairo. In short, it uses tiles of Core Animation's CALayer to display contents from an OpenGL or Cairo rendering so that the window can provide partial damage updates. Partial damage is not generally available when using OpenGL as the whole buffer is flipped even if you only submitted a small change using a scissor rect. Thankfully, this speeds up Cairo rendering a bit too by using IOSurface to upload contents to the display server. We use the tiling system we do for OpenGL which reduces overall complexity and differences between them. A New Buffer ============ GdkMacosBuffer is a wrapper around an IOSurfaceRef. The term buffer was used because 1) surface is already used and 2) it loosely maps to a front/back buffer semantic. However, it appears that IOSurfaceRef contents are being retained in some fashion (likely in the compositor result) so we can update the same IOSurfaceRef without flipping as long as we're fast. This appears to be what Chromium does as well, but Firefox uses two IOSurfaceRef and flips between them. We would like to avoid two surfaces because it doubles the GPU VRAM requirements of the application. Changes to Windows ================== Previously, the NSWindow would dynamically change between different types of NSView based on the renderer being used. This is no longer necessary as we just have a single NSView type, GdkMacosView, which inherits from GdkMacosBaseView just to keep the tedius stuff separate from the machinery of GdkMacosView. We can merge those someday if we are okay with that. Changes to Views ================ GdkMacosCairoView, GdkMacosCairoSubView, GdkMacosGLView have all been removed and replaced with GdkMacosView. This new view has a single CALayer (GdkMacosLayer) attached to it which itself has sublayers. The contents of the CALayer is populated with an IOSurfaceRef which we allocated with the GdkMacosSurface. The surface is replaced when the NSWindow resizes. Changes to Layers ================= We now have a dedicated GdkMacosLayer which contains sublayers of GdkMacosTile. The tile has a maximum size of 128x128 pixels in device units. The GdkMacosTile is partitioned by splitting both the transparent region (window bounds minus opaque area) and then by splitting the opaque area. A tile has either translucent contents (and therefore is not opaque) or has opaque contents (and therefore is opaque). An opaque tile never contains transparent contents. As such, the opaque tiles contain a black background so that Core Animation will consider the tile's bounds as opaque. This can be verified with "Quartz Debug -> Show opaque regions". Changes to Cairo ================ GTK 4 cannot currently use cairo-quartz because of how CSS borders are rendered. It simply causes errors in the cairo_quartz_surface_t backend. Since we are restricted to using cairo_image_surface_t (which happens to be faster anyway) we can use the IOSurfaceBaseAddress() to obtain a mapping of the IOSurfaceRef in user-space. It always uses BGRA 32-bit with alpha channel even if we will discard the alpha channel as that is necessary to hit the fast paths in other parts of the platform. Note that while Cairo says CAIRO_FORMAT_ARGB32, it is really 32-bit BGRA on little-endian as we expect. OpenGL will render flipped (Quartz Native Co-ordinates) while Cairo renders with 0,O in the top-left. We could use cairo_translate() and cairo_scale() to reverse this, but it looks like some cairo things may not look quite as right if we do so. To reduce the chances of one-off bugs this continues to draw as Cairo would normally, but instead uses an CGAffineTransform in the tiles and some CGRect translation when swapping buffers to get the same effect. Changes to OpenGL ================= To simplify things, removal of all NSOpenGL* related components have been removed and we strictly use the Core GL (CGL*) API. This probably should have been done long ago anyay. Most examples found in the browsers to use IOSurfaceRef with OpenGL are using Legacy GL and there is still work underway to make this fit in with the rest of how the GSK GL renderer works. Since IOSurfaceRef bound to a texture/framebuffer will not have a default framebuffer ID of 0, we needed to add a default framebuffer id to the GdkGLContext. GskGLRenderer can use this to setup the command queue in such a way that our IOSurface destination has been glBindFramebuffer() as if it were the default drawable. This stuff is pretty slight-of-hand, so where things are and what needs flushing when and where has been a bit of an experiment to see what actually works to get synchronization across subsystems. Efficient Damages ================= After we draw with Cairo, we unlock the IOSurfaceRef and the contents are uploaded to the GPU. To make the contents visible to the app, we must clear the tiles contents with `layer.contents=nil;` and then re-apply the IOSurfaceRef. Since the buffer has likely not changed, we only do this if the tile overlaps the damage region. This gives the effect of having more tightly controlled damage regions even though updating the layer would damage be the whole window (as it is with OpenGL/Metal today with the exception of scissor-rect). This too can be verified usign "Quartz Debug -> Flash screen udpates". Frame Synchronized Resize ========================= In GTK 4, we have the ability to perform sizing changes from compute-size during the layout phase. Since the macOS backend already tracks window resizes manually, we can avoid doing the setFrame: immediately and instead do it within the frame clock's layout phase. Doing so gives us vastly better resize experience as we're more likely to get the size-change and updated-contents in the same frame on screen. It makes things feel "connected" in a way they weren't before. Some additional effort to tweak gravity during the process is also necessary but we were already doing that in the GTK 4 backend. Backporting =========== The design here has made an attempt to make it possible to backport by keeping GdkMacosBuffer, GdkMacosLayer, and GdkMacosTile fairly independent. There may be an opportunity to integrate this into GTK 3's quartz backend with a fair bit of work. Doing so could improve the situation for applications which are damage-rich such as The GIMP.
2022-02-14 10:20:19 +00:00
#define CHECK(error,cgl_error) _CHECK_CGL(error, G_STRLOC, cgl_error)
static inline gboolean
_CHECK_CGL (GError **error,
const char *location,
CGLError cgl_error)
{
if (cgl_error != kCGLNoError)
{
g_log ("Core OpenGL",
G_LOG_LEVEL_CRITICAL,
"%s: %s",
location, CGLErrorString (cgl_error));
g_set_error (error,
GDK_GL_ERROR,
GDK_GL_ERROR_NOT_AVAILABLE,
"%s",
CGLErrorString (cgl_error));
return FALSE;
}
macos: implement GL context This implements the basics for a GdkGLContext on macOS. Currently, rendering only is fully working for the GskCairoRenderer case where we read back pixels into a cairo surface for rendering. More work on synchronization is required for the GL on GskGLRenderer case. When we attempt to render a surface itself with GL, the context will ensure that the new GdkMacosGLView is placed within the NSWindow. In other cases, we use a dummy NSView and NSWindow for backing the NSOpenGLContext to ensure that we can get accelerated drawing. This gets GtkGLArea working when running with GSK_RENDERER=cairo.
2020-10-29 17:33:22 +00:00
macos: use CALayer and IOSurface for rendering This provides a major shift in how we draw both when accelerated OpenGL as well as software rendering with Cairo. In short, it uses tiles of Core Animation's CALayer to display contents from an OpenGL or Cairo rendering so that the window can provide partial damage updates. Partial damage is not generally available when using OpenGL as the whole buffer is flipped even if you only submitted a small change using a scissor rect. Thankfully, this speeds up Cairo rendering a bit too by using IOSurface to upload contents to the display server. We use the tiling system we do for OpenGL which reduces overall complexity and differences between them. A New Buffer ============ GdkMacosBuffer is a wrapper around an IOSurfaceRef. The term buffer was used because 1) surface is already used and 2) it loosely maps to a front/back buffer semantic. However, it appears that IOSurfaceRef contents are being retained in some fashion (likely in the compositor result) so we can update the same IOSurfaceRef without flipping as long as we're fast. This appears to be what Chromium does as well, but Firefox uses two IOSurfaceRef and flips between them. We would like to avoid two surfaces because it doubles the GPU VRAM requirements of the application. Changes to Windows ================== Previously, the NSWindow would dynamically change between different types of NSView based on the renderer being used. This is no longer necessary as we just have a single NSView type, GdkMacosView, which inherits from GdkMacosBaseView just to keep the tedius stuff separate from the machinery of GdkMacosView. We can merge those someday if we are okay with that. Changes to Views ================ GdkMacosCairoView, GdkMacosCairoSubView, GdkMacosGLView have all been removed and replaced with GdkMacosView. This new view has a single CALayer (GdkMacosLayer) attached to it which itself has sublayers. The contents of the CALayer is populated with an IOSurfaceRef which we allocated with the GdkMacosSurface. The surface is replaced when the NSWindow resizes. Changes to Layers ================= We now have a dedicated GdkMacosLayer which contains sublayers of GdkMacosTile. The tile has a maximum size of 128x128 pixels in device units. The GdkMacosTile is partitioned by splitting both the transparent region (window bounds minus opaque area) and then by splitting the opaque area. A tile has either translucent contents (and therefore is not opaque) or has opaque contents (and therefore is opaque). An opaque tile never contains transparent contents. As such, the opaque tiles contain a black background so that Core Animation will consider the tile's bounds as opaque. This can be verified with "Quartz Debug -> Show opaque regions". Changes to Cairo ================ GTK 4 cannot currently use cairo-quartz because of how CSS borders are rendered. It simply causes errors in the cairo_quartz_surface_t backend. Since we are restricted to using cairo_image_surface_t (which happens to be faster anyway) we can use the IOSurfaceBaseAddress() to obtain a mapping of the IOSurfaceRef in user-space. It always uses BGRA 32-bit with alpha channel even if we will discard the alpha channel as that is necessary to hit the fast paths in other parts of the platform. Note that while Cairo says CAIRO_FORMAT_ARGB32, it is really 32-bit BGRA on little-endian as we expect. OpenGL will render flipped (Quartz Native Co-ordinates) while Cairo renders with 0,O in the top-left. We could use cairo_translate() and cairo_scale() to reverse this, but it looks like some cairo things may not look quite as right if we do so. To reduce the chances of one-off bugs this continues to draw as Cairo would normally, but instead uses an CGAffineTransform in the tiles and some CGRect translation when swapping buffers to get the same effect. Changes to OpenGL ================= To simplify things, removal of all NSOpenGL* related components have been removed and we strictly use the Core GL (CGL*) API. This probably should have been done long ago anyay. Most examples found in the browsers to use IOSurfaceRef with OpenGL are using Legacy GL and there is still work underway to make this fit in with the rest of how the GSK GL renderer works. Since IOSurfaceRef bound to a texture/framebuffer will not have a default framebuffer ID of 0, we needed to add a default framebuffer id to the GdkGLContext. GskGLRenderer can use this to setup the command queue in such a way that our IOSurface destination has been glBindFramebuffer() as if it were the default drawable. This stuff is pretty slight-of-hand, so where things are and what needs flushing when and where has been a bit of an experiment to see what actually works to get synchronization across subsystems. Efficient Damages ================= After we draw with Cairo, we unlock the IOSurfaceRef and the contents are uploaded to the GPU. To make the contents visible to the app, we must clear the tiles contents with `layer.contents=nil;` and then re-apply the IOSurfaceRef. Since the buffer has likely not changed, we only do this if the tile overlaps the damage region. This gives the effect of having more tightly controlled damage regions even though updating the layer would damage be the whole window (as it is with OpenGL/Metal today with the exception of scissor-rect). This too can be verified usign "Quartz Debug -> Flash screen udpates". Frame Synchronized Resize ========================= In GTK 4, we have the ability to perform sizing changes from compute-size during the layout phase. Since the macOS backend already tracks window resizes manually, we can avoid doing the setFrame: immediately and instead do it within the frame clock's layout phase. Doing so gives us vastly better resize experience as we're more likely to get the size-change and updated-contents in the same frame on screen. It makes things feel "connected" in a way they weren't before. Some additional effort to tweak gravity during the process is also necessary but we were already doing that in the GTK 4 backend. Backporting =========== The design here has made an attempt to make it possible to backport by keeping GdkMacosBuffer, GdkMacosLayer, and GdkMacosTile fairly independent. There may be an opportunity to integrate this into GTK 3's quartz backend with a fair bit of work. Doing so could improve the situation for applications which are damage-rich such as The GIMP.
2022-02-14 10:20:19 +00:00
return TRUE;
}
macos: prototype new GDK backend for macOS This is fairly substantial rewrite of the GDK backend for quartz and renamed to macOS to allow for a greenfield implementation. Many things have come across from the quartz implementation fairly intact such as the eventloop integration design and discovery of event windows from the NSEvent. However much has been changed to fit in with the new GDK design and how removal of child GdkWindow have been completely eliminated. Furthermore, the new GdkPopup allows for regular NSWindow to be used to provide popovers unlike the previous implementation. The object design more closely follows the ideal for a GDK backend. Views have been broken out into subclasses so that we can support multiple GSK renderer paths such as GL and Cairo (and Metal in the future). However mixed mode GL and Cairo will not be supported. Currently only the Cairo renderer has been implemented. A new frame clock implementation using CVDisplayLink provides more accurate information about when to draw drawing the next frame. Some testing will need to be done here to understand the power implications of this. This implementation has also gained edge snapping for CSD windows. Some work was also done to ensure that CSD windows have opaque regions registered with the display server. ** This is still very much a work-in-progress ** Some outstanding work that needs to be done: - Finish a GL context for macOS and alternate NSView for GL rendering (possibly using speciailized CALayer for OpenGL). - Input rework to ensure that we don't loose remapping of keys that was dropped from GDK during GTK 4 development. - Make sure input methods continue to work. - Drag-n-Drop is still very much a work in progress - High resolution input scrolling needs various work in GDK to land first before we can plumb that to NSEvent. - gtk/ has a number of things based on GDK_WINDOWING_QUARTZ that need to be updated to use the macOS backend. But this is good enough to start playing with and breaking things which is what I'd like to see.
2020-04-23 23:36:46 +00:00
macos: use CALayer and IOSurface for rendering This provides a major shift in how we draw both when accelerated OpenGL as well as software rendering with Cairo. In short, it uses tiles of Core Animation's CALayer to display contents from an OpenGL or Cairo rendering so that the window can provide partial damage updates. Partial damage is not generally available when using OpenGL as the whole buffer is flipped even if you only submitted a small change using a scissor rect. Thankfully, this speeds up Cairo rendering a bit too by using IOSurface to upload contents to the display server. We use the tiling system we do for OpenGL which reduces overall complexity and differences between them. A New Buffer ============ GdkMacosBuffer is a wrapper around an IOSurfaceRef. The term buffer was used because 1) surface is already used and 2) it loosely maps to a front/back buffer semantic. However, it appears that IOSurfaceRef contents are being retained in some fashion (likely in the compositor result) so we can update the same IOSurfaceRef without flipping as long as we're fast. This appears to be what Chromium does as well, but Firefox uses two IOSurfaceRef and flips between them. We would like to avoid two surfaces because it doubles the GPU VRAM requirements of the application. Changes to Windows ================== Previously, the NSWindow would dynamically change between different types of NSView based on the renderer being used. This is no longer necessary as we just have a single NSView type, GdkMacosView, which inherits from GdkMacosBaseView just to keep the tedius stuff separate from the machinery of GdkMacosView. We can merge those someday if we are okay with that. Changes to Views ================ GdkMacosCairoView, GdkMacosCairoSubView, GdkMacosGLView have all been removed and replaced with GdkMacosView. This new view has a single CALayer (GdkMacosLayer) attached to it which itself has sublayers. The contents of the CALayer is populated with an IOSurfaceRef which we allocated with the GdkMacosSurface. The surface is replaced when the NSWindow resizes. Changes to Layers ================= We now have a dedicated GdkMacosLayer which contains sublayers of GdkMacosTile. The tile has a maximum size of 128x128 pixels in device units. The GdkMacosTile is partitioned by splitting both the transparent region (window bounds minus opaque area) and then by splitting the opaque area. A tile has either translucent contents (and therefore is not opaque) or has opaque contents (and therefore is opaque). An opaque tile never contains transparent contents. As such, the opaque tiles contain a black background so that Core Animation will consider the tile's bounds as opaque. This can be verified with "Quartz Debug -> Show opaque regions". Changes to Cairo ================ GTK 4 cannot currently use cairo-quartz because of how CSS borders are rendered. It simply causes errors in the cairo_quartz_surface_t backend. Since we are restricted to using cairo_image_surface_t (which happens to be faster anyway) we can use the IOSurfaceBaseAddress() to obtain a mapping of the IOSurfaceRef in user-space. It always uses BGRA 32-bit with alpha channel even if we will discard the alpha channel as that is necessary to hit the fast paths in other parts of the platform. Note that while Cairo says CAIRO_FORMAT_ARGB32, it is really 32-bit BGRA on little-endian as we expect. OpenGL will render flipped (Quartz Native Co-ordinates) while Cairo renders with 0,O in the top-left. We could use cairo_translate() and cairo_scale() to reverse this, but it looks like some cairo things may not look quite as right if we do so. To reduce the chances of one-off bugs this continues to draw as Cairo would normally, but instead uses an CGAffineTransform in the tiles and some CGRect translation when swapping buffers to get the same effect. Changes to OpenGL ================= To simplify things, removal of all NSOpenGL* related components have been removed and we strictly use the Core GL (CGL*) API. This probably should have been done long ago anyay. Most examples found in the browsers to use IOSurfaceRef with OpenGL are using Legacy GL and there is still work underway to make this fit in with the rest of how the GSK GL renderer works. Since IOSurfaceRef bound to a texture/framebuffer will not have a default framebuffer ID of 0, we needed to add a default framebuffer id to the GdkGLContext. GskGLRenderer can use this to setup the command queue in such a way that our IOSurface destination has been glBindFramebuffer() as if it were the default drawable. This stuff is pretty slight-of-hand, so where things are and what needs flushing when and where has been a bit of an experiment to see what actually works to get synchronization across subsystems. Efficient Damages ================= After we draw with Cairo, we unlock the IOSurfaceRef and the contents are uploaded to the GPU. To make the contents visible to the app, we must clear the tiles contents with `layer.contents=nil;` and then re-apply the IOSurfaceRef. Since the buffer has likely not changed, we only do this if the tile overlaps the damage region. This gives the effect of having more tightly controlled damage regions even though updating the layer would damage be the whole window (as it is with OpenGL/Metal today with the exception of scissor-rect). This too can be verified usign "Quartz Debug -> Flash screen udpates". Frame Synchronized Resize ========================= In GTK 4, we have the ability to perform sizing changes from compute-size during the layout phase. Since the macOS backend already tracks window resizes manually, we can avoid doing the setFrame: immediately and instead do it within the frame clock's layout phase. Doing so gives us vastly better resize experience as we're more likely to get the size-change and updated-contents in the same frame on screen. It makes things feel "connected" in a way they weren't before. Some additional effort to tweak gravity during the process is also necessary but we were already doing that in the GTK 4 backend. Backporting =========== The design here has made an attempt to make it possible to backport by keeping GdkMacosBuffer, GdkMacosLayer, and GdkMacosTile fairly independent. There may be an opportunity to integrate this into GTK 3's quartz backend with a fair bit of work. Doing so could improve the situation for applications which are damage-rich such as The GIMP.
2022-02-14 10:20:19 +00:00
/* Apple's OpenGL implementation does not contain the extension to
* perform log handler callbacks when errors occur. Therefore, to aid in
* tracking down issues we have a CHECK_GL() macro that can wrap GL
* calls and check for an error afterwards.
*
* To make this easier, we use a statement expression, as this will
* always be using something GCC-compatible on macOS.
*/
#define CHECK_GL(error,func) _CHECK_GL(error, G_STRLOC, ({ func; glGetError(); }))
static inline gboolean
_CHECK_GL (GError **error,
const char *location,
GLenum gl_error)
{
const char *msg;
switch (gl_error)
{
case GL_INVALID_ENUM:
msg = "invalid enum";
break;
case GL_INVALID_VALUE:
msg = "invalid value";
break;
case GL_INVALID_OPERATION:
msg = "invalid operation";
break;
case GL_INVALID_FRAMEBUFFER_OPERATION:
msg = "invalid framebuffer operation";
break;
case GL_OUT_OF_MEMORY:
msg = "out of memory";
break;
default:
msg = "unknown error";
break;
}
if (gl_error != GL_NO_ERROR)
{
g_log ("OpenGL",
G_LOG_LEVEL_CRITICAL,
"%s: %s", location, msg);
if (error != NULL)
g_set_error (error,
GDK_GL_ERROR,
GDK_GL_ERROR_NOT_AVAILABLE,
"%s", msg);
return FALSE;
}
return TRUE;
}
static gboolean
check_framebuffer_status (GLenum target)
{
switch (glCheckFramebufferStatus (target))
{
case GL_FRAMEBUFFER_COMPLETE:
return TRUE;
case GL_FRAMEBUFFER_UNDEFINED:
g_critical ("Framebuffer is undefined");
return FALSE;
case GL_FRAMEBUFFER_INCOMPLETE_ATTACHMENT:
g_critical ("Framebuffer has incomplete attachment");
return FALSE;
case GL_FRAMEBUFFER_INCOMPLETE_MISSING_ATTACHMENT:
g_critical ("Framebuffer has missing attachment");
return FALSE;
case GL_FRAMEBUFFER_INCOMPLETE_DRAW_BUFFER:
g_critical ("Framebuffer has incomplete draw buffer");
return FALSE;
case GL_FRAMEBUFFER_INCOMPLETE_READ_BUFFER:
g_critical ("Framebuffer has incomplete read buffer");
return FALSE;
case GL_FRAMEBUFFER_UNSUPPORTED:
g_critical ("Framebuffer is unsupported");
return FALSE;
case GL_FRAMEBUFFER_INCOMPLETE_MULTISAMPLE:
g_critical ("Framebuffer has incomplete multisample");
return FALSE;
default:
g_critical ("Framebuffer has unknown error");
return FALSE;
}
}
macos: prototype new GDK backend for macOS This is fairly substantial rewrite of the GDK backend for quartz and renamed to macOS to allow for a greenfield implementation. Many things have come across from the quartz implementation fairly intact such as the eventloop integration design and discovery of event windows from the NSEvent. However much has been changed to fit in with the new GDK design and how removal of child GdkWindow have been completely eliminated. Furthermore, the new GdkPopup allows for regular NSWindow to be used to provide popovers unlike the previous implementation. The object design more closely follows the ideal for a GDK backend. Views have been broken out into subclasses so that we can support multiple GSK renderer paths such as GL and Cairo (and Metal in the future). However mixed mode GL and Cairo will not be supported. Currently only the Cairo renderer has been implemented. A new frame clock implementation using CVDisplayLink provides more accurate information about when to draw drawing the next frame. Some testing will need to be done here to understand the power implications of this. This implementation has also gained edge snapping for CSD windows. Some work was also done to ensure that CSD windows have opaque regions registered with the display server. ** This is still very much a work-in-progress ** Some outstanding work that needs to be done: - Finish a GL context for macOS and alternate NSView for GL rendering (possibly using speciailized CALayer for OpenGL). - Input rework to ensure that we don't loose remapping of keys that was dropped from GDK during GTK 4 development. - Make sure input methods continue to work. - Drag-n-Drop is still very much a work in progress - High resolution input scrolling needs various work in GDK to land first before we can plumb that to NSEvent. - gtk/ has a number of things based on GDK_WINDOWING_QUARTZ that need to be updated to use the macOS backend. But this is good enough to start playing with and breaking things which is what I'd like to see.
2020-04-23 23:36:46 +00:00
macos: implement GL context This implements the basics for a GdkGLContext on macOS. Currently, rendering only is fully working for the GskCairoRenderer case where we read back pixels into a cairo surface for rendering. More work on synchronization is required for the GL on GskGLRenderer case. When we attempt to render a surface itself with GL, the context will ensure that the new GdkMacosGLView is placed within the NSWindow. In other cases, we use a dummy NSView and NSWindow for backing the NSOpenGLContext to ensure that we can get accelerated drawing. This gets GtkGLArea working when running with GSK_RENDERER=cairo.
2020-10-29 17:33:22 +00:00
static const char *
get_renderer_name (GLint id)
{
static char renderer_name[32];
switch (id & kCGLRendererIDMatchingMask)
{
case kCGLRendererGenericID: return "Generic";
case kCGLRendererGenericFloatID: return "Generic Float";
case kCGLRendererAppleSWID: return "Apple Software Renderer";
case kCGLRendererATIRage128ID: return "ATI Rage 128";
case kCGLRendererATIRadeonID: return "ATI Radeon";
case kCGLRendererATIRageProID: return "ATI Rage Pro";
case kCGLRendererATIRadeon8500ID: return "ATI Radeon 8500";
case kCGLRendererATIRadeon9700ID: return "ATI Radeon 9700";
case kCGLRendererATIRadeonX1000ID: return "ATI Radeon X1000";
case kCGLRendererATIRadeonX2000ID: return "ATI Radeon X2000";
case kCGLRendererATIRadeonX3000ID: return "ATI Radeon X3000";
case kCGLRendererATIRadeonX4000ID: return "ATI Radeon X4000";
case kCGLRendererGeForce2MXID: return "GeForce 2 MX";
case kCGLRendererGeForce3ID: return "GeForce 3";
case kCGLRendererGeForceFXID: return "GeForce FX";
case kCGLRendererGeForce8xxxID: return "GeForce 8xxx";
case kCGLRendererGeForceID: return "GeForce";
case kCGLRendererVTBladeXP2ID: return "VT Blade XP 2";
case kCGLRendererIntel900ID: return "Intel 900";
case kCGLRendererIntelX3100ID: return "Intel X3100";
case kCGLRendererIntelHDID: return "Intel HD";
case kCGLRendererIntelHD4000ID: return "Intel HD 4000";
case kCGLRendererIntelHD5000ID: return "Intel HD 5000";
case kCGLRendererMesa3DFXID: return "Mesa 3DFX";
default:
snprintf (renderer_name, sizeof renderer_name, "0x%08x", id & kCGLRendererIDMatchingMask);
renderer_name[sizeof renderer_name-1] = 0;
return renderer_name;
}
}
macos: use CALayer and IOSurface for rendering This provides a major shift in how we draw both when accelerated OpenGL as well as software rendering with Cairo. In short, it uses tiles of Core Animation's CALayer to display contents from an OpenGL or Cairo rendering so that the window can provide partial damage updates. Partial damage is not generally available when using OpenGL as the whole buffer is flipped even if you only submitted a small change using a scissor rect. Thankfully, this speeds up Cairo rendering a bit too by using IOSurface to upload contents to the display server. We use the tiling system we do for OpenGL which reduces overall complexity and differences between them. A New Buffer ============ GdkMacosBuffer is a wrapper around an IOSurfaceRef. The term buffer was used because 1) surface is already used and 2) it loosely maps to a front/back buffer semantic. However, it appears that IOSurfaceRef contents are being retained in some fashion (likely in the compositor result) so we can update the same IOSurfaceRef without flipping as long as we're fast. This appears to be what Chromium does as well, but Firefox uses two IOSurfaceRef and flips between them. We would like to avoid two surfaces because it doubles the GPU VRAM requirements of the application. Changes to Windows ================== Previously, the NSWindow would dynamically change between different types of NSView based on the renderer being used. This is no longer necessary as we just have a single NSView type, GdkMacosView, which inherits from GdkMacosBaseView just to keep the tedius stuff separate from the machinery of GdkMacosView. We can merge those someday if we are okay with that. Changes to Views ================ GdkMacosCairoView, GdkMacosCairoSubView, GdkMacosGLView have all been removed and replaced with GdkMacosView. This new view has a single CALayer (GdkMacosLayer) attached to it which itself has sublayers. The contents of the CALayer is populated with an IOSurfaceRef which we allocated with the GdkMacosSurface. The surface is replaced when the NSWindow resizes. Changes to Layers ================= We now have a dedicated GdkMacosLayer which contains sublayers of GdkMacosTile. The tile has a maximum size of 128x128 pixels in device units. The GdkMacosTile is partitioned by splitting both the transparent region (window bounds minus opaque area) and then by splitting the opaque area. A tile has either translucent contents (and therefore is not opaque) or has opaque contents (and therefore is opaque). An opaque tile never contains transparent contents. As such, the opaque tiles contain a black background so that Core Animation will consider the tile's bounds as opaque. This can be verified with "Quartz Debug -> Show opaque regions". Changes to Cairo ================ GTK 4 cannot currently use cairo-quartz because of how CSS borders are rendered. It simply causes errors in the cairo_quartz_surface_t backend. Since we are restricted to using cairo_image_surface_t (which happens to be faster anyway) we can use the IOSurfaceBaseAddress() to obtain a mapping of the IOSurfaceRef in user-space. It always uses BGRA 32-bit with alpha channel even if we will discard the alpha channel as that is necessary to hit the fast paths in other parts of the platform. Note that while Cairo says CAIRO_FORMAT_ARGB32, it is really 32-bit BGRA on little-endian as we expect. OpenGL will render flipped (Quartz Native Co-ordinates) while Cairo renders with 0,O in the top-left. We could use cairo_translate() and cairo_scale() to reverse this, but it looks like some cairo things may not look quite as right if we do so. To reduce the chances of one-off bugs this continues to draw as Cairo would normally, but instead uses an CGAffineTransform in the tiles and some CGRect translation when swapping buffers to get the same effect. Changes to OpenGL ================= To simplify things, removal of all NSOpenGL* related components have been removed and we strictly use the Core GL (CGL*) API. This probably should have been done long ago anyay. Most examples found in the browsers to use IOSurfaceRef with OpenGL are using Legacy GL and there is still work underway to make this fit in with the rest of how the GSK GL renderer works. Since IOSurfaceRef bound to a texture/framebuffer will not have a default framebuffer ID of 0, we needed to add a default framebuffer id to the GdkGLContext. GskGLRenderer can use this to setup the command queue in such a way that our IOSurface destination has been glBindFramebuffer() as if it were the default drawable. This stuff is pretty slight-of-hand, so where things are and what needs flushing when and where has been a bit of an experiment to see what actually works to get synchronization across subsystems. Efficient Damages ================= After we draw with Cairo, we unlock the IOSurfaceRef and the contents are uploaded to the GPU. To make the contents visible to the app, we must clear the tiles contents with `layer.contents=nil;` and then re-apply the IOSurfaceRef. Since the buffer has likely not changed, we only do this if the tile overlaps the damage region. This gives the effect of having more tightly controlled damage regions even though updating the layer would damage be the whole window (as it is with OpenGL/Metal today with the exception of scissor-rect). This too can be verified usign "Quartz Debug -> Flash screen udpates". Frame Synchronized Resize ========================= In GTK 4, we have the ability to perform sizing changes from compute-size during the layout phase. Since the macOS backend already tracks window resizes manually, we can avoid doing the setFrame: immediately and instead do it within the frame clock's layout phase. Doing so gives us vastly better resize experience as we're more likely to get the size-change and updated-contents in the same frame on screen. It makes things feel "connected" in a way they weren't before. Some additional effort to tweak gravity during the process is also necessary but we were already doing that in the GTK 4 backend. Backporting =========== The design here has made an attempt to make it possible to backport by keeping GdkMacosBuffer, GdkMacosLayer, and GdkMacosTile fairly independent. There may be an opportunity to integrate this into GTK 3's quartz backend with a fair bit of work. Doing so could improve the situation for applications which are damage-rich such as The GIMP.
2022-02-14 10:20:19 +00:00
static GLuint
create_texture (CGLContextObj cgl_context,
GLuint target,
IOSurfaceRef io_surface,
guint width,
guint height)
macos: implement GL context This implements the basics for a GdkGLContext on macOS. Currently, rendering only is fully working for the GskCairoRenderer case where we read back pixels into a cairo surface for rendering. More work on synchronization is required for the GL on GskGLRenderer case. When we attempt to render a surface itself with GL, the context will ensure that the new GdkMacosGLView is placed within the NSWindow. In other cases, we use a dummy NSView and NSWindow for backing the NSOpenGLContext to ensure that we can get accelerated drawing. This gets GtkGLArea working when running with GSK_RENDERER=cairo.
2020-10-29 17:33:22 +00:00
{
macos: use CALayer and IOSurface for rendering This provides a major shift in how we draw both when accelerated OpenGL as well as software rendering with Cairo. In short, it uses tiles of Core Animation's CALayer to display contents from an OpenGL or Cairo rendering so that the window can provide partial damage updates. Partial damage is not generally available when using OpenGL as the whole buffer is flipped even if you only submitted a small change using a scissor rect. Thankfully, this speeds up Cairo rendering a bit too by using IOSurface to upload contents to the display server. We use the tiling system we do for OpenGL which reduces overall complexity and differences between them. A New Buffer ============ GdkMacosBuffer is a wrapper around an IOSurfaceRef. The term buffer was used because 1) surface is already used and 2) it loosely maps to a front/back buffer semantic. However, it appears that IOSurfaceRef contents are being retained in some fashion (likely in the compositor result) so we can update the same IOSurfaceRef without flipping as long as we're fast. This appears to be what Chromium does as well, but Firefox uses two IOSurfaceRef and flips between them. We would like to avoid two surfaces because it doubles the GPU VRAM requirements of the application. Changes to Windows ================== Previously, the NSWindow would dynamically change between different types of NSView based on the renderer being used. This is no longer necessary as we just have a single NSView type, GdkMacosView, which inherits from GdkMacosBaseView just to keep the tedius stuff separate from the machinery of GdkMacosView. We can merge those someday if we are okay with that. Changes to Views ================ GdkMacosCairoView, GdkMacosCairoSubView, GdkMacosGLView have all been removed and replaced with GdkMacosView. This new view has a single CALayer (GdkMacosLayer) attached to it which itself has sublayers. The contents of the CALayer is populated with an IOSurfaceRef which we allocated with the GdkMacosSurface. The surface is replaced when the NSWindow resizes. Changes to Layers ================= We now have a dedicated GdkMacosLayer which contains sublayers of GdkMacosTile. The tile has a maximum size of 128x128 pixels in device units. The GdkMacosTile is partitioned by splitting both the transparent region (window bounds minus opaque area) and then by splitting the opaque area. A tile has either translucent contents (and therefore is not opaque) or has opaque contents (and therefore is opaque). An opaque tile never contains transparent contents. As such, the opaque tiles contain a black background so that Core Animation will consider the tile's bounds as opaque. This can be verified with "Quartz Debug -> Show opaque regions". Changes to Cairo ================ GTK 4 cannot currently use cairo-quartz because of how CSS borders are rendered. It simply causes errors in the cairo_quartz_surface_t backend. Since we are restricted to using cairo_image_surface_t (which happens to be faster anyway) we can use the IOSurfaceBaseAddress() to obtain a mapping of the IOSurfaceRef in user-space. It always uses BGRA 32-bit with alpha channel even if we will discard the alpha channel as that is necessary to hit the fast paths in other parts of the platform. Note that while Cairo says CAIRO_FORMAT_ARGB32, it is really 32-bit BGRA on little-endian as we expect. OpenGL will render flipped (Quartz Native Co-ordinates) while Cairo renders with 0,O in the top-left. We could use cairo_translate() and cairo_scale() to reverse this, but it looks like some cairo things may not look quite as right if we do so. To reduce the chances of one-off bugs this continues to draw as Cairo would normally, but instead uses an CGAffineTransform in the tiles and some CGRect translation when swapping buffers to get the same effect. Changes to OpenGL ================= To simplify things, removal of all NSOpenGL* related components have been removed and we strictly use the Core GL (CGL*) API. This probably should have been done long ago anyay. Most examples found in the browsers to use IOSurfaceRef with OpenGL are using Legacy GL and there is still work underway to make this fit in with the rest of how the GSK GL renderer works. Since IOSurfaceRef bound to a texture/framebuffer will not have a default framebuffer ID of 0, we needed to add a default framebuffer id to the GdkGLContext. GskGLRenderer can use this to setup the command queue in such a way that our IOSurface destination has been glBindFramebuffer() as if it were the default drawable. This stuff is pretty slight-of-hand, so where things are and what needs flushing when and where has been a bit of an experiment to see what actually works to get synchronization across subsystems. Efficient Damages ================= After we draw with Cairo, we unlock the IOSurfaceRef and the contents are uploaded to the GPU. To make the contents visible to the app, we must clear the tiles contents with `layer.contents=nil;` and then re-apply the IOSurfaceRef. Since the buffer has likely not changed, we only do this if the tile overlaps the damage region. This gives the effect of having more tightly controlled damage regions even though updating the layer would damage be the whole window (as it is with OpenGL/Metal today with the exception of scissor-rect). This too can be verified usign "Quartz Debug -> Flash screen udpates". Frame Synchronized Resize ========================= In GTK 4, we have the ability to perform sizing changes from compute-size during the layout phase. Since the macOS backend already tracks window resizes manually, we can avoid doing the setFrame: immediately and instead do it within the frame clock's layout phase. Doing so gives us vastly better resize experience as we're more likely to get the size-change and updated-contents in the same frame on screen. It makes things feel "connected" in a way they weren't before. Some additional effort to tweak gravity during the process is also necessary but we were already doing that in the GTK 4 backend. Backporting =========== The design here has made an attempt to make it possible to backport by keeping GdkMacosBuffer, GdkMacosLayer, and GdkMacosTile fairly independent. There may be an opportunity to integrate this into GTK 3's quartz backend with a fair bit of work. Doing so could improve the situation for applications which are damage-rich such as The GIMP.
2022-02-14 10:20:19 +00:00
GLuint texture = 0;
if (!CHECK_GL (NULL, glActiveTexture (GL_TEXTURE0)) ||
!CHECK_GL (NULL, glGenTextures (1, &texture)) ||
!CHECK_GL (NULL, glBindTexture (target, texture)) ||
!CHECK (NULL, CGLTexImageIOSurface2D (cgl_context,
target,
GL_RGBA,
width,
height,
GL_BGRA,
GL_UNSIGNED_INT_8_8_8_8_REV,
io_surface,
0)) ||
!CHECK_GL (NULL, glTexParameteri (target, GL_TEXTURE_BASE_LEVEL, 0)) ||
!CHECK_GL (NULL, glTexParameteri (target, GL_TEXTURE_MIN_FILTER, GL_NEAREST)) ||
!CHECK_GL (NULL, glTexParameteri (target, GL_TEXTURE_MAG_FILTER, GL_NEAREST)) ||
!CHECK_GL (NULL, glTexParameteri (target, GL_TEXTURE_WRAP_S, GL_CLAMP_TO_EDGE)) ||
!CHECK_GL (NULL, glTexParameteri (target, GL_TEXTURE_WRAP_R, GL_CLAMP_TO_EDGE)) ||
!CHECK_GL (NULL, glTexParameteri (target, GL_TEXTURE_WRAP_T, GL_CLAMP_TO_EDGE)) ||
!CHECK_GL (NULL, glBindTexture (target, 0)))
macos: implement GL context This implements the basics for a GdkGLContext on macOS. Currently, rendering only is fully working for the GskCairoRenderer case where we read back pixels into a cairo surface for rendering. More work on synchronization is required for the GL on GskGLRenderer case. When we attempt to render a surface itself with GL, the context will ensure that the new GdkMacosGLView is placed within the NSWindow. In other cases, we use a dummy NSView and NSWindow for backing the NSOpenGLContext to ensure that we can get accelerated drawing. This gets GtkGLArea working when running with GSK_RENDERER=cairo.
2020-10-29 17:33:22 +00:00
{
macos: use CALayer and IOSurface for rendering This provides a major shift in how we draw both when accelerated OpenGL as well as software rendering with Cairo. In short, it uses tiles of Core Animation's CALayer to display contents from an OpenGL or Cairo rendering so that the window can provide partial damage updates. Partial damage is not generally available when using OpenGL as the whole buffer is flipped even if you only submitted a small change using a scissor rect. Thankfully, this speeds up Cairo rendering a bit too by using IOSurface to upload contents to the display server. We use the tiling system we do for OpenGL which reduces overall complexity and differences between them. A New Buffer ============ GdkMacosBuffer is a wrapper around an IOSurfaceRef. The term buffer was used because 1) surface is already used and 2) it loosely maps to a front/back buffer semantic. However, it appears that IOSurfaceRef contents are being retained in some fashion (likely in the compositor result) so we can update the same IOSurfaceRef without flipping as long as we're fast. This appears to be what Chromium does as well, but Firefox uses two IOSurfaceRef and flips between them. We would like to avoid two surfaces because it doubles the GPU VRAM requirements of the application. Changes to Windows ================== Previously, the NSWindow would dynamically change between different types of NSView based on the renderer being used. This is no longer necessary as we just have a single NSView type, GdkMacosView, which inherits from GdkMacosBaseView just to keep the tedius stuff separate from the machinery of GdkMacosView. We can merge those someday if we are okay with that. Changes to Views ================ GdkMacosCairoView, GdkMacosCairoSubView, GdkMacosGLView have all been removed and replaced with GdkMacosView. This new view has a single CALayer (GdkMacosLayer) attached to it which itself has sublayers. The contents of the CALayer is populated with an IOSurfaceRef which we allocated with the GdkMacosSurface. The surface is replaced when the NSWindow resizes. Changes to Layers ================= We now have a dedicated GdkMacosLayer which contains sublayers of GdkMacosTile. The tile has a maximum size of 128x128 pixels in device units. The GdkMacosTile is partitioned by splitting both the transparent region (window bounds minus opaque area) and then by splitting the opaque area. A tile has either translucent contents (and therefore is not opaque) or has opaque contents (and therefore is opaque). An opaque tile never contains transparent contents. As such, the opaque tiles contain a black background so that Core Animation will consider the tile's bounds as opaque. This can be verified with "Quartz Debug -> Show opaque regions". Changes to Cairo ================ GTK 4 cannot currently use cairo-quartz because of how CSS borders are rendered. It simply causes errors in the cairo_quartz_surface_t backend. Since we are restricted to using cairo_image_surface_t (which happens to be faster anyway) we can use the IOSurfaceBaseAddress() to obtain a mapping of the IOSurfaceRef in user-space. It always uses BGRA 32-bit with alpha channel even if we will discard the alpha channel as that is necessary to hit the fast paths in other parts of the platform. Note that while Cairo says CAIRO_FORMAT_ARGB32, it is really 32-bit BGRA on little-endian as we expect. OpenGL will render flipped (Quartz Native Co-ordinates) while Cairo renders with 0,O in the top-left. We could use cairo_translate() and cairo_scale() to reverse this, but it looks like some cairo things may not look quite as right if we do so. To reduce the chances of one-off bugs this continues to draw as Cairo would normally, but instead uses an CGAffineTransform in the tiles and some CGRect translation when swapping buffers to get the same effect. Changes to OpenGL ================= To simplify things, removal of all NSOpenGL* related components have been removed and we strictly use the Core GL (CGL*) API. This probably should have been done long ago anyay. Most examples found in the browsers to use IOSurfaceRef with OpenGL are using Legacy GL and there is still work underway to make this fit in with the rest of how the GSK GL renderer works. Since IOSurfaceRef bound to a texture/framebuffer will not have a default framebuffer ID of 0, we needed to add a default framebuffer id to the GdkGLContext. GskGLRenderer can use this to setup the command queue in such a way that our IOSurface destination has been glBindFramebuffer() as if it were the default drawable. This stuff is pretty slight-of-hand, so where things are and what needs flushing when and where has been a bit of an experiment to see what actually works to get synchronization across subsystems. Efficient Damages ================= After we draw with Cairo, we unlock the IOSurfaceRef and the contents are uploaded to the GPU. To make the contents visible to the app, we must clear the tiles contents with `layer.contents=nil;` and then re-apply the IOSurfaceRef. Since the buffer has likely not changed, we only do this if the tile overlaps the damage region. This gives the effect of having more tightly controlled damage regions even though updating the layer would damage be the whole window (as it is with OpenGL/Metal today with the exception of scissor-rect). This too can be verified usign "Quartz Debug -> Flash screen udpates". Frame Synchronized Resize ========================= In GTK 4, we have the ability to perform sizing changes from compute-size during the layout phase. Since the macOS backend already tracks window resizes manually, we can avoid doing the setFrame: immediately and instead do it within the frame clock's layout phase. Doing so gives us vastly better resize experience as we're more likely to get the size-change and updated-contents in the same frame on screen. It makes things feel "connected" in a way they weren't before. Some additional effort to tweak gravity during the process is also necessary but we were already doing that in the GTK 4 backend. Backporting =========== The design here has made an attempt to make it possible to backport by keeping GdkMacosBuffer, GdkMacosLayer, and GdkMacosTile fairly independent. There may be an opportunity to integrate this into GTK 3's quartz backend with a fair bit of work. Doing so could improve the situation for applications which are damage-rich such as The GIMP.
2022-02-14 10:20:19 +00:00
glDeleteTextures (1, &texture);
return 0;
macos: implement GL context This implements the basics for a GdkGLContext on macOS. Currently, rendering only is fully working for the GskCairoRenderer case where we read back pixels into a cairo surface for rendering. More work on synchronization is required for the GL on GskGLRenderer case. When we attempt to render a surface itself with GL, the context will ensure that the new GdkMacosGLView is placed within the NSWindow. In other cases, we use a dummy NSView and NSWindow for backing the NSOpenGLContext to ensure that we can get accelerated drawing. This gets GtkGLArea working when running with GSK_RENDERER=cairo.
2020-10-29 17:33:22 +00:00
}
macos: use CALayer and IOSurface for rendering This provides a major shift in how we draw both when accelerated OpenGL as well as software rendering with Cairo. In short, it uses tiles of Core Animation's CALayer to display contents from an OpenGL or Cairo rendering so that the window can provide partial damage updates. Partial damage is not generally available when using OpenGL as the whole buffer is flipped even if you only submitted a small change using a scissor rect. Thankfully, this speeds up Cairo rendering a bit too by using IOSurface to upload contents to the display server. We use the tiling system we do for OpenGL which reduces overall complexity and differences between them. A New Buffer ============ GdkMacosBuffer is a wrapper around an IOSurfaceRef. The term buffer was used because 1) surface is already used and 2) it loosely maps to a front/back buffer semantic. However, it appears that IOSurfaceRef contents are being retained in some fashion (likely in the compositor result) so we can update the same IOSurfaceRef without flipping as long as we're fast. This appears to be what Chromium does as well, but Firefox uses two IOSurfaceRef and flips between them. We would like to avoid two surfaces because it doubles the GPU VRAM requirements of the application. Changes to Windows ================== Previously, the NSWindow would dynamically change between different types of NSView based on the renderer being used. This is no longer necessary as we just have a single NSView type, GdkMacosView, which inherits from GdkMacosBaseView just to keep the tedius stuff separate from the machinery of GdkMacosView. We can merge those someday if we are okay with that. Changes to Views ================ GdkMacosCairoView, GdkMacosCairoSubView, GdkMacosGLView have all been removed and replaced with GdkMacosView. This new view has a single CALayer (GdkMacosLayer) attached to it which itself has sublayers. The contents of the CALayer is populated with an IOSurfaceRef which we allocated with the GdkMacosSurface. The surface is replaced when the NSWindow resizes. Changes to Layers ================= We now have a dedicated GdkMacosLayer which contains sublayers of GdkMacosTile. The tile has a maximum size of 128x128 pixels in device units. The GdkMacosTile is partitioned by splitting both the transparent region (window bounds minus opaque area) and then by splitting the opaque area. A tile has either translucent contents (and therefore is not opaque) or has opaque contents (and therefore is opaque). An opaque tile never contains transparent contents. As such, the opaque tiles contain a black background so that Core Animation will consider the tile's bounds as opaque. This can be verified with "Quartz Debug -> Show opaque regions". Changes to Cairo ================ GTK 4 cannot currently use cairo-quartz because of how CSS borders are rendered. It simply causes errors in the cairo_quartz_surface_t backend. Since we are restricted to using cairo_image_surface_t (which happens to be faster anyway) we can use the IOSurfaceBaseAddress() to obtain a mapping of the IOSurfaceRef in user-space. It always uses BGRA 32-bit with alpha channel even if we will discard the alpha channel as that is necessary to hit the fast paths in other parts of the platform. Note that while Cairo says CAIRO_FORMAT_ARGB32, it is really 32-bit BGRA on little-endian as we expect. OpenGL will render flipped (Quartz Native Co-ordinates) while Cairo renders with 0,O in the top-left. We could use cairo_translate() and cairo_scale() to reverse this, but it looks like some cairo things may not look quite as right if we do so. To reduce the chances of one-off bugs this continues to draw as Cairo would normally, but instead uses an CGAffineTransform in the tiles and some CGRect translation when swapping buffers to get the same effect. Changes to OpenGL ================= To simplify things, removal of all NSOpenGL* related components have been removed and we strictly use the Core GL (CGL*) API. This probably should have been done long ago anyay. Most examples found in the browsers to use IOSurfaceRef with OpenGL are using Legacy GL and there is still work underway to make this fit in with the rest of how the GSK GL renderer works. Since IOSurfaceRef bound to a texture/framebuffer will not have a default framebuffer ID of 0, we needed to add a default framebuffer id to the GdkGLContext. GskGLRenderer can use this to setup the command queue in such a way that our IOSurface destination has been glBindFramebuffer() as if it were the default drawable. This stuff is pretty slight-of-hand, so where things are and what needs flushing when and where has been a bit of an experiment to see what actually works to get synchronization across subsystems. Efficient Damages ================= After we draw with Cairo, we unlock the IOSurfaceRef and the contents are uploaded to the GPU. To make the contents visible to the app, we must clear the tiles contents with `layer.contents=nil;` and then re-apply the IOSurfaceRef. Since the buffer has likely not changed, we only do this if the tile overlaps the damage region. This gives the effect of having more tightly controlled damage regions even though updating the layer would damage be the whole window (as it is with OpenGL/Metal today with the exception of scissor-rect). This too can be verified usign "Quartz Debug -> Flash screen udpates". Frame Synchronized Resize ========================= In GTK 4, we have the ability to perform sizing changes from compute-size during the layout phase. Since the macOS backend already tracks window resizes manually, we can avoid doing the setFrame: immediately and instead do it within the frame clock's layout phase. Doing so gives us vastly better resize experience as we're more likely to get the size-change and updated-contents in the same frame on screen. It makes things feel "connected" in a way they weren't before. Some additional effort to tweak gravity during the process is also necessary but we were already doing that in the GTK 4 backend. Backporting =========== The design here has made an attempt to make it possible to backport by keeping GdkMacosBuffer, GdkMacosLayer, and GdkMacosTile fairly independent. There may be an opportunity to integrate this into GTK 3's quartz backend with a fair bit of work. Doing so could improve the situation for applications which are damage-rich such as The GIMP.
2022-02-14 10:20:19 +00:00
return texture;
macos: implement GL context This implements the basics for a GdkGLContext on macOS. Currently, rendering only is fully working for the GskCairoRenderer case where we read back pixels into a cairo surface for rendering. More work on synchronization is required for the GL on GskGLRenderer case. When we attempt to render a surface itself with GL, the context will ensure that the new GdkMacosGLView is placed within the NSWindow. In other cases, we use a dummy NSView and NSWindow for backing the NSOpenGLContext to ensure that we can get accelerated drawing. This gets GtkGLArea working when running with GSK_RENDERER=cairo.
2020-10-29 17:33:22 +00:00
}
macos: use CALayer and IOSurface for rendering This provides a major shift in how we draw both when accelerated OpenGL as well as software rendering with Cairo. In short, it uses tiles of Core Animation's CALayer to display contents from an OpenGL or Cairo rendering so that the window can provide partial damage updates. Partial damage is not generally available when using OpenGL as the whole buffer is flipped even if you only submitted a small change using a scissor rect. Thankfully, this speeds up Cairo rendering a bit too by using IOSurface to upload contents to the display server. We use the tiling system we do for OpenGL which reduces overall complexity and differences between them. A New Buffer ============ GdkMacosBuffer is a wrapper around an IOSurfaceRef. The term buffer was used because 1) surface is already used and 2) it loosely maps to a front/back buffer semantic. However, it appears that IOSurfaceRef contents are being retained in some fashion (likely in the compositor result) so we can update the same IOSurfaceRef without flipping as long as we're fast. This appears to be what Chromium does as well, but Firefox uses two IOSurfaceRef and flips between them. We would like to avoid two surfaces because it doubles the GPU VRAM requirements of the application. Changes to Windows ================== Previously, the NSWindow would dynamically change between different types of NSView based on the renderer being used. This is no longer necessary as we just have a single NSView type, GdkMacosView, which inherits from GdkMacosBaseView just to keep the tedius stuff separate from the machinery of GdkMacosView. We can merge those someday if we are okay with that. Changes to Views ================ GdkMacosCairoView, GdkMacosCairoSubView, GdkMacosGLView have all been removed and replaced with GdkMacosView. This new view has a single CALayer (GdkMacosLayer) attached to it which itself has sublayers. The contents of the CALayer is populated with an IOSurfaceRef which we allocated with the GdkMacosSurface. The surface is replaced when the NSWindow resizes. Changes to Layers ================= We now have a dedicated GdkMacosLayer which contains sublayers of GdkMacosTile. The tile has a maximum size of 128x128 pixels in device units. The GdkMacosTile is partitioned by splitting both the transparent region (window bounds minus opaque area) and then by splitting the opaque area. A tile has either translucent contents (and therefore is not opaque) or has opaque contents (and therefore is opaque). An opaque tile never contains transparent contents. As such, the opaque tiles contain a black background so that Core Animation will consider the tile's bounds as opaque. This can be verified with "Quartz Debug -> Show opaque regions". Changes to Cairo ================ GTK 4 cannot currently use cairo-quartz because of how CSS borders are rendered. It simply causes errors in the cairo_quartz_surface_t backend. Since we are restricted to using cairo_image_surface_t (which happens to be faster anyway) we can use the IOSurfaceBaseAddress() to obtain a mapping of the IOSurfaceRef in user-space. It always uses BGRA 32-bit with alpha channel even if we will discard the alpha channel as that is necessary to hit the fast paths in other parts of the platform. Note that while Cairo says CAIRO_FORMAT_ARGB32, it is really 32-bit BGRA on little-endian as we expect. OpenGL will render flipped (Quartz Native Co-ordinates) while Cairo renders with 0,O in the top-left. We could use cairo_translate() and cairo_scale() to reverse this, but it looks like some cairo things may not look quite as right if we do so. To reduce the chances of one-off bugs this continues to draw as Cairo would normally, but instead uses an CGAffineTransform in the tiles and some CGRect translation when swapping buffers to get the same effect. Changes to OpenGL ================= To simplify things, removal of all NSOpenGL* related components have been removed and we strictly use the Core GL (CGL*) API. This probably should have been done long ago anyay. Most examples found in the browsers to use IOSurfaceRef with OpenGL are using Legacy GL and there is still work underway to make this fit in with the rest of how the GSK GL renderer works. Since IOSurfaceRef bound to a texture/framebuffer will not have a default framebuffer ID of 0, we needed to add a default framebuffer id to the GdkGLContext. GskGLRenderer can use this to setup the command queue in such a way that our IOSurface destination has been glBindFramebuffer() as if it were the default drawable. This stuff is pretty slight-of-hand, so where things are and what needs flushing when and where has been a bit of an experiment to see what actually works to get synchronization across subsystems. Efficient Damages ================= After we draw with Cairo, we unlock the IOSurfaceRef and the contents are uploaded to the GPU. To make the contents visible to the app, we must clear the tiles contents with `layer.contents=nil;` and then re-apply the IOSurfaceRef. Since the buffer has likely not changed, we only do this if the tile overlaps the damage region. This gives the effect of having more tightly controlled damage regions even though updating the layer would damage be the whole window (as it is with OpenGL/Metal today with the exception of scissor-rect). This too can be verified usign "Quartz Debug -> Flash screen udpates". Frame Synchronized Resize ========================= In GTK 4, we have the ability to perform sizing changes from compute-size during the layout phase. Since the macOS backend already tracks window resizes manually, we can avoid doing the setFrame: immediately and instead do it within the frame clock's layout phase. Doing so gives us vastly better resize experience as we're more likely to get the size-change and updated-contents in the same frame on screen. It makes things feel "connected" in a way they weren't before. Some additional effort to tweak gravity during the process is also necessary but we were already doing that in the GTK 4 backend. Backporting =========== The design here has made an attempt to make it possible to backport by keeping GdkMacosBuffer, GdkMacosLayer, and GdkMacosTile fairly independent. There may be an opportunity to integrate this into GTK 3's quartz backend with a fair bit of work. Doing so could improve the situation for applications which are damage-rich such as The GIMP.
2022-02-14 10:20:19 +00:00
static void
gdk_macos_gl_context_allocate (GdkMacosGLContext *self)
macos: implement GL context This implements the basics for a GdkGLContext on macOS. Currently, rendering only is fully working for the GskCairoRenderer case where we read back pixels into a cairo surface for rendering. More work on synchronization is required for the GL on GskGLRenderer case. When we attempt to render a surface itself with GL, the context will ensure that the new GdkMacosGLView is placed within the NSWindow. In other cases, we use a dummy NSView and NSWindow for backing the NSOpenGLContext to ensure that we can get accelerated drawing. This gets GtkGLArea working when running with GSK_RENDERER=cairo.
2020-10-29 17:33:22 +00:00
{
macos: use CALayer and IOSurface for rendering This provides a major shift in how we draw both when accelerated OpenGL as well as software rendering with Cairo. In short, it uses tiles of Core Animation's CALayer to display contents from an OpenGL or Cairo rendering so that the window can provide partial damage updates. Partial damage is not generally available when using OpenGL as the whole buffer is flipped even if you only submitted a small change using a scissor rect. Thankfully, this speeds up Cairo rendering a bit too by using IOSurface to upload contents to the display server. We use the tiling system we do for OpenGL which reduces overall complexity and differences between them. A New Buffer ============ GdkMacosBuffer is a wrapper around an IOSurfaceRef. The term buffer was used because 1) surface is already used and 2) it loosely maps to a front/back buffer semantic. However, it appears that IOSurfaceRef contents are being retained in some fashion (likely in the compositor result) so we can update the same IOSurfaceRef without flipping as long as we're fast. This appears to be what Chromium does as well, but Firefox uses two IOSurfaceRef and flips between them. We would like to avoid two surfaces because it doubles the GPU VRAM requirements of the application. Changes to Windows ================== Previously, the NSWindow would dynamically change between different types of NSView based on the renderer being used. This is no longer necessary as we just have a single NSView type, GdkMacosView, which inherits from GdkMacosBaseView just to keep the tedius stuff separate from the machinery of GdkMacosView. We can merge those someday if we are okay with that. Changes to Views ================ GdkMacosCairoView, GdkMacosCairoSubView, GdkMacosGLView have all been removed and replaced with GdkMacosView. This new view has a single CALayer (GdkMacosLayer) attached to it which itself has sublayers. The contents of the CALayer is populated with an IOSurfaceRef which we allocated with the GdkMacosSurface. The surface is replaced when the NSWindow resizes. Changes to Layers ================= We now have a dedicated GdkMacosLayer which contains sublayers of GdkMacosTile. The tile has a maximum size of 128x128 pixels in device units. The GdkMacosTile is partitioned by splitting both the transparent region (window bounds minus opaque area) and then by splitting the opaque area. A tile has either translucent contents (and therefore is not opaque) or has opaque contents (and therefore is opaque). An opaque tile never contains transparent contents. As such, the opaque tiles contain a black background so that Core Animation will consider the tile's bounds as opaque. This can be verified with "Quartz Debug -> Show opaque regions". Changes to Cairo ================ GTK 4 cannot currently use cairo-quartz because of how CSS borders are rendered. It simply causes errors in the cairo_quartz_surface_t backend. Since we are restricted to using cairo_image_surface_t (which happens to be faster anyway) we can use the IOSurfaceBaseAddress() to obtain a mapping of the IOSurfaceRef in user-space. It always uses BGRA 32-bit with alpha channel even if we will discard the alpha channel as that is necessary to hit the fast paths in other parts of the platform. Note that while Cairo says CAIRO_FORMAT_ARGB32, it is really 32-bit BGRA on little-endian as we expect. OpenGL will render flipped (Quartz Native Co-ordinates) while Cairo renders with 0,O in the top-left. We could use cairo_translate() and cairo_scale() to reverse this, but it looks like some cairo things may not look quite as right if we do so. To reduce the chances of one-off bugs this continues to draw as Cairo would normally, but instead uses an CGAffineTransform in the tiles and some CGRect translation when swapping buffers to get the same effect. Changes to OpenGL ================= To simplify things, removal of all NSOpenGL* related components have been removed and we strictly use the Core GL (CGL*) API. This probably should have been done long ago anyay. Most examples found in the browsers to use IOSurfaceRef with OpenGL are using Legacy GL and there is still work underway to make this fit in with the rest of how the GSK GL renderer works. Since IOSurfaceRef bound to a texture/framebuffer will not have a default framebuffer ID of 0, we needed to add a default framebuffer id to the GdkGLContext. GskGLRenderer can use this to setup the command queue in such a way that our IOSurface destination has been glBindFramebuffer() as if it were the default drawable. This stuff is pretty slight-of-hand, so where things are and what needs flushing when and where has been a bit of an experiment to see what actually works to get synchronization across subsystems. Efficient Damages ================= After we draw with Cairo, we unlock the IOSurfaceRef and the contents are uploaded to the GPU. To make the contents visible to the app, we must clear the tiles contents with `layer.contents=nil;` and then re-apply the IOSurfaceRef. Since the buffer has likely not changed, we only do this if the tile overlaps the damage region. This gives the effect of having more tightly controlled damage regions even though updating the layer would damage be the whole window (as it is with OpenGL/Metal today with the exception of scissor-rect). This too can be verified usign "Quartz Debug -> Flash screen udpates". Frame Synchronized Resize ========================= In GTK 4, we have the ability to perform sizing changes from compute-size during the layout phase. Since the macOS backend already tracks window resizes manually, we can avoid doing the setFrame: immediately and instead do it within the frame clock's layout phase. Doing so gives us vastly better resize experience as we're more likely to get the size-change and updated-contents in the same frame on screen. It makes things feel "connected" in a way they weren't before. Some additional effort to tweak gravity during the process is also necessary but we were already doing that in the GTK 4 backend. Backporting =========== The design here has made an attempt to make it possible to backport by keeping GdkMacosBuffer, GdkMacosLayer, and GdkMacosTile fairly independent. There may be an opportunity to integrate this into GTK 3's quartz backend with a fair bit of work. Doing so could improve the situation for applications which are damage-rich such as The GIMP.
2022-02-14 10:20:19 +00:00
GdkSurface *surface;
GLint opaque;
macos: implement GL context This implements the basics for a GdkGLContext on macOS. Currently, rendering only is fully working for the GskCairoRenderer case where we read back pixels into a cairo surface for rendering. More work on synchronization is required for the GL on GskGLRenderer case. When we attempt to render a surface itself with GL, the context will ensure that the new GdkMacosGLView is placed within the NSWindow. In other cases, we use a dummy NSView and NSWindow for backing the NSOpenGLContext to ensure that we can get accelerated drawing. This gets GtkGLArea working when running with GSK_RENDERER=cairo.
2020-10-29 17:33:22 +00:00
macos: use CALayer and IOSurface for rendering This provides a major shift in how we draw both when accelerated OpenGL as well as software rendering with Cairo. In short, it uses tiles of Core Animation's CALayer to display contents from an OpenGL or Cairo rendering so that the window can provide partial damage updates. Partial damage is not generally available when using OpenGL as the whole buffer is flipped even if you only submitted a small change using a scissor rect. Thankfully, this speeds up Cairo rendering a bit too by using IOSurface to upload contents to the display server. We use the tiling system we do for OpenGL which reduces overall complexity and differences between them. A New Buffer ============ GdkMacosBuffer is a wrapper around an IOSurfaceRef. The term buffer was used because 1) surface is already used and 2) it loosely maps to a front/back buffer semantic. However, it appears that IOSurfaceRef contents are being retained in some fashion (likely in the compositor result) so we can update the same IOSurfaceRef without flipping as long as we're fast. This appears to be what Chromium does as well, but Firefox uses two IOSurfaceRef and flips between them. We would like to avoid two surfaces because it doubles the GPU VRAM requirements of the application. Changes to Windows ================== Previously, the NSWindow would dynamically change between different types of NSView based on the renderer being used. This is no longer necessary as we just have a single NSView type, GdkMacosView, which inherits from GdkMacosBaseView just to keep the tedius stuff separate from the machinery of GdkMacosView. We can merge those someday if we are okay with that. Changes to Views ================ GdkMacosCairoView, GdkMacosCairoSubView, GdkMacosGLView have all been removed and replaced with GdkMacosView. This new view has a single CALayer (GdkMacosLayer) attached to it which itself has sublayers. The contents of the CALayer is populated with an IOSurfaceRef which we allocated with the GdkMacosSurface. The surface is replaced when the NSWindow resizes. Changes to Layers ================= We now have a dedicated GdkMacosLayer which contains sublayers of GdkMacosTile. The tile has a maximum size of 128x128 pixels in device units. The GdkMacosTile is partitioned by splitting both the transparent region (window bounds minus opaque area) and then by splitting the opaque area. A tile has either translucent contents (and therefore is not opaque) or has opaque contents (and therefore is opaque). An opaque tile never contains transparent contents. As such, the opaque tiles contain a black background so that Core Animation will consider the tile's bounds as opaque. This can be verified with "Quartz Debug -> Show opaque regions". Changes to Cairo ================ GTK 4 cannot currently use cairo-quartz because of how CSS borders are rendered. It simply causes errors in the cairo_quartz_surface_t backend. Since we are restricted to using cairo_image_surface_t (which happens to be faster anyway) we can use the IOSurfaceBaseAddress() to obtain a mapping of the IOSurfaceRef in user-space. It always uses BGRA 32-bit with alpha channel even if we will discard the alpha channel as that is necessary to hit the fast paths in other parts of the platform. Note that while Cairo says CAIRO_FORMAT_ARGB32, it is really 32-bit BGRA on little-endian as we expect. OpenGL will render flipped (Quartz Native Co-ordinates) while Cairo renders with 0,O in the top-left. We could use cairo_translate() and cairo_scale() to reverse this, but it looks like some cairo things may not look quite as right if we do so. To reduce the chances of one-off bugs this continues to draw as Cairo would normally, but instead uses an CGAffineTransform in the tiles and some CGRect translation when swapping buffers to get the same effect. Changes to OpenGL ================= To simplify things, removal of all NSOpenGL* related components have been removed and we strictly use the Core GL (CGL*) API. This probably should have been done long ago anyay. Most examples found in the browsers to use IOSurfaceRef with OpenGL are using Legacy GL and there is still work underway to make this fit in with the rest of how the GSK GL renderer works. Since IOSurfaceRef bound to a texture/framebuffer will not have a default framebuffer ID of 0, we needed to add a default framebuffer id to the GdkGLContext. GskGLRenderer can use this to setup the command queue in such a way that our IOSurface destination has been glBindFramebuffer() as if it were the default drawable. This stuff is pretty slight-of-hand, so where things are and what needs flushing when and where has been a bit of an experiment to see what actually works to get synchronization across subsystems. Efficient Damages ================= After we draw with Cairo, we unlock the IOSurfaceRef and the contents are uploaded to the GPU. To make the contents visible to the app, we must clear the tiles contents with `layer.contents=nil;` and then re-apply the IOSurfaceRef. Since the buffer has likely not changed, we only do this if the tile overlaps the damage region. This gives the effect of having more tightly controlled damage regions even though updating the layer would damage be the whole window (as it is with OpenGL/Metal today with the exception of scissor-rect). This too can be verified usign "Quartz Debug -> Flash screen udpates". Frame Synchronized Resize ========================= In GTK 4, we have the ability to perform sizing changes from compute-size during the layout phase. Since the macOS backend already tracks window resizes manually, we can avoid doing the setFrame: immediately and instead do it within the frame clock's layout phase. Doing so gives us vastly better resize experience as we're more likely to get the size-change and updated-contents in the same frame on screen. It makes things feel "connected" in a way they weren't before. Some additional effort to tweak gravity during the process is also necessary but we were already doing that in the GTK 4 backend. Backporting =========== The design here has made an attempt to make it possible to backport by keeping GdkMacosBuffer, GdkMacosLayer, and GdkMacosTile fairly independent. There may be an opportunity to integrate this into GTK 3's quartz backend with a fair bit of work. Doing so could improve the situation for applications which are damage-rich such as The GIMP.
2022-02-14 10:20:19 +00:00
g_assert (GDK_IS_MACOS_GL_CONTEXT (self));
g_assert (self->cgl_context != NULL);
g_assert (self->target != 0);
g_assert (self->texture != 0 || self->fbo == 0);
g_assert (self->fbo != 0 || self->texture == 0);
macos: implement GL context This implements the basics for a GdkGLContext on macOS. Currently, rendering only is fully working for the GskCairoRenderer case where we read back pixels into a cairo surface for rendering. More work on synchronization is required for the GL on GskGLRenderer case. When we attempt to render a surface itself with GL, the context will ensure that the new GdkMacosGLView is placed within the NSWindow. In other cases, we use a dummy NSView and NSWindow for backing the NSOpenGLContext to ensure that we can get accelerated drawing. This gets GtkGLArea working when running with GSK_RENDERER=cairo.
2020-10-29 17:33:22 +00:00
macos: use CALayer and IOSurface for rendering This provides a major shift in how we draw both when accelerated OpenGL as well as software rendering with Cairo. In short, it uses tiles of Core Animation's CALayer to display contents from an OpenGL or Cairo rendering so that the window can provide partial damage updates. Partial damage is not generally available when using OpenGL as the whole buffer is flipped even if you only submitted a small change using a scissor rect. Thankfully, this speeds up Cairo rendering a bit too by using IOSurface to upload contents to the display server. We use the tiling system we do for OpenGL which reduces overall complexity and differences between them. A New Buffer ============ GdkMacosBuffer is a wrapper around an IOSurfaceRef. The term buffer was used because 1) surface is already used and 2) it loosely maps to a front/back buffer semantic. However, it appears that IOSurfaceRef contents are being retained in some fashion (likely in the compositor result) so we can update the same IOSurfaceRef without flipping as long as we're fast. This appears to be what Chromium does as well, but Firefox uses two IOSurfaceRef and flips between them. We would like to avoid two surfaces because it doubles the GPU VRAM requirements of the application. Changes to Windows ================== Previously, the NSWindow would dynamically change between different types of NSView based on the renderer being used. This is no longer necessary as we just have a single NSView type, GdkMacosView, which inherits from GdkMacosBaseView just to keep the tedius stuff separate from the machinery of GdkMacosView. We can merge those someday if we are okay with that. Changes to Views ================ GdkMacosCairoView, GdkMacosCairoSubView, GdkMacosGLView have all been removed and replaced with GdkMacosView. This new view has a single CALayer (GdkMacosLayer) attached to it which itself has sublayers. The contents of the CALayer is populated with an IOSurfaceRef which we allocated with the GdkMacosSurface. The surface is replaced when the NSWindow resizes. Changes to Layers ================= We now have a dedicated GdkMacosLayer which contains sublayers of GdkMacosTile. The tile has a maximum size of 128x128 pixels in device units. The GdkMacosTile is partitioned by splitting both the transparent region (window bounds minus opaque area) and then by splitting the opaque area. A tile has either translucent contents (and therefore is not opaque) or has opaque contents (and therefore is opaque). An opaque tile never contains transparent contents. As such, the opaque tiles contain a black background so that Core Animation will consider the tile's bounds as opaque. This can be verified with "Quartz Debug -> Show opaque regions". Changes to Cairo ================ GTK 4 cannot currently use cairo-quartz because of how CSS borders are rendered. It simply causes errors in the cairo_quartz_surface_t backend. Since we are restricted to using cairo_image_surface_t (which happens to be faster anyway) we can use the IOSurfaceBaseAddress() to obtain a mapping of the IOSurfaceRef in user-space. It always uses BGRA 32-bit with alpha channel even if we will discard the alpha channel as that is necessary to hit the fast paths in other parts of the platform. Note that while Cairo says CAIRO_FORMAT_ARGB32, it is really 32-bit BGRA on little-endian as we expect. OpenGL will render flipped (Quartz Native Co-ordinates) while Cairo renders with 0,O in the top-left. We could use cairo_translate() and cairo_scale() to reverse this, but it looks like some cairo things may not look quite as right if we do so. To reduce the chances of one-off bugs this continues to draw as Cairo would normally, but instead uses an CGAffineTransform in the tiles and some CGRect translation when swapping buffers to get the same effect. Changes to OpenGL ================= To simplify things, removal of all NSOpenGL* related components have been removed and we strictly use the Core GL (CGL*) API. This probably should have been done long ago anyay. Most examples found in the browsers to use IOSurfaceRef with OpenGL are using Legacy GL and there is still work underway to make this fit in with the rest of how the GSK GL renderer works. Since IOSurfaceRef bound to a texture/framebuffer will not have a default framebuffer ID of 0, we needed to add a default framebuffer id to the GdkGLContext. GskGLRenderer can use this to setup the command queue in such a way that our IOSurface destination has been glBindFramebuffer() as if it were the default drawable. This stuff is pretty slight-of-hand, so where things are and what needs flushing when and where has been a bit of an experiment to see what actually works to get synchronization across subsystems. Efficient Damages ================= After we draw with Cairo, we unlock the IOSurfaceRef and the contents are uploaded to the GPU. To make the contents visible to the app, we must clear the tiles contents with `layer.contents=nil;` and then re-apply the IOSurfaceRef. Since the buffer has likely not changed, we only do this if the tile overlaps the damage region. This gives the effect of having more tightly controlled damage regions even though updating the layer would damage be the whole window (as it is with OpenGL/Metal today with the exception of scissor-rect). This too can be verified usign "Quartz Debug -> Flash screen udpates". Frame Synchronized Resize ========================= In GTK 4, we have the ability to perform sizing changes from compute-size during the layout phase. Since the macOS backend already tracks window resizes manually, we can avoid doing the setFrame: immediately and instead do it within the frame clock's layout phase. Doing so gives us vastly better resize experience as we're more likely to get the size-change and updated-contents in the same frame on screen. It makes things feel "connected" in a way they weren't before. Some additional effort to tweak gravity during the process is also necessary but we were already doing that in the GTK 4 backend. Backporting =========== The design here has made an attempt to make it possible to backport by keeping GdkMacosBuffer, GdkMacosLayer, and GdkMacosTile fairly independent. There may be an opportunity to integrate this into GTK 3's quartz backend with a fair bit of work. Doing so could improve the situation for applications which are damage-rich such as The GIMP.
2022-02-14 10:20:19 +00:00
if (!(surface = gdk_draw_context_get_surface (GDK_DRAW_CONTEXT (self))))
return;
macos: implement GL context This implements the basics for a GdkGLContext on macOS. Currently, rendering only is fully working for the GskCairoRenderer case where we read back pixels into a cairo surface for rendering. More work on synchronization is required for the GL on GskGLRenderer case. When we attempt to render a surface itself with GL, the context will ensure that the new GdkMacosGLView is placed within the NSWindow. In other cases, we use a dummy NSView and NSWindow for backing the NSOpenGLContext to ensure that we can get accelerated drawing. This gets GtkGLArea working when running with GSK_RENDERER=cairo.
2020-10-29 17:33:22 +00:00
macos: use CALayer and IOSurface for rendering This provides a major shift in how we draw both when accelerated OpenGL as well as software rendering with Cairo. In short, it uses tiles of Core Animation's CALayer to display contents from an OpenGL or Cairo rendering so that the window can provide partial damage updates. Partial damage is not generally available when using OpenGL as the whole buffer is flipped even if you only submitted a small change using a scissor rect. Thankfully, this speeds up Cairo rendering a bit too by using IOSurface to upload contents to the display server. We use the tiling system we do for OpenGL which reduces overall complexity and differences between them. A New Buffer ============ GdkMacosBuffer is a wrapper around an IOSurfaceRef. The term buffer was used because 1) surface is already used and 2) it loosely maps to a front/back buffer semantic. However, it appears that IOSurfaceRef contents are being retained in some fashion (likely in the compositor result) so we can update the same IOSurfaceRef without flipping as long as we're fast. This appears to be what Chromium does as well, but Firefox uses two IOSurfaceRef and flips between them. We would like to avoid two surfaces because it doubles the GPU VRAM requirements of the application. Changes to Windows ================== Previously, the NSWindow would dynamically change between different types of NSView based on the renderer being used. This is no longer necessary as we just have a single NSView type, GdkMacosView, which inherits from GdkMacosBaseView just to keep the tedius stuff separate from the machinery of GdkMacosView. We can merge those someday if we are okay with that. Changes to Views ================ GdkMacosCairoView, GdkMacosCairoSubView, GdkMacosGLView have all been removed and replaced with GdkMacosView. This new view has a single CALayer (GdkMacosLayer) attached to it which itself has sublayers. The contents of the CALayer is populated with an IOSurfaceRef which we allocated with the GdkMacosSurface. The surface is replaced when the NSWindow resizes. Changes to Layers ================= We now have a dedicated GdkMacosLayer which contains sublayers of GdkMacosTile. The tile has a maximum size of 128x128 pixels in device units. The GdkMacosTile is partitioned by splitting both the transparent region (window bounds minus opaque area) and then by splitting the opaque area. A tile has either translucent contents (and therefore is not opaque) or has opaque contents (and therefore is opaque). An opaque tile never contains transparent contents. As such, the opaque tiles contain a black background so that Core Animation will consider the tile's bounds as opaque. This can be verified with "Quartz Debug -> Show opaque regions". Changes to Cairo ================ GTK 4 cannot currently use cairo-quartz because of how CSS borders are rendered. It simply causes errors in the cairo_quartz_surface_t backend. Since we are restricted to using cairo_image_surface_t (which happens to be faster anyway) we can use the IOSurfaceBaseAddress() to obtain a mapping of the IOSurfaceRef in user-space. It always uses BGRA 32-bit with alpha channel even if we will discard the alpha channel as that is necessary to hit the fast paths in other parts of the platform. Note that while Cairo says CAIRO_FORMAT_ARGB32, it is really 32-bit BGRA on little-endian as we expect. OpenGL will render flipped (Quartz Native Co-ordinates) while Cairo renders with 0,O in the top-left. We could use cairo_translate() and cairo_scale() to reverse this, but it looks like some cairo things may not look quite as right if we do so. To reduce the chances of one-off bugs this continues to draw as Cairo would normally, but instead uses an CGAffineTransform in the tiles and some CGRect translation when swapping buffers to get the same effect. Changes to OpenGL ================= To simplify things, removal of all NSOpenGL* related components have been removed and we strictly use the Core GL (CGL*) API. This probably should have been done long ago anyay. Most examples found in the browsers to use IOSurfaceRef with OpenGL are using Legacy GL and there is still work underway to make this fit in with the rest of how the GSK GL renderer works. Since IOSurfaceRef bound to a texture/framebuffer will not have a default framebuffer ID of 0, we needed to add a default framebuffer id to the GdkGLContext. GskGLRenderer can use this to setup the command queue in such a way that our IOSurface destination has been glBindFramebuffer() as if it were the default drawable. This stuff is pretty slight-of-hand, so where things are and what needs flushing when and where has been a bit of an experiment to see what actually works to get synchronization across subsystems. Efficient Damages ================= After we draw with Cairo, we unlock the IOSurfaceRef and the contents are uploaded to the GPU. To make the contents visible to the app, we must clear the tiles contents with `layer.contents=nil;` and then re-apply the IOSurfaceRef. Since the buffer has likely not changed, we only do this if the tile overlaps the damage region. This gives the effect of having more tightly controlled damage regions even though updating the layer would damage be the whole window (as it is with OpenGL/Metal today with the exception of scissor-rect). This too can be verified usign "Quartz Debug -> Flash screen udpates". Frame Synchronized Resize ========================= In GTK 4, we have the ability to perform sizing changes from compute-size during the layout phase. Since the macOS backend already tracks window resizes manually, we can avoid doing the setFrame: immediately and instead do it within the frame clock's layout phase. Doing so gives us vastly better resize experience as we're more likely to get the size-change and updated-contents in the same frame on screen. It makes things feel "connected" in a way they weren't before. Some additional effort to tweak gravity during the process is also necessary but we were already doing that in the GTK 4 backend. Backporting =========== The design here has made an attempt to make it possible to backport by keeping GdkMacosBuffer, GdkMacosLayer, and GdkMacosTile fairly independent. There may be an opportunity to integrate this into GTK 3's quartz backend with a fair bit of work. Doing so could improve the situation for applications which are damage-rich such as The GIMP.
2022-02-14 10:20:19 +00:00
/* Alter to an opaque surface if necessary */
opaque = _gdk_macos_surface_is_opaque (GDK_MACOS_SURFACE (surface));
if (opaque != self->last_opaque)
{
self->last_opaque = !!opaque;
if (!CHECK (NULL, CGLSetParameter (self->cgl_context, kCGLCPSurfaceOpacity, &opaque)))
return;
}
macos: implement GL context This implements the basics for a GdkGLContext on macOS. Currently, rendering only is fully working for the GskCairoRenderer case where we read back pixels into a cairo surface for rendering. More work on synchronization is required for the GL on GskGLRenderer case. When we attempt to render a surface itself with GL, the context will ensure that the new GdkMacosGLView is placed within the NSWindow. In other cases, we use a dummy NSView and NSWindow for backing the NSOpenGLContext to ensure that we can get accelerated drawing. This gets GtkGLArea working when running with GSK_RENDERER=cairo.
2020-10-29 17:33:22 +00:00
macos: use CALayer and IOSurface for rendering This provides a major shift in how we draw both when accelerated OpenGL as well as software rendering with Cairo. In short, it uses tiles of Core Animation's CALayer to display contents from an OpenGL or Cairo rendering so that the window can provide partial damage updates. Partial damage is not generally available when using OpenGL as the whole buffer is flipped even if you only submitted a small change using a scissor rect. Thankfully, this speeds up Cairo rendering a bit too by using IOSurface to upload contents to the display server. We use the tiling system we do for OpenGL which reduces overall complexity and differences between them. A New Buffer ============ GdkMacosBuffer is a wrapper around an IOSurfaceRef. The term buffer was used because 1) surface is already used and 2) it loosely maps to a front/back buffer semantic. However, it appears that IOSurfaceRef contents are being retained in some fashion (likely in the compositor result) so we can update the same IOSurfaceRef without flipping as long as we're fast. This appears to be what Chromium does as well, but Firefox uses two IOSurfaceRef and flips between them. We would like to avoid two surfaces because it doubles the GPU VRAM requirements of the application. Changes to Windows ================== Previously, the NSWindow would dynamically change between different types of NSView based on the renderer being used. This is no longer necessary as we just have a single NSView type, GdkMacosView, which inherits from GdkMacosBaseView just to keep the tedius stuff separate from the machinery of GdkMacosView. We can merge those someday if we are okay with that. Changes to Views ================ GdkMacosCairoView, GdkMacosCairoSubView, GdkMacosGLView have all been removed and replaced with GdkMacosView. This new view has a single CALayer (GdkMacosLayer) attached to it which itself has sublayers. The contents of the CALayer is populated with an IOSurfaceRef which we allocated with the GdkMacosSurface. The surface is replaced when the NSWindow resizes. Changes to Layers ================= We now have a dedicated GdkMacosLayer which contains sublayers of GdkMacosTile. The tile has a maximum size of 128x128 pixels in device units. The GdkMacosTile is partitioned by splitting both the transparent region (window bounds minus opaque area) and then by splitting the opaque area. A tile has either translucent contents (and therefore is not opaque) or has opaque contents (and therefore is opaque). An opaque tile never contains transparent contents. As such, the opaque tiles contain a black background so that Core Animation will consider the tile's bounds as opaque. This can be verified with "Quartz Debug -> Show opaque regions". Changes to Cairo ================ GTK 4 cannot currently use cairo-quartz because of how CSS borders are rendered. It simply causes errors in the cairo_quartz_surface_t backend. Since we are restricted to using cairo_image_surface_t (which happens to be faster anyway) we can use the IOSurfaceBaseAddress() to obtain a mapping of the IOSurfaceRef in user-space. It always uses BGRA 32-bit with alpha channel even if we will discard the alpha channel as that is necessary to hit the fast paths in other parts of the platform. Note that while Cairo says CAIRO_FORMAT_ARGB32, it is really 32-bit BGRA on little-endian as we expect. OpenGL will render flipped (Quartz Native Co-ordinates) while Cairo renders with 0,O in the top-left. We could use cairo_translate() and cairo_scale() to reverse this, but it looks like some cairo things may not look quite as right if we do so. To reduce the chances of one-off bugs this continues to draw as Cairo would normally, but instead uses an CGAffineTransform in the tiles and some CGRect translation when swapping buffers to get the same effect. Changes to OpenGL ================= To simplify things, removal of all NSOpenGL* related components have been removed and we strictly use the Core GL (CGL*) API. This probably should have been done long ago anyay. Most examples found in the browsers to use IOSurfaceRef with OpenGL are using Legacy GL and there is still work underway to make this fit in with the rest of how the GSK GL renderer works. Since IOSurfaceRef bound to a texture/framebuffer will not have a default framebuffer ID of 0, we needed to add a default framebuffer id to the GdkGLContext. GskGLRenderer can use this to setup the command queue in such a way that our IOSurface destination has been glBindFramebuffer() as if it were the default drawable. This stuff is pretty slight-of-hand, so where things are and what needs flushing when and where has been a bit of an experiment to see what actually works to get synchronization across subsystems. Efficient Damages ================= After we draw with Cairo, we unlock the IOSurfaceRef and the contents are uploaded to the GPU. To make the contents visible to the app, we must clear the tiles contents with `layer.contents=nil;` and then re-apply the IOSurfaceRef. Since the buffer has likely not changed, we only do this if the tile overlaps the damage region. This gives the effect of having more tightly controlled damage regions even though updating the layer would damage be the whole window (as it is with OpenGL/Metal today with the exception of scissor-rect). This too can be verified usign "Quartz Debug -> Flash screen udpates". Frame Synchronized Resize ========================= In GTK 4, we have the ability to perform sizing changes from compute-size during the layout phase. Since the macOS backend already tracks window resizes manually, we can avoid doing the setFrame: immediately and instead do it within the frame clock's layout phase. Doing so gives us vastly better resize experience as we're more likely to get the size-change and updated-contents in the same frame on screen. It makes things feel "connected" in a way they weren't before. Some additional effort to tweak gravity during the process is also necessary but we were already doing that in the GTK 4 backend. Backporting =========== The design here has made an attempt to make it possible to backport by keeping GdkMacosBuffer, GdkMacosLayer, and GdkMacosTile fairly independent. There may be an opportunity to integrate this into GTK 3's quartz backend with a fair bit of work. Doing so could improve the situation for applications which are damage-rich such as The GIMP.
2022-02-14 10:20:19 +00:00
if (self->texture == 0)
{
GdkMacosBuffer *buffer;
IOSurfaceRef io_surface;
guint width;
guint height;
GLuint texture = 0;
GLuint fbo = 0;
buffer = _gdk_macos_surface_get_buffer (GDK_MACOS_SURFACE (surface));
io_surface = _gdk_macos_buffer_get_native (buffer);
width = _gdk_macos_buffer_get_width (buffer);
height = _gdk_macos_buffer_get_height (buffer);
/* We might need to re-enforce our CGL context here to keep
* video playing correctly. Something, somewhere, might have
* changed the context without touching GdkGLContext.
*
* Without this, video_player often breaks in gtk-demo when using
* the GStreamer backend.
*/
CGLSetCurrentContext (self->cgl_context);
if (!(texture = create_texture (self->cgl_context, self->target, io_surface, width, height)) ||
!CHECK_GL (NULL, glGenFramebuffers (1, &fbo)) ||
!CHECK_GL (NULL, glBindFramebuffer (GL_FRAMEBUFFER, fbo)) ||
!CHECK_GL (NULL, glBindTexture (self->target, texture)) ||
!CHECK_GL (NULL, glFramebufferTexture2D (GL_FRAMEBUFFER,
GL_COLOR_ATTACHMENT0,
self->target,
texture,
0)) ||
!check_framebuffer_status (GL_FRAMEBUFFER))
{
glDeleteFramebuffers (1, &fbo);
glDeleteTextures (1, &texture);
return;
}
glBindTexture (self->target, 0);
glBindFramebuffer (GL_FRAMEBUFFER, 0);
self->texture = texture;
self->fbo = fbo;
}
macos: implement GL context This implements the basics for a GdkGLContext on macOS. Currently, rendering only is fully working for the GskCairoRenderer case where we read back pixels into a cairo surface for rendering. More work on synchronization is required for the GL on GskGLRenderer case. When we attempt to render a surface itself with GL, the context will ensure that the new GdkMacosGLView is placed within the NSWindow. In other cases, we use a dummy NSView and NSWindow for backing the NSOpenGLContext to ensure that we can get accelerated drawing. This gets GtkGLArea working when running with GSK_RENDERER=cairo.
2020-10-29 17:33:22 +00:00
}
macos: use CALayer and IOSurface for rendering This provides a major shift in how we draw both when accelerated OpenGL as well as software rendering with Cairo. In short, it uses tiles of Core Animation's CALayer to display contents from an OpenGL or Cairo rendering so that the window can provide partial damage updates. Partial damage is not generally available when using OpenGL as the whole buffer is flipped even if you only submitted a small change using a scissor rect. Thankfully, this speeds up Cairo rendering a bit too by using IOSurface to upload contents to the display server. We use the tiling system we do for OpenGL which reduces overall complexity and differences between them. A New Buffer ============ GdkMacosBuffer is a wrapper around an IOSurfaceRef. The term buffer was used because 1) surface is already used and 2) it loosely maps to a front/back buffer semantic. However, it appears that IOSurfaceRef contents are being retained in some fashion (likely in the compositor result) so we can update the same IOSurfaceRef without flipping as long as we're fast. This appears to be what Chromium does as well, but Firefox uses two IOSurfaceRef and flips between them. We would like to avoid two surfaces because it doubles the GPU VRAM requirements of the application. Changes to Windows ================== Previously, the NSWindow would dynamically change between different types of NSView based on the renderer being used. This is no longer necessary as we just have a single NSView type, GdkMacosView, which inherits from GdkMacosBaseView just to keep the tedius stuff separate from the machinery of GdkMacosView. We can merge those someday if we are okay with that. Changes to Views ================ GdkMacosCairoView, GdkMacosCairoSubView, GdkMacosGLView have all been removed and replaced with GdkMacosView. This new view has a single CALayer (GdkMacosLayer) attached to it which itself has sublayers. The contents of the CALayer is populated with an IOSurfaceRef which we allocated with the GdkMacosSurface. The surface is replaced when the NSWindow resizes. Changes to Layers ================= We now have a dedicated GdkMacosLayer which contains sublayers of GdkMacosTile. The tile has a maximum size of 128x128 pixels in device units. The GdkMacosTile is partitioned by splitting both the transparent region (window bounds minus opaque area) and then by splitting the opaque area. A tile has either translucent contents (and therefore is not opaque) or has opaque contents (and therefore is opaque). An opaque tile never contains transparent contents. As such, the opaque tiles contain a black background so that Core Animation will consider the tile's bounds as opaque. This can be verified with "Quartz Debug -> Show opaque regions". Changes to Cairo ================ GTK 4 cannot currently use cairo-quartz because of how CSS borders are rendered. It simply causes errors in the cairo_quartz_surface_t backend. Since we are restricted to using cairo_image_surface_t (which happens to be faster anyway) we can use the IOSurfaceBaseAddress() to obtain a mapping of the IOSurfaceRef in user-space. It always uses BGRA 32-bit with alpha channel even if we will discard the alpha channel as that is necessary to hit the fast paths in other parts of the platform. Note that while Cairo says CAIRO_FORMAT_ARGB32, it is really 32-bit BGRA on little-endian as we expect. OpenGL will render flipped (Quartz Native Co-ordinates) while Cairo renders with 0,O in the top-left. We could use cairo_translate() and cairo_scale() to reverse this, but it looks like some cairo things may not look quite as right if we do so. To reduce the chances of one-off bugs this continues to draw as Cairo would normally, but instead uses an CGAffineTransform in the tiles and some CGRect translation when swapping buffers to get the same effect. Changes to OpenGL ================= To simplify things, removal of all NSOpenGL* related components have been removed and we strictly use the Core GL (CGL*) API. This probably should have been done long ago anyay. Most examples found in the browsers to use IOSurfaceRef with OpenGL are using Legacy GL and there is still work underway to make this fit in with the rest of how the GSK GL renderer works. Since IOSurfaceRef bound to a texture/framebuffer will not have a default framebuffer ID of 0, we needed to add a default framebuffer id to the GdkGLContext. GskGLRenderer can use this to setup the command queue in such a way that our IOSurface destination has been glBindFramebuffer() as if it were the default drawable. This stuff is pretty slight-of-hand, so where things are and what needs flushing when and where has been a bit of an experiment to see what actually works to get synchronization across subsystems. Efficient Damages ================= After we draw with Cairo, we unlock the IOSurfaceRef and the contents are uploaded to the GPU. To make the contents visible to the app, we must clear the tiles contents with `layer.contents=nil;` and then re-apply the IOSurfaceRef. Since the buffer has likely not changed, we only do this if the tile overlaps the damage region. This gives the effect of having more tightly controlled damage regions even though updating the layer would damage be the whole window (as it is with OpenGL/Metal today with the exception of scissor-rect). This too can be verified usign "Quartz Debug -> Flash screen udpates". Frame Synchronized Resize ========================= In GTK 4, we have the ability to perform sizing changes from compute-size during the layout phase. Since the macOS backend already tracks window resizes manually, we can avoid doing the setFrame: immediately and instead do it within the frame clock's layout phase. Doing so gives us vastly better resize experience as we're more likely to get the size-change and updated-contents in the same frame on screen. It makes things feel "connected" in a way they weren't before. Some additional effort to tweak gravity during the process is also necessary but we were already doing that in the GTK 4 backend. Backporting =========== The design here has made an attempt to make it possible to backport by keeping GdkMacosBuffer, GdkMacosLayer, and GdkMacosTile fairly independent. There may be an opportunity to integrate this into GTK 3's quartz backend with a fair bit of work. Doing so could improve the situation for applications which are damage-rich such as The GIMP.
2022-02-14 10:20:19 +00:00
static void
gdk_macos_gl_context_release (GdkMacosGLContext *self)
macos: implement GL context This implements the basics for a GdkGLContext on macOS. Currently, rendering only is fully working for the GskCairoRenderer case where we read back pixels into a cairo surface for rendering. More work on synchronization is required for the GL on GskGLRenderer case. When we attempt to render a surface itself with GL, the context will ensure that the new GdkMacosGLView is placed within the NSWindow. In other cases, we use a dummy NSView and NSWindow for backing the NSOpenGLContext to ensure that we can get accelerated drawing. This gets GtkGLArea working when running with GSK_RENDERER=cairo.
2020-10-29 17:33:22 +00:00
{
g_assert (GDK_IS_MACOS_GL_CONTEXT (self));
macos: use CALayer and IOSurface for rendering This provides a major shift in how we draw both when accelerated OpenGL as well as software rendering with Cairo. In short, it uses tiles of Core Animation's CALayer to display contents from an OpenGL or Cairo rendering so that the window can provide partial damage updates. Partial damage is not generally available when using OpenGL as the whole buffer is flipped even if you only submitted a small change using a scissor rect. Thankfully, this speeds up Cairo rendering a bit too by using IOSurface to upload contents to the display server. We use the tiling system we do for OpenGL which reduces overall complexity and differences between them. A New Buffer ============ GdkMacosBuffer is a wrapper around an IOSurfaceRef. The term buffer was used because 1) surface is already used and 2) it loosely maps to a front/back buffer semantic. However, it appears that IOSurfaceRef contents are being retained in some fashion (likely in the compositor result) so we can update the same IOSurfaceRef without flipping as long as we're fast. This appears to be what Chromium does as well, but Firefox uses two IOSurfaceRef and flips between them. We would like to avoid two surfaces because it doubles the GPU VRAM requirements of the application. Changes to Windows ================== Previously, the NSWindow would dynamically change between different types of NSView based on the renderer being used. This is no longer necessary as we just have a single NSView type, GdkMacosView, which inherits from GdkMacosBaseView just to keep the tedius stuff separate from the machinery of GdkMacosView. We can merge those someday if we are okay with that. Changes to Views ================ GdkMacosCairoView, GdkMacosCairoSubView, GdkMacosGLView have all been removed and replaced with GdkMacosView. This new view has a single CALayer (GdkMacosLayer) attached to it which itself has sublayers. The contents of the CALayer is populated with an IOSurfaceRef which we allocated with the GdkMacosSurface. The surface is replaced when the NSWindow resizes. Changes to Layers ================= We now have a dedicated GdkMacosLayer which contains sublayers of GdkMacosTile. The tile has a maximum size of 128x128 pixels in device units. The GdkMacosTile is partitioned by splitting both the transparent region (window bounds minus opaque area) and then by splitting the opaque area. A tile has either translucent contents (and therefore is not opaque) or has opaque contents (and therefore is opaque). An opaque tile never contains transparent contents. As such, the opaque tiles contain a black background so that Core Animation will consider the tile's bounds as opaque. This can be verified with "Quartz Debug -> Show opaque regions". Changes to Cairo ================ GTK 4 cannot currently use cairo-quartz because of how CSS borders are rendered. It simply causes errors in the cairo_quartz_surface_t backend. Since we are restricted to using cairo_image_surface_t (which happens to be faster anyway) we can use the IOSurfaceBaseAddress() to obtain a mapping of the IOSurfaceRef in user-space. It always uses BGRA 32-bit with alpha channel even if we will discard the alpha channel as that is necessary to hit the fast paths in other parts of the platform. Note that while Cairo says CAIRO_FORMAT_ARGB32, it is really 32-bit BGRA on little-endian as we expect. OpenGL will render flipped (Quartz Native Co-ordinates) while Cairo renders with 0,O in the top-left. We could use cairo_translate() and cairo_scale() to reverse this, but it looks like some cairo things may not look quite as right if we do so. To reduce the chances of one-off bugs this continues to draw as Cairo would normally, but instead uses an CGAffineTransform in the tiles and some CGRect translation when swapping buffers to get the same effect. Changes to OpenGL ================= To simplify things, removal of all NSOpenGL* related components have been removed and we strictly use the Core GL (CGL*) API. This probably should have been done long ago anyay. Most examples found in the browsers to use IOSurfaceRef with OpenGL are using Legacy GL and there is still work underway to make this fit in with the rest of how the GSK GL renderer works. Since IOSurfaceRef bound to a texture/framebuffer will not have a default framebuffer ID of 0, we needed to add a default framebuffer id to the GdkGLContext. GskGLRenderer can use this to setup the command queue in such a way that our IOSurface destination has been glBindFramebuffer() as if it were the default drawable. This stuff is pretty slight-of-hand, so where things are and what needs flushing when and where has been a bit of an experiment to see what actually works to get synchronization across subsystems. Efficient Damages ================= After we draw with Cairo, we unlock the IOSurfaceRef and the contents are uploaded to the GPU. To make the contents visible to the app, we must clear the tiles contents with `layer.contents=nil;` and then re-apply the IOSurfaceRef. Since the buffer has likely not changed, we only do this if the tile overlaps the damage region. This gives the effect of having more tightly controlled damage regions even though updating the layer would damage be the whole window (as it is with OpenGL/Metal today with the exception of scissor-rect). This too can be verified usign "Quartz Debug -> Flash screen udpates". Frame Synchronized Resize ========================= In GTK 4, we have the ability to perform sizing changes from compute-size during the layout phase. Since the macOS backend already tracks window resizes manually, we can avoid doing the setFrame: immediately and instead do it within the frame clock's layout phase. Doing so gives us vastly better resize experience as we're more likely to get the size-change and updated-contents in the same frame on screen. It makes things feel "connected" in a way they weren't before. Some additional effort to tweak gravity during the process is also necessary but we were already doing that in the GTK 4 backend. Backporting =========== The design here has made an attempt to make it possible to backport by keeping GdkMacosBuffer, GdkMacosLayer, and GdkMacosTile fairly independent. There may be an opportunity to integrate this into GTK 3's quartz backend with a fair bit of work. Doing so could improve the situation for applications which are damage-rich such as The GIMP.
2022-02-14 10:20:19 +00:00
g_assert (self->texture != 0 || self->fbo == 0);
g_assert (self->fbo != 0 || self->texture == 0);
macos: implement GL context This implements the basics for a GdkGLContext on macOS. Currently, rendering only is fully working for the GskCairoRenderer case where we read back pixels into a cairo surface for rendering. More work on synchronization is required for the GL on GskGLRenderer case. When we attempt to render a surface itself with GL, the context will ensure that the new GdkMacosGLView is placed within the NSWindow. In other cases, we use a dummy NSView and NSWindow for backing the NSOpenGLContext to ensure that we can get accelerated drawing. This gets GtkGLArea working when running with GSK_RENDERER=cairo.
2020-10-29 17:33:22 +00:00
macos: use CALayer and IOSurface for rendering This provides a major shift in how we draw both when accelerated OpenGL as well as software rendering with Cairo. In short, it uses tiles of Core Animation's CALayer to display contents from an OpenGL or Cairo rendering so that the window can provide partial damage updates. Partial damage is not generally available when using OpenGL as the whole buffer is flipped even if you only submitted a small change using a scissor rect. Thankfully, this speeds up Cairo rendering a bit too by using IOSurface to upload contents to the display server. We use the tiling system we do for OpenGL which reduces overall complexity and differences between them. A New Buffer ============ GdkMacosBuffer is a wrapper around an IOSurfaceRef. The term buffer was used because 1) surface is already used and 2) it loosely maps to a front/back buffer semantic. However, it appears that IOSurfaceRef contents are being retained in some fashion (likely in the compositor result) so we can update the same IOSurfaceRef without flipping as long as we're fast. This appears to be what Chromium does as well, but Firefox uses two IOSurfaceRef and flips between them. We would like to avoid two surfaces because it doubles the GPU VRAM requirements of the application. Changes to Windows ================== Previously, the NSWindow would dynamically change between different types of NSView based on the renderer being used. This is no longer necessary as we just have a single NSView type, GdkMacosView, which inherits from GdkMacosBaseView just to keep the tedius stuff separate from the machinery of GdkMacosView. We can merge those someday if we are okay with that. Changes to Views ================ GdkMacosCairoView, GdkMacosCairoSubView, GdkMacosGLView have all been removed and replaced with GdkMacosView. This new view has a single CALayer (GdkMacosLayer) attached to it which itself has sublayers. The contents of the CALayer is populated with an IOSurfaceRef which we allocated with the GdkMacosSurface. The surface is replaced when the NSWindow resizes. Changes to Layers ================= We now have a dedicated GdkMacosLayer which contains sublayers of GdkMacosTile. The tile has a maximum size of 128x128 pixels in device units. The GdkMacosTile is partitioned by splitting both the transparent region (window bounds minus opaque area) and then by splitting the opaque area. A tile has either translucent contents (and therefore is not opaque) or has opaque contents (and therefore is opaque). An opaque tile never contains transparent contents. As such, the opaque tiles contain a black background so that Core Animation will consider the tile's bounds as opaque. This can be verified with "Quartz Debug -> Show opaque regions". Changes to Cairo ================ GTK 4 cannot currently use cairo-quartz because of how CSS borders are rendered. It simply causes errors in the cairo_quartz_surface_t backend. Since we are restricted to using cairo_image_surface_t (which happens to be faster anyway) we can use the IOSurfaceBaseAddress() to obtain a mapping of the IOSurfaceRef in user-space. It always uses BGRA 32-bit with alpha channel even if we will discard the alpha channel as that is necessary to hit the fast paths in other parts of the platform. Note that while Cairo says CAIRO_FORMAT_ARGB32, it is really 32-bit BGRA on little-endian as we expect. OpenGL will render flipped (Quartz Native Co-ordinates) while Cairo renders with 0,O in the top-left. We could use cairo_translate() and cairo_scale() to reverse this, but it looks like some cairo things may not look quite as right if we do so. To reduce the chances of one-off bugs this continues to draw as Cairo would normally, but instead uses an CGAffineTransform in the tiles and some CGRect translation when swapping buffers to get the same effect. Changes to OpenGL ================= To simplify things, removal of all NSOpenGL* related components have been removed and we strictly use the Core GL (CGL*) API. This probably should have been done long ago anyay. Most examples found in the browsers to use IOSurfaceRef with OpenGL are using Legacy GL and there is still work underway to make this fit in with the rest of how the GSK GL renderer works. Since IOSurfaceRef bound to a texture/framebuffer will not have a default framebuffer ID of 0, we needed to add a default framebuffer id to the GdkGLContext. GskGLRenderer can use this to setup the command queue in such a way that our IOSurface destination has been glBindFramebuffer() as if it were the default drawable. This stuff is pretty slight-of-hand, so where things are and what needs flushing when and where has been a bit of an experiment to see what actually works to get synchronization across subsystems. Efficient Damages ================= After we draw with Cairo, we unlock the IOSurfaceRef and the contents are uploaded to the GPU. To make the contents visible to the app, we must clear the tiles contents with `layer.contents=nil;` and then re-apply the IOSurfaceRef. Since the buffer has likely not changed, we only do this if the tile overlaps the damage region. This gives the effect of having more tightly controlled damage regions even though updating the layer would damage be the whole window (as it is with OpenGL/Metal today with the exception of scissor-rect). This too can be verified usign "Quartz Debug -> Flash screen udpates". Frame Synchronized Resize ========================= In GTK 4, we have the ability to perform sizing changes from compute-size during the layout phase. Since the macOS backend already tracks window resizes manually, we can avoid doing the setFrame: immediately and instead do it within the frame clock's layout phase. Doing so gives us vastly better resize experience as we're more likely to get the size-change and updated-contents in the same frame on screen. It makes things feel "connected" in a way they weren't before. Some additional effort to tweak gravity during the process is also necessary but we were already doing that in the GTK 4 backend. Backporting =========== The design here has made an attempt to make it possible to backport by keeping GdkMacosBuffer, GdkMacosLayer, and GdkMacosTile fairly independent. There may be an opportunity to integrate this into GTK 3's quartz backend with a fair bit of work. Doing so could improve the situation for applications which are damage-rich such as The GIMP.
2022-02-14 10:20:19 +00:00
glBindFramebuffer (GL_FRAMEBUFFER, 0);
glActiveTexture (GL_TEXTURE0);
glBindTexture (self->target, 0);
macos: implement GL context This implements the basics for a GdkGLContext on macOS. Currently, rendering only is fully working for the GskCairoRenderer case where we read back pixels into a cairo surface for rendering. More work on synchronization is required for the GL on GskGLRenderer case. When we attempt to render a surface itself with GL, the context will ensure that the new GdkMacosGLView is placed within the NSWindow. In other cases, we use a dummy NSView and NSWindow for backing the NSOpenGLContext to ensure that we can get accelerated drawing. This gets GtkGLArea working when running with GSK_RENDERER=cairo.
2020-10-29 17:33:22 +00:00
macos: use CALayer and IOSurface for rendering This provides a major shift in how we draw both when accelerated OpenGL as well as software rendering with Cairo. In short, it uses tiles of Core Animation's CALayer to display contents from an OpenGL or Cairo rendering so that the window can provide partial damage updates. Partial damage is not generally available when using OpenGL as the whole buffer is flipped even if you only submitted a small change using a scissor rect. Thankfully, this speeds up Cairo rendering a bit too by using IOSurface to upload contents to the display server. We use the tiling system we do for OpenGL which reduces overall complexity and differences between them. A New Buffer ============ GdkMacosBuffer is a wrapper around an IOSurfaceRef. The term buffer was used because 1) surface is already used and 2) it loosely maps to a front/back buffer semantic. However, it appears that IOSurfaceRef contents are being retained in some fashion (likely in the compositor result) so we can update the same IOSurfaceRef without flipping as long as we're fast. This appears to be what Chromium does as well, but Firefox uses two IOSurfaceRef and flips between them. We would like to avoid two surfaces because it doubles the GPU VRAM requirements of the application. Changes to Windows ================== Previously, the NSWindow would dynamically change between different types of NSView based on the renderer being used. This is no longer necessary as we just have a single NSView type, GdkMacosView, which inherits from GdkMacosBaseView just to keep the tedius stuff separate from the machinery of GdkMacosView. We can merge those someday if we are okay with that. Changes to Views ================ GdkMacosCairoView, GdkMacosCairoSubView, GdkMacosGLView have all been removed and replaced with GdkMacosView. This new view has a single CALayer (GdkMacosLayer) attached to it which itself has sublayers. The contents of the CALayer is populated with an IOSurfaceRef which we allocated with the GdkMacosSurface. The surface is replaced when the NSWindow resizes. Changes to Layers ================= We now have a dedicated GdkMacosLayer which contains sublayers of GdkMacosTile. The tile has a maximum size of 128x128 pixels in device units. The GdkMacosTile is partitioned by splitting both the transparent region (window bounds minus opaque area) and then by splitting the opaque area. A tile has either translucent contents (and therefore is not opaque) or has opaque contents (and therefore is opaque). An opaque tile never contains transparent contents. As such, the opaque tiles contain a black background so that Core Animation will consider the tile's bounds as opaque. This can be verified with "Quartz Debug -> Show opaque regions". Changes to Cairo ================ GTK 4 cannot currently use cairo-quartz because of how CSS borders are rendered. It simply causes errors in the cairo_quartz_surface_t backend. Since we are restricted to using cairo_image_surface_t (which happens to be faster anyway) we can use the IOSurfaceBaseAddress() to obtain a mapping of the IOSurfaceRef in user-space. It always uses BGRA 32-bit with alpha channel even if we will discard the alpha channel as that is necessary to hit the fast paths in other parts of the platform. Note that while Cairo says CAIRO_FORMAT_ARGB32, it is really 32-bit BGRA on little-endian as we expect. OpenGL will render flipped (Quartz Native Co-ordinates) while Cairo renders with 0,O in the top-left. We could use cairo_translate() and cairo_scale() to reverse this, but it looks like some cairo things may not look quite as right if we do so. To reduce the chances of one-off bugs this continues to draw as Cairo would normally, but instead uses an CGAffineTransform in the tiles and some CGRect translation when swapping buffers to get the same effect. Changes to OpenGL ================= To simplify things, removal of all NSOpenGL* related components have been removed and we strictly use the Core GL (CGL*) API. This probably should have been done long ago anyay. Most examples found in the browsers to use IOSurfaceRef with OpenGL are using Legacy GL and there is still work underway to make this fit in with the rest of how the GSK GL renderer works. Since IOSurfaceRef bound to a texture/framebuffer will not have a default framebuffer ID of 0, we needed to add a default framebuffer id to the GdkGLContext. GskGLRenderer can use this to setup the command queue in such a way that our IOSurface destination has been glBindFramebuffer() as if it were the default drawable. This stuff is pretty slight-of-hand, so where things are and what needs flushing when and where has been a bit of an experiment to see what actually works to get synchronization across subsystems. Efficient Damages ================= After we draw with Cairo, we unlock the IOSurfaceRef and the contents are uploaded to the GPU. To make the contents visible to the app, we must clear the tiles contents with `layer.contents=nil;` and then re-apply the IOSurfaceRef. Since the buffer has likely not changed, we only do this if the tile overlaps the damage region. This gives the effect of having more tightly controlled damage regions even though updating the layer would damage be the whole window (as it is with OpenGL/Metal today with the exception of scissor-rect). This too can be verified usign "Quartz Debug -> Flash screen udpates". Frame Synchronized Resize ========================= In GTK 4, we have the ability to perform sizing changes from compute-size during the layout phase. Since the macOS backend already tracks window resizes manually, we can avoid doing the setFrame: immediately and instead do it within the frame clock's layout phase. Doing so gives us vastly better resize experience as we're more likely to get the size-change and updated-contents in the same frame on screen. It makes things feel "connected" in a way they weren't before. Some additional effort to tweak gravity during the process is also necessary but we were already doing that in the GTK 4 backend. Backporting =========== The design here has made an attempt to make it possible to backport by keeping GdkMacosBuffer, GdkMacosLayer, and GdkMacosTile fairly independent. There may be an opportunity to integrate this into GTK 3's quartz backend with a fair bit of work. Doing so could improve the situation for applications which are damage-rich such as The GIMP.
2022-02-14 10:20:19 +00:00
if (self->fbo != 0)
macos: implement GL context This implements the basics for a GdkGLContext on macOS. Currently, rendering only is fully working for the GskCairoRenderer case where we read back pixels into a cairo surface for rendering. More work on synchronization is required for the GL on GskGLRenderer case. When we attempt to render a surface itself with GL, the context will ensure that the new GdkMacosGLView is placed within the NSWindow. In other cases, we use a dummy NSView and NSWindow for backing the NSOpenGLContext to ensure that we can get accelerated drawing. This gets GtkGLArea working when running with GSK_RENDERER=cairo.
2020-10-29 17:33:22 +00:00
{
macos: use CALayer and IOSurface for rendering This provides a major shift in how we draw both when accelerated OpenGL as well as software rendering with Cairo. In short, it uses tiles of Core Animation's CALayer to display contents from an OpenGL or Cairo rendering so that the window can provide partial damage updates. Partial damage is not generally available when using OpenGL as the whole buffer is flipped even if you only submitted a small change using a scissor rect. Thankfully, this speeds up Cairo rendering a bit too by using IOSurface to upload contents to the display server. We use the tiling system we do for OpenGL which reduces overall complexity and differences between them. A New Buffer ============ GdkMacosBuffer is a wrapper around an IOSurfaceRef. The term buffer was used because 1) surface is already used and 2) it loosely maps to a front/back buffer semantic. However, it appears that IOSurfaceRef contents are being retained in some fashion (likely in the compositor result) so we can update the same IOSurfaceRef without flipping as long as we're fast. This appears to be what Chromium does as well, but Firefox uses two IOSurfaceRef and flips between them. We would like to avoid two surfaces because it doubles the GPU VRAM requirements of the application. Changes to Windows ================== Previously, the NSWindow would dynamically change between different types of NSView based on the renderer being used. This is no longer necessary as we just have a single NSView type, GdkMacosView, which inherits from GdkMacosBaseView just to keep the tedius stuff separate from the machinery of GdkMacosView. We can merge those someday if we are okay with that. Changes to Views ================ GdkMacosCairoView, GdkMacosCairoSubView, GdkMacosGLView have all been removed and replaced with GdkMacosView. This new view has a single CALayer (GdkMacosLayer) attached to it which itself has sublayers. The contents of the CALayer is populated with an IOSurfaceRef which we allocated with the GdkMacosSurface. The surface is replaced when the NSWindow resizes. Changes to Layers ================= We now have a dedicated GdkMacosLayer which contains sublayers of GdkMacosTile. The tile has a maximum size of 128x128 pixels in device units. The GdkMacosTile is partitioned by splitting both the transparent region (window bounds minus opaque area) and then by splitting the opaque area. A tile has either translucent contents (and therefore is not opaque) or has opaque contents (and therefore is opaque). An opaque tile never contains transparent contents. As such, the opaque tiles contain a black background so that Core Animation will consider the tile's bounds as opaque. This can be verified with "Quartz Debug -> Show opaque regions". Changes to Cairo ================ GTK 4 cannot currently use cairo-quartz because of how CSS borders are rendered. It simply causes errors in the cairo_quartz_surface_t backend. Since we are restricted to using cairo_image_surface_t (which happens to be faster anyway) we can use the IOSurfaceBaseAddress() to obtain a mapping of the IOSurfaceRef in user-space. It always uses BGRA 32-bit with alpha channel even if we will discard the alpha channel as that is necessary to hit the fast paths in other parts of the platform. Note that while Cairo says CAIRO_FORMAT_ARGB32, it is really 32-bit BGRA on little-endian as we expect. OpenGL will render flipped (Quartz Native Co-ordinates) while Cairo renders with 0,O in the top-left. We could use cairo_translate() and cairo_scale() to reverse this, but it looks like some cairo things may not look quite as right if we do so. To reduce the chances of one-off bugs this continues to draw as Cairo would normally, but instead uses an CGAffineTransform in the tiles and some CGRect translation when swapping buffers to get the same effect. Changes to OpenGL ================= To simplify things, removal of all NSOpenGL* related components have been removed and we strictly use the Core GL (CGL*) API. This probably should have been done long ago anyay. Most examples found in the browsers to use IOSurfaceRef with OpenGL are using Legacy GL and there is still work underway to make this fit in with the rest of how the GSK GL renderer works. Since IOSurfaceRef bound to a texture/framebuffer will not have a default framebuffer ID of 0, we needed to add a default framebuffer id to the GdkGLContext. GskGLRenderer can use this to setup the command queue in such a way that our IOSurface destination has been glBindFramebuffer() as if it were the default drawable. This stuff is pretty slight-of-hand, so where things are and what needs flushing when and where has been a bit of an experiment to see what actually works to get synchronization across subsystems. Efficient Damages ================= After we draw with Cairo, we unlock the IOSurfaceRef and the contents are uploaded to the GPU. To make the contents visible to the app, we must clear the tiles contents with `layer.contents=nil;` and then re-apply the IOSurfaceRef. Since the buffer has likely not changed, we only do this if the tile overlaps the damage region. This gives the effect of having more tightly controlled damage regions even though updating the layer would damage be the whole window (as it is with OpenGL/Metal today with the exception of scissor-rect). This too can be verified usign "Quartz Debug -> Flash screen udpates". Frame Synchronized Resize ========================= In GTK 4, we have the ability to perform sizing changes from compute-size during the layout phase. Since the macOS backend already tracks window resizes manually, we can avoid doing the setFrame: immediately and instead do it within the frame clock's layout phase. Doing so gives us vastly better resize experience as we're more likely to get the size-change and updated-contents in the same frame on screen. It makes things feel "connected" in a way they weren't before. Some additional effort to tweak gravity during the process is also necessary but we were already doing that in the GTK 4 backend. Backporting =========== The design here has made an attempt to make it possible to backport by keeping GdkMacosBuffer, GdkMacosLayer, and GdkMacosTile fairly independent. There may be an opportunity to integrate this into GTK 3's quartz backend with a fair bit of work. Doing so could improve the situation for applications which are damage-rich such as The GIMP.
2022-02-14 10:20:19 +00:00
glDeleteFramebuffers (1, &self->fbo);
self->fbo = 0;
}
macos: always create dummy window/view for GL context This simplifies the creation by always creating the dummy views, and then removes it if necessary by detecting the begin_frame/end_frame pair.
2020-11-04 22:40:11 +00:00
macos: use CALayer and IOSurface for rendering This provides a major shift in how we draw both when accelerated OpenGL as well as software rendering with Cairo. In short, it uses tiles of Core Animation's CALayer to display contents from an OpenGL or Cairo rendering so that the window can provide partial damage updates. Partial damage is not generally available when using OpenGL as the whole buffer is flipped even if you only submitted a small change using a scissor rect. Thankfully, this speeds up Cairo rendering a bit too by using IOSurface to upload contents to the display server. We use the tiling system we do for OpenGL which reduces overall complexity and differences between them. A New Buffer ============ GdkMacosBuffer is a wrapper around an IOSurfaceRef. The term buffer was used because 1) surface is already used and 2) it loosely maps to a front/back buffer semantic. However, it appears that IOSurfaceRef contents are being retained in some fashion (likely in the compositor result) so we can update the same IOSurfaceRef without flipping as long as we're fast. This appears to be what Chromium does as well, but Firefox uses two IOSurfaceRef and flips between them. We would like to avoid two surfaces because it doubles the GPU VRAM requirements of the application. Changes to Windows ================== Previously, the NSWindow would dynamically change between different types of NSView based on the renderer being used. This is no longer necessary as we just have a single NSView type, GdkMacosView, which inherits from GdkMacosBaseView just to keep the tedius stuff separate from the machinery of GdkMacosView. We can merge those someday if we are okay with that. Changes to Views ================ GdkMacosCairoView, GdkMacosCairoSubView, GdkMacosGLView have all been removed and replaced with GdkMacosView. This new view has a single CALayer (GdkMacosLayer) attached to it which itself has sublayers. The contents of the CALayer is populated with an IOSurfaceRef which we allocated with the GdkMacosSurface. The surface is replaced when the NSWindow resizes. Changes to Layers ================= We now have a dedicated GdkMacosLayer which contains sublayers of GdkMacosTile. The tile has a maximum size of 128x128 pixels in device units. The GdkMacosTile is partitioned by splitting both the transparent region (window bounds minus opaque area) and then by splitting the opaque area. A tile has either translucent contents (and therefore is not opaque) or has opaque contents (and therefore is opaque). An opaque tile never contains transparent contents. As such, the opaque tiles contain a black background so that Core Animation will consider the tile's bounds as opaque. This can be verified with "Quartz Debug -> Show opaque regions". Changes to Cairo ================ GTK 4 cannot currently use cairo-quartz because of how CSS borders are rendered. It simply causes errors in the cairo_quartz_surface_t backend. Since we are restricted to using cairo_image_surface_t (which happens to be faster anyway) we can use the IOSurfaceBaseAddress() to obtain a mapping of the IOSurfaceRef in user-space. It always uses BGRA 32-bit with alpha channel even if we will discard the alpha channel as that is necessary to hit the fast paths in other parts of the platform. Note that while Cairo says CAIRO_FORMAT_ARGB32, it is really 32-bit BGRA on little-endian as we expect. OpenGL will render flipped (Quartz Native Co-ordinates) while Cairo renders with 0,O in the top-left. We could use cairo_translate() and cairo_scale() to reverse this, but it looks like some cairo things may not look quite as right if we do so. To reduce the chances of one-off bugs this continues to draw as Cairo would normally, but instead uses an CGAffineTransform in the tiles and some CGRect translation when swapping buffers to get the same effect. Changes to OpenGL ================= To simplify things, removal of all NSOpenGL* related components have been removed and we strictly use the Core GL (CGL*) API. This probably should have been done long ago anyay. Most examples found in the browsers to use IOSurfaceRef with OpenGL are using Legacy GL and there is still work underway to make this fit in with the rest of how the GSK GL renderer works. Since IOSurfaceRef bound to a texture/framebuffer will not have a default framebuffer ID of 0, we needed to add a default framebuffer id to the GdkGLContext. GskGLRenderer can use this to setup the command queue in such a way that our IOSurface destination has been glBindFramebuffer() as if it were the default drawable. This stuff is pretty slight-of-hand, so where things are and what needs flushing when and where has been a bit of an experiment to see what actually works to get synchronization across subsystems. Efficient Damages ================= After we draw with Cairo, we unlock the IOSurfaceRef and the contents are uploaded to the GPU. To make the contents visible to the app, we must clear the tiles contents with `layer.contents=nil;` and then re-apply the IOSurfaceRef. Since the buffer has likely not changed, we only do this if the tile overlaps the damage region. This gives the effect of having more tightly controlled damage regions even though updating the layer would damage be the whole window (as it is with OpenGL/Metal today with the exception of scissor-rect). This too can be verified usign "Quartz Debug -> Flash screen udpates". Frame Synchronized Resize ========================= In GTK 4, we have the ability to perform sizing changes from compute-size during the layout phase. Since the macOS backend already tracks window resizes manually, we can avoid doing the setFrame: immediately and instead do it within the frame clock's layout phase. Doing so gives us vastly better resize experience as we're more likely to get the size-change and updated-contents in the same frame on screen. It makes things feel "connected" in a way they weren't before. Some additional effort to tweak gravity during the process is also necessary but we were already doing that in the GTK 4 backend. Backporting =========== The design here has made an attempt to make it possible to backport by keeping GdkMacosBuffer, GdkMacosLayer, and GdkMacosTile fairly independent. There may be an opportunity to integrate this into GTK 3's quartz backend with a fair bit of work. Doing so could improve the situation for applications which are damage-rich such as The GIMP.
2022-02-14 10:20:19 +00:00
if (self->texture != 0)
{
glDeleteTextures (1, &self->texture);
self->texture = 0;
macos: implement GL context This implements the basics for a GdkGLContext on macOS. Currently, rendering only is fully working for the GskCairoRenderer case where we read back pixels into a cairo surface for rendering. More work on synchronization is required for the GL on GskGLRenderer case. When we attempt to render a surface itself with GL, the context will ensure that the new GdkMacosGLView is placed within the NSWindow. In other cases, we use a dummy NSView and NSWindow for backing the NSOpenGLContext to ensure that we can get accelerated drawing. This gets GtkGLArea working when running with GSK_RENDERER=cairo.
2020-10-29 17:33:22 +00:00
}
macos: use CALayer and IOSurface for rendering This provides a major shift in how we draw both when accelerated OpenGL as well as software rendering with Cairo. In short, it uses tiles of Core Animation's CALayer to display contents from an OpenGL or Cairo rendering so that the window can provide partial damage updates. Partial damage is not generally available when using OpenGL as the whole buffer is flipped even if you only submitted a small change using a scissor rect. Thankfully, this speeds up Cairo rendering a bit too by using IOSurface to upload contents to the display server. We use the tiling system we do for OpenGL which reduces overall complexity and differences between them. A New Buffer ============ GdkMacosBuffer is a wrapper around an IOSurfaceRef. The term buffer was used because 1) surface is already used and 2) it loosely maps to a front/back buffer semantic. However, it appears that IOSurfaceRef contents are being retained in some fashion (likely in the compositor result) so we can update the same IOSurfaceRef without flipping as long as we're fast. This appears to be what Chromium does as well, but Firefox uses two IOSurfaceRef and flips between them. We would like to avoid two surfaces because it doubles the GPU VRAM requirements of the application. Changes to Windows ================== Previously, the NSWindow would dynamically change between different types of NSView based on the renderer being used. This is no longer necessary as we just have a single NSView type, GdkMacosView, which inherits from GdkMacosBaseView just to keep the tedius stuff separate from the machinery of GdkMacosView. We can merge those someday if we are okay with that. Changes to Views ================ GdkMacosCairoView, GdkMacosCairoSubView, GdkMacosGLView have all been removed and replaced with GdkMacosView. This new view has a single CALayer (GdkMacosLayer) attached to it which itself has sublayers. The contents of the CALayer is populated with an IOSurfaceRef which we allocated with the GdkMacosSurface. The surface is replaced when the NSWindow resizes. Changes to Layers ================= We now have a dedicated GdkMacosLayer which contains sublayers of GdkMacosTile. The tile has a maximum size of 128x128 pixels in device units. The GdkMacosTile is partitioned by splitting both the transparent region (window bounds minus opaque area) and then by splitting the opaque area. A tile has either translucent contents (and therefore is not opaque) or has opaque contents (and therefore is opaque). An opaque tile never contains transparent contents. As such, the opaque tiles contain a black background so that Core Animation will consider the tile's bounds as opaque. This can be verified with "Quartz Debug -> Show opaque regions". Changes to Cairo ================ GTK 4 cannot currently use cairo-quartz because of how CSS borders are rendered. It simply causes errors in the cairo_quartz_surface_t backend. Since we are restricted to using cairo_image_surface_t (which happens to be faster anyway) we can use the IOSurfaceBaseAddress() to obtain a mapping of the IOSurfaceRef in user-space. It always uses BGRA 32-bit with alpha channel even if we will discard the alpha channel as that is necessary to hit the fast paths in other parts of the platform. Note that while Cairo says CAIRO_FORMAT_ARGB32, it is really 32-bit BGRA on little-endian as we expect. OpenGL will render flipped (Quartz Native Co-ordinates) while Cairo renders with 0,O in the top-left. We could use cairo_translate() and cairo_scale() to reverse this, but it looks like some cairo things may not look quite as right if we do so. To reduce the chances of one-off bugs this continues to draw as Cairo would normally, but instead uses an CGAffineTransform in the tiles and some CGRect translation when swapping buffers to get the same effect. Changes to OpenGL ================= To simplify things, removal of all NSOpenGL* related components have been removed and we strictly use the Core GL (CGL*) API. This probably should have been done long ago anyay. Most examples found in the browsers to use IOSurfaceRef with OpenGL are using Legacy GL and there is still work underway to make this fit in with the rest of how the GSK GL renderer works. Since IOSurfaceRef bound to a texture/framebuffer will not have a default framebuffer ID of 0, we needed to add a default framebuffer id to the GdkGLContext. GskGLRenderer can use this to setup the command queue in such a way that our IOSurface destination has been glBindFramebuffer() as if it were the default drawable. This stuff is pretty slight-of-hand, so where things are and what needs flushing when and where has been a bit of an experiment to see what actually works to get synchronization across subsystems. Efficient Damages ================= After we draw with Cairo, we unlock the IOSurfaceRef and the contents are uploaded to the GPU. To make the contents visible to the app, we must clear the tiles contents with `layer.contents=nil;` and then re-apply the IOSurfaceRef. Since the buffer has likely not changed, we only do this if the tile overlaps the damage region. This gives the effect of having more tightly controlled damage regions even though updating the layer would damage be the whole window (as it is with OpenGL/Metal today with the exception of scissor-rect). This too can be verified usign "Quartz Debug -> Flash screen udpates". Frame Synchronized Resize ========================= In GTK 4, we have the ability to perform sizing changes from compute-size during the layout phase. Since the macOS backend already tracks window resizes manually, we can avoid doing the setFrame: immediately and instead do it within the frame clock's layout phase. Doing so gives us vastly better resize experience as we're more likely to get the size-change and updated-contents in the same frame on screen. It makes things feel "connected" in a way they weren't before. Some additional effort to tweak gravity during the process is also necessary but we were already doing that in the GTK 4 backend. Backporting =========== The design here has made an attempt to make it possible to backport by keeping GdkMacosBuffer, GdkMacosLayer, and GdkMacosTile fairly independent. There may be an opportunity to integrate this into GTK 3's quartz backend with a fair bit of work. Doing so could improve the situation for applications which are damage-rich such as The GIMP.
2022-02-14 10:20:19 +00:00
}
static CGLPixelFormatObj
create_pixel_format (int major,
int minor,
GError **error)
{
CGLPixelFormatAttribute attrs[] = {
kCGLPFAOpenGLProfile, (CGLPixelFormatAttribute)kCGLOGLPVersion_Legacy,
kCGLPFAAllowOfflineRenderers, /* allow sharing across GPUs */
kCGLPFAColorSize, 24,
kCGLPFAAlphaSize, 8,
0
};
CGLPixelFormatObj format = NULL;
GLint n_format = 1;
if (major == 3 && minor == 2)
attrs[1] = (CGLPixelFormatAttribute)kCGLOGLPVersion_GL3_Core;
else if (major == 4 && minor == 1)
attrs[1] = (CGLPixelFormatAttribute)kCGLOGLPVersion_GL4_Core;
macos: implement GL context This implements the basics for a GdkGLContext on macOS. Currently, rendering only is fully working for the GskCairoRenderer case where we read back pixels into a cairo surface for rendering. More work on synchronization is required for the GL on GskGLRenderer case. When we attempt to render a surface itself with GL, the context will ensure that the new GdkMacosGLView is placed within the NSWindow. In other cases, we use a dummy NSView and NSWindow for backing the NSOpenGLContext to ensure that we can get accelerated drawing. This gets GtkGLArea working when running with GSK_RENDERER=cairo.
2020-10-29 17:33:22 +00:00
macos: use CALayer and IOSurface for rendering This provides a major shift in how we draw both when accelerated OpenGL as well as software rendering with Cairo. In short, it uses tiles of Core Animation's CALayer to display contents from an OpenGL or Cairo rendering so that the window can provide partial damage updates. Partial damage is not generally available when using OpenGL as the whole buffer is flipped even if you only submitted a small change using a scissor rect. Thankfully, this speeds up Cairo rendering a bit too by using IOSurface to upload contents to the display server. We use the tiling system we do for OpenGL which reduces overall complexity and differences between them. A New Buffer ============ GdkMacosBuffer is a wrapper around an IOSurfaceRef. The term buffer was used because 1) surface is already used and 2) it loosely maps to a front/back buffer semantic. However, it appears that IOSurfaceRef contents are being retained in some fashion (likely in the compositor result) so we can update the same IOSurfaceRef without flipping as long as we're fast. This appears to be what Chromium does as well, but Firefox uses two IOSurfaceRef and flips between them. We would like to avoid two surfaces because it doubles the GPU VRAM requirements of the application. Changes to Windows ================== Previously, the NSWindow would dynamically change between different types of NSView based on the renderer being used. This is no longer necessary as we just have a single NSView type, GdkMacosView, which inherits from GdkMacosBaseView just to keep the tedius stuff separate from the machinery of GdkMacosView. We can merge those someday if we are okay with that. Changes to Views ================ GdkMacosCairoView, GdkMacosCairoSubView, GdkMacosGLView have all been removed and replaced with GdkMacosView. This new view has a single CALayer (GdkMacosLayer) attached to it which itself has sublayers. The contents of the CALayer is populated with an IOSurfaceRef which we allocated with the GdkMacosSurface. The surface is replaced when the NSWindow resizes. Changes to Layers ================= We now have a dedicated GdkMacosLayer which contains sublayers of GdkMacosTile. The tile has a maximum size of 128x128 pixels in device units. The GdkMacosTile is partitioned by splitting both the transparent region (window bounds minus opaque area) and then by splitting the opaque area. A tile has either translucent contents (and therefore is not opaque) or has opaque contents (and therefore is opaque). An opaque tile never contains transparent contents. As such, the opaque tiles contain a black background so that Core Animation will consider the tile's bounds as opaque. This can be verified with "Quartz Debug -> Show opaque regions". Changes to Cairo ================ GTK 4 cannot currently use cairo-quartz because of how CSS borders are rendered. It simply causes errors in the cairo_quartz_surface_t backend. Since we are restricted to using cairo_image_surface_t (which happens to be faster anyway) we can use the IOSurfaceBaseAddress() to obtain a mapping of the IOSurfaceRef in user-space. It always uses BGRA 32-bit with alpha channel even if we will discard the alpha channel as that is necessary to hit the fast paths in other parts of the platform. Note that while Cairo says CAIRO_FORMAT_ARGB32, it is really 32-bit BGRA on little-endian as we expect. OpenGL will render flipped (Quartz Native Co-ordinates) while Cairo renders with 0,O in the top-left. We could use cairo_translate() and cairo_scale() to reverse this, but it looks like some cairo things may not look quite as right if we do so. To reduce the chances of one-off bugs this continues to draw as Cairo would normally, but instead uses an CGAffineTransform in the tiles and some CGRect translation when swapping buffers to get the same effect. Changes to OpenGL ================= To simplify things, removal of all NSOpenGL* related components have been removed and we strictly use the Core GL (CGL*) API. This probably should have been done long ago anyay. Most examples found in the browsers to use IOSurfaceRef with OpenGL are using Legacy GL and there is still work underway to make this fit in with the rest of how the GSK GL renderer works. Since IOSurfaceRef bound to a texture/framebuffer will not have a default framebuffer ID of 0, we needed to add a default framebuffer id to the GdkGLContext. GskGLRenderer can use this to setup the command queue in such a way that our IOSurface destination has been glBindFramebuffer() as if it were the default drawable. This stuff is pretty slight-of-hand, so where things are and what needs flushing when and where has been a bit of an experiment to see what actually works to get synchronization across subsystems. Efficient Damages ================= After we draw with Cairo, we unlock the IOSurfaceRef and the contents are uploaded to the GPU. To make the contents visible to the app, we must clear the tiles contents with `layer.contents=nil;` and then re-apply the IOSurfaceRef. Since the buffer has likely not changed, we only do this if the tile overlaps the damage region. This gives the effect of having more tightly controlled damage regions even though updating the layer would damage be the whole window (as it is with OpenGL/Metal today with the exception of scissor-rect). This too can be verified usign "Quartz Debug -> Flash screen udpates". Frame Synchronized Resize ========================= In GTK 4, we have the ability to perform sizing changes from compute-size during the layout phase. Since the macOS backend already tracks window resizes manually, we can avoid doing the setFrame: immediately and instead do it within the frame clock's layout phase. Doing so gives us vastly better resize experience as we're more likely to get the size-change and updated-contents in the same frame on screen. It makes things feel "connected" in a way they weren't before. Some additional effort to tweak gravity during the process is also necessary but we were already doing that in the GTK 4 backend. Backporting =========== The design here has made an attempt to make it possible to backport by keeping GdkMacosBuffer, GdkMacosLayer, and GdkMacosTile fairly independent. There may be an opportunity to integrate this into GTK 3's quartz backend with a fair bit of work. Doing so could improve the situation for applications which are damage-rich such as The GIMP.
2022-02-14 10:20:19 +00:00
if (!CHECK (error, CGLChoosePixelFormat (attrs, &format, &n_format)))
return NULL;
return g_steal_pointer (&format);
macos: implement GL context This implements the basics for a GdkGLContext on macOS. Currently, rendering only is fully working for the GskCairoRenderer case where we read back pixels into a cairo surface for rendering. More work on synchronization is required for the GL on GskGLRenderer case. When we attempt to render a surface itself with GL, the context will ensure that the new GdkMacosGLView is placed within the NSWindow. In other cases, we use a dummy NSView and NSWindow for backing the NSOpenGLContext to ensure that we can get accelerated drawing. This gets GtkGLArea working when running with GSK_RENDERER=cairo.
2020-10-29 17:33:22 +00:00
}
macos: Fix return value
2021-11-03 21:53:31 +00:00
static GdkGLAPI
macos: implement GL context This implements the basics for a GdkGLContext on macOS. Currently, rendering only is fully working for the GskCairoRenderer case where we read back pixels into a cairo surface for rendering. More work on synchronization is required for the GL on GskGLRenderer case. When we attempt to render a surface itself with GL, the context will ensure that the new GdkMacosGLView is placed within the NSWindow. In other cases, we use a dummy NSView and NSWindow for backing the NSOpenGLContext to ensure that we can get accelerated drawing. This gets GtkGLArea working when running with GSK_RENDERER=cairo.
2020-10-29 17:33:22 +00:00
gdk_macos_gl_context_real_realize (GdkGLContext *context,
GError **error)
{
GdkMacosGLContext *self = (GdkMacosGLContext *)context;
GdkSurface *surface;
gdk: Add GdkDisplay::init_gl vfunc The vfunc is called to initialize GL and it returns a "base" context that GDK then uses as the context all others are shared with. So the GL context share tree now looks like: + context from init_gl - context1 - context2 ... So this is a flat tree now, the complexity is gone. The only caveat is that backends now need to create a GL context when initializing GL so some refactoring was needed. Two new functions have been added: * gdk_display_prepare_gl() This is public API and can be used to ensure that GL has been initialized or if not, retrieve an error to display (or debug-print). * gdk_display_get_gl_context() This is a private function to retrieve the base context from init_gl(). It replaces gdk_surface_get_shared_data_context().
2021-07-04 23:57:03 +00:00
GdkDisplay *display;
macos: use CALayer and IOSurface for rendering This provides a major shift in how we draw both when accelerated OpenGL as well as software rendering with Cairo. In short, it uses tiles of Core Animation's CALayer to display contents from an OpenGL or Cairo rendering so that the window can provide partial damage updates. Partial damage is not generally available when using OpenGL as the whole buffer is flipped even if you only submitted a small change using a scissor rect. Thankfully, this speeds up Cairo rendering a bit too by using IOSurface to upload contents to the display server. We use the tiling system we do for OpenGL which reduces overall complexity and differences between them. A New Buffer ============ GdkMacosBuffer is a wrapper around an IOSurfaceRef. The term buffer was used because 1) surface is already used and 2) it loosely maps to a front/back buffer semantic. However, it appears that IOSurfaceRef contents are being retained in some fashion (likely in the compositor result) so we can update the same IOSurfaceRef without flipping as long as we're fast. This appears to be what Chromium does as well, but Firefox uses two IOSurfaceRef and flips between them. We would like to avoid two surfaces because it doubles the GPU VRAM requirements of the application. Changes to Windows ================== Previously, the NSWindow would dynamically change between different types of NSView based on the renderer being used. This is no longer necessary as we just have a single NSView type, GdkMacosView, which inherits from GdkMacosBaseView just to keep the tedius stuff separate from the machinery of GdkMacosView. We can merge those someday if we are okay with that. Changes to Views ================ GdkMacosCairoView, GdkMacosCairoSubView, GdkMacosGLView have all been removed and replaced with GdkMacosView. This new view has a single CALayer (GdkMacosLayer) attached to it which itself has sublayers. The contents of the CALayer is populated with an IOSurfaceRef which we allocated with the GdkMacosSurface. The surface is replaced when the NSWindow resizes. Changes to Layers ================= We now have a dedicated GdkMacosLayer which contains sublayers of GdkMacosTile. The tile has a maximum size of 128x128 pixels in device units. The GdkMacosTile is partitioned by splitting both the transparent region (window bounds minus opaque area) and then by splitting the opaque area. A tile has either translucent contents (and therefore is not opaque) or has opaque contents (and therefore is opaque). An opaque tile never contains transparent contents. As such, the opaque tiles contain a black background so that Core Animation will consider the tile's bounds as opaque. This can be verified with "Quartz Debug -> Show opaque regions". Changes to Cairo ================ GTK 4 cannot currently use cairo-quartz because of how CSS borders are rendered. It simply causes errors in the cairo_quartz_surface_t backend. Since we are restricted to using cairo_image_surface_t (which happens to be faster anyway) we can use the IOSurfaceBaseAddress() to obtain a mapping of the IOSurfaceRef in user-space. It always uses BGRA 32-bit with alpha channel even if we will discard the alpha channel as that is necessary to hit the fast paths in other parts of the platform. Note that while Cairo says CAIRO_FORMAT_ARGB32, it is really 32-bit BGRA on little-endian as we expect. OpenGL will render flipped (Quartz Native Co-ordinates) while Cairo renders with 0,O in the top-left. We could use cairo_translate() and cairo_scale() to reverse this, but it looks like some cairo things may not look quite as right if we do so. To reduce the chances of one-off bugs this continues to draw as Cairo would normally, but instead uses an CGAffineTransform in the tiles and some CGRect translation when swapping buffers to get the same effect. Changes to OpenGL ================= To simplify things, removal of all NSOpenGL* related components have been removed and we strictly use the Core GL (CGL*) API. This probably should have been done long ago anyay. Most examples found in the browsers to use IOSurfaceRef with OpenGL are using Legacy GL and there is still work underway to make this fit in with the rest of how the GSK GL renderer works. Since IOSurfaceRef bound to a texture/framebuffer will not have a default framebuffer ID of 0, we needed to add a default framebuffer id to the GdkGLContext. GskGLRenderer can use this to setup the command queue in such a way that our IOSurface destination has been glBindFramebuffer() as if it were the default drawable. This stuff is pretty slight-of-hand, so where things are and what needs flushing when and where has been a bit of an experiment to see what actually works to get synchronization across subsystems. Efficient Damages ================= After we draw with Cairo, we unlock the IOSurfaceRef and the contents are uploaded to the GPU. To make the contents visible to the app, we must clear the tiles contents with `layer.contents=nil;` and then re-apply the IOSurfaceRef. Since the buffer has likely not changed, we only do this if the tile overlaps the damage region. This gives the effect of having more tightly controlled damage regions even though updating the layer would damage be the whole window (as it is with OpenGL/Metal today with the exception of scissor-rect). This too can be verified usign "Quartz Debug -> Flash screen udpates". Frame Synchronized Resize ========================= In GTK 4, we have the ability to perform sizing changes from compute-size during the layout phase. Since the macOS backend already tracks window resizes manually, we can avoid doing the setFrame: immediately and instead do it within the frame clock's layout phase. Doing so gives us vastly better resize experience as we're more likely to get the size-change and updated-contents in the same frame on screen. It makes things feel "connected" in a way they weren't before. Some additional effort to tweak gravity during the process is also necessary but we were already doing that in the GTK 4 backend. Backporting =========== The design here has made an attempt to make it possible to backport by keeping GdkMacosBuffer, GdkMacosLayer, and GdkMacosTile fairly independent. There may be an opportunity to integrate this into GTK 3's quartz backend with a fair bit of work. Doing so could improve the situation for applications which are damage-rich such as The GIMP.
2022-02-14 10:20:19 +00:00
CGLPixelFormatObj pixelFormat;
CGLContextObj shared_gl_context = nil;
macos: use CGLSetParameter and CGLEnable We don't need to go through the NSOpenGLContext for these. We can just use the C API directly. It's also clearer what is using CGLEnable() vs CGLSetParameter().
2020-12-07 19:47:51 +00:00
CGLContextObj cgl_context;
macos: use CALayer and IOSurface for rendering This provides a major shift in how we draw both when accelerated OpenGL as well as software rendering with Cairo. In short, it uses tiles of Core Animation's CALayer to display contents from an OpenGL or Cairo rendering so that the window can provide partial damage updates. Partial damage is not generally available when using OpenGL as the whole buffer is flipped even if you only submitted a small change using a scissor rect. Thankfully, this speeds up Cairo rendering a bit too by using IOSurface to upload contents to the display server. We use the tiling system we do for OpenGL which reduces overall complexity and differences between them. A New Buffer ============ GdkMacosBuffer is a wrapper around an IOSurfaceRef. The term buffer was used because 1) surface is already used and 2) it loosely maps to a front/back buffer semantic. However, it appears that IOSurfaceRef contents are being retained in some fashion (likely in the compositor result) so we can update the same IOSurfaceRef without flipping as long as we're fast. This appears to be what Chromium does as well, but Firefox uses two IOSurfaceRef and flips between them. We would like to avoid two surfaces because it doubles the GPU VRAM requirements of the application. Changes to Windows ================== Previously, the NSWindow would dynamically change between different types of NSView based on the renderer being used. This is no longer necessary as we just have a single NSView type, GdkMacosView, which inherits from GdkMacosBaseView just to keep the tedius stuff separate from the machinery of GdkMacosView. We can merge those someday if we are okay with that. Changes to Views ================ GdkMacosCairoView, GdkMacosCairoSubView, GdkMacosGLView have all been removed and replaced with GdkMacosView. This new view has a single CALayer (GdkMacosLayer) attached to it which itself has sublayers. The contents of the CALayer is populated with an IOSurfaceRef which we allocated with the GdkMacosSurface. The surface is replaced when the NSWindow resizes. Changes to Layers ================= We now have a dedicated GdkMacosLayer which contains sublayers of GdkMacosTile. The tile has a maximum size of 128x128 pixels in device units. The GdkMacosTile is partitioned by splitting both the transparent region (window bounds minus opaque area) and then by splitting the opaque area. A tile has either translucent contents (and therefore is not opaque) or has opaque contents (and therefore is opaque). An opaque tile never contains transparent contents. As such, the opaque tiles contain a black background so that Core Animation will consider the tile's bounds as opaque. This can be verified with "Quartz Debug -> Show opaque regions". Changes to Cairo ================ GTK 4 cannot currently use cairo-quartz because of how CSS borders are rendered. It simply causes errors in the cairo_quartz_surface_t backend. Since we are restricted to using cairo_image_surface_t (which happens to be faster anyway) we can use the IOSurfaceBaseAddress() to obtain a mapping of the IOSurfaceRef in user-space. It always uses BGRA 32-bit with alpha channel even if we will discard the alpha channel as that is necessary to hit the fast paths in other parts of the platform. Note that while Cairo says CAIRO_FORMAT_ARGB32, it is really 32-bit BGRA on little-endian as we expect. OpenGL will render flipped (Quartz Native Co-ordinates) while Cairo renders with 0,O in the top-left. We could use cairo_translate() and cairo_scale() to reverse this, but it looks like some cairo things may not look quite as right if we do so. To reduce the chances of one-off bugs this continues to draw as Cairo would normally, but instead uses an CGAffineTransform in the tiles and some CGRect translation when swapping buffers to get the same effect. Changes to OpenGL ================= To simplify things, removal of all NSOpenGL* related components have been removed and we strictly use the Core GL (CGL*) API. This probably should have been done long ago anyay. Most examples found in the browsers to use IOSurfaceRef with OpenGL are using Legacy GL and there is still work underway to make this fit in with the rest of how the GSK GL renderer works. Since IOSurfaceRef bound to a texture/framebuffer will not have a default framebuffer ID of 0, we needed to add a default framebuffer id to the GdkGLContext. GskGLRenderer can use this to setup the command queue in such a way that our IOSurface destination has been glBindFramebuffer() as if it were the default drawable. This stuff is pretty slight-of-hand, so where things are and what needs flushing when and where has been a bit of an experiment to see what actually works to get synchronization across subsystems. Efficient Damages ================= After we draw with Cairo, we unlock the IOSurfaceRef and the contents are uploaded to the GPU. To make the contents visible to the app, we must clear the tiles contents with `layer.contents=nil;` and then re-apply the IOSurfaceRef. Since the buffer has likely not changed, we only do this if the tile overlaps the damage region. This gives the effect of having more tightly controlled damage regions even though updating the layer would damage be the whole window (as it is with OpenGL/Metal today with the exception of scissor-rect). This too can be verified usign "Quartz Debug -> Flash screen udpates". Frame Synchronized Resize ========================= In GTK 4, we have the ability to perform sizing changes from compute-size during the layout phase. Since the macOS backend already tracks window resizes manually, we can avoid doing the setFrame: immediately and instead do it within the frame clock's layout phase. Doing so gives us vastly better resize experience as we're more likely to get the size-change and updated-contents in the same frame on screen. It makes things feel "connected" in a way they weren't before. Some additional effort to tweak gravity during the process is also necessary but we were already doing that in the GTK 4 backend. Backporting =========== The design here has made an attempt to make it possible to backport by keeping GdkMacosBuffer, GdkMacosLayer, and GdkMacosTile fairly independent. There may be an opportunity to integrate this into GTK 3's quartz backend with a fair bit of work. Doing so could improve the situation for applications which are damage-rich such as The GIMP.
2022-02-14 10:20:19 +00:00
CGLContextObj existing;
macos: implement GL context This implements the basics for a GdkGLContext on macOS. Currently, rendering only is fully working for the GskCairoRenderer case where we read back pixels into a cairo surface for rendering. More work on synchronization is required for the GL on GskGLRenderer case. When we attempt to render a surface itself with GL, the context will ensure that the new GdkMacosGLView is placed within the NSWindow. In other cases, we use a dummy NSView and NSWindow for backing the NSOpenGLContext to ensure that we can get accelerated drawing. This gets GtkGLArea working when running with GSK_RENDERER=cairo.
2020-10-29 17:33:22 +00:00
GdkGLContext *shared;
GLint sync_to_framerate = 1;
GLint validate = 0;
macos: use CALayer and IOSurface for rendering This provides a major shift in how we draw both when accelerated OpenGL as well as software rendering with Cairo. In short, it uses tiles of Core Animation's CALayer to display contents from an OpenGL or Cairo rendering so that the window can provide partial damage updates. Partial damage is not generally available when using OpenGL as the whole buffer is flipped even if you only submitted a small change using a scissor rect. Thankfully, this speeds up Cairo rendering a bit too by using IOSurface to upload contents to the display server. We use the tiling system we do for OpenGL which reduces overall complexity and differences between them. A New Buffer ============ GdkMacosBuffer is a wrapper around an IOSurfaceRef. The term buffer was used because 1) surface is already used and 2) it loosely maps to a front/back buffer semantic. However, it appears that IOSurfaceRef contents are being retained in some fashion (likely in the compositor result) so we can update the same IOSurfaceRef without flipping as long as we're fast. This appears to be what Chromium does as well, but Firefox uses two IOSurfaceRef and flips between them. We would like to avoid two surfaces because it doubles the GPU VRAM requirements of the application. Changes to Windows ================== Previously, the NSWindow would dynamically change between different types of NSView based on the renderer being used. This is no longer necessary as we just have a single NSView type, GdkMacosView, which inherits from GdkMacosBaseView just to keep the tedius stuff separate from the machinery of GdkMacosView. We can merge those someday if we are okay with that. Changes to Views ================ GdkMacosCairoView, GdkMacosCairoSubView, GdkMacosGLView have all been removed and replaced with GdkMacosView. This new view has a single CALayer (GdkMacosLayer) attached to it which itself has sublayers. The contents of the CALayer is populated with an IOSurfaceRef which we allocated with the GdkMacosSurface. The surface is replaced when the NSWindow resizes. Changes to Layers ================= We now have a dedicated GdkMacosLayer which contains sublayers of GdkMacosTile. The tile has a maximum size of 128x128 pixels in device units. The GdkMacosTile is partitioned by splitting both the transparent region (window bounds minus opaque area) and then by splitting the opaque area. A tile has either translucent contents (and therefore is not opaque) or has opaque contents (and therefore is opaque). An opaque tile never contains transparent contents. As such, the opaque tiles contain a black background so that Core Animation will consider the tile's bounds as opaque. This can be verified with "Quartz Debug -> Show opaque regions". Changes to Cairo ================ GTK 4 cannot currently use cairo-quartz because of how CSS borders are rendered. It simply causes errors in the cairo_quartz_surface_t backend. Since we are restricted to using cairo_image_surface_t (which happens to be faster anyway) we can use the IOSurfaceBaseAddress() to obtain a mapping of the IOSurfaceRef in user-space. It always uses BGRA 32-bit with alpha channel even if we will discard the alpha channel as that is necessary to hit the fast paths in other parts of the platform. Note that while Cairo says CAIRO_FORMAT_ARGB32, it is really 32-bit BGRA on little-endian as we expect. OpenGL will render flipped (Quartz Native Co-ordinates) while Cairo renders with 0,O in the top-left. We could use cairo_translate() and cairo_scale() to reverse this, but it looks like some cairo things may not look quite as right if we do so. To reduce the chances of one-off bugs this continues to draw as Cairo would normally, but instead uses an CGAffineTransform in the tiles and some CGRect translation when swapping buffers to get the same effect. Changes to OpenGL ================= To simplify things, removal of all NSOpenGL* related components have been removed and we strictly use the Core GL (CGL*) API. This probably should have been done long ago anyay. Most examples found in the browsers to use IOSurfaceRef with OpenGL are using Legacy GL and there is still work underway to make this fit in with the rest of how the GSK GL renderer works. Since IOSurfaceRef bound to a texture/framebuffer will not have a default framebuffer ID of 0, we needed to add a default framebuffer id to the GdkGLContext. GskGLRenderer can use this to setup the command queue in such a way that our IOSurface destination has been glBindFramebuffer() as if it were the default drawable. This stuff is pretty slight-of-hand, so where things are and what needs flushing when and where has been a bit of an experiment to see what actually works to get synchronization across subsystems. Efficient Damages ================= After we draw with Cairo, we unlock the IOSurfaceRef and the contents are uploaded to the GPU. To make the contents visible to the app, we must clear the tiles contents with `layer.contents=nil;` and then re-apply the IOSurfaceRef. Since the buffer has likely not changed, we only do this if the tile overlaps the damage region. This gives the effect of having more tightly controlled damage regions even though updating the layer would damage be the whole window (as it is with OpenGL/Metal today with the exception of scissor-rect). This too can be verified usign "Quartz Debug -> Flash screen udpates". Frame Synchronized Resize ========================= In GTK 4, we have the ability to perform sizing changes from compute-size during the layout phase. Since the macOS backend already tracks window resizes manually, we can avoid doing the setFrame: immediately and instead do it within the frame clock's layout phase. Doing so gives us vastly better resize experience as we're more likely to get the size-change and updated-contents in the same frame on screen. It makes things feel "connected" in a way they weren't before. Some additional effort to tweak gravity during the process is also necessary but we were already doing that in the GTK 4 backend. Backporting =========== The design here has made an attempt to make it possible to backport by keeping GdkMacosBuffer, GdkMacosLayer, and GdkMacosTile fairly independent. There may be an opportunity to integrate this into GTK 3's quartz backend with a fair bit of work. Doing so could improve the situation for applications which are damage-rich such as The GIMP.
2022-02-14 10:20:19 +00:00
GLint renderer_id = 0;
macos: use CGLSetParameter and CGLEnable We don't need to go through the NSOpenGLContext for these. We can just use the C API directly. It's also clearer what is using CGLEnable() vs CGLSetParameter().
2020-12-07 19:47:51 +00:00
GLint swapRect[4];
macos: implement GL context This implements the basics for a GdkGLContext on macOS. Currently, rendering only is fully working for the GskCairoRenderer case where we read back pixels into a cairo surface for rendering. More work on synchronization is required for the GL on GskGLRenderer case. When we attempt to render a surface itself with GL, the context will ensure that the new GdkMacosGLView is placed within the NSWindow. In other cases, we use a dummy NSView and NSWindow for backing the NSOpenGLContext to ensure that we can get accelerated drawing. This gets GtkGLArea working when running with GSK_RENDERER=cairo.
2020-10-29 17:33:22 +00:00
int major, minor;
g_assert (GDK_IS_MACOS_GL_CONTEXT (self));
macos: use CALayer and IOSurface for rendering This provides a major shift in how we draw both when accelerated OpenGL as well as software rendering with Cairo. In short, it uses tiles of Core Animation's CALayer to display contents from an OpenGL or Cairo rendering so that the window can provide partial damage updates. Partial damage is not generally available when using OpenGL as the whole buffer is flipped even if you only submitted a small change using a scissor rect. Thankfully, this speeds up Cairo rendering a bit too by using IOSurface to upload contents to the display server. We use the tiling system we do for OpenGL which reduces overall complexity and differences between them. A New Buffer ============ GdkMacosBuffer is a wrapper around an IOSurfaceRef. The term buffer was used because 1) surface is already used and 2) it loosely maps to a front/back buffer semantic. However, it appears that IOSurfaceRef contents are being retained in some fashion (likely in the compositor result) so we can update the same IOSurfaceRef without flipping as long as we're fast. This appears to be what Chromium does as well, but Firefox uses two IOSurfaceRef and flips between them. We would like to avoid two surfaces because it doubles the GPU VRAM requirements of the application. Changes to Windows ================== Previously, the NSWindow would dynamically change between different types of NSView based on the renderer being used. This is no longer necessary as we just have a single NSView type, GdkMacosView, which inherits from GdkMacosBaseView just to keep the tedius stuff separate from the machinery of GdkMacosView. We can merge those someday if we are okay with that. Changes to Views ================ GdkMacosCairoView, GdkMacosCairoSubView, GdkMacosGLView have all been removed and replaced with GdkMacosView. This new view has a single CALayer (GdkMacosLayer) attached to it which itself has sublayers. The contents of the CALayer is populated with an IOSurfaceRef which we allocated with the GdkMacosSurface. The surface is replaced when the NSWindow resizes. Changes to Layers ================= We now have a dedicated GdkMacosLayer which contains sublayers of GdkMacosTile. The tile has a maximum size of 128x128 pixels in device units. The GdkMacosTile is partitioned by splitting both the transparent region (window bounds minus opaque area) and then by splitting the opaque area. A tile has either translucent contents (and therefore is not opaque) or has opaque contents (and therefore is opaque). An opaque tile never contains transparent contents. As such, the opaque tiles contain a black background so that Core Animation will consider the tile's bounds as opaque. This can be verified with "Quartz Debug -> Show opaque regions". Changes to Cairo ================ GTK 4 cannot currently use cairo-quartz because of how CSS borders are rendered. It simply causes errors in the cairo_quartz_surface_t backend. Since we are restricted to using cairo_image_surface_t (which happens to be faster anyway) we can use the IOSurfaceBaseAddress() to obtain a mapping of the IOSurfaceRef in user-space. It always uses BGRA 32-bit with alpha channel even if we will discard the alpha channel as that is necessary to hit the fast paths in other parts of the platform. Note that while Cairo says CAIRO_FORMAT_ARGB32, it is really 32-bit BGRA on little-endian as we expect. OpenGL will render flipped (Quartz Native Co-ordinates) while Cairo renders with 0,O in the top-left. We could use cairo_translate() and cairo_scale() to reverse this, but it looks like some cairo things may not look quite as right if we do so. To reduce the chances of one-off bugs this continues to draw as Cairo would normally, but instead uses an CGAffineTransform in the tiles and some CGRect translation when swapping buffers to get the same effect. Changes to OpenGL ================= To simplify things, removal of all NSOpenGL* related components have been removed and we strictly use the Core GL (CGL*) API. This probably should have been done long ago anyay. Most examples found in the browsers to use IOSurfaceRef with OpenGL are using Legacy GL and there is still work underway to make this fit in with the rest of how the GSK GL renderer works. Since IOSurfaceRef bound to a texture/framebuffer will not have a default framebuffer ID of 0, we needed to add a default framebuffer id to the GdkGLContext. GskGLRenderer can use this to setup the command queue in such a way that our IOSurface destination has been glBindFramebuffer() as if it were the default drawable. This stuff is pretty slight-of-hand, so where things are and what needs flushing when and where has been a bit of an experiment to see what actually works to get synchronization across subsystems. Efficient Damages ================= After we draw with Cairo, we unlock the IOSurfaceRef and the contents are uploaded to the GPU. To make the contents visible to the app, we must clear the tiles contents with `layer.contents=nil;` and then re-apply the IOSurfaceRef. Since the buffer has likely not changed, we only do this if the tile overlaps the damage region. This gives the effect of having more tightly controlled damage regions even though updating the layer would damage be the whole window (as it is with OpenGL/Metal today with the exception of scissor-rect). This too can be verified usign "Quartz Debug -> Flash screen udpates". Frame Synchronized Resize ========================= In GTK 4, we have the ability to perform sizing changes from compute-size during the layout phase. Since the macOS backend already tracks window resizes manually, we can avoid doing the setFrame: immediately and instead do it within the frame clock's layout phase. Doing so gives us vastly better resize experience as we're more likely to get the size-change and updated-contents in the same frame on screen. It makes things feel "connected" in a way they weren't before. Some additional effort to tweak gravity during the process is also necessary but we were already doing that in the GTK 4 backend. Backporting =========== The design here has made an attempt to make it possible to backport by keeping GdkMacosBuffer, GdkMacosLayer, and GdkMacosTile fairly independent. There may be an opportunity to integrate this into GTK 3's quartz backend with a fair bit of work. Doing so could improve the situation for applications which are damage-rich such as The GIMP.
2022-02-14 10:20:19 +00:00
if (self->cgl_context != nil)
gl: Check allowed APIs in realize() Add gdk_gl_context_is_api_allowed() for backends and make them use it. Finally, have them return the final API as the return value (or 0 on error). And then use that api instead of a use_es boolean flag. Fixes #4221
2021-10-07 20:10:39 +00:00
return GDK_GL_API_GL;
if (!gdk_gl_context_is_api_allowed (context, GDK_GL_API_GL, error))
return 0;
macos: implement GL context This implements the basics for a GdkGLContext on macOS. Currently, rendering only is fully working for the GskCairoRenderer case where we read back pixels into a cairo surface for rendering. More work on synchronization is required for the GL on GskGLRenderer case. When we attempt to render a surface itself with GL, the context will ensure that the new GdkMacosGLView is placed within the NSWindow. In other cases, we use a dummy NSView and NSWindow for backing the NSOpenGLContext to ensure that we can get accelerated drawing. This gets GtkGLArea working when running with GSK_RENDERER=cairo.
2020-10-29 17:33:22 +00:00
macos: use CALayer and IOSurface for rendering This provides a major shift in how we draw both when accelerated OpenGL as well as software rendering with Cairo. In short, it uses tiles of Core Animation's CALayer to display contents from an OpenGL or Cairo rendering so that the window can provide partial damage updates. Partial damage is not generally available when using OpenGL as the whole buffer is flipped even if you only submitted a small change using a scissor rect. Thankfully, this speeds up Cairo rendering a bit too by using IOSurface to upload contents to the display server. We use the tiling system we do for OpenGL which reduces overall complexity and differences between them. A New Buffer ============ GdkMacosBuffer is a wrapper around an IOSurfaceRef. The term buffer was used because 1) surface is already used and 2) it loosely maps to a front/back buffer semantic. However, it appears that IOSurfaceRef contents are being retained in some fashion (likely in the compositor result) so we can update the same IOSurfaceRef without flipping as long as we're fast. This appears to be what Chromium does as well, but Firefox uses two IOSurfaceRef and flips between them. We would like to avoid two surfaces because it doubles the GPU VRAM requirements of the application. Changes to Windows ================== Previously, the NSWindow would dynamically change between different types of NSView based on the renderer being used. This is no longer necessary as we just have a single NSView type, GdkMacosView, which inherits from GdkMacosBaseView just to keep the tedius stuff separate from the machinery of GdkMacosView. We can merge those someday if we are okay with that. Changes to Views ================ GdkMacosCairoView, GdkMacosCairoSubView, GdkMacosGLView have all been removed and replaced with GdkMacosView. This new view has a single CALayer (GdkMacosLayer) attached to it which itself has sublayers. The contents of the CALayer is populated with an IOSurfaceRef which we allocated with the GdkMacosSurface. The surface is replaced when the NSWindow resizes. Changes to Layers ================= We now have a dedicated GdkMacosLayer which contains sublayers of GdkMacosTile. The tile has a maximum size of 128x128 pixels in device units. The GdkMacosTile is partitioned by splitting both the transparent region (window bounds minus opaque area) and then by splitting the opaque area. A tile has either translucent contents (and therefore is not opaque) or has opaque contents (and therefore is opaque). An opaque tile never contains transparent contents. As such, the opaque tiles contain a black background so that Core Animation will consider the tile's bounds as opaque. This can be verified with "Quartz Debug -> Show opaque regions". Changes to Cairo ================ GTK 4 cannot currently use cairo-quartz because of how CSS borders are rendered. It simply causes errors in the cairo_quartz_surface_t backend. Since we are restricted to using cairo_image_surface_t (which happens to be faster anyway) we can use the IOSurfaceBaseAddress() to obtain a mapping of the IOSurfaceRef in user-space. It always uses BGRA 32-bit with alpha channel even if we will discard the alpha channel as that is necessary to hit the fast paths in other parts of the platform. Note that while Cairo says CAIRO_FORMAT_ARGB32, it is really 32-bit BGRA on little-endian as we expect. OpenGL will render flipped (Quartz Native Co-ordinates) while Cairo renders with 0,O in the top-left. We could use cairo_translate() and cairo_scale() to reverse this, but it looks like some cairo things may not look quite as right if we do so. To reduce the chances of one-off bugs this continues to draw as Cairo would normally, but instead uses an CGAffineTransform in the tiles and some CGRect translation when swapping buffers to get the same effect. Changes to OpenGL ================= To simplify things, removal of all NSOpenGL* related components have been removed and we strictly use the Core GL (CGL*) API. This probably should have been done long ago anyay. Most examples found in the browsers to use IOSurfaceRef with OpenGL are using Legacy GL and there is still work underway to make this fit in with the rest of how the GSK GL renderer works. Since IOSurfaceRef bound to a texture/framebuffer will not have a default framebuffer ID of 0, we needed to add a default framebuffer id to the GdkGLContext. GskGLRenderer can use this to setup the command queue in such a way that our IOSurface destination has been glBindFramebuffer() as if it were the default drawable. This stuff is pretty slight-of-hand, so where things are and what needs flushing when and where has been a bit of an experiment to see what actually works to get synchronization across subsystems. Efficient Damages ================= After we draw with Cairo, we unlock the IOSurfaceRef and the contents are uploaded to the GPU. To make the contents visible to the app, we must clear the tiles contents with `layer.contents=nil;` and then re-apply the IOSurfaceRef. Since the buffer has likely not changed, we only do this if the tile overlaps the damage region. This gives the effect of having more tightly controlled damage regions even though updating the layer would damage be the whole window (as it is with OpenGL/Metal today with the exception of scissor-rect). This too can be verified usign "Quartz Debug -> Flash screen udpates". Frame Synchronized Resize ========================= In GTK 4, we have the ability to perform sizing changes from compute-size during the layout phase. Since the macOS backend already tracks window resizes manually, we can avoid doing the setFrame: immediately and instead do it within the frame clock's layout phase. Doing so gives us vastly better resize experience as we're more likely to get the size-change and updated-contents in the same frame on screen. It makes things feel "connected" in a way they weren't before. Some additional effort to tweak gravity during the process is also necessary but we were already doing that in the GTK 4 backend. Backporting =========== The design here has made an attempt to make it possible to backport by keeping GdkMacosBuffer, GdkMacosLayer, and GdkMacosTile fairly independent. There may be an opportunity to integrate this into GTK 3's quartz backend with a fair bit of work. Doing so could improve the situation for applications which are damage-rich such as The GIMP.
2022-02-14 10:20:19 +00:00
existing = CGLGetCurrentContext ();
macos: implement GL context This implements the basics for a GdkGLContext on macOS. Currently, rendering only is fully working for the GskCairoRenderer case where we read back pixels into a cairo surface for rendering. More work on synchronization is required for the GL on GskGLRenderer case. When we attempt to render a surface itself with GL, the context will ensure that the new GdkMacosGLView is placed within the NSWindow. In other cases, we use a dummy NSView and NSWindow for backing the NSOpenGLContext to ensure that we can get accelerated drawing. This gets GtkGLArea working when running with GSK_RENDERER=cairo.
2020-10-29 17:33:22 +00:00
gdk_gl_context_get_required_version (context, &major, &minor);
gdk: Add GdkDisplay::init_gl vfunc The vfunc is called to initialize GL and it returns a "base" context that GDK then uses as the context all others are shared with. So the GL context share tree now looks like: + context from init_gl - context1 - context2 ... So this is a flat tree now, the complexity is gone. The only caveat is that backends now need to create a GL context when initializing GL so some refactoring was needed. Two new functions have been added: * gdk_display_prepare_gl() This is public API and can be used to ensure that GL has been initialized or if not, retrieve an error to display (or debug-print). * gdk_display_get_gl_context() This is a private function to retrieve the base context from init_gl(). It replaces gdk_surface_get_shared_data_context().
2021-07-04 23:57:03 +00:00
display = gdk_gl_context_get_display (context);
shared = gdk_display_get_gl_context (display);
macos: implement GL context This implements the basics for a GdkGLContext on macOS. Currently, rendering only is fully working for the GskCairoRenderer case where we read back pixels into a cairo surface for rendering. More work on synchronization is required for the GL on GskGLRenderer case. When we attempt to render a surface itself with GL, the context will ensure that the new GdkMacosGLView is placed within the NSWindow. In other cases, we use a dummy NSView and NSWindow for backing the NSOpenGLContext to ensure that we can get accelerated drawing. This gets GtkGLArea working when running with GSK_RENDERER=cairo.
2020-10-29 17:33:22 +00:00
if (shared != NULL)
{
macos: use CALayer and IOSurface for rendering This provides a major shift in how we draw both when accelerated OpenGL as well as software rendering with Cairo. In short, it uses tiles of Core Animation's CALayer to display contents from an OpenGL or Cairo rendering so that the window can provide partial damage updates. Partial damage is not generally available when using OpenGL as the whole buffer is flipped even if you only submitted a small change using a scissor rect. Thankfully, this speeds up Cairo rendering a bit too by using IOSurface to upload contents to the display server. We use the tiling system we do for OpenGL which reduces overall complexity and differences between them. A New Buffer ============ GdkMacosBuffer is a wrapper around an IOSurfaceRef. The term buffer was used because 1) surface is already used and 2) it loosely maps to a front/back buffer semantic. However, it appears that IOSurfaceRef contents are being retained in some fashion (likely in the compositor result) so we can update the same IOSurfaceRef without flipping as long as we're fast. This appears to be what Chromium does as well, but Firefox uses two IOSurfaceRef and flips between them. We would like to avoid two surfaces because it doubles the GPU VRAM requirements of the application. Changes to Windows ================== Previously, the NSWindow would dynamically change between different types of NSView based on the renderer being used. This is no longer necessary as we just have a single NSView type, GdkMacosView, which inherits from GdkMacosBaseView just to keep the tedius stuff separate from the machinery of GdkMacosView. We can merge those someday if we are okay with that. Changes to Views ================ GdkMacosCairoView, GdkMacosCairoSubView, GdkMacosGLView have all been removed and replaced with GdkMacosView. This new view has a single CALayer (GdkMacosLayer) attached to it which itself has sublayers. The contents of the CALayer is populated with an IOSurfaceRef which we allocated with the GdkMacosSurface. The surface is replaced when the NSWindow resizes. Changes to Layers ================= We now have a dedicated GdkMacosLayer which contains sublayers of GdkMacosTile. The tile has a maximum size of 128x128 pixels in device units. The GdkMacosTile is partitioned by splitting both the transparent region (window bounds minus opaque area) and then by splitting the opaque area. A tile has either translucent contents (and therefore is not opaque) or has opaque contents (and therefore is opaque). An opaque tile never contains transparent contents. As such, the opaque tiles contain a black background so that Core Animation will consider the tile's bounds as opaque. This can be verified with "Quartz Debug -> Show opaque regions". Changes to Cairo ================ GTK 4 cannot currently use cairo-quartz because of how CSS borders are rendered. It simply causes errors in the cairo_quartz_surface_t backend. Since we are restricted to using cairo_image_surface_t (which happens to be faster anyway) we can use the IOSurfaceBaseAddress() to obtain a mapping of the IOSurfaceRef in user-space. It always uses BGRA 32-bit with alpha channel even if we will discard the alpha channel as that is necessary to hit the fast paths in other parts of the platform. Note that while Cairo says CAIRO_FORMAT_ARGB32, it is really 32-bit BGRA on little-endian as we expect. OpenGL will render flipped (Quartz Native Co-ordinates) while Cairo renders with 0,O in the top-left. We could use cairo_translate() and cairo_scale() to reverse this, but it looks like some cairo things may not look quite as right if we do so. To reduce the chances of one-off bugs this continues to draw as Cairo would normally, but instead uses an CGAffineTransform in the tiles and some CGRect translation when swapping buffers to get the same effect. Changes to OpenGL ================= To simplify things, removal of all NSOpenGL* related components have been removed and we strictly use the Core GL (CGL*) API. This probably should have been done long ago anyay. Most examples found in the browsers to use IOSurfaceRef with OpenGL are using Legacy GL and there is still work underway to make this fit in with the rest of how the GSK GL renderer works. Since IOSurfaceRef bound to a texture/framebuffer will not have a default framebuffer ID of 0, we needed to add a default framebuffer id to the GdkGLContext. GskGLRenderer can use this to setup the command queue in such a way that our IOSurface destination has been glBindFramebuffer() as if it were the default drawable. This stuff is pretty slight-of-hand, so where things are and what needs flushing when and where has been a bit of an experiment to see what actually works to get synchronization across subsystems. Efficient Damages ================= After we draw with Cairo, we unlock the IOSurfaceRef and the contents are uploaded to the GPU. To make the contents visible to the app, we must clear the tiles contents with `layer.contents=nil;` and then re-apply the IOSurfaceRef. Since the buffer has likely not changed, we only do this if the tile overlaps the damage region. This gives the effect of having more tightly controlled damage regions even though updating the layer would damage be the whole window (as it is with OpenGL/Metal today with the exception of scissor-rect). This too can be verified usign "Quartz Debug -> Flash screen udpates". Frame Synchronized Resize ========================= In GTK 4, we have the ability to perform sizing changes from compute-size during the layout phase. Since the macOS backend already tracks window resizes manually, we can avoid doing the setFrame: immediately and instead do it within the frame clock's layout phase. Doing so gives us vastly better resize experience as we're more likely to get the size-change and updated-contents in the same frame on screen. It makes things feel "connected" in a way they weren't before. Some additional effort to tweak gravity during the process is also necessary but we were already doing that in the GTK 4 backend. Backporting =========== The design here has made an attempt to make it possible to backport by keeping GdkMacosBuffer, GdkMacosLayer, and GdkMacosTile fairly independent. There may be an opportunity to integrate this into GTK 3's quartz backend with a fair bit of work. Doing so could improve the situation for applications which are damage-rich such as The GIMP.
2022-02-14 10:20:19 +00:00
if (!(shared_gl_context = GDK_MACOS_GL_CONTEXT (shared)->cgl_context))
{
g_set_error_literal (error,
GDK_GL_ERROR,
GDK_GL_ERROR_NOT_AVAILABLE,
"Cannot access shared CGLContextObj");
return 0;
}
macos: implement GL context This implements the basics for a GdkGLContext on macOS. Currently, rendering only is fully working for the GskCairoRenderer case where we read back pixels into a cairo surface for rendering. More work on synchronization is required for the GL on GskGLRenderer case. When we attempt to render a surface itself with GL, the context will ensure that the new GdkMacosGLView is placed within the NSWindow. In other cases, we use a dummy NSView and NSWindow for backing the NSOpenGLContext to ensure that we can get accelerated drawing. This gets GtkGLArea working when running with GSK_RENDERER=cairo.
2020-10-29 17:33:22 +00:00
}
gdk: Add GdkDisplay::init_gl vfunc The vfunc is called to initialize GL and it returns a "base" context that GDK then uses as the context all others are shared with. So the GL context share tree now looks like: + context from init_gl - context1 - context2 ... So this is a flat tree now, the complexity is gone. The only caveat is that backends now need to create a GL context when initializing GL so some refactoring was needed. Two new functions have been added: * gdk_display_prepare_gl() This is public API and can be used to ensure that GL has been initialized or if not, retrieve an error to display (or debug-print). * gdk_display_get_gl_context() This is a private function to retrieve the base context from init_gl(). It replaces gdk_surface_get_shared_data_context().
2021-07-04 23:57:03 +00:00
GDK_DISPLAY_NOTE (display,
macos: implement GL context This implements the basics for a GdkGLContext on macOS. Currently, rendering only is fully working for the GskCairoRenderer case where we read back pixels into a cairo surface for rendering. More work on synchronization is required for the GL on GskGLRenderer case. When we attempt to render a surface itself with GL, the context will ensure that the new GdkMacosGLView is placed within the NSWindow. In other cases, we use a dummy NSView and NSWindow for backing the NSOpenGLContext to ensure that we can get accelerated drawing. This gets GtkGLArea working when running with GSK_RENDERER=cairo.
2020-10-29 17:33:22 +00:00
OPENGL,
macos: use CALayer and IOSurface for rendering This provides a major shift in how we draw both when accelerated OpenGL as well as software rendering with Cairo. In short, it uses tiles of Core Animation's CALayer to display contents from an OpenGL or Cairo rendering so that the window can provide partial damage updates. Partial damage is not generally available when using OpenGL as the whole buffer is flipped even if you only submitted a small change using a scissor rect. Thankfully, this speeds up Cairo rendering a bit too by using IOSurface to upload contents to the display server. We use the tiling system we do for OpenGL which reduces overall complexity and differences between them. A New Buffer ============ GdkMacosBuffer is a wrapper around an IOSurfaceRef. The term buffer was used because 1) surface is already used and 2) it loosely maps to a front/back buffer semantic. However, it appears that IOSurfaceRef contents are being retained in some fashion (likely in the compositor result) so we can update the same IOSurfaceRef without flipping as long as we're fast. This appears to be what Chromium does as well, but Firefox uses two IOSurfaceRef and flips between them. We would like to avoid two surfaces because it doubles the GPU VRAM requirements of the application. Changes to Windows ================== Previously, the NSWindow would dynamically change between different types of NSView based on the renderer being used. This is no longer necessary as we just have a single NSView type, GdkMacosView, which inherits from GdkMacosBaseView just to keep the tedius stuff separate from the machinery of GdkMacosView. We can merge those someday if we are okay with that. Changes to Views ================ GdkMacosCairoView, GdkMacosCairoSubView, GdkMacosGLView have all been removed and replaced with GdkMacosView. This new view has a single CALayer (GdkMacosLayer) attached to it which itself has sublayers. The contents of the CALayer is populated with an IOSurfaceRef which we allocated with the GdkMacosSurface. The surface is replaced when the NSWindow resizes. Changes to Layers ================= We now have a dedicated GdkMacosLayer which contains sublayers of GdkMacosTile. The tile has a maximum size of 128x128 pixels in device units. The GdkMacosTile is partitioned by splitting both the transparent region (window bounds minus opaque area) and then by splitting the opaque area. A tile has either translucent contents (and therefore is not opaque) or has opaque contents (and therefore is opaque). An opaque tile never contains transparent contents. As such, the opaque tiles contain a black background so that Core Animation will consider the tile's bounds as opaque. This can be verified with "Quartz Debug -> Show opaque regions". Changes to Cairo ================ GTK 4 cannot currently use cairo-quartz because of how CSS borders are rendered. It simply causes errors in the cairo_quartz_surface_t backend. Since we are restricted to using cairo_image_surface_t (which happens to be faster anyway) we can use the IOSurfaceBaseAddress() to obtain a mapping of the IOSurfaceRef in user-space. It always uses BGRA 32-bit with alpha channel even if we will discard the alpha channel as that is necessary to hit the fast paths in other parts of the platform. Note that while Cairo says CAIRO_FORMAT_ARGB32, it is really 32-bit BGRA on little-endian as we expect. OpenGL will render flipped (Quartz Native Co-ordinates) while Cairo renders with 0,O in the top-left. We could use cairo_translate() and cairo_scale() to reverse this, but it looks like some cairo things may not look quite as right if we do so. To reduce the chances of one-off bugs this continues to draw as Cairo would normally, but instead uses an CGAffineTransform in the tiles and some CGRect translation when swapping buffers to get the same effect. Changes to OpenGL ================= To simplify things, removal of all NSOpenGL* related components have been removed and we strictly use the Core GL (CGL*) API. This probably should have been done long ago anyay. Most examples found in the browsers to use IOSurfaceRef with OpenGL are using Legacy GL and there is still work underway to make this fit in with the rest of how the GSK GL renderer works. Since IOSurfaceRef bound to a texture/framebuffer will not have a default framebuffer ID of 0, we needed to add a default framebuffer id to the GdkGLContext. GskGLRenderer can use this to setup the command queue in such a way that our IOSurface destination has been glBindFramebuffer() as if it were the default drawable. This stuff is pretty slight-of-hand, so where things are and what needs flushing when and where has been a bit of an experiment to see what actually works to get synchronization across subsystems. Efficient Damages ================= After we draw with Cairo, we unlock the IOSurfaceRef and the contents are uploaded to the GPU. To make the contents visible to the app, we must clear the tiles contents with `layer.contents=nil;` and then re-apply the IOSurfaceRef. Since the buffer has likely not changed, we only do this if the tile overlaps the damage region. This gives the effect of having more tightly controlled damage regions even though updating the layer would damage be the whole window (as it is with OpenGL/Metal today with the exception of scissor-rect). This too can be verified usign "Quartz Debug -> Flash screen udpates". Frame Synchronized Resize ========================= In GTK 4, we have the ability to perform sizing changes from compute-size during the layout phase. Since the macOS backend already tracks window resizes manually, we can avoid doing the setFrame: immediately and instead do it within the frame clock's layout phase. Doing so gives us vastly better resize experience as we're more likely to get the size-change and updated-contents in the same frame on screen. It makes things feel "connected" in a way they weren't before. Some additional effort to tweak gravity during the process is also necessary but we were already doing that in the GTK 4 backend. Backporting =========== The design here has made an attempt to make it possible to backport by keeping GdkMacosBuffer, GdkMacosLayer, and GdkMacosTile fairly independent. There may be an opportunity to integrate this into GTK 3's quartz backend with a fair bit of work. Doing so could improve the situation for applications which are damage-rich such as The GIMP.
2022-02-14 10:20:19 +00:00
g_message ("Creating CGLContextObj (version %d.%d)",
macos: implement GL context This implements the basics for a GdkGLContext on macOS. Currently, rendering only is fully working for the GskCairoRenderer case where we read back pixels into a cairo surface for rendering. More work on synchronization is required for the GL on GskGLRenderer case. When we attempt to render a surface itself with GL, the context will ensure that the new GdkMacosGLView is placed within the NSWindow. In other cases, we use a dummy NSView and NSWindow for backing the NSOpenGLContext to ensure that we can get accelerated drawing. This gets GtkGLArea working when running with GSK_RENDERER=cairo.
2020-10-29 17:33:22 +00:00
major, minor));
if (!(pixelFormat = create_pixel_format (major, minor, error)))
gl: Check allowed APIs in realize() Add gdk_gl_context_is_api_allowed() for backends and make them use it. Finally, have them return the final API as the return value (or 0 on error). And then use that api instead of a use_es boolean flag. Fixes #4221
2021-10-07 20:10:39 +00:00
return 0;
macos: implement GL context This implements the basics for a GdkGLContext on macOS. Currently, rendering only is fully working for the GskCairoRenderer case where we read back pixels into a cairo surface for rendering. More work on synchronization is required for the GL on GskGLRenderer case. When we attempt to render a surface itself with GL, the context will ensure that the new GdkMacosGLView is placed within the NSWindow. In other cases, we use a dummy NSView and NSWindow for backing the NSOpenGLContext to ensure that we can get accelerated drawing. This gets GtkGLArea working when running with GSK_RENDERER=cairo.
2020-10-29 17:33:22 +00:00
macos: use CALayer and IOSurface for rendering This provides a major shift in how we draw both when accelerated OpenGL as well as software rendering with Cairo. In short, it uses tiles of Core Animation's CALayer to display contents from an OpenGL or Cairo rendering so that the window can provide partial damage updates. Partial damage is not generally available when using OpenGL as the whole buffer is flipped even if you only submitted a small change using a scissor rect. Thankfully, this speeds up Cairo rendering a bit too by using IOSurface to upload contents to the display server. We use the tiling system we do for OpenGL which reduces overall complexity and differences between them. A New Buffer ============ GdkMacosBuffer is a wrapper around an IOSurfaceRef. The term buffer was used because 1) surface is already used and 2) it loosely maps to a front/back buffer semantic. However, it appears that IOSurfaceRef contents are being retained in some fashion (likely in the compositor result) so we can update the same IOSurfaceRef without flipping as long as we're fast. This appears to be what Chromium does as well, but Firefox uses two IOSurfaceRef and flips between them. We would like to avoid two surfaces because it doubles the GPU VRAM requirements of the application. Changes to Windows ================== Previously, the NSWindow would dynamically change between different types of NSView based on the renderer being used. This is no longer necessary as we just have a single NSView type, GdkMacosView, which inherits from GdkMacosBaseView just to keep the tedius stuff separate from the machinery of GdkMacosView. We can merge those someday if we are okay with that. Changes to Views ================ GdkMacosCairoView, GdkMacosCairoSubView, GdkMacosGLView have all been removed and replaced with GdkMacosView. This new view has a single CALayer (GdkMacosLayer) attached to it which itself has sublayers. The contents of the CALayer is populated with an IOSurfaceRef which we allocated with the GdkMacosSurface. The surface is replaced when the NSWindow resizes. Changes to Layers ================= We now have a dedicated GdkMacosLayer which contains sublayers of GdkMacosTile. The tile has a maximum size of 128x128 pixels in device units. The GdkMacosTile is partitioned by splitting both the transparent region (window bounds minus opaque area) and then by splitting the opaque area. A tile has either translucent contents (and therefore is not opaque) or has opaque contents (and therefore is opaque). An opaque tile never contains transparent contents. As such, the opaque tiles contain a black background so that Core Animation will consider the tile's bounds as opaque. This can be verified with "Quartz Debug -> Show opaque regions". Changes to Cairo ================ GTK 4 cannot currently use cairo-quartz because of how CSS borders are rendered. It simply causes errors in the cairo_quartz_surface_t backend. Since we are restricted to using cairo_image_surface_t (which happens to be faster anyway) we can use the IOSurfaceBaseAddress() to obtain a mapping of the IOSurfaceRef in user-space. It always uses BGRA 32-bit with alpha channel even if we will discard the alpha channel as that is necessary to hit the fast paths in other parts of the platform. Note that while Cairo says CAIRO_FORMAT_ARGB32, it is really 32-bit BGRA on little-endian as we expect. OpenGL will render flipped (Quartz Native Co-ordinates) while Cairo renders with 0,O in the top-left. We could use cairo_translate() and cairo_scale() to reverse this, but it looks like some cairo things may not look quite as right if we do so. To reduce the chances of one-off bugs this continues to draw as Cairo would normally, but instead uses an CGAffineTransform in the tiles and some CGRect translation when swapping buffers to get the same effect. Changes to OpenGL ================= To simplify things, removal of all NSOpenGL* related components have been removed and we strictly use the Core GL (CGL*) API. This probably should have been done long ago anyay. Most examples found in the browsers to use IOSurfaceRef with OpenGL are using Legacy GL and there is still work underway to make this fit in with the rest of how the GSK GL renderer works. Since IOSurfaceRef bound to a texture/framebuffer will not have a default framebuffer ID of 0, we needed to add a default framebuffer id to the GdkGLContext. GskGLRenderer can use this to setup the command queue in such a way that our IOSurface destination has been glBindFramebuffer() as if it were the default drawable. This stuff is pretty slight-of-hand, so where things are and what needs flushing when and where has been a bit of an experiment to see what actually works to get synchronization across subsystems. Efficient Damages ================= After we draw with Cairo, we unlock the IOSurfaceRef and the contents are uploaded to the GPU. To make the contents visible to the app, we must clear the tiles contents with `layer.contents=nil;` and then re-apply the IOSurfaceRef. Since the buffer has likely not changed, we only do this if the tile overlaps the damage region. This gives the effect of having more tightly controlled damage regions even though updating the layer would damage be the whole window (as it is with OpenGL/Metal today with the exception of scissor-rect). This too can be verified usign "Quartz Debug -> Flash screen udpates". Frame Synchronized Resize ========================= In GTK 4, we have the ability to perform sizing changes from compute-size during the layout phase. Since the macOS backend already tracks window resizes manually, we can avoid doing the setFrame: immediately and instead do it within the frame clock's layout phase. Doing so gives us vastly better resize experience as we're more likely to get the size-change and updated-contents in the same frame on screen. It makes things feel "connected" in a way they weren't before. Some additional effort to tweak gravity during the process is also necessary but we were already doing that in the GTK 4 backend. Backporting =========== The design here has made an attempt to make it possible to backport by keeping GdkMacosBuffer, GdkMacosLayer, and GdkMacosTile fairly independent. There may be an opportunity to integrate this into GTK 3's quartz backend with a fair bit of work. Doing so could improve the situation for applications which are damage-rich such as The GIMP.
2022-02-14 10:20:19 +00:00
if (!CHECK (error, CGLCreateContext (pixelFormat, shared_gl_context, &cgl_context)))
macos: implement GL context This implements the basics for a GdkGLContext on macOS. Currently, rendering only is fully working for the GskCairoRenderer case where we read back pixels into a cairo surface for rendering. More work on synchronization is required for the GL on GskGLRenderer case. When we attempt to render a surface itself with GL, the context will ensure that the new GdkMacosGLView is placed within the NSWindow. In other cases, we use a dummy NSView and NSWindow for backing the NSOpenGLContext to ensure that we can get accelerated drawing. This gets GtkGLArea working when running with GSK_RENDERER=cairo.
2020-10-29 17:33:22 +00:00
{
macos: use CALayer and IOSurface for rendering This provides a major shift in how we draw both when accelerated OpenGL as well as software rendering with Cairo. In short, it uses tiles of Core Animation's CALayer to display contents from an OpenGL or Cairo rendering so that the window can provide partial damage updates. Partial damage is not generally available when using OpenGL as the whole buffer is flipped even if you only submitted a small change using a scissor rect. Thankfully, this speeds up Cairo rendering a bit too by using IOSurface to upload contents to the display server. We use the tiling system we do for OpenGL which reduces overall complexity and differences between them. A New Buffer ============ GdkMacosBuffer is a wrapper around an IOSurfaceRef. The term buffer was used because 1) surface is already used and 2) it loosely maps to a front/back buffer semantic. However, it appears that IOSurfaceRef contents are being retained in some fashion (likely in the compositor result) so we can update the same IOSurfaceRef without flipping as long as we're fast. This appears to be what Chromium does as well, but Firefox uses two IOSurfaceRef and flips between them. We would like to avoid two surfaces because it doubles the GPU VRAM requirements of the application. Changes to Windows ================== Previously, the NSWindow would dynamically change between different types of NSView based on the renderer being used. This is no longer necessary as we just have a single NSView type, GdkMacosView, which inherits from GdkMacosBaseView just to keep the tedius stuff separate from the machinery of GdkMacosView. We can merge those someday if we are okay with that. Changes to Views ================ GdkMacosCairoView, GdkMacosCairoSubView, GdkMacosGLView have all been removed and replaced with GdkMacosView. This new view has a single CALayer (GdkMacosLayer) attached to it which itself has sublayers. The contents of the CALayer is populated with an IOSurfaceRef which we allocated with the GdkMacosSurface. The surface is replaced when the NSWindow resizes. Changes to Layers ================= We now have a dedicated GdkMacosLayer which contains sublayers of GdkMacosTile. The tile has a maximum size of 128x128 pixels in device units. The GdkMacosTile is partitioned by splitting both the transparent region (window bounds minus opaque area) and then by splitting the opaque area. A tile has either translucent contents (and therefore is not opaque) or has opaque contents (and therefore is opaque). An opaque tile never contains transparent contents. As such, the opaque tiles contain a black background so that Core Animation will consider the tile's bounds as opaque. This can be verified with "Quartz Debug -> Show opaque regions". Changes to Cairo ================ GTK 4 cannot currently use cairo-quartz because of how CSS borders are rendered. It simply causes errors in the cairo_quartz_surface_t backend. Since we are restricted to using cairo_image_surface_t (which happens to be faster anyway) we can use the IOSurfaceBaseAddress() to obtain a mapping of the IOSurfaceRef in user-space. It always uses BGRA 32-bit with alpha channel even if we will discard the alpha channel as that is necessary to hit the fast paths in other parts of the platform. Note that while Cairo says CAIRO_FORMAT_ARGB32, it is really 32-bit BGRA on little-endian as we expect. OpenGL will render flipped (Quartz Native Co-ordinates) while Cairo renders with 0,O in the top-left. We could use cairo_translate() and cairo_scale() to reverse this, but it looks like some cairo things may not look quite as right if we do so. To reduce the chances of one-off bugs this continues to draw as Cairo would normally, but instead uses an CGAffineTransform in the tiles and some CGRect translation when swapping buffers to get the same effect. Changes to OpenGL ================= To simplify things, removal of all NSOpenGL* related components have been removed and we strictly use the Core GL (CGL*) API. This probably should have been done long ago anyay. Most examples found in the browsers to use IOSurfaceRef with OpenGL are using Legacy GL and there is still work underway to make this fit in with the rest of how the GSK GL renderer works. Since IOSurfaceRef bound to a texture/framebuffer will not have a default framebuffer ID of 0, we needed to add a default framebuffer id to the GdkGLContext. GskGLRenderer can use this to setup the command queue in such a way that our IOSurface destination has been glBindFramebuffer() as if it were the default drawable. This stuff is pretty slight-of-hand, so where things are and what needs flushing when and where has been a bit of an experiment to see what actually works to get synchronization across subsystems. Efficient Damages ================= After we draw with Cairo, we unlock the IOSurfaceRef and the contents are uploaded to the GPU. To make the contents visible to the app, we must clear the tiles contents with `layer.contents=nil;` and then re-apply the IOSurfaceRef. Since the buffer has likely not changed, we only do this if the tile overlaps the damage region. This gives the effect of having more tightly controlled damage regions even though updating the layer would damage be the whole window (as it is with OpenGL/Metal today with the exception of scissor-rect). This too can be verified usign "Quartz Debug -> Flash screen udpates". Frame Synchronized Resize ========================= In GTK 4, we have the ability to perform sizing changes from compute-size during the layout phase. Since the macOS backend already tracks window resizes manually, we can avoid doing the setFrame: immediately and instead do it within the frame clock's layout phase. Doing so gives us vastly better resize experience as we're more likely to get the size-change and updated-contents in the same frame on screen. It makes things feel "connected" in a way they weren't before. Some additional effort to tweak gravity during the process is also necessary but we were already doing that in the GTK 4 backend. Backporting =========== The design here has made an attempt to make it possible to backport by keeping GdkMacosBuffer, GdkMacosLayer, and GdkMacosTile fairly independent. There may be an opportunity to integrate this into GTK 3's quartz backend with a fair bit of work. Doing so could improve the situation for applications which are damage-rich such as The GIMP.
2022-02-14 10:20:19 +00:00
CGLReleasePixelFormat (pixelFormat);
gl: Check allowed APIs in realize() Add gdk_gl_context_is_api_allowed() for backends and make them use it. Finally, have them return the final API as the return value (or 0 on error). And then use that api instead of a use_es boolean flag. Fixes #4221
2021-10-07 20:10:39 +00:00
return 0;
macos: implement GL context This implements the basics for a GdkGLContext on macOS. Currently, rendering only is fully working for the GskCairoRenderer case where we read back pixels into a cairo surface for rendering. More work on synchronization is required for the GL on GskGLRenderer case. When we attempt to render a surface itself with GL, the context will ensure that the new GdkMacosGLView is placed within the NSWindow. In other cases, we use a dummy NSView and NSWindow for backing the NSOpenGLContext to ensure that we can get accelerated drawing. This gets GtkGLArea working when running with GSK_RENDERER=cairo.
2020-10-29 17:33:22 +00:00
}
macos: use CALayer and IOSurface for rendering This provides a major shift in how we draw both when accelerated OpenGL as well as software rendering with Cairo. In short, it uses tiles of Core Animation's CALayer to display contents from an OpenGL or Cairo rendering so that the window can provide partial damage updates. Partial damage is not generally available when using OpenGL as the whole buffer is flipped even if you only submitted a small change using a scissor rect. Thankfully, this speeds up Cairo rendering a bit too by using IOSurface to upload contents to the display server. We use the tiling system we do for OpenGL which reduces overall complexity and differences between them. A New Buffer ============ GdkMacosBuffer is a wrapper around an IOSurfaceRef. The term buffer was used because 1) surface is already used and 2) it loosely maps to a front/back buffer semantic. However, it appears that IOSurfaceRef contents are being retained in some fashion (likely in the compositor result) so we can update the same IOSurfaceRef without flipping as long as we're fast. This appears to be what Chromium does as well, but Firefox uses two IOSurfaceRef and flips between them. We would like to avoid two surfaces because it doubles the GPU VRAM requirements of the application. Changes to Windows ================== Previously, the NSWindow would dynamically change between different types of NSView based on the renderer being used. This is no longer necessary as we just have a single NSView type, GdkMacosView, which inherits from GdkMacosBaseView just to keep the tedius stuff separate from the machinery of GdkMacosView. We can merge those someday if we are okay with that. Changes to Views ================ GdkMacosCairoView, GdkMacosCairoSubView, GdkMacosGLView have all been removed and replaced with GdkMacosView. This new view has a single CALayer (GdkMacosLayer) attached to it which itself has sublayers. The contents of the CALayer is populated with an IOSurfaceRef which we allocated with the GdkMacosSurface. The surface is replaced when the NSWindow resizes. Changes to Layers ================= We now have a dedicated GdkMacosLayer which contains sublayers of GdkMacosTile. The tile has a maximum size of 128x128 pixels in device units. The GdkMacosTile is partitioned by splitting both the transparent region (window bounds minus opaque area) and then by splitting the opaque area. A tile has either translucent contents (and therefore is not opaque) or has opaque contents (and therefore is opaque). An opaque tile never contains transparent contents. As such, the opaque tiles contain a black background so that Core Animation will consider the tile's bounds as opaque. This can be verified with "Quartz Debug -> Show opaque regions". Changes to Cairo ================ GTK 4 cannot currently use cairo-quartz because of how CSS borders are rendered. It simply causes errors in the cairo_quartz_surface_t backend. Since we are restricted to using cairo_image_surface_t (which happens to be faster anyway) we can use the IOSurfaceBaseAddress() to obtain a mapping of the IOSurfaceRef in user-space. It always uses BGRA 32-bit with alpha channel even if we will discard the alpha channel as that is necessary to hit the fast paths in other parts of the platform. Note that while Cairo says CAIRO_FORMAT_ARGB32, it is really 32-bit BGRA on little-endian as we expect. OpenGL will render flipped (Quartz Native Co-ordinates) while Cairo renders with 0,O in the top-left. We could use cairo_translate() and cairo_scale() to reverse this, but it looks like some cairo things may not look quite as right if we do so. To reduce the chances of one-off bugs this continues to draw as Cairo would normally, but instead uses an CGAffineTransform in the tiles and some CGRect translation when swapping buffers to get the same effect. Changes to OpenGL ================= To simplify things, removal of all NSOpenGL* related components have been removed and we strictly use the Core GL (CGL*) API. This probably should have been done long ago anyay. Most examples found in the browsers to use IOSurfaceRef with OpenGL are using Legacy GL and there is still work underway to make this fit in with the rest of how the GSK GL renderer works. Since IOSurfaceRef bound to a texture/framebuffer will not have a default framebuffer ID of 0, we needed to add a default framebuffer id to the GdkGLContext. GskGLRenderer can use this to setup the command queue in such a way that our IOSurface destination has been glBindFramebuffer() as if it were the default drawable. This stuff is pretty slight-of-hand, so where things are and what needs flushing when and where has been a bit of an experiment to see what actually works to get synchronization across subsystems. Efficient Damages ================= After we draw with Cairo, we unlock the IOSurfaceRef and the contents are uploaded to the GPU. To make the contents visible to the app, we must clear the tiles contents with `layer.contents=nil;` and then re-apply the IOSurfaceRef. Since the buffer has likely not changed, we only do this if the tile overlaps the damage region. This gives the effect of having more tightly controlled damage regions even though updating the layer would damage be the whole window (as it is with OpenGL/Metal today with the exception of scissor-rect). This too can be verified usign "Quartz Debug -> Flash screen udpates". Frame Synchronized Resize ========================= In GTK 4, we have the ability to perform sizing changes from compute-size during the layout phase. Since the macOS backend already tracks window resizes manually, we can avoid doing the setFrame: immediately and instead do it within the frame clock's layout phase. Doing so gives us vastly better resize experience as we're more likely to get the size-change and updated-contents in the same frame on screen. It makes things feel "connected" in a way they weren't before. Some additional effort to tweak gravity during the process is also necessary but we were already doing that in the GTK 4 backend. Backporting =========== The design here has made an attempt to make it possible to backport by keeping GdkMacosBuffer, GdkMacosLayer, and GdkMacosTile fairly independent. There may be an opportunity to integrate this into GTK 3's quartz backend with a fair bit of work. Doing so could improve the situation for applications which are damage-rich such as The GIMP.
2022-02-14 10:20:19 +00:00
CGLSetCurrentContext (cgl_context);
CGLReleasePixelFormat (pixelFormat);
macos: use CGLSetParameter and CGLEnable We don't need to go through the NSOpenGLContext for these. We can just use the C API directly. It's also clearer what is using CGLEnable() vs CGLSetParameter().
2020-12-07 19:47:51 +00:00
macos: use CALayer and IOSurface for rendering This provides a major shift in how we draw both when accelerated OpenGL as well as software rendering with Cairo. In short, it uses tiles of Core Animation's CALayer to display contents from an OpenGL or Cairo rendering so that the window can provide partial damage updates. Partial damage is not generally available when using OpenGL as the whole buffer is flipped even if you only submitted a small change using a scissor rect. Thankfully, this speeds up Cairo rendering a bit too by using IOSurface to upload contents to the display server. We use the tiling system we do for OpenGL which reduces overall complexity and differences between them. A New Buffer ============ GdkMacosBuffer is a wrapper around an IOSurfaceRef. The term buffer was used because 1) surface is already used and 2) it loosely maps to a front/back buffer semantic. However, it appears that IOSurfaceRef contents are being retained in some fashion (likely in the compositor result) so we can update the same IOSurfaceRef without flipping as long as we're fast. This appears to be what Chromium does as well, but Firefox uses two IOSurfaceRef and flips between them. We would like to avoid two surfaces because it doubles the GPU VRAM requirements of the application. Changes to Windows ================== Previously, the NSWindow would dynamically change between different types of NSView based on the renderer being used. This is no longer necessary as we just have a single NSView type, GdkMacosView, which inherits from GdkMacosBaseView just to keep the tedius stuff separate from the machinery of GdkMacosView. We can merge those someday if we are okay with that. Changes to Views ================ GdkMacosCairoView, GdkMacosCairoSubView, GdkMacosGLView have all been removed and replaced with GdkMacosView. This new view has a single CALayer (GdkMacosLayer) attached to it which itself has sublayers. The contents of the CALayer is populated with an IOSurfaceRef which we allocated with the GdkMacosSurface. The surface is replaced when the NSWindow resizes. Changes to Layers ================= We now have a dedicated GdkMacosLayer which contains sublayers of GdkMacosTile. The tile has a maximum size of 128x128 pixels in device units. The GdkMacosTile is partitioned by splitting both the transparent region (window bounds minus opaque area) and then by splitting the opaque area. A tile has either translucent contents (and therefore is not opaque) or has opaque contents (and therefore is opaque). An opaque tile never contains transparent contents. As such, the opaque tiles contain a black background so that Core Animation will consider the tile's bounds as opaque. This can be verified with "Quartz Debug -> Show opaque regions". Changes to Cairo ================ GTK 4 cannot currently use cairo-quartz because of how CSS borders are rendered. It simply causes errors in the cairo_quartz_surface_t backend. Since we are restricted to using cairo_image_surface_t (which happens to be faster anyway) we can use the IOSurfaceBaseAddress() to obtain a mapping of the IOSurfaceRef in user-space. It always uses BGRA 32-bit with alpha channel even if we will discard the alpha channel as that is necessary to hit the fast paths in other parts of the platform. Note that while Cairo says CAIRO_FORMAT_ARGB32, it is really 32-bit BGRA on little-endian as we expect. OpenGL will render flipped (Quartz Native Co-ordinates) while Cairo renders with 0,O in the top-left. We could use cairo_translate() and cairo_scale() to reverse this, but it looks like some cairo things may not look quite as right if we do so. To reduce the chances of one-off bugs this continues to draw as Cairo would normally, but instead uses an CGAffineTransform in the tiles and some CGRect translation when swapping buffers to get the same effect. Changes to OpenGL ================= To simplify things, removal of all NSOpenGL* related components have been removed and we strictly use the Core GL (CGL*) API. This probably should have been done long ago anyay. Most examples found in the browsers to use IOSurfaceRef with OpenGL are using Legacy GL and there is still work underway to make this fit in with the rest of how the GSK GL renderer works. Since IOSurfaceRef bound to a texture/framebuffer will not have a default framebuffer ID of 0, we needed to add a default framebuffer id to the GdkGLContext. GskGLRenderer can use this to setup the command queue in such a way that our IOSurface destination has been glBindFramebuffer() as if it were the default drawable. This stuff is pretty slight-of-hand, so where things are and what needs flushing when and where has been a bit of an experiment to see what actually works to get synchronization across subsystems. Efficient Damages ================= After we draw with Cairo, we unlock the IOSurfaceRef and the contents are uploaded to the GPU. To make the contents visible to the app, we must clear the tiles contents with `layer.contents=nil;` and then re-apply the IOSurfaceRef. Since the buffer has likely not changed, we only do this if the tile overlaps the damage region. This gives the effect of having more tightly controlled damage regions even though updating the layer would damage be the whole window (as it is with OpenGL/Metal today with the exception of scissor-rect). This too can be verified usign "Quartz Debug -> Flash screen udpates". Frame Synchronized Resize ========================= In GTK 4, we have the ability to perform sizing changes from compute-size during the layout phase. Since the macOS backend already tracks window resizes manually, we can avoid doing the setFrame: immediately and instead do it within the frame clock's layout phase. Doing so gives us vastly better resize experience as we're more likely to get the size-change and updated-contents in the same frame on screen. It makes things feel "connected" in a way they weren't before. Some additional effort to tweak gravity during the process is also necessary but we were already doing that in the GTK 4 backend. Backporting =========== The design here has made an attempt to make it possible to backport by keeping GdkMacosBuffer, GdkMacosLayer, and GdkMacosTile fairly independent. There may be an opportunity to integrate this into GTK 3's quartz backend with a fair bit of work. Doing so could improve the situation for applications which are damage-rich such as The GIMP.
2022-02-14 10:20:19 +00:00
if (validate)
CHECK (NULL, CGLEnable (cgl_context, kCGLCEStateValidation));
macos: use CGLSetParameter and CGLEnable We don't need to go through the NSOpenGLContext for these. We can just use the C API directly. It's also clearer what is using CGLEnable() vs CGLSetParameter().
2020-12-07 19:47:51 +00:00
macos: use CALayer and IOSurface for rendering This provides a major shift in how we draw both when accelerated OpenGL as well as software rendering with Cairo. In short, it uses tiles of Core Animation's CALayer to display contents from an OpenGL or Cairo rendering so that the window can provide partial damage updates. Partial damage is not generally available when using OpenGL as the whole buffer is flipped even if you only submitted a small change using a scissor rect. Thankfully, this speeds up Cairo rendering a bit too by using IOSurface to upload contents to the display server. We use the tiling system we do for OpenGL which reduces overall complexity and differences between them. A New Buffer ============ GdkMacosBuffer is a wrapper around an IOSurfaceRef. The term buffer was used because 1) surface is already used and 2) it loosely maps to a front/back buffer semantic. However, it appears that IOSurfaceRef contents are being retained in some fashion (likely in the compositor result) so we can update the same IOSurfaceRef without flipping as long as we're fast. This appears to be what Chromium does as well, but Firefox uses two IOSurfaceRef and flips between them. We would like to avoid two surfaces because it doubles the GPU VRAM requirements of the application. Changes to Windows ================== Previously, the NSWindow would dynamically change between different types of NSView based on the renderer being used. This is no longer necessary as we just have a single NSView type, GdkMacosView, which inherits from GdkMacosBaseView just to keep the tedius stuff separate from the machinery of GdkMacosView. We can merge those someday if we are okay with that. Changes to Views ================ GdkMacosCairoView, GdkMacosCairoSubView, GdkMacosGLView have all been removed and replaced with GdkMacosView. This new view has a single CALayer (GdkMacosLayer) attached to it which itself has sublayers. The contents of the CALayer is populated with an IOSurfaceRef which we allocated with the GdkMacosSurface. The surface is replaced when the NSWindow resizes. Changes to Layers ================= We now have a dedicated GdkMacosLayer which contains sublayers of GdkMacosTile. The tile has a maximum size of 128x128 pixels in device units. The GdkMacosTile is partitioned by splitting both the transparent region (window bounds minus opaque area) and then by splitting the opaque area. A tile has either translucent contents (and therefore is not opaque) or has opaque contents (and therefore is opaque). An opaque tile never contains transparent contents. As such, the opaque tiles contain a black background so that Core Animation will consider the tile's bounds as opaque. This can be verified with "Quartz Debug -> Show opaque regions". Changes to Cairo ================ GTK 4 cannot currently use cairo-quartz because of how CSS borders are rendered. It simply causes errors in the cairo_quartz_surface_t backend. Since we are restricted to using cairo_image_surface_t (which happens to be faster anyway) we can use the IOSurfaceBaseAddress() to obtain a mapping of the IOSurfaceRef in user-space. It always uses BGRA 32-bit with alpha channel even if we will discard the alpha channel as that is necessary to hit the fast paths in other parts of the platform. Note that while Cairo says CAIRO_FORMAT_ARGB32, it is really 32-bit BGRA on little-endian as we expect. OpenGL will render flipped (Quartz Native Co-ordinates) while Cairo renders with 0,O in the top-left. We could use cairo_translate() and cairo_scale() to reverse this, but it looks like some cairo things may not look quite as right if we do so. To reduce the chances of one-off bugs this continues to draw as Cairo would normally, but instead uses an CGAffineTransform in the tiles and some CGRect translation when swapping buffers to get the same effect. Changes to OpenGL ================= To simplify things, removal of all NSOpenGL* related components have been removed and we strictly use the Core GL (CGL*) API. This probably should have been done long ago anyay. Most examples found in the browsers to use IOSurfaceRef with OpenGL are using Legacy GL and there is still work underway to make this fit in with the rest of how the GSK GL renderer works. Since IOSurfaceRef bound to a texture/framebuffer will not have a default framebuffer ID of 0, we needed to add a default framebuffer id to the GdkGLContext. GskGLRenderer can use this to setup the command queue in such a way that our IOSurface destination has been glBindFramebuffer() as if it were the default drawable. This stuff is pretty slight-of-hand, so where things are and what needs flushing when and where has been a bit of an experiment to see what actually works to get synchronization across subsystems. Efficient Damages ================= After we draw with Cairo, we unlock the IOSurfaceRef and the contents are uploaded to the GPU. To make the contents visible to the app, we must clear the tiles contents with `layer.contents=nil;` and then re-apply the IOSurfaceRef. Since the buffer has likely not changed, we only do this if the tile overlaps the damage region. This gives the effect of having more tightly controlled damage regions even though updating the layer would damage be the whole window (as it is with OpenGL/Metal today with the exception of scissor-rect). This too can be verified usign "Quartz Debug -> Flash screen udpates". Frame Synchronized Resize ========================= In GTK 4, we have the ability to perform sizing changes from compute-size during the layout phase. Since the macOS backend already tracks window resizes manually, we can avoid doing the setFrame: immediately and instead do it within the frame clock's layout phase. Doing so gives us vastly better resize experience as we're more likely to get the size-change and updated-contents in the same frame on screen. It makes things feel "connected" in a way they weren't before. Some additional effort to tweak gravity during the process is also necessary but we were already doing that in the GTK 4 backend. Backporting =========== The design here has made an attempt to make it possible to backport by keeping GdkMacosBuffer, GdkMacosLayer, and GdkMacosTile fairly independent. There may be an opportunity to integrate this into GTK 3's quartz backend with a fair bit of work. Doing so could improve the situation for applications which are damage-rich such as The GIMP.
2022-02-14 10:20:19 +00:00
if (!CHECK (error, CGLSetParameter (cgl_context, kCGLCPSwapInterval, &sync_to_framerate)) ||
!CHECK (error, CGLGetParameter (cgl_context, kCGLCPCurrentRendererID, &renderer_id)))
{
CGLReleaseContext (cgl_context);
return 0;
}
macos: use CGLSetParameter and CGLEnable We don't need to go through the NSOpenGLContext for these. We can just use the C API directly. It's also clearer what is using CGLEnable() vs CGLSetParameter().
2020-12-07 19:47:51 +00:00
macos: use CALayer and IOSurface for rendering This provides a major shift in how we draw both when accelerated OpenGL as well as software rendering with Cairo. In short, it uses tiles of Core Animation's CALayer to display contents from an OpenGL or Cairo rendering so that the window can provide partial damage updates. Partial damage is not generally available when using OpenGL as the whole buffer is flipped even if you only submitted a small change using a scissor rect. Thankfully, this speeds up Cairo rendering a bit too by using IOSurface to upload contents to the display server. We use the tiling system we do for OpenGL which reduces overall complexity and differences between them. A New Buffer ============ GdkMacosBuffer is a wrapper around an IOSurfaceRef. The term buffer was used because 1) surface is already used and 2) it loosely maps to a front/back buffer semantic. However, it appears that IOSurfaceRef contents are being retained in some fashion (likely in the compositor result) so we can update the same IOSurfaceRef without flipping as long as we're fast. This appears to be what Chromium does as well, but Firefox uses two IOSurfaceRef and flips between them. We would like to avoid two surfaces because it doubles the GPU VRAM requirements of the application. Changes to Windows ================== Previously, the NSWindow would dynamically change between different types of NSView based on the renderer being used. This is no longer necessary as we just have a single NSView type, GdkMacosView, which inherits from GdkMacosBaseView just to keep the tedius stuff separate from the machinery of GdkMacosView. We can merge those someday if we are okay with that. Changes to Views ================ GdkMacosCairoView, GdkMacosCairoSubView, GdkMacosGLView have all been removed and replaced with GdkMacosView. This new view has a single CALayer (GdkMacosLayer) attached to it which itself has sublayers. The contents of the CALayer is populated with an IOSurfaceRef which we allocated with the GdkMacosSurface. The surface is replaced when the NSWindow resizes. Changes to Layers ================= We now have a dedicated GdkMacosLayer which contains sublayers of GdkMacosTile. The tile has a maximum size of 128x128 pixels in device units. The GdkMacosTile is partitioned by splitting both the transparent region (window bounds minus opaque area) and then by splitting the opaque area. A tile has either translucent contents (and therefore is not opaque) or has opaque contents (and therefore is opaque). An opaque tile never contains transparent contents. As such, the opaque tiles contain a black background so that Core Animation will consider the tile's bounds as opaque. This can be verified with "Quartz Debug -> Show opaque regions". Changes to Cairo ================ GTK 4 cannot currently use cairo-quartz because of how CSS borders are rendered. It simply causes errors in the cairo_quartz_surface_t backend. Since we are restricted to using cairo_image_surface_t (which happens to be faster anyway) we can use the IOSurfaceBaseAddress() to obtain a mapping of the IOSurfaceRef in user-space. It always uses BGRA 32-bit with alpha channel even if we will discard the alpha channel as that is necessary to hit the fast paths in other parts of the platform. Note that while Cairo says CAIRO_FORMAT_ARGB32, it is really 32-bit BGRA on little-endian as we expect. OpenGL will render flipped (Quartz Native Co-ordinates) while Cairo renders with 0,O in the top-left. We could use cairo_translate() and cairo_scale() to reverse this, but it looks like some cairo things may not look quite as right if we do so. To reduce the chances of one-off bugs this continues to draw as Cairo would normally, but instead uses an CGAffineTransform in the tiles and some CGRect translation when swapping buffers to get the same effect. Changes to OpenGL ================= To simplify things, removal of all NSOpenGL* related components have been removed and we strictly use the Core GL (CGL*) API. This probably should have been done long ago anyay. Most examples found in the browsers to use IOSurfaceRef with OpenGL are using Legacy GL and there is still work underway to make this fit in with the rest of how the GSK GL renderer works. Since IOSurfaceRef bound to a texture/framebuffer will not have a default framebuffer ID of 0, we needed to add a default framebuffer id to the GdkGLContext. GskGLRenderer can use this to setup the command queue in such a way that our IOSurface destination has been glBindFramebuffer() as if it were the default drawable. This stuff is pretty slight-of-hand, so where things are and what needs flushing when and where has been a bit of an experiment to see what actually works to get synchronization across subsystems. Efficient Damages ================= After we draw with Cairo, we unlock the IOSurfaceRef and the contents are uploaded to the GPU. To make the contents visible to the app, we must clear the tiles contents with `layer.contents=nil;` and then re-apply the IOSurfaceRef. Since the buffer has likely not changed, we only do this if the tile overlaps the damage region. This gives the effect of having more tightly controlled damage regions even though updating the layer would damage be the whole window (as it is with OpenGL/Metal today with the exception of scissor-rect). This too can be verified usign "Quartz Debug -> Flash screen udpates". Frame Synchronized Resize ========================= In GTK 4, we have the ability to perform sizing changes from compute-size during the layout phase. Since the macOS backend already tracks window resizes manually, we can avoid doing the setFrame: immediately and instead do it within the frame clock's layout phase. Doing so gives us vastly better resize experience as we're more likely to get the size-change and updated-contents in the same frame on screen. It makes things feel "connected" in a way they weren't before. Some additional effort to tweak gravity during the process is also necessary but we were already doing that in the GTK 4 backend. Backporting =========== The design here has made an attempt to make it possible to backport by keeping GdkMacosBuffer, GdkMacosLayer, and GdkMacosTile fairly independent. There may be an opportunity to integrate this into GTK 3's quartz backend with a fair bit of work. Doing so could improve the situation for applications which are damage-rich such as The GIMP.
2022-02-14 10:20:19 +00:00
surface = gdk_draw_context_get_surface (GDK_DRAW_CONTEXT (context));
macos: implement GL context This implements the basics for a GdkGLContext on macOS. Currently, rendering only is fully working for the GskCairoRenderer case where we read back pixels into a cairo surface for rendering. More work on synchronization is required for the GL on GskGLRenderer case. When we attempt to render a surface itself with GL, the context will ensure that the new GdkMacosGLView is placed within the NSWindow. In other cases, we use a dummy NSView and NSWindow for backing the NSOpenGLContext to ensure that we can get accelerated drawing. This gets GtkGLArea working when running with GSK_RENDERER=cairo.
2020-10-29 17:33:22 +00:00
macos: use CALayer and IOSurface for rendering This provides a major shift in how we draw both when accelerated OpenGL as well as software rendering with Cairo. In short, it uses tiles of Core Animation's CALayer to display contents from an OpenGL or Cairo rendering so that the window can provide partial damage updates. Partial damage is not generally available when using OpenGL as the whole buffer is flipped even if you only submitted a small change using a scissor rect. Thankfully, this speeds up Cairo rendering a bit too by using IOSurface to upload contents to the display server. We use the tiling system we do for OpenGL which reduces overall complexity and differences between them. A New Buffer ============ GdkMacosBuffer is a wrapper around an IOSurfaceRef. The term buffer was used because 1) surface is already used and 2) it loosely maps to a front/back buffer semantic. However, it appears that IOSurfaceRef contents are being retained in some fashion (likely in the compositor result) so we can update the same IOSurfaceRef without flipping as long as we're fast. This appears to be what Chromium does as well, but Firefox uses two IOSurfaceRef and flips between them. We would like to avoid two surfaces because it doubles the GPU VRAM requirements of the application. Changes to Windows ================== Previously, the NSWindow would dynamically change between different types of NSView based on the renderer being used. This is no longer necessary as we just have a single NSView type, GdkMacosView, which inherits from GdkMacosBaseView just to keep the tedius stuff separate from the machinery of GdkMacosView. We can merge those someday if we are okay with that. Changes to Views ================ GdkMacosCairoView, GdkMacosCairoSubView, GdkMacosGLView have all been removed and replaced with GdkMacosView. This new view has a single CALayer (GdkMacosLayer) attached to it which itself has sublayers. The contents of the CALayer is populated with an IOSurfaceRef which we allocated with the GdkMacosSurface. The surface is replaced when the NSWindow resizes. Changes to Layers ================= We now have a dedicated GdkMacosLayer which contains sublayers of GdkMacosTile. The tile has a maximum size of 128x128 pixels in device units. The GdkMacosTile is partitioned by splitting both the transparent region (window bounds minus opaque area) and then by splitting the opaque area. A tile has either translucent contents (and therefore is not opaque) or has opaque contents (and therefore is opaque). An opaque tile never contains transparent contents. As such, the opaque tiles contain a black background so that Core Animation will consider the tile's bounds as opaque. This can be verified with "Quartz Debug -> Show opaque regions". Changes to Cairo ================ GTK 4 cannot currently use cairo-quartz because of how CSS borders are rendered. It simply causes errors in the cairo_quartz_surface_t backend. Since we are restricted to using cairo_image_surface_t (which happens to be faster anyway) we can use the IOSurfaceBaseAddress() to obtain a mapping of the IOSurfaceRef in user-space. It always uses BGRA 32-bit with alpha channel even if we will discard the alpha channel as that is necessary to hit the fast paths in other parts of the platform. Note that while Cairo says CAIRO_FORMAT_ARGB32, it is really 32-bit BGRA on little-endian as we expect. OpenGL will render flipped (Quartz Native Co-ordinates) while Cairo renders with 0,O in the top-left. We could use cairo_translate() and cairo_scale() to reverse this, but it looks like some cairo things may not look quite as right if we do so. To reduce the chances of one-off bugs this continues to draw as Cairo would normally, but instead uses an CGAffineTransform in the tiles and some CGRect translation when swapping buffers to get the same effect. Changes to OpenGL ================= To simplify things, removal of all NSOpenGL* related components have been removed and we strictly use the Core GL (CGL*) API. This probably should have been done long ago anyay. Most examples found in the browsers to use IOSurfaceRef with OpenGL are using Legacy GL and there is still work underway to make this fit in with the rest of how the GSK GL renderer works. Since IOSurfaceRef bound to a texture/framebuffer will not have a default framebuffer ID of 0, we needed to add a default framebuffer id to the GdkGLContext. GskGLRenderer can use this to setup the command queue in such a way that our IOSurface destination has been glBindFramebuffer() as if it were the default drawable. This stuff is pretty slight-of-hand, so where things are and what needs flushing when and where has been a bit of an experiment to see what actually works to get synchronization across subsystems. Efficient Damages ================= After we draw with Cairo, we unlock the IOSurfaceRef and the contents are uploaded to the GPU. To make the contents visible to the app, we must clear the tiles contents with `layer.contents=nil;` and then re-apply the IOSurfaceRef. Since the buffer has likely not changed, we only do this if the tile overlaps the damage region. This gives the effect of having more tightly controlled damage regions even though updating the layer would damage be the whole window (as it is with OpenGL/Metal today with the exception of scissor-rect). This too can be verified usign "Quartz Debug -> Flash screen udpates". Frame Synchronized Resize ========================= In GTK 4, we have the ability to perform sizing changes from compute-size during the layout phase. Since the macOS backend already tracks window resizes manually, we can avoid doing the setFrame: immediately and instead do it within the frame clock's layout phase. Doing so gives us vastly better resize experience as we're more likely to get the size-change and updated-contents in the same frame on screen. It makes things feel "connected" in a way they weren't before. Some additional effort to tweak gravity during the process is also necessary but we were already doing that in the GTK 4 backend. Backporting =========== The design here has made an attempt to make it possible to backport by keeping GdkMacosBuffer, GdkMacosLayer, and GdkMacosTile fairly independent. There may be an opportunity to integrate this into GTK 3's quartz backend with a fair bit of work. Doing so could improve the situation for applications which are damage-rich such as The GIMP.
2022-02-14 10:20:19 +00:00
if (surface != NULL)
{
/* Setup initial swap rectangle. We might not actually need this
* anymore though as we are rendering to an IOSurface and we have
* a scissor clip when rendering to it.
*/
swapRect[0] = 0;
swapRect[1] = 0;
swapRect[2] = surface->width;
swapRect[3] = surface->height;
CGLSetParameter (cgl_context, kCGLCPSwapRectangle, swapRect);
CGLEnable (cgl_context, kCGLCESwapRectangle);
}
macos: implement GL context This implements the basics for a GdkGLContext on macOS. Currently, rendering only is fully working for the GskCairoRenderer case where we read back pixels into a cairo surface for rendering. More work on synchronization is required for the GL on GskGLRenderer case. When we attempt to render a surface itself with GL, the context will ensure that the new GdkMacosGLView is placed within the NSWindow. In other cases, we use a dummy NSView and NSWindow for backing the NSOpenGLContext to ensure that we can get accelerated drawing. This gets GtkGLArea working when running with GSK_RENDERER=cairo.
2020-10-29 17:33:22 +00:00
gdk: Add GdkDisplay::init_gl vfunc The vfunc is called to initialize GL and it returns a "base" context that GDK then uses as the context all others are shared with. So the GL context share tree now looks like: + context from init_gl - context1 - context2 ... So this is a flat tree now, the complexity is gone. The only caveat is that backends now need to create a GL context when initializing GL so some refactoring was needed. Two new functions have been added: * gdk_display_prepare_gl() This is public API and can be used to ensure that GL has been initialized or if not, retrieve an error to display (or debug-print). * gdk_display_get_gl_context() This is a private function to retrieve the base context from init_gl(). It replaces gdk_surface_get_shared_data_context().
2021-07-04 23:57:03 +00:00
GDK_DISPLAY_NOTE (display,
macos: implement GL context This implements the basics for a GdkGLContext on macOS. Currently, rendering only is fully working for the GskCairoRenderer case where we read back pixels into a cairo surface for rendering. More work on synchronization is required for the GL on GskGLRenderer case. When we attempt to render a surface itself with GL, the context will ensure that the new GdkMacosGLView is placed within the NSWindow. In other cases, we use a dummy NSView and NSWindow for backing the NSOpenGLContext to ensure that we can get accelerated drawing. This gets GtkGLArea working when running with GSK_RENDERER=cairo.
2020-10-29 17:33:22 +00:00
OPENGL,
macos: use CALayer and IOSurface for rendering This provides a major shift in how we draw both when accelerated OpenGL as well as software rendering with Cairo. In short, it uses tiles of Core Animation's CALayer to display contents from an OpenGL or Cairo rendering so that the window can provide partial damage updates. Partial damage is not generally available when using OpenGL as the whole buffer is flipped even if you only submitted a small change using a scissor rect. Thankfully, this speeds up Cairo rendering a bit too by using IOSurface to upload contents to the display server. We use the tiling system we do for OpenGL which reduces overall complexity and differences between them. A New Buffer ============ GdkMacosBuffer is a wrapper around an IOSurfaceRef. The term buffer was used because 1) surface is already used and 2) it loosely maps to a front/back buffer semantic. However, it appears that IOSurfaceRef contents are being retained in some fashion (likely in the compositor result) so we can update the same IOSurfaceRef without flipping as long as we're fast. This appears to be what Chromium does as well, but Firefox uses two IOSurfaceRef and flips between them. We would like to avoid two surfaces because it doubles the GPU VRAM requirements of the application. Changes to Windows ================== Previously, the NSWindow would dynamically change between different types of NSView based on the renderer being used. This is no longer necessary as we just have a single NSView type, GdkMacosView, which inherits from GdkMacosBaseView just to keep the tedius stuff separate from the machinery of GdkMacosView. We can merge those someday if we are okay with that. Changes to Views ================ GdkMacosCairoView, GdkMacosCairoSubView, GdkMacosGLView have all been removed and replaced with GdkMacosView. This new view has a single CALayer (GdkMacosLayer) attached to it which itself has sublayers. The contents of the CALayer is populated with an IOSurfaceRef which we allocated with the GdkMacosSurface. The surface is replaced when the NSWindow resizes. Changes to Layers ================= We now have a dedicated GdkMacosLayer which contains sublayers of GdkMacosTile. The tile has a maximum size of 128x128 pixels in device units. The GdkMacosTile is partitioned by splitting both the transparent region (window bounds minus opaque area) and then by splitting the opaque area. A tile has either translucent contents (and therefore is not opaque) or has opaque contents (and therefore is opaque). An opaque tile never contains transparent contents. As such, the opaque tiles contain a black background so that Core Animation will consider the tile's bounds as opaque. This can be verified with "Quartz Debug -> Show opaque regions". Changes to Cairo ================ GTK 4 cannot currently use cairo-quartz because of how CSS borders are rendered. It simply causes errors in the cairo_quartz_surface_t backend. Since we are restricted to using cairo_image_surface_t (which happens to be faster anyway) we can use the IOSurfaceBaseAddress() to obtain a mapping of the IOSurfaceRef in user-space. It always uses BGRA 32-bit with alpha channel even if we will discard the alpha channel as that is necessary to hit the fast paths in other parts of the platform. Note that while Cairo says CAIRO_FORMAT_ARGB32, it is really 32-bit BGRA on little-endian as we expect. OpenGL will render flipped (Quartz Native Co-ordinates) while Cairo renders with 0,O in the top-left. We could use cairo_translate() and cairo_scale() to reverse this, but it looks like some cairo things may not look quite as right if we do so. To reduce the chances of one-off bugs this continues to draw as Cairo would normally, but instead uses an CGAffineTransform in the tiles and some CGRect translation when swapping buffers to get the same effect. Changes to OpenGL ================= To simplify things, removal of all NSOpenGL* related components have been removed and we strictly use the Core GL (CGL*) API. This probably should have been done long ago anyay. Most examples found in the browsers to use IOSurfaceRef with OpenGL are using Legacy GL and there is still work underway to make this fit in with the rest of how the GSK GL renderer works. Since IOSurfaceRef bound to a texture/framebuffer will not have a default framebuffer ID of 0, we needed to add a default framebuffer id to the GdkGLContext. GskGLRenderer can use this to setup the command queue in such a way that our IOSurface destination has been glBindFramebuffer() as if it were the default drawable. This stuff is pretty slight-of-hand, so where things are and what needs flushing when and where has been a bit of an experiment to see what actually works to get synchronization across subsystems. Efficient Damages ================= After we draw with Cairo, we unlock the IOSurfaceRef and the contents are uploaded to the GPU. To make the contents visible to the app, we must clear the tiles contents with `layer.contents=nil;` and then re-apply the IOSurfaceRef. Since the buffer has likely not changed, we only do this if the tile overlaps the damage region. This gives the effect of having more tightly controlled damage regions even though updating the layer would damage be the whole window (as it is with OpenGL/Metal today with the exception of scissor-rect). This too can be verified usign "Quartz Debug -> Flash screen udpates". Frame Synchronized Resize ========================= In GTK 4, we have the ability to perform sizing changes from compute-size during the layout phase. Since the macOS backend already tracks window resizes manually, we can avoid doing the setFrame: immediately and instead do it within the frame clock's layout phase. Doing so gives us vastly better resize experience as we're more likely to get the size-change and updated-contents in the same frame on screen. It makes things feel "connected" in a way they weren't before. Some additional effort to tweak gravity during the process is also necessary but we were already doing that in the GTK 4 backend. Backporting =========== The design here has made an attempt to make it possible to backport by keeping GdkMacosBuffer, GdkMacosLayer, and GdkMacosTile fairly independent. There may be an opportunity to integrate this into GTK 3's quartz backend with a fair bit of work. Doing so could improve the situation for applications which are damage-rich such as The GIMP.
2022-02-14 10:20:19 +00:00
g_message ("Created CGLContextObj@%p using %s",
cgl_context,
macos: implement GL context This implements the basics for a GdkGLContext on macOS. Currently, rendering only is fully working for the GskCairoRenderer case where we read back pixels into a cairo surface for rendering. More work on synchronization is required for the GL on GskGLRenderer case. When we attempt to render a surface itself with GL, the context will ensure that the new GdkMacosGLView is placed within the NSWindow. In other cases, we use a dummy NSView and NSWindow for backing the NSOpenGLContext to ensure that we can get accelerated drawing. This gets GtkGLArea working when running with GSK_RENDERER=cairo.
2020-10-29 17:33:22 +00:00
get_renderer_name (renderer_id)));
macos: use CALayer and IOSurface for rendering This provides a major shift in how we draw both when accelerated OpenGL as well as software rendering with Cairo. In short, it uses tiles of Core Animation's CALayer to display contents from an OpenGL or Cairo rendering so that the window can provide partial damage updates. Partial damage is not generally available when using OpenGL as the whole buffer is flipped even if you only submitted a small change using a scissor rect. Thankfully, this speeds up Cairo rendering a bit too by using IOSurface to upload contents to the display server. We use the tiling system we do for OpenGL which reduces overall complexity and differences between them. A New Buffer ============ GdkMacosBuffer is a wrapper around an IOSurfaceRef. The term buffer was used because 1) surface is already used and 2) it loosely maps to a front/back buffer semantic. However, it appears that IOSurfaceRef contents are being retained in some fashion (likely in the compositor result) so we can update the same IOSurfaceRef without flipping as long as we're fast. This appears to be what Chromium does as well, but Firefox uses two IOSurfaceRef and flips between them. We would like to avoid two surfaces because it doubles the GPU VRAM requirements of the application. Changes to Windows ================== Previously, the NSWindow would dynamically change between different types of NSView based on the renderer being used. This is no longer necessary as we just have a single NSView type, GdkMacosView, which inherits from GdkMacosBaseView just to keep the tedius stuff separate from the machinery of GdkMacosView. We can merge those someday if we are okay with that. Changes to Views ================ GdkMacosCairoView, GdkMacosCairoSubView, GdkMacosGLView have all been removed and replaced with GdkMacosView. This new view has a single CALayer (GdkMacosLayer) attached to it which itself has sublayers. The contents of the CALayer is populated with an IOSurfaceRef which we allocated with the GdkMacosSurface. The surface is replaced when the NSWindow resizes. Changes to Layers ================= We now have a dedicated GdkMacosLayer which contains sublayers of GdkMacosTile. The tile has a maximum size of 128x128 pixels in device units. The GdkMacosTile is partitioned by splitting both the transparent region (window bounds minus opaque area) and then by splitting the opaque area. A tile has either translucent contents (and therefore is not opaque) or has opaque contents (and therefore is opaque). An opaque tile never contains transparent contents. As such, the opaque tiles contain a black background so that Core Animation will consider the tile's bounds as opaque. This can be verified with "Quartz Debug -> Show opaque regions". Changes to Cairo ================ GTK 4 cannot currently use cairo-quartz because of how CSS borders are rendered. It simply causes errors in the cairo_quartz_surface_t backend. Since we are restricted to using cairo_image_surface_t (which happens to be faster anyway) we can use the IOSurfaceBaseAddress() to obtain a mapping of the IOSurfaceRef in user-space. It always uses BGRA 32-bit with alpha channel even if we will discard the alpha channel as that is necessary to hit the fast paths in other parts of the platform. Note that while Cairo says CAIRO_FORMAT_ARGB32, it is really 32-bit BGRA on little-endian as we expect. OpenGL will render flipped (Quartz Native Co-ordinates) while Cairo renders with 0,O in the top-left. We could use cairo_translate() and cairo_scale() to reverse this, but it looks like some cairo things may not look quite as right if we do so. To reduce the chances of one-off bugs this continues to draw as Cairo would normally, but instead uses an CGAffineTransform in the tiles and some CGRect translation when swapping buffers to get the same effect. Changes to OpenGL ================= To simplify things, removal of all NSOpenGL* related components have been removed and we strictly use the Core GL (CGL*) API. This probably should have been done long ago anyay. Most examples found in the browsers to use IOSurfaceRef with OpenGL are using Legacy GL and there is still work underway to make this fit in with the rest of how the GSK GL renderer works. Since IOSurfaceRef bound to a texture/framebuffer will not have a default framebuffer ID of 0, we needed to add a default framebuffer id to the GdkGLContext. GskGLRenderer can use this to setup the command queue in such a way that our IOSurface destination has been glBindFramebuffer() as if it were the default drawable. This stuff is pretty slight-of-hand, so where things are and what needs flushing when and where has been a bit of an experiment to see what actually works to get synchronization across subsystems. Efficient Damages ================= After we draw with Cairo, we unlock the IOSurfaceRef and the contents are uploaded to the GPU. To make the contents visible to the app, we must clear the tiles contents with `layer.contents=nil;` and then re-apply the IOSurfaceRef. Since the buffer has likely not changed, we only do this if the tile overlaps the damage region. This gives the effect of having more tightly controlled damage regions even though updating the layer would damage be the whole window (as it is with OpenGL/Metal today with the exception of scissor-rect). This too can be verified usign "Quartz Debug -> Flash screen udpates". Frame Synchronized Resize ========================= In GTK 4, we have the ability to perform sizing changes from compute-size during the layout phase. Since the macOS backend already tracks window resizes manually, we can avoid doing the setFrame: immediately and instead do it within the frame clock's layout phase. Doing so gives us vastly better resize experience as we're more likely to get the size-change and updated-contents in the same frame on screen. It makes things feel "connected" in a way they weren't before. Some additional effort to tweak gravity during the process is also necessary but we were already doing that in the GTK 4 backend. Backporting =========== The design here has made an attempt to make it possible to backport by keeping GdkMacosBuffer, GdkMacosLayer, and GdkMacosTile fairly independent. There may be an opportunity to integrate this into GTK 3's quartz backend with a fair bit of work. Doing so could improve the situation for applications which are damage-rich such as The GIMP.
2022-02-14 10:20:19 +00:00
self->cgl_context = g_steal_pointer (&cgl_context);
macos: implement GL context This implements the basics for a GdkGLContext on macOS. Currently, rendering only is fully working for the GskCairoRenderer case where we read back pixels into a cairo surface for rendering. More work on synchronization is required for the GL on GskGLRenderer case. When we attempt to render a surface itself with GL, the context will ensure that the new GdkMacosGLView is placed within the NSWindow. In other cases, we use a dummy NSView and NSWindow for backing the NSOpenGLContext to ensure that we can get accelerated drawing. This gets GtkGLArea working when running with GSK_RENDERER=cairo.
2020-10-29 17:33:22 +00:00
if (existing != NULL)
macos: use CALayer and IOSurface for rendering This provides a major shift in how we draw both when accelerated OpenGL as well as software rendering with Cairo. In short, it uses tiles of Core Animation's CALayer to display contents from an OpenGL or Cairo rendering so that the window can provide partial damage updates. Partial damage is not generally available when using OpenGL as the whole buffer is flipped even if you only submitted a small change using a scissor rect. Thankfully, this speeds up Cairo rendering a bit too by using IOSurface to upload contents to the display server. We use the tiling system we do for OpenGL which reduces overall complexity and differences between them. A New Buffer ============ GdkMacosBuffer is a wrapper around an IOSurfaceRef. The term buffer was used because 1) surface is already used and 2) it loosely maps to a front/back buffer semantic. However, it appears that IOSurfaceRef contents are being retained in some fashion (likely in the compositor result) so we can update the same IOSurfaceRef without flipping as long as we're fast. This appears to be what Chromium does as well, but Firefox uses two IOSurfaceRef and flips between them. We would like to avoid two surfaces because it doubles the GPU VRAM requirements of the application. Changes to Windows ================== Previously, the NSWindow would dynamically change between different types of NSView based on the renderer being used. This is no longer necessary as we just have a single NSView type, GdkMacosView, which inherits from GdkMacosBaseView just to keep the tedius stuff separate from the machinery of GdkMacosView. We can merge those someday if we are okay with that. Changes to Views ================ GdkMacosCairoView, GdkMacosCairoSubView, GdkMacosGLView have all been removed and replaced with GdkMacosView. This new view has a single CALayer (GdkMacosLayer) attached to it which itself has sublayers. The contents of the CALayer is populated with an IOSurfaceRef which we allocated with the GdkMacosSurface. The surface is replaced when the NSWindow resizes. Changes to Layers ================= We now have a dedicated GdkMacosLayer which contains sublayers of GdkMacosTile. The tile has a maximum size of 128x128 pixels in device units. The GdkMacosTile is partitioned by splitting both the transparent region (window bounds minus opaque area) and then by splitting the opaque area. A tile has either translucent contents (and therefore is not opaque) or has opaque contents (and therefore is opaque). An opaque tile never contains transparent contents. As such, the opaque tiles contain a black background so that Core Animation will consider the tile's bounds as opaque. This can be verified with "Quartz Debug -> Show opaque regions". Changes to Cairo ================ GTK 4 cannot currently use cairo-quartz because of how CSS borders are rendered. It simply causes errors in the cairo_quartz_surface_t backend. Since we are restricted to using cairo_image_surface_t (which happens to be faster anyway) we can use the IOSurfaceBaseAddress() to obtain a mapping of the IOSurfaceRef in user-space. It always uses BGRA 32-bit with alpha channel even if we will discard the alpha channel as that is necessary to hit the fast paths in other parts of the platform. Note that while Cairo says CAIRO_FORMAT_ARGB32, it is really 32-bit BGRA on little-endian as we expect. OpenGL will render flipped (Quartz Native Co-ordinates) while Cairo renders with 0,O in the top-left. We could use cairo_translate() and cairo_scale() to reverse this, but it looks like some cairo things may not look quite as right if we do so. To reduce the chances of one-off bugs this continues to draw as Cairo would normally, but instead uses an CGAffineTransform in the tiles and some CGRect translation when swapping buffers to get the same effect. Changes to OpenGL ================= To simplify things, removal of all NSOpenGL* related components have been removed and we strictly use the Core GL (CGL*) API. This probably should have been done long ago anyay. Most examples found in the browsers to use IOSurfaceRef with OpenGL are using Legacy GL and there is still work underway to make this fit in with the rest of how the GSK GL renderer works. Since IOSurfaceRef bound to a texture/framebuffer will not have a default framebuffer ID of 0, we needed to add a default framebuffer id to the GdkGLContext. GskGLRenderer can use this to setup the command queue in such a way that our IOSurface destination has been glBindFramebuffer() as if it were the default drawable. This stuff is pretty slight-of-hand, so where things are and what needs flushing when and where has been a bit of an experiment to see what actually works to get synchronization across subsystems. Efficient Damages ================= After we draw with Cairo, we unlock the IOSurfaceRef and the contents are uploaded to the GPU. To make the contents visible to the app, we must clear the tiles contents with `layer.contents=nil;` and then re-apply the IOSurfaceRef. Since the buffer has likely not changed, we only do this if the tile overlaps the damage region. This gives the effect of having more tightly controlled damage regions even though updating the layer would damage be the whole window (as it is with OpenGL/Metal today with the exception of scissor-rect). This too can be verified usign "Quartz Debug -> Flash screen udpates". Frame Synchronized Resize ========================= In GTK 4, we have the ability to perform sizing changes from compute-size during the layout phase. Since the macOS backend already tracks window resizes manually, we can avoid doing the setFrame: immediately and instead do it within the frame clock's layout phase. Doing so gives us vastly better resize experience as we're more likely to get the size-change and updated-contents in the same frame on screen. It makes things feel "connected" in a way they weren't before. Some additional effort to tweak gravity during the process is also necessary but we were already doing that in the GTK 4 backend. Backporting =========== The design here has made an attempt to make it possible to backport by keeping GdkMacosBuffer, GdkMacosLayer, and GdkMacosTile fairly independent. There may be an opportunity to integrate this into GTK 3's quartz backend with a fair bit of work. Doing so could improve the situation for applications which are damage-rich such as The GIMP.
2022-02-14 10:20:19 +00:00
CGLSetCurrentContext (existing);
macos: implement GL context This implements the basics for a GdkGLContext on macOS. Currently, rendering only is fully working for the GskCairoRenderer case where we read back pixels into a cairo surface for rendering. More work on synchronization is required for the GL on GskGLRenderer case. When we attempt to render a surface itself with GL, the context will ensure that the new GdkMacosGLView is placed within the NSWindow. In other cases, we use a dummy NSView and NSWindow for backing the NSOpenGLContext to ensure that we can get accelerated drawing. This gets GtkGLArea working when running with GSK_RENDERER=cairo.
2020-10-29 17:33:22 +00:00
gl: Check allowed APIs in realize() Add gdk_gl_context_is_api_allowed() for backends and make them use it. Finally, have them return the final API as the return value (or 0 on error). And then use that api instead of a use_es boolean flag. Fixes #4221
2021-10-07 20:10:39 +00:00
return GDK_GL_API_GL;
macos: implement GL context This implements the basics for a GdkGLContext on macOS. Currently, rendering only is fully working for the GskCairoRenderer case where we read back pixels into a cairo surface for rendering. More work on synchronization is required for the GL on GskGLRenderer case. When we attempt to render a surface itself with GL, the context will ensure that the new GdkMacosGLView is placed within the NSWindow. In other cases, we use a dummy NSView and NSWindow for backing the NSOpenGLContext to ensure that we can get accelerated drawing. This gets GtkGLArea working when running with GSK_RENDERER=cairo.
2020-10-29 17:33:22 +00:00
}
static void
gdk_macos_gl_context_begin_frame (GdkDrawContext *context,
gdk: hdr => high depth The term "hdr" is so overloaded, we shouldn't use them anywhere, except from maybe describing all of this work in blog posts and other marketing materials. So do renames: * hdr => high_depth * request_hdr => prefers_high_depth This more accurately describes what is going on.
2021-10-06 16:25:30 +00:00
gboolean prefers_high_depth,
macos: use CALayer and IOSurface for rendering This provides a major shift in how we draw both when accelerated OpenGL as well as software rendering with Cairo. In short, it uses tiles of Core Animation's CALayer to display contents from an OpenGL or Cairo rendering so that the window can provide partial damage updates. Partial damage is not generally available when using OpenGL as the whole buffer is flipped even if you only submitted a small change using a scissor rect. Thankfully, this speeds up Cairo rendering a bit too by using IOSurface to upload contents to the display server. We use the tiling system we do for OpenGL which reduces overall complexity and differences between them. A New Buffer ============ GdkMacosBuffer is a wrapper around an IOSurfaceRef. The term buffer was used because 1) surface is already used and 2) it loosely maps to a front/back buffer semantic. However, it appears that IOSurfaceRef contents are being retained in some fashion (likely in the compositor result) so we can update the same IOSurfaceRef without flipping as long as we're fast. This appears to be what Chromium does as well, but Firefox uses two IOSurfaceRef and flips between them. We would like to avoid two surfaces because it doubles the GPU VRAM requirements of the application. Changes to Windows ================== Previously, the NSWindow would dynamically change between different types of NSView based on the renderer being used. This is no longer necessary as we just have a single NSView type, GdkMacosView, which inherits from GdkMacosBaseView just to keep the tedius stuff separate from the machinery of GdkMacosView. We can merge those someday if we are okay with that. Changes to Views ================ GdkMacosCairoView, GdkMacosCairoSubView, GdkMacosGLView have all been removed and replaced with GdkMacosView. This new view has a single CALayer (GdkMacosLayer) attached to it which itself has sublayers. The contents of the CALayer is populated with an IOSurfaceRef which we allocated with the GdkMacosSurface. The surface is replaced when the NSWindow resizes. Changes to Layers ================= We now have a dedicated GdkMacosLayer which contains sublayers of GdkMacosTile. The tile has a maximum size of 128x128 pixels in device units. The GdkMacosTile is partitioned by splitting both the transparent region (window bounds minus opaque area) and then by splitting the opaque area. A tile has either translucent contents (and therefore is not opaque) or has opaque contents (and therefore is opaque). An opaque tile never contains transparent contents. As such, the opaque tiles contain a black background so that Core Animation will consider the tile's bounds as opaque. This can be verified with "Quartz Debug -> Show opaque regions". Changes to Cairo ================ GTK 4 cannot currently use cairo-quartz because of how CSS borders are rendered. It simply causes errors in the cairo_quartz_surface_t backend. Since we are restricted to using cairo_image_surface_t (which happens to be faster anyway) we can use the IOSurfaceBaseAddress() to obtain a mapping of the IOSurfaceRef in user-space. It always uses BGRA 32-bit with alpha channel even if we will discard the alpha channel as that is necessary to hit the fast paths in other parts of the platform. Note that while Cairo says CAIRO_FORMAT_ARGB32, it is really 32-bit BGRA on little-endian as we expect. OpenGL will render flipped (Quartz Native Co-ordinates) while Cairo renders with 0,O in the top-left. We could use cairo_translate() and cairo_scale() to reverse this, but it looks like some cairo things may not look quite as right if we do so. To reduce the chances of one-off bugs this continues to draw as Cairo would normally, but instead uses an CGAffineTransform in the tiles and some CGRect translation when swapping buffers to get the same effect. Changes to OpenGL ================= To simplify things, removal of all NSOpenGL* related components have been removed and we strictly use the Core GL (CGL*) API. This probably should have been done long ago anyay. Most examples found in the browsers to use IOSurfaceRef with OpenGL are using Legacy GL and there is still work underway to make this fit in with the rest of how the GSK GL renderer works. Since IOSurfaceRef bound to a texture/framebuffer will not have a default framebuffer ID of 0, we needed to add a default framebuffer id to the GdkGLContext. GskGLRenderer can use this to setup the command queue in such a way that our IOSurface destination has been glBindFramebuffer() as if it were the default drawable. This stuff is pretty slight-of-hand, so where things are and what needs flushing when and where has been a bit of an experiment to see what actually works to get synchronization across subsystems. Efficient Damages ================= After we draw with Cairo, we unlock the IOSurfaceRef and the contents are uploaded to the GPU. To make the contents visible to the app, we must clear the tiles contents with `layer.contents=nil;` and then re-apply the IOSurfaceRef. Since the buffer has likely not changed, we only do this if the tile overlaps the damage region. This gives the effect of having more tightly controlled damage regions even though updating the layer would damage be the whole window (as it is with OpenGL/Metal today with the exception of scissor-rect). This too can be verified usign "Quartz Debug -> Flash screen udpates". Frame Synchronized Resize ========================= In GTK 4, we have the ability to perform sizing changes from compute-size during the layout phase. Since the macOS backend already tracks window resizes manually, we can avoid doing the setFrame: immediately and instead do it within the frame clock's layout phase. Doing so gives us vastly better resize experience as we're more likely to get the size-change and updated-contents in the same frame on screen. It makes things feel "connected" in a way they weren't before. Some additional effort to tweak gravity during the process is also necessary but we were already doing that in the GTK 4 backend. Backporting =========== The design here has made an attempt to make it possible to backport by keeping GdkMacosBuffer, GdkMacosLayer, and GdkMacosTile fairly independent. There may be an opportunity to integrate this into GTK 3's quartz backend with a fair bit of work. Doing so could improve the situation for applications which are damage-rich such as The GIMP.
2022-02-14 10:20:19 +00:00
cairo_region_t *region)
macos: implement GL context This implements the basics for a GdkGLContext on macOS. Currently, rendering only is fully working for the GskCairoRenderer case where we read back pixels into a cairo surface for rendering. More work on synchronization is required for the GL on GskGLRenderer case. When we attempt to render a surface itself with GL, the context will ensure that the new GdkMacosGLView is placed within the NSWindow. In other cases, we use a dummy NSView and NSWindow for backing the NSOpenGLContext to ensure that we can get accelerated drawing. This gets GtkGLArea working when running with GSK_RENDERER=cairo.
2020-10-29 17:33:22 +00:00
{
GdkMacosGLContext *self = (GdkMacosGLContext *)context;
macos: use CALayer and IOSurface for rendering This provides a major shift in how we draw both when accelerated OpenGL as well as software rendering with Cairo. In short, it uses tiles of Core Animation's CALayer to display contents from an OpenGL or Cairo rendering so that the window can provide partial damage updates. Partial damage is not generally available when using OpenGL as the whole buffer is flipped even if you only submitted a small change using a scissor rect. Thankfully, this speeds up Cairo rendering a bit too by using IOSurface to upload contents to the display server. We use the tiling system we do for OpenGL which reduces overall complexity and differences between them. A New Buffer ============ GdkMacosBuffer is a wrapper around an IOSurfaceRef. The term buffer was used because 1) surface is already used and 2) it loosely maps to a front/back buffer semantic. However, it appears that IOSurfaceRef contents are being retained in some fashion (likely in the compositor result) so we can update the same IOSurfaceRef without flipping as long as we're fast. This appears to be what Chromium does as well, but Firefox uses two IOSurfaceRef and flips between them. We would like to avoid two surfaces because it doubles the GPU VRAM requirements of the application. Changes to Windows ================== Previously, the NSWindow would dynamically change between different types of NSView based on the renderer being used. This is no longer necessary as we just have a single NSView type, GdkMacosView, which inherits from GdkMacosBaseView just to keep the tedius stuff separate from the machinery of GdkMacosView. We can merge those someday if we are okay with that. Changes to Views ================ GdkMacosCairoView, GdkMacosCairoSubView, GdkMacosGLView have all been removed and replaced with GdkMacosView. This new view has a single CALayer (GdkMacosLayer) attached to it which itself has sublayers. The contents of the CALayer is populated with an IOSurfaceRef which we allocated with the GdkMacosSurface. The surface is replaced when the NSWindow resizes. Changes to Layers ================= We now have a dedicated GdkMacosLayer which contains sublayers of GdkMacosTile. The tile has a maximum size of 128x128 pixels in device units. The GdkMacosTile is partitioned by splitting both the transparent region (window bounds minus opaque area) and then by splitting the opaque area. A tile has either translucent contents (and therefore is not opaque) or has opaque contents (and therefore is opaque). An opaque tile never contains transparent contents. As such, the opaque tiles contain a black background so that Core Animation will consider the tile's bounds as opaque. This can be verified with "Quartz Debug -> Show opaque regions". Changes to Cairo ================ GTK 4 cannot currently use cairo-quartz because of how CSS borders are rendered. It simply causes errors in the cairo_quartz_surface_t backend. Since we are restricted to using cairo_image_surface_t (which happens to be faster anyway) we can use the IOSurfaceBaseAddress() to obtain a mapping of the IOSurfaceRef in user-space. It always uses BGRA 32-bit with alpha channel even if we will discard the alpha channel as that is necessary to hit the fast paths in other parts of the platform. Note that while Cairo says CAIRO_FORMAT_ARGB32, it is really 32-bit BGRA on little-endian as we expect. OpenGL will render flipped (Quartz Native Co-ordinates) while Cairo renders with 0,O in the top-left. We could use cairo_translate() and cairo_scale() to reverse this, but it looks like some cairo things may not look quite as right if we do so. To reduce the chances of one-off bugs this continues to draw as Cairo would normally, but instead uses an CGAffineTransform in the tiles and some CGRect translation when swapping buffers to get the same effect. Changes to OpenGL ================= To simplify things, removal of all NSOpenGL* related components have been removed and we strictly use the Core GL (CGL*) API. This probably should have been done long ago anyay. Most examples found in the browsers to use IOSurfaceRef with OpenGL are using Legacy GL and there is still work underway to make this fit in with the rest of how the GSK GL renderer works. Since IOSurfaceRef bound to a texture/framebuffer will not have a default framebuffer ID of 0, we needed to add a default framebuffer id to the GdkGLContext. GskGLRenderer can use this to setup the command queue in such a way that our IOSurface destination has been glBindFramebuffer() as if it were the default drawable. This stuff is pretty slight-of-hand, so where things are and what needs flushing when and where has been a bit of an experiment to see what actually works to get synchronization across subsystems. Efficient Damages ================= After we draw with Cairo, we unlock the IOSurfaceRef and the contents are uploaded to the GPU. To make the contents visible to the app, we must clear the tiles contents with `layer.contents=nil;` and then re-apply the IOSurfaceRef. Since the buffer has likely not changed, we only do this if the tile overlaps the damage region. This gives the effect of having more tightly controlled damage regions even though updating the layer would damage be the whole window (as it is with OpenGL/Metal today with the exception of scissor-rect). This too can be verified usign "Quartz Debug -> Flash screen udpates". Frame Synchronized Resize ========================= In GTK 4, we have the ability to perform sizing changes from compute-size during the layout phase. Since the macOS backend already tracks window resizes manually, we can avoid doing the setFrame: immediately and instead do it within the frame clock's layout phase. Doing so gives us vastly better resize experience as we're more likely to get the size-change and updated-contents in the same frame on screen. It makes things feel "connected" in a way they weren't before. Some additional effort to tweak gravity during the process is also necessary but we were already doing that in the GTK 4 backend. Backporting =========== The design here has made an attempt to make it possible to backport by keeping GdkMacosBuffer, GdkMacosLayer, and GdkMacosTile fairly independent. There may be an opportunity to integrate this into GTK 3's quartz backend with a fair bit of work. Doing so could improve the situation for applications which are damage-rich such as The GIMP.
2022-02-14 10:20:19 +00:00
GdkMacosBuffer *buffer;
cairo_region_t *copy;
macos: use CGLSetParameter and CGLEnable We don't need to go through the NSOpenGLContext for these. We can just use the C API directly. It's also clearer what is using CGLEnable() vs CGLSetParameter().
2020-12-07 19:47:51 +00:00
GdkSurface *surface;
macos: implement GL context This implements the basics for a GdkGLContext on macOS. Currently, rendering only is fully working for the GskCairoRenderer case where we read back pixels into a cairo surface for rendering. More work on synchronization is required for the GL on GskGLRenderer case. When we attempt to render a surface itself with GL, the context will ensure that the new GdkMacosGLView is placed within the NSWindow. In other cases, we use a dummy NSView and NSWindow for backing the NSOpenGLContext to ensure that we can get accelerated drawing. This gets GtkGLArea working when running with GSK_RENDERER=cairo.
2020-10-29 17:33:22 +00:00
g_assert (GDK_IS_MACOS_GL_CONTEXT (self));
macos: use CALayer and IOSurface for rendering This provides a major shift in how we draw both when accelerated OpenGL as well as software rendering with Cairo. In short, it uses tiles of Core Animation's CALayer to display contents from an OpenGL or Cairo rendering so that the window can provide partial damage updates. Partial damage is not generally available when using OpenGL as the whole buffer is flipped even if you only submitted a small change using a scissor rect. Thankfully, this speeds up Cairo rendering a bit too by using IOSurface to upload contents to the display server. We use the tiling system we do for OpenGL which reduces overall complexity and differences between them. A New Buffer ============ GdkMacosBuffer is a wrapper around an IOSurfaceRef. The term buffer was used because 1) surface is already used and 2) it loosely maps to a front/back buffer semantic. However, it appears that IOSurfaceRef contents are being retained in some fashion (likely in the compositor result) so we can update the same IOSurfaceRef without flipping as long as we're fast. This appears to be what Chromium does as well, but Firefox uses two IOSurfaceRef and flips between them. We would like to avoid two surfaces because it doubles the GPU VRAM requirements of the application. Changes to Windows ================== Previously, the NSWindow would dynamically change between different types of NSView based on the renderer being used. This is no longer necessary as we just have a single NSView type, GdkMacosView, which inherits from GdkMacosBaseView just to keep the tedius stuff separate from the machinery of GdkMacosView. We can merge those someday if we are okay with that. Changes to Views ================ GdkMacosCairoView, GdkMacosCairoSubView, GdkMacosGLView have all been removed and replaced with GdkMacosView. This new view has a single CALayer (GdkMacosLayer) attached to it which itself has sublayers. The contents of the CALayer is populated with an IOSurfaceRef which we allocated with the GdkMacosSurface. The surface is replaced when the NSWindow resizes. Changes to Layers ================= We now have a dedicated GdkMacosLayer which contains sublayers of GdkMacosTile. The tile has a maximum size of 128x128 pixels in device units. The GdkMacosTile is partitioned by splitting both the transparent region (window bounds minus opaque area) and then by splitting the opaque area. A tile has either translucent contents (and therefore is not opaque) or has opaque contents (and therefore is opaque). An opaque tile never contains transparent contents. As such, the opaque tiles contain a black background so that Core Animation will consider the tile's bounds as opaque. This can be verified with "Quartz Debug -> Show opaque regions". Changes to Cairo ================ GTK 4 cannot currently use cairo-quartz because of how CSS borders are rendered. It simply causes errors in the cairo_quartz_surface_t backend. Since we are restricted to using cairo_image_surface_t (which happens to be faster anyway) we can use the IOSurfaceBaseAddress() to obtain a mapping of the IOSurfaceRef in user-space. It always uses BGRA 32-bit with alpha channel even if we will discard the alpha channel as that is necessary to hit the fast paths in other parts of the platform. Note that while Cairo says CAIRO_FORMAT_ARGB32, it is really 32-bit BGRA on little-endian as we expect. OpenGL will render flipped (Quartz Native Co-ordinates) while Cairo renders with 0,O in the top-left. We could use cairo_translate() and cairo_scale() to reverse this, but it looks like some cairo things may not look quite as right if we do so. To reduce the chances of one-off bugs this continues to draw as Cairo would normally, but instead uses an CGAffineTransform in the tiles and some CGRect translation when swapping buffers to get the same effect. Changes to OpenGL ================= To simplify things, removal of all NSOpenGL* related components have been removed and we strictly use the Core GL (CGL*) API. This probably should have been done long ago anyay. Most examples found in the browsers to use IOSurfaceRef with OpenGL are using Legacy GL and there is still work underway to make this fit in with the rest of how the GSK GL renderer works. Since IOSurfaceRef bound to a texture/framebuffer will not have a default framebuffer ID of 0, we needed to add a default framebuffer id to the GdkGLContext. GskGLRenderer can use this to setup the command queue in such a way that our IOSurface destination has been glBindFramebuffer() as if it were the default drawable. This stuff is pretty slight-of-hand, so where things are and what needs flushing when and where has been a bit of an experiment to see what actually works to get synchronization across subsystems. Efficient Damages ================= After we draw with Cairo, we unlock the IOSurfaceRef and the contents are uploaded to the GPU. To make the contents visible to the app, we must clear the tiles contents with `layer.contents=nil;` and then re-apply the IOSurfaceRef. Since the buffer has likely not changed, we only do this if the tile overlaps the damage region. This gives the effect of having more tightly controlled damage regions even though updating the layer would damage be the whole window (as it is with OpenGL/Metal today with the exception of scissor-rect). This too can be verified usign "Quartz Debug -> Flash screen udpates". Frame Synchronized Resize ========================= In GTK 4, we have the ability to perform sizing changes from compute-size during the layout phase. Since the macOS backend already tracks window resizes manually, we can avoid doing the setFrame: immediately and instead do it within the frame clock's layout phase. Doing so gives us vastly better resize experience as we're more likely to get the size-change and updated-contents in the same frame on screen. It makes things feel "connected" in a way they weren't before. Some additional effort to tweak gravity during the process is also necessary but we were already doing that in the GTK 4 backend. Backporting =========== The design here has made an attempt to make it possible to backport by keeping GdkMacosBuffer, GdkMacosLayer, and GdkMacosTile fairly independent. There may be an opportunity to integrate this into GTK 3's quartz backend with a fair bit of work. Doing so could improve the situation for applications which are damage-rich such as The GIMP.
2022-02-14 10:20:19 +00:00
copy = cairo_region_copy (region);
macos: use CGLSetParameter and CGLEnable We don't need to go through the NSOpenGLContext for these. We can just use the C API directly. It's also clearer what is using CGLEnable() vs CGLSetParameter().
2020-12-07 19:47:51 +00:00
surface = gdk_draw_context_get_surface (context);
macos: use CALayer and IOSurface for rendering This provides a major shift in how we draw both when accelerated OpenGL as well as software rendering with Cairo. In short, it uses tiles of Core Animation's CALayer to display contents from an OpenGL or Cairo rendering so that the window can provide partial damage updates. Partial damage is not generally available when using OpenGL as the whole buffer is flipped even if you only submitted a small change using a scissor rect. Thankfully, this speeds up Cairo rendering a bit too by using IOSurface to upload contents to the display server. We use the tiling system we do for OpenGL which reduces overall complexity and differences between them. A New Buffer ============ GdkMacosBuffer is a wrapper around an IOSurfaceRef. The term buffer was used because 1) surface is already used and 2) it loosely maps to a front/back buffer semantic. However, it appears that IOSurfaceRef contents are being retained in some fashion (likely in the compositor result) so we can update the same IOSurfaceRef without flipping as long as we're fast. This appears to be what Chromium does as well, but Firefox uses two IOSurfaceRef and flips between them. We would like to avoid two surfaces because it doubles the GPU VRAM requirements of the application. Changes to Windows ================== Previously, the NSWindow would dynamically change between different types of NSView based on the renderer being used. This is no longer necessary as we just have a single NSView type, GdkMacosView, which inherits from GdkMacosBaseView just to keep the tedius stuff separate from the machinery of GdkMacosView. We can merge those someday if we are okay with that. Changes to Views ================ GdkMacosCairoView, GdkMacosCairoSubView, GdkMacosGLView have all been removed and replaced with GdkMacosView. This new view has a single CALayer (GdkMacosLayer) attached to it which itself has sublayers. The contents of the CALayer is populated with an IOSurfaceRef which we allocated with the GdkMacosSurface. The surface is replaced when the NSWindow resizes. Changes to Layers ================= We now have a dedicated GdkMacosLayer which contains sublayers of GdkMacosTile. The tile has a maximum size of 128x128 pixels in device units. The GdkMacosTile is partitioned by splitting both the transparent region (window bounds minus opaque area) and then by splitting the opaque area. A tile has either translucent contents (and therefore is not opaque) or has opaque contents (and therefore is opaque). An opaque tile never contains transparent contents. As such, the opaque tiles contain a black background so that Core Animation will consider the tile's bounds as opaque. This can be verified with "Quartz Debug -> Show opaque regions". Changes to Cairo ================ GTK 4 cannot currently use cairo-quartz because of how CSS borders are rendered. It simply causes errors in the cairo_quartz_surface_t backend. Since we are restricted to using cairo_image_surface_t (which happens to be faster anyway) we can use the IOSurfaceBaseAddress() to obtain a mapping of the IOSurfaceRef in user-space. It always uses BGRA 32-bit with alpha channel even if we will discard the alpha channel as that is necessary to hit the fast paths in other parts of the platform. Note that while Cairo says CAIRO_FORMAT_ARGB32, it is really 32-bit BGRA on little-endian as we expect. OpenGL will render flipped (Quartz Native Co-ordinates) while Cairo renders with 0,O in the top-left. We could use cairo_translate() and cairo_scale() to reverse this, but it looks like some cairo things may not look quite as right if we do so. To reduce the chances of one-off bugs this continues to draw as Cairo would normally, but instead uses an CGAffineTransform in the tiles and some CGRect translation when swapping buffers to get the same effect. Changes to OpenGL ================= To simplify things, removal of all NSOpenGL* related components have been removed and we strictly use the Core GL (CGL*) API. This probably should have been done long ago anyay. Most examples found in the browsers to use IOSurfaceRef with OpenGL are using Legacy GL and there is still work underway to make this fit in with the rest of how the GSK GL renderer works. Since IOSurfaceRef bound to a texture/framebuffer will not have a default framebuffer ID of 0, we needed to add a default framebuffer id to the GdkGLContext. GskGLRenderer can use this to setup the command queue in such a way that our IOSurface destination has been glBindFramebuffer() as if it were the default drawable. This stuff is pretty slight-of-hand, so where things are and what needs flushing when and where has been a bit of an experiment to see what actually works to get synchronization across subsystems. Efficient Damages ================= After we draw with Cairo, we unlock the IOSurfaceRef and the contents are uploaded to the GPU. To make the contents visible to the app, we must clear the tiles contents with `layer.contents=nil;` and then re-apply the IOSurfaceRef. Since the buffer has likely not changed, we only do this if the tile overlaps the damage region. This gives the effect of having more tightly controlled damage regions even though updating the layer would damage be the whole window (as it is with OpenGL/Metal today with the exception of scissor-rect). This too can be verified usign "Quartz Debug -> Flash screen udpates". Frame Synchronized Resize ========================= In GTK 4, we have the ability to perform sizing changes from compute-size during the layout phase. Since the macOS backend already tracks window resizes manually, we can avoid doing the setFrame: immediately and instead do it within the frame clock's layout phase. Doing so gives us vastly better resize experience as we're more likely to get the size-change and updated-contents in the same frame on screen. It makes things feel "connected" in a way they weren't before. Some additional effort to tweak gravity during the process is also necessary but we were already doing that in the GTK 4 backend. Backporting =========== The design here has made an attempt to make it possible to backport by keeping GdkMacosBuffer, GdkMacosLayer, and GdkMacosTile fairly independent. There may be an opportunity to integrate this into GTK 3's quartz backend with a fair bit of work. Doing so could improve the situation for applications which are damage-rich such as The GIMP.
2022-02-14 10:20:19 +00:00
buffer = _gdk_macos_surface_get_buffer (GDK_MACOS_SURFACE (surface));
macos: use CGLSetParameter and CGLEnable We don't need to go through the NSOpenGLContext for these. We can just use the C API directly. It's also clearer what is using CGLEnable() vs CGLSetParameter().
2020-12-07 19:47:51 +00:00
macos: use CALayer and IOSurface for rendering This provides a major shift in how we draw both when accelerated OpenGL as well as software rendering with Cairo. In short, it uses tiles of Core Animation's CALayer to display contents from an OpenGL or Cairo rendering so that the window can provide partial damage updates. Partial damage is not generally available when using OpenGL as the whole buffer is flipped even if you only submitted a small change using a scissor rect. Thankfully, this speeds up Cairo rendering a bit too by using IOSurface to upload contents to the display server. We use the tiling system we do for OpenGL which reduces overall complexity and differences between them. A New Buffer ============ GdkMacosBuffer is a wrapper around an IOSurfaceRef. The term buffer was used because 1) surface is already used and 2) it loosely maps to a front/back buffer semantic. However, it appears that IOSurfaceRef contents are being retained in some fashion (likely in the compositor result) so we can update the same IOSurfaceRef without flipping as long as we're fast. This appears to be what Chromium does as well, but Firefox uses two IOSurfaceRef and flips between them. We would like to avoid two surfaces because it doubles the GPU VRAM requirements of the application. Changes to Windows ================== Previously, the NSWindow would dynamically change between different types of NSView based on the renderer being used. This is no longer necessary as we just have a single NSView type, GdkMacosView, which inherits from GdkMacosBaseView just to keep the tedius stuff separate from the machinery of GdkMacosView. We can merge those someday if we are okay with that. Changes to Views ================ GdkMacosCairoView, GdkMacosCairoSubView, GdkMacosGLView have all been removed and replaced with GdkMacosView. This new view has a single CALayer (GdkMacosLayer) attached to it which itself has sublayers. The contents of the CALayer is populated with an IOSurfaceRef which we allocated with the GdkMacosSurface. The surface is replaced when the NSWindow resizes. Changes to Layers ================= We now have a dedicated GdkMacosLayer which contains sublayers of GdkMacosTile. The tile has a maximum size of 128x128 pixels in device units. The GdkMacosTile is partitioned by splitting both the transparent region (window bounds minus opaque area) and then by splitting the opaque area. A tile has either translucent contents (and therefore is not opaque) or has opaque contents (and therefore is opaque). An opaque tile never contains transparent contents. As such, the opaque tiles contain a black background so that Core Animation will consider the tile's bounds as opaque. This can be verified with "Quartz Debug -> Show opaque regions". Changes to Cairo ================ GTK 4 cannot currently use cairo-quartz because of how CSS borders are rendered. It simply causes errors in the cairo_quartz_surface_t backend. Since we are restricted to using cairo_image_surface_t (which happens to be faster anyway) we can use the IOSurfaceBaseAddress() to obtain a mapping of the IOSurfaceRef in user-space. It always uses BGRA 32-bit with alpha channel even if we will discard the alpha channel as that is necessary to hit the fast paths in other parts of the platform. Note that while Cairo says CAIRO_FORMAT_ARGB32, it is really 32-bit BGRA on little-endian as we expect. OpenGL will render flipped (Quartz Native Co-ordinates) while Cairo renders with 0,O in the top-left. We could use cairo_translate() and cairo_scale() to reverse this, but it looks like some cairo things may not look quite as right if we do so. To reduce the chances of one-off bugs this continues to draw as Cairo would normally, but instead uses an CGAffineTransform in the tiles and some CGRect translation when swapping buffers to get the same effect. Changes to OpenGL ================= To simplify things, removal of all NSOpenGL* related components have been removed and we strictly use the Core GL (CGL*) API. This probably should have been done long ago anyay. Most examples found in the browsers to use IOSurfaceRef with OpenGL are using Legacy GL and there is still work underway to make this fit in with the rest of how the GSK GL renderer works. Since IOSurfaceRef bound to a texture/framebuffer will not have a default framebuffer ID of 0, we needed to add a default framebuffer id to the GdkGLContext. GskGLRenderer can use this to setup the command queue in such a way that our IOSurface destination has been glBindFramebuffer() as if it were the default drawable. This stuff is pretty slight-of-hand, so where things are and what needs flushing when and where has been a bit of an experiment to see what actually works to get synchronization across subsystems. Efficient Damages ================= After we draw with Cairo, we unlock the IOSurfaceRef and the contents are uploaded to the GPU. To make the contents visible to the app, we must clear the tiles contents with `layer.contents=nil;` and then re-apply the IOSurfaceRef. Since the buffer has likely not changed, we only do this if the tile overlaps the damage region. This gives the effect of having more tightly controlled damage regions even though updating the layer would damage be the whole window (as it is with OpenGL/Metal today with the exception of scissor-rect). This too can be verified usign "Quartz Debug -> Flash screen udpates". Frame Synchronized Resize ========================= In GTK 4, we have the ability to perform sizing changes from compute-size during the layout phase. Since the macOS backend already tracks window resizes manually, we can avoid doing the setFrame: immediately and instead do it within the frame clock's layout phase. Doing so gives us vastly better resize experience as we're more likely to get the size-change and updated-contents in the same frame on screen. It makes things feel "connected" in a way they weren't before. Some additional effort to tweak gravity during the process is also necessary but we were already doing that in the GTK 4 backend. Backporting =========== The design here has made an attempt to make it possible to backport by keeping GdkMacosBuffer, GdkMacosLayer, and GdkMacosTile fairly independent. There may be an opportunity to integrate this into GTK 3's quartz backend with a fair bit of work. Doing so could improve the situation for applications which are damage-rich such as The GIMP.
2022-02-14 10:20:19 +00:00
_gdk_macos_buffer_set_flipped (buffer, TRUE);
macos: use opaque GL context when possible This is better for situations where the window is decorated and therefore we can rely on window system clipping of rounded corners.
2020-11-05 21:34:29 +00:00
macos: use CALayer and IOSurface for rendering This provides a major shift in how we draw both when accelerated OpenGL as well as software rendering with Cairo. In short, it uses tiles of Core Animation's CALayer to display contents from an OpenGL or Cairo rendering so that the window can provide partial damage updates. Partial damage is not generally available when using OpenGL as the whole buffer is flipped even if you only submitted a small change using a scissor rect. Thankfully, this speeds up Cairo rendering a bit too by using IOSurface to upload contents to the display server. We use the tiling system we do for OpenGL which reduces overall complexity and differences between them. A New Buffer ============ GdkMacosBuffer is a wrapper around an IOSurfaceRef. The term buffer was used because 1) surface is already used and 2) it loosely maps to a front/back buffer semantic. However, it appears that IOSurfaceRef contents are being retained in some fashion (likely in the compositor result) so we can update the same IOSurfaceRef without flipping as long as we're fast. This appears to be what Chromium does as well, but Firefox uses two IOSurfaceRef and flips between them. We would like to avoid two surfaces because it doubles the GPU VRAM requirements of the application. Changes to Windows ================== Previously, the NSWindow would dynamically change between different types of NSView based on the renderer being used. This is no longer necessary as we just have a single NSView type, GdkMacosView, which inherits from GdkMacosBaseView just to keep the tedius stuff separate from the machinery of GdkMacosView. We can merge those someday if we are okay with that. Changes to Views ================ GdkMacosCairoView, GdkMacosCairoSubView, GdkMacosGLView have all been removed and replaced with GdkMacosView. This new view has a single CALayer (GdkMacosLayer) attached to it which itself has sublayers. The contents of the CALayer is populated with an IOSurfaceRef which we allocated with the GdkMacosSurface. The surface is replaced when the NSWindow resizes. Changes to Layers ================= We now have a dedicated GdkMacosLayer which contains sublayers of GdkMacosTile. The tile has a maximum size of 128x128 pixels in device units. The GdkMacosTile is partitioned by splitting both the transparent region (window bounds minus opaque area) and then by splitting the opaque area. A tile has either translucent contents (and therefore is not opaque) or has opaque contents (and therefore is opaque). An opaque tile never contains transparent contents. As such, the opaque tiles contain a black background so that Core Animation will consider the tile's bounds as opaque. This can be verified with "Quartz Debug -> Show opaque regions". Changes to Cairo ================ GTK 4 cannot currently use cairo-quartz because of how CSS borders are rendered. It simply causes errors in the cairo_quartz_surface_t backend. Since we are restricted to using cairo_image_surface_t (which happens to be faster anyway) we can use the IOSurfaceBaseAddress() to obtain a mapping of the IOSurfaceRef in user-space. It always uses BGRA 32-bit with alpha channel even if we will discard the alpha channel as that is necessary to hit the fast paths in other parts of the platform. Note that while Cairo says CAIRO_FORMAT_ARGB32, it is really 32-bit BGRA on little-endian as we expect. OpenGL will render flipped (Quartz Native Co-ordinates) while Cairo renders with 0,O in the top-left. We could use cairo_translate() and cairo_scale() to reverse this, but it looks like some cairo things may not look quite as right if we do so. To reduce the chances of one-off bugs this continues to draw as Cairo would normally, but instead uses an CGAffineTransform in the tiles and some CGRect translation when swapping buffers to get the same effect. Changes to OpenGL ================= To simplify things, removal of all NSOpenGL* related components have been removed and we strictly use the Core GL (CGL*) API. This probably should have been done long ago anyay. Most examples found in the browsers to use IOSurfaceRef with OpenGL are using Legacy GL and there is still work underway to make this fit in with the rest of how the GSK GL renderer works. Since IOSurfaceRef bound to a texture/framebuffer will not have a default framebuffer ID of 0, we needed to add a default framebuffer id to the GdkGLContext. GskGLRenderer can use this to setup the command queue in such a way that our IOSurface destination has been glBindFramebuffer() as if it were the default drawable. This stuff is pretty slight-of-hand, so where things are and what needs flushing when and where has been a bit of an experiment to see what actually works to get synchronization across subsystems. Efficient Damages ================= After we draw with Cairo, we unlock the IOSurfaceRef and the contents are uploaded to the GPU. To make the contents visible to the app, we must clear the tiles contents with `layer.contents=nil;` and then re-apply the IOSurfaceRef. Since the buffer has likely not changed, we only do this if the tile overlaps the damage region. This gives the effect of having more tightly controlled damage regions even though updating the layer would damage be the whole window (as it is with OpenGL/Metal today with the exception of scissor-rect). This too can be verified usign "Quartz Debug -> Flash screen udpates". Frame Synchronized Resize ========================= In GTK 4, we have the ability to perform sizing changes from compute-size during the layout phase. Since the macOS backend already tracks window resizes manually, we can avoid doing the setFrame: immediately and instead do it within the frame clock's layout phase. Doing so gives us vastly better resize experience as we're more likely to get the size-change and updated-contents in the same frame on screen. It makes things feel "connected" in a way they weren't before. Some additional effort to tweak gravity during the process is also necessary but we were already doing that in the GTK 4 backend. Backporting =========== The design here has made an attempt to make it possible to backport by keeping GdkMacosBuffer, GdkMacosLayer, and GdkMacosTile fairly independent. There may be an opportunity to integrate this into GTK 3's quartz backend with a fair bit of work. Doing so could improve the situation for applications which are damage-rich such as The GIMP.
2022-02-14 10:20:19 +00:00
/* Create our render target and bind it */
gdk_gl_context_make_current (GDK_GL_CONTEXT (self));
gdk_macos_gl_context_allocate (self);
macos: implement GL context This implements the basics for a GdkGLContext on macOS. Currently, rendering only is fully working for the GskCairoRenderer case where we read back pixels into a cairo surface for rendering. More work on synchronization is required for the GL on GskGLRenderer case. When we attempt to render a surface itself with GL, the context will ensure that the new GdkMacosGLView is placed within the NSWindow. In other cases, we use a dummy NSView and NSWindow for backing the NSOpenGLContext to ensure that we can get accelerated drawing. This gets GtkGLArea working when running with GSK_RENDERER=cairo.
2020-10-29 17:33:22 +00:00
macos: use CALayer and IOSurface for rendering This provides a major shift in how we draw both when accelerated OpenGL as well as software rendering with Cairo. In short, it uses tiles of Core Animation's CALayer to display contents from an OpenGL or Cairo rendering so that the window can provide partial damage updates. Partial damage is not generally available when using OpenGL as the whole buffer is flipped even if you only submitted a small change using a scissor rect. Thankfully, this speeds up Cairo rendering a bit too by using IOSurface to upload contents to the display server. We use the tiling system we do for OpenGL which reduces overall complexity and differences between them. A New Buffer ============ GdkMacosBuffer is a wrapper around an IOSurfaceRef. The term buffer was used because 1) surface is already used and 2) it loosely maps to a front/back buffer semantic. However, it appears that IOSurfaceRef contents are being retained in some fashion (likely in the compositor result) so we can update the same IOSurfaceRef without flipping as long as we're fast. This appears to be what Chromium does as well, but Firefox uses two IOSurfaceRef and flips between them. We would like to avoid two surfaces because it doubles the GPU VRAM requirements of the application. Changes to Windows ================== Previously, the NSWindow would dynamically change between different types of NSView based on the renderer being used. This is no longer necessary as we just have a single NSView type, GdkMacosView, which inherits from GdkMacosBaseView just to keep the tedius stuff separate from the machinery of GdkMacosView. We can merge those someday if we are okay with that. Changes to Views ================ GdkMacosCairoView, GdkMacosCairoSubView, GdkMacosGLView have all been removed and replaced with GdkMacosView. This new view has a single CALayer (GdkMacosLayer) attached to it which itself has sublayers. The contents of the CALayer is populated with an IOSurfaceRef which we allocated with the GdkMacosSurface. The surface is replaced when the NSWindow resizes. Changes to Layers ================= We now have a dedicated GdkMacosLayer which contains sublayers of GdkMacosTile. The tile has a maximum size of 128x128 pixels in device units. The GdkMacosTile is partitioned by splitting both the transparent region (window bounds minus opaque area) and then by splitting the opaque area. A tile has either translucent contents (and therefore is not opaque) or has opaque contents (and therefore is opaque). An opaque tile never contains transparent contents. As such, the opaque tiles contain a black background so that Core Animation will consider the tile's bounds as opaque. This can be verified with "Quartz Debug -> Show opaque regions". Changes to Cairo ================ GTK 4 cannot currently use cairo-quartz because of how CSS borders are rendered. It simply causes errors in the cairo_quartz_surface_t backend. Since we are restricted to using cairo_image_surface_t (which happens to be faster anyway) we can use the IOSurfaceBaseAddress() to obtain a mapping of the IOSurfaceRef in user-space. It always uses BGRA 32-bit with alpha channel even if we will discard the alpha channel as that is necessary to hit the fast paths in other parts of the platform. Note that while Cairo says CAIRO_FORMAT_ARGB32, it is really 32-bit BGRA on little-endian as we expect. OpenGL will render flipped (Quartz Native Co-ordinates) while Cairo renders with 0,O in the top-left. We could use cairo_translate() and cairo_scale() to reverse this, but it looks like some cairo things may not look quite as right if we do so. To reduce the chances of one-off bugs this continues to draw as Cairo would normally, but instead uses an CGAffineTransform in the tiles and some CGRect translation when swapping buffers to get the same effect. Changes to OpenGL ================= To simplify things, removal of all NSOpenGL* related components have been removed and we strictly use the Core GL (CGL*) API. This probably should have been done long ago anyay. Most examples found in the browsers to use IOSurfaceRef with OpenGL are using Legacy GL and there is still work underway to make this fit in with the rest of how the GSK GL renderer works. Since IOSurfaceRef bound to a texture/framebuffer will not have a default framebuffer ID of 0, we needed to add a default framebuffer id to the GdkGLContext. GskGLRenderer can use this to setup the command queue in such a way that our IOSurface destination has been glBindFramebuffer() as if it were the default drawable. This stuff is pretty slight-of-hand, so where things are and what needs flushing when and where has been a bit of an experiment to see what actually works to get synchronization across subsystems. Efficient Damages ================= After we draw with Cairo, we unlock the IOSurfaceRef and the contents are uploaded to the GPU. To make the contents visible to the app, we must clear the tiles contents with `layer.contents=nil;` and then re-apply the IOSurfaceRef. Since the buffer has likely not changed, we only do this if the tile overlaps the damage region. This gives the effect of having more tightly controlled damage regions even though updating the layer would damage be the whole window (as it is with OpenGL/Metal today with the exception of scissor-rect). This too can be verified usign "Quartz Debug -> Flash screen udpates". Frame Synchronized Resize ========================= In GTK 4, we have the ability to perform sizing changes from compute-size during the layout phase. Since the macOS backend already tracks window resizes manually, we can avoid doing the setFrame: immediately and instead do it within the frame clock's layout phase. Doing so gives us vastly better resize experience as we're more likely to get the size-change and updated-contents in the same frame on screen. It makes things feel "connected" in a way they weren't before. Some additional effort to tweak gravity during the process is also necessary but we were already doing that in the GTK 4 backend. Backporting =========== The design here has made an attempt to make it possible to backport by keeping GdkMacosBuffer, GdkMacosLayer, and GdkMacosTile fairly independent. There may be an opportunity to integrate this into GTK 3's quartz backend with a fair bit of work. Doing so could improve the situation for applications which are damage-rich such as The GIMP.
2022-02-14 10:20:19 +00:00
GDK_DRAW_CONTEXT_CLASS (gdk_macos_gl_context_parent_class)->begin_frame (context, prefers_high_depth, region);
macos: use CGLSetParameter and CGLEnable We don't need to go through the NSOpenGLContext for these. We can just use the C API directly. It's also clearer what is using CGLEnable() vs CGLSetParameter().
2020-12-07 19:47:51 +00:00
macos: use CALayer and IOSurface for rendering This provides a major shift in how we draw both when accelerated OpenGL as well as software rendering with Cairo. In short, it uses tiles of Core Animation's CALayer to display contents from an OpenGL or Cairo rendering so that the window can provide partial damage updates. Partial damage is not generally available when using OpenGL as the whole buffer is flipped even if you only submitted a small change using a scissor rect. Thankfully, this speeds up Cairo rendering a bit too by using IOSurface to upload contents to the display server. We use the tiling system we do for OpenGL which reduces overall complexity and differences between them. A New Buffer ============ GdkMacosBuffer is a wrapper around an IOSurfaceRef. The term buffer was used because 1) surface is already used and 2) it loosely maps to a front/back buffer semantic. However, it appears that IOSurfaceRef contents are being retained in some fashion (likely in the compositor result) so we can update the same IOSurfaceRef without flipping as long as we're fast. This appears to be what Chromium does as well, but Firefox uses two IOSurfaceRef and flips between them. We would like to avoid two surfaces because it doubles the GPU VRAM requirements of the application. Changes to Windows ================== Previously, the NSWindow would dynamically change between different types of NSView based on the renderer being used. This is no longer necessary as we just have a single NSView type, GdkMacosView, which inherits from GdkMacosBaseView just to keep the tedius stuff separate from the machinery of GdkMacosView. We can merge those someday if we are okay with that. Changes to Views ================ GdkMacosCairoView, GdkMacosCairoSubView, GdkMacosGLView have all been removed and replaced with GdkMacosView. This new view has a single CALayer (GdkMacosLayer) attached to it which itself has sublayers. The contents of the CALayer is populated with an IOSurfaceRef which we allocated with the GdkMacosSurface. The surface is replaced when the NSWindow resizes. Changes to Layers ================= We now have a dedicated GdkMacosLayer which contains sublayers of GdkMacosTile. The tile has a maximum size of 128x128 pixels in device units. The GdkMacosTile is partitioned by splitting both the transparent region (window bounds minus opaque area) and then by splitting the opaque area. A tile has either translucent contents (and therefore is not opaque) or has opaque contents (and therefore is opaque). An opaque tile never contains transparent contents. As such, the opaque tiles contain a black background so that Core Animation will consider the tile's bounds as opaque. This can be verified with "Quartz Debug -> Show opaque regions". Changes to Cairo ================ GTK 4 cannot currently use cairo-quartz because of how CSS borders are rendered. It simply causes errors in the cairo_quartz_surface_t backend. Since we are restricted to using cairo_image_surface_t (which happens to be faster anyway) we can use the IOSurfaceBaseAddress() to obtain a mapping of the IOSurfaceRef in user-space. It always uses BGRA 32-bit with alpha channel even if we will discard the alpha channel as that is necessary to hit the fast paths in other parts of the platform. Note that while Cairo says CAIRO_FORMAT_ARGB32, it is really 32-bit BGRA on little-endian as we expect. OpenGL will render flipped (Quartz Native Co-ordinates) while Cairo renders with 0,O in the top-left. We could use cairo_translate() and cairo_scale() to reverse this, but it looks like some cairo things may not look quite as right if we do so. To reduce the chances of one-off bugs this continues to draw as Cairo would normally, but instead uses an CGAffineTransform in the tiles and some CGRect translation when swapping buffers to get the same effect. Changes to OpenGL ================= To simplify things, removal of all NSOpenGL* related components have been removed and we strictly use the Core GL (CGL*) API. This probably should have been done long ago anyay. Most examples found in the browsers to use IOSurfaceRef with OpenGL are using Legacy GL and there is still work underway to make this fit in with the rest of how the GSK GL renderer works. Since IOSurfaceRef bound to a texture/framebuffer will not have a default framebuffer ID of 0, we needed to add a default framebuffer id to the GdkGLContext. GskGLRenderer can use this to setup the command queue in such a way that our IOSurface destination has been glBindFramebuffer() as if it were the default drawable. This stuff is pretty slight-of-hand, so where things are and what needs flushing when and where has been a bit of an experiment to see what actually works to get synchronization across subsystems. Efficient Damages ================= After we draw with Cairo, we unlock the IOSurfaceRef and the contents are uploaded to the GPU. To make the contents visible to the app, we must clear the tiles contents with `layer.contents=nil;` and then re-apply the IOSurfaceRef. Since the buffer has likely not changed, we only do this if the tile overlaps the damage region. This gives the effect of having more tightly controlled damage regions even though updating the layer would damage be the whole window (as it is with OpenGL/Metal today with the exception of scissor-rect). This too can be verified usign "Quartz Debug -> Flash screen udpates". Frame Synchronized Resize ========================= In GTK 4, we have the ability to perform sizing changes from compute-size during the layout phase. Since the macOS backend already tracks window resizes manually, we can avoid doing the setFrame: immediately and instead do it within the frame clock's layout phase. Doing so gives us vastly better resize experience as we're more likely to get the size-change and updated-contents in the same frame on screen. It makes things feel "connected" in a way they weren't before. Some additional effort to tweak gravity during the process is also necessary but we were already doing that in the GTK 4 backend. Backporting =========== The design here has made an attempt to make it possible to backport by keeping GdkMacosBuffer, GdkMacosLayer, and GdkMacosTile fairly independent. There may be an opportunity to integrate this into GTK 3's quartz backend with a fair bit of work. Doing so could improve the situation for applications which are damage-rich such as The GIMP.
2022-02-14 10:20:19 +00:00
g_clear_pointer (&self->damage, cairo_region_destroy);
self->damage = g_steal_pointer (&copy);
macos: implement GL context This implements the basics for a GdkGLContext on macOS. Currently, rendering only is fully working for the GskCairoRenderer case where we read back pixels into a cairo surface for rendering. More work on synchronization is required for the GL on GskGLRenderer case. When we attempt to render a surface itself with GL, the context will ensure that the new GdkMacosGLView is placed within the NSWindow. In other cases, we use a dummy NSView and NSWindow for backing the NSOpenGLContext to ensure that we can get accelerated drawing. This gets GtkGLArea working when running with GSK_RENDERER=cairo.
2020-10-29 17:33:22 +00:00
macos: use CALayer and IOSurface for rendering This provides a major shift in how we draw both when accelerated OpenGL as well as software rendering with Cairo. In short, it uses tiles of Core Animation's CALayer to display contents from an OpenGL or Cairo rendering so that the window can provide partial damage updates. Partial damage is not generally available when using OpenGL as the whole buffer is flipped even if you only submitted a small change using a scissor rect. Thankfully, this speeds up Cairo rendering a bit too by using IOSurface to upload contents to the display server. We use the tiling system we do for OpenGL which reduces overall complexity and differences between them. A New Buffer ============ GdkMacosBuffer is a wrapper around an IOSurfaceRef. The term buffer was used because 1) surface is already used and 2) it loosely maps to a front/back buffer semantic. However, it appears that IOSurfaceRef contents are being retained in some fashion (likely in the compositor result) so we can update the same IOSurfaceRef without flipping as long as we're fast. This appears to be what Chromium does as well, but Firefox uses two IOSurfaceRef and flips between them. We would like to avoid two surfaces because it doubles the GPU VRAM requirements of the application. Changes to Windows ================== Previously, the NSWindow would dynamically change between different types of NSView based on the renderer being used. This is no longer necessary as we just have a single NSView type, GdkMacosView, which inherits from GdkMacosBaseView just to keep the tedius stuff separate from the machinery of GdkMacosView. We can merge those someday if we are okay with that. Changes to Views ================ GdkMacosCairoView, GdkMacosCairoSubView, GdkMacosGLView have all been removed and replaced with GdkMacosView. This new view has a single CALayer (GdkMacosLayer) attached to it which itself has sublayers. The contents of the CALayer is populated with an IOSurfaceRef which we allocated with the GdkMacosSurface. The surface is replaced when the NSWindow resizes. Changes to Layers ================= We now have a dedicated GdkMacosLayer which contains sublayers of GdkMacosTile. The tile has a maximum size of 128x128 pixels in device units. The GdkMacosTile is partitioned by splitting both the transparent region (window bounds minus opaque area) and then by splitting the opaque area. A tile has either translucent contents (and therefore is not opaque) or has opaque contents (and therefore is opaque). An opaque tile never contains transparent contents. As such, the opaque tiles contain a black background so that Core Animation will consider the tile's bounds as opaque. This can be verified with "Quartz Debug -> Show opaque regions". Changes to Cairo ================ GTK 4 cannot currently use cairo-quartz because of how CSS borders are rendered. It simply causes errors in the cairo_quartz_surface_t backend. Since we are restricted to using cairo_image_surface_t (which happens to be faster anyway) we can use the IOSurfaceBaseAddress() to obtain a mapping of the IOSurfaceRef in user-space. It always uses BGRA 32-bit with alpha channel even if we will discard the alpha channel as that is necessary to hit the fast paths in other parts of the platform. Note that while Cairo says CAIRO_FORMAT_ARGB32, it is really 32-bit BGRA on little-endian as we expect. OpenGL will render flipped (Quartz Native Co-ordinates) while Cairo renders with 0,O in the top-left. We could use cairo_translate() and cairo_scale() to reverse this, but it looks like some cairo things may not look quite as right if we do so. To reduce the chances of one-off bugs this continues to draw as Cairo would normally, but instead uses an CGAffineTransform in the tiles and some CGRect translation when swapping buffers to get the same effect. Changes to OpenGL ================= To simplify things, removal of all NSOpenGL* related components have been removed and we strictly use the Core GL (CGL*) API. This probably should have been done long ago anyay. Most examples found in the browsers to use IOSurfaceRef with OpenGL are using Legacy GL and there is still work underway to make this fit in with the rest of how the GSK GL renderer works. Since IOSurfaceRef bound to a texture/framebuffer will not have a default framebuffer ID of 0, we needed to add a default framebuffer id to the GdkGLContext. GskGLRenderer can use this to setup the command queue in such a way that our IOSurface destination has been glBindFramebuffer() as if it were the default drawable. This stuff is pretty slight-of-hand, so where things are and what needs flushing when and where has been a bit of an experiment to see what actually works to get synchronization across subsystems. Efficient Damages ================= After we draw with Cairo, we unlock the IOSurfaceRef and the contents are uploaded to the GPU. To make the contents visible to the app, we must clear the tiles contents with `layer.contents=nil;` and then re-apply the IOSurfaceRef. Since the buffer has likely not changed, we only do this if the tile overlaps the damage region. This gives the effect of having more tightly controlled damage regions even though updating the layer would damage be the whole window (as it is with OpenGL/Metal today with the exception of scissor-rect). This too can be verified usign "Quartz Debug -> Flash screen udpates". Frame Synchronized Resize ========================= In GTK 4, we have the ability to perform sizing changes from compute-size during the layout phase. Since the macOS backend already tracks window resizes manually, we can avoid doing the setFrame: immediately and instead do it within the frame clock's layout phase. Doing so gives us vastly better resize experience as we're more likely to get the size-change and updated-contents in the same frame on screen. It makes things feel "connected" in a way they weren't before. Some additional effort to tweak gravity during the process is also necessary but we were already doing that in the GTK 4 backend. Backporting =========== The design here has made an attempt to make it possible to backport by keeping GdkMacosBuffer, GdkMacosLayer, and GdkMacosTile fairly independent. There may be an opportunity to integrate this into GTK 3's quartz backend with a fair bit of work. Doing so could improve the situation for applications which are damage-rich such as The GIMP.
2022-02-14 10:20:19 +00:00
gdk_gl_context_make_current (GDK_GL_CONTEXT (self));
CHECK_GL (NULL, glBindFramebuffer (GL_FRAMEBUFFER, self->fbo));
macos: implement GL context This implements the basics for a GdkGLContext on macOS. Currently, rendering only is fully working for the GskCairoRenderer case where we read back pixels into a cairo surface for rendering. More work on synchronization is required for the GL on GskGLRenderer case. When we attempt to render a surface itself with GL, the context will ensure that the new GdkMacosGLView is placed within the NSWindow. In other cases, we use a dummy NSView and NSWindow for backing the NSOpenGLContext to ensure that we can get accelerated drawing. This gets GtkGLArea working when running with GSK_RENDERER=cairo.
2020-10-29 17:33:22 +00:00
}
macos: prototype new GDK backend for macOS This is fairly substantial rewrite of the GDK backend for quartz and renamed to macOS to allow for a greenfield implementation. Many things have come across from the quartz implementation fairly intact such as the eventloop integration design and discovery of event windows from the NSEvent. However much has been changed to fit in with the new GDK design and how removal of child GdkWindow have been completely eliminated. Furthermore, the new GdkPopup allows for regular NSWindow to be used to provide popovers unlike the previous implementation. The object design more closely follows the ideal for a GDK backend. Views have been broken out into subclasses so that we can support multiple GSK renderer paths such as GL and Cairo (and Metal in the future). However mixed mode GL and Cairo will not be supported. Currently only the Cairo renderer has been implemented. A new frame clock implementation using CVDisplayLink provides more accurate information about when to draw drawing the next frame. Some testing will need to be done here to understand the power implications of this. This implementation has also gained edge snapping for CSD windows. Some work was also done to ensure that CSD windows have opaque regions registered with the display server. ** This is still very much a work-in-progress ** Some outstanding work that needs to be done: - Finish a GL context for macOS and alternate NSView for GL rendering (possibly using speciailized CALayer for OpenGL). - Input rework to ensure that we don't loose remapping of keys that was dropped from GDK during GTK 4 development. - Make sure input methods continue to work. - Drag-n-Drop is still very much a work in progress - High resolution input scrolling needs various work in GDK to land first before we can plumb that to NSEvent. - gtk/ has a number of things based on GDK_WINDOWING_QUARTZ that need to be updated to use the macOS backend. But this is good enough to start playing with and breaking things which is what I'd like to see.
2020-04-23 23:36:46 +00:00
static void
gdk_macos_gl_context_end_frame (GdkDrawContext *context,
cairo_region_t *painted)
{
GdkMacosGLContext *self = GDK_MACOS_GL_CONTEXT (context);
macos: use CALayer and IOSurface for rendering This provides a major shift in how we draw both when accelerated OpenGL as well as software rendering with Cairo. In short, it uses tiles of Core Animation's CALayer to display contents from an OpenGL or Cairo rendering so that the window can provide partial damage updates. Partial damage is not generally available when using OpenGL as the whole buffer is flipped even if you only submitted a small change using a scissor rect. Thankfully, this speeds up Cairo rendering a bit too by using IOSurface to upload contents to the display server. We use the tiling system we do for OpenGL which reduces overall complexity and differences between them. A New Buffer ============ GdkMacosBuffer is a wrapper around an IOSurfaceRef. The term buffer was used because 1) surface is already used and 2) it loosely maps to a front/back buffer semantic. However, it appears that IOSurfaceRef contents are being retained in some fashion (likely in the compositor result) so we can update the same IOSurfaceRef without flipping as long as we're fast. This appears to be what Chromium does as well, but Firefox uses two IOSurfaceRef and flips between them. We would like to avoid two surfaces because it doubles the GPU VRAM requirements of the application. Changes to Windows ================== Previously, the NSWindow would dynamically change between different types of NSView based on the renderer being used. This is no longer necessary as we just have a single NSView type, GdkMacosView, which inherits from GdkMacosBaseView just to keep the tedius stuff separate from the machinery of GdkMacosView. We can merge those someday if we are okay with that. Changes to Views ================ GdkMacosCairoView, GdkMacosCairoSubView, GdkMacosGLView have all been removed and replaced with GdkMacosView. This new view has a single CALayer (GdkMacosLayer) attached to it which itself has sublayers. The contents of the CALayer is populated with an IOSurfaceRef which we allocated with the GdkMacosSurface. The surface is replaced when the NSWindow resizes. Changes to Layers ================= We now have a dedicated GdkMacosLayer which contains sublayers of GdkMacosTile. The tile has a maximum size of 128x128 pixels in device units. The GdkMacosTile is partitioned by splitting both the transparent region (window bounds minus opaque area) and then by splitting the opaque area. A tile has either translucent contents (and therefore is not opaque) or has opaque contents (and therefore is opaque). An opaque tile never contains transparent contents. As such, the opaque tiles contain a black background so that Core Animation will consider the tile's bounds as opaque. This can be verified with "Quartz Debug -> Show opaque regions". Changes to Cairo ================ GTK 4 cannot currently use cairo-quartz because of how CSS borders are rendered. It simply causes errors in the cairo_quartz_surface_t backend. Since we are restricted to using cairo_image_surface_t (which happens to be faster anyway) we can use the IOSurfaceBaseAddress() to obtain a mapping of the IOSurfaceRef in user-space. It always uses BGRA 32-bit with alpha channel even if we will discard the alpha channel as that is necessary to hit the fast paths in other parts of the platform. Note that while Cairo says CAIRO_FORMAT_ARGB32, it is really 32-bit BGRA on little-endian as we expect. OpenGL will render flipped (Quartz Native Co-ordinates) while Cairo renders with 0,O in the top-left. We could use cairo_translate() and cairo_scale() to reverse this, but it looks like some cairo things may not look quite as right if we do so. To reduce the chances of one-off bugs this continues to draw as Cairo would normally, but instead uses an CGAffineTransform in the tiles and some CGRect translation when swapping buffers to get the same effect. Changes to OpenGL ================= To simplify things, removal of all NSOpenGL* related components have been removed and we strictly use the Core GL (CGL*) API. This probably should have been done long ago anyay. Most examples found in the browsers to use IOSurfaceRef with OpenGL are using Legacy GL and there is still work underway to make this fit in with the rest of how the GSK GL renderer works. Since IOSurfaceRef bound to a texture/framebuffer will not have a default framebuffer ID of 0, we needed to add a default framebuffer id to the GdkGLContext. GskGLRenderer can use this to setup the command queue in such a way that our IOSurface destination has been glBindFramebuffer() as if it were the default drawable. This stuff is pretty slight-of-hand, so where things are and what needs flushing when and where has been a bit of an experiment to see what actually works to get synchronization across subsystems. Efficient Damages ================= After we draw with Cairo, we unlock the IOSurfaceRef and the contents are uploaded to the GPU. To make the contents visible to the app, we must clear the tiles contents with `layer.contents=nil;` and then re-apply the IOSurfaceRef. Since the buffer has likely not changed, we only do this if the tile overlaps the damage region. This gives the effect of having more tightly controlled damage regions even though updating the layer would damage be the whole window (as it is with OpenGL/Metal today with the exception of scissor-rect). This too can be verified usign "Quartz Debug -> Flash screen udpates". Frame Synchronized Resize ========================= In GTK 4, we have the ability to perform sizing changes from compute-size during the layout phase. Since the macOS backend already tracks window resizes manually, we can avoid doing the setFrame: immediately and instead do it within the frame clock's layout phase. Doing so gives us vastly better resize experience as we're more likely to get the size-change and updated-contents in the same frame on screen. It makes things feel "connected" in a way they weren't before. Some additional effort to tweak gravity during the process is also necessary but we were already doing that in the GTK 4 backend. Backporting =========== The design here has made an attempt to make it possible to backport by keeping GdkMacosBuffer, GdkMacosLayer, and GdkMacosTile fairly independent. There may be an opportunity to integrate this into GTK 3's quartz backend with a fair bit of work. Doing so could improve the situation for applications which are damage-rich such as The GIMP.
2022-02-14 10:20:19 +00:00
GdkSurface *surface;
cairo_rectangle_int_t flush_rect;
GLint swapRect[4];
macos: prototype new GDK backend for macOS This is fairly substantial rewrite of the GDK backend for quartz and renamed to macOS to allow for a greenfield implementation. Many things have come across from the quartz implementation fairly intact such as the eventloop integration design and discovery of event windows from the NSEvent. However much has been changed to fit in with the new GDK design and how removal of child GdkWindow have been completely eliminated. Furthermore, the new GdkPopup allows for regular NSWindow to be used to provide popovers unlike the previous implementation. The object design more closely follows the ideal for a GDK backend. Views have been broken out into subclasses so that we can support multiple GSK renderer paths such as GL and Cairo (and Metal in the future). However mixed mode GL and Cairo will not be supported. Currently only the Cairo renderer has been implemented. A new frame clock implementation using CVDisplayLink provides more accurate information about when to draw drawing the next frame. Some testing will need to be done here to understand the power implications of this. This implementation has also gained edge snapping for CSD windows. Some work was also done to ensure that CSD windows have opaque regions registered with the display server. ** This is still very much a work-in-progress ** Some outstanding work that needs to be done: - Finish a GL context for macOS and alternate NSView for GL rendering (possibly using speciailized CALayer for OpenGL). - Input rework to ensure that we don't loose remapping of keys that was dropped from GDK during GTK 4 development. - Make sure input methods continue to work. - Drag-n-Drop is still very much a work in progress - High resolution input scrolling needs various work in GDK to land first before we can plumb that to NSEvent. - gtk/ has a number of things based on GDK_WINDOWING_QUARTZ that need to be updated to use the macOS backend. But this is good enough to start playing with and breaking things which is what I'd like to see.
2020-04-23 23:36:46 +00:00
g_assert (GDK_IS_MACOS_GL_CONTEXT (self));
macos: use CALayer and IOSurface for rendering This provides a major shift in how we draw both when accelerated OpenGL as well as software rendering with Cairo. In short, it uses tiles of Core Animation's CALayer to display contents from an OpenGL or Cairo rendering so that the window can provide partial damage updates. Partial damage is not generally available when using OpenGL as the whole buffer is flipped even if you only submitted a small change using a scissor rect. Thankfully, this speeds up Cairo rendering a bit too by using IOSurface to upload contents to the display server. We use the tiling system we do for OpenGL which reduces overall complexity and differences between them. A New Buffer ============ GdkMacosBuffer is a wrapper around an IOSurfaceRef. The term buffer was used because 1) surface is already used and 2) it loosely maps to a front/back buffer semantic. However, it appears that IOSurfaceRef contents are being retained in some fashion (likely in the compositor result) so we can update the same IOSurfaceRef without flipping as long as we're fast. This appears to be what Chromium does as well, but Firefox uses two IOSurfaceRef and flips between them. We would like to avoid two surfaces because it doubles the GPU VRAM requirements of the application. Changes to Windows ================== Previously, the NSWindow would dynamically change between different types of NSView based on the renderer being used. This is no longer necessary as we just have a single NSView type, GdkMacosView, which inherits from GdkMacosBaseView just to keep the tedius stuff separate from the machinery of GdkMacosView. We can merge those someday if we are okay with that. Changes to Views ================ GdkMacosCairoView, GdkMacosCairoSubView, GdkMacosGLView have all been removed and replaced with GdkMacosView. This new view has a single CALayer (GdkMacosLayer) attached to it which itself has sublayers. The contents of the CALayer is populated with an IOSurfaceRef which we allocated with the GdkMacosSurface. The surface is replaced when the NSWindow resizes. Changes to Layers ================= We now have a dedicated GdkMacosLayer which contains sublayers of GdkMacosTile. The tile has a maximum size of 128x128 pixels in device units. The GdkMacosTile is partitioned by splitting both the transparent region (window bounds minus opaque area) and then by splitting the opaque area. A tile has either translucent contents (and therefore is not opaque) or has opaque contents (and therefore is opaque). An opaque tile never contains transparent contents. As such, the opaque tiles contain a black background so that Core Animation will consider the tile's bounds as opaque. This can be verified with "Quartz Debug -> Show opaque regions". Changes to Cairo ================ GTK 4 cannot currently use cairo-quartz because of how CSS borders are rendered. It simply causes errors in the cairo_quartz_surface_t backend. Since we are restricted to using cairo_image_surface_t (which happens to be faster anyway) we can use the IOSurfaceBaseAddress() to obtain a mapping of the IOSurfaceRef in user-space. It always uses BGRA 32-bit with alpha channel even if we will discard the alpha channel as that is necessary to hit the fast paths in other parts of the platform. Note that while Cairo says CAIRO_FORMAT_ARGB32, it is really 32-bit BGRA on little-endian as we expect. OpenGL will render flipped (Quartz Native Co-ordinates) while Cairo renders with 0,O in the top-left. We could use cairo_translate() and cairo_scale() to reverse this, but it looks like some cairo things may not look quite as right if we do so. To reduce the chances of one-off bugs this continues to draw as Cairo would normally, but instead uses an CGAffineTransform in the tiles and some CGRect translation when swapping buffers to get the same effect. Changes to OpenGL ================= To simplify things, removal of all NSOpenGL* related components have been removed and we strictly use the Core GL (CGL*) API. This probably should have been done long ago anyay. Most examples found in the browsers to use IOSurfaceRef with OpenGL are using Legacy GL and there is still work underway to make this fit in with the rest of how the GSK GL renderer works. Since IOSurfaceRef bound to a texture/framebuffer will not have a default framebuffer ID of 0, we needed to add a default framebuffer id to the GdkGLContext. GskGLRenderer can use this to setup the command queue in such a way that our IOSurface destination has been glBindFramebuffer() as if it were the default drawable. This stuff is pretty slight-of-hand, so where things are and what needs flushing when and where has been a bit of an experiment to see what actually works to get synchronization across subsystems. Efficient Damages ================= After we draw with Cairo, we unlock the IOSurfaceRef and the contents are uploaded to the GPU. To make the contents visible to the app, we must clear the tiles contents with `layer.contents=nil;` and then re-apply the IOSurfaceRef. Since the buffer has likely not changed, we only do this if the tile overlaps the damage region. This gives the effect of having more tightly controlled damage regions even though updating the layer would damage be the whole window (as it is with OpenGL/Metal today with the exception of scissor-rect). This too can be verified usign "Quartz Debug -> Flash screen udpates". Frame Synchronized Resize ========================= In GTK 4, we have the ability to perform sizing changes from compute-size during the layout phase. Since the macOS backend already tracks window resizes manually, we can avoid doing the setFrame: immediately and instead do it within the frame clock's layout phase. Doing so gives us vastly better resize experience as we're more likely to get the size-change and updated-contents in the same frame on screen. It makes things feel "connected" in a way they weren't before. Some additional effort to tweak gravity during the process is also necessary but we were already doing that in the GTK 4 backend. Backporting =========== The design here has made an attempt to make it possible to backport by keeping GdkMacosBuffer, GdkMacosLayer, and GdkMacosTile fairly independent. There may be an opportunity to integrate this into GTK 3's quartz backend with a fair bit of work. Doing so could improve the situation for applications which are damage-rich such as The GIMP.
2022-02-14 10:20:19 +00:00
g_assert (self->cgl_context != nil);
macos: implement GL context This implements the basics for a GdkGLContext on macOS. Currently, rendering only is fully working for the GskCairoRenderer case where we read back pixels into a cairo surface for rendering. More work on synchronization is required for the GL on GskGLRenderer case. When we attempt to render a surface itself with GL, the context will ensure that the new GdkMacosGLView is placed within the NSWindow. In other cases, we use a dummy NSView and NSWindow for backing the NSOpenGLContext to ensure that we can get accelerated drawing. This gets GtkGLArea working when running with GSK_RENDERER=cairo.
2020-10-29 17:33:22 +00:00
GDK_DRAW_CONTEXT_CLASS (gdk_macos_gl_context_parent_class)->end_frame (context, painted);
macos: use CALayer and IOSurface for rendering This provides a major shift in how we draw both when accelerated OpenGL as well as software rendering with Cairo. In short, it uses tiles of Core Animation's CALayer to display contents from an OpenGL or Cairo rendering so that the window can provide partial damage updates. Partial damage is not generally available when using OpenGL as the whole buffer is flipped even if you only submitted a small change using a scissor rect. Thankfully, this speeds up Cairo rendering a bit too by using IOSurface to upload contents to the display server. We use the tiling system we do for OpenGL which reduces overall complexity and differences between them. A New Buffer ============ GdkMacosBuffer is a wrapper around an IOSurfaceRef. The term buffer was used because 1) surface is already used and 2) it loosely maps to a front/back buffer semantic. However, it appears that IOSurfaceRef contents are being retained in some fashion (likely in the compositor result) so we can update the same IOSurfaceRef without flipping as long as we're fast. This appears to be what Chromium does as well, but Firefox uses two IOSurfaceRef and flips between them. We would like to avoid two surfaces because it doubles the GPU VRAM requirements of the application. Changes to Windows ================== Previously, the NSWindow would dynamically change between different types of NSView based on the renderer being used. This is no longer necessary as we just have a single NSView type, GdkMacosView, which inherits from GdkMacosBaseView just to keep the tedius stuff separate from the machinery of GdkMacosView. We can merge those someday if we are okay with that. Changes to Views ================ GdkMacosCairoView, GdkMacosCairoSubView, GdkMacosGLView have all been removed and replaced with GdkMacosView. This new view has a single CALayer (GdkMacosLayer) attached to it which itself has sublayers. The contents of the CALayer is populated with an IOSurfaceRef which we allocated with the GdkMacosSurface. The surface is replaced when the NSWindow resizes. Changes to Layers ================= We now have a dedicated GdkMacosLayer which contains sublayers of GdkMacosTile. The tile has a maximum size of 128x128 pixels in device units. The GdkMacosTile is partitioned by splitting both the transparent region (window bounds minus opaque area) and then by splitting the opaque area. A tile has either translucent contents (and therefore is not opaque) or has opaque contents (and therefore is opaque). An opaque tile never contains transparent contents. As such, the opaque tiles contain a black background so that Core Animation will consider the tile's bounds as opaque. This can be verified with "Quartz Debug -> Show opaque regions". Changes to Cairo ================ GTK 4 cannot currently use cairo-quartz because of how CSS borders are rendered. It simply causes errors in the cairo_quartz_surface_t backend. Since we are restricted to using cairo_image_surface_t (which happens to be faster anyway) we can use the IOSurfaceBaseAddress() to obtain a mapping of the IOSurfaceRef in user-space. It always uses BGRA 32-bit with alpha channel even if we will discard the alpha channel as that is necessary to hit the fast paths in other parts of the platform. Note that while Cairo says CAIRO_FORMAT_ARGB32, it is really 32-bit BGRA on little-endian as we expect. OpenGL will render flipped (Quartz Native Co-ordinates) while Cairo renders with 0,O in the top-left. We could use cairo_translate() and cairo_scale() to reverse this, but it looks like some cairo things may not look quite as right if we do so. To reduce the chances of one-off bugs this continues to draw as Cairo would normally, but instead uses an CGAffineTransform in the tiles and some CGRect translation when swapping buffers to get the same effect. Changes to OpenGL ================= To simplify things, removal of all NSOpenGL* related components have been removed and we strictly use the Core GL (CGL*) API. This probably should have been done long ago anyay. Most examples found in the browsers to use IOSurfaceRef with OpenGL are using Legacy GL and there is still work underway to make this fit in with the rest of how the GSK GL renderer works. Since IOSurfaceRef bound to a texture/framebuffer will not have a default framebuffer ID of 0, we needed to add a default framebuffer id to the GdkGLContext. GskGLRenderer can use this to setup the command queue in such a way that our IOSurface destination has been glBindFramebuffer() as if it were the default drawable. This stuff is pretty slight-of-hand, so where things are and what needs flushing when and where has been a bit of an experiment to see what actually works to get synchronization across subsystems. Efficient Damages ================= After we draw with Cairo, we unlock the IOSurfaceRef and the contents are uploaded to the GPU. To make the contents visible to the app, we must clear the tiles contents with `layer.contents=nil;` and then re-apply the IOSurfaceRef. Since the buffer has likely not changed, we only do this if the tile overlaps the damage region. This gives the effect of having more tightly controlled damage regions even though updating the layer would damage be the whole window (as it is with OpenGL/Metal today with the exception of scissor-rect). This too can be verified usign "Quartz Debug -> Flash screen udpates". Frame Synchronized Resize ========================= In GTK 4, we have the ability to perform sizing changes from compute-size during the layout phase. Since the macOS backend already tracks window resizes manually, we can avoid doing the setFrame: immediately and instead do it within the frame clock's layout phase. Doing so gives us vastly better resize experience as we're more likely to get the size-change and updated-contents in the same frame on screen. It makes things feel "connected" in a way they weren't before. Some additional effort to tweak gravity during the process is also necessary but we were already doing that in the GTK 4 backend. Backporting =========== The design here has made an attempt to make it possible to backport by keeping GdkMacosBuffer, GdkMacosLayer, and GdkMacosTile fairly independent. There may be an opportunity to integrate this into GTK 3's quartz backend with a fair bit of work. Doing so could improve the situation for applications which are damage-rich such as The GIMP.
2022-02-14 10:20:19 +00:00
surface = gdk_draw_context_get_surface (context);
gdk_gl_context_make_current (GDK_GL_CONTEXT (self));
/* Coordinates are in display coordinates, where as flush_rect is
* in GDK coordinates. Must flip Y to match display coordinates where
* 0,0 is the bottom-left corner.
*/
cairo_region_get_extents (painted, &flush_rect);
swapRect[0] = flush_rect.x; /* left */
swapRect[1] = surface->height - flush_rect.y; /* bottom */
swapRect[2] = flush_rect.width; /* width */
swapRect[3] = flush_rect.height; /* height */
CGLSetParameter (self->cgl_context, kCGLCPSwapRectangle, swapRect);
gdk_macos_gl_context_release (self);
glFlush ();
/* Begin a Core Animation transaction so that all changes we
* make within the window are seen atomically.
*/
[CATransaction begin];
[CATransaction setDisableActions:YES];
_gdk_macos_surface_swap_buffers (GDK_MACOS_SURFACE (surface), painted);
[CATransaction commit];
macos: prototype new GDK backend for macOS This is fairly substantial rewrite of the GDK backend for quartz and renamed to macOS to allow for a greenfield implementation. Many things have come across from the quartz implementation fairly intact such as the eventloop integration design and discovery of event windows from the NSEvent. However much has been changed to fit in with the new GDK design and how removal of child GdkWindow have been completely eliminated. Furthermore, the new GdkPopup allows for regular NSWindow to be used to provide popovers unlike the previous implementation. The object design more closely follows the ideal for a GDK backend. Views have been broken out into subclasses so that we can support multiple GSK renderer paths such as GL and Cairo (and Metal in the future). However mixed mode GL and Cairo will not be supported. Currently only the Cairo renderer has been implemented. A new frame clock implementation using CVDisplayLink provides more accurate information about when to draw drawing the next frame. Some testing will need to be done here to understand the power implications of this. This implementation has also gained edge snapping for CSD windows. Some work was also done to ensure that CSD windows have opaque regions registered with the display server. ** This is still very much a work-in-progress ** Some outstanding work that needs to be done: - Finish a GL context for macOS and alternate NSView for GL rendering (possibly using speciailized CALayer for OpenGL). - Input rework to ensure that we don't loose remapping of keys that was dropped from GDK during GTK 4 development. - Make sure input methods continue to work. - Drag-n-Drop is still very much a work in progress - High resolution input scrolling needs various work in GDK to land first before we can plumb that to NSEvent. - gtk/ has a number of things based on GDK_WINDOWING_QUARTZ that need to be updated to use the macOS backend. But this is good enough to start playing with and breaking things which is what I'd like to see.
2020-04-23 23:36:46 +00:00
}
macos: implement GL context This implements the basics for a GdkGLContext on macOS. Currently, rendering only is fully working for the GskCairoRenderer case where we read back pixels into a cairo surface for rendering. More work on synchronization is required for the GL on GskGLRenderer case. When we attempt to render a surface itself with GL, the context will ensure that the new GdkMacosGLView is placed within the NSWindow. In other cases, we use a dummy NSView and NSWindow for backing the NSOpenGLContext to ensure that we can get accelerated drawing. This gets GtkGLArea working when running with GSK_RENDERER=cairo.
2020-10-29 17:33:22 +00:00
static void
gdk_macos_gl_context_surface_resized (GdkDrawContext *draw_context)
{
GdkMacosGLContext *self = (GdkMacosGLContext *)draw_context;
g_assert (GDK_IS_MACOS_GL_CONTEXT (self));
macos: clip damage when swapping buffers This ensures that we only copy the changed area.
2020-11-05 01:08:43 +00:00
g_clear_pointer (&self->damage, cairo_region_destroy);
}
gl: Move vfunc Instead of Display::make_gl_context_current() we now have GLContext::clear_current() GLContext::make_current() This fits better with the backends (we can actually implement clearCurrent on macOS now) and makes it easier to implement different GL backends for backends (like EGL/GLX on X11). We also pass a surfaceless boolean to make_current() so the calling code can decide if a surface needs to be bound or not, because the backends were all doing whatever, which was very counterproductive.
2021-07-07 02:40:34 +00:00
static gboolean
gdk_macos_gl_context_clear_current (GdkGLContext *context)
{
GdkMacosGLContext *self = GDK_MACOS_GL_CONTEXT (context);
g_return_val_if_fail (GDK_IS_MACOS_GL_CONTEXT (self), FALSE);
macos: use CALayer and IOSurface for rendering This provides a major shift in how we draw both when accelerated OpenGL as well as software rendering with Cairo. In short, it uses tiles of Core Animation's CALayer to display contents from an OpenGL or Cairo rendering so that the window can provide partial damage updates. Partial damage is not generally available when using OpenGL as the whole buffer is flipped even if you only submitted a small change using a scissor rect. Thankfully, this speeds up Cairo rendering a bit too by using IOSurface to upload contents to the display server. We use the tiling system we do for OpenGL which reduces overall complexity and differences between them. A New Buffer ============ GdkMacosBuffer is a wrapper around an IOSurfaceRef. The term buffer was used because 1) surface is already used and 2) it loosely maps to a front/back buffer semantic. However, it appears that IOSurfaceRef contents are being retained in some fashion (likely in the compositor result) so we can update the same IOSurfaceRef without flipping as long as we're fast. This appears to be what Chromium does as well, but Firefox uses two IOSurfaceRef and flips between them. We would like to avoid two surfaces because it doubles the GPU VRAM requirements of the application. Changes to Windows ================== Previously, the NSWindow would dynamically change between different types of NSView based on the renderer being used. This is no longer necessary as we just have a single NSView type, GdkMacosView, which inherits from GdkMacosBaseView just to keep the tedius stuff separate from the machinery of GdkMacosView. We can merge those someday if we are okay with that. Changes to Views ================ GdkMacosCairoView, GdkMacosCairoSubView, GdkMacosGLView have all been removed and replaced with GdkMacosView. This new view has a single CALayer (GdkMacosLayer) attached to it which itself has sublayers. The contents of the CALayer is populated with an IOSurfaceRef which we allocated with the GdkMacosSurface. The surface is replaced when the NSWindow resizes. Changes to Layers ================= We now have a dedicated GdkMacosLayer which contains sublayers of GdkMacosTile. The tile has a maximum size of 128x128 pixels in device units. The GdkMacosTile is partitioned by splitting both the transparent region (window bounds minus opaque area) and then by splitting the opaque area. A tile has either translucent contents (and therefore is not opaque) or has opaque contents (and therefore is opaque). An opaque tile never contains transparent contents. As such, the opaque tiles contain a black background so that Core Animation will consider the tile's bounds as opaque. This can be verified with "Quartz Debug -> Show opaque regions". Changes to Cairo ================ GTK 4 cannot currently use cairo-quartz because of how CSS borders are rendered. It simply causes errors in the cairo_quartz_surface_t backend. Since we are restricted to using cairo_image_surface_t (which happens to be faster anyway) we can use the IOSurfaceBaseAddress() to obtain a mapping of the IOSurfaceRef in user-space. It always uses BGRA 32-bit with alpha channel even if we will discard the alpha channel as that is necessary to hit the fast paths in other parts of the platform. Note that while Cairo says CAIRO_FORMAT_ARGB32, it is really 32-bit BGRA on little-endian as we expect. OpenGL will render flipped (Quartz Native Co-ordinates) while Cairo renders with 0,O in the top-left. We could use cairo_translate() and cairo_scale() to reverse this, but it looks like some cairo things may not look quite as right if we do so. To reduce the chances of one-off bugs this continues to draw as Cairo would normally, but instead uses an CGAffineTransform in the tiles and some CGRect translation when swapping buffers to get the same effect. Changes to OpenGL ================= To simplify things, removal of all NSOpenGL* related components have been removed and we strictly use the Core GL (CGL*) API. This probably should have been done long ago anyay. Most examples found in the browsers to use IOSurfaceRef with OpenGL are using Legacy GL and there is still work underway to make this fit in with the rest of how the GSK GL renderer works. Since IOSurfaceRef bound to a texture/framebuffer will not have a default framebuffer ID of 0, we needed to add a default framebuffer id to the GdkGLContext. GskGLRenderer can use this to setup the command queue in such a way that our IOSurface destination has been glBindFramebuffer() as if it were the default drawable. This stuff is pretty slight-of-hand, so where things are and what needs flushing when and where has been a bit of an experiment to see what actually works to get synchronization across subsystems. Efficient Damages ================= After we draw with Cairo, we unlock the IOSurfaceRef and the contents are uploaded to the GPU. To make the contents visible to the app, we must clear the tiles contents with `layer.contents=nil;` and then re-apply the IOSurfaceRef. Since the buffer has likely not changed, we only do this if the tile overlaps the damage region. This gives the effect of having more tightly controlled damage regions even though updating the layer would damage be the whole window (as it is with OpenGL/Metal today with the exception of scissor-rect). This too can be verified usign "Quartz Debug -> Flash screen udpates". Frame Synchronized Resize ========================= In GTK 4, we have the ability to perform sizing changes from compute-size during the layout phase. Since the macOS backend already tracks window resizes manually, we can avoid doing the setFrame: immediately and instead do it within the frame clock's layout phase. Doing so gives us vastly better resize experience as we're more likely to get the size-change and updated-contents in the same frame on screen. It makes things feel "connected" in a way they weren't before. Some additional effort to tweak gravity during the process is also necessary but we were already doing that in the GTK 4 backend. Backporting =========== The design here has made an attempt to make it possible to backport by keeping GdkMacosBuffer, GdkMacosLayer, and GdkMacosTile fairly independent. There may be an opportunity to integrate this into GTK 3's quartz backend with a fair bit of work. Doing so could improve the situation for applications which are damage-rich such as The GIMP.
2022-02-14 10:20:19 +00:00
if (self->cgl_context == CGLGetCurrentContext ())
gl: Move vfunc Instead of Display::make_gl_context_current() we now have GLContext::clear_current() GLContext::make_current() This fits better with the backends (we can actually implement clearCurrent on macOS now) and makes it easier to implement different GL backends for backends (like EGL/GLX on X11). We also pass a surfaceless boolean to make_current() so the calling code can decide if a surface needs to be bound or not, because the backends were all doing whatever, which was very counterproductive.
2021-07-07 02:40:34 +00:00
{
macos: use CALayer and IOSurface for rendering This provides a major shift in how we draw both when accelerated OpenGL as well as software rendering with Cairo. In short, it uses tiles of Core Animation's CALayer to display contents from an OpenGL or Cairo rendering so that the window can provide partial damage updates. Partial damage is not generally available when using OpenGL as the whole buffer is flipped even if you only submitted a small change using a scissor rect. Thankfully, this speeds up Cairo rendering a bit too by using IOSurface to upload contents to the display server. We use the tiling system we do for OpenGL which reduces overall complexity and differences between them. A New Buffer ============ GdkMacosBuffer is a wrapper around an IOSurfaceRef. The term buffer was used because 1) surface is already used and 2) it loosely maps to a front/back buffer semantic. However, it appears that IOSurfaceRef contents are being retained in some fashion (likely in the compositor result) so we can update the same IOSurfaceRef without flipping as long as we're fast. This appears to be what Chromium does as well, but Firefox uses two IOSurfaceRef and flips between them. We would like to avoid two surfaces because it doubles the GPU VRAM requirements of the application. Changes to Windows ================== Previously, the NSWindow would dynamically change between different types of NSView based on the renderer being used. This is no longer necessary as we just have a single NSView type, GdkMacosView, which inherits from GdkMacosBaseView just to keep the tedius stuff separate from the machinery of GdkMacosView. We can merge those someday if we are okay with that. Changes to Views ================ GdkMacosCairoView, GdkMacosCairoSubView, GdkMacosGLView have all been removed and replaced with GdkMacosView. This new view has a single CALayer (GdkMacosLayer) attached to it which itself has sublayers. The contents of the CALayer is populated with an IOSurfaceRef which we allocated with the GdkMacosSurface. The surface is replaced when the NSWindow resizes. Changes to Layers ================= We now have a dedicated GdkMacosLayer which contains sublayers of GdkMacosTile. The tile has a maximum size of 128x128 pixels in device units. The GdkMacosTile is partitioned by splitting both the transparent region (window bounds minus opaque area) and then by splitting the opaque area. A tile has either translucent contents (and therefore is not opaque) or has opaque contents (and therefore is opaque). An opaque tile never contains transparent contents. As such, the opaque tiles contain a black background so that Core Animation will consider the tile's bounds as opaque. This can be verified with "Quartz Debug -> Show opaque regions". Changes to Cairo ================ GTK 4 cannot currently use cairo-quartz because of how CSS borders are rendered. It simply causes errors in the cairo_quartz_surface_t backend. Since we are restricted to using cairo_image_surface_t (which happens to be faster anyway) we can use the IOSurfaceBaseAddress() to obtain a mapping of the IOSurfaceRef in user-space. It always uses BGRA 32-bit with alpha channel even if we will discard the alpha channel as that is necessary to hit the fast paths in other parts of the platform. Note that while Cairo says CAIRO_FORMAT_ARGB32, it is really 32-bit BGRA on little-endian as we expect. OpenGL will render flipped (Quartz Native Co-ordinates) while Cairo renders with 0,O in the top-left. We could use cairo_translate() and cairo_scale() to reverse this, but it looks like some cairo things may not look quite as right if we do so. To reduce the chances of one-off bugs this continues to draw as Cairo would normally, but instead uses an CGAffineTransform in the tiles and some CGRect translation when swapping buffers to get the same effect. Changes to OpenGL ================= To simplify things, removal of all NSOpenGL* related components have been removed and we strictly use the Core GL (CGL*) API. This probably should have been done long ago anyay. Most examples found in the browsers to use IOSurfaceRef with OpenGL are using Legacy GL and there is still work underway to make this fit in with the rest of how the GSK GL renderer works. Since IOSurfaceRef bound to a texture/framebuffer will not have a default framebuffer ID of 0, we needed to add a default framebuffer id to the GdkGLContext. GskGLRenderer can use this to setup the command queue in such a way that our IOSurface destination has been glBindFramebuffer() as if it were the default drawable. This stuff is pretty slight-of-hand, so where things are and what needs flushing when and where has been a bit of an experiment to see what actually works to get synchronization across subsystems. Efficient Damages ================= After we draw with Cairo, we unlock the IOSurfaceRef and the contents are uploaded to the GPU. To make the contents visible to the app, we must clear the tiles contents with `layer.contents=nil;` and then re-apply the IOSurfaceRef. Since the buffer has likely not changed, we only do this if the tile overlaps the damage region. This gives the effect of having more tightly controlled damage regions even though updating the layer would damage be the whole window (as it is with OpenGL/Metal today with the exception of scissor-rect). This too can be verified usign "Quartz Debug -> Flash screen udpates". Frame Synchronized Resize ========================= In GTK 4, we have the ability to perform sizing changes from compute-size during the layout phase. Since the macOS backend already tracks window resizes manually, we can avoid doing the setFrame: immediately and instead do it within the frame clock's layout phase. Doing so gives us vastly better resize experience as we're more likely to get the size-change and updated-contents in the same frame on screen. It makes things feel "connected" in a way they weren't before. Some additional effort to tweak gravity during the process is also necessary but we were already doing that in the GTK 4 backend. Backporting =========== The design here has made an attempt to make it possible to backport by keeping GdkMacosBuffer, GdkMacosLayer, and GdkMacosTile fairly independent. There may be an opportunity to integrate this into GTK 3's quartz backend with a fair bit of work. Doing so could improve the situation for applications which are damage-rich such as The GIMP.
2022-02-14 10:20:19 +00:00
glFlush ();
CGLSetCurrentContext (NULL);
gl: Move vfunc Instead of Display::make_gl_context_current() we now have GLContext::clear_current() GLContext::make_current() This fits better with the backends (we can actually implement clearCurrent on macOS now) and makes it easier to implement different GL backends for backends (like EGL/GLX on X11). We also pass a surfaceless boolean to make_current() so the calling code can decide if a surface needs to be bound or not, because the backends were all doing whatever, which was very counterproductive.
2021-07-07 02:40:34 +00:00
}
return TRUE;
}
static gboolean
gdk_macos_gl_context_make_current (GdkGLContext *context,
gboolean surfaceless)
{
GdkMacosGLContext *self = GDK_MACOS_GL_CONTEXT (context);
macos: use CALayer and IOSurface for rendering This provides a major shift in how we draw both when accelerated OpenGL as well as software rendering with Cairo. In short, it uses tiles of Core Animation's CALayer to display contents from an OpenGL or Cairo rendering so that the window can provide partial damage updates. Partial damage is not generally available when using OpenGL as the whole buffer is flipped even if you only submitted a small change using a scissor rect. Thankfully, this speeds up Cairo rendering a bit too by using IOSurface to upload contents to the display server. We use the tiling system we do for OpenGL which reduces overall complexity and differences between them. A New Buffer ============ GdkMacosBuffer is a wrapper around an IOSurfaceRef. The term buffer was used because 1) surface is already used and 2) it loosely maps to a front/back buffer semantic. However, it appears that IOSurfaceRef contents are being retained in some fashion (likely in the compositor result) so we can update the same IOSurfaceRef without flipping as long as we're fast. This appears to be what Chromium does as well, but Firefox uses two IOSurfaceRef and flips between them. We would like to avoid two surfaces because it doubles the GPU VRAM requirements of the application. Changes to Windows ================== Previously, the NSWindow would dynamically change between different types of NSView based on the renderer being used. This is no longer necessary as we just have a single NSView type, GdkMacosView, which inherits from GdkMacosBaseView just to keep the tedius stuff separate from the machinery of GdkMacosView. We can merge those someday if we are okay with that. Changes to Views ================ GdkMacosCairoView, GdkMacosCairoSubView, GdkMacosGLView have all been removed and replaced with GdkMacosView. This new view has a single CALayer (GdkMacosLayer) attached to it which itself has sublayers. The contents of the CALayer is populated with an IOSurfaceRef which we allocated with the GdkMacosSurface. The surface is replaced when the NSWindow resizes. Changes to Layers ================= We now have a dedicated GdkMacosLayer which contains sublayers of GdkMacosTile. The tile has a maximum size of 128x128 pixels in device units. The GdkMacosTile is partitioned by splitting both the transparent region (window bounds minus opaque area) and then by splitting the opaque area. A tile has either translucent contents (and therefore is not opaque) or has opaque contents (and therefore is opaque). An opaque tile never contains transparent contents. As such, the opaque tiles contain a black background so that Core Animation will consider the tile's bounds as opaque. This can be verified with "Quartz Debug -> Show opaque regions". Changes to Cairo ================ GTK 4 cannot currently use cairo-quartz because of how CSS borders are rendered. It simply causes errors in the cairo_quartz_surface_t backend. Since we are restricted to using cairo_image_surface_t (which happens to be faster anyway) we can use the IOSurfaceBaseAddress() to obtain a mapping of the IOSurfaceRef in user-space. It always uses BGRA 32-bit with alpha channel even if we will discard the alpha channel as that is necessary to hit the fast paths in other parts of the platform. Note that while Cairo says CAIRO_FORMAT_ARGB32, it is really 32-bit BGRA on little-endian as we expect. OpenGL will render flipped (Quartz Native Co-ordinates) while Cairo renders with 0,O in the top-left. We could use cairo_translate() and cairo_scale() to reverse this, but it looks like some cairo things may not look quite as right if we do so. To reduce the chances of one-off bugs this continues to draw as Cairo would normally, but instead uses an CGAffineTransform in the tiles and some CGRect translation when swapping buffers to get the same effect. Changes to OpenGL ================= To simplify things, removal of all NSOpenGL* related components have been removed and we strictly use the Core GL (CGL*) API. This probably should have been done long ago anyay. Most examples found in the browsers to use IOSurfaceRef with OpenGL are using Legacy GL and there is still work underway to make this fit in with the rest of how the GSK GL renderer works. Since IOSurfaceRef bound to a texture/framebuffer will not have a default framebuffer ID of 0, we needed to add a default framebuffer id to the GdkGLContext. GskGLRenderer can use this to setup the command queue in such a way that our IOSurface destination has been glBindFramebuffer() as if it were the default drawable. This stuff is pretty slight-of-hand, so where things are and what needs flushing when and where has been a bit of an experiment to see what actually works to get synchronization across subsystems. Efficient Damages ================= After we draw with Cairo, we unlock the IOSurfaceRef and the contents are uploaded to the GPU. To make the contents visible to the app, we must clear the tiles contents with `layer.contents=nil;` and then re-apply the IOSurfaceRef. Since the buffer has likely not changed, we only do this if the tile overlaps the damage region. This gives the effect of having more tightly controlled damage regions even though updating the layer would damage be the whole window (as it is with OpenGL/Metal today with the exception of scissor-rect). This too can be verified usign "Quartz Debug -> Flash screen udpates". Frame Synchronized Resize ========================= In GTK 4, we have the ability to perform sizing changes from compute-size during the layout phase. Since the macOS backend already tracks window resizes manually, we can avoid doing the setFrame: immediately and instead do it within the frame clock's layout phase. Doing so gives us vastly better resize experience as we're more likely to get the size-change and updated-contents in the same frame on screen. It makes things feel "connected" in a way they weren't before. Some additional effort to tweak gravity during the process is also necessary but we were already doing that in the GTK 4 backend. Backporting =========== The design here has made an attempt to make it possible to backport by keeping GdkMacosBuffer, GdkMacosLayer, and GdkMacosTile fairly independent. There may be an opportunity to integrate this into GTK 3's quartz backend with a fair bit of work. Doing so could improve the situation for applications which are damage-rich such as The GIMP.
2022-02-14 10:20:19 +00:00
CGLContextObj current;
gl: Move vfunc Instead of Display::make_gl_context_current() we now have GLContext::clear_current() GLContext::make_current() This fits better with the backends (we can actually implement clearCurrent on macOS now) and makes it easier to implement different GL backends for backends (like EGL/GLX on X11). We also pass a surfaceless boolean to make_current() so the calling code can decide if a surface needs to be bound or not, because the backends were all doing whatever, which was very counterproductive.
2021-07-07 02:40:34 +00:00
g_return_val_if_fail (GDK_IS_MACOS_GL_CONTEXT (self), FALSE);
macos: use CALayer and IOSurface for rendering This provides a major shift in how we draw both when accelerated OpenGL as well as software rendering with Cairo. In short, it uses tiles of Core Animation's CALayer to display contents from an OpenGL or Cairo rendering so that the window can provide partial damage updates. Partial damage is not generally available when using OpenGL as the whole buffer is flipped even if you only submitted a small change using a scissor rect. Thankfully, this speeds up Cairo rendering a bit too by using IOSurface to upload contents to the display server. We use the tiling system we do for OpenGL which reduces overall complexity and differences between them. A New Buffer ============ GdkMacosBuffer is a wrapper around an IOSurfaceRef. The term buffer was used because 1) surface is already used and 2) it loosely maps to a front/back buffer semantic. However, it appears that IOSurfaceRef contents are being retained in some fashion (likely in the compositor result) so we can update the same IOSurfaceRef without flipping as long as we're fast. This appears to be what Chromium does as well, but Firefox uses two IOSurfaceRef and flips between them. We would like to avoid two surfaces because it doubles the GPU VRAM requirements of the application. Changes to Windows ================== Previously, the NSWindow would dynamically change between different types of NSView based on the renderer being used. This is no longer necessary as we just have a single NSView type, GdkMacosView, which inherits from GdkMacosBaseView just to keep the tedius stuff separate from the machinery of GdkMacosView. We can merge those someday if we are okay with that. Changes to Views ================ GdkMacosCairoView, GdkMacosCairoSubView, GdkMacosGLView have all been removed and replaced with GdkMacosView. This new view has a single CALayer (GdkMacosLayer) attached to it which itself has sublayers. The contents of the CALayer is populated with an IOSurfaceRef which we allocated with the GdkMacosSurface. The surface is replaced when the NSWindow resizes. Changes to Layers ================= We now have a dedicated GdkMacosLayer which contains sublayers of GdkMacosTile. The tile has a maximum size of 128x128 pixels in device units. The GdkMacosTile is partitioned by splitting both the transparent region (window bounds minus opaque area) and then by splitting the opaque area. A tile has either translucent contents (and therefore is not opaque) or has opaque contents (and therefore is opaque). An opaque tile never contains transparent contents. As such, the opaque tiles contain a black background so that Core Animation will consider the tile's bounds as opaque. This can be verified with "Quartz Debug -> Show opaque regions". Changes to Cairo ================ GTK 4 cannot currently use cairo-quartz because of how CSS borders are rendered. It simply causes errors in the cairo_quartz_surface_t backend. Since we are restricted to using cairo_image_surface_t (which happens to be faster anyway) we can use the IOSurfaceBaseAddress() to obtain a mapping of the IOSurfaceRef in user-space. It always uses BGRA 32-bit with alpha channel even if we will discard the alpha channel as that is necessary to hit the fast paths in other parts of the platform. Note that while Cairo says CAIRO_FORMAT_ARGB32, it is really 32-bit BGRA on little-endian as we expect. OpenGL will render flipped (Quartz Native Co-ordinates) while Cairo renders with 0,O in the top-left. We could use cairo_translate() and cairo_scale() to reverse this, but it looks like some cairo things may not look quite as right if we do so. To reduce the chances of one-off bugs this continues to draw as Cairo would normally, but instead uses an CGAffineTransform in the tiles and some CGRect translation when swapping buffers to get the same effect. Changes to OpenGL ================= To simplify things, removal of all NSOpenGL* related components have been removed and we strictly use the Core GL (CGL*) API. This probably should have been done long ago anyay. Most examples found in the browsers to use IOSurfaceRef with OpenGL are using Legacy GL and there is still work underway to make this fit in with the rest of how the GSK GL renderer works. Since IOSurfaceRef bound to a texture/framebuffer will not have a default framebuffer ID of 0, we needed to add a default framebuffer id to the GdkGLContext. GskGLRenderer can use this to setup the command queue in such a way that our IOSurface destination has been glBindFramebuffer() as if it were the default drawable. This stuff is pretty slight-of-hand, so where things are and what needs flushing when and where has been a bit of an experiment to see what actually works to get synchronization across subsystems. Efficient Damages ================= After we draw with Cairo, we unlock the IOSurfaceRef and the contents are uploaded to the GPU. To make the contents visible to the app, we must clear the tiles contents with `layer.contents=nil;` and then re-apply the IOSurfaceRef. Since the buffer has likely not changed, we only do this if the tile overlaps the damage region. This gives the effect of having more tightly controlled damage regions even though updating the layer would damage be the whole window (as it is with OpenGL/Metal today with the exception of scissor-rect). This too can be verified usign "Quartz Debug -> Flash screen udpates". Frame Synchronized Resize ========================= In GTK 4, we have the ability to perform sizing changes from compute-size during the layout phase. Since the macOS backend already tracks window resizes manually, we can avoid doing the setFrame: immediately and instead do it within the frame clock's layout phase. Doing so gives us vastly better resize experience as we're more likely to get the size-change and updated-contents in the same frame on screen. It makes things feel "connected" in a way they weren't before. Some additional effort to tweak gravity during the process is also necessary but we were already doing that in the GTK 4 backend. Backporting =========== The design here has made an attempt to make it possible to backport by keeping GdkMacosBuffer, GdkMacosLayer, and GdkMacosTile fairly independent. There may be an opportunity to integrate this into GTK 3's quartz backend with a fair bit of work. Doing so could improve the situation for applications which are damage-rich such as The GIMP.
2022-02-14 10:20:19 +00:00
current = CGLGetCurrentContext ();
gl: Move vfunc Instead of Display::make_gl_context_current() we now have GLContext::clear_current() GLContext::make_current() This fits better with the backends (we can actually implement clearCurrent on macOS now) and makes it easier to implement different GL backends for backends (like EGL/GLX on X11). We also pass a surfaceless boolean to make_current() so the calling code can decide if a surface needs to be bound or not, because the backends were all doing whatever, which was very counterproductive.
2021-07-07 02:40:34 +00:00
macos: use CALayer and IOSurface for rendering This provides a major shift in how we draw both when accelerated OpenGL as well as software rendering with Cairo. In short, it uses tiles of Core Animation's CALayer to display contents from an OpenGL or Cairo rendering so that the window can provide partial damage updates. Partial damage is not generally available when using OpenGL as the whole buffer is flipped even if you only submitted a small change using a scissor rect. Thankfully, this speeds up Cairo rendering a bit too by using IOSurface to upload contents to the display server. We use the tiling system we do for OpenGL which reduces overall complexity and differences between them. A New Buffer ============ GdkMacosBuffer is a wrapper around an IOSurfaceRef. The term buffer was used because 1) surface is already used and 2) it loosely maps to a front/back buffer semantic. However, it appears that IOSurfaceRef contents are being retained in some fashion (likely in the compositor result) so we can update the same IOSurfaceRef without flipping as long as we're fast. This appears to be what Chromium does as well, but Firefox uses two IOSurfaceRef and flips between them. We would like to avoid two surfaces because it doubles the GPU VRAM requirements of the application. Changes to Windows ================== Previously, the NSWindow would dynamically change between different types of NSView based on the renderer being used. This is no longer necessary as we just have a single NSView type, GdkMacosView, which inherits from GdkMacosBaseView just to keep the tedius stuff separate from the machinery of GdkMacosView. We can merge those someday if we are okay with that. Changes to Views ================ GdkMacosCairoView, GdkMacosCairoSubView, GdkMacosGLView have all been removed and replaced with GdkMacosView. This new view has a single CALayer (GdkMacosLayer) attached to it which itself has sublayers. The contents of the CALayer is populated with an IOSurfaceRef which we allocated with the GdkMacosSurface. The surface is replaced when the NSWindow resizes. Changes to Layers ================= We now have a dedicated GdkMacosLayer which contains sublayers of GdkMacosTile. The tile has a maximum size of 128x128 pixels in device units. The GdkMacosTile is partitioned by splitting both the transparent region (window bounds minus opaque area) and then by splitting the opaque area. A tile has either translucent contents (and therefore is not opaque) or has opaque contents (and therefore is opaque). An opaque tile never contains transparent contents. As such, the opaque tiles contain a black background so that Core Animation will consider the tile's bounds as opaque. This can be verified with "Quartz Debug -> Show opaque regions". Changes to Cairo ================ GTK 4 cannot currently use cairo-quartz because of how CSS borders are rendered. It simply causes errors in the cairo_quartz_surface_t backend. Since we are restricted to using cairo_image_surface_t (which happens to be faster anyway) we can use the IOSurfaceBaseAddress() to obtain a mapping of the IOSurfaceRef in user-space. It always uses BGRA 32-bit with alpha channel even if we will discard the alpha channel as that is necessary to hit the fast paths in other parts of the platform. Note that while Cairo says CAIRO_FORMAT_ARGB32, it is really 32-bit BGRA on little-endian as we expect. OpenGL will render flipped (Quartz Native Co-ordinates) while Cairo renders with 0,O in the top-left. We could use cairo_translate() and cairo_scale() to reverse this, but it looks like some cairo things may not look quite as right if we do so. To reduce the chances of one-off bugs this continues to draw as Cairo would normally, but instead uses an CGAffineTransform in the tiles and some CGRect translation when swapping buffers to get the same effect. Changes to OpenGL ================= To simplify things, removal of all NSOpenGL* related components have been removed and we strictly use the Core GL (CGL*) API. This probably should have been done long ago anyay. Most examples found in the browsers to use IOSurfaceRef with OpenGL are using Legacy GL and there is still work underway to make this fit in with the rest of how the GSK GL renderer works. Since IOSurfaceRef bound to a texture/framebuffer will not have a default framebuffer ID of 0, we needed to add a default framebuffer id to the GdkGLContext. GskGLRenderer can use this to setup the command queue in such a way that our IOSurface destination has been glBindFramebuffer() as if it were the default drawable. This stuff is pretty slight-of-hand, so where things are and what needs flushing when and where has been a bit of an experiment to see what actually works to get synchronization across subsystems. Efficient Damages ================= After we draw with Cairo, we unlock the IOSurfaceRef and the contents are uploaded to the GPU. To make the contents visible to the app, we must clear the tiles contents with `layer.contents=nil;` and then re-apply the IOSurfaceRef. Since the buffer has likely not changed, we only do this if the tile overlaps the damage region. This gives the effect of having more tightly controlled damage regions even though updating the layer would damage be the whole window (as it is with OpenGL/Metal today with the exception of scissor-rect). This too can be verified usign "Quartz Debug -> Flash screen udpates". Frame Synchronized Resize ========================= In GTK 4, we have the ability to perform sizing changes from compute-size during the layout phase. Since the macOS backend already tracks window resizes manually, we can avoid doing the setFrame: immediately and instead do it within the frame clock's layout phase. Doing so gives us vastly better resize experience as we're more likely to get the size-change and updated-contents in the same frame on screen. It makes things feel "connected" in a way they weren't before. Some additional effort to tweak gravity during the process is also necessary but we were already doing that in the GTK 4 backend. Backporting =========== The design here has made an attempt to make it possible to backport by keeping GdkMacosBuffer, GdkMacosLayer, and GdkMacosTile fairly independent. There may be an opportunity to integrate this into GTK 3's quartz backend with a fair bit of work. Doing so could improve the situation for applications which are damage-rich such as The GIMP.
2022-02-14 10:20:19 +00:00
if (self->cgl_context != current)
gl: Move vfunc Instead of Display::make_gl_context_current() we now have GLContext::clear_current() GLContext::make_current() This fits better with the backends (we can actually implement clearCurrent on macOS now) and makes it easier to implement different GL backends for backends (like EGL/GLX on X11). We also pass a surfaceless boolean to make_current() so the calling code can decide if a surface needs to be bound or not, because the backends were all doing whatever, which was very counterproductive.
2021-07-07 02:40:34 +00:00
{
/* The OpenGL mac programming guide suggests that glFlush() is called
* before switching current contexts to ensure that the drawing commands
* are submitted.
macos: use CALayer and IOSurface for rendering This provides a major shift in how we draw both when accelerated OpenGL as well as software rendering with Cairo. In short, it uses tiles of Core Animation's CALayer to display contents from an OpenGL or Cairo rendering so that the window can provide partial damage updates. Partial damage is not generally available when using OpenGL as the whole buffer is flipped even if you only submitted a small change using a scissor rect. Thankfully, this speeds up Cairo rendering a bit too by using IOSurface to upload contents to the display server. We use the tiling system we do for OpenGL which reduces overall complexity and differences between them. A New Buffer ============ GdkMacosBuffer is a wrapper around an IOSurfaceRef. The term buffer was used because 1) surface is already used and 2) it loosely maps to a front/back buffer semantic. However, it appears that IOSurfaceRef contents are being retained in some fashion (likely in the compositor result) so we can update the same IOSurfaceRef without flipping as long as we're fast. This appears to be what Chromium does as well, but Firefox uses two IOSurfaceRef and flips between them. We would like to avoid two surfaces because it doubles the GPU VRAM requirements of the application. Changes to Windows ================== Previously, the NSWindow would dynamically change between different types of NSView based on the renderer being used. This is no longer necessary as we just have a single NSView type, GdkMacosView, which inherits from GdkMacosBaseView just to keep the tedius stuff separate from the machinery of GdkMacosView. We can merge those someday if we are okay with that. Changes to Views ================ GdkMacosCairoView, GdkMacosCairoSubView, GdkMacosGLView have all been removed and replaced with GdkMacosView. This new view has a single CALayer (GdkMacosLayer) attached to it which itself has sublayers. The contents of the CALayer is populated with an IOSurfaceRef which we allocated with the GdkMacosSurface. The surface is replaced when the NSWindow resizes. Changes to Layers ================= We now have a dedicated GdkMacosLayer which contains sublayers of GdkMacosTile. The tile has a maximum size of 128x128 pixels in device units. The GdkMacosTile is partitioned by splitting both the transparent region (window bounds minus opaque area) and then by splitting the opaque area. A tile has either translucent contents (and therefore is not opaque) or has opaque contents (and therefore is opaque). An opaque tile never contains transparent contents. As such, the opaque tiles contain a black background so that Core Animation will consider the tile's bounds as opaque. This can be verified with "Quartz Debug -> Show opaque regions". Changes to Cairo ================ GTK 4 cannot currently use cairo-quartz because of how CSS borders are rendered. It simply causes errors in the cairo_quartz_surface_t backend. Since we are restricted to using cairo_image_surface_t (which happens to be faster anyway) we can use the IOSurfaceBaseAddress() to obtain a mapping of the IOSurfaceRef in user-space. It always uses BGRA 32-bit with alpha channel even if we will discard the alpha channel as that is necessary to hit the fast paths in other parts of the platform. Note that while Cairo says CAIRO_FORMAT_ARGB32, it is really 32-bit BGRA on little-endian as we expect. OpenGL will render flipped (Quartz Native Co-ordinates) while Cairo renders with 0,O in the top-left. We could use cairo_translate() and cairo_scale() to reverse this, but it looks like some cairo things may not look quite as right if we do so. To reduce the chances of one-off bugs this continues to draw as Cairo would normally, but instead uses an CGAffineTransform in the tiles and some CGRect translation when swapping buffers to get the same effect. Changes to OpenGL ================= To simplify things, removal of all NSOpenGL* related components have been removed and we strictly use the Core GL (CGL*) API. This probably should have been done long ago anyay. Most examples found in the browsers to use IOSurfaceRef with OpenGL are using Legacy GL and there is still work underway to make this fit in with the rest of how the GSK GL renderer works. Since IOSurfaceRef bound to a texture/framebuffer will not have a default framebuffer ID of 0, we needed to add a default framebuffer id to the GdkGLContext. GskGLRenderer can use this to setup the command queue in such a way that our IOSurface destination has been glBindFramebuffer() as if it were the default drawable. This stuff is pretty slight-of-hand, so where things are and what needs flushing when and where has been a bit of an experiment to see what actually works to get synchronization across subsystems. Efficient Damages ================= After we draw with Cairo, we unlock the IOSurfaceRef and the contents are uploaded to the GPU. To make the contents visible to the app, we must clear the tiles contents with `layer.contents=nil;` and then re-apply the IOSurfaceRef. Since the buffer has likely not changed, we only do this if the tile overlaps the damage region. This gives the effect of having more tightly controlled damage regions even though updating the layer would damage be the whole window (as it is with OpenGL/Metal today with the exception of scissor-rect). This too can be verified usign "Quartz Debug -> Flash screen udpates". Frame Synchronized Resize ========================= In GTK 4, we have the ability to perform sizing changes from compute-size during the layout phase. Since the macOS backend already tracks window resizes manually, we can avoid doing the setFrame: immediately and instead do it within the frame clock's layout phase. Doing so gives us vastly better resize experience as we're more likely to get the size-change and updated-contents in the same frame on screen. It makes things feel "connected" in a way they weren't before. Some additional effort to tweak gravity during the process is also necessary but we were already doing that in the GTK 4 backend. Backporting =========== The design here has made an attempt to make it possible to backport by keeping GdkMacosBuffer, GdkMacosLayer, and GdkMacosTile fairly independent. There may be an opportunity to integrate this into GTK 3's quartz backend with a fair bit of work. Doing so could improve the situation for applications which are damage-rich such as The GIMP.
2022-02-14 10:20:19 +00:00
*
* TODO: investigate if we need this because we may switch contexts
* durring composition and only need it when returning to a
* previous context that uses the other context.
gl: Move vfunc Instead of Display::make_gl_context_current() we now have GLContext::clear_current() GLContext::make_current() This fits better with the backends (we can actually implement clearCurrent on macOS now) and makes it easier to implement different GL backends for backends (like EGL/GLX on X11). We also pass a surfaceless boolean to make_current() so the calling code can decide if a surface needs to be bound or not, because the backends were all doing whatever, which was very counterproductive.
2021-07-07 02:40:34 +00:00
*/
if (current != NULL)
glFlush ();
macos: use CALayer and IOSurface for rendering This provides a major shift in how we draw both when accelerated OpenGL as well as software rendering with Cairo. In short, it uses tiles of Core Animation's CALayer to display contents from an OpenGL or Cairo rendering so that the window can provide partial damage updates. Partial damage is not generally available when using OpenGL as the whole buffer is flipped even if you only submitted a small change using a scissor rect. Thankfully, this speeds up Cairo rendering a bit too by using IOSurface to upload contents to the display server. We use the tiling system we do for OpenGL which reduces overall complexity and differences between them. A New Buffer ============ GdkMacosBuffer is a wrapper around an IOSurfaceRef. The term buffer was used because 1) surface is already used and 2) it loosely maps to a front/back buffer semantic. However, it appears that IOSurfaceRef contents are being retained in some fashion (likely in the compositor result) so we can update the same IOSurfaceRef without flipping as long as we're fast. This appears to be what Chromium does as well, but Firefox uses two IOSurfaceRef and flips between them. We would like to avoid two surfaces because it doubles the GPU VRAM requirements of the application. Changes to Windows ================== Previously, the NSWindow would dynamically change between different types of NSView based on the renderer being used. This is no longer necessary as we just have a single NSView type, GdkMacosView, which inherits from GdkMacosBaseView just to keep the tedius stuff separate from the machinery of GdkMacosView. We can merge those someday if we are okay with that. Changes to Views ================ GdkMacosCairoView, GdkMacosCairoSubView, GdkMacosGLView have all been removed and replaced with GdkMacosView. This new view has a single CALayer (GdkMacosLayer) attached to it which itself has sublayers. The contents of the CALayer is populated with an IOSurfaceRef which we allocated with the GdkMacosSurface. The surface is replaced when the NSWindow resizes. Changes to Layers ================= We now have a dedicated GdkMacosLayer which contains sublayers of GdkMacosTile. The tile has a maximum size of 128x128 pixels in device units. The GdkMacosTile is partitioned by splitting both the transparent region (window bounds minus opaque area) and then by splitting the opaque area. A tile has either translucent contents (and therefore is not opaque) or has opaque contents (and therefore is opaque). An opaque tile never contains transparent contents. As such, the opaque tiles contain a black background so that Core Animation will consider the tile's bounds as opaque. This can be verified with "Quartz Debug -> Show opaque regions". Changes to Cairo ================ GTK 4 cannot currently use cairo-quartz because of how CSS borders are rendered. It simply causes errors in the cairo_quartz_surface_t backend. Since we are restricted to using cairo_image_surface_t (which happens to be faster anyway) we can use the IOSurfaceBaseAddress() to obtain a mapping of the IOSurfaceRef in user-space. It always uses BGRA 32-bit with alpha channel even if we will discard the alpha channel as that is necessary to hit the fast paths in other parts of the platform. Note that while Cairo says CAIRO_FORMAT_ARGB32, it is really 32-bit BGRA on little-endian as we expect. OpenGL will render flipped (Quartz Native Co-ordinates) while Cairo renders with 0,O in the top-left. We could use cairo_translate() and cairo_scale() to reverse this, but it looks like some cairo things may not look quite as right if we do so. To reduce the chances of one-off bugs this continues to draw as Cairo would normally, but instead uses an CGAffineTransform in the tiles and some CGRect translation when swapping buffers to get the same effect. Changes to OpenGL ================= To simplify things, removal of all NSOpenGL* related components have been removed and we strictly use the Core GL (CGL*) API. This probably should have been done long ago anyay. Most examples found in the browsers to use IOSurfaceRef with OpenGL are using Legacy GL and there is still work underway to make this fit in with the rest of how the GSK GL renderer works. Since IOSurfaceRef bound to a texture/framebuffer will not have a default framebuffer ID of 0, we needed to add a default framebuffer id to the GdkGLContext. GskGLRenderer can use this to setup the command queue in such a way that our IOSurface destination has been glBindFramebuffer() as if it were the default drawable. This stuff is pretty slight-of-hand, so where things are and what needs flushing when and where has been a bit of an experiment to see what actually works to get synchronization across subsystems. Efficient Damages ================= After we draw with Cairo, we unlock the IOSurfaceRef and the contents are uploaded to the GPU. To make the contents visible to the app, we must clear the tiles contents with `layer.contents=nil;` and then re-apply the IOSurfaceRef. Since the buffer has likely not changed, we only do this if the tile overlaps the damage region. This gives the effect of having more tightly controlled damage regions even though updating the layer would damage be the whole window (as it is with OpenGL/Metal today with the exception of scissor-rect). This too can be verified usign "Quartz Debug -> Flash screen udpates". Frame Synchronized Resize ========================= In GTK 4, we have the ability to perform sizing changes from compute-size during the layout phase. Since the macOS backend already tracks window resizes manually, we can avoid doing the setFrame: immediately and instead do it within the frame clock's layout phase. Doing so gives us vastly better resize experience as we're more likely to get the size-change and updated-contents in the same frame on screen. It makes things feel "connected" in a way they weren't before. Some additional effort to tweak gravity during the process is also necessary but we were already doing that in the GTK 4 backend. Backporting =========== The design here has made an attempt to make it possible to backport by keeping GdkMacosBuffer, GdkMacosLayer, and GdkMacosTile fairly independent. There may be an opportunity to integrate this into GTK 3's quartz backend with a fair bit of work. Doing so could improve the situation for applications which are damage-rich such as The GIMP.
2022-02-14 10:20:19 +00:00
CGLSetCurrentContext (self->cgl_context);
gl: Move vfunc Instead of Display::make_gl_context_current() we now have GLContext::clear_current() GLContext::make_current() This fits better with the backends (we can actually implement clearCurrent on macOS now) and makes it easier to implement different GL backends for backends (like EGL/GLX on X11). We also pass a surfaceless boolean to make_current() so the calling code can decide if a surface needs to be bound or not, because the backends were all doing whatever, which was very counterproductive.
2021-07-07 02:40:34 +00:00
}
return TRUE;
}
macos: clip damage when swapping buffers This ensures that we only copy the changed area.
2020-11-05 01:08:43 +00:00
static cairo_region_t *
gdk_macos_gl_context_get_damage (GdkGLContext *context)
{
GdkMacosGLContext *self = (GdkMacosGLContext *)context;
macos: use CALayer and IOSurface for rendering This provides a major shift in how we draw both when accelerated OpenGL as well as software rendering with Cairo. In short, it uses tiles of Core Animation's CALayer to display contents from an OpenGL or Cairo rendering so that the window can provide partial damage updates. Partial damage is not generally available when using OpenGL as the whole buffer is flipped even if you only submitted a small change using a scissor rect. Thankfully, this speeds up Cairo rendering a bit too by using IOSurface to upload contents to the display server. We use the tiling system we do for OpenGL which reduces overall complexity and differences between them. A New Buffer ============ GdkMacosBuffer is a wrapper around an IOSurfaceRef. The term buffer was used because 1) surface is already used and 2) it loosely maps to a front/back buffer semantic. However, it appears that IOSurfaceRef contents are being retained in some fashion (likely in the compositor result) so we can update the same IOSurfaceRef without flipping as long as we're fast. This appears to be what Chromium does as well, but Firefox uses two IOSurfaceRef and flips between them. We would like to avoid two surfaces because it doubles the GPU VRAM requirements of the application. Changes to Windows ================== Previously, the NSWindow would dynamically change between different types of NSView based on the renderer being used. This is no longer necessary as we just have a single NSView type, GdkMacosView, which inherits from GdkMacosBaseView just to keep the tedius stuff separate from the machinery of GdkMacosView. We can merge those someday if we are okay with that. Changes to Views ================ GdkMacosCairoView, GdkMacosCairoSubView, GdkMacosGLView have all been removed and replaced with GdkMacosView. This new view has a single CALayer (GdkMacosLayer) attached to it which itself has sublayers. The contents of the CALayer is populated with an IOSurfaceRef which we allocated with the GdkMacosSurface. The surface is replaced when the NSWindow resizes. Changes to Layers ================= We now have a dedicated GdkMacosLayer which contains sublayers of GdkMacosTile. The tile has a maximum size of 128x128 pixels in device units. The GdkMacosTile is partitioned by splitting both the transparent region (window bounds minus opaque area) and then by splitting the opaque area. A tile has either translucent contents (and therefore is not opaque) or has opaque contents (and therefore is opaque). An opaque tile never contains transparent contents. As such, the opaque tiles contain a black background so that Core Animation will consider the tile's bounds as opaque. This can be verified with "Quartz Debug -> Show opaque regions". Changes to Cairo ================ GTK 4 cannot currently use cairo-quartz because of how CSS borders are rendered. It simply causes errors in the cairo_quartz_surface_t backend. Since we are restricted to using cairo_image_surface_t (which happens to be faster anyway) we can use the IOSurfaceBaseAddress() to obtain a mapping of the IOSurfaceRef in user-space. It always uses BGRA 32-bit with alpha channel even if we will discard the alpha channel as that is necessary to hit the fast paths in other parts of the platform. Note that while Cairo says CAIRO_FORMAT_ARGB32, it is really 32-bit BGRA on little-endian as we expect. OpenGL will render flipped (Quartz Native Co-ordinates) while Cairo renders with 0,O in the top-left. We could use cairo_translate() and cairo_scale() to reverse this, but it looks like some cairo things may not look quite as right if we do so. To reduce the chances of one-off bugs this continues to draw as Cairo would normally, but instead uses an CGAffineTransform in the tiles and some CGRect translation when swapping buffers to get the same effect. Changes to OpenGL ================= To simplify things, removal of all NSOpenGL* related components have been removed and we strictly use the Core GL (CGL*) API. This probably should have been done long ago anyay. Most examples found in the browsers to use IOSurfaceRef with OpenGL are using Legacy GL and there is still work underway to make this fit in with the rest of how the GSK GL renderer works. Since IOSurfaceRef bound to a texture/framebuffer will not have a default framebuffer ID of 0, we needed to add a default framebuffer id to the GdkGLContext. GskGLRenderer can use this to setup the command queue in such a way that our IOSurface destination has been glBindFramebuffer() as if it were the default drawable. This stuff is pretty slight-of-hand, so where things are and what needs flushing when and where has been a bit of an experiment to see what actually works to get synchronization across subsystems. Efficient Damages ================= After we draw with Cairo, we unlock the IOSurfaceRef and the contents are uploaded to the GPU. To make the contents visible to the app, we must clear the tiles contents with `layer.contents=nil;` and then re-apply the IOSurfaceRef. Since the buffer has likely not changed, we only do this if the tile overlaps the damage region. This gives the effect of having more tightly controlled damage regions even though updating the layer would damage be the whole window (as it is with OpenGL/Metal today with the exception of scissor-rect). This too can be verified usign "Quartz Debug -> Flash screen udpates". Frame Synchronized Resize ========================= In GTK 4, we have the ability to perform sizing changes from compute-size during the layout phase. Since the macOS backend already tracks window resizes manually, we can avoid doing the setFrame: immediately and instead do it within the frame clock's layout phase. Doing so gives us vastly better resize experience as we're more likely to get the size-change and updated-contents in the same frame on screen. It makes things feel "connected" in a way they weren't before. Some additional effort to tweak gravity during the process is also necessary but we were already doing that in the GTK 4 backend. Backporting =========== The design here has made an attempt to make it possible to backport by keeping GdkMacosBuffer, GdkMacosLayer, and GdkMacosTile fairly independent. There may be an opportunity to integrate this into GTK 3's quartz backend with a fair bit of work. Doing so could improve the situation for applications which are damage-rich such as The GIMP.
2022-02-14 10:20:19 +00:00
if (self->damage)
macos: clip damage when swapping buffers This ensures that we only copy the changed area.
2020-11-05 01:08:43 +00:00
return cairo_region_copy (self->damage);
return GDK_GL_CONTEXT_CLASS (gdk_macos_gl_context_parent_class)->get_damage (context);
macos: implement GL context This implements the basics for a GdkGLContext on macOS. Currently, rendering only is fully working for the GskCairoRenderer case where we read back pixels into a cairo surface for rendering. More work on synchronization is required for the GL on GskGLRenderer case. When we attempt to render a surface itself with GL, the context will ensure that the new GdkMacosGLView is placed within the NSWindow. In other cases, we use a dummy NSView and NSWindow for backing the NSOpenGLContext to ensure that we can get accelerated drawing. This gets GtkGLArea working when running with GSK_RENDERER=cairo.
2020-10-29 17:33:22 +00:00
}
macos: use CALayer and IOSurface for rendering This provides a major shift in how we draw both when accelerated OpenGL as well as software rendering with Cairo. In short, it uses tiles of Core Animation's CALayer to display contents from an OpenGL or Cairo rendering so that the window can provide partial damage updates. Partial damage is not generally available when using OpenGL as the whole buffer is flipped even if you only submitted a small change using a scissor rect. Thankfully, this speeds up Cairo rendering a bit too by using IOSurface to upload contents to the display server. We use the tiling system we do for OpenGL which reduces overall complexity and differences between them. A New Buffer ============ GdkMacosBuffer is a wrapper around an IOSurfaceRef. The term buffer was used because 1) surface is already used and 2) it loosely maps to a front/back buffer semantic. However, it appears that IOSurfaceRef contents are being retained in some fashion (likely in the compositor result) so we can update the same IOSurfaceRef without flipping as long as we're fast. This appears to be what Chromium does as well, but Firefox uses two IOSurfaceRef and flips between them. We would like to avoid two surfaces because it doubles the GPU VRAM requirements of the application. Changes to Windows ================== Previously, the NSWindow would dynamically change between different types of NSView based on the renderer being used. This is no longer necessary as we just have a single NSView type, GdkMacosView, which inherits from GdkMacosBaseView just to keep the tedius stuff separate from the machinery of GdkMacosView. We can merge those someday if we are okay with that. Changes to Views ================ GdkMacosCairoView, GdkMacosCairoSubView, GdkMacosGLView have all been removed and replaced with GdkMacosView. This new view has a single CALayer (GdkMacosLayer) attached to it which itself has sublayers. The contents of the CALayer is populated with an IOSurfaceRef which we allocated with the GdkMacosSurface. The surface is replaced when the NSWindow resizes. Changes to Layers ================= We now have a dedicated GdkMacosLayer which contains sublayers of GdkMacosTile. The tile has a maximum size of 128x128 pixels in device units. The GdkMacosTile is partitioned by splitting both the transparent region (window bounds minus opaque area) and then by splitting the opaque area. A tile has either translucent contents (and therefore is not opaque) or has opaque contents (and therefore is opaque). An opaque tile never contains transparent contents. As such, the opaque tiles contain a black background so that Core Animation will consider the tile's bounds as opaque. This can be verified with "Quartz Debug -> Show opaque regions". Changes to Cairo ================ GTK 4 cannot currently use cairo-quartz because of how CSS borders are rendered. It simply causes errors in the cairo_quartz_surface_t backend. Since we are restricted to using cairo_image_surface_t (which happens to be faster anyway) we can use the IOSurfaceBaseAddress() to obtain a mapping of the IOSurfaceRef in user-space. It always uses BGRA 32-bit with alpha channel even if we will discard the alpha channel as that is necessary to hit the fast paths in other parts of the platform. Note that while Cairo says CAIRO_FORMAT_ARGB32, it is really 32-bit BGRA on little-endian as we expect. OpenGL will render flipped (Quartz Native Co-ordinates) while Cairo renders with 0,O in the top-left. We could use cairo_translate() and cairo_scale() to reverse this, but it looks like some cairo things may not look quite as right if we do so. To reduce the chances of one-off bugs this continues to draw as Cairo would normally, but instead uses an CGAffineTransform in the tiles and some CGRect translation when swapping buffers to get the same effect. Changes to OpenGL ================= To simplify things, removal of all NSOpenGL* related components have been removed and we strictly use the Core GL (CGL*) API. This probably should have been done long ago anyay. Most examples found in the browsers to use IOSurfaceRef with OpenGL are using Legacy GL and there is still work underway to make this fit in with the rest of how the GSK GL renderer works. Since IOSurfaceRef bound to a texture/framebuffer will not have a default framebuffer ID of 0, we needed to add a default framebuffer id to the GdkGLContext. GskGLRenderer can use this to setup the command queue in such a way that our IOSurface destination has been glBindFramebuffer() as if it were the default drawable. This stuff is pretty slight-of-hand, so where things are and what needs flushing when and where has been a bit of an experiment to see what actually works to get synchronization across subsystems. Efficient Damages ================= After we draw with Cairo, we unlock the IOSurfaceRef and the contents are uploaded to the GPU. To make the contents visible to the app, we must clear the tiles contents with `layer.contents=nil;` and then re-apply the IOSurfaceRef. Since the buffer has likely not changed, we only do this if the tile overlaps the damage region. This gives the effect of having more tightly controlled damage regions even though updating the layer would damage be the whole window (as it is with OpenGL/Metal today with the exception of scissor-rect). This too can be verified usign "Quartz Debug -> Flash screen udpates". Frame Synchronized Resize ========================= In GTK 4, we have the ability to perform sizing changes from compute-size during the layout phase. Since the macOS backend already tracks window resizes manually, we can avoid doing the setFrame: immediately and instead do it within the frame clock's layout phase. Doing so gives us vastly better resize experience as we're more likely to get the size-change and updated-contents in the same frame on screen. It makes things feel "connected" in a way they weren't before. Some additional effort to tweak gravity during the process is also necessary but we were already doing that in the GTK 4 backend. Backporting =========== The design here has made an attempt to make it possible to backport by keeping GdkMacosBuffer, GdkMacosLayer, and GdkMacosTile fairly independent. There may be an opportunity to integrate this into GTK 3's quartz backend with a fair bit of work. Doing so could improve the situation for applications which are damage-rich such as The GIMP.
2022-02-14 10:20:19 +00:00
static guint
gdk_macos_gl_context_get_default_framebuffer (GdkGLContext *context)
{
return GDK_MACOS_GL_CONTEXT (context)->fbo;
}
macos: prototype new GDK backend for macOS This is fairly substantial rewrite of the GDK backend for quartz and renamed to macOS to allow for a greenfield implementation. Many things have come across from the quartz implementation fairly intact such as the eventloop integration design and discovery of event windows from the NSEvent. However much has been changed to fit in with the new GDK design and how removal of child GdkWindow have been completely eliminated. Furthermore, the new GdkPopup allows for regular NSWindow to be used to provide popovers unlike the previous implementation. The object design more closely follows the ideal for a GDK backend. Views have been broken out into subclasses so that we can support multiple GSK renderer paths such as GL and Cairo (and Metal in the future). However mixed mode GL and Cairo will not be supported. Currently only the Cairo renderer has been implemented. A new frame clock implementation using CVDisplayLink provides more accurate information about when to draw drawing the next frame. Some testing will need to be done here to understand the power implications of this. This implementation has also gained edge snapping for CSD windows. Some work was also done to ensure that CSD windows have opaque regions registered with the display server. ** This is still very much a work-in-progress ** Some outstanding work that needs to be done: - Finish a GL context for macOS and alternate NSView for GL rendering (possibly using speciailized CALayer for OpenGL). - Input rework to ensure that we don't loose remapping of keys that was dropped from GDK during GTK 4 development. - Make sure input methods continue to work. - Drag-n-Drop is still very much a work in progress - High resolution input scrolling needs various work in GDK to land first before we can plumb that to NSEvent. - gtk/ has a number of things based on GDK_WINDOWING_QUARTZ that need to be updated to use the macOS backend. But this is good enough to start playing with and breaking things which is what I'd like to see.
2020-04-23 23:36:46 +00:00
static void
gdk_macos_gl_context_dispose (GObject *gobject)
{
macos: implement GL context This implements the basics for a GdkGLContext on macOS. Currently, rendering only is fully working for the GskCairoRenderer case where we read back pixels into a cairo surface for rendering. More work on synchronization is required for the GL on GskGLRenderer case. When we attempt to render a surface itself with GL, the context will ensure that the new GdkMacosGLView is placed within the NSWindow. In other cases, we use a dummy NSView and NSWindow for backing the NSOpenGLContext to ensure that we can get accelerated drawing. This gets GtkGLArea working when running with GSK_RENDERER=cairo.
2020-10-29 17:33:22 +00:00
GdkMacosGLContext *self = GDK_MACOS_GL_CONTEXT (gobject);
macos: use CALayer and IOSurface for rendering This provides a major shift in how we draw both when accelerated OpenGL as well as software rendering with Cairo. In short, it uses tiles of Core Animation's CALayer to display contents from an OpenGL or Cairo rendering so that the window can provide partial damage updates. Partial damage is not generally available when using OpenGL as the whole buffer is flipped even if you only submitted a small change using a scissor rect. Thankfully, this speeds up Cairo rendering a bit too by using IOSurface to upload contents to the display server. We use the tiling system we do for OpenGL which reduces overall complexity and differences between them. A New Buffer ============ GdkMacosBuffer is a wrapper around an IOSurfaceRef. The term buffer was used because 1) surface is already used and 2) it loosely maps to a front/back buffer semantic. However, it appears that IOSurfaceRef contents are being retained in some fashion (likely in the compositor result) so we can update the same IOSurfaceRef without flipping as long as we're fast. This appears to be what Chromium does as well, but Firefox uses two IOSurfaceRef and flips between them. We would like to avoid two surfaces because it doubles the GPU VRAM requirements of the application. Changes to Windows ================== Previously, the NSWindow would dynamically change between different types of NSView based on the renderer being used. This is no longer necessary as we just have a single NSView type, GdkMacosView, which inherits from GdkMacosBaseView just to keep the tedius stuff separate from the machinery of GdkMacosView. We can merge those someday if we are okay with that. Changes to Views ================ GdkMacosCairoView, GdkMacosCairoSubView, GdkMacosGLView have all been removed and replaced with GdkMacosView. This new view has a single CALayer (GdkMacosLayer) attached to it which itself has sublayers. The contents of the CALayer is populated with an IOSurfaceRef which we allocated with the GdkMacosSurface. The surface is replaced when the NSWindow resizes. Changes to Layers ================= We now have a dedicated GdkMacosLayer which contains sublayers of GdkMacosTile. The tile has a maximum size of 128x128 pixels in device units. The GdkMacosTile is partitioned by splitting both the transparent region (window bounds minus opaque area) and then by splitting the opaque area. A tile has either translucent contents (and therefore is not opaque) or has opaque contents (and therefore is opaque). An opaque tile never contains transparent contents. As such, the opaque tiles contain a black background so that Core Animation will consider the tile's bounds as opaque. This can be verified with "Quartz Debug -> Show opaque regions". Changes to Cairo ================ GTK 4 cannot currently use cairo-quartz because of how CSS borders are rendered. It simply causes errors in the cairo_quartz_surface_t backend. Since we are restricted to using cairo_image_surface_t (which happens to be faster anyway) we can use the IOSurfaceBaseAddress() to obtain a mapping of the IOSurfaceRef in user-space. It always uses BGRA 32-bit with alpha channel even if we will discard the alpha channel as that is necessary to hit the fast paths in other parts of the platform. Note that while Cairo says CAIRO_FORMAT_ARGB32, it is really 32-bit BGRA on little-endian as we expect. OpenGL will render flipped (Quartz Native Co-ordinates) while Cairo renders with 0,O in the top-left. We could use cairo_translate() and cairo_scale() to reverse this, but it looks like some cairo things may not look quite as right if we do so. To reduce the chances of one-off bugs this continues to draw as Cairo would normally, but instead uses an CGAffineTransform in the tiles and some CGRect translation when swapping buffers to get the same effect. Changes to OpenGL ================= To simplify things, removal of all NSOpenGL* related components have been removed and we strictly use the Core GL (CGL*) API. This probably should have been done long ago anyay. Most examples found in the browsers to use IOSurfaceRef with OpenGL are using Legacy GL and there is still work underway to make this fit in with the rest of how the GSK GL renderer works. Since IOSurfaceRef bound to a texture/framebuffer will not have a default framebuffer ID of 0, we needed to add a default framebuffer id to the GdkGLContext. GskGLRenderer can use this to setup the command queue in such a way that our IOSurface destination has been glBindFramebuffer() as if it were the default drawable. This stuff is pretty slight-of-hand, so where things are and what needs flushing when and where has been a bit of an experiment to see what actually works to get synchronization across subsystems. Efficient Damages ================= After we draw with Cairo, we unlock the IOSurfaceRef and the contents are uploaded to the GPU. To make the contents visible to the app, we must clear the tiles contents with `layer.contents=nil;` and then re-apply the IOSurfaceRef. Since the buffer has likely not changed, we only do this if the tile overlaps the damage region. This gives the effect of having more tightly controlled damage regions even though updating the layer would damage be the whole window (as it is with OpenGL/Metal today with the exception of scissor-rect). This too can be verified usign "Quartz Debug -> Flash screen udpates". Frame Synchronized Resize ========================= In GTK 4, we have the ability to perform sizing changes from compute-size during the layout phase. Since the macOS backend already tracks window resizes manually, we can avoid doing the setFrame: immediately and instead do it within the frame clock's layout phase. Doing so gives us vastly better resize experience as we're more likely to get the size-change and updated-contents in the same frame on screen. It makes things feel "connected" in a way they weren't before. Some additional effort to tweak gravity during the process is also necessary but we were already doing that in the GTK 4 backend. Backporting =========== The design here has made an attempt to make it possible to backport by keeping GdkMacosBuffer, GdkMacosLayer, and GdkMacosTile fairly independent. There may be an opportunity to integrate this into GTK 3's quartz backend with a fair bit of work. Doing so could improve the situation for applications which are damage-rich such as The GIMP.
2022-02-14 10:20:19 +00:00
self->texture = 0;
self->fbo = 0;
macos: implement GL context This implements the basics for a GdkGLContext on macOS. Currently, rendering only is fully working for the GskCairoRenderer case where we read back pixels into a cairo surface for rendering. More work on synchronization is required for the GL on GskGLRenderer case. When we attempt to render a surface itself with GL, the context will ensure that the new GdkMacosGLView is placed within the NSWindow. In other cases, we use a dummy NSView and NSWindow for backing the NSOpenGLContext to ensure that we can get accelerated drawing. This gets GtkGLArea working when running with GSK_RENDERER=cairo.
2020-10-29 17:33:22 +00:00
macos: use CALayer and IOSurface for rendering This provides a major shift in how we draw both when accelerated OpenGL as well as software rendering with Cairo. In short, it uses tiles of Core Animation's CALayer to display contents from an OpenGL or Cairo rendering so that the window can provide partial damage updates. Partial damage is not generally available when using OpenGL as the whole buffer is flipped even if you only submitted a small change using a scissor rect. Thankfully, this speeds up Cairo rendering a bit too by using IOSurface to upload contents to the display server. We use the tiling system we do for OpenGL which reduces overall complexity and differences between them. A New Buffer ============ GdkMacosBuffer is a wrapper around an IOSurfaceRef. The term buffer was used because 1) surface is already used and 2) it loosely maps to a front/back buffer semantic. However, it appears that IOSurfaceRef contents are being retained in some fashion (likely in the compositor result) so we can update the same IOSurfaceRef without flipping as long as we're fast. This appears to be what Chromium does as well, but Firefox uses two IOSurfaceRef and flips between them. We would like to avoid two surfaces because it doubles the GPU VRAM requirements of the application. Changes to Windows ================== Previously, the NSWindow would dynamically change between different types of NSView based on the renderer being used. This is no longer necessary as we just have a single NSView type, GdkMacosView, which inherits from GdkMacosBaseView just to keep the tedius stuff separate from the machinery of GdkMacosView. We can merge those someday if we are okay with that. Changes to Views ================ GdkMacosCairoView, GdkMacosCairoSubView, GdkMacosGLView have all been removed and replaced with GdkMacosView. This new view has a single CALayer (GdkMacosLayer) attached to it which itself has sublayers. The contents of the CALayer is populated with an IOSurfaceRef which we allocated with the GdkMacosSurface. The surface is replaced when the NSWindow resizes. Changes to Layers ================= We now have a dedicated GdkMacosLayer which contains sublayers of GdkMacosTile. The tile has a maximum size of 128x128 pixels in device units. The GdkMacosTile is partitioned by splitting both the transparent region (window bounds minus opaque area) and then by splitting the opaque area. A tile has either translucent contents (and therefore is not opaque) or has opaque contents (and therefore is opaque). An opaque tile never contains transparent contents. As such, the opaque tiles contain a black background so that Core Animation will consider the tile's bounds as opaque. This can be verified with "Quartz Debug -> Show opaque regions". Changes to Cairo ================ GTK 4 cannot currently use cairo-quartz because of how CSS borders are rendered. It simply causes errors in the cairo_quartz_surface_t backend. Since we are restricted to using cairo_image_surface_t (which happens to be faster anyway) we can use the IOSurfaceBaseAddress() to obtain a mapping of the IOSurfaceRef in user-space. It always uses BGRA 32-bit with alpha channel even if we will discard the alpha channel as that is necessary to hit the fast paths in other parts of the platform. Note that while Cairo says CAIRO_FORMAT_ARGB32, it is really 32-bit BGRA on little-endian as we expect. OpenGL will render flipped (Quartz Native Co-ordinates) while Cairo renders with 0,O in the top-left. We could use cairo_translate() and cairo_scale() to reverse this, but it looks like some cairo things may not look quite as right if we do so. To reduce the chances of one-off bugs this continues to draw as Cairo would normally, but instead uses an CGAffineTransform in the tiles and some CGRect translation when swapping buffers to get the same effect. Changes to OpenGL ================= To simplify things, removal of all NSOpenGL* related components have been removed and we strictly use the Core GL (CGL*) API. This probably should have been done long ago anyay. Most examples found in the browsers to use IOSurfaceRef with OpenGL are using Legacy GL and there is still work underway to make this fit in with the rest of how the GSK GL renderer works. Since IOSurfaceRef bound to a texture/framebuffer will not have a default framebuffer ID of 0, we needed to add a default framebuffer id to the GdkGLContext. GskGLRenderer can use this to setup the command queue in such a way that our IOSurface destination has been glBindFramebuffer() as if it were the default drawable. This stuff is pretty slight-of-hand, so where things are and what needs flushing when and where has been a bit of an experiment to see what actually works to get synchronization across subsystems. Efficient Damages ================= After we draw with Cairo, we unlock the IOSurfaceRef and the contents are uploaded to the GPU. To make the contents visible to the app, we must clear the tiles contents with `layer.contents=nil;` and then re-apply the IOSurfaceRef. Since the buffer has likely not changed, we only do this if the tile overlaps the damage region. This gives the effect of having more tightly controlled damage regions even though updating the layer would damage be the whole window (as it is with OpenGL/Metal today with the exception of scissor-rect). This too can be verified usign "Quartz Debug -> Flash screen udpates". Frame Synchronized Resize ========================= In GTK 4, we have the ability to perform sizing changes from compute-size during the layout phase. Since the macOS backend already tracks window resizes manually, we can avoid doing the setFrame: immediately and instead do it within the frame clock's layout phase. Doing so gives us vastly better resize experience as we're more likely to get the size-change and updated-contents in the same frame on screen. It makes things feel "connected" in a way they weren't before. Some additional effort to tweak gravity during the process is also necessary but we were already doing that in the GTK 4 backend. Backporting =========== The design here has made an attempt to make it possible to backport by keeping GdkMacosBuffer, GdkMacosLayer, and GdkMacosTile fairly independent. There may be an opportunity to integrate this into GTK 3's quartz backend with a fair bit of work. Doing so could improve the situation for applications which are damage-rich such as The GIMP.
2022-02-14 10:20:19 +00:00
if (self->cgl_context != nil)
macos: implement GL context This implements the basics for a GdkGLContext on macOS. Currently, rendering only is fully working for the GskCairoRenderer case where we read back pixels into a cairo surface for rendering. More work on synchronization is required for the GL on GskGLRenderer case. When we attempt to render a surface itself with GL, the context will ensure that the new GdkMacosGLView is placed within the NSWindow. In other cases, we use a dummy NSView and NSWindow for backing the NSOpenGLContext to ensure that we can get accelerated drawing. This gets GtkGLArea working when running with GSK_RENDERER=cairo.
2020-10-29 17:33:22 +00:00
{
macos: use CALayer and IOSurface for rendering This provides a major shift in how we draw both when accelerated OpenGL as well as software rendering with Cairo. In short, it uses tiles of Core Animation's CALayer to display contents from an OpenGL or Cairo rendering so that the window can provide partial damage updates. Partial damage is not generally available when using OpenGL as the whole buffer is flipped even if you only submitted a small change using a scissor rect. Thankfully, this speeds up Cairo rendering a bit too by using IOSurface to upload contents to the display server. We use the tiling system we do for OpenGL which reduces overall complexity and differences between them. A New Buffer ============ GdkMacosBuffer is a wrapper around an IOSurfaceRef. The term buffer was used because 1) surface is already used and 2) it loosely maps to a front/back buffer semantic. However, it appears that IOSurfaceRef contents are being retained in some fashion (likely in the compositor result) so we can update the same IOSurfaceRef without flipping as long as we're fast. This appears to be what Chromium does as well, but Firefox uses two IOSurfaceRef and flips between them. We would like to avoid two surfaces because it doubles the GPU VRAM requirements of the application. Changes to Windows ================== Previously, the NSWindow would dynamically change between different types of NSView based on the renderer being used. This is no longer necessary as we just have a single NSView type, GdkMacosView, which inherits from GdkMacosBaseView just to keep the tedius stuff separate from the machinery of GdkMacosView. We can merge those someday if we are okay with that. Changes to Views ================ GdkMacosCairoView, GdkMacosCairoSubView, GdkMacosGLView have all been removed and replaced with GdkMacosView. This new view has a single CALayer (GdkMacosLayer) attached to it which itself has sublayers. The contents of the CALayer is populated with an IOSurfaceRef which we allocated with the GdkMacosSurface. The surface is replaced when the NSWindow resizes. Changes to Layers ================= We now have a dedicated GdkMacosLayer which contains sublayers of GdkMacosTile. The tile has a maximum size of 128x128 pixels in device units. The GdkMacosTile is partitioned by splitting both the transparent region (window bounds minus opaque area) and then by splitting the opaque area. A tile has either translucent contents (and therefore is not opaque) or has opaque contents (and therefore is opaque). An opaque tile never contains transparent contents. As such, the opaque tiles contain a black background so that Core Animation will consider the tile's bounds as opaque. This can be verified with "Quartz Debug -> Show opaque regions". Changes to Cairo ================ GTK 4 cannot currently use cairo-quartz because of how CSS borders are rendered. It simply causes errors in the cairo_quartz_surface_t backend. Since we are restricted to using cairo_image_surface_t (which happens to be faster anyway) we can use the IOSurfaceBaseAddress() to obtain a mapping of the IOSurfaceRef in user-space. It always uses BGRA 32-bit with alpha channel even if we will discard the alpha channel as that is necessary to hit the fast paths in other parts of the platform. Note that while Cairo says CAIRO_FORMAT_ARGB32, it is really 32-bit BGRA on little-endian as we expect. OpenGL will render flipped (Quartz Native Co-ordinates) while Cairo renders with 0,O in the top-left. We could use cairo_translate() and cairo_scale() to reverse this, but it looks like some cairo things may not look quite as right if we do so. To reduce the chances of one-off bugs this continues to draw as Cairo would normally, but instead uses an CGAffineTransform in the tiles and some CGRect translation when swapping buffers to get the same effect. Changes to OpenGL ================= To simplify things, removal of all NSOpenGL* related components have been removed and we strictly use the Core GL (CGL*) API. This probably should have been done long ago anyay. Most examples found in the browsers to use IOSurfaceRef with OpenGL are using Legacy GL and there is still work underway to make this fit in with the rest of how the GSK GL renderer works. Since IOSurfaceRef bound to a texture/framebuffer will not have a default framebuffer ID of 0, we needed to add a default framebuffer id to the GdkGLContext. GskGLRenderer can use this to setup the command queue in such a way that our IOSurface destination has been glBindFramebuffer() as if it were the default drawable. This stuff is pretty slight-of-hand, so where things are and what needs flushing when and where has been a bit of an experiment to see what actually works to get synchronization across subsystems. Efficient Damages ================= After we draw with Cairo, we unlock the IOSurfaceRef and the contents are uploaded to the GPU. To make the contents visible to the app, we must clear the tiles contents with `layer.contents=nil;` and then re-apply the IOSurfaceRef. Since the buffer has likely not changed, we only do this if the tile overlaps the damage region. This gives the effect of having more tightly controlled damage regions even though updating the layer would damage be the whole window (as it is with OpenGL/Metal today with the exception of scissor-rect). This too can be verified usign "Quartz Debug -> Flash screen udpates". Frame Synchronized Resize ========================= In GTK 4, we have the ability to perform sizing changes from compute-size during the layout phase. Since the macOS backend already tracks window resizes manually, we can avoid doing the setFrame: immediately and instead do it within the frame clock's layout phase. Doing so gives us vastly better resize experience as we're more likely to get the size-change and updated-contents in the same frame on screen. It makes things feel "connected" in a way they weren't before. Some additional effort to tweak gravity during the process is also necessary but we were already doing that in the GTK 4 backend. Backporting =========== The design here has made an attempt to make it possible to backport by keeping GdkMacosBuffer, GdkMacosLayer, and GdkMacosTile fairly independent. There may be an opportunity to integrate this into GTK 3's quartz backend with a fair bit of work. Doing so could improve the situation for applications which are damage-rich such as The GIMP.
2022-02-14 10:20:19 +00:00
CGLContextObj cgl_context = g_steal_pointer (&self->cgl_context);
macos: implement GL context This implements the basics for a GdkGLContext on macOS. Currently, rendering only is fully working for the GskCairoRenderer case where we read back pixels into a cairo surface for rendering. More work on synchronization is required for the GL on GskGLRenderer case. When we attempt to render a surface itself with GL, the context will ensure that the new GdkMacosGLView is placed within the NSWindow. In other cases, we use a dummy NSView and NSWindow for backing the NSOpenGLContext to ensure that we can get accelerated drawing. This gets GtkGLArea working when running with GSK_RENDERER=cairo.
2020-10-29 17:33:22 +00:00
macos: use CALayer and IOSurface for rendering This provides a major shift in how we draw both when accelerated OpenGL as well as software rendering with Cairo. In short, it uses tiles of Core Animation's CALayer to display contents from an OpenGL or Cairo rendering so that the window can provide partial damage updates. Partial damage is not generally available when using OpenGL as the whole buffer is flipped even if you only submitted a small change using a scissor rect. Thankfully, this speeds up Cairo rendering a bit too by using IOSurface to upload contents to the display server. We use the tiling system we do for OpenGL which reduces overall complexity and differences between them. A New Buffer ============ GdkMacosBuffer is a wrapper around an IOSurfaceRef. The term buffer was used because 1) surface is already used and 2) it loosely maps to a front/back buffer semantic. However, it appears that IOSurfaceRef contents are being retained in some fashion (likely in the compositor result) so we can update the same IOSurfaceRef without flipping as long as we're fast. This appears to be what Chromium does as well, but Firefox uses two IOSurfaceRef and flips between them. We would like to avoid two surfaces because it doubles the GPU VRAM requirements of the application. Changes to Windows ================== Previously, the NSWindow would dynamically change between different types of NSView based on the renderer being used. This is no longer necessary as we just have a single NSView type, GdkMacosView, which inherits from GdkMacosBaseView just to keep the tedius stuff separate from the machinery of GdkMacosView. We can merge those someday if we are okay with that. Changes to Views ================ GdkMacosCairoView, GdkMacosCairoSubView, GdkMacosGLView have all been removed and replaced with GdkMacosView. This new view has a single CALayer (GdkMacosLayer) attached to it which itself has sublayers. The contents of the CALayer is populated with an IOSurfaceRef which we allocated with the GdkMacosSurface. The surface is replaced when the NSWindow resizes. Changes to Layers ================= We now have a dedicated GdkMacosLayer which contains sublayers of GdkMacosTile. The tile has a maximum size of 128x128 pixels in device units. The GdkMacosTile is partitioned by splitting both the transparent region (window bounds minus opaque area) and then by splitting the opaque area. A tile has either translucent contents (and therefore is not opaque) or has opaque contents (and therefore is opaque). An opaque tile never contains transparent contents. As such, the opaque tiles contain a black background so that Core Animation will consider the tile's bounds as opaque. This can be verified with "Quartz Debug -> Show opaque regions". Changes to Cairo ================ GTK 4 cannot currently use cairo-quartz because of how CSS borders are rendered. It simply causes errors in the cairo_quartz_surface_t backend. Since we are restricted to using cairo_image_surface_t (which happens to be faster anyway) we can use the IOSurfaceBaseAddress() to obtain a mapping of the IOSurfaceRef in user-space. It always uses BGRA 32-bit with alpha channel even if we will discard the alpha channel as that is necessary to hit the fast paths in other parts of the platform. Note that while Cairo says CAIRO_FORMAT_ARGB32, it is really 32-bit BGRA on little-endian as we expect. OpenGL will render flipped (Quartz Native Co-ordinates) while Cairo renders with 0,O in the top-left. We could use cairo_translate() and cairo_scale() to reverse this, but it looks like some cairo things may not look quite as right if we do so. To reduce the chances of one-off bugs this continues to draw as Cairo would normally, but instead uses an CGAffineTransform in the tiles and some CGRect translation when swapping buffers to get the same effect. Changes to OpenGL ================= To simplify things, removal of all NSOpenGL* related components have been removed and we strictly use the Core GL (CGL*) API. This probably should have been done long ago anyay. Most examples found in the browsers to use IOSurfaceRef with OpenGL are using Legacy GL and there is still work underway to make this fit in with the rest of how the GSK GL renderer works. Since IOSurfaceRef bound to a texture/framebuffer will not have a default framebuffer ID of 0, we needed to add a default framebuffer id to the GdkGLContext. GskGLRenderer can use this to setup the command queue in such a way that our IOSurface destination has been glBindFramebuffer() as if it were the default drawable. This stuff is pretty slight-of-hand, so where things are and what needs flushing when and where has been a bit of an experiment to see what actually works to get synchronization across subsystems. Efficient Damages ================= After we draw with Cairo, we unlock the IOSurfaceRef and the contents are uploaded to the GPU. To make the contents visible to the app, we must clear the tiles contents with `layer.contents=nil;` and then re-apply the IOSurfaceRef. Since the buffer has likely not changed, we only do this if the tile overlaps the damage region. This gives the effect of having more tightly controlled damage regions even though updating the layer would damage be the whole window (as it is with OpenGL/Metal today with the exception of scissor-rect). This too can be verified usign "Quartz Debug -> Flash screen udpates". Frame Synchronized Resize ========================= In GTK 4, we have the ability to perform sizing changes from compute-size during the layout phase. Since the macOS backend already tracks window resizes manually, we can avoid doing the setFrame: immediately and instead do it within the frame clock's layout phase. Doing so gives us vastly better resize experience as we're more likely to get the size-change and updated-contents in the same frame on screen. It makes things feel "connected" in a way they weren't before. Some additional effort to tweak gravity during the process is also necessary but we were already doing that in the GTK 4 backend. Backporting =========== The design here has made an attempt to make it possible to backport by keeping GdkMacosBuffer, GdkMacosLayer, and GdkMacosTile fairly independent. There may be an opportunity to integrate this into GTK 3's quartz backend with a fair bit of work. Doing so could improve the situation for applications which are damage-rich such as The GIMP.
2022-02-14 10:20:19 +00:00
if (cgl_context == CGLGetCurrentContext ())
CGLSetCurrentContext (NULL);
macos: implement GL context This implements the basics for a GdkGLContext on macOS. Currently, rendering only is fully working for the GskCairoRenderer case where we read back pixels into a cairo surface for rendering. More work on synchronization is required for the GL on GskGLRenderer case. When we attempt to render a surface itself with GL, the context will ensure that the new GdkMacosGLView is placed within the NSWindow. In other cases, we use a dummy NSView and NSWindow for backing the NSOpenGLContext to ensure that we can get accelerated drawing. This gets GtkGLArea working when running with GSK_RENDERER=cairo.
2020-10-29 17:33:22 +00:00
macos: use CALayer and IOSurface for rendering This provides a major shift in how we draw both when accelerated OpenGL as well as software rendering with Cairo. In short, it uses tiles of Core Animation's CALayer to display contents from an OpenGL or Cairo rendering so that the window can provide partial damage updates. Partial damage is not generally available when using OpenGL as the whole buffer is flipped even if you only submitted a small change using a scissor rect. Thankfully, this speeds up Cairo rendering a bit too by using IOSurface to upload contents to the display server. We use the tiling system we do for OpenGL which reduces overall complexity and differences between them. A New Buffer ============ GdkMacosBuffer is a wrapper around an IOSurfaceRef. The term buffer was used because 1) surface is already used and 2) it loosely maps to a front/back buffer semantic. However, it appears that IOSurfaceRef contents are being retained in some fashion (likely in the compositor result) so we can update the same IOSurfaceRef without flipping as long as we're fast. This appears to be what Chromium does as well, but Firefox uses two IOSurfaceRef and flips between them. We would like to avoid two surfaces because it doubles the GPU VRAM requirements of the application. Changes to Windows ================== Previously, the NSWindow would dynamically change between different types of NSView based on the renderer being used. This is no longer necessary as we just have a single NSView type, GdkMacosView, which inherits from GdkMacosBaseView just to keep the tedius stuff separate from the machinery of GdkMacosView. We can merge those someday if we are okay with that. Changes to Views ================ GdkMacosCairoView, GdkMacosCairoSubView, GdkMacosGLView have all been removed and replaced with GdkMacosView. This new view has a single CALayer (GdkMacosLayer) attached to it which itself has sublayers. The contents of the CALayer is populated with an IOSurfaceRef which we allocated with the GdkMacosSurface. The surface is replaced when the NSWindow resizes. Changes to Layers ================= We now have a dedicated GdkMacosLayer which contains sublayers of GdkMacosTile. The tile has a maximum size of 128x128 pixels in device units. The GdkMacosTile is partitioned by splitting both the transparent region (window bounds minus opaque area) and then by splitting the opaque area. A tile has either translucent contents (and therefore is not opaque) or has opaque contents (and therefore is opaque). An opaque tile never contains transparent contents. As such, the opaque tiles contain a black background so that Core Animation will consider the tile's bounds as opaque. This can be verified with "Quartz Debug -> Show opaque regions". Changes to Cairo ================ GTK 4 cannot currently use cairo-quartz because of how CSS borders are rendered. It simply causes errors in the cairo_quartz_surface_t backend. Since we are restricted to using cairo_image_surface_t (which happens to be faster anyway) we can use the IOSurfaceBaseAddress() to obtain a mapping of the IOSurfaceRef in user-space. It always uses BGRA 32-bit with alpha channel even if we will discard the alpha channel as that is necessary to hit the fast paths in other parts of the platform. Note that while Cairo says CAIRO_FORMAT_ARGB32, it is really 32-bit BGRA on little-endian as we expect. OpenGL will render flipped (Quartz Native Co-ordinates) while Cairo renders with 0,O in the top-left. We could use cairo_translate() and cairo_scale() to reverse this, but it looks like some cairo things may not look quite as right if we do so. To reduce the chances of one-off bugs this continues to draw as Cairo would normally, but instead uses an CGAffineTransform in the tiles and some CGRect translation when swapping buffers to get the same effect. Changes to OpenGL ================= To simplify things, removal of all NSOpenGL* related components have been removed and we strictly use the Core GL (CGL*) API. This probably should have been done long ago anyay. Most examples found in the browsers to use IOSurfaceRef with OpenGL are using Legacy GL and there is still work underway to make this fit in with the rest of how the GSK GL renderer works. Since IOSurfaceRef bound to a texture/framebuffer will not have a default framebuffer ID of 0, we needed to add a default framebuffer id to the GdkGLContext. GskGLRenderer can use this to setup the command queue in such a way that our IOSurface destination has been glBindFramebuffer() as if it were the default drawable. This stuff is pretty slight-of-hand, so where things are and what needs flushing when and where has been a bit of an experiment to see what actually works to get synchronization across subsystems. Efficient Damages ================= After we draw with Cairo, we unlock the IOSurfaceRef and the contents are uploaded to the GPU. To make the contents visible to the app, we must clear the tiles contents with `layer.contents=nil;` and then re-apply the IOSurfaceRef. Since the buffer has likely not changed, we only do this if the tile overlaps the damage region. This gives the effect of having more tightly controlled damage regions even though updating the layer would damage be the whole window (as it is with OpenGL/Metal today with the exception of scissor-rect). This too can be verified usign "Quartz Debug -> Flash screen udpates". Frame Synchronized Resize ========================= In GTK 4, we have the ability to perform sizing changes from compute-size during the layout phase. Since the macOS backend already tracks window resizes manually, we can avoid doing the setFrame: immediately and instead do it within the frame clock's layout phase. Doing so gives us vastly better resize experience as we're more likely to get the size-change and updated-contents in the same frame on screen. It makes things feel "connected" in a way they weren't before. Some additional effort to tweak gravity during the process is also necessary but we were already doing that in the GTK 4 backend. Backporting =========== The design here has made an attempt to make it possible to backport by keeping GdkMacosBuffer, GdkMacosLayer, and GdkMacosTile fairly independent. There may be an opportunity to integrate this into GTK 3's quartz backend with a fair bit of work. Doing so could improve the situation for applications which are damage-rich such as The GIMP.
2022-02-14 10:20:19 +00:00
CGLClearDrawable (cgl_context);
CGLDestroyContext (cgl_context);
macos: prototype new GDK backend for macOS This is fairly substantial rewrite of the GDK backend for quartz and renamed to macOS to allow for a greenfield implementation. Many things have come across from the quartz implementation fairly intact such as the eventloop integration design and discovery of event windows from the NSEvent. However much has been changed to fit in with the new GDK design and how removal of child GdkWindow have been completely eliminated. Furthermore, the new GdkPopup allows for regular NSWindow to be used to provide popovers unlike the previous implementation. The object design more closely follows the ideal for a GDK backend. Views have been broken out into subclasses so that we can support multiple GSK renderer paths such as GL and Cairo (and Metal in the future). However mixed mode GL and Cairo will not be supported. Currently only the Cairo renderer has been implemented. A new frame clock implementation using CVDisplayLink provides more accurate information about when to draw drawing the next frame. Some testing will need to be done here to understand the power implications of this. This implementation has also gained edge snapping for CSD windows. Some work was also done to ensure that CSD windows have opaque regions registered with the display server. ** This is still very much a work-in-progress ** Some outstanding work that needs to be done: - Finish a GL context for macOS and alternate NSView for GL rendering (possibly using speciailized CALayer for OpenGL). - Input rework to ensure that we don't loose remapping of keys that was dropped from GDK during GTK 4 development. - Make sure input methods continue to work. - Drag-n-Drop is still very much a work in progress - High resolution input scrolling needs various work in GDK to land first before we can plumb that to NSEvent. - gtk/ has a number of things based on GDK_WINDOWING_QUARTZ that need to be updated to use the macOS backend. But this is good enough to start playing with and breaking things which is what I'd like to see.
2020-04-23 23:36:46 +00:00
}
macos: clip damage when swapping buffers This ensures that we only copy the changed area.
2020-11-05 01:08:43 +00:00
g_clear_pointer (&self->damage, cairo_region_destroy);
macos: prototype new GDK backend for macOS This is fairly substantial rewrite of the GDK backend for quartz and renamed to macOS to allow for a greenfield implementation. Many things have come across from the quartz implementation fairly intact such as the eventloop integration design and discovery of event windows from the NSEvent. However much has been changed to fit in with the new GDK design and how removal of child GdkWindow have been completely eliminated. Furthermore, the new GdkPopup allows for regular NSWindow to be used to provide popovers unlike the previous implementation. The object design more closely follows the ideal for a GDK backend. Views have been broken out into subclasses so that we can support multiple GSK renderer paths such as GL and Cairo (and Metal in the future). However mixed mode GL and Cairo will not be supported. Currently only the Cairo renderer has been implemented. A new frame clock implementation using CVDisplayLink provides more accurate information about when to draw drawing the next frame. Some testing will need to be done here to understand the power implications of this. This implementation has also gained edge snapping for CSD windows. Some work was also done to ensure that CSD windows have opaque regions registered with the display server. ** This is still very much a work-in-progress ** Some outstanding work that needs to be done: - Finish a GL context for macOS and alternate NSView for GL rendering (possibly using speciailized CALayer for OpenGL). - Input rework to ensure that we don't loose remapping of keys that was dropped from GDK during GTK 4 development. - Make sure input methods continue to work. - Drag-n-Drop is still very much a work in progress - High resolution input scrolling needs various work in GDK to land first before we can plumb that to NSEvent. - gtk/ has a number of things based on GDK_WINDOWING_QUARTZ that need to be updated to use the macOS backend. But this is good enough to start playing with and breaking things which is what I'd like to see.
2020-04-23 23:36:46 +00:00
G_OBJECT_CLASS (gdk_macos_gl_context_parent_class)->dispose (gobject);
}
static void
gdk_macos_gl_context_class_init (GdkMacosGLContextClass *klass)
{
GObjectClass *object_class = G_OBJECT_CLASS (klass);
GdkDrawContextClass *draw_context_class = GDK_DRAW_CONTEXT_CLASS (klass);
macos: implement GL context This implements the basics for a GdkGLContext on macOS. Currently, rendering only is fully working for the GskCairoRenderer case where we read back pixels into a cairo surface for rendering. More work on synchronization is required for the GL on GskGLRenderer case. When we attempt to render a surface itself with GL, the context will ensure that the new GdkMacosGLView is placed within the NSWindow. In other cases, we use a dummy NSView and NSWindow for backing the NSOpenGLContext to ensure that we can get accelerated drawing. This gets GtkGLArea working when running with GSK_RENDERER=cairo.
2020-10-29 17:33:22 +00:00
GdkGLContextClass *gl_class = GDK_GL_CONTEXT_CLASS (klass);
macos: prototype new GDK backend for macOS This is fairly substantial rewrite of the GDK backend for quartz and renamed to macOS to allow for a greenfield implementation. Many things have come across from the quartz implementation fairly intact such as the eventloop integration design and discovery of event windows from the NSEvent. However much has been changed to fit in with the new GDK design and how removal of child GdkWindow have been completely eliminated. Furthermore, the new GdkPopup allows for regular NSWindow to be used to provide popovers unlike the previous implementation. The object design more closely follows the ideal for a GDK backend. Views have been broken out into subclasses so that we can support multiple GSK renderer paths such as GL and Cairo (and Metal in the future). However mixed mode GL and Cairo will not be supported. Currently only the Cairo renderer has been implemented. A new frame clock implementation using CVDisplayLink provides more accurate information about when to draw drawing the next frame. Some testing will need to be done here to understand the power implications of this. This implementation has also gained edge snapping for CSD windows. Some work was also done to ensure that CSD windows have opaque regions registered with the display server. ** This is still very much a work-in-progress ** Some outstanding work that needs to be done: - Finish a GL context for macOS and alternate NSView for GL rendering (possibly using speciailized CALayer for OpenGL). - Input rework to ensure that we don't loose remapping of keys that was dropped from GDK during GTK 4 development. - Make sure input methods continue to work. - Drag-n-Drop is still very much a work in progress - High resolution input scrolling needs various work in GDK to land first before we can plumb that to NSEvent. - gtk/ has a number of things based on GDK_WINDOWING_QUARTZ that need to be updated to use the macOS backend. But this is good enough to start playing with and breaking things which is what I'd like to see.
2020-04-23 23:36:46 +00:00
object_class->dispose = gdk_macos_gl_context_dispose;
macos: implement GL context This implements the basics for a GdkGLContext on macOS. Currently, rendering only is fully working for the GskCairoRenderer case where we read back pixels into a cairo surface for rendering. More work on synchronization is required for the GL on GskGLRenderer case. When we attempt to render a surface itself with GL, the context will ensure that the new GdkMacosGLView is placed within the NSWindow. In other cases, we use a dummy NSView and NSWindow for backing the NSOpenGLContext to ensure that we can get accelerated drawing. This gets GtkGLArea working when running with GSK_RENDERER=cairo.
2020-10-29 17:33:22 +00:00
draw_context_class->begin_frame = gdk_macos_gl_context_begin_frame;
macos: prototype new GDK backend for macOS This is fairly substantial rewrite of the GDK backend for quartz and renamed to macOS to allow for a greenfield implementation. Many things have come across from the quartz implementation fairly intact such as the eventloop integration design and discovery of event windows from the NSEvent. However much has been changed to fit in with the new GDK design and how removal of child GdkWindow have been completely eliminated. Furthermore, the new GdkPopup allows for regular NSWindow to be used to provide popovers unlike the previous implementation. The object design more closely follows the ideal for a GDK backend. Views have been broken out into subclasses so that we can support multiple GSK renderer paths such as GL and Cairo (and Metal in the future). However mixed mode GL and Cairo will not be supported. Currently only the Cairo renderer has been implemented. A new frame clock implementation using CVDisplayLink provides more accurate information about when to draw drawing the next frame. Some testing will need to be done here to understand the power implications of this. This implementation has also gained edge snapping for CSD windows. Some work was also done to ensure that CSD windows have opaque regions registered with the display server. ** This is still very much a work-in-progress ** Some outstanding work that needs to be done: - Finish a GL context for macOS and alternate NSView for GL rendering (possibly using speciailized CALayer for OpenGL). - Input rework to ensure that we don't loose remapping of keys that was dropped from GDK during GTK 4 development. - Make sure input methods continue to work. - Drag-n-Drop is still very much a work in progress - High resolution input scrolling needs various work in GDK to land first before we can plumb that to NSEvent. - gtk/ has a number of things based on GDK_WINDOWING_QUARTZ that need to be updated to use the macOS backend. But this is good enough to start playing with and breaking things which is what I'd like to see.
2020-04-23 23:36:46 +00:00
draw_context_class->end_frame = gdk_macos_gl_context_end_frame;
macos: implement GL context This implements the basics for a GdkGLContext on macOS. Currently, rendering only is fully working for the GskCairoRenderer case where we read back pixels into a cairo surface for rendering. More work on synchronization is required for the GL on GskGLRenderer case. When we attempt to render a surface itself with GL, the context will ensure that the new GdkMacosGLView is placed within the NSWindow. In other cases, we use a dummy NSView and NSWindow for backing the NSOpenGLContext to ensure that we can get accelerated drawing. This gets GtkGLArea working when running with GSK_RENDERER=cairo.
2020-10-29 17:33:22 +00:00
draw_context_class->surface_resized = gdk_macos_gl_context_surface_resized;
macos: clip damage when swapping buffers This ensures that we only copy the changed area.
2020-11-05 01:08:43 +00:00
gl_class->get_damage = gdk_macos_gl_context_get_damage;
macos: Actually set the vfuncs
2021-09-24 20:11:22 +00:00
gl_class->clear_current = gdk_macos_gl_context_clear_current;
gl_class->make_current = gdk_macos_gl_context_make_current;
macos: implement GL context This implements the basics for a GdkGLContext on macOS. Currently, rendering only is fully working for the GskCairoRenderer case where we read back pixels into a cairo surface for rendering. More work on synchronization is required for the GL on GskGLRenderer case. When we attempt to render a surface itself with GL, the context will ensure that the new GdkMacosGLView is placed within the NSWindow. In other cases, we use a dummy NSView and NSWindow for backing the NSOpenGLContext to ensure that we can get accelerated drawing. This gets GtkGLArea working when running with GSK_RENDERER=cairo.
2020-10-29 17:33:22 +00:00
gl_class->realize = gdk_macos_gl_context_real_realize;
macos: use CALayer and IOSurface for rendering This provides a major shift in how we draw both when accelerated OpenGL as well as software rendering with Cairo. In short, it uses tiles of Core Animation's CALayer to display contents from an OpenGL or Cairo rendering so that the window can provide partial damage updates. Partial damage is not generally available when using OpenGL as the whole buffer is flipped even if you only submitted a small change using a scissor rect. Thankfully, this speeds up Cairo rendering a bit too by using IOSurface to upload contents to the display server. We use the tiling system we do for OpenGL which reduces overall complexity and differences between them. A New Buffer ============ GdkMacosBuffer is a wrapper around an IOSurfaceRef. The term buffer was used because 1) surface is already used and 2) it loosely maps to a front/back buffer semantic. However, it appears that IOSurfaceRef contents are being retained in some fashion (likely in the compositor result) so we can update the same IOSurfaceRef without flipping as long as we're fast. This appears to be what Chromium does as well, but Firefox uses two IOSurfaceRef and flips between them. We would like to avoid two surfaces because it doubles the GPU VRAM requirements of the application. Changes to Windows ================== Previously, the NSWindow would dynamically change between different types of NSView based on the renderer being used. This is no longer necessary as we just have a single NSView type, GdkMacosView, which inherits from GdkMacosBaseView just to keep the tedius stuff separate from the machinery of GdkMacosView. We can merge those someday if we are okay with that. Changes to Views ================ GdkMacosCairoView, GdkMacosCairoSubView, GdkMacosGLView have all been removed and replaced with GdkMacosView. This new view has a single CALayer (GdkMacosLayer) attached to it which itself has sublayers. The contents of the CALayer is populated with an IOSurfaceRef which we allocated with the GdkMacosSurface. The surface is replaced when the NSWindow resizes. Changes to Layers ================= We now have a dedicated GdkMacosLayer which contains sublayers of GdkMacosTile. The tile has a maximum size of 128x128 pixels in device units. The GdkMacosTile is partitioned by splitting both the transparent region (window bounds minus opaque area) and then by splitting the opaque area. A tile has either translucent contents (and therefore is not opaque) or has opaque contents (and therefore is opaque). An opaque tile never contains transparent contents. As such, the opaque tiles contain a black background so that Core Animation will consider the tile's bounds as opaque. This can be verified with "Quartz Debug -> Show opaque regions". Changes to Cairo ================ GTK 4 cannot currently use cairo-quartz because of how CSS borders are rendered. It simply causes errors in the cairo_quartz_surface_t backend. Since we are restricted to using cairo_image_surface_t (which happens to be faster anyway) we can use the IOSurfaceBaseAddress() to obtain a mapping of the IOSurfaceRef in user-space. It always uses BGRA 32-bit with alpha channel even if we will discard the alpha channel as that is necessary to hit the fast paths in other parts of the platform. Note that while Cairo says CAIRO_FORMAT_ARGB32, it is really 32-bit BGRA on little-endian as we expect. OpenGL will render flipped (Quartz Native Co-ordinates) while Cairo renders with 0,O in the top-left. We could use cairo_translate() and cairo_scale() to reverse this, but it looks like some cairo things may not look quite as right if we do so. To reduce the chances of one-off bugs this continues to draw as Cairo would normally, but instead uses an CGAffineTransform in the tiles and some CGRect translation when swapping buffers to get the same effect. Changes to OpenGL ================= To simplify things, removal of all NSOpenGL* related components have been removed and we strictly use the Core GL (CGL*) API. This probably should have been done long ago anyay. Most examples found in the browsers to use IOSurfaceRef with OpenGL are using Legacy GL and there is still work underway to make this fit in with the rest of how the GSK GL renderer works. Since IOSurfaceRef bound to a texture/framebuffer will not have a default framebuffer ID of 0, we needed to add a default framebuffer id to the GdkGLContext. GskGLRenderer can use this to setup the command queue in such a way that our IOSurface destination has been glBindFramebuffer() as if it were the default drawable. This stuff is pretty slight-of-hand, so where things are and what needs flushing when and where has been a bit of an experiment to see what actually works to get synchronization across subsystems. Efficient Damages ================= After we draw with Cairo, we unlock the IOSurfaceRef and the contents are uploaded to the GPU. To make the contents visible to the app, we must clear the tiles contents with `layer.contents=nil;` and then re-apply the IOSurfaceRef. Since the buffer has likely not changed, we only do this if the tile overlaps the damage region. This gives the effect of having more tightly controlled damage regions even though updating the layer would damage be the whole window (as it is with OpenGL/Metal today with the exception of scissor-rect). This too can be verified usign "Quartz Debug -> Flash screen udpates". Frame Synchronized Resize ========================= In GTK 4, we have the ability to perform sizing changes from compute-size during the layout phase. Since the macOS backend already tracks window resizes manually, we can avoid doing the setFrame: immediately and instead do it within the frame clock's layout phase. Doing so gives us vastly better resize experience as we're more likely to get the size-change and updated-contents in the same frame on screen. It makes things feel "connected" in a way they weren't before. Some additional effort to tweak gravity during the process is also necessary but we were already doing that in the GTK 4 backend. Backporting =========== The design here has made an attempt to make it possible to backport by keeping GdkMacosBuffer, GdkMacosLayer, and GdkMacosTile fairly independent. There may be an opportunity to integrate this into GTK 3's quartz backend with a fair bit of work. Doing so could improve the situation for applications which are damage-rich such as The GIMP.
2022-02-14 10:20:19 +00:00
gl_class->get_default_framebuffer = gdk_macos_gl_context_get_default_framebuffer;
gdk: Make sure only one GL backend is used Creative people managed to create an X11 display and a Wayland display at once, thereby getting EGL and GLX involved in a fight to the death over the ownership of the glFoo() symbolspace. A way to force such a fight with available tools here is (on Wayland) running something like: GTK_INSPECTOR_DISPLAY=:1 GTK_DEBUG=interactive gtk4-demo Related: xdg-desktop-portal-gnome#5
2021-09-23 23:47:03 +00:00
gl_class->backend_type = GDK_GL_CGL;
macos: prototype new GDK backend for macOS This is fairly substantial rewrite of the GDK backend for quartz and renamed to macOS to allow for a greenfield implementation. Many things have come across from the quartz implementation fairly intact such as the eventloop integration design and discovery of event windows from the NSEvent. However much has been changed to fit in with the new GDK design and how removal of child GdkWindow have been completely eliminated. Furthermore, the new GdkPopup allows for regular NSWindow to be used to provide popovers unlike the previous implementation. The object design more closely follows the ideal for a GDK backend. Views have been broken out into subclasses so that we can support multiple GSK renderer paths such as GL and Cairo (and Metal in the future). However mixed mode GL and Cairo will not be supported. Currently only the Cairo renderer has been implemented. A new frame clock implementation using CVDisplayLink provides more accurate information about when to draw drawing the next frame. Some testing will need to be done here to understand the power implications of this. This implementation has also gained edge snapping for CSD windows. Some work was also done to ensure that CSD windows have opaque regions registered with the display server. ** This is still very much a work-in-progress ** Some outstanding work that needs to be done: - Finish a GL context for macOS and alternate NSView for GL rendering (possibly using speciailized CALayer for OpenGL). - Input rework to ensure that we don't loose remapping of keys that was dropped from GDK during GTK 4 development. - Make sure input methods continue to work. - Drag-n-Drop is still very much a work in progress - High resolution input scrolling needs various work in GDK to land first before we can plumb that to NSEvent. - gtk/ has a number of things based on GDK_WINDOWING_QUARTZ that need to be updated to use the macOS backend. But this is good enough to start playing with and breaking things which is what I'd like to see.
2020-04-23 23:36:46 +00:00
}
static void
gdk_macos_gl_context_init (GdkMacosGLContext *self)
{
macos: use CALayer and IOSurface for rendering This provides a major shift in how we draw both when accelerated OpenGL as well as software rendering with Cairo. In short, it uses tiles of Core Animation's CALayer to display contents from an OpenGL or Cairo rendering so that the window can provide partial damage updates. Partial damage is not generally available when using OpenGL as the whole buffer is flipped even if you only submitted a small change using a scissor rect. Thankfully, this speeds up Cairo rendering a bit too by using IOSurface to upload contents to the display server. We use the tiling system we do for OpenGL which reduces overall complexity and differences between them. A New Buffer ============ GdkMacosBuffer is a wrapper around an IOSurfaceRef. The term buffer was used because 1) surface is already used and 2) it loosely maps to a front/back buffer semantic. However, it appears that IOSurfaceRef contents are being retained in some fashion (likely in the compositor result) so we can update the same IOSurfaceRef without flipping as long as we're fast. This appears to be what Chromium does as well, but Firefox uses two IOSurfaceRef and flips between them. We would like to avoid two surfaces because it doubles the GPU VRAM requirements of the application. Changes to Windows ================== Previously, the NSWindow would dynamically change between different types of NSView based on the renderer being used. This is no longer necessary as we just have a single NSView type, GdkMacosView, which inherits from GdkMacosBaseView just to keep the tedius stuff separate from the machinery of GdkMacosView. We can merge those someday if we are okay with that. Changes to Views ================ GdkMacosCairoView, GdkMacosCairoSubView, GdkMacosGLView have all been removed and replaced with GdkMacosView. This new view has a single CALayer (GdkMacosLayer) attached to it which itself has sublayers. The contents of the CALayer is populated with an IOSurfaceRef which we allocated with the GdkMacosSurface. The surface is replaced when the NSWindow resizes. Changes to Layers ================= We now have a dedicated GdkMacosLayer which contains sublayers of GdkMacosTile. The tile has a maximum size of 128x128 pixels in device units. The GdkMacosTile is partitioned by splitting both the transparent region (window bounds minus opaque area) and then by splitting the opaque area. A tile has either translucent contents (and therefore is not opaque) or has opaque contents (and therefore is opaque). An opaque tile never contains transparent contents. As such, the opaque tiles contain a black background so that Core Animation will consider the tile's bounds as opaque. This can be verified with "Quartz Debug -> Show opaque regions". Changes to Cairo ================ GTK 4 cannot currently use cairo-quartz because of how CSS borders are rendered. It simply causes errors in the cairo_quartz_surface_t backend. Since we are restricted to using cairo_image_surface_t (which happens to be faster anyway) we can use the IOSurfaceBaseAddress() to obtain a mapping of the IOSurfaceRef in user-space. It always uses BGRA 32-bit with alpha channel even if we will discard the alpha channel as that is necessary to hit the fast paths in other parts of the platform. Note that while Cairo says CAIRO_FORMAT_ARGB32, it is really 32-bit BGRA on little-endian as we expect. OpenGL will render flipped (Quartz Native Co-ordinates) while Cairo renders with 0,O in the top-left. We could use cairo_translate() and cairo_scale() to reverse this, but it looks like some cairo things may not look quite as right if we do so. To reduce the chances of one-off bugs this continues to draw as Cairo would normally, but instead uses an CGAffineTransform in the tiles and some CGRect translation when swapping buffers to get the same effect. Changes to OpenGL ================= To simplify things, removal of all NSOpenGL* related components have been removed and we strictly use the Core GL (CGL*) API. This probably should have been done long ago anyay. Most examples found in the browsers to use IOSurfaceRef with OpenGL are using Legacy GL and there is still work underway to make this fit in with the rest of how the GSK GL renderer works. Since IOSurfaceRef bound to a texture/framebuffer will not have a default framebuffer ID of 0, we needed to add a default framebuffer id to the GdkGLContext. GskGLRenderer can use this to setup the command queue in such a way that our IOSurface destination has been glBindFramebuffer() as if it were the default drawable. This stuff is pretty slight-of-hand, so where things are and what needs flushing when and where has been a bit of an experiment to see what actually works to get synchronization across subsystems. Efficient Damages ================= After we draw with Cairo, we unlock the IOSurfaceRef and the contents are uploaded to the GPU. To make the contents visible to the app, we must clear the tiles contents with `layer.contents=nil;` and then re-apply the IOSurfaceRef. Since the buffer has likely not changed, we only do this if the tile overlaps the damage region. This gives the effect of having more tightly controlled damage regions even though updating the layer would damage be the whole window (as it is with OpenGL/Metal today with the exception of scissor-rect). This too can be verified usign "Quartz Debug -> Flash screen udpates". Frame Synchronized Resize ========================= In GTK 4, we have the ability to perform sizing changes from compute-size during the layout phase. Since the macOS backend already tracks window resizes manually, we can avoid doing the setFrame: immediately and instead do it within the frame clock's layout phase. Doing so gives us vastly better resize experience as we're more likely to get the size-change and updated-contents in the same frame on screen. It makes things feel "connected" in a way they weren't before. Some additional effort to tweak gravity during the process is also necessary but we were already doing that in the GTK 4 backend. Backporting =========== The design here has made an attempt to make it possible to backport by keeping GdkMacosBuffer, GdkMacosLayer, and GdkMacosTile fairly independent. There may be an opportunity to integrate this into GTK 3's quartz backend with a fair bit of work. Doing so could improve the situation for applications which are damage-rich such as The GIMP.
2022-02-14 10:20:19 +00:00
self->target = GL_TEXTURE_RECTANGLE;
macos: prototype new GDK backend for macOS This is fairly substantial rewrite of the GDK backend for quartz and renamed to macOS to allow for a greenfield implementation. Many things have come across from the quartz implementation fairly intact such as the eventloop integration design and discovery of event windows from the NSEvent. However much has been changed to fit in with the new GDK design and how removal of child GdkWindow have been completely eliminated. Furthermore, the new GdkPopup allows for regular NSWindow to be used to provide popovers unlike the previous implementation. The object design more closely follows the ideal for a GDK backend. Views have been broken out into subclasses so that we can support multiple GSK renderer paths such as GL and Cairo (and Metal in the future). However mixed mode GL and Cairo will not be supported. Currently only the Cairo renderer has been implemented. A new frame clock implementation using CVDisplayLink provides more accurate information about when to draw drawing the next frame. Some testing will need to be done here to understand the power implications of this. This implementation has also gained edge snapping for CSD windows. Some work was also done to ensure that CSD windows have opaque regions registered with the display server. ** This is still very much a work-in-progress ** Some outstanding work that needs to be done: - Finish a GL context for macOS and alternate NSView for GL rendering (possibly using speciailized CALayer for OpenGL). - Input rework to ensure that we don't loose remapping of keys that was dropped from GDK during GTK 4 development. - Make sure input methods continue to work. - Drag-n-Drop is still very much a work in progress - High resolution input scrolling needs various work in GDK to land first before we can plumb that to NSEvent. - gtk/ has a number of things based on GDK_WINDOWING_QUARTZ that need to be updated to use the macOS backend. But this is good enough to start playing with and breaking things which is what I'd like to see.
2020-04-23 23:36:46 +00:00
}
G_GNUC_END_IGNORE_DEPRECATIONS
Reference in New Issue Copy Permalink