Instead of basing the worksWhenModal property on the window type, which
will include both windows inside the current modal session (as intend),
but also windows below the current modal session (wrongly), we check
to see if the window is a transient child of the current top level
modal window.
Going via NSApp.modalWindow means we also catch cases where the top
level modal session is run by a native window, not part of the modal
session stack in the Cocoa event dispatcher.
The new logic relies on windows such as popups, dialogs, etc to set
the correct transient parent, but this seems to be the case already.
To ensure the window tag is also updated, we call setWorksWhenModal
on modal session changes. We could change worksWhenModal into e.g.
shouldWorkWhenModal and always use the setter, but that would mean
the initial window tag update in [NSWindow _commonAwake] would pick
up the incorrect value. And if the window tag is not updated after
that due to the workaround in [QNSPanel setWorksWhenModal:] being
compiled out, the window would not be possible to order front,
which is worse than being able to order front a window with
worksWhenModal=NO.
Fixes: QTBUG-76654
Task-number: QTBUG-71480
Change-Id: I38b14422d274dcc03b4c7d5ef87066e282ed9111
Reviewed-by: Timur Pocheptsov <timur.pocheptsov@qt.io>
Reviewed-by: Tor Arne Vestbø <tor.arne.vestbo@qt.io>
f4889e63c7