Fix scheduling of the frame clock when we don't receive "frame drawn"
messages from the compositor.
If we received "frame drawn" events recently, then the "smooth frame
time" would be in sync with the vsync time. When we don't receive frame
drawn events, the "smooth frame time" is simply incremented by constant
multiples of the refresh interval. In both cases we can use this smooth
time as the basis for scheduling the next clock cycle.
By only using the "smooth frame time" as a basis we also benefit from
more consistent scheduling cadence. If, for example, we got "frame
drawn" events, then didn't receive them for a few frames, we would still
be in sync when we start receiving these events again.
When an animation is started while the application is idle, that often
happens as a result of some external event. This can be an input event,
an expired timer, data arriving over the network etc. The result is that
the first animation clock cycle could be scheduled at some random time,
as opposed to follow up cycles which are usually scheduled right after a
vsync.
Since the frame time we report to the application is correlated to the
time when the frame clock was scheduled to run, this can result in
uneven times reported in the first few animation frames. In order to fix
that, we measure the phase of the first clock cycle - i.e. the offset
between the first cycle and the preceding vsync. Once we start receiving
"frame drawn" signals, the cadence of the frame clock scheduling becomes
tied to the vsync. In order to maintain the regularity of the reported
frame times, we adjust subsequent reported frame times with the
aforementioned phase.
When the application does not receive "frame drawn" signals we schedule
the clock to run more or less at intervals equal to the last known
refresh interval. In order to minimize clock skew we have to aim for
exact intervals.
We try to step the frame clock in whole refresh_interval steps, but to
avoid drift and rounding issues we additionally try to converge it to
be synced to the physical vblank (actually the time we get the
frame-drawn message from the compositor, but these are tied together).
However, the convergence to vsync only really makes sense if the new
frame_time actually is tied to the vsync. It may very well be that
some other kind of event (say a network or mouse event) triggered
the redraw, and not a vsync presentation.
We used to assume that all frames that are close in time (< 4 frames
apart) were regular and thus tied to the vsync, but there is really no
guarantee of that. Even non regular times could be rapid.
This commit changes the code to only do the convergence-to-real-time
if the cause of the clock cycle was a thaw (i.e. last frame drawn and
animating). Paint cycles for any other kind of reason are always
scheduled an integer number of frames after the last cycle that was
caused by a thaw.
When we get to a paint cycle we now know if this was caused by a
thaw, which typically means last frame was drawn, or some other event.
In the first case the time of the cycle is tied to the vblank in some
sense, and in the others it is essentially random. We can use this
information to compute better frame times. (Will be done in later
commits.)
When we run the frameclock RUN_FLUSH_IDLE idle before the paint,
then gdk_frame_clock_flush_idle() sets
```
priv->phase = GDK_FRAME_CLOCK_PHASE_BEFORE_PAINT
```
at the end if there is a paint comming.
But, before doing the paint cycle it may handle other X events, and
during that time the phase is set to BEFORE_PAINT. This means that the
current check on whether we're inside a paint is wrong:
```
if (priv->phase != GDK_FRAME_CLOCK_PHASE_NONE &&
priv->phase != GDK_FRAME_CLOCK_PHASE_FLUSH_EVENTS)
return priv->smoothed_frame_time_base;
```
This caused us to sometimes use this smoothed_frame_time_base even
though we previously reported a later value during PHASE_NONE, thus
being non-monotonic.
We can't just additionally check for the BEGIN_PAINT phase though,
becasue if we are in the paint loop actually doing that phase we
should use the time base. Instead we check for `!(BEFORE_PAINT &&
in_paint_idle)`.
A call to frame gdk_frame_clock_get_frame_time() outside of the paint
cycle could report an un-error-corrected frame time, and later a
corrected value could be earlier than the previously reported value.
We now always store the latest reported time so we can ensure
monotonicity.
In commit c6901a8b, the frame clock reported time was changed from
simply reporting the time we ran the frame clock cycle to reporting a
smoothed value that increased by the frame interval each time it was
called.
However, this change caused some problems, such as:
https://gitlab.gnome.org/GNOME/gtk/-/merge_requests/1415https://gitlab.gnome.org/GNOME/gtk/-/merge_requests/1416https://gitlab.gnome.org/GNOME/gtk/-/merge_requests/1482
I think a lot of this is caused by the fact that we just overwrote the
old frame time with the smoothed, monotonous timestamp, breaking
some things that relied on knowing the actual time something happened.
This is a new approach to doing the smoothing that is more explicit.
The "frame_time" we store is the actual time we ran the update cycle,
and then we separately compute and store the derived smoothed time and
its period, allowing us to easily return a smoothed time at any time
by rounding the time difference to an integer number of frames.
The initial frame_time can be somewhat arbitrary, as it depends on the
first cycle which is not driven by the frame clock. But follow-up
cycles are typically tied to the the compositor sending the drawn
signal. It may happen that the initial frame is exactly in the middle
between two frames where jitter causes us to randomly round in
different directions when rounding to nearest frame. To fix this we
additionally do a quadratic convergence towards the "real" time,
during presentation driven clock cycles (i.e. when the frame times are
small).
This drops the marks for before/after-paint as they are internal
things that very rarely use any time, and also flush/resume-events
as any events reported here will get separate marks so will be easy
to see anyway.
Also, we rename the entire frameclock cycle to "frameclock cycle"
rather than "paint_idle" which is rather cryptic.
These don't take a duration, instead they call g_get_monotonic_time() to
and subtract the start time for it.
Almost all our calls are like this, and this makes the callsites clearer
and avoids inlining the clock call into the call site.
When we use if (GDK_PROFILER_IS_RUNNING) this means we get an
inlined if (FALSE) when the compiler support is not compiled in, which
gets rid of all the related code completely.
We also expand to G_UNLIKELY(gdk_profiler_is_running ()) in the supported
case which might cause somewhat better code generation.
usec is the scale of the monotonic timer which is where we get almost
all the times from. The only actual source of nsec is the opengl
GPU time (but who knows what the actual resulution of that is).
Changing this to usec allows us to get rid of " * 1000" in a *lot* of
places all over the codebase, which are ugly and confusing.
Instead of reporting the frame clock phases as defined,
report the duration of the signal emissions, which is more
useful for tracking down what is taking time.
We were adding incomplete frame timings to the
profile, which lead to occasional nonsense
numbers. Instead, only add timings to the profile
once we marked them as complete. This also
gives us an opportunity to add the presentation
time as a marker.
Since commit 3b2f9395, the frame time may be set into the future, so
only ensure monotonicity, and don't store the offset. This prevents the
frame time from becoming out of sync with g_get_monotonic_time().
Fixes#1612
The main GDK thread lock is not portable and deprecated.
The only reason why gdk_threads_add_timeout() and
gdk_threads_add_timeout_full() exist is to allow invoking a callback
with the GDK lock held, in case 3rd party libraries still use the
deprecated gdk_threads_enter()/gdk_threads_leave() API.
Since we're removing the GDK lock, and we're releasing a new major API,
such code cannot exist any more; this means we can use the GLib API for
installing timeout callbacks.
https://bugzilla.gnome.org/show_bug.cgi?id=793124
This fixes stuttering in animations that rely on the regularity of
gdk_frame_clock_get_frame_time.
https://bugzilla.gnome.org/show_bug.cgi?id=787665
BEFORE
gdkgears:
58 FPS and visibly stuttering
gnome-maps on a 59.95Hz monitor:
"paint" g_get_monotonic_time +17278μs, gdk_frame_clock_get_frame_time +17278μs
"paint" g_get_monotonic_time +17449μs, gdk_frame_clock_get_frame_time +17426μs
"paint" g_get_monotonic_time +17620μs, gdk_frame_clock_get_frame_time +17600μs
AFTER
gdkgears:
60 FPS and smoother
gnome-maps on a 59.95Hz monitor:
"paint" g_get_monotonic_time +18228μs, gdk_frame_clock_get_frame_time +16680μs
"paint" g_get_monotonic_time +15010μs, gdk_frame_clock_get_frame_time +16680μs
"paint" g_get_monotonic_time +17134μs, gdk_frame_clock_get_frame_time +16680μs
This patch makes that work using 1 of 2 options:
1. Add all missing enums to the switch statement
or
2. Cast the switch argument to a uint to avoid having to do that (mostly
for GdkEventType).
I even found a bug while doing that: clearing a GtkImage with a surface
did not notify thae surface property.
The reason for enabling this flag even though it is tedious at times is
that it is very useful when adding values to an enum, because it makes
GTK immediately warn about all the switch statements where this enum is
relevant.
And I expect changes to enums to be frequent during the GTK4 development
cycle.
These were showing up higher in Sysprof profiles.
The simple fix is to avoid the emit_by_name() and let the interface emit
the signals directly. No function preconditions are provided since these
are internal API.
In the case where the layout phase queued a layout we don't
want to progress to the paint phase with invalid allocations, so
we loop the layout. This shouldn't normally happen, but it may
happen in some edge cases like if user/wm resizes clash with
natural window size changes from a gtk widget. This should not
generally loop though, so we detect this after 4 cycles and
print a warning.
This was detected because of an issue in GtkWindow where it
seems to incorrectly handle the case of a user interactive resize.
It seems gtk_window_move_resize() believes that configure_request_size_changed
changed due to hitting some corner case so it calls
gtk_widget_queue_resize_no_redraw(), marking the window as need_alloc
after the layout phase. This commit fixes the issue, but we should
also look into if we can fix that.
If no updates, redraws, or repaints have been scheduled for this frame,
we will skip immediately to RESUME_EVENTS, and no GdkFrameTimings will
be created.
https://bugzilla.gnome.org/show_bug.cgi?id=694732
Commit 1db87c897f accidentally removed
a check for !in_paint_idle in maybe_start_idle which causes us
to create a paint loop whenever something requests a phase
inside the paint_idle.
The default windows timer resolution is 16msec, which is too little
for fluent animations (say at 60Hz). So, while a paint clock is
active we temporarily raise the timer resolution to 1 msec.
Add an API to start or stop continually updating the frame clock.
This is a slight convenience for applcations and avoids the problem
of getting one more frame run after an animation stops, but the
primary motivation for this is because it looks like we might have
to use timeBeginPeriod()/timeEndPeriod() on Windows to get reasonably
accurate timing, and for that we'll need to know if there is an
animation running.
https://bugzilla.gnome.org/show_bug.cgi?id=693934
* remove gdk_frame_clock_get_frame_time_val(); a convenience
function that would rarely be used.
* remove gdk_frame_clock_get_requested() and
::frame-requested signal; while we might want to eventually
be able to track the requested phases for a clock, we don't
have a current use case.
* Make gdk_frame_clock_freeze/thaw() private: they are only
used within GTK+ and have complex semantics.
* Remove gdk_frame_clock_get_last_complete(). Another convenience
function that I don't have a current use case for.
* Rename:
gdk_frame_clock_get_start() => gdk_frame_clock_get_history_start()
gdk_frame_clocK_get_current_frame_timings() => gdk_frame_clock_get_timings()
Since we're not exporting the ability to create your own frame
clock for now, remove the setters for GdkFrameTimings fields.
Also remove all setters and getters for fields that are more
about implementation than about quantities that are meaningful
to the applcation and just access the fields directly within
GDK.
Now that GdkFrameClock is a class, not interface, there's no real advantage
to splitting the frame history into an aggregate object, so directly
merge it into GdkFrameClock.
It's unlikely that anyone will want to have, say, a GtkWidget that
also acts as a GdkFrameClock, so an abstract base class is as
flexible as making GdkFrameClock an interface, but has advantages:
- If we decide to never make implementing your own frame clock
possible, we can remove the virtualization.
- We can put functionality like history into the base class.
- Avoids the oddity of a interface without a public interface
VTable, which may cause problems for language bindings.
For an operation like synchronizing audio to video playback, we need to
be able to predict the time that a frame will be presented. The details
of this depend on the windowing system, so make the backend predict
a presentation time for ::begin-frame and set it on the GdkFrameTimings.
The timing algorithm of GdkFrameClockIdle is adjusted to give predictable
presentation times for frames that are not throttled by the windowing
system.
Helper functions:
gdk_frame_clock_get_current_frame_timings()
gdk_frame_clock_get_refresh_info()
are added for operations that would otherwise be needed multiple times
in different locations.
https://bugzilla.gnome.org/show_bug.cgi?id=685460
Add functions that tell us whether the main loop slept before we drew
a frame. Blocking with the frame clock frozen doesn't count as sleeping.
We'll use this to advertise to the compositor whether we
are drawing as fast as possible (and it should do the same) or timing
frames carefully (and it should do the same.)
https://bugzilla.gnome.org/show_bug.cgi?id=685460
