Fix scheduling of the frame clock when we don't receive "frame drawn"
messages from the compositor.
If we received "frame drawn" events recently, then the "smooth frame
time" would be in sync with the vsync time. When we don't receive frame
drawn events, the "smooth frame time" is simply incremented by constant
multiples of the refresh interval. In both cases we can use this smooth
time as the basis for scheduling the next clock cycle.
By only using the "smooth frame time" as a basis we also benefit from
more consistent scheduling cadence. If, for example, we got "frame
drawn" events, then didn't receive them for a few frames, we would still
be in sync when we start receiving these events again.
When an animation is started while the application is idle, that often
happens as a result of some external event. This can be an input event,
an expired timer, data arriving over the network etc. The result is that
the first animation clock cycle could be scheduled at some random time,
as opposed to follow up cycles which are usually scheduled right after a
vsync.
Since the frame time we report to the application is correlated to the
time when the frame clock was scheduled to run, this can result in
uneven times reported in the first few animation frames. In order to fix
that, we measure the phase of the first clock cycle - i.e. the offset
between the first cycle and the preceding vsync. Once we start receiving
"frame drawn" signals, the cadence of the frame clock scheduling becomes
tied to the vsync. In order to maintain the regularity of the reported
frame times, we adjust subsequent reported frame times with the
aforementioned phase.
When the application does not receive "frame drawn" signals we schedule
the clock to run more or less at intervals equal to the last known
refresh interval. In order to minimize clock skew we have to aim for
exact intervals.
(cherry picked from commit f5de46670b)
We try to step the frame clock in whole refresh_interval steps, but to
avoid drift and rounding issues we additionally try to converge it to
be synced to the physical vblank (actually the time we get the
frame-drawn message from the compositor, but these are tied together).
However, the convergence to vsync only really makes sense if the new
frame_time actually is tied to the vsync. It may very well be that
some other kind of event (say a network or mouse event) triggered
the redraw, and not a vsync presentation.
We used to assume that all frames that are close in time (< 4 frames
apart) were regular and thus tied to the vsync, but there is really no
guarantee of that. Even non regular times could be rapid.
This commit changes the code to only do the convergence-to-real-time
if the cause of the clock cycle was a thaw (i.e. last frame drawn and
animating). Paint cycles for any other kind of reason are always
scheduled an integer number of frames after the last cycle that was
caused by a thaw.
(cherry picked from commit 91af8a705b)
When we get to a paint cycle we now know if this was caused by a
thaw, which typically means last frame was drawn, or some other event.
In the first case the time of the cycle is tied to the vblank in some
sense, and in the others it is essentially random. We can use this
information to compute better frame times. (Will be done in later
commits.)
(cherry picked from commit 82c314f1af)
When we run the frameclock RUN_FLUSH_IDLE idle before the paint,
then gdk_frame_clock_flush_idle() sets
```
priv->phase = GDK_FRAME_CLOCK_PHASE_BEFORE_PAINT
```
at the end if there is a paint comming.
But, before doing the paint cycle it may handle other X events, and
during that time the phase is set to BEFORE_PAINT. This means that the
current check on whether we're inside a paint is wrong:
```
if (priv->phase != GDK_FRAME_CLOCK_PHASE_NONE &&
priv->phase != GDK_FRAME_CLOCK_PHASE_FLUSH_EVENTS)
return priv->smoothed_frame_time_base;
```
This caused us to sometimes use this smoothed_frame_time_base even
though we previously reported a later value during PHASE_NONE, thus
being non-monotonic.
We can't just additionally check for the BEGIN_PAINT phase though,
becasue if we are in the paint loop actually doing that phase we
should use the time base. Instead we check for `!(BEFORE_PAINT &&
in_paint_idle)`.
(cherry picked from commit a36e2bc764)
A call to frame gdk_frame_clock_get_frame_time() outside of the paint
cycle could report an un-error-corrected frame time, and later a
corrected value could be earlier than the previously reported value.
We now always store the latest reported time so we can ensure
monotonicity.
(cherry picked from commit a27fed47e0)
In commit c6901a8b, the frame clock reported time was changed from
simply reporting the time we ran the frame clock cycle to reporting a
smoothed value that increased by the frame interval each time it was
called.
However, this change caused some problems, such as:
https://gitlab.gnome.org/GNOME/gtk/-/merge_requests/1415https://gitlab.gnome.org/GNOME/gtk/-/merge_requests/1416https://gitlab.gnome.org/GNOME/gtk/-/merge_requests/1482
I think a lot of this is caused by the fact that we just overwrote the
old frame time with the smoothed, monotonous timestamp, breaking
some things that relied on knowing the actual time something happened.
This is a new approach to doing the smoothing that is more explicit.
The "frame_time" we store is the actual time we ran the update cycle,
and then we separately compute and store the derived smoothed time and
its period, allowing us to easily return a smoothed time at any time
by rounding the time difference to an integer number of frames.
The initial frame_time can be somewhat arbitrary, as it depends on the
first cycle which is not driven by the frame clock. But follow-up
cycles are typically tied to the the compositor sending the drawn
signal. It may happen that the initial frame is exactly in the middle
between two frames where jitter causes us to randomly round in
different directions when rounding to nearest frame. To fix this we
additionally do a quadratic convergence towards the "real" time,
during presentation driven clock cycles (i.e. when the frame times are
small).
(cherry picked from commit 9ef3e70040)
This is a backport of the GdkProfiler from master. It does not include
the pixel bandwidth numbers that come from gdkdrawcontext.c since there
does not seem to be an analog in 3.x.
Additionally, this implements the recent changes for SYsprof's D-Bus
profiler API which adds a Capabilities property and an options hash-table
to the D-Bus interface for forward portability.
Since commit 3b2f9395, the frame time may be set into the future, so
only ensure monotonicity, and don't store the offset. This prevents the
frame time from becoming out of sync with g_get_monotonic_time().
Fixes#1612
This fixes stuttering in animations that rely on the regularity of
gdk_frame_clock_get_frame_time.
https://bugzilla.gnome.org/show_bug.cgi?id=787665
BEFORE
gdkgears:
58 FPS and visibly stuttering
gnome-maps on a 59.95Hz monitor:
"paint" g_get_monotonic_time +17278μs, gdk_frame_clock_get_frame_time +17278μs
"paint" g_get_monotonic_time +17449μs, gdk_frame_clock_get_frame_time +17426μs
"paint" g_get_monotonic_time +17620μs, gdk_frame_clock_get_frame_time +17600μs
AFTER
gdkgears:
60 FPS and smoother
gnome-maps on a 59.95Hz monitor:
"paint" g_get_monotonic_time +18228μs, gdk_frame_clock_get_frame_time +16680μs
"paint" g_get_monotonic_time +15010μs, gdk_frame_clock_get_frame_time +16680μs
"paint" g_get_monotonic_time +17134μs, gdk_frame_clock_get_frame_time +16680μs
These were showing up higher in Sysprof profiles.
The simple fix is to avoid the emit_by_name() and let the interface emit
the signals directly. No function preconditions are provided since these
are internal API.
In the case where the layout phase queued a layout we don't
want to progress to the paint phase with invalid allocations, so
we loop the layout. This shouldn't normally happen, but it may
happen in some edge cases like if user/wm resizes clash with
natural window size changes from a gtk widget. This should not
generally loop though, so we detect this after 4 cycles and
print a warning.
This was detected because of an issue in GtkWindow where it
seems to incorrectly handle the case of a user interactive resize.
It seems gtk_window_move_resize() believes that configure_request_size_changed
changed due to hitting some corner case so it calls
gtk_widget_queue_resize_no_redraw(), marking the window as need_alloc
after the layout phase. This commit fixes the issue, but we should
also look into if we can fix that.
If no updates, redraws, or repaints have been scheduled for this frame,
we will skip immediately to RESUME_EVENTS, and no GdkFrameTimings will
be created.
https://bugzilla.gnome.org/show_bug.cgi?id=694732
Commit 1db87c897f accidentally removed
a check for !in_paint_idle in maybe_start_idle which causes us
to create a paint loop whenever something requests a phase
inside the paint_idle.
The default windows timer resolution is 16msec, which is too little
for fluent animations (say at 60Hz). So, while a paint clock is
active we temporarily raise the timer resolution to 1 msec.
Add an API to start or stop continually updating the frame clock.
This is a slight convenience for applcations and avoids the problem
of getting one more frame run after an animation stops, but the
primary motivation for this is because it looks like we might have
to use timeBeginPeriod()/timeEndPeriod() on Windows to get reasonably
accurate timing, and for that we'll need to know if there is an
animation running.
https://bugzilla.gnome.org/show_bug.cgi?id=693934
* remove gdk_frame_clock_get_frame_time_val(); a convenience
function that would rarely be used.
* remove gdk_frame_clock_get_requested() and
::frame-requested signal; while we might want to eventually
be able to track the requested phases for a clock, we don't
have a current use case.
* Make gdk_frame_clock_freeze/thaw() private: they are only
used within GTK+ and have complex semantics.
* Remove gdk_frame_clock_get_last_complete(). Another convenience
function that I don't have a current use case for.
* Rename:
gdk_frame_clock_get_start() => gdk_frame_clock_get_history_start()
gdk_frame_clocK_get_current_frame_timings() => gdk_frame_clock_get_timings()
Since we're not exporting the ability to create your own frame
clock for now, remove the setters for GdkFrameTimings fields.
Also remove all setters and getters for fields that are more
about implementation than about quantities that are meaningful
to the applcation and just access the fields directly within
GDK.
Now that GdkFrameClock is a class, not interface, there's no real advantage
to splitting the frame history into an aggregate object, so directly
merge it into GdkFrameClock.
It's unlikely that anyone will want to have, say, a GtkWidget that
also acts as a GdkFrameClock, so an abstract base class is as
flexible as making GdkFrameClock an interface, but has advantages:
- If we decide to never make implementing your own frame clock
possible, we can remove the virtualization.
- We can put functionality like history into the base class.
- Avoids the oddity of a interface without a public interface
VTable, which may cause problems for language bindings.
For an operation like synchronizing audio to video playback, we need to
be able to predict the time that a frame will be presented. The details
of this depend on the windowing system, so make the backend predict
a presentation time for ::begin-frame and set it on the GdkFrameTimings.
The timing algorithm of GdkFrameClockIdle is adjusted to give predictable
presentation times for frames that are not throttled by the windowing
system.
Helper functions:
gdk_frame_clock_get_current_frame_timings()
gdk_frame_clock_get_refresh_info()
are added for operations that would otherwise be needed multiple times
in different locations.
https://bugzilla.gnome.org/show_bug.cgi?id=685460
Add functions that tell us whether the main loop slept before we drew
a frame. Blocking with the frame clock frozen doesn't count as sleeping.
We'll use this to advertise to the compositor whether we
are drawing as fast as possible (and it should do the same) or timing
frames carefully (and it should do the same.)
https://bugzilla.gnome.org/show_bug.cgi?id=685460
Don't start the idle if we're in the middle of painting a frame -
this will prevent us from getting the timing right when starting
the idle after the frame.
https://bugzilla.gnome.org/show_bug.cgi?id=685460
In order to be able to track statistics about how well we are drawing,
and in order to be able to do sophisticated things with frame timing
like predicting per-frame latencies and synchronizing audio with video,
we need to be able to track exactly when previous frames were drawn
to the screen.
Information about each frame is stored in a new GdkFrameTimings object.
A new GdkFrameHistory object is added which keeps a queue of recent
GdkFrameTimings (this is added to avoid further complicating the
implementation of GdkFrameClock.)
https://bugzilla.gnome.org/show_bug.cgi?id=685460
Keeping events paused after the end of a frame put us in a
weird state where we had to process and queue events - so that
we would get the message from the compositor - but not deliver
them. Instead resume events before ending the frame.
https://bugzilla.gnome.org/show_bug.cgi?id=685460
When we have pending motion events, instead of delivering them
directly, request the new FLUSH_EVENTS phase of the frame clock.
This allows us to compress repeated motion events sent to the
same window.
In the FLUSH_EVENTS phase, which occur at priority GDK_PRIORITY_EVENTS + 1,
we deliver any pending motion events then turn off event delivery
until the end of the next frame. Turning off event delivery means
that we'll reliably paint the compressed motion events even if more
have arrived.
Add a motion-compression test case which demonstrates behavior when
an application takes too long handle motion events. It is unusable
without this patch but behaves fine with the patch.
https://bugzilla.gnome.org/show_bug.cgi?id=685460
Switch GtkStyleContext to using GdkFrameClock. To do this, add a new
UPDATE phase to GdkFrameClock.
Add a GdkFrameClockTarget interface with a single set_clock() method,
and use this to deal with the fact that GtkWidget only has a frame
clock when realized.
https://bugzilla.gnome.org/show_bug.cgi?id=685460
If the backend is throttling paints, then the frame clock will be
frozen at the end of the frame. If not, then we need to add throttling,
so wait until 16ms after the start of the frame before beginning the
next frame.
https://bugzilla.gnome.org/show_bug.cgi?id=685460
Add the ability to freeze a frame clock, which pauses its operation,
then thaw it again later to resume.
Initially this is used to implement freezing updates when we are
waiting for ConfigureNotify in response to changing the size of
a toplevel.
We need a per-window clock for this to work properly, so add that
for the X11 backend.
https://bugzilla.gnome.org/show_bug.cgi?id=685460
Instead of having gdk_frame_clock_request_frame() have
gdk_frame_clock_request_phase() where we can say what phase we need.
This allows us to know if we get a frame-request during layout whether
it's just a request for drawing from the layout, or whether another
layout phase is needed.
https://bugzilla.gnome.org/show_bug.cgi?id=685460
