AuroraMiddleware/gtk2
1
0
Fork 0
forked from AuroraMiddleware/gtk
Code Issues Pull Requests Projects Releases Wiki Activity
b23f793753
gtk2/gtk/gtksortlistmodel.c

1062 lines
29 KiB
C
Raw Normal View History

GtkSortListModel: Add
2018-09-14 02:34:40 +00:00
/*
* Copyright © 2018 Benjamin Otte
*
* This library is free software; you can redistribute it and/or
* modify it under the terms of the GNU Lesser General Public
* License as published by the Free Software Foundation; either
* version 2.1 of the License, or (at your option) any later version.
*
* This library is distributed in the hope that it will be useful,
* but WITHOUT ANY WARRANTY; without even the implied warranty of
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
* Lesser General Public License for more details.
*
* You should have received a copy of the GNU Lesser General Public
* License along with this library. If not, see <http://www.gnu.org/licenses/>.
*
* Authors: Benjamin Otte <otte@gnome.org>
*/
#include "config.h"
#include "gtksortlistmodel.h"
sortlistmodel: Make key generation part of the step function SSave the missing keys as a bitset and iterate over that bitset in the step function. Solves the problem with a large UI block at the beginning of a sort operation when all the keys were generated, in particular when key generation was slow. Benchmarks for maximum time taken by a single main loop callback: initial sort with complex GFileInfo keys old new 32,000 items 137ms 3ms 128,000 items 520ms 31ms initial sort with string keys old new 32,000 items 187ms 1ms 128,000 items 804ms 3ms
2020-07-21 23:43:59 +00:00
#include "gtkbitset.h"
GtkSortListModel: Add
2018-09-14 02:34:40 +00:00
#include "gtkintl.h"
#include "gtkprivate.h"
sortlistmodel: Use GtkSortKeys This massively speeds up sorting with expensive sort functions that it's the most worthwhile optimization of this whole branch. It's slower for simple sort functions though. It's also quite a lot slower when the model doesn't support sort keys (like GtkCustomSorter), but all the other sorters do support keys. Of course, this depends on the number of items in the model - the number of comparisons scales O(N * log N) while the overhead for key handling scales O(N). So as the log N part grows, generating keys gets more and more beneficial. Benchmarks: initial sort of a GFileInfo model with display-name keys items keys 8,000 items 715ms 50ms 64,000 items --- 554ms initial sort of a GFileInfo model with complex keys items keys 64,000 items 340ms 295ms 128,000 items 641ms 605ms removing half a GFileInfo model with display-name keys (no comparisons, just key freeing overhead of a complex sorter) items keys 512,000 items 14ms 21ms 2,048,000 items 40ms 62ms removing half a GFileInfo model with complex keys (no comparisons, just key freeing overhead of a complex sorter) items keys 512,000 items 90ms 237ms 2,048,000 items 247ms 601ms
2020-07-21 01:40:28 +00:00
#include "gtksorterprivate.h"
sortlistmodel: Use timsort Simply replace the old qsort() call with a timsort() call. This is ultimately relevant because timsort is a LOT faster in merging to already sorted lists (think items-chaged adding some items) or reversing an existing list (think columnview sort order changes). Benchmarks: initially sorting the model qsort timsort 128,000 items 124ms 111ms 256,000 items 264ms 250ms
2020-07-17 00:47:22 +00:00
#include "gtktimsortprivate.h"
GtkSortListModel: Add
2018-09-14 02:34:40 +00:00
timsort: Add gtk_tim_sort_set_max_merge_size() Makes the SOrtListModel responsive when incrementally sorting. By making it configurable we can avoid losting performance in the non-incremental case.
2020-07-18 02:45:46 +00:00
/* The maximum amount of items to merge for a single merge step
*
* Making this smaller will result in more steps, which has more overhead and slows
* down total sort time.
* Making it larger will result in fewer steps, which increases the time taken for
* a single step.
*
* As merges are the most expensive steps, this is essentially a tunable for the
* longest time spent in gtk_tim_sort_step().
*
* Note that this should be reset to 0 when not doing incremental sorting to get
* rid of all the overhead.
*/
#define GTK_SORT_MAX_MERGE_SIZE (1024)
sortlistmodel: Make the sort callback useful 1. Run step() for a while to avoid very short steps This way, we batch items-changed() emissions. 2. Track the change region accurately This way, we can avoid invalidating the whole list if our step just touched a small part of a huge list. As this is a merge sort, this is a common occurence when we're buys merging chunks: The rest of the model outside those chunks isn't changed. Note that the tracking is accurate: It determines the minimum change region in the model. This will be important, because the testsuite is going to test this.
2020-07-20 23:40:06 +00:00
/* Time we spend in the sort callback before returning to the main loop
*
* Increasing this number will make the callback take longer and potentially
* reduce responsiveness of an application, but will increase the amount of
* work done per step. And we emit an ::items-changed() signal after every
* step, so if we can avoid that, we recuce the overhead in the list widget
* and in turn reduce the total sort time.
*/
#define GTK_SORT_STEP_TIME_US (1000) /* 1 millisecond */
GtkSortListModel: Add
2018-09-14 02:34:40 +00:00
/**
* SECTION:gtksortlistmodel
* @title: GtkSortListModel
docs: Miscellaneous doc fixes Additions and correction all over the place, in GDK and GTK docs.
2019-02-24 20:29:08 +00:00
* @short_description: A list model that sorts its items
Redo sort list model with GtkSorter Reshuffle the api to take full advantage of GtkSorter. Update all callers.
2019-12-03 04:44:22 +00:00
* @see_also: #GListModel, #GtkSorter
GtkSortListModel: Add
2018-09-14 02:34:40 +00:00
*
* #GtkSortListModel is a list model that takes a list model and
Redo sort list model with GtkSorter Reshuffle the api to take full advantage of GtkSorter. Update all callers.
2019-12-03 04:44:22 +00:00
* sorts its elements according to a #GtkSorter.
GtkSortListModel: Add
2018-09-14 02:34:40 +00:00
*
* #GtkSortListModel is a generic model and because of that it
* cannot take advantage of any external knowledge when sorting.
* If you run into performance issues with #GtkSortListModel, it
* is strongly recommended that you write your own sorting list
* model.
*/
enum {
PROP_0,
sortlistmodel: Add an incremental property Also refactor a large part of the sortmodel to make this convenient. A large amount of time has been spent on getting items-changed regions minimized.
2020-07-21 00:50:45 +00:00
PROP_INCREMENTAL,
GtkSortListModel: Add
2018-09-14 02:34:40 +00:00
PROP_MODEL,
sortlistmodel: Add progress estimation
2020-07-22 00:50:58 +00:00
PROP_PENDING,
Redo sort list model with GtkSorter Reshuffle the api to take full advantage of GtkSorter. Update all callers.
2019-12-03 04:44:22 +00:00
PROP_SORTER,
GtkSortListModel: Add
2018-09-14 02:34:40 +00:00
NUM_PROPERTIES
};
struct _GtkSortListModel
{
GObject parent_instance;
GListModel *model;
Redo sort list model with GtkSorter Reshuffle the api to take full advantage of GtkSorter. Update all callers.
2019-12-03 04:44:22 +00:00
GtkSorter *sorter;
sortlistmodel: Add an incremental property Also refactor a large part of the sortmodel to make this convenient. A large amount of time has been spent on getting items-changed regions minimized.
2020-07-21 00:50:45 +00:00
gboolean incremental;
GtkSortListModel: Add
2018-09-14 02:34:40 +00:00
sortlistmodel: Make sorting incremental This is just an experiment so far to see how long it takes to sort.
2020-07-20 23:46:09 +00:00
GtkTimSort sort; /* ongoing sort operation */
guint sort_cb; /* 0 or current ongoing sort callback */
sortlistmodel: Split the SortItem into 2 arrays Instead of one item keeping the item + its position and sorting that list, keep the items in 1 array and put the positions into a 2nd array. This is generally slower while sorting, but allows multiple improvements: 1. We can replace items with keys This allows avoiding multiple slow lookups when using complex comparisons 2. We can keep multiple position arrays This allows doing a sorting in the background without actually emitting items-changed() until the array is completely sorted. 3. The main list tracks the items in the original model So only a single memmove() is necessary there, while the old version had to upgrade the position in every item. Benchmarks: sorting a model of simple strings old new 256,000 items 256ms 268ms 512,000 items 569ms 638ms sorting a model of file trees, directories first, by size old new 64,000 items 350ms 364ms 128,000 items 667ms 691ms removing half the model old new 512,000 items 24ms 15ms 1,024,000 items 49ms 25ms
2020-07-21 01:09:10 +00:00
guint n_items;
sortlistmodel: Use GtkSortKeys This massively speeds up sorting with expensive sort functions that it's the most worthwhile optimization of this whole branch. It's slower for simple sort functions though. It's also quite a lot slower when the model doesn't support sort keys (like GtkCustomSorter), but all the other sorters do support keys. Of course, this depends on the number of items in the model - the number of comparisons scales O(N * log N) while the overhead for key handling scales O(N). So as the log N part grows, generating keys gets more and more beneficial. Benchmarks: initial sort of a GFileInfo model with display-name keys items keys 8,000 items 715ms 50ms 64,000 items --- 554ms initial sort of a GFileInfo model with complex keys items keys 64,000 items 340ms 295ms 128,000 items 641ms 605ms removing half a GFileInfo model with display-name keys (no comparisons, just key freeing overhead of a complex sorter) items keys 512,000 items 14ms 21ms 2,048,000 items 40ms 62ms removing half a GFileInfo model with complex keys (no comparisons, just key freeing overhead of a complex sorter) items keys 512,000 items 90ms 237ms 2,048,000 items 247ms 601ms
2020-07-21 01:40:28 +00:00
GtkSortKeys *sort_keys;
gsize key_size;
gpointer keys;
sortlistmodel: Make key generation part of the step function SSave the missing keys as a bitset and iterate over that bitset in the step function. Solves the problem with a large UI block at the beginning of a sort operation when all the keys were generated, in particular when key generation was slow. Benchmarks for maximum time taken by a single main loop callback: initial sort with complex GFileInfo keys old new 32,000 items 137ms 3ms 128,000 items 520ms 31ms initial sort with string keys old new 32,000 items 187ms 1ms 128,000 items 804ms 3ms
2020-07-21 23:43:59 +00:00
GtkBitset *missing_keys;
sortlistmodel: Split the SortItem into 2 arrays Instead of one item keeping the item + its position and sorting that list, keep the items in 1 array and put the positions into a 2nd array. This is generally slower while sorting, but allows multiple improvements: 1. We can replace items with keys This allows avoiding multiple slow lookups when using complex comparisons 2. We can keep multiple position arrays This allows doing a sorting in the background without actually emitting items-changed() until the array is completely sorted. 3. The main list tracks the items in the original model So only a single memmove() is necessary there, while the old version had to upgrade the position in every item. Benchmarks: sorting a model of simple strings old new 256,000 items 256ms 268ms 512,000 items 569ms 638ms sorting a model of file trees, directories first, by size old new 64,000 items 350ms 364ms 128,000 items 667ms 691ms removing half the model old new 512,000 items 24ms 15ms 1,024,000 items 49ms 25ms
2020-07-21 01:09:10 +00:00
gpointer *positions;
GtkSortListModel: Add
2018-09-14 02:34:40 +00:00
};
struct _GtkSortListModelClass
{
GObjectClass parent_class;
};
static GParamSpec *properties[NUM_PROPERTIES] = { NULL, };
sortlistmodel: Split the SortItem into 2 arrays Instead of one item keeping the item + its position and sorting that list, keep the items in 1 array and put the positions into a 2nd array. This is generally slower while sorting, but allows multiple improvements: 1. We can replace items with keys This allows avoiding multiple slow lookups when using complex comparisons 2. We can keep multiple position arrays This allows doing a sorting in the background without actually emitting items-changed() until the array is completely sorted. 3. The main list tracks the items in the original model So only a single memmove() is necessary there, while the old version had to upgrade the position in every item. Benchmarks: sorting a model of simple strings old new 256,000 items 256ms 268ms 512,000 items 569ms 638ms sorting a model of file trees, directories first, by size old new 64,000 items 350ms 364ms 128,000 items 667ms 691ms removing half the model old new 512,000 items 24ms 15ms 1,024,000 items 49ms 25ms
2020-07-21 01:09:10 +00:00
static guint
pos_from_key (GtkSortListModel *self,
gpointer key)
{
sortlistmodel: Use GtkSortKeys This massively speeds up sorting with expensive sort functions that it's the most worthwhile optimization of this whole branch. It's slower for simple sort functions though. It's also quite a lot slower when the model doesn't support sort keys (like GtkCustomSorter), but all the other sorters do support keys. Of course, this depends on the number of items in the model - the number of comparisons scales O(N * log N) while the overhead for key handling scales O(N). So as the log N part grows, generating keys gets more and more beneficial. Benchmarks: initial sort of a GFileInfo model with display-name keys items keys 8,000 items 715ms 50ms 64,000 items --- 554ms initial sort of a GFileInfo model with complex keys items keys 64,000 items 340ms 295ms 128,000 items 641ms 605ms removing half a GFileInfo model with display-name keys (no comparisons, just key freeing overhead of a complex sorter) items keys 512,000 items 14ms 21ms 2,048,000 items 40ms 62ms removing half a GFileInfo model with complex keys (no comparisons, just key freeing overhead of a complex sorter) items keys 512,000 items 90ms 237ms 2,048,000 items 247ms 601ms
2020-07-21 01:40:28 +00:00
guint pos = ((char *) key - (char *) self->keys) / self->key_size;
sortlistmodel: Split the SortItem into 2 arrays Instead of one item keeping the item + its position and sorting that list, keep the items in 1 array and put the positions into a 2nd array. This is generally slower while sorting, but allows multiple improvements: 1. We can replace items with keys This allows avoiding multiple slow lookups when using complex comparisons 2. We can keep multiple position arrays This allows doing a sorting in the background without actually emitting items-changed() until the array is completely sorted. 3. The main list tracks the items in the original model So only a single memmove() is necessary there, while the old version had to upgrade the position in every item. Benchmarks: sorting a model of simple strings old new 256,000 items 256ms 268ms 512,000 items 569ms 638ms sorting a model of file trees, directories first, by size old new 64,000 items 350ms 364ms 128,000 items 667ms 691ms removing half the model old new 512,000 items 24ms 15ms 1,024,000 items 49ms 25ms
2020-07-21 01:09:10 +00:00
g_assert (pos < self->n_items);
return pos;
}
static gpointer
key_from_pos (GtkSortListModel *self,
guint pos)
{
sortlistmodel: Use GtkSortKeys This massively speeds up sorting with expensive sort functions that it's the most worthwhile optimization of this whole branch. It's slower for simple sort functions though. It's also quite a lot slower when the model doesn't support sort keys (like GtkCustomSorter), but all the other sorters do support keys. Of course, this depends on the number of items in the model - the number of comparisons scales O(N * log N) while the overhead for key handling scales O(N). So as the log N part grows, generating keys gets more and more beneficial. Benchmarks: initial sort of a GFileInfo model with display-name keys items keys 8,000 items 715ms 50ms 64,000 items --- 554ms initial sort of a GFileInfo model with complex keys items keys 64,000 items 340ms 295ms 128,000 items 641ms 605ms removing half a GFileInfo model with display-name keys (no comparisons, just key freeing overhead of a complex sorter) items keys 512,000 items 14ms 21ms 2,048,000 items 40ms 62ms removing half a GFileInfo model with complex keys (no comparisons, just key freeing overhead of a complex sorter) items keys 512,000 items 90ms 237ms 2,048,000 items 247ms 601ms
2020-07-21 01:40:28 +00:00
return (char *) self->keys + self->key_size * pos;
sortlistmodel: Split the SortItem into 2 arrays Instead of one item keeping the item + its position and sorting that list, keep the items in 1 array and put the positions into a 2nd array. This is generally slower while sorting, but allows multiple improvements: 1. We can replace items with keys This allows avoiding multiple slow lookups when using complex comparisons 2. We can keep multiple position arrays This allows doing a sorting in the background without actually emitting items-changed() until the array is completely sorted. 3. The main list tracks the items in the original model So only a single memmove() is necessary there, while the old version had to upgrade the position in every item. Benchmarks: sorting a model of simple strings old new 256,000 items 256ms 268ms 512,000 items 569ms 638ms sorting a model of file trees, directories first, by size old new 64,000 items 350ms 364ms 128,000 items 667ms 691ms removing half the model old new 512,000 items 24ms 15ms 1,024,000 items 49ms 25ms
2020-07-21 01:09:10 +00:00
}
GtkSortListModel: Add
2018-09-14 02:34:40 +00:00
static GType
gtk_sort_list_model_get_item_type (GListModel *list)
{
listmodels: Stop respecting item-type Simplify all view model APIs and always return G_TYPE_OBJECT as the item-type for every model. It turns out nobody uses item-type anyway. So instead of adding lots of APIs, forcing people to think about it and trying to figure out how to handle filter or map models that modify item types, just having an easy life is a better approach. All the models need to be able to deal with any type of object going through anyway.
2020-07-04 19:47:48 +00:00
return G_TYPE_OBJECT;
GtkSortListModel: Add
2018-09-14 02:34:40 +00:00
}
static guint
gtk_sort_list_model_get_n_items (GListModel *list)
{
GtkSortListModel *self = GTK_SORT_LIST_MODEL (list);
if (self->model == NULL)
return 0;
return g_list_model_get_n_items (self->model);
}
static gpointer
gtk_sort_list_model_get_item (GListModel *list,
guint position)
{
GtkSortListModel *self = GTK_SORT_LIST_MODEL (list);
if (self->model == NULL)
return NULL;
sortlistmodel: Split the SortItem into 2 arrays Instead of one item keeping the item + its position and sorting that list, keep the items in 1 array and put the positions into a 2nd array. This is generally slower while sorting, but allows multiple improvements: 1. We can replace items with keys This allows avoiding multiple slow lookups when using complex comparisons 2. We can keep multiple position arrays This allows doing a sorting in the background without actually emitting items-changed() until the array is completely sorted. 3. The main list tracks the items in the original model So only a single memmove() is necessary there, while the old version had to upgrade the position in every item. Benchmarks: sorting a model of simple strings old new 256,000 items 256ms 268ms 512,000 items 569ms 638ms sorting a model of file trees, directories first, by size old new 64,000 items 350ms 364ms 128,000 items 667ms 691ms removing half the model old new 512,000 items 24ms 15ms 1,024,000 items 49ms 25ms
2020-07-21 01:09:10 +00:00
if (self->positions)
position = pos_from_key (self, self->positions[position]);
GtkSortListModel: Add
2018-09-14 02:34:40 +00:00
sortlistmodel: Split the SortItem into 2 arrays Instead of one item keeping the item + its position and sorting that list, keep the items in 1 array and put the positions into a 2nd array. This is generally slower while sorting, but allows multiple improvements: 1. We can replace items with keys This allows avoiding multiple slow lookups when using complex comparisons 2. We can keep multiple position arrays This allows doing a sorting in the background without actually emitting items-changed() until the array is completely sorted. 3. The main list tracks the items in the original model So only a single memmove() is necessary there, while the old version had to upgrade the position in every item. Benchmarks: sorting a model of simple strings old new 256,000 items 256ms 268ms 512,000 items 569ms 638ms sorting a model of file trees, directories first, by size old new 64,000 items 350ms 364ms 128,000 items 667ms 691ms removing half the model old new 512,000 items 24ms 15ms 1,024,000 items 49ms 25ms
2020-07-21 01:09:10 +00:00
return g_list_model_get_item (self->model, position);
GtkSortListModel: Add
2018-09-14 02:34:40 +00:00
}
static void
gtk_sort_list_model_model_init (GListModelInterface *iface)
{
iface->get_item_type = gtk_sort_list_model_get_item_type;
iface->get_n_items = gtk_sort_list_model_get_n_items;
iface->get_item = gtk_sort_list_model_get_item;
}
G_DEFINE_TYPE_WITH_CODE (GtkSortListModel, gtk_sort_list_model, G_TYPE_OBJECT,
G_IMPLEMENT_INTERFACE (G_TYPE_LIST_MODEL, gtk_sort_list_model_model_init))
sortlistmodel: Make sorting incremental This is just an experiment so far to see how long it takes to sort.
2020-07-20 23:46:09 +00:00
static gboolean
gtk_sort_list_model_is_sorting (GtkSortListModel *self)
{
return self->sort_cb != 0;
}
static void
sortlistmodel: Add an incremental property Also refactor a large part of the sortmodel to make this convenient. A large amount of time has been spent on getting items-changed regions minimized.
2020-07-21 00:50:45 +00:00
gtk_sort_list_model_stop_sorting (GtkSortListModel *self,
gsize *runs)
sortlistmodel: Make sorting incremental This is just an experiment so far to see how long it takes to sort.
2020-07-20 23:46:09 +00:00
{
if (self->sort_cb == 0)
sortlistmodel: Add an incremental property Also refactor a large part of the sortmodel to make this convenient. A large amount of time has been spent on getting items-changed regions minimized.
2020-07-21 00:50:45 +00:00
{
if (runs)
{
sortlistmodel: Split the SortItem into 2 arrays Instead of one item keeping the item + its position and sorting that list, keep the items in 1 array and put the positions into a 2nd array. This is generally slower while sorting, but allows multiple improvements: 1. We can replace items with keys This allows avoiding multiple slow lookups when using complex comparisons 2. We can keep multiple position arrays This allows doing a sorting in the background without actually emitting items-changed() until the array is completely sorted. 3. The main list tracks the items in the original model So only a single memmove() is necessary there, while the old version had to upgrade the position in every item. Benchmarks: sorting a model of simple strings old new 256,000 items 256ms 268ms 512,000 items 569ms 638ms sorting a model of file trees, directories first, by size old new 64,000 items 350ms 364ms 128,000 items 667ms 691ms removing half the model old new 512,000 items 24ms 15ms 1,024,000 items 49ms 25ms
2020-07-21 01:09:10 +00:00
runs[0] = self->n_items;
sortlistmodel: Add an incremental property Also refactor a large part of the sortmodel to make this convenient. A large amount of time has been spent on getting items-changed regions minimized.
2020-07-21 00:50:45 +00:00
runs[1] = 0;
}
return;
}
sortlistmodel: Make sorting incremental This is just an experiment so far to see how long it takes to sort.
2020-07-20 23:46:09 +00:00
sortlistmodel: Add an incremental property Also refactor a large part of the sortmodel to make this convenient. A large amount of time has been spent on getting items-changed regions minimized.
2020-07-21 00:50:45 +00:00
if (runs)
gtk_tim_sort_get_runs (&self->sort, runs);
sortlistmodel: Make sorting incremental This is just an experiment so far to see how long it takes to sort.
2020-07-20 23:46:09 +00:00
gtk_tim_sort_finish (&self->sort);
g_clear_handle_id (&self->sort_cb, g_source_remove);
sortlistmodel: Add progress estimation
2020-07-22 00:50:58 +00:00
g_object_notify_by_pspec (G_OBJECT (self), properties[PROP_PENDING]);
sortlistmodel: Make sorting incremental This is just an experiment so far to see how long it takes to sort.
2020-07-20 23:46:09 +00:00
}
sortlistmodel: Make the sort callback useful 1. Run step() for a while to avoid very short steps This way, we batch items-changed() emissions. 2. Track the change region accurately This way, we can avoid invalidating the whole list if our step just touched a small part of a huge list. As this is a merge sort, this is a common occurence when we're buys merging chunks: The rest of the model outside those chunks isn't changed. Note that the tracking is accurate: It determines the minimum change region in the model. This will be important, because the testsuite is going to test this.
2020-07-20 23:40:06 +00:00
static gboolean
gtk_sort_list_model_sort_step (GtkSortListModel *self,
sortlistmodel: Add an incremental property Also refactor a large part of the sortmodel to make this convenient. A large amount of time has been spent on getting items-changed regions minimized.
2020-07-21 00:50:45 +00:00
gboolean finish,
sortlistmodel: Make the sort callback useful 1. Run step() for a while to avoid very short steps This way, we batch items-changed() emissions. 2. Track the change region accurately This way, we can avoid invalidating the whole list if our step just touched a small part of a huge list. As this is a merge sort, this is a common occurence when we're buys merging chunks: The rest of the model outside those chunks isn't changed. Note that the tracking is accurate: It determines the minimum change region in the model. This will be important, because the testsuite is going to test this.
2020-07-20 23:40:06 +00:00
guint *out_position,
guint *out_n_items)
{
gint64 end_time = g_get_monotonic_time ();
gboolean result = FALSE;
GtkTimSortRun change;
sortlistmodel: Split the SortItem into 2 arrays Instead of one item keeping the item + its position and sorting that list, keep the items in 1 array and put the positions into a 2nd array. This is generally slower while sorting, but allows multiple improvements: 1. We can replace items with keys This allows avoiding multiple slow lookups when using complex comparisons 2. We can keep multiple position arrays This allows doing a sorting in the background without actually emitting items-changed() until the array is completely sorted. 3. The main list tracks the items in the original model So only a single memmove() is necessary there, while the old version had to upgrade the position in every item. Benchmarks: sorting a model of simple strings old new 256,000 items 256ms 268ms 512,000 items 569ms 638ms sorting a model of file trees, directories first, by size old new 64,000 items 350ms 364ms 128,000 items 667ms 691ms removing half the model old new 512,000 items 24ms 15ms 1,024,000 items 49ms 25ms
2020-07-21 01:09:10 +00:00
gpointer *start_change, *end_change;
sortlistmodel: Make the sort callback useful 1. Run step() for a while to avoid very short steps This way, we batch items-changed() emissions. 2. Track the change region accurately This way, we can avoid invalidating the whole list if our step just touched a small part of a huge list. As this is a merge sort, this is a common occurence when we're buys merging chunks: The rest of the model outside those chunks isn't changed. Note that the tracking is accurate: It determines the minimum change region in the model. This will be important, because the testsuite is going to test this.
2020-07-20 23:40:06 +00:00
end_time += GTK_SORT_STEP_TIME_US;
sortlistmodel: Make key generation part of the step function SSave the missing keys as a bitset and iterate over that bitset in the step function. Solves the problem with a large UI block at the beginning of a sort operation when all the keys were generated, in particular when key generation was slow. Benchmarks for maximum time taken by a single main loop callback: initial sort with complex GFileInfo keys old new 32,000 items 137ms 3ms 128,000 items 520ms 31ms initial sort with string keys old new 32,000 items 187ms 1ms 128,000 items 804ms 3ms
2020-07-21 23:43:59 +00:00
if (!gtk_bitset_is_empty (self->missing_keys))
{
GtkBitsetIter iter;
guint pos;
for (gtk_bitset_iter_init_first (&iter, self->missing_keys, &pos);
gtk_bitset_iter_is_valid (&iter);
gtk_bitset_iter_next (&iter, &pos))
{
gpointer item = g_list_model_get_item (self->model, pos);
gtk_sort_keys_init_key (self->sort_keys, item, key_from_pos (self, pos));
g_object_unref (item);
if (g_get_monotonic_time () >= end_time && !finish)
{
gtk_bitset_remove_range_closed (self->missing_keys, 0, pos);
*out_position = 0;
*out_n_items = 0;
return TRUE;
}
}
result = TRUE;
gtk_bitset_remove_all (self->missing_keys);
}
sortlistmodel: Split the SortItem into 2 arrays Instead of one item keeping the item + its position and sorting that list, keep the items in 1 array and put the positions into a 2nd array. This is generally slower while sorting, but allows multiple improvements: 1. We can replace items with keys This allows avoiding multiple slow lookups when using complex comparisons 2. We can keep multiple position arrays This allows doing a sorting in the background without actually emitting items-changed() until the array is completely sorted. 3. The main list tracks the items in the original model So only a single memmove() is necessary there, while the old version had to upgrade the position in every item. Benchmarks: sorting a model of simple strings old new 256,000 items 256ms 268ms 512,000 items 569ms 638ms sorting a model of file trees, directories first, by size old new 64,000 items 350ms 364ms 128,000 items 667ms 691ms removing half the model old new 512,000 items 24ms 15ms 1,024,000 items 49ms 25ms
2020-07-21 01:09:10 +00:00
end_change = self->positions;
start_change = self->positions + self->n_items;
sortlistmodel: Make the sort callback useful 1. Run step() for a while to avoid very short steps This way, we batch items-changed() emissions. 2. Track the change region accurately This way, we can avoid invalidating the whole list if our step just touched a small part of a huge list. As this is a merge sort, this is a common occurence when we're buys merging chunks: The rest of the model outside those chunks isn't changed. Note that the tracking is accurate: It determines the minimum change region in the model. This will be important, because the testsuite is going to test this.
2020-07-20 23:40:06 +00:00
while (gtk_tim_sort_step (&self->sort, &change))
{
result = TRUE;
if (change.len)
{
sortlistmodel: Split the SortItem into 2 arrays Instead of one item keeping the item + its position and sorting that list, keep the items in 1 array and put the positions into a 2nd array. This is generally slower while sorting, but allows multiple improvements: 1. We can replace items with keys This allows avoiding multiple slow lookups when using complex comparisons 2. We can keep multiple position arrays This allows doing a sorting in the background without actually emitting items-changed() until the array is completely sorted. 3. The main list tracks the items in the original model So only a single memmove() is necessary there, while the old version had to upgrade the position in every item. Benchmarks: sorting a model of simple strings old new 256,000 items 256ms 268ms 512,000 items 569ms 638ms sorting a model of file trees, directories first, by size old new 64,000 items 350ms 364ms 128,000 items 667ms 691ms removing half the model old new 512,000 items 24ms 15ms 1,024,000 items 49ms 25ms
2020-07-21 01:09:10 +00:00
start_change = MIN (start_change, (gpointer *) change.base);
end_change = MAX (end_change, ((gpointer *) change.base) + change.len);
sortlistmodel: Make the sort callback useful 1. Run step() for a while to avoid very short steps This way, we batch items-changed() emissions. 2. Track the change region accurately This way, we can avoid invalidating the whole list if our step just touched a small part of a huge list. As this is a merge sort, this is a common occurence when we're buys merging chunks: The rest of the model outside those chunks isn't changed. Note that the tracking is accurate: It determines the minimum change region in the model. This will be important, because the testsuite is going to test this.
2020-07-20 23:40:06 +00:00
}
sortlistmodel: Add an incremental property Also refactor a large part of the sortmodel to make this convenient. A large amount of time has been spent on getting items-changed regions minimized.
2020-07-21 00:50:45 +00:00
if (g_get_monotonic_time () >= end_time && !finish)
sortlistmodel: Make the sort callback useful 1. Run step() for a while to avoid very short steps This way, we batch items-changed() emissions. 2. Track the change region accurately This way, we can avoid invalidating the whole list if our step just touched a small part of a huge list. As this is a merge sort, this is a common occurence when we're buys merging chunks: The rest of the model outside those chunks isn't changed. Note that the tracking is accurate: It determines the minimum change region in the model. This will be important, because the testsuite is going to test this.
2020-07-20 23:40:06 +00:00
break;
}
if (start_change < end_change)
{
sortlistmodel: Split the SortItem into 2 arrays Instead of one item keeping the item + its position and sorting that list, keep the items in 1 array and put the positions into a 2nd array. This is generally slower while sorting, but allows multiple improvements: 1. We can replace items with keys This allows avoiding multiple slow lookups when using complex comparisons 2. We can keep multiple position arrays This allows doing a sorting in the background without actually emitting items-changed() until the array is completely sorted. 3. The main list tracks the items in the original model So only a single memmove() is necessary there, while the old version had to upgrade the position in every item. Benchmarks: sorting a model of simple strings old new 256,000 items 256ms 268ms 512,000 items 569ms 638ms sorting a model of file trees, directories first, by size old new 64,000 items 350ms 364ms 128,000 items 667ms 691ms removing half the model old new 512,000 items 24ms 15ms 1,024,000 items 49ms 25ms
2020-07-21 01:09:10 +00:00
*out_position = start_change - self->positions;
sortlistmodel: Make the sort callback useful 1. Run step() for a while to avoid very short steps This way, we batch items-changed() emissions. 2. Track the change region accurately This way, we can avoid invalidating the whole list if our step just touched a small part of a huge list. As this is a merge sort, this is a common occurence when we're buys merging chunks: The rest of the model outside those chunks isn't changed. Note that the tracking is accurate: It determines the minimum change region in the model. This will be important, because the testsuite is going to test this.
2020-07-20 23:40:06 +00:00
*out_n_items = end_change - start_change;
}
else
{
*out_position = 0;
*out_n_items = 0;
}
return result;
}
sortlistmodel: Make sorting incremental This is just an experiment so far to see how long it takes to sort.
2020-07-20 23:46:09 +00:00
static gboolean
gtk_sort_list_model_sort_cb (gpointer data)
{
GtkSortListModel *self = data;
sortlistmodel: Make the sort callback useful 1. Run step() for a while to avoid very short steps This way, we batch items-changed() emissions. 2. Track the change region accurately This way, we can avoid invalidating the whole list if our step just touched a small part of a huge list. As this is a merge sort, this is a common occurence when we're buys merging chunks: The rest of the model outside those chunks isn't changed. Note that the tracking is accurate: It determines the minimum change region in the model. This will be important, because the testsuite is going to test this.
2020-07-20 23:40:06 +00:00
guint pos, n_items;
sortlistmodel: Make sorting incremental This is just an experiment so far to see how long it takes to sort.
2020-07-20 23:46:09 +00:00
sortlistmodel: Add an incremental property Also refactor a large part of the sortmodel to make this convenient. A large amount of time has been spent on getting items-changed regions minimized.
2020-07-21 00:50:45 +00:00
if (gtk_sort_list_model_sort_step (self, FALSE, &pos, &n_items))
sortlistmodel: Make sorting incremental This is just an experiment so far to see how long it takes to sort.
2020-07-20 23:46:09 +00:00
{
sortlistmodel: Make the sort callback useful 1. Run step() for a while to avoid very short steps This way, we batch items-changed() emissions. 2. Track the change region accurately This way, we can avoid invalidating the whole list if our step just touched a small part of a huge list. As this is a merge sort, this is a common occurence when we're buys merging chunks: The rest of the model outside those chunks isn't changed. Note that the tracking is accurate: It determines the minimum change region in the model. This will be important, because the testsuite is going to test this.
2020-07-20 23:40:06 +00:00
if (n_items)
g_list_model_items_changed (G_LIST_MODEL (self), pos, n_items, n_items);
sortlistmodel: Add progress estimation
2020-07-22 00:50:58 +00:00
g_object_notify_by_pspec (G_OBJECT (self), properties[PROP_PENDING]);
sortlistmodel: Make sorting incremental This is just an experiment so far to see how long it takes to sort.
2020-07-20 23:46:09 +00:00
return G_SOURCE_CONTINUE;
}
sortlistmodel: Add an incremental property Also refactor a large part of the sortmodel to make this convenient. A large amount of time has been spent on getting items-changed regions minimized.
2020-07-21 00:50:45 +00:00
gtk_sort_list_model_stop_sorting (self, NULL);
sortlistmodel: Make sorting incremental This is just an experiment so far to see how long it takes to sort.
2020-07-20 23:46:09 +00:00
return G_SOURCE_REMOVE;
}
sortlistmodel: Add an incremental property Also refactor a large part of the sortmodel to make this convenient. A large amount of time has been spent on getting items-changed regions minimized.
2020-07-21 00:50:45 +00:00
static int
sort_func (gconstpointer a,
gconstpointer b,
gpointer data)
sortlistmodel: Make sorting incremental This is just an experiment so far to see how long it takes to sort.
2020-07-20 23:46:09 +00:00
{
sortlistmodel: Use GtkSortKeys This massively speeds up sorting with expensive sort functions that it's the most worthwhile optimization of this whole branch. It's slower for simple sort functions though. It's also quite a lot slower when the model doesn't support sort keys (like GtkCustomSorter), but all the other sorters do support keys. Of course, this depends on the number of items in the model - the number of comparisons scales O(N * log N) while the overhead for key handling scales O(N). So as the log N part grows, generating keys gets more and more beneficial. Benchmarks: initial sort of a GFileInfo model with display-name keys items keys 8,000 items 715ms 50ms 64,000 items --- 554ms initial sort of a GFileInfo model with complex keys items keys 64,000 items 340ms 295ms 128,000 items 641ms 605ms removing half a GFileInfo model with display-name keys (no comparisons, just key freeing overhead of a complex sorter) items keys 512,000 items 14ms 21ms 2,048,000 items 40ms 62ms removing half a GFileInfo model with complex keys (no comparisons, just key freeing overhead of a complex sorter) items keys 512,000 items 90ms 237ms 2,048,000 items 247ms 601ms
2020-07-21 01:40:28 +00:00
gpointer *sa = (gpointer *) a;
gpointer *sb = (gpointer *) b;
int result;
sortlistmodel: Add an incremental property Also refactor a large part of the sortmodel to make this convenient. A large amount of time has been spent on getting items-changed regions minimized.
2020-07-21 00:50:45 +00:00
sortlistmodel: Make sort stable again Previously, the sort was not stable when items were added/removed while sorting or the sort algorithm was changed. Now the sort looks at the item position (via the key's location in the keys array) to make sure each comparison stays stable with respect to this position.
2020-07-21 02:06:13 +00:00
result = gtk_sort_keys_compare (data, *sa, *sb);
if (result)
return result;
return *sa < *sb ? -1 : 1;
sortlistmodel: Add an incremental property Also refactor a large part of the sortmodel to make this convenient. A large amount of time has been spent on getting items-changed regions minimized.
2020-07-21 00:50:45 +00:00
}
static gboolean
gtk_sort_list_model_start_sorting (GtkSortListModel *self,
gsize *runs)
{
g_assert (self->sort_cb == 0);
gtk_tim_sort_init (&self->sort,
sortlistmodel: Split the SortItem into 2 arrays Instead of one item keeping the item + its position and sorting that list, keep the items in 1 array and put the positions into a 2nd array. This is generally slower while sorting, but allows multiple improvements: 1. We can replace items with keys This allows avoiding multiple slow lookups when using complex comparisons 2. We can keep multiple position arrays This allows doing a sorting in the background without actually emitting items-changed() until the array is completely sorted. 3. The main list tracks the items in the original model So only a single memmove() is necessary there, while the old version had to upgrade the position in every item. Benchmarks: sorting a model of simple strings old new 256,000 items 256ms 268ms 512,000 items 569ms 638ms sorting a model of file trees, directories first, by size old new 64,000 items 350ms 364ms 128,000 items 667ms 691ms removing half the model old new 512,000 items 24ms 15ms 1,024,000 items 49ms 25ms
2020-07-21 01:09:10 +00:00
self->positions,
self->n_items,
sizeof (gpointer),
sortlistmodel: Add an incremental property Also refactor a large part of the sortmodel to make this convenient. A large amount of time has been spent on getting items-changed regions minimized.
2020-07-21 00:50:45 +00:00
sort_func,
sortlistmodel: Use GtkSortKeys This massively speeds up sorting with expensive sort functions that it's the most worthwhile optimization of this whole branch. It's slower for simple sort functions though. It's also quite a lot slower when the model doesn't support sort keys (like GtkCustomSorter), but all the other sorters do support keys. Of course, this depends on the number of items in the model - the number of comparisons scales O(N * log N) while the overhead for key handling scales O(N). So as the log N part grows, generating keys gets more and more beneficial. Benchmarks: initial sort of a GFileInfo model with display-name keys items keys 8,000 items 715ms 50ms 64,000 items --- 554ms initial sort of a GFileInfo model with complex keys items keys 64,000 items 340ms 295ms 128,000 items 641ms 605ms removing half a GFileInfo model with display-name keys (no comparisons, just key freeing overhead of a complex sorter) items keys 512,000 items 14ms 21ms 2,048,000 items 40ms 62ms removing half a GFileInfo model with complex keys (no comparisons, just key freeing overhead of a complex sorter) items keys 512,000 items 90ms 237ms 2,048,000 items 247ms 601ms
2020-07-21 01:40:28 +00:00
self->sort_keys);
sortlistmodel: Add an incremental property Also refactor a large part of the sortmodel to make this convenient. A large amount of time has been spent on getting items-changed regions minimized.
2020-07-21 00:50:45 +00:00
if (runs)
gtk_tim_sort_set_runs (&self->sort, runs);
if (self->incremental)
gtk_tim_sort_set_max_merge_size (&self->sort, GTK_SORT_MAX_MERGE_SIZE);
if (!self->incremental)
return FALSE;
sortlistmodel: Make sorting incremental This is just an experiment so far to see how long it takes to sort.
2020-07-20 23:46:09 +00:00
self->sort_cb = g_idle_add (gtk_sort_list_model_sort_cb, self);
sortlistmodel: Add progress estimation
2020-07-22 00:50:58 +00:00
g_object_notify_by_pspec (G_OBJECT (self), properties[PROP_PENDING]);
sortlistmodel: Add an incremental property Also refactor a large part of the sortmodel to make this convenient. A large amount of time has been spent on getting items-changed regions minimized.
2020-07-21 00:50:45 +00:00
return TRUE;
sortlistmodel: Make sorting incremental This is just an experiment so far to see how long it takes to sort.
2020-07-20 23:46:09 +00:00
}
GtkSortListModel: Add
2018-09-14 02:34:40 +00:00
static void
sortlistmodel: Add an incremental property Also refactor a large part of the sortmodel to make this convenient. A large amount of time has been spent on getting items-changed regions minimized.
2020-07-21 00:50:45 +00:00
gtk_sort_list_model_finish_sorting (GtkSortListModel *self,
guint *pos,
guint *n_items)
GtkSortListModel: Add
2018-09-14 02:34:40 +00:00
{
sortlistmodel: Add an incremental property Also refactor a large part of the sortmodel to make this convenient. A large amount of time has been spent on getting items-changed regions minimized.
2020-07-21 00:50:45 +00:00
gtk_tim_sort_set_max_merge_size (&self->sort, 0);
gtk_sort_list_model_sort_step (self, TRUE, pos, n_items);
gtk_tim_sort_finish (&self->sort);
gtk_sort_list_model_stop_sorting (self, NULL);
}
sortlistmodel: Use GtkSortKeys This massively speeds up sorting with expensive sort functions that it's the most worthwhile optimization of this whole branch. It's slower for simple sort functions though. It's also quite a lot slower when the model doesn't support sort keys (like GtkCustomSorter), but all the other sorters do support keys. Of course, this depends on the number of items in the model - the number of comparisons scales O(N * log N) while the overhead for key handling scales O(N). So as the log N part grows, generating keys gets more and more beneficial. Benchmarks: initial sort of a GFileInfo model with display-name keys items keys 8,000 items 715ms 50ms 64,000 items --- 554ms initial sort of a GFileInfo model with complex keys items keys 64,000 items 340ms 295ms 128,000 items 641ms 605ms removing half a GFileInfo model with display-name keys (no comparisons, just key freeing overhead of a complex sorter) items keys 512,000 items 14ms 21ms 2,048,000 items 40ms 62ms removing half a GFileInfo model with complex keys (no comparisons, just key freeing overhead of a complex sorter) items keys 512,000 items 90ms 237ms 2,048,000 items 247ms 601ms
2020-07-21 01:40:28 +00:00
static void
sortlistmodel: Make key generation part of the step function SSave the missing keys as a bitset and iterate over that bitset in the step function. Solves the problem with a large UI block at the beginning of a sort operation when all the keys were generated, in particular when key generation was slow. Benchmarks for maximum time taken by a single main loop callback: initial sort with complex GFileInfo keys old new 32,000 items 137ms 3ms 128,000 items 520ms 31ms initial sort with string keys old new 32,000 items 187ms 1ms 128,000 items 804ms 3ms
2020-07-21 23:43:59 +00:00
gtk_sort_list_model_clear_sort_keys (GtkSortListModel *self,
guint position,
guint n_items)
sortlistmodel: Use GtkSortKeys This massively speeds up sorting with expensive sort functions that it's the most worthwhile optimization of this whole branch. It's slower for simple sort functions though. It's also quite a lot slower when the model doesn't support sort keys (like GtkCustomSorter), but all the other sorters do support keys. Of course, this depends on the number of items in the model - the number of comparisons scales O(N * log N) while the overhead for key handling scales O(N). So as the log N part grows, generating keys gets more and more beneficial. Benchmarks: initial sort of a GFileInfo model with display-name keys items keys 8,000 items 715ms 50ms 64,000 items --- 554ms initial sort of a GFileInfo model with complex keys items keys 64,000 items 340ms 295ms 128,000 items 641ms 605ms removing half a GFileInfo model with display-name keys (no comparisons, just key freeing overhead of a complex sorter) items keys 512,000 items 14ms 21ms 2,048,000 items 40ms 62ms removing half a GFileInfo model with complex keys (no comparisons, just key freeing overhead of a complex sorter) items keys 512,000 items 90ms 237ms 2,048,000 items 247ms 601ms
2020-07-21 01:40:28 +00:00
{
sortlistmodel: Make key generation part of the step function SSave the missing keys as a bitset and iterate over that bitset in the step function. Solves the problem with a large UI block at the beginning of a sort operation when all the keys were generated, in particular when key generation was slow. Benchmarks for maximum time taken by a single main loop callback: initial sort with complex GFileInfo keys old new 32,000 items 137ms 3ms 128,000 items 520ms 31ms initial sort with string keys old new 32,000 items 187ms 1ms 128,000 items 804ms 3ms
2020-07-21 23:43:59 +00:00
GtkBitsetIter iter;
GtkBitset *clear;
guint pos;
sortlistmodel: Use GtkSortKeys This massively speeds up sorting with expensive sort functions that it's the most worthwhile optimization of this whole branch. It's slower for simple sort functions though. It's also quite a lot slower when the model doesn't support sort keys (like GtkCustomSorter), but all the other sorters do support keys. Of course, this depends on the number of items in the model - the number of comparisons scales O(N * log N) while the overhead for key handling scales O(N). So as the log N part grows, generating keys gets more and more beneficial. Benchmarks: initial sort of a GFileInfo model with display-name keys items keys 8,000 items 715ms 50ms 64,000 items --- 554ms initial sort of a GFileInfo model with complex keys items keys 64,000 items 340ms 295ms 128,000 items 641ms 605ms removing half a GFileInfo model with display-name keys (no comparisons, just key freeing overhead of a complex sorter) items keys 512,000 items 14ms 21ms 2,048,000 items 40ms 62ms removing half a GFileInfo model with complex keys (no comparisons, just key freeing overhead of a complex sorter) items keys 512,000 items 90ms 237ms 2,048,000 items 247ms 601ms
2020-07-21 01:40:28 +00:00
sortlistmodel: Make key generation part of the step function SSave the missing keys as a bitset and iterate over that bitset in the step function. Solves the problem with a large UI block at the beginning of a sort operation when all the keys were generated, in particular when key generation was slow. Benchmarks for maximum time taken by a single main loop callback: initial sort with complex GFileInfo keys old new 32,000 items 137ms 3ms 128,000 items 520ms 31ms initial sort with string keys old new 32,000 items 187ms 1ms 128,000 items 804ms 3ms
2020-07-21 23:43:59 +00:00
if (!gtk_sort_keys_needs_clear_key (self->sort_keys))
return;
clear = gtk_bitset_new_range (position, n_items);
gtk_bitset_subtract (clear, self->missing_keys);
sortlistmodel: Use GtkSortKeys This massively speeds up sorting with expensive sort functions that it's the most worthwhile optimization of this whole branch. It's slower for simple sort functions though. It's also quite a lot slower when the model doesn't support sort keys (like GtkCustomSorter), but all the other sorters do support keys. Of course, this depends on the number of items in the model - the number of comparisons scales O(N * log N) while the overhead for key handling scales O(N). So as the log N part grows, generating keys gets more and more beneficial. Benchmarks: initial sort of a GFileInfo model with display-name keys items keys 8,000 items 715ms 50ms 64,000 items --- 554ms initial sort of a GFileInfo model with complex keys items keys 64,000 items 340ms 295ms 128,000 items 641ms 605ms removing half a GFileInfo model with display-name keys (no comparisons, just key freeing overhead of a complex sorter) items keys 512,000 items 14ms 21ms 2,048,000 items 40ms 62ms removing half a GFileInfo model with complex keys (no comparisons, just key freeing overhead of a complex sorter) items keys 512,000 items 90ms 237ms 2,048,000 items 247ms 601ms
2020-07-21 01:40:28 +00:00
sortlistmodel: Make key generation part of the step function SSave the missing keys as a bitset and iterate over that bitset in the step function. Solves the problem with a large UI block at the beginning of a sort operation when all the keys were generated, in particular when key generation was slow. Benchmarks for maximum time taken by a single main loop callback: initial sort with complex GFileInfo keys old new 32,000 items 137ms 3ms 128,000 items 520ms 31ms initial sort with string keys old new 32,000 items 187ms 1ms 128,000 items 804ms 3ms
2020-07-21 23:43:59 +00:00
for (gtk_bitset_iter_init_first (&iter, clear, &pos);
gtk_bitset_iter_is_valid (&iter);
gtk_bitset_iter_next (&iter, &pos))
{
gtk_sort_keys_clear_key (self->sort_keys, key_from_pos (self, pos));
sortlistmodel: Use GtkSortKeys This massively speeds up sorting with expensive sort functions that it's the most worthwhile optimization of this whole branch. It's slower for simple sort functions though. It's also quite a lot slower when the model doesn't support sort keys (like GtkCustomSorter), but all the other sorters do support keys. Of course, this depends on the number of items in the model - the number of comparisons scales O(N * log N) while the overhead for key handling scales O(N). So as the log N part grows, generating keys gets more and more beneficial. Benchmarks: initial sort of a GFileInfo model with display-name keys items keys 8,000 items 715ms 50ms 64,000 items --- 554ms initial sort of a GFileInfo model with complex keys items keys 64,000 items 340ms 295ms 128,000 items 641ms 605ms removing half a GFileInfo model with display-name keys (no comparisons, just key freeing overhead of a complex sorter) items keys 512,000 items 14ms 21ms 2,048,000 items 40ms 62ms removing half a GFileInfo model with complex keys (no comparisons, just key freeing overhead of a complex sorter) items keys 512,000 items 90ms 237ms 2,048,000 items 247ms 601ms
2020-07-21 01:40:28 +00:00
}
sortlistmodel: Make key generation part of the step function SSave the missing keys as a bitset and iterate over that bitset in the step function. Solves the problem with a large UI block at the beginning of a sort operation when all the keys were generated, in particular when key generation was slow. Benchmarks for maximum time taken by a single main loop callback: initial sort with complex GFileInfo keys old new 32,000 items 137ms 3ms 128,000 items 520ms 31ms initial sort with string keys old new 32,000 items 187ms 1ms 128,000 items 804ms 3ms
2020-07-21 23:43:59 +00:00
gtk_bitset_unref (clear);
}
static void
gtk_sort_list_model_clear_keys (GtkSortListModel *self)
{
gtk_sort_list_model_clear_sort_keys (self, 0, self->n_items);
g_clear_pointer (&self->missing_keys, gtk_bitset_unref);
sortlistmodel: Use GtkSortKeys This massively speeds up sorting with expensive sort functions that it's the most worthwhile optimization of this whole branch. It's slower for simple sort functions though. It's also quite a lot slower when the model doesn't support sort keys (like GtkCustomSorter), but all the other sorters do support keys. Of course, this depends on the number of items in the model - the number of comparisons scales O(N * log N) while the overhead for key handling scales O(N). So as the log N part grows, generating keys gets more and more beneficial. Benchmarks: initial sort of a GFileInfo model with display-name keys items keys 8,000 items 715ms 50ms 64,000 items --- 554ms initial sort of a GFileInfo model with complex keys items keys 64,000 items 340ms 295ms 128,000 items 641ms 605ms removing half a GFileInfo model with display-name keys (no comparisons, just key freeing overhead of a complex sorter) items keys 512,000 items 14ms 21ms 2,048,000 items 40ms 62ms removing half a GFileInfo model with complex keys (no comparisons, just key freeing overhead of a complex sorter) items keys 512,000 items 90ms 237ms 2,048,000 items 247ms 601ms
2020-07-21 01:40:28 +00:00
g_clear_pointer (&self->keys, g_free);
g_clear_pointer (&self->sort_keys, gtk_sort_keys_unref);
self->key_size = 0;
}
sortlistmodel: Add an incremental property Also refactor a large part of the sortmodel to make this convenient. A large amount of time has been spent on getting items-changed regions minimized.
2020-07-21 00:50:45 +00:00
static void
gtk_sort_list_model_clear_items (GtkSortListModel *self,
guint *pos,
guint *n_items)
{
gtk_sort_list_model_stop_sorting (self, NULL);
sortlistmodel: Use GtkSortKeys This massively speeds up sorting with expensive sort functions that it's the most worthwhile optimization of this whole branch. It's slower for simple sort functions though. It's also quite a lot slower when the model doesn't support sort keys (like GtkCustomSorter), but all the other sorters do support keys. Of course, this depends on the number of items in the model - the number of comparisons scales O(N * log N) while the overhead for key handling scales O(N). So as the log N part grows, generating keys gets more and more beneficial. Benchmarks: initial sort of a GFileInfo model with display-name keys items keys 8,000 items 715ms 50ms 64,000 items --- 554ms initial sort of a GFileInfo model with complex keys items keys 64,000 items 340ms 295ms 128,000 items 641ms 605ms removing half a GFileInfo model with display-name keys (no comparisons, just key freeing overhead of a complex sorter) items keys 512,000 items 14ms 21ms 2,048,000 items 40ms 62ms removing half a GFileInfo model with complex keys (no comparisons, just key freeing overhead of a complex sorter) items keys 512,000 items 90ms 237ms 2,048,000 items 247ms 601ms
2020-07-21 01:40:28 +00:00
if (self->sort_keys == NULL)
sortlistmodel: Split the SortItem into 2 arrays Instead of one item keeping the item + its position and sorting that list, keep the items in 1 array and put the positions into a 2nd array. This is generally slower while sorting, but allows multiple improvements: 1. We can replace items with keys This allows avoiding multiple slow lookups when using complex comparisons 2. We can keep multiple position arrays This allows doing a sorting in the background without actually emitting items-changed() until the array is completely sorted. 3. The main list tracks the items in the original model So only a single memmove() is necessary there, while the old version had to upgrade the position in every item. Benchmarks: sorting a model of simple strings old new 256,000 items 256ms 268ms 512,000 items 569ms 638ms sorting a model of file trees, directories first, by size old new 64,000 items 350ms 364ms 128,000 items 667ms 691ms removing half the model old new 512,000 items 24ms 15ms 1,024,000 items 49ms 25ms
2020-07-21 01:09:10 +00:00
{
if (pos || n_items)
*pos = *n_items = 0;
return;
}
sortlistmodel: Add an incremental property Also refactor a large part of the sortmodel to make this convenient. A large amount of time has been spent on getting items-changed regions minimized.
2020-07-21 00:50:45 +00:00
if (pos || n_items)
{
guint start, end;
sortlistmodel: Split the SortItem into 2 arrays Instead of one item keeping the item + its position and sorting that list, keep the items in 1 array and put the positions into a 2nd array. This is generally slower while sorting, but allows multiple improvements: 1. We can replace items with keys This allows avoiding multiple slow lookups when using complex comparisons 2. We can keep multiple position arrays This allows doing a sorting in the background without actually emitting items-changed() until the array is completely sorted. 3. The main list tracks the items in the original model So only a single memmove() is necessary there, while the old version had to upgrade the position in every item. Benchmarks: sorting a model of simple strings old new 256,000 items 256ms 268ms 512,000 items 569ms 638ms sorting a model of file trees, directories first, by size old new 64,000 items 350ms 364ms 128,000 items 667ms 691ms removing half the model old new 512,000 items 24ms 15ms 1,024,000 items 49ms 25ms
2020-07-21 01:09:10 +00:00
for (start = 0; start < self->n_items; start++)
sortlistmodel: Add an incremental property Also refactor a large part of the sortmodel to make this convenient. A large amount of time has been spent on getting items-changed regions minimized.
2020-07-21 00:50:45 +00:00
{
sortlistmodel: Split the SortItem into 2 arrays Instead of one item keeping the item + its position and sorting that list, keep the items in 1 array and put the positions into a 2nd array. This is generally slower while sorting, but allows multiple improvements: 1. We can replace items with keys This allows avoiding multiple slow lookups when using complex comparisons 2. We can keep multiple position arrays This allows doing a sorting in the background without actually emitting items-changed() until the array is completely sorted. 3. The main list tracks the items in the original model So only a single memmove() is necessary there, while the old version had to upgrade the position in every item. Benchmarks: sorting a model of simple strings old new 256,000 items 256ms 268ms 512,000 items 569ms 638ms sorting a model of file trees, directories first, by size old new 64,000 items 350ms 364ms 128,000 items 667ms 691ms removing half the model old new 512,000 items 24ms 15ms 1,024,000 items 49ms 25ms
2020-07-21 01:09:10 +00:00
if (pos_from_key (self, self->positions[start]) != + start)
sortlistmodel: Add an incremental property Also refactor a large part of the sortmodel to make this convenient. A large amount of time has been spent on getting items-changed regions minimized.
2020-07-21 00:50:45 +00:00
break;
}
sortlistmodel: Split the SortItem into 2 arrays Instead of one item keeping the item + its position and sorting that list, keep the items in 1 array and put the positions into a 2nd array. This is generally slower while sorting, but allows multiple improvements: 1. We can replace items with keys This allows avoiding multiple slow lookups when using complex comparisons 2. We can keep multiple position arrays This allows doing a sorting in the background without actually emitting items-changed() until the array is completely sorted. 3. The main list tracks the items in the original model So only a single memmove() is necessary there, while the old version had to upgrade the position in every item. Benchmarks: sorting a model of simple strings old new 256,000 items 256ms 268ms 512,000 items 569ms 638ms sorting a model of file trees, directories first, by size old new 64,000 items 350ms 364ms 128,000 items 667ms 691ms removing half the model old new 512,000 items 24ms 15ms 1,024,000 items 49ms 25ms
2020-07-21 01:09:10 +00:00
for (end = self->n_items; end > start; end--)
sortlistmodel: Add an incremental property Also refactor a large part of the sortmodel to make this convenient. A large amount of time has been spent on getting items-changed regions minimized.
2020-07-21 00:50:45 +00:00
{
sortlistmodel: Split the SortItem into 2 arrays Instead of one item keeping the item + its position and sorting that list, keep the items in 1 array and put the positions into a 2nd array. This is generally slower while sorting, but allows multiple improvements: 1. We can replace items with keys This allows avoiding multiple slow lookups when using complex comparisons 2. We can keep multiple position arrays This allows doing a sorting in the background without actually emitting items-changed() until the array is completely sorted. 3. The main list tracks the items in the original model So only a single memmove() is necessary there, while the old version had to upgrade the position in every item. Benchmarks: sorting a model of simple strings old new 256,000 items 256ms 268ms 512,000 items 569ms 638ms sorting a model of file trees, directories first, by size old new 64,000 items 350ms 364ms 128,000 items 667ms 691ms removing half the model old new 512,000 items 24ms 15ms 1,024,000 items 49ms 25ms
2020-07-21 01:09:10 +00:00
if (pos_from_key (self, self->positions[end - 1]) != end - 1)
sortlistmodel: Add an incremental property Also refactor a large part of the sortmodel to make this convenient. A large amount of time has been spent on getting items-changed regions minimized.
2020-07-21 00:50:45 +00:00
break;
}
*n_items = end - start;
if (*n_items == 0)
*pos = 0;
else
*pos = start;
}
sortlistmodel: Split the SortItem into 2 arrays Instead of one item keeping the item + its position and sorting that list, keep the items in 1 array and put the positions into a 2nd array. This is generally slower while sorting, but allows multiple improvements: 1. We can replace items with keys This allows avoiding multiple slow lookups when using complex comparisons 2. We can keep multiple position arrays This allows doing a sorting in the background without actually emitting items-changed() until the array is completely sorted. 3. The main list tracks the items in the original model So only a single memmove() is necessary there, while the old version had to upgrade the position in every item. Benchmarks: sorting a model of simple strings old new 256,000 items 256ms 268ms 512,000 items 569ms 638ms sorting a model of file trees, directories first, by size old new 64,000 items 350ms 364ms 128,000 items 667ms 691ms removing half the model old new 512,000 items 24ms 15ms 1,024,000 items 49ms 25ms
2020-07-21 01:09:10 +00:00
g_clear_pointer (&self->positions, g_free);
sortlistmodel: Use GtkSortKeys This massively speeds up sorting with expensive sort functions that it's the most worthwhile optimization of this whole branch. It's slower for simple sort functions though. It's also quite a lot slower when the model doesn't support sort keys (like GtkCustomSorter), but all the other sorters do support keys. Of course, this depends on the number of items in the model - the number of comparisons scales O(N * log N) while the overhead for key handling scales O(N). So as the log N part grows, generating keys gets more and more beneficial. Benchmarks: initial sort of a GFileInfo model with display-name keys items keys 8,000 items 715ms 50ms 64,000 items --- 554ms initial sort of a GFileInfo model with complex keys items keys 64,000 items 340ms 295ms 128,000 items 641ms 605ms removing half a GFileInfo model with display-name keys (no comparisons, just key freeing overhead of a complex sorter) items keys 512,000 items 14ms 21ms 2,048,000 items 40ms 62ms removing half a GFileInfo model with complex keys (no comparisons, just key freeing overhead of a complex sorter) items keys 512,000 items 90ms 237ms 2,048,000 items 247ms 601ms
2020-07-21 01:40:28 +00:00
gtk_sort_list_model_clear_keys (self);
sortlistmodel: Replace with an array-based model This is the dumbest possible sortmodel using an array: Just grab all the items, put them in the array, qsort() the array. Some benchmarks (setting a new model): 125,000 items - old: 549ms new: 115ms 250,000 items - new: 250ms This performance can not be kept for simple additions and removals though.
2020-07-16 23:56:18 +00:00
}
GtkSortListModel: Add
2018-09-14 02:34:40 +00:00
sortlistmodel: Track item positions The model now tracks the original positions on top of just the items so that it can remove items in an items-changed emission. It now takes twice as much memory but removes items much faster. Benchmarks: Removing 50% of a model: before after 250,000 items 135ms 10ms 500,000 items 300ms 25ms Removing 1 item: 4,000 items 2.2ms 0ms 8,000 items 4.6ms 0ms 500,000 items --- 0.01ms
2020-07-17 00:28:42 +00:00
static gboolean
gtk_sort_list_model_should_sort (GtkSortListModel *self)
{
return self->sorter != NULL &&
self->model != NULL &&
gtk_sorter_get_order (self->sorter) != GTK_SORTER_ORDER_NONE;
}
sortlistmodel: Use GtkSortKeys This massively speeds up sorting with expensive sort functions that it's the most worthwhile optimization of this whole branch. It's slower for simple sort functions though. It's also quite a lot slower when the model doesn't support sort keys (like GtkCustomSorter), but all the other sorters do support keys. Of course, this depends on the number of items in the model - the number of comparisons scales O(N * log N) while the overhead for key handling scales O(N). So as the log N part grows, generating keys gets more and more beneficial. Benchmarks: initial sort of a GFileInfo model with display-name keys items keys 8,000 items 715ms 50ms 64,000 items --- 554ms initial sort of a GFileInfo model with complex keys items keys 64,000 items 340ms 295ms 128,000 items 641ms 605ms removing half a GFileInfo model with display-name keys (no comparisons, just key freeing overhead of a complex sorter) items keys 512,000 items 14ms 21ms 2,048,000 items 40ms 62ms removing half a GFileInfo model with complex keys (no comparisons, just key freeing overhead of a complex sorter) items keys 512,000 items 90ms 237ms 2,048,000 items 247ms 601ms
2020-07-21 01:40:28 +00:00
static void
gtk_sort_list_model_create_keys (GtkSortListModel *self)
{
g_assert (self->keys == NULL);
g_assert (self->sort_keys == NULL);
g_assert (self->key_size == 0);
self->sort_keys = gtk_sorter_get_keys (self->sorter);
multisorter: Implement GtkSortKeys
2020-07-20 20:24:36 +00:00
self->key_size = gtk_sort_keys_get_key_size (self->sort_keys);
sortlistmodel: Use GtkSortKeys This massively speeds up sorting with expensive sort functions that it's the most worthwhile optimization of this whole branch. It's slower for simple sort functions though. It's also quite a lot slower when the model doesn't support sort keys (like GtkCustomSorter), but all the other sorters do support keys. Of course, this depends on the number of items in the model - the number of comparisons scales O(N * log N) while the overhead for key handling scales O(N). So as the log N part grows, generating keys gets more and more beneficial. Benchmarks: initial sort of a GFileInfo model with display-name keys items keys 8,000 items 715ms 50ms 64,000 items --- 554ms initial sort of a GFileInfo model with complex keys items keys 64,000 items 340ms 295ms 128,000 items 641ms 605ms removing half a GFileInfo model with display-name keys (no comparisons, just key freeing overhead of a complex sorter) items keys 512,000 items 14ms 21ms 2,048,000 items 40ms 62ms removing half a GFileInfo model with complex keys (no comparisons, just key freeing overhead of a complex sorter) items keys 512,000 items 90ms 237ms 2,048,000 items 247ms 601ms
2020-07-21 01:40:28 +00:00
self->keys = g_malloc_n (self->n_items, self->key_size);
sortlistmodel: Make key generation part of the step function SSave the missing keys as a bitset and iterate over that bitset in the step function. Solves the problem with a large UI block at the beginning of a sort operation when all the keys were generated, in particular when key generation was slow. Benchmarks for maximum time taken by a single main loop callback: initial sort with complex GFileInfo keys old new 32,000 items 137ms 3ms 128,000 items 520ms 31ms initial sort with string keys old new 32,000 items 187ms 1ms 128,000 items 804ms 3ms
2020-07-21 23:43:59 +00:00
self->missing_keys = gtk_bitset_new_range (0, self->n_items);
sortlistmodel: Use GtkSortKeys This massively speeds up sorting with expensive sort functions that it's the most worthwhile optimization of this whole branch. It's slower for simple sort functions though. It's also quite a lot slower when the model doesn't support sort keys (like GtkCustomSorter), but all the other sorters do support keys. Of course, this depends on the number of items in the model - the number of comparisons scales O(N * log N) while the overhead for key handling scales O(N). So as the log N part grows, generating keys gets more and more beneficial. Benchmarks: initial sort of a GFileInfo model with display-name keys items keys 8,000 items 715ms 50ms 64,000 items --- 554ms initial sort of a GFileInfo model with complex keys items keys 64,000 items 340ms 295ms 128,000 items 641ms 605ms removing half a GFileInfo model with display-name keys (no comparisons, just key freeing overhead of a complex sorter) items keys 512,000 items 14ms 21ms 2,048,000 items 40ms 62ms removing half a GFileInfo model with complex keys (no comparisons, just key freeing overhead of a complex sorter) items keys 512,000 items 90ms 237ms 2,048,000 items 247ms 601ms
2020-07-21 01:40:28 +00:00
}
sortlistmodel: Replace with an array-based model This is the dumbest possible sortmodel using an array: Just grab all the items, put them in the array, qsort() the array. Some benchmarks (setting a new model): 125,000 items - old: 549ms new: 115ms 250,000 items - new: 250ms This performance can not be kept for simple additions and removals though.
2020-07-16 23:56:18 +00:00
static void
gtk_sort_list_model_create_items (GtkSortListModel *self)
{
sortlistmodel: Split the SortItem into 2 arrays Instead of one item keeping the item + its position and sorting that list, keep the items in 1 array and put the positions into a 2nd array. This is generally slower while sorting, but allows multiple improvements: 1. We can replace items with keys This allows avoiding multiple slow lookups when using complex comparisons 2. We can keep multiple position arrays This allows doing a sorting in the background without actually emitting items-changed() until the array is completely sorted. 3. The main list tracks the items in the original model So only a single memmove() is necessary there, while the old version had to upgrade the position in every item. Benchmarks: sorting a model of simple strings old new 256,000 items 256ms 268ms 512,000 items 569ms 638ms sorting a model of file trees, directories first, by size old new 64,000 items 350ms 364ms 128,000 items 667ms 691ms removing half the model old new 512,000 items 24ms 15ms 1,024,000 items 49ms 25ms
2020-07-21 01:09:10 +00:00
guint i;
GtkSortListModel: Add
2018-09-14 02:34:40 +00:00
sortlistmodel: Track item positions The model now tracks the original positions on top of just the items so that it can remove items in an items-changed emission. It now takes twice as much memory but removes items much faster. Benchmarks: Removing 50% of a model: before after 250,000 items 135ms 10ms 500,000 items 300ms 25ms Removing 1 item: 4,000 items 2.2ms 0ms 8,000 items 4.6ms 0ms 500,000 items --- 0.01ms
2020-07-17 00:28:42 +00:00
if (!gtk_sort_list_model_should_sort (self))
sortlistmodel: Replace with an array-based model This is the dumbest possible sortmodel using an array: Just grab all the items, put them in the array, qsort() the array. Some benchmarks (setting a new model): 125,000 items - old: 549ms new: 115ms 250,000 items - new: 250ms This performance can not be kept for simple additions and removals though.
2020-07-16 23:56:18 +00:00
return;
GtkSortListModel: Add
2018-09-14 02:34:40 +00:00
sortlistmodel: Use GtkSortKeys This massively speeds up sorting with expensive sort functions that it's the most worthwhile optimization of this whole branch. It's slower for simple sort functions though. It's also quite a lot slower when the model doesn't support sort keys (like GtkCustomSorter), but all the other sorters do support keys. Of course, this depends on the number of items in the model - the number of comparisons scales O(N * log N) while the overhead for key handling scales O(N). So as the log N part grows, generating keys gets more and more beneficial. Benchmarks: initial sort of a GFileInfo model with display-name keys items keys 8,000 items 715ms 50ms 64,000 items --- 554ms initial sort of a GFileInfo model with complex keys items keys 64,000 items 340ms 295ms 128,000 items 641ms 605ms removing half a GFileInfo model with display-name keys (no comparisons, just key freeing overhead of a complex sorter) items keys 512,000 items 14ms 21ms 2,048,000 items 40ms 62ms removing half a GFileInfo model with complex keys (no comparisons, just key freeing overhead of a complex sorter) items keys 512,000 items 90ms 237ms 2,048,000 items 247ms 601ms
2020-07-21 01:40:28 +00:00
g_assert (self->sort_keys == NULL);
sortlistmodel: Split the SortItem into 2 arrays Instead of one item keeping the item + its position and sorting that list, keep the items in 1 array and put the positions into a 2nd array. This is generally slower while sorting, but allows multiple improvements: 1. We can replace items with keys This allows avoiding multiple slow lookups when using complex comparisons 2. We can keep multiple position arrays This allows doing a sorting in the background without actually emitting items-changed() until the array is completely sorted. 3. The main list tracks the items in the original model So only a single memmove() is necessary there, while the old version had to upgrade the position in every item. Benchmarks: sorting a model of simple strings old new 256,000 items 256ms 268ms 512,000 items 569ms 638ms sorting a model of file trees, directories first, by size old new 64,000 items 350ms 364ms 128,000 items 667ms 691ms removing half the model old new 512,000 items 24ms 15ms 1,024,000 items 49ms 25ms
2020-07-21 01:09:10 +00:00
self->positions = g_new (gpointer, self->n_items);
sortlistmodel: Use GtkSortKeys This massively speeds up sorting with expensive sort functions that it's the most worthwhile optimization of this whole branch. It's slower for simple sort functions though. It's also quite a lot slower when the model doesn't support sort keys (like GtkCustomSorter), but all the other sorters do support keys. Of course, this depends on the number of items in the model - the number of comparisons scales O(N * log N) while the overhead for key handling scales O(N). So as the log N part grows, generating keys gets more and more beneficial. Benchmarks: initial sort of a GFileInfo model with display-name keys items keys 8,000 items 715ms 50ms 64,000 items --- 554ms initial sort of a GFileInfo model with complex keys items keys 64,000 items 340ms 295ms 128,000 items 641ms 605ms removing half a GFileInfo model with display-name keys (no comparisons, just key freeing overhead of a complex sorter) items keys 512,000 items 14ms 21ms 2,048,000 items 40ms 62ms removing half a GFileInfo model with complex keys (no comparisons, just key freeing overhead of a complex sorter) items keys 512,000 items 90ms 237ms 2,048,000 items 247ms 601ms
2020-07-21 01:40:28 +00:00
gtk_sort_list_model_create_keys (self);
sortlistmodel: Split the SortItem into 2 arrays Instead of one item keeping the item + its position and sorting that list, keep the items in 1 array and put the positions into a 2nd array. This is generally slower while sorting, but allows multiple improvements: 1. We can replace items with keys This allows avoiding multiple slow lookups when using complex comparisons 2. We can keep multiple position arrays This allows doing a sorting in the background without actually emitting items-changed() until the array is completely sorted. 3. The main list tracks the items in the original model So only a single memmove() is necessary there, while the old version had to upgrade the position in every item. Benchmarks: sorting a model of simple strings old new 256,000 items 256ms 268ms 512,000 items 569ms 638ms sorting a model of file trees, directories first, by size old new 64,000 items 350ms 364ms 128,000 items 667ms 691ms removing half the model old new 512,000 items 24ms 15ms 1,024,000 items 49ms 25ms
2020-07-21 01:09:10 +00:00
for (i = 0; i < self->n_items; i++)
sortlistmodel: Use GtkSortKeys This massively speeds up sorting with expensive sort functions that it's the most worthwhile optimization of this whole branch. It's slower for simple sort functions though. It's also quite a lot slower when the model doesn't support sort keys (like GtkCustomSorter), but all the other sorters do support keys. Of course, this depends on the number of items in the model - the number of comparisons scales O(N * log N) while the overhead for key handling scales O(N). So as the log N part grows, generating keys gets more and more beneficial. Benchmarks: initial sort of a GFileInfo model with display-name keys items keys 8,000 items 715ms 50ms 64,000 items --- 554ms initial sort of a GFileInfo model with complex keys items keys 64,000 items 340ms 295ms 128,000 items 641ms 605ms removing half a GFileInfo model with display-name keys (no comparisons, just key freeing overhead of a complex sorter) items keys 512,000 items 14ms 21ms 2,048,000 items 40ms 62ms removing half a GFileInfo model with complex keys (no comparisons, just key freeing overhead of a complex sorter) items keys 512,000 items 90ms 237ms 2,048,000 items 247ms 601ms
2020-07-21 01:40:28 +00:00
self->positions[i] = key_from_pos (self, i);
GtkSortListModel: Add
2018-09-14 02:34:40 +00:00
}
sortlistmodel: Split the SortItem into 2 arrays Instead of one item keeping the item + its position and sorting that list, keep the items in 1 array and put the positions into a 2nd array. This is generally slower while sorting, but allows multiple improvements: 1. We can replace items with keys This allows avoiding multiple slow lookups when using complex comparisons 2. We can keep multiple position arrays This allows doing a sorting in the background without actually emitting items-changed() until the array is completely sorted. 3. The main list tracks the items in the original model So only a single memmove() is necessary there, while the old version had to upgrade the position in every item. Benchmarks: sorting a model of simple strings old new 256,000 items 256ms 268ms 512,000 items 569ms 638ms sorting a model of file trees, directories first, by size old new 64,000 items 350ms 364ms 128,000 items 667ms 691ms removing half the model old new 512,000 items 24ms 15ms 1,024,000 items 49ms 25ms
2020-07-21 01:09:10 +00:00
/* This realloc()s the arrays but does not set the added values. */
sortlistmodel: Track item positions The model now tracks the original positions on top of just the items so that it can remove items in an items-changed emission. It now takes twice as much memory but removes items much faster. Benchmarks: Removing 50% of a model: before after 250,000 items 135ms 10ms 500,000 items 300ms 25ms Removing 1 item: 4,000 items 2.2ms 0ms 8,000 items 4.6ms 0ms 500,000 items --- 0.01ms
2020-07-17 00:28:42 +00:00
static void
sortlistmodel: Add an incremental property Also refactor a large part of the sortmodel to make this convenient. A large amount of time has been spent on getting items-changed regions minimized.
2020-07-21 00:50:45 +00:00
gtk_sort_list_model_update_items (GtkSortListModel *self,
gsize runs[GTK_TIM_SORT_MAX_PENDING + 1],
sortlistmodel: Track item positions The model now tracks the original positions on top of just the items so that it can remove items in an items-changed emission. It now takes twice as much memory but removes items much faster. Benchmarks: Removing 50% of a model: before after 250,000 items 135ms 10ms 500,000 items 300ms 25ms Removing 1 item: 4,000 items 2.2ms 0ms 8,000 items 4.6ms 0ms 500,000 items --- 0.01ms
2020-07-17 00:28:42 +00:00
guint position,
guint removed,
guint added,
guint *unmodified_start,
guint *unmodified_end)
{
guint i, n_items, valid;
sortlistmodel: Properly compute runs When updating a (partially) sorted model, take the known runs for the existing sort and apply them to the new sort. That way, we don't have to check the whole model again. Benchmarks: appending half the items to a model of strings old new 512,000 items 437ms 389ms 1,024,000 items 1006ms 914ms appending 10% of the items to a model of strings old new 512,000 items 206ms 132ms 1,024,000 items 438ms 301ms appending 1 item to a model of strings old new 64,000 items 1.8ms 0.00ms 512,000 items --- 0.01ms
2020-07-21 02:50:05 +00:00
guint run_index, valid_run, valid_run_end, run_end;
sortlistmodel: Track item positions The model now tracks the original positions on top of just the items so that it can remove items in an items-changed emission. It now takes twice as much memory but removes items much faster. Benchmarks: Removing 50% of a model: before after 250,000 items 135ms 10ms 500,000 items 300ms 25ms Removing 1 item: 4,000 items 2.2ms 0ms 8,000 items 4.6ms 0ms 500,000 items --- 0.01ms
2020-07-17 00:28:42 +00:00
guint start, end;
sortlistmodel: Split the SortItem into 2 arrays Instead of one item keeping the item + its position and sorting that list, keep the items in 1 array and put the positions into a 2nd array. This is generally slower while sorting, but allows multiple improvements: 1. We can replace items with keys This allows avoiding multiple slow lookups when using complex comparisons 2. We can keep multiple position arrays This allows doing a sorting in the background without actually emitting items-changed() until the array is completely sorted. 3. The main list tracks the items in the original model So only a single memmove() is necessary there, while the old version had to upgrade the position in every item. Benchmarks: sorting a model of simple strings old new 256,000 items 256ms 268ms 512,000 items 569ms 638ms sorting a model of file trees, directories first, by size old new 64,000 items 350ms 364ms 128,000 items 667ms 691ms removing half the model old new 512,000 items 24ms 15ms 1,024,000 items 49ms 25ms
2020-07-21 01:09:10 +00:00
gpointer *old_keys;
sortlistmodel: Track item positions The model now tracks the original positions on top of just the items so that it can remove items in an items-changed emission. It now takes twice as much memory but removes items much faster. Benchmarks: Removing 50% of a model: before after 250,000 items 135ms 10ms 500,000 items 300ms 25ms Removing 1 item: 4,000 items 2.2ms 0ms 8,000 items 4.6ms 0ms 500,000 items --- 0.01ms
2020-07-17 00:28:42 +00:00
sortlistmodel: Split the SortItem into 2 arrays Instead of one item keeping the item + its position and sorting that list, keep the items in 1 array and put the positions into a 2nd array. This is generally slower while sorting, but allows multiple improvements: 1. We can replace items with keys This allows avoiding multiple slow lookups when using complex comparisons 2. We can keep multiple position arrays This allows doing a sorting in the background without actually emitting items-changed() until the array is completely sorted. 3. The main list tracks the items in the original model So only a single memmove() is necessary there, while the old version had to upgrade the position in every item. Benchmarks: sorting a model of simple strings old new 256,000 items 256ms 268ms 512,000 items 569ms 638ms sorting a model of file trees, directories first, by size old new 64,000 items 350ms 364ms 128,000 items 667ms 691ms removing half the model old new 512,000 items 24ms 15ms 1,024,000 items 49ms 25ms
2020-07-21 01:09:10 +00:00
n_items = self->n_items;
sortlistmodel: Track item positions The model now tracks the original positions on top of just the items so that it can remove items in an items-changed emission. It now takes twice as much memory but removes items much faster. Benchmarks: Removing 50% of a model: before after 250,000 items 135ms 10ms 500,000 items 300ms 25ms Removing 1 item: 4,000 items 2.2ms 0ms 8,000 items 4.6ms 0ms 500,000 items --- 0.01ms
2020-07-17 00:28:42 +00:00
start = n_items;
end = n_items;
sortlistmodel: Split the SortItem into 2 arrays Instead of one item keeping the item + its position and sorting that list, keep the items in 1 array and put the positions into a 2nd array. This is generally slower while sorting, but allows multiple improvements: 1. We can replace items with keys This allows avoiding multiple slow lookups when using complex comparisons 2. We can keep multiple position arrays This allows doing a sorting in the background without actually emitting items-changed() until the array is completely sorted. 3. The main list tracks the items in the original model So only a single memmove() is necessary there, while the old version had to upgrade the position in every item. Benchmarks: sorting a model of simple strings old new 256,000 items 256ms 268ms 512,000 items 569ms 638ms sorting a model of file trees, directories first, by size old new 64,000 items 350ms 364ms 128,000 items 667ms 691ms removing half the model old new 512,000 items 24ms 15ms 1,024,000 items 49ms 25ms
2020-07-21 01:09:10 +00:00
/* first, move the keys over */
old_keys = self->keys;
sortlistmodel: Make key generation part of the step function SSave the missing keys as a bitset and iterate over that bitset in the step function. Solves the problem with a large UI block at the beginning of a sort operation when all the keys were generated, in particular when key generation was slow. Benchmarks for maximum time taken by a single main loop callback: initial sort with complex GFileInfo keys old new 32,000 items 137ms 3ms 128,000 items 520ms 31ms initial sort with string keys old new 32,000 items 187ms 1ms 128,000 items 804ms 3ms
2020-07-21 23:43:59 +00:00
gtk_sort_list_model_clear_sort_keys (self, position, removed);
sortlistmodel: Use GtkSortKeys This massively speeds up sorting with expensive sort functions that it's the most worthwhile optimization of this whole branch. It's slower for simple sort functions though. It's also quite a lot slower when the model doesn't support sort keys (like GtkCustomSorter), but all the other sorters do support keys. Of course, this depends on the number of items in the model - the number of comparisons scales O(N * log N) while the overhead for key handling scales O(N). So as the log N part grows, generating keys gets more and more beneficial. Benchmarks: initial sort of a GFileInfo model with display-name keys items keys 8,000 items 715ms 50ms 64,000 items --- 554ms initial sort of a GFileInfo model with complex keys items keys 64,000 items 340ms 295ms 128,000 items 641ms 605ms removing half a GFileInfo model with display-name keys (no comparisons, just key freeing overhead of a complex sorter) items keys 512,000 items 14ms 21ms 2,048,000 items 40ms 62ms removing half a GFileInfo model with complex keys (no comparisons, just key freeing overhead of a complex sorter) items keys 512,000 items 90ms 237ms 2,048,000 items 247ms 601ms
2020-07-21 01:40:28 +00:00
sortlistmodel: Split the SortItem into 2 arrays Instead of one item keeping the item + its position and sorting that list, keep the items in 1 array and put the positions into a 2nd array. This is generally slower while sorting, but allows multiple improvements: 1. We can replace items with keys This allows avoiding multiple slow lookups when using complex comparisons 2. We can keep multiple position arrays This allows doing a sorting in the background without actually emitting items-changed() until the array is completely sorted. 3. The main list tracks the items in the original model So only a single memmove() is necessary there, while the old version had to upgrade the position in every item. Benchmarks: sorting a model of simple strings old new 256,000 items 256ms 268ms 512,000 items 569ms 638ms sorting a model of file trees, directories first, by size old new 64,000 items 350ms 364ms 128,000 items 667ms 691ms removing half the model old new 512,000 items 24ms 15ms 1,024,000 items 49ms 25ms
2020-07-21 01:09:10 +00:00
if (removed > added)
{
sortlistmodel: Use GtkSortKeys This massively speeds up sorting with expensive sort functions that it's the most worthwhile optimization of this whole branch. It's slower for simple sort functions though. It's also quite a lot slower when the model doesn't support sort keys (like GtkCustomSorter), but all the other sorters do support keys. Of course, this depends on the number of items in the model - the number of comparisons scales O(N * log N) while the overhead for key handling scales O(N). So as the log N part grows, generating keys gets more and more beneficial. Benchmarks: initial sort of a GFileInfo model with display-name keys items keys 8,000 items 715ms 50ms 64,000 items --- 554ms initial sort of a GFileInfo model with complex keys items keys 64,000 items 340ms 295ms 128,000 items 641ms 605ms removing half a GFileInfo model with display-name keys (no comparisons, just key freeing overhead of a complex sorter) items keys 512,000 items 14ms 21ms 2,048,000 items 40ms 62ms removing half a GFileInfo model with complex keys (no comparisons, just key freeing overhead of a complex sorter) items keys 512,000 items 90ms 237ms 2,048,000 items 247ms 601ms
2020-07-21 01:40:28 +00:00
memmove (key_from_pos (self, position + added),
key_from_pos (self, position + removed),
self->key_size * (n_items - position - removed));
self->keys = g_realloc_n (self->keys, n_items - removed + added, self->key_size);
sortlistmodel: Split the SortItem into 2 arrays Instead of one item keeping the item + its position and sorting that list, keep the items in 1 array and put the positions into a 2nd array. This is generally slower while sorting, but allows multiple improvements: 1. We can replace items with keys This allows avoiding multiple slow lookups when using complex comparisons 2. We can keep multiple position arrays This allows doing a sorting in the background without actually emitting items-changed() until the array is completely sorted. 3. The main list tracks the items in the original model So only a single memmove() is necessary there, while the old version had to upgrade the position in every item. Benchmarks: sorting a model of simple strings old new 256,000 items 256ms 268ms 512,000 items 569ms 638ms sorting a model of file trees, directories first, by size old new 64,000 items 350ms 364ms 128,000 items 667ms 691ms removing half the model old new 512,000 items 24ms 15ms 1,024,000 items 49ms 25ms
2020-07-21 01:09:10 +00:00
}
else if (removed < added)
{
sortlistmodel: Use GtkSortKeys This massively speeds up sorting with expensive sort functions that it's the most worthwhile optimization of this whole branch. It's slower for simple sort functions though. It's also quite a lot slower when the model doesn't support sort keys (like GtkCustomSorter), but all the other sorters do support keys. Of course, this depends on the number of items in the model - the number of comparisons scales O(N * log N) while the overhead for key handling scales O(N). So as the log N part grows, generating keys gets more and more beneficial. Benchmarks: initial sort of a GFileInfo model with display-name keys items keys 8,000 items 715ms 50ms 64,000 items --- 554ms initial sort of a GFileInfo model with complex keys items keys 64,000 items 340ms 295ms 128,000 items 641ms 605ms removing half a GFileInfo model with display-name keys (no comparisons, just key freeing overhead of a complex sorter) items keys 512,000 items 14ms 21ms 2,048,000 items 40ms 62ms removing half a GFileInfo model with complex keys (no comparisons, just key freeing overhead of a complex sorter) items keys 512,000 items 90ms 237ms 2,048,000 items 247ms 601ms
2020-07-21 01:40:28 +00:00
self->keys = g_realloc_n (self->keys, n_items - removed + added, self->key_size);
memmove (key_from_pos (self, position + added),
key_from_pos (self, position + removed),
self->key_size * (n_items - position - removed));
sortlistmodel: Split the SortItem into 2 arrays Instead of one item keeping the item + its position and sorting that list, keep the items in 1 array and put the positions into a 2nd array. This is generally slower while sorting, but allows multiple improvements: 1. We can replace items with keys This allows avoiding multiple slow lookups when using complex comparisons 2. We can keep multiple position arrays This allows doing a sorting in the background without actually emitting items-changed() until the array is completely sorted. 3. The main list tracks the items in the original model So only a single memmove() is necessary there, while the old version had to upgrade the position in every item. Benchmarks: sorting a model of simple strings old new 256,000 items 256ms 268ms 512,000 items 569ms 638ms sorting a model of file trees, directories first, by size old new 64,000 items 350ms 364ms 128,000 items 667ms 691ms removing half the model old new 512,000 items 24ms 15ms 1,024,000 items 49ms 25ms
2020-07-21 01:09:10 +00:00
}
/* then, update the positions */
sortlistmodel: Track item positions The model now tracks the original positions on top of just the items so that it can remove items in an items-changed emission. It now takes twice as much memory but removes items much faster. Benchmarks: Removing 50% of a model: before after 250,000 items 135ms 10ms 500,000 items 300ms 25ms Removing 1 item: 4,000 items 2.2ms 0ms 8,000 items 4.6ms 0ms 500,000 items --- 0.01ms
2020-07-17 00:28:42 +00:00
valid = 0;
sortlistmodel: Properly compute runs When updating a (partially) sorted model, take the known runs for the existing sort and apply them to the new sort. That way, we don't have to check the whole model again. Benchmarks: appending half the items to a model of strings old new 512,000 items 437ms 389ms 1,024,000 items 1006ms 914ms appending 10% of the items to a model of strings old new 512,000 items 206ms 132ms 1,024,000 items 438ms 301ms appending 1 item to a model of strings old new 64,000 items 1.8ms 0.00ms 512,000 items --- 0.01ms
2020-07-21 02:50:05 +00:00
valid_run = 0;
valid_run_end = 0;
run_index = 0;
run_end = 0;
for (i = 0; i < n_items;)
sortlistmodel: Track item positions The model now tracks the original positions on top of just the items so that it can remove items in an items-changed emission. It now takes twice as much memory but removes items much faster. Benchmarks: Removing 50% of a model: before after 250,000 items 135ms 10ms 500,000 items 300ms 25ms Removing 1 item: 4,000 items 2.2ms 0ms 8,000 items 4.6ms 0ms 500,000 items --- 0.01ms
2020-07-17 00:28:42 +00:00
{
sortlistmodel: Properly compute runs When updating a (partially) sorted model, take the known runs for the existing sort and apply them to the new sort. That way, we don't have to check the whole model again. Benchmarks: appending half the items to a model of strings old new 512,000 items 437ms 389ms 1,024,000 items 1006ms 914ms appending 10% of the items to a model of strings old new 512,000 items 206ms 132ms 1,024,000 items 438ms 301ms appending 1 item to a model of strings old new 64,000 items 1.8ms 0.00ms 512,000 items --- 0.01ms
2020-07-21 02:50:05 +00:00
if (runs[run_index] == 0)
{
run_end = n_items;
valid_run_end = G_MAXUINT;
}
else
{
run_end += runs[run_index++];
sortlistmodel: Track item positions The model now tracks the original positions on top of just the items so that it can remove items in an items-changed emission. It now takes twice as much memory but removes items much faster. Benchmarks: Removing 50% of a model: before after 250,000 items 135ms 10ms 500,000 items 300ms 25ms Removing 1 item: 4,000 items 2.2ms 0ms 8,000 items 4.6ms 0ms 500,000 items --- 0.01ms
2020-07-17 00:28:42 +00:00
}
sortlistmodel: Split the SortItem into 2 arrays Instead of one item keeping the item + its position and sorting that list, keep the items in 1 array and put the positions into a 2nd array. This is generally slower while sorting, but allows multiple improvements: 1. We can replace items with keys This allows avoiding multiple slow lookups when using complex comparisons 2. We can keep multiple position arrays This allows doing a sorting in the background without actually emitting items-changed() until the array is completely sorted. 3. The main list tracks the items in the original model So only a single memmove() is necessary there, while the old version had to upgrade the position in every item. Benchmarks: sorting a model of simple strings old new 256,000 items 256ms 268ms 512,000 items 569ms 638ms sorting a model of file trees, directories first, by size old new 64,000 items 350ms 364ms 128,000 items 667ms 691ms removing half the model old new 512,000 items 24ms 15ms 1,024,000 items 49ms 25ms
2020-07-21 01:09:10 +00:00
sortlistmodel: Properly compute runs When updating a (partially) sorted model, take the known runs for the existing sort and apply them to the new sort. That way, we don't have to check the whole model again. Benchmarks: appending half the items to a model of strings old new 512,000 items 437ms 389ms 1,024,000 items 1006ms 914ms appending 10% of the items to a model of strings old new 512,000 items 206ms 132ms 1,024,000 items 438ms 301ms appending 1 item to a model of strings old new 64,000 items 1.8ms 0.00ms 512,000 items --- 0.01ms
2020-07-21 02:50:05 +00:00
for (; i < run_end; i++)
{
guint pos = ((char *) self->positions[i] - (char *) old_keys) / self->key_size;
if (pos >= position + removed)
pos = pos - removed + added;
else if (pos >= position)
{
start = MIN (start, valid);
end = n_items - i - 1;
continue;
}
self->positions[valid] = key_from_pos (self, pos);
valid++;
}
sortlistmodel: Add an incremental property Also refactor a large part of the sortmodel to make this convenient. A large amount of time has been spent on getting items-changed regions minimized.
2020-07-21 00:50:45 +00:00
sortlistmodel: Properly compute runs When updating a (partially) sorted model, take the known runs for the existing sort and apply them to the new sort. That way, we don't have to check the whole model again. Benchmarks: appending half the items to a model of strings old new 512,000 items 437ms 389ms 1,024,000 items 1006ms 914ms appending 10% of the items to a model of strings old new 512,000 items 206ms 132ms 1,024,000 items 438ms 301ms appending 1 item to a model of strings old new 64,000 items 1.8ms 0.00ms 512,000 items --- 0.01ms
2020-07-21 02:50:05 +00:00
if (valid_run_end < valid)
{
runs[valid_run++] = valid - valid_run_end;
valid_run_end = valid;
}
}
g_assert (i == n_items);
sortlistmodel: Track item positions The model now tracks the original positions on top of just the items so that it can remove items in an items-changed emission. It now takes twice as much memory but removes items much faster. Benchmarks: Removing 50% of a model: before after 250,000 items 135ms 10ms 500,000 items 300ms 25ms Removing 1 item: 4,000 items 2.2ms 0ms 8,000 items 4.6ms 0ms 500,000 items --- 0.01ms
2020-07-17 00:28:42 +00:00
g_assert (valid == n_items - removed);
sortlistmodel: Properly compute runs When updating a (partially) sorted model, take the known runs for the existing sort and apply them to the new sort. That way, we don't have to check the whole model again. Benchmarks: appending half the items to a model of strings old new 512,000 items 437ms 389ms 1,024,000 items 1006ms 914ms appending 10% of the items to a model of strings old new 512,000 items 206ms 132ms 1,024,000 items 438ms 301ms appending 1 item to a model of strings old new 64,000 items 1.8ms 0.00ms 512,000 items --- 0.01ms
2020-07-21 02:50:05 +00:00
runs[valid_run] = 0;
self->positions = g_renew (gpointer, self->positions, n_items - removed + added);
sortlistmodel: Split the SortItem into 2 arrays Instead of one item keeping the item + its position and sorting that list, keep the items in 1 array and put the positions into a 2nd array. This is generally slower while sorting, but allows multiple improvements: 1. We can replace items with keys This allows avoiding multiple slow lookups when using complex comparisons 2. We can keep multiple position arrays This allows doing a sorting in the background without actually emitting items-changed() until the array is completely sorted. 3. The main list tracks the items in the original model So only a single memmove() is necessary there, while the old version had to upgrade the position in every item. Benchmarks: sorting a model of simple strings old new 256,000 items 256ms 268ms 512,000 items 569ms 638ms sorting a model of file trees, directories first, by size old new 64,000 items 350ms 364ms 128,000 items 667ms 691ms removing half the model old new 512,000 items 24ms 15ms 1,024,000 items 49ms 25ms
2020-07-21 01:09:10 +00:00
sortlistmodel: Make key generation part of the step function SSave the missing keys as a bitset and iterate over that bitset in the step function. Solves the problem with a large UI block at the beginning of a sort operation when all the keys were generated, in particular when key generation was slow. Benchmarks for maximum time taken by a single main loop callback: initial sort with complex GFileInfo keys old new 32,000 items 137ms 3ms 128,000 items 520ms 31ms initial sort with string keys old new 32,000 items 187ms 1ms 128,000 items 804ms 3ms
2020-07-21 23:43:59 +00:00
if (self->missing_keys)
{
gtk_bitset_splice (self->missing_keys, position, removed, added);
gtk_bitset_add_range (self->missing_keys, position, added);
}
sortlistmodel: Split the SortItem into 2 arrays Instead of one item keeping the item + its position and sorting that list, keep the items in 1 array and put the positions into a 2nd array. This is generally slower while sorting, but allows multiple improvements: 1. We can replace items with keys This allows avoiding multiple slow lookups when using complex comparisons 2. We can keep multiple position arrays This allows doing a sorting in the background without actually emitting items-changed() until the array is completely sorted. 3. The main list tracks the items in the original model So only a single memmove() is necessary there, while the old version had to upgrade the position in every item. Benchmarks: sorting a model of simple strings old new 256,000 items 256ms 268ms 512,000 items 569ms 638ms sorting a model of file trees, directories first, by size old new 64,000 items 350ms 364ms 128,000 items 667ms 691ms removing half the model old new 512,000 items 24ms 15ms 1,024,000 items 49ms 25ms
2020-07-21 01:09:10 +00:00
self->n_items = n_items - removed + added;
for (i = 0; i < added; i++)
{
self->positions[self->n_items - added + i] = key_from_pos (self, position + i);
}
sortlistmodel: Track item positions The model now tracks the original positions on top of just the items so that it can remove items in an items-changed emission. It now takes twice as much memory but removes items much faster. Benchmarks: Removing 50% of a model: before after 250,000 items 135ms 10ms 500,000 items 300ms 25ms Removing 1 item: 4,000 items 2.2ms 0ms 8,000 items 4.6ms 0ms 500,000 items --- 0.01ms
2020-07-17 00:28:42 +00:00
*unmodified_start = start;
*unmodified_end = end;
}
GtkSortListModel: Add
2018-09-14 02:34:40 +00:00
static void
gtk_sort_list_model_items_changed_cb (GListModel *model,
guint position,
guint removed,
guint added,
GtkSortListModel *self)
{
sortlistmodel: Add an incremental property Also refactor a large part of the sortmodel to make this convenient. A large amount of time has been spent on getting items-changed regions minimized.
2020-07-21 00:50:45 +00:00
gsize runs[GTK_TIM_SORT_MAX_PENDING + 1];
sortlistmodel: Track item positions The model now tracks the original positions on top of just the items so that it can remove items in an items-changed emission. It now takes twice as much memory but removes items much faster. Benchmarks: Removing 50% of a model: before after 250,000 items 135ms 10ms 500,000 items 300ms 25ms Removing 1 item: 4,000 items 2.2ms 0ms 8,000 items 4.6ms 0ms 500,000 items --- 0.01ms
2020-07-17 00:28:42 +00:00
guint i, n_items, start, end;
sortlistmodel: Make sorting incremental This is just an experiment so far to see how long it takes to sort.
2020-07-20 23:46:09 +00:00
gboolean was_sorting;
GtkSortListModel: Add
2018-09-14 02:34:40 +00:00
sortlistmodel: Track item positions The model now tracks the original positions on top of just the items so that it can remove items in an items-changed emission. It now takes twice as much memory but removes items much faster. Benchmarks: Removing 50% of a model: before after 250,000 items 135ms 10ms 500,000 items 300ms 25ms Removing 1 item: 4,000 items 2.2ms 0ms 8,000 items 4.6ms 0ms 500,000 items --- 0.01ms
2020-07-17 00:28:42 +00:00
if (removed == 0 && added == 0)
return;
GtkSortListModel: Add
2018-09-14 02:34:40 +00:00
sortlistmodel: Track item positions The model now tracks the original positions on top of just the items so that it can remove items in an items-changed emission. It now takes twice as much memory but removes items much faster. Benchmarks: Removing 50% of a model: before after 250,000 items 135ms 10ms 500,000 items 300ms 25ms Removing 1 item: 4,000 items 2.2ms 0ms 8,000 items 4.6ms 0ms 500,000 items --- 0.01ms
2020-07-17 00:28:42 +00:00
if (!gtk_sort_list_model_should_sort (self))
{
sortlistmodel: Split the SortItem into 2 arrays Instead of one item keeping the item + its position and sorting that list, keep the items in 1 array and put the positions into a 2nd array. This is generally slower while sorting, but allows multiple improvements: 1. We can replace items with keys This allows avoiding multiple slow lookups when using complex comparisons 2. We can keep multiple position arrays This allows doing a sorting in the background without actually emitting items-changed() until the array is completely sorted. 3. The main list tracks the items in the original model So only a single memmove() is necessary there, while the old version had to upgrade the position in every item. Benchmarks: sorting a model of simple strings old new 256,000 items 256ms 268ms 512,000 items 569ms 638ms sorting a model of file trees, directories first, by size old new 64,000 items 350ms 364ms 128,000 items 667ms 691ms removing half the model old new 512,000 items 24ms 15ms 1,024,000 items 49ms 25ms
2020-07-21 01:09:10 +00:00
self->n_items = self->n_items - removed + added;
sortlistmodel: Track item positions The model now tracks the original positions on top of just the items so that it can remove items in an items-changed emission. It now takes twice as much memory but removes items much faster. Benchmarks: Removing 50% of a model: before after 250,000 items 135ms 10ms 500,000 items 300ms 25ms Removing 1 item: 4,000 items 2.2ms 0ms 8,000 items 4.6ms 0ms 500,000 items --- 0.01ms
2020-07-17 00:28:42 +00:00
g_list_model_items_changed (G_LIST_MODEL (self), position, removed, added);
return;
}
sortlistmodel: Make sorting incremental This is just an experiment so far to see how long it takes to sort.
2020-07-20 23:46:09 +00:00
was_sorting = gtk_sort_list_model_is_sorting (self);
sortlistmodel: Add an incremental property Also refactor a large part of the sortmodel to make this convenient. A large amount of time has been spent on getting items-changed regions minimized.
2020-07-21 00:50:45 +00:00
gtk_sort_list_model_stop_sorting (self, runs);
sortlistmodel: Make sorting incremental This is just an experiment so far to see how long it takes to sort.
2020-07-20 23:46:09 +00:00
sortlistmodel: Add an incremental property Also refactor a large part of the sortmodel to make this convenient. A large amount of time has been spent on getting items-changed regions minimized.
2020-07-21 00:50:45 +00:00
gtk_sort_list_model_update_items (self, runs, position, removed, added, &start, &end);
sortlistmodel: Make sorting incremental This is just an experiment so far to see how long it takes to sort.
2020-07-20 23:46:09 +00:00
sortlistmodel: Track item positions The model now tracks the original positions on top of just the items so that it can remove items in an items-changed emission. It now takes twice as much memory but removes items much faster. Benchmarks: Removing 50% of a model: before after 250,000 items 135ms 10ms 500,000 items 300ms 25ms Removing 1 item: 4,000 items 2.2ms 0ms 8,000 items 4.6ms 0ms 500,000 items --- 0.01ms
2020-07-17 00:28:42 +00:00
if (added > 0)
{
sortlistmodel: Split the SortItem into 2 arrays Instead of one item keeping the item + its position and sorting that list, keep the items in 1 array and put the positions into a 2nd array. This is generally slower while sorting, but allows multiple improvements: 1. We can replace items with keys This allows avoiding multiple slow lookups when using complex comparisons 2. We can keep multiple position arrays This allows doing a sorting in the background without actually emitting items-changed() until the array is completely sorted. 3. The main list tracks the items in the original model So only a single memmove() is necessary there, while the old version had to upgrade the position in every item. Benchmarks: sorting a model of simple strings old new 256,000 items 256ms 268ms 512,000 items 569ms 638ms sorting a model of file trees, directories first, by size old new 64,000 items 350ms 364ms 128,000 items 667ms 691ms removing half the model old new 512,000 items 24ms 15ms 1,024,000 items 49ms 25ms
2020-07-21 01:09:10 +00:00
if (gtk_sort_list_model_start_sorting (self, runs))
sortlistmodel: Track item positions The model now tracks the original positions on top of just the items so that it can remove items in an items-changed emission. It now takes twice as much memory but removes items much faster. Benchmarks: Removing 50% of a model: before after 250,000 items 135ms 10ms 500,000 items 300ms 25ms Removing 1 item: 4,000 items 2.2ms 0ms 8,000 items 4.6ms 0ms 500,000 items --- 0.01ms
2020-07-17 00:28:42 +00:00
{
sortlistmodel: Split the SortItem into 2 arrays Instead of one item keeping the item + its position and sorting that list, keep the items in 1 array and put the positions into a 2nd array. This is generally slower while sorting, but allows multiple improvements: 1. We can replace items with keys This allows avoiding multiple slow lookups when using complex comparisons 2. We can keep multiple position arrays This allows doing a sorting in the background without actually emitting items-changed() until the array is completely sorted. 3. The main list tracks the items in the original model So only a single memmove() is necessary there, while the old version had to upgrade the position in every item. Benchmarks: sorting a model of simple strings old new 256,000 items 256ms 268ms 512,000 items 569ms 638ms sorting a model of file trees, directories first, by size old new 64,000 items 350ms 364ms 128,000 items 667ms 691ms removing half the model old new 512,000 items 24ms 15ms 1,024,000 items 49ms 25ms
2020-07-21 01:09:10 +00:00
end = 0;
sortlistmodel: Track item positions The model now tracks the original positions on top of just the items so that it can remove items in an items-changed emission. It now takes twice as much memory but removes items much faster. Benchmarks: Removing 50% of a model: before after 250,000 items 135ms 10ms 500,000 items 300ms 25ms Removing 1 item: 4,000 items 2.2ms 0ms 8,000 items 4.6ms 0ms 500,000 items --- 0.01ms
2020-07-17 00:28:42 +00:00
}
sortlistmodel: Split the SortItem into 2 arrays Instead of one item keeping the item + its position and sorting that list, keep the items in 1 array and put the positions into a 2nd array. This is generally slower while sorting, but allows multiple improvements: 1. We can replace items with keys This allows avoiding multiple slow lookups when using complex comparisons 2. We can keep multiple position arrays This allows doing a sorting in the background without actually emitting items-changed() until the array is completely sorted. 3. The main list tracks the items in the original model So only a single memmove() is necessary there, while the old version had to upgrade the position in every item. Benchmarks: sorting a model of simple strings old new 256,000 items 256ms 268ms 512,000 items 569ms 638ms sorting a model of file trees, directories first, by size old new 64,000 items 350ms 364ms 128,000 items 667ms 691ms removing half the model old new 512,000 items 24ms 15ms 1,024,000 items 49ms 25ms
2020-07-21 01:09:10 +00:00
else
sortlistmodel: Add an incremental property Also refactor a large part of the sortmodel to make this convenient. A large amount of time has been spent on getting items-changed regions minimized.
2020-07-21 00:50:45 +00:00
{
guint pos, n;
gtk_sort_list_model_finish_sorting (self, &pos, &n);
sortlistmodel: Split the SortItem into 2 arrays Instead of one item keeping the item + its position and sorting that list, keep the items in 1 array and put the positions into a 2nd array. This is generally slower while sorting, but allows multiple improvements: 1. We can replace items with keys This allows avoiding multiple slow lookups when using complex comparisons 2. We can keep multiple position arrays This allows doing a sorting in the background without actually emitting items-changed() until the array is completely sorted. 3. The main list tracks the items in the original model So only a single memmove() is necessary there, while the old version had to upgrade the position in every item. Benchmarks: sorting a model of simple strings old new 256,000 items 256ms 268ms 512,000 items 569ms 638ms sorting a model of file trees, directories first, by size old new 64,000 items 350ms 364ms 128,000 items 667ms 691ms removing half the model old new 512,000 items 24ms 15ms 1,024,000 items 49ms 25ms
2020-07-21 01:09:10 +00:00
if (n)
start = MIN (start, pos);
/* find first item that was added */
for (i = 0; i < end; i++)
{
pos = pos_from_key (self, self->positions[self->n_items - i - 1]);
if (pos >= position && pos < position + added)
{
end = i;
break;
}
}
sortlistmodel: Add an incremental property Also refactor a large part of the sortmodel to make this convenient. A large amount of time has been spent on getting items-changed regions minimized.
2020-07-21 00:50:45 +00:00
}
sortlistmodel: Make sorting incremental This is just an experiment so far to see how long it takes to sort.
2020-07-20 23:46:09 +00:00
}
else
{
if (was_sorting)
sortlistmodel: Add an incremental property Also refactor a large part of the sortmodel to make this convenient. A large amount of time has been spent on getting items-changed regions minimized.
2020-07-21 00:50:45 +00:00
gtk_sort_list_model_start_sorting (self, runs);
sortlistmodel: Track item positions The model now tracks the original positions on top of just the items so that it can remove items in an items-changed emission. It now takes twice as much memory but removes items much faster. Benchmarks: Removing 50% of a model: before after 250,000 items 135ms 10ms 500,000 items 300ms 25ms Removing 1 item: 4,000 items 2.2ms 0ms 8,000 items 4.6ms 0ms 500,000 items --- 0.01ms
2020-07-17 00:28:42 +00:00
}
sortlistmodel: Split the SortItem into 2 arrays Instead of one item keeping the item + its position and sorting that list, keep the items in 1 array and put the positions into a 2nd array. This is generally slower while sorting, but allows multiple improvements: 1. We can replace items with keys This allows avoiding multiple slow lookups when using complex comparisons 2. We can keep multiple position arrays This allows doing a sorting in the background without actually emitting items-changed() until the array is completely sorted. 3. The main list tracks the items in the original model So only a single memmove() is necessary there, while the old version had to upgrade the position in every item. Benchmarks: sorting a model of simple strings old new 256,000 items 256ms 268ms 512,000 items 569ms 638ms sorting a model of file trees, directories first, by size old new 64,000 items 350ms 364ms 128,000 items 667ms 691ms removing half the model old new 512,000 items 24ms 15ms 1,024,000 items 49ms 25ms
2020-07-21 01:09:10 +00:00
n_items = self->n_items - start - end;
sortlistmodel: Track item positions The model now tracks the original positions on top of just the items so that it can remove items in an items-changed emission. It now takes twice as much memory but removes items much faster. Benchmarks: Removing 50% of a model: before after 250,000 items 135ms 10ms 500,000 items 300ms 25ms Removing 1 item: 4,000 items 2.2ms 0ms 8,000 items 4.6ms 0ms 500,000 items --- 0.01ms
2020-07-17 00:28:42 +00:00
g_list_model_items_changed (G_LIST_MODEL (self), start, n_items - added + removed, n_items);
GtkSortListModel: Add
2018-09-14 02:34:40 +00:00
}
static void
gtk_sort_list_model_set_property (GObject *object,
guint prop_id,
const GValue *value,
GParamSpec *pspec)
{
GtkSortListModel *self = GTK_SORT_LIST_MODEL (object);
switch (prop_id)
{
sortlistmodel: Add an incremental property Also refactor a large part of the sortmodel to make this convenient. A large amount of time has been spent on getting items-changed regions minimized.
2020-07-21 00:50:45 +00:00
case PROP_INCREMENTAL:
gtk_sort_list_model_set_incremental (self, g_value_get_boolean (value));
break;
GtkSortListModel: Add
2018-09-14 02:34:40 +00:00
case PROP_MODEL:
gtk_sort_list_model_set_model (self, g_value_get_object (value));
break;
Redo sort list model with GtkSorter Reshuffle the api to take full advantage of GtkSorter. Update all callers.
2019-12-03 04:44:22 +00:00
case PROP_SORTER:
gtk_sort_list_model_set_sorter (self, g_value_get_object (value));
break;
GtkSortListModel: Add
2018-09-14 02:34:40 +00:00
default:
G_OBJECT_WARN_INVALID_PROPERTY_ID (object, prop_id, pspec);
break;
}
}
static void
gtk_sort_list_model_get_property (GObject *object,
guint prop_id,
GValue *value,
GParamSpec *pspec)
{
GtkSortListModel *self = GTK_SORT_LIST_MODEL (object);
switch (prop_id)
{
sortlistmodel: Add an incremental property Also refactor a large part of the sortmodel to make this convenient. A large amount of time has been spent on getting items-changed regions minimized.
2020-07-21 00:50:45 +00:00
case PROP_INCREMENTAL:
g_value_set_boolean (value, self->incremental);
break;
GtkSortListModel: Add
2018-09-14 02:34:40 +00:00
case PROP_MODEL:
g_value_set_object (value, self->model);
break;
sortlistmodel: Add progress estimation
2020-07-22 00:50:58 +00:00
case PROP_PENDING:
g_value_set_uint (value, gtk_sort_list_model_get_pending (self));
break;
Redo sort list model with GtkSorter Reshuffle the api to take full advantage of GtkSorter. Update all callers.
2019-12-03 04:44:22 +00:00
case PROP_SORTER:
g_value_set_object (value, self->sorter);
break;
GtkSortListModel: Add
2018-09-14 02:34:40 +00:00
default:
G_OBJECT_WARN_INVALID_PROPERTY_ID (object, prop_id, pspec);
break;
}
}
Redo sort list model with GtkSorter Reshuffle the api to take full advantage of GtkSorter. Update all callers.
2019-12-03 04:44:22 +00:00
static void
gtk_sort_list_model_sorter_changed_cb (GtkSorter *sorter,
int change,
GtkSortListModel *self)
{
sortlistmodel: Add an incremental property Also refactor a large part of the sortmodel to make this convenient. A large amount of time has been spent on getting items-changed regions minimized.
2020-07-21 00:50:45 +00:00
guint pos, n_items;
sortlistmodel: Replace with an array-based model This is the dumbest possible sortmodel using an array: Just grab all the items, put them in the array, qsort() the array. Some benchmarks (setting a new model): 125,000 items - old: 549ms new: 115ms 250,000 items - new: 250ms This performance can not be kept for simple additions and removals though.
2020-07-16 23:56:18 +00:00
sortlistmodel: Split the SortItem into 2 arrays Instead of one item keeping the item + its position and sorting that list, keep the items in 1 array and put the positions into a 2nd array. This is generally slower while sorting, but allows multiple improvements: 1. We can replace items with keys This allows avoiding multiple slow lookups when using complex comparisons 2. We can keep multiple position arrays This allows doing a sorting in the background without actually emitting items-changed() until the array is completely sorted. 3. The main list tracks the items in the original model So only a single memmove() is necessary there, while the old version had to upgrade the position in every item. Benchmarks: sorting a model of simple strings old new 256,000 items 256ms 268ms 512,000 items 569ms 638ms sorting a model of file trees, directories first, by size old new 64,000 items 350ms 364ms 128,000 items 667ms 691ms removing half the model old new 512,000 items 24ms 15ms 1,024,000 items 49ms 25ms
2020-07-21 01:09:10 +00:00
if (gtk_sort_list_model_should_sort (self))
sortlistmodel: Add an incremental property Also refactor a large part of the sortmodel to make this convenient. A large amount of time has been spent on getting items-changed regions minimized.
2020-07-21 00:50:45 +00:00
{
gtk_sort_list_model_stop_sorting (self, NULL);
sortlistmodel: Replace with an array-based model This is the dumbest possible sortmodel using an array: Just grab all the items, put them in the array, qsort() the array. Some benchmarks (setting a new model): 125,000 items - old: 549ms new: 115ms 250,000 items - new: 250ms This performance can not be kept for simple additions and removals though.
2020-07-16 23:56:18 +00:00
sortlistmodel: Use GtkSortKeys This massively speeds up sorting with expensive sort functions that it's the most worthwhile optimization of this whole branch. It's slower for simple sort functions though. It's also quite a lot slower when the model doesn't support sort keys (like GtkCustomSorter), but all the other sorters do support keys. Of course, this depends on the number of items in the model - the number of comparisons scales O(N * log N) while the overhead for key handling scales O(N). So as the log N part grows, generating keys gets more and more beneficial. Benchmarks: initial sort of a GFileInfo model with display-name keys items keys 8,000 items 715ms 50ms 64,000 items --- 554ms initial sort of a GFileInfo model with complex keys items keys 64,000 items 340ms 295ms 128,000 items 641ms 605ms removing half a GFileInfo model with display-name keys (no comparisons, just key freeing overhead of a complex sorter) items keys 512,000 items 14ms 21ms 2,048,000 items 40ms 62ms removing half a GFileInfo model with complex keys (no comparisons, just key freeing overhead of a complex sorter) items keys 512,000 items 90ms 237ms 2,048,000 items 247ms 601ms
2020-07-21 01:40:28 +00:00
if (self->sort_keys == NULL)
{
gtk_sort_list_model_create_items (self);
}
else
{
GtkSortKeys *new_keys = gtk_sorter_get_keys (sorter);
if (!gtk_sort_keys_is_compatible (new_keys, self->sort_keys))
{
char *old_keys = self->keys;
gsize old_key_size = self->key_size;
guint i;
gtk_sort_list_model_clear_keys (self);
gtk_sort_list_model_create_keys (self);
for (i = 0; i < self->n_items; i++)
self->positions[i] = key_from_pos (self, ((char *) self->positions[i] - old_keys) / old_key_size);
gtk_sort_keys_unref (new_keys);
}
else
{
gtk_sort_keys_unref (self->sort_keys);
self->sort_keys = new_keys;
}
}
sortlistmodel: Add an incremental property Also refactor a large part of the sortmodel to make this convenient. A large amount of time has been spent on getting items-changed regions minimized.
2020-07-21 00:50:45 +00:00
if (gtk_sort_list_model_start_sorting (self, NULL))
pos = n_items = 0;
else
gtk_sort_list_model_finish_sorting (self, &pos, &n_items);
}
sortlistmodel: Split the SortItem into 2 arrays Instead of one item keeping the item + its position and sorting that list, keep the items in 1 array and put the positions into a 2nd array. This is generally slower while sorting, but allows multiple improvements: 1. We can replace items with keys This allows avoiding multiple slow lookups when using complex comparisons 2. We can keep multiple position arrays This allows doing a sorting in the background without actually emitting items-changed() until the array is completely sorted. 3. The main list tracks the items in the original model So only a single memmove() is necessary there, while the old version had to upgrade the position in every item. Benchmarks: sorting a model of simple strings old new 256,000 items 256ms 268ms 512,000 items 569ms 638ms sorting a model of file trees, directories first, by size old new 64,000 items 350ms 364ms 128,000 items 667ms 691ms removing half the model old new 512,000 items 24ms 15ms 1,024,000 items 49ms 25ms
2020-07-21 01:09:10 +00:00
else
{
gtk_sort_list_model_clear_items (self, &pos, &n_items);
}
sortlistmodel: Replace with an array-based model This is the dumbest possible sortmodel using an array: Just grab all the items, put them in the array, qsort() the array. Some benchmarks (setting a new model): 125,000 items - old: 549ms new: 115ms 250,000 items - new: 250ms This performance can not be kept for simple additions and removals though.
2020-07-16 23:56:18 +00:00
sortlistmodel: Add an incremental property Also refactor a large part of the sortmodel to make this convenient. A large amount of time has been spent on getting items-changed regions minimized.
2020-07-21 00:50:45 +00:00
if (n_items > 0)
g_list_model_items_changed (G_LIST_MODEL (self), pos, n_items, n_items);
Redo sort list model with GtkSorter Reshuffle the api to take full advantage of GtkSorter. Update all callers.
2019-12-03 04:44:22 +00:00
}
GtkSortListModel: Add
2018-09-14 02:34:40 +00:00
static void
gtk_sort_list_model_clear_model (GtkSortListModel *self)
{
if (self->model == NULL)
return;
g_signal_handlers_disconnect_by_func (self->model, gtk_sort_list_model_items_changed_cb, self);
g_clear_object (&self->model);
sortlistmodel: Add an incremental property Also refactor a large part of the sortmodel to make this convenient. A large amount of time has been spent on getting items-changed regions minimized.
2020-07-21 00:50:45 +00:00
gtk_sort_list_model_clear_items (self, NULL, NULL);
sortlistmodel: Split the SortItem into 2 arrays Instead of one item keeping the item + its position and sorting that list, keep the items in 1 array and put the positions into a 2nd array. This is generally slower while sorting, but allows multiple improvements: 1. We can replace items with keys This allows avoiding multiple slow lookups when using complex comparisons 2. We can keep multiple position arrays This allows doing a sorting in the background without actually emitting items-changed() until the array is completely sorted. 3. The main list tracks the items in the original model So only a single memmove() is necessary there, while the old version had to upgrade the position in every item. Benchmarks: sorting a model of simple strings old new 256,000 items 256ms 268ms 512,000 items 569ms 638ms sorting a model of file trees, directories first, by size old new 64,000 items 350ms 364ms 128,000 items 667ms 691ms removing half the model old new 512,000 items 24ms 15ms 1,024,000 items 49ms 25ms
2020-07-21 01:09:10 +00:00
self->n_items = 0;
GtkSortListModel: Add
2018-09-14 02:34:40 +00:00
}
Redo sort list model with GtkSorter Reshuffle the api to take full advantage of GtkSorter. Update all callers.
2019-12-03 04:44:22 +00:00
static void
gtk_sort_list_model_clear_sorter (GtkSortListModel *self)
{
if (self->sorter == NULL)
return;
g_signal_handlers_disconnect_by_func (self->sorter, gtk_sort_list_model_sorter_changed_cb, self);
g_clear_object (&self->sorter);
}
GtkSortListModel: Add
2018-09-14 02:34:40 +00:00
static void
gtk_sort_list_model_dispose (GObject *object)
{
GtkSortListModel *self = GTK_SORT_LIST_MODEL (object);
gtk_sort_list_model_clear_model (self);
Redo sort list model with GtkSorter Reshuffle the api to take full advantage of GtkSorter. Update all callers.
2019-12-03 04:44:22 +00:00
gtk_sort_list_model_clear_sorter (self);
GtkSortListModel: Add
2018-09-14 02:34:40 +00:00
G_OBJECT_CLASS (gtk_sort_list_model_parent_class)->dispose (object);
};
static void
gtk_sort_list_model_class_init (GtkSortListModelClass *class)
{
GObjectClass *gobject_class = G_OBJECT_CLASS (class);
gobject_class->set_property = gtk_sort_list_model_set_property;
gobject_class->get_property = gtk_sort_list_model_get_property;
gobject_class->dispose = gtk_sort_list_model_dispose;
/**
sortlistmodel: Add an incremental property Also refactor a large part of the sortmodel to make this convenient. A large amount of time has been spent on getting items-changed regions minimized.
2020-07-21 00:50:45 +00:00
* GtkSortListModel:incremental:
GtkSortListModel: Add
2018-09-14 02:34:40 +00:00
*
sortlistmodel: Add an incremental property Also refactor a large part of the sortmodel to make this convenient. A large amount of time has been spent on getting items-changed regions minimized.
2020-07-21 00:50:45 +00:00
* If the model should sort items incrementally
GtkSortListModel: Add
2018-09-14 02:34:40 +00:00
*/
sortlistmodel: Add an incremental property Also refactor a large part of the sortmodel to make this convenient. A large amount of time has been spent on getting items-changed regions minimized.
2020-07-21 00:50:45 +00:00
properties[PROP_INCREMENTAL] =
g_param_spec_boolean ("incremental",
P_("Incremental"),
P_("Sort items incrementally"),
FALSE,
Redo sort list model with GtkSorter Reshuffle the api to take full advantage of GtkSorter. Update all callers.
2019-12-03 04:44:22 +00:00
GTK_PARAM_READWRITE | G_PARAM_EXPLICIT_NOTIFY);
GtkSortListModel: Add
2018-09-14 02:34:40 +00:00
/**
* GtkSortListModel:model:
*
* The model being sorted
*/
properties[PROP_MODEL] =
g_param_spec_object ("model",
P_("Model"),
P_("The model being sorted"),
G_TYPE_LIST_MODEL,
sortlistmodel: Remove forgotten G_PARAM_CONSTRUCT_ONLY
2020-07-07 21:18:46 +00:00
GTK_PARAM_READWRITE | G_PARAM_EXPLICIT_NOTIFY);
GtkSortListModel: Add
2018-09-14 02:34:40 +00:00
sortlistmodel: Add progress estimation
2020-07-22 00:50:58 +00:00
/**
* GtkSortListModel:pending:
*
* Estimate of unsorted items remaining
*/
properties[PROP_PENDING] =
g_param_spec_uint ("pending",
P_("Pending"),
P_("Estimate of unsorted items remaining"),
0, G_MAXUINT, 0,
GTK_PARAM_READABLE | G_PARAM_EXPLICIT_NOTIFY);
sortlistmodel: Add an incremental property Also refactor a large part of the sortmodel to make this convenient. A large amount of time has been spent on getting items-changed regions minimized.
2020-07-21 00:50:45 +00:00
/**
* GtkSortListModel:sorter:
*
* The sorter for this model
*/
properties[PROP_SORTER] =
g_param_spec_object ("sorter",
P_("Sorter"),
P_("The sorter for this model"),
GTK_TYPE_SORTER,
GTK_PARAM_READWRITE | G_PARAM_EXPLICIT_NOTIFY);
GtkSortListModel: Add
2018-09-14 02:34:40 +00:00
g_object_class_install_properties (gobject_class, NUM_PROPERTIES, properties);
}
static void
gtk_sort_list_model_init (GtkSortListModel *self)
{
}
/**
* gtk_sort_list_model_new:
listmodels: Stop respecting item-type Simplify all view model APIs and always return G_TYPE_OBJECT as the item-type for every model. It turns out nobody uses item-type anyway. So instead of adding lots of APIs, forcing people to think about it and trying to figure out how to handle filter or map models that modify item types, just having an easy life is a better approach. All the models need to be able to deal with any type of object going through anyway.
2020-07-04 19:47:48 +00:00
* @model: (allow-none): the model to sort
Redo sort list model with GtkSorter Reshuffle the api to take full advantage of GtkSorter. Update all callers.
2019-12-03 04:44:22 +00:00
* @sorter: (allow-none): the #GtkSorter to sort @model with
GtkSortListModel: Add
2018-09-14 02:34:40 +00:00
*
Redo sort list model with GtkSorter Reshuffle the api to take full advantage of GtkSorter. Update all callers.
2019-12-03 04:44:22 +00:00
* Creates a new sort list model that uses the @sorter to sort @model.
GtkSortListModel: Add
2018-09-14 02:34:40 +00:00
*
* Returns: a new #GtkSortListModel
**/
GtkSortListModel *
Redo sort list model with GtkSorter Reshuffle the api to take full advantage of GtkSorter. Update all callers.
2019-12-03 04:44:22 +00:00
gtk_sort_list_model_new (GListModel *model,
GtkSorter *sorter)
GtkSortListModel: Add
2018-09-14 02:34:40 +00:00
{
GtkSortListModel *result;
listmodels: Stop respecting item-type Simplify all view model APIs and always return G_TYPE_OBJECT as the item-type for every model. It turns out nobody uses item-type anyway. So instead of adding lots of APIs, forcing people to think about it and trying to figure out how to handle filter or map models that modify item types, just having an easy life is a better approach. All the models need to be able to deal with any type of object going through anyway.
2020-07-04 19:47:48 +00:00
g_return_val_if_fail (model == NULL || G_IS_LIST_MODEL (model), NULL);
Redo sort list model with GtkSorter Reshuffle the api to take full advantage of GtkSorter. Update all callers.
2019-12-03 04:44:22 +00:00
g_return_val_if_fail (sorter == NULL || GTK_IS_SORTER (sorter), NULL);
GtkSortListModel: Add
2018-09-14 02:34:40 +00:00
result = g_object_new (GTK_TYPE_SORT_LIST_MODEL,
"model", model,
Redo sort list model with GtkSorter Reshuffle the api to take full advantage of GtkSorter. Update all callers.
2019-12-03 04:44:22 +00:00
"sorter", sorter,
GtkSortListModel: Add
2018-09-14 02:34:40 +00:00
NULL);
return result;
}
/**
* gtk_sort_list_model_set_model:
* @self: a #GtkSortListModel
* @model: (allow-none): The model to be sorted
*
* Sets the model to be sorted. The @model's item type must conform to
* the item type of @self.
**/
void
gtk_sort_list_model_set_model (GtkSortListModel *self,
GListModel *model)
{
sortlistmodel: Split the SortItem into 2 arrays Instead of one item keeping the item + its position and sorting that list, keep the items in 1 array and put the positions into a 2nd array. This is generally slower while sorting, but allows multiple improvements: 1. We can replace items with keys This allows avoiding multiple slow lookups when using complex comparisons 2. We can keep multiple position arrays This allows doing a sorting in the background without actually emitting items-changed() until the array is completely sorted. 3. The main list tracks the items in the original model So only a single memmove() is necessary there, while the old version had to upgrade the position in every item. Benchmarks: sorting a model of simple strings old new 256,000 items 256ms 268ms 512,000 items 569ms 638ms sorting a model of file trees, directories first, by size old new 64,000 items 350ms 364ms 128,000 items 667ms 691ms removing half the model old new 512,000 items 24ms 15ms 1,024,000 items 49ms 25ms
2020-07-21 01:09:10 +00:00
guint removed;
GtkSortListModel: Add
2018-09-14 02:34:40 +00:00
g_return_if_fail (GTK_IS_SORT_LIST_MODEL (self));
g_return_if_fail (model == NULL || G_IS_LIST_MODEL (model));
if (self->model == model)
return;
removed = g_list_model_get_n_items (G_LIST_MODEL (self));
gtk_sort_list_model_clear_model (self);
if (model)
{
sortlistmodel: Add an incremental property Also refactor a large part of the sortmodel to make this convenient. A large amount of time has been spent on getting items-changed regions minimized.
2020-07-21 00:50:45 +00:00
guint ignore1, ignore2;
GtkSortListModel: Add
2018-09-14 02:34:40 +00:00
self->model = g_object_ref (model);
sortlistmodel: Split the SortItem into 2 arrays Instead of one item keeping the item + its position and sorting that list, keep the items in 1 array and put the positions into a 2nd array. This is generally slower while sorting, but allows multiple improvements: 1. We can replace items with keys This allows avoiding multiple slow lookups when using complex comparisons 2. We can keep multiple position arrays This allows doing a sorting in the background without actually emitting items-changed() until the array is completely sorted. 3. The main list tracks the items in the original model So only a single memmove() is necessary there, while the old version had to upgrade the position in every item. Benchmarks: sorting a model of simple strings old new 256,000 items 256ms 268ms 512,000 items 569ms 638ms sorting a model of file trees, directories first, by size old new 64,000 items 350ms 364ms 128,000 items 667ms 691ms removing half the model old new 512,000 items 24ms 15ms 1,024,000 items 49ms 25ms
2020-07-21 01:09:10 +00:00
self->n_items = g_list_model_get_n_items (model);
GtkSortListModel: Add
2018-09-14 02:34:40 +00:00
g_signal_connect (model, "items-changed", G_CALLBACK (gtk_sort_list_model_items_changed_cb), self);
sortlistmodel: Split the SortItem into 2 arrays Instead of one item keeping the item + its position and sorting that list, keep the items in 1 array and put the positions into a 2nd array. This is generally slower while sorting, but allows multiple improvements: 1. We can replace items with keys This allows avoiding multiple slow lookups when using complex comparisons 2. We can keep multiple position arrays This allows doing a sorting in the background without actually emitting items-changed() until the array is completely sorted. 3. The main list tracks the items in the original model So only a single memmove() is necessary there, while the old version had to upgrade the position in every item. Benchmarks: sorting a model of simple strings old new 256,000 items 256ms 268ms 512,000 items 569ms 638ms sorting a model of file trees, directories first, by size old new 64,000 items 350ms 364ms 128,000 items 667ms 691ms removing half the model old new 512,000 items 24ms 15ms 1,024,000 items 49ms 25ms
2020-07-21 01:09:10 +00:00
if (gtk_sort_list_model_should_sort (self))
{
gtk_sort_list_model_create_items (self);
if (!gtk_sort_list_model_start_sorting (self, NULL))
gtk_sort_list_model_finish_sorting (self, &ignore1, &ignore2);
}
GtkSortListModel: Add
2018-09-14 02:34:40 +00:00
}
sortlistmodel: Split the SortItem into 2 arrays Instead of one item keeping the item + its position and sorting that list, keep the items in 1 array and put the positions into a 2nd array. This is generally slower while sorting, but allows multiple improvements: 1. We can replace items with keys This allows avoiding multiple slow lookups when using complex comparisons 2. We can keep multiple position arrays This allows doing a sorting in the background without actually emitting items-changed() until the array is completely sorted. 3. The main list tracks the items in the original model So only a single memmove() is necessary there, while the old version had to upgrade the position in every item. Benchmarks: sorting a model of simple strings old new 256,000 items 256ms 268ms 512,000 items 569ms 638ms sorting a model of file trees, directories first, by size old new 64,000 items 350ms 364ms 128,000 items 667ms 691ms removing half the model old new 512,000 items 24ms 15ms 1,024,000 items 49ms 25ms
2020-07-21 01:09:10 +00:00
if (removed > 0 || self->n_items > 0)
g_list_model_items_changed (G_LIST_MODEL (self), 0, removed, self->n_items);
GtkSortListModel: Add
2018-09-14 02:34:40 +00:00
g_object_notify_by_pspec (G_OBJECT (self), properties[PROP_MODEL]);
}
/**
* gtk_sort_list_model_get_model:
* @self: a #GtkSortListModel
*
* Gets the model currently sorted or %NULL if none.
*
* Returns: (nullable) (transfer none): The model that gets sorted
**/
GListModel *
gtk_sort_list_model_get_model (GtkSortListModel *self)
{
g_return_val_if_fail (GTK_IS_SORT_LIST_MODEL (self), NULL);
return self->model;
}
Redo sort list model with GtkSorter Reshuffle the api to take full advantage of GtkSorter. Update all callers.
2019-12-03 04:44:22 +00:00
/**
* gtk_sort_list_model_set_sorter:
* @self: a #GtkSortListModel
* @sorter: (allow-none): the #GtkSorter to sort @model with
*
* Sets a new sorter on @self.
*/
void
gtk_sort_list_model_set_sorter (GtkSortListModel *self,
GtkSorter *sorter)
{
g_return_if_fail (GTK_IS_SORT_LIST_MODEL (self));
g_return_if_fail (sorter == NULL || GTK_IS_SORTER (sorter));
gtk_sort_list_model_clear_sorter (self);
if (sorter)
{
self->sorter = g_object_ref (sorter);
g_signal_connect (sorter, "changed", G_CALLBACK (gtk_sort_list_model_sorter_changed_cb), self);
}
sortlistmodel: Split the SortItem into 2 arrays Instead of one item keeping the item + its position and sorting that list, keep the items in 1 array and put the positions into a 2nd array. This is generally slower while sorting, but allows multiple improvements: 1. We can replace items with keys This allows avoiding multiple slow lookups when using complex comparisons 2. We can keep multiple position arrays This allows doing a sorting in the background without actually emitting items-changed() until the array is completely sorted. 3. The main list tracks the items in the original model So only a single memmove() is necessary there, while the old version had to upgrade the position in every item. Benchmarks: sorting a model of simple strings old new 256,000 items 256ms 268ms 512,000 items 569ms 638ms sorting a model of file trees, directories first, by size old new 64,000 items 350ms 364ms 128,000 items 667ms 691ms removing half the model old new 512,000 items 24ms 15ms 1,024,000 items 49ms 25ms
2020-07-21 01:09:10 +00:00
gtk_sort_list_model_sorter_changed_cb (sorter, GTK_SORTER_CHANGE_DIFFERENT, self);
Redo sort list model with GtkSorter Reshuffle the api to take full advantage of GtkSorter. Update all callers.
2019-12-03 04:44:22 +00:00
g_object_notify_by_pspec (G_OBJECT (self), properties[PROP_SORTER]);
}
/**
* gtk_sort_list_model_get_sorter:
sortlistmodel: Replace with an array-based model This is the dumbest possible sortmodel using an array: Just grab all the items, put them in the array, qsort() the array. Some benchmarks (setting a new model): 125,000 items - old: 549ms new: 115ms 250,000 items - new: 250ms This performance can not be kept for simple additions and removals though.
2020-07-16 23:56:18 +00:00
* @self: a #GtkSortListModel
Redo sort list model with GtkSorter Reshuffle the api to take full advantage of GtkSorter. Update all callers.
2019-12-03 04:44:22 +00:00
*
* Gets the sorter that is used to sort @self.
*
* Returns: (nullable) (transfer none): the sorter of #self
*/
GtkSorter *
gtk_sort_list_model_get_sorter (GtkSortListModel *self)
{
g_return_val_if_fail (GTK_IS_SORT_LIST_MODEL (self), NULL);
return self->sorter;
}
sortlistmodel: Add an incremental property Also refactor a large part of the sortmodel to make this convenient. A large amount of time has been spent on getting items-changed regions minimized.
2020-07-21 00:50:45 +00:00
/**
* gtk_sort_list_model_set_incremental:
* @self: a #GtkSortListModel
* @incremental: %TRUE to sort incrementally
*
* Sets the sort model to do an incremental sort.
*
* When incremental sorting is enabled, the sortlistmodel will not do
* a complete sort immediately, but will instead queue an idle handler that
* incrementally sorts the items towards their correct position. This of
* course means that items do not instantly appear in the right place. It
* also means that the total sorting time is a lot slower.
*
* When your filter blocks the UI while sorting, you might consider
* turning this on. Depending on your model and sorters, this may become
* interesting around 10,000 to 100,000 items.
*
* By default, incremental sorting is disabled.
*/
void
gtk_sort_list_model_set_incremental (GtkSortListModel *self,
gboolean incremental)
{
g_return_if_fail (GTK_IS_SORT_LIST_MODEL (self));
if (self->incremental == incremental)
return;
self->incremental = incremental;
if (!incremental && gtk_sort_list_model_is_sorting (self))
{
guint pos, n_items;
gtk_sort_list_model_finish_sorting (self, &pos, &n_items);
if (n_items)
g_list_model_items_changed (G_LIST_MODEL (self), pos, n_items, n_items);
}
g_object_notify_by_pspec (G_OBJECT (self), properties[PROP_INCREMENTAL]);
}
/**
* gtk_sort_list_model_get_incremental:
* @self: a #GtkSortListModel
*
* Returns whether incremental sorting was enabled via
* gtk_sort_list_model_set_incremental().
*
* Returns: %TRUE if incremental sorting is enabled
*/
gboolean
gtk_sort_list_model_get_incremental (GtkSortListModel *self)
{
g_return_val_if_fail (GTK_IS_SORT_LIST_MODEL (self), FALSE);
return self->incremental;
}
sortlistmodel: Add progress estimation
2020-07-22 00:50:58 +00:00
/**
* gtk_sort_list_model_get_pending:
* @self: a #GtkSortListModel
*
* Estimates progress of an ongoing sorting operation
*
* The estimate is the number of items that would still need to be
* sorted to finish the sorting operation if this was a linear
* algorithm. So this number is not related to how many items are
* already correctly sorted.
*
* If you want to estimate the progress, you can use code like this:
* |[<!-- language="C" -->
* double progress = 1.0 - (double) gtk_sort_list_model_get_pending (self)
* / MAX (1, g_list_model_get_n_items (G_LIST_MODEL (sort)));
* ]|
*
* If no sort operation is ongoing - in particular when
* #GtkSortListModel:incremental is %FALSE - this function returns 0.
*
* Returns: a progress estimate of remaining items to sort
**/
guint
gtk_sort_list_model_get_pending (GtkSortListModel *self)
{
g_return_val_if_fail (GTK_IS_SORT_LIST_MODEL (self), FALSE);
if (self->sort_cb == 0)
return 0;
/* We do a random guess that 50% of time is spent generating keys
* and the other 50% is spent actually sorting.
*
* This is of course massively wrong, but it depends on the sorter
* in use, and estimating this correctly is hard, so this will have
* to be good enough.
*/
if (!gtk_bitset_is_empty (self->missing_keys))
{
return (self->n_items + gtk_bitset_get_size (self->missing_keys)) / 2;
}
else
{
return (self->n_items - gtk_tim_sort_get_progress (&self->sort)) / 2;
}
}
Reference in New Issue Copy Permalink