5c2f2ebfdb
`zstreamtest --newapi` (and `--opaqueapi`) create and destroy way too many threads resulting in failure of tsan tests, and potentially connected to the qemu flaky tests. This is because, at each test, the nb of threads can be changed (random). The `--no-big-tests` directive reduce this choice to 1/2 threads, in order to limit memory usage, especially for qemu and 32-bits builds. Unfortunately, swapping between 1 and 2 threads is enough to constantly create/destroy new mtctx. This patch takes advantage of the following property : via compress_generic, no internal mtctx is needed for nbThreads < 2. As a consequence, when nbThreads == 2, the currently active mtctx is necessarily good. This dramatically reduces the nb of thread creations when invoking `zstreamtest --newapi --no-big-tests` (only when parent cctx itself is created, which is randomized to 1/256 tests). Expected outcome : - at a minimum : tsan tests shall now work continuously without exploding the thread counter - at best : flaky qemu tests on `zstreamtest --newapi --no-big-tests` may stop being flaky, due to less stress from constant thread creation/destruction Real world impact : minimal, I don't expect users to constantly change `nbThreads` between each invocation. If `nbThreads` remains stable, existing implementation re-uses existing mtctx. Also : `zstreamtest --newapi` but without `--no-big-tests` doesn't benefit as much, since this test can select a random `nbThreads` value between 1 and 4. The current patch only reduces opportunity to free/create mtctx (for example : 2->1->2 doesn't need a new mtctx) but doesn't completely eliminate it, since `nbThreads` can still change between 2/3/4. A more complete solution could be to only use 2 out of 4 allocated threads, thus keeping the pool at a constant size. This would require a larger change to `POOL_*` api though. |
||
---|---|---|
.. | ||
fse_compress.c | ||
huf_compress.c | ||
zstd_compress_internal.h | ||
zstd_compress.c | ||
zstd_double_fast.c | ||
zstd_double_fast.h | ||
zstd_fast.c | ||
zstd_fast.h | ||
zstd_lazy.c | ||
zstd_lazy.h | ||
zstd_ldm.c | ||
zstd_ldm.h | ||
zstd_opt.c | ||
zstd_opt.h | ||
zstdmt_compress.c | ||
zstdmt_compress.h |