Commit Graph

16 Commits

Author SHA1 Message Date
Fabian Kosmale
090394ab6f tst_qdecompresshelper: Disable big data tests under ASAN
They are already slow without ASAN, and risk running into timeouts with
ASAN enabled.

Change-Id: I427b990066d8ffa838ea90a318176fbcba400852
Reviewed-by: Mårten Nordheim <marten.nordheim@qt.io>
2022-11-30 03:50:37 +00:00
Lucie Gérard
05fc3aef53 Use SPDX license identifiers
Replace the current license disclaimer in files by
a SPDX-License-Identifier.
Files that have to be modified by hand are modified.
License files are organized under LICENSES directory.

Task-number: QTBUG-67283
Change-Id: Id880c92784c40f3bbde861c0d93f58151c18b9f1
Reviewed-by: Qt CI Bot <qt_ci_bot@qt-project.org>
Reviewed-by: Lars Knoll <lars.knoll@qt.io>
Reviewed-by: Jörg Bornemann <joerg.bornemann@qt.io>
2022-05-16 16:37:38 +02:00
Mårten Nordheim
d642c16fe7 QNetworkReply: update decompress error message and handling
The error message was quite vague since it would then not require
any additional translations. However, in hindsight this was a mistake
since now developers just thought their downloads were being corrupted.

Pick-to: 6.2 6.3
Fixes: QTBUG-101942
Change-Id: Ie9af42510ca027d15248e5bcf21e836e709898d9
Reviewed-by: Timur Pocheptsov <timur.pocheptsov@qt.io>
2022-04-27 18:31:01 +02:00
Pasi Petäjäjärvi
39310a6817 Handle situation when device running test does not have enough RAM
CI tests for QNX are run in QEMU which does not have enough RAM
to run this test successfully.

Pick-to: 6.2 6.3
Change-Id: Idab3e8b6a1e0ae3eddf5aedb82e6784a74ae9a3a
Reviewed-by: Volker Hilsheimer <volker.hilsheimer@qt.io>
Reviewed-by: Mårten Nordheim <marten.nordheim@qt.io>
2022-02-09 14:17:53 +00:00
Mårten Nordheim
b41d5f6293 QDecompressHelper: Skip double-decompression if download is small
To retain backwards compatibility with some QNetworkReply usage,
namely connecting solely to finished-signal and allocating a buffer to
read into, but without storing the entire decompressed file in memory
until read, we may decompress the file twice.

With this patch users can now avoid this double decompression if the
amount of buffered data stays below 10 MiB. This means any file smaller
than 10 MiB will never need to be decompressed twice to know the size of
it. On top of that, if the data is handled as it arrives (e.g. in
readyRead) and the buffer is kept below 10 MiB it won't need to
decompress twice either.

This is active as long as "countDecompressed" is true, though it
currently always is in QNetworkAccessManger, with a future goal to make
it possible to control with public API. Since it requires the user to
potentially adapt their usage of QNetworkReply.

In this patch we also stop tracking the amount of unhandled uncompressed
bytes (uncompressedBytes) in favor of tracking the total amount of bytes
which has been read() by the user of QDecompressHelper (totalBytesRead),
since we can more intuitively work out the total amount of unread bytes
using this value.

Change-Id: Ie3d8d6e39a18343fcf9b610f45c7fe7e4cd4e474
Reviewed-by: Timur Pocheptsov <timur.pocheptsov@qt.io>
2021-08-04 19:31:52 +02:00
Mårten Nordheim
d33040548f QNetworkRequest: Rename (set)minimumArchiveBombSize
To (set)decompressedSafetyCheckThreshold, as suggested on the API review.

Task-number: QTBUG-94407
Change-Id: Iffc52691022939ae46703de8a0416355487b716f
Reviewed-by: Timur Pocheptsov <timur.pocheptsov@qt.io>
Reviewed-by: Edward Welbourne <edward.welbourne@qt.io>
2021-07-02 13:05:32 +00:00
Mårten Nordheim
69982182a3 QNetworkRequest: Add API to set a minimum archive bomb size
Fixes: QTBUG-91870
Change-Id: Ia23e8b8bcfdf65a91fe57e739242a355c681c9e6
Reviewed-by: Edward Welbourne <edward.welbourne@qt.io>
2021-05-31 17:25:20 +02:00
Mårten Nordheim
4da22d6534 tst_QDecompressHelper: Move pointer size check inside the test
For some reason the test case was (no longer?) detected by moc

Change-Id: I20f20de76479f7d9db43fae985cefe0869a7e203
Reviewed-by: Timur Pocheptsov <timur.pocheptsov@qt.io>
Reviewed-by: Edward Welbourne <edward.welbourne@qt.io>
2021-05-21 21:13:47 +02:00
Øystein Heskestad
be73ca7eb1 Make qdecompresshelper archive bomb check only trigger for large files
This is to avoid false positives.
By default files are large if uncompressed size > 10 MB. Only configurable internally.
Also add auto tests.

Task-number: QTBUG-91392
Pick-to: 6.0 6.1
Change-Id: I32258cb7c957f2a23a05157ba4ed5c0af2ba585e
Reviewed-by: Mårten Nordheim <marten.nordheim@qt.io>
Reviewed-by: Edward Welbourne <edward.welbourne@qt.io>
2021-03-05 15:13:53 +01:00
David Skoland
27d96b4789 Replace QtTest headers with QTest
Complete search and replace of QtTest and QtTest/QtTest with QTest, as
QtTest includes the whole module. Replace all such instances with
correct header includes. See Jira task for more discussion.

Fixes: QTBUG-88831
Change-Id: I981cfae18a1cabcabcabee376016b086d9d01f44
Pick-to: 6.0
Reviewed-by: Volker Hilsheimer <volker.hilsheimer@qt.io>
2020-12-22 15:20:30 +01:00
Mårten Nordheim
a07f35409b QDecompressHelper: limit decompression ratio
To avoid potential decompression bombs. This is implemented with just
a simple check that the ratio doesn't pass some hardcoded preset.

Change-Id: I17246f0f43e73280cdb35a8f03d65885f5678ad6
Reviewed-by: Thiago Macieira <thiago.macieira@intel.com>
2020-09-22 19:08:53 +02:00
Mårten Nordheim
d40f88e8d2 QDecompressHelper: Introduce zstd support
Also take this opportunity to reshuffle the content-encodings in the
intended ordering since the ordering is used to signify priority.

Task-number: QTBUG-83269
Change-Id: I022eecf1ba03b54dbd9c98a9d63d05fb05fd2124
Reviewed-by: Thiago Macieira <thiago.macieira@intel.com>
2020-08-14 13:17:11 +02:00
Ulf Hermann
3e25d09b08 QDecompressHelper: Change order of encodings
Before the introduction of QDecompressHelper gzip was preferred over
deflate. The change seems to be accidental. Amends commit
7b76379a89.

Change-Id: I70f33d551912465d63f49ea3db1ac3575d19a92d
Reviewed-by: Mårten Nordheim <marten.nordheim@qt.io>
2020-08-12 08:27:09 +00:00
Mårten Nordheim
ad1a5bf63f QDecompressHelper: Add brotli support
Task-number: QTBUG-83269
Change-Id: If23b098ee76a4892e4c2c6ce5c635688d8d9138d
Reviewed-by: Timur Pocheptsov <timur.pocheptsov@qt.io>
2020-08-06 20:56:42 +02:00
Mårten Nordheim
7b76379a89 Make use of QDecompressHelper for HTTP downloads
Changes are not too big for now. Just replaces use of the previous
calls to the zlib decompression function. And initialize
QDecompressHelper when we know the content-encoding.

Task-number: QTBUG-83269
Change-Id: I41358feaef2e7ac5f48f14e3f95ec094e0c110b7
Reviewed-by: Timur Pocheptsov <timur.pocheptsov@qt.io>
2020-08-05 17:19:10 +02:00
Mårten Nordheim
07b008425a Privately introducing QDecompressHelper for network purposes
To support streaming decompression in QNAM.
Will also be used to refactor existing decompression code in QNAM.

Task-number: QTBUG-83269
Change-Id: Iecf3e359734163f15686c949f75d41fa4794a00e
Reviewed-by: Thiago Macieira <thiago.macieira@intel.com>
2020-08-05 09:17:11 +02:00