Commit Graph

45819 Commits

Author SHA1 Message Date
Lars Knoll
4d67b9dcfb Get rid of QTextCodec in QTextStream
Use QStringConverter instead. Also change the default
encoding of QTextStream to utf8.

Change-Id: I30682e75fe0462d1a937539f773640c83a2d82e1
Reviewed-by: Thiago Macieira <thiago.macieira@intel.com>
2020-05-14 07:50:19 +02:00
Lars Knoll
33bb695a28 Start porting QTextStream away from QTextCodec
As a first step add setEncoding/encoding() methods that use the
QStringConverter::Encoding enum, and port all uses of setCodec()/
codec() over to the new API.

Internally QTextStream still uses QTextCodec, this will be ported
over to QStringConverter in a follow-up change.

Change-Id: Icd764cf47b449b57f4ebd010c2dad89e6717d6c0
Reviewed-by: Thiago Macieira <thiago.macieira@intel.com>
2020-05-14 07:50:13 +02:00
Lars Knoll
8835c64f79 Get rid of the QTextCodec dependency in QXmlStreamReader
Use QStringDecoder to convert the data instead.

[ChangeLog][Important Behavior Changes] QXmlStreamWriter
always encodes XML in UTF-8, and QXmlStreamReader is limited to
XML files encoded in Unicode encodings (UTF-8, UTF-16 and UTF-32)
and latin1 (ISO-8859-1).

Change-Id: I10da612b951f4312ddaf63a89587697777dd8dc1
Reviewed-by: Lars Knoll <lars.knoll@qt.io>
2020-05-14 07:50:05 +02:00
Lars Knoll
b8db123341 Add a method to determine the encoding for encoded HTML data
This is a replacement for Qt::codecForHtml().

Change-Id: I31f03518fd9c70507cbd210a8bcf405b6a0106b1
Reviewed-by: Thiago Macieira <thiago.macieira@intel.com>
2020-05-14 07:49:05 +02:00
Lars Knoll
13af1312f7 Add QStringConverter::encodingForData()
Add method that tries to determine the encoding of the data
from an initial byte order mark.

Change-Id: I348c51a3d4db9b434af53359b739a7e17acfc760
Reviewed-by: Thiago Macieira <thiago.macieira@intel.com>
2020-05-14 07:48:55 +02:00
Lars Knoll
a639bcda1e Add methods to convert between encoding and name to QStringConverter
Add static methods that allow converting between a name for an
encoding and the Encoding enum.

Change-Id: I12bc503cf757ea31d3ca8d5e1f1216efddcb16d4
Reviewed-by: Thiago Macieira <thiago.macieira@intel.com>
2020-05-14 07:48:49 +02:00
Lars Knoll
3ce9162ab5 Construct a string converter by name
Add a constructor, that allows constructing a string converter by
name. This is required in some cases and also makes it possible to
(in the future) extend the API to 3rd party encodings.

Also add a name() accessor.

Change-Id: I606d6ce9405ee967f76197b803615e27c5b001cf
Reviewed-by: Thiago Macieira <thiago.macieira@intel.com>
2020-05-14 07:48:42 +02:00
Lars Knoll
2d43f735b4 Remove QTextCodec dependency from the cbor reader
Change-Id: I5abc60846d95931860133b9318690b457bb886d7
Reviewed-by: Thiago Macieira <thiago.macieira@intel.com>
Reviewed-by: Alex Blasche <alexander.blasche@qt.io>
2020-05-14 07:48:35 +02:00
Lars Knoll
a383d79772 Get rid of QSettings::iniCodec()
Always encodee INI files as utf-8 in Qt6. This is mostly backwards
compatible, as old ini files would encode all non ascii characters.

[ChangeLog][Important behavioral changes] QSettings will now always
encode INI files as utf-8 (and the iniCodec/setIniCode methods are
removed). This is a change from Qt 5 and earlier, where QSettings would
by default escape all non ascii characters. The behavior is equivalent to
what you got in Qt5 by setting a utf-8 iniCodec on the settings object.
Settings files written in Qt 5 will still be readable in Qt 6 (unless
an iniCodec different from utf-8 was used), but to read Qt6 based ini
files in Qt 5 applications, setting the iniCodec to utf-8 is required.

Change-Id: Ic7dffcca17779bd5e3dae50d42ce633170289f6c
Reviewed-by: Thiago Macieira <thiago.macieira@intel.com>
Reviewed-by: Alex Blasche <alexander.blasche@qt.io>
2020-05-14 07:48:28 +02:00
Lars Knoll
1cab047d08 Remove QTextCodec dependency from QCoreGobalData
As we want to move text codecs out of Qt Core, disentangle the
dependency, but moving the global codec data into qtextcodec.*.

Change-Id: Id7498423c7c4f9f42fd00c450947305d2af8c4be
Reviewed-by: Thiago Macieira <thiago.macieira@intel.com>
Reviewed-by: Alex Blasche <alexander.blasche@qt.io>
2020-05-14 07:48:20 +02:00
Lars Knoll
940665eff5 Add some incremental tests
Feed the data one by one to the encoder or decoder to
verify that the handling of incremental decoding is
correct.

Change-Id: I565e4f1872e00859026334f7662b6778772e159d
Reviewed-by: Thiago Macieira <thiago.macieira@intel.com>
2020-05-14 07:48:13 +02:00
Lars Knoll
4b2edde373 Cleanup QUtf32::convertToUnicode
Cleanup the implementation and improve performance by
handling the first char outside of the main loop.

Also avoid one copy of the data when using QStringConverter.

Change-Id: Ie698e62de1864352612a4dddc907cb139e7e6407
Reviewed-by: Thiago Macieira <thiago.macieira@intel.com>
2020-05-14 07:48:07 +02:00
Lars Knoll
b1d8ce32cd Refactor QUtf32::convertFromUnicode
Implement proper state handling, and avoid a copy when using
it through QStringConverter.

Change-Id: I201fe966601c424c337e452e359a2e71f76354ad
Reviewed-by: Thiago Macieira <thiago.macieira@intel.com>
2020-05-14 07:48:00 +02:00
Lars Knoll
d8997ad797 Clean up QUtf16::convertTo/FromUnicode
Clean up the method, and refactor it so we can avoid one
copy of the data when using QStringConverter.

Make the conversion to unicode more by avoiding conditions in
the inner loop and doing a memcpy if endianness matches.

Change-Id: I869daf861f886d69b67a1b223ac2238498b609ac
Reviewed-by: Thiago Macieira <thiago.macieira@intel.com>
2020-05-14 07:47:53 +02:00
Lars Knoll
5dcfd0ac2f Cleanup state handling in QUtf8::convertFromUnicode
And optimize the method so we can avoid a copy of
the data.

Change-Id: Ic267150db80358dbc4010bb1db2af5c0eb97dc65
Reviewed-by: Thiago Macieira <thiago.macieira@intel.com>
2020-05-14 07:47:47 +02:00
Lars Knoll
618620bc5d Ensure the conversion methods in qstringconverter always get a valid state
Make sure that the conversion methods always get a valid state. This is
already the ecase then using the new QStringConverter API, ensure the
old QTextCodec API also passes in a valid state.

This helps simplify the logic inside those methods.

Change-Id: I1945e98cdefd46bf1427e11984337f1d62abcaa2
Reviewed-by: Thiago Macieira <thiago.macieira@intel.com>
Reviewed-by: Alex Blasche <alexander.blasche@qt.io>
2020-05-14 07:47:40 +02:00
Lars Knoll
cab0d57d1e Clean up the Flag handling in QStringConverter
IgnoreHeader was a rather badly defined enum, in addition the
utf8 and utf16 codecs where handling BOMs somewhat different
for stateless decoding.

Fix this by introducing explicit flags for writing a bom when
encoding and not skipping the initial bom when decoding.

Source compatibility for QTextCodec is done with a couple of
static constexpr variables.

Change-Id: I0b2d94f84c937cec1e0494c16ef448c00382691d
Reviewed-by: Thiago Macieira <thiago.macieira@intel.com>
2020-05-14 07:47:33 +02:00
Lars Knoll
542ded462e Enable using QStringConverter together with QStringBuilder
The way the new converters are structured allows us to
use them together with QStringBuilder. Like this, we
can avoid additional and unnecessary copies of the
data.

Change-Id: I168da3860537fe81a1eb62632e4d9a6680f86af1
Reviewed-by: Thiago Macieira <thiago.macieira@intel.com>
2020-05-14 07:47:27 +02:00
Lars Knoll
9ada6e5719 Remove QTextCodec dependency from qfilesystemiterator
Assume Unix systems are utf-8 based nowadays. glib has been
assuming this for quite some time already, and all Linux and BSD
systems shipped in the last 10 years assume utf-8 for 8-bit strings.

Utf-8 is also the encoding used by macOS and QNX since a very long time.

File systems where file names are not encoded in utf-8 can usually be
translated transparently to utf8 by specifying appropriate mount
options.

Change-Id: I1970496db24e59dee8efb79ba025355a3ce87387
Reviewed-by: Thiago Macieira <thiago.macieira@intel.com>
2020-05-14 07:47:20 +02:00
Lars Knoll
d9870dc233 Remove QTextCodec dependency from QString
Change-Id: I34cc24c263684f7c04ee49205f9181476bc16c34
Reviewed-by: Thiago Macieira <thiago.macieira@intel.com>
2020-05-14 07:47:13 +02:00
Lars Knoll
2c7f9565ed Always write XML documents as UTF-8
Remove support for setting a codec different from UTF-8
for writing XML files.

All XML readers today can handle UTF-8, and there is no
reason anymore to write a file in a different encoding.

Change-Id: If89fb2d2474a2b55644d9bed7473c11ad91033eb
Reviewed-by: Simon Hausmann <hausmann@gmail.com>
2020-05-14 07:47:06 +02:00
Lars Knoll
99632c2217 Allow moving of string converter objects
They are still not copyable, but can be moved.

Change-Id: Id66e35be4ecdaa781ecb9212d646d224b1767913
Reviewed-by: Thiago Macieira <thiago.macieira@intel.com>
2020-05-14 07:46:59 +02:00
Lars Knoll
7b93bedb60 Add Latin1 to the set of supported encodings in QStringConverter
Latin1 is the only non Unicode encoding that is still being used
to some extent. Current web site statistics show that it is
being used in ~2% of all web sites. An additional 1% of web sites
use Windows1251 (which is almost the same as latin1).

As it's trivial to support this encoding, we keep it supported
in QStringConverter.

Change-Id: I0eff53a490b6c43d3e474107e7823be245d1715a
Reviewed-by: Thiago Macieira <thiago.macieira@intel.com>
2020-05-14 07:46:51 +02:00
Lars Knoll
94e210faea Move local8bit conversion over to qutfsupport
Local8Bit is always UTF-8 except for Windows platforms.
Also add a Locale encoding to QStringConverter.

Change-Id: I8d729931fd4c1d7fc6857696b6442a44def3fd9d
Reviewed-by: Thiago Macieira <thiago.macieira@intel.com>
2020-05-14 07:46:45 +02:00
Lars Knoll
ea0a08c898 Move the UTF conversion methods to qstringconverter
Separate them from the qutfcodec, so that the codec
can later on be moved out of Qt Core.

Fix the QUtf methods to take qsizetype instead of int
for length arguments.

This also makes it possible to not build QTextCodec into
the bootstrap lib anymore.

Change-Id: I0b4f83139d61b19c651520a2f3a5012aa7e85cb8
Reviewed-by: Thiago Macieira <thiago.macieira@intel.com>
2020-05-14 07:46:38 +02:00
Lars Knoll
f64a6bd638 Start work on a new API to replace QTextCodec
The new QStringEncoder and QStringDecoder classes
(with a common QStringConverter base class) are
there to replace QTextCodec in Qt 6.

It currently uses a trivial wrapper around the utf
encoding functionality.

Added some autotests, mostly copied from the text codec
tests.

Change-Id: Ib6eeee55fba918b9424be244cbda9dfd5096f7eb
Reviewed-by: Thiago Macieira <thiago.macieira@intel.com>
2020-05-14 07:46:14 +02:00
Stefan Löffler
09ee4282e5 Fix invalid text layout data when a full layout run is interrupted
When a QTextDocument is laid out, this is done in "chunks" to keep
programs responsive. During the layout process, blocks are split into
lines. When a full (re)layout is interrupted (e.g. because a
QHighlighter changes some formats before the layout for all chunks is
completed), later chunks are skipped. This results in invalid data
(e.g., blocks not split into lines).
This change ensures that full layout runs of the root frame are
completed even after interruptions.

Fixes: QTBUG-20354
Change-Id: I041c73a532a5abe74d577ca49810191b5594dca2
Reviewed-by: Eskil Abrahamsen Blomfeldt <eskil.abrahamsen-blomfeldt@qt.io>
2020-05-14 07:20:55 +02:00
Jan Grulich
844967d297 Support opening directories over portal
Recent version of xdg-desktop-portal got support for opening directories
through the portal, which means we no longer need to rely on opening the
dialog inside sandbox, hoping we have permissions to user directories.

[ChangeLog][Linux] QFileDialog will open directories through the portal
if required version of xdg-desktop-portal is running on the system.

Change-Id: Ifc9035e268f1cc8d9d6a93480e651e0d9e1e9929
Reviewed-by: Thiago Macieira <thiago.macieira@intel.com>
2020-05-14 07:03:55 +02:00
Marc Mutz
f437c8c5f9 QUrlIdna: port containsProhibitedOutput() to QStringIterator
Also Extract Method isProhibitedOutput(), which, in order to preserve
history, is placed in the unnamed namespace and violates the
indentation rules. But this code is delicate enough to be left
undisturbed. At least for now.

By said Extract Method, we can static_assert on the invalidity of the
QStringIterator::next() argument, which we assume will be rejected, so
that we continue to reject malformed surrogate pairs.

Also fix a comment that suggested the function would actively remove
("strip") malformed content, when in fact it only detects it.

Change-Id: I4185cbac71fb147e2f2036dbaf052af20bd1003f
Reviewed-by: Lars Knoll <lars.knoll@qt.io>
Reviewed-by: Thiago Macieira <thiago.macieira@intel.com>
2020-05-13 19:30:26 +00:00
Marc Mutz
c59665b0ec QSettings: fix UB (signed integer overflow) on parsing long hex/oct escapes
The code did not limit the length of hex and octal escape sequences,
but used an int as the accumulator, which causes UB on overflow.

Due to the use of the QChar(int) constructor when appending escapeVal,
only the lowest 16 bit of the value were appended to the result
string. An test case encoding this behavior explicitly suggests this
is intended behavior.

It therefore suffices to use an unsigned 16-bit value as the
accumulator (unsigned, because that doesn't cause UB on overflow, 16
bits, because that's all we care for).

For future-proofing, use char16_t as the accumulator.

Pick-to: 5.15
Change-Id: I07e7ebf1f312276b2bbcb08e4360c66a3b9522ca
Reviewed-by: Lars Knoll <lars.knoll@qt.io>
Reviewed-by: Thiago Macieira <thiago.macieira@intel.com>
2020-05-13 21:26:35 +02:00
Marc Mutz
72f6aaa7d4 QUrlIdna: port NameprepCaseFoldingEntry from int-ish to char-ish
Change-Id: I43171aae8188cc68d9c03716451c6427486cc55c
Reviewed-by: Lars Knoll <lars.knoll@qt.io>
2020-05-13 21:26:31 +02:00
Marc Mutz
f5a56cf9bb QString: deprecate fromUtf16(ushort*)/fromUcs4(uint*)
... in favor of existing char16_t* and char32_t* versions.

Swap the implementations around so the deprecated version is inline.

[ChangeLog][QtCore][QString] The fromUtf16(const ushort*) and
fromUcs4(const uint*) functions have been deprecated in favor of
the char16_t and char32_t versions, resp.

Change-Id: I94ff63ae98d190267da0b1c943d3d9c4e825ef10
Reviewed-by: Volker Hilsheimer <volker.hilsheimer@qt.io>
Reviewed-by: Thiago Macieira <thiago.macieira@intel.com>
2020-05-13 19:26:25 +00:00
Joerg Bornemann
46fb39ef5a Fix crash on macOS with -qtlibinfix'ed Qt
When Qt is configured with -qtlibinfix Foo then the prefix determination code
in QLibraryInfo crashed, because it still assumed the library name to be
"QtCore".

Fixes: QTBUG-83912
Pick-to: 5.15
Change-Id: I104e115ac6904fc0a04ac45d68d26d4e02aba721
Reviewed-by: Alexandru Croitor <alexandru.croitor@qt.io>
2020-05-13 15:12:54 +00:00
Volker Hilsheimer
92df790f46 QApplication: refactor delivery and propagation of wheel events
Handle wheel grabbing via wheel_widget in a single place, and
propagate events in the same way for all (spontaneous) events.

Handle ScrollMomentum the same way as ScrollUpdate to allow
partial sequences.

Fix the incorrect ignoring of wheel events by default; like all
other input events, they are now again accepted by default and
ignored in the default event handler implementation of QWidget.
This way, implementing the handle suffices to accept the event.
Note that QWidget::wheelEvent doesn't need to be changed, as the
event is ignored there today (an oversight of the change made in
f253f4c3, perhaps).

This also fixes changing of direction of a wheel event while
the event sequence is grabbed by a widget.

Change-Id: Ia0f03c14dede80322d690ca50d085898a0497dbe
Fixes: QTBUG-67032
Task-number: QTBUG-79102
Reviewed-by: Shawn Rutledge <shawn.rutledge@qt.io>
2020-05-13 15:07:55 +00:00
Allan Sandfeld Jensen
01b2ea83aa Avoid format conversion in backing store texture upload
Use glPixelStorei where possible

Pick-To: 5.15, 5.12
Fixes: QTBUG-84189
Change-Id: Iadf039b5c6d8e7b6bb11d031a94683343dee0dc6
Reviewed-by: Laszlo Agocs <laszlo.agocs@qt.io>
2020-05-13 16:52:11 +02:00
Marc Mutz
1f7eb3332e QUrlIdna: replace manual pointer handling with std::begin()/end()
This is more readable, because it follows usual iterator patterns.

Change-Id: I56900286ddbca57a86aa499020f175e8add17fd9
Reviewed-by: Lars Knoll <lars.knoll@qt.io>
2020-05-13 15:23:06 +02:00
Marc Mutz
a504f63f2e Port qt_punycodeEncoder() to QStringView
Change-Id: I264e67bc08413f8a39e2d16c774bfd2c76c320ac
Reviewed-by: Lars Knoll <lars.knoll@qt.io>
2020-05-13 13:22:09 +00:00
Marc Mutz
e201d58540 QString: fix premature pessimisation in arg(char-ish)
Instead of converting the char or QChar to a QString, convert them to
a QStringView, saving the memory allocation.

Pick-to: 5.15
Change-Id: I023712ec00172f1c8570e762817bfc3a95e01125
Reviewed-by: Lars Knoll <lars.knoll@qt.io>
2020-05-13 14:38:58 +02:00
Marc Mutz
90f864a4c5 QString: remove op+=(char) and op+=(QChar::SpecialCharacter)
The former messes in bad ways with the overload set (it, fatally,
attracts char16_t, e.g.). The latter was probably added in response to
ambiguities between (char) and (QChar). While it's harmless now,
remove it, since it no longer pulls its weight.

The no-ascii warning is now coming from QChar(char), so the protection
isn't lost.

[ChangeLog][QtCore][QString] The += operators taking char and
QChar::SpecialCharacter have been removed as they cause adding a
char16_t to QString to call the char overload, losing information. The
append() function was not affected.

Change-Id: I57116314bcc71c0d9476159513c0c10048239db3
Reviewed-by: Volker Hilsheimer <volker.hilsheimer@qt.io>
Reviewed-by: Lars Knoll <lars.knoll@qt.io>
2020-05-13 14:38:48 +02:00
Volker Hilsheimer
082eebe6a4 Remove deprecated QDesktopServices APIs
Change-Id: Ic21ad2938b20c1aa3ae499a921c9cff92f615816
Reviewed-by: Lars Knoll <lars.knoll@qt.io>
2020-05-13 14:38:23 +02:00
Tor Arne Vestbø
c856033851 Remove QPlatformNativeInterface functions for CGImage conversion
QMacExtras no longer use them. Clients who need the functionality
should use QImage::toCGImage(), or qt_mac_toQImage() from QtGui's
private qcoregraphics_p.h, until we've made a public verion of it.

Task-number: QTBUG-83252
Change-Id: If74d05afe8f4ce6728d8c503bd97f7632583fbed
Reviewed-by: Volker Hilsheimer <volker.hilsheimer@qt.io>
Reviewed-by: Timur Pocheptsov <timur.pocheptsov@qt.io>
2020-05-13 14:30:43 +02:00
Friedemann Kleint
7cafcd4867 QtCore/Windows: fix deprecated QChar conversions
global\qlogging.cpp:1698:91: warning: 'constexpr QChar::QChar(char)' is deprecated: Use fromUtf8, QStringLiteral, or QLatin1String [-Wdeprecated-declarations]

Change-Id: I3c8a9499c9504bb6198dc187cb79239f705e2d39
Reviewed-by: Marc Mutz <marc.mutz@kdab.com>
2020-05-13 12:07:52 +00:00
Mårten Nordheim
80e0d0e08e QNetworkReply/http2: Add a contentEncoding test
Will be useful when DecompressHelper gets taken into use for both.

Task-number: QTBUG-83269
Change-Id: Iaf253219bed193025c2b82d6609f4dcc4de33df8
Reviewed-by: Timur Pocheptsov <timur.pocheptsov@qt.io>
2020-05-13 13:54:44 +02:00
Tony Sarajärvi
1736746228 Blacklist imageAtRightAlignedTab in SLES
Task-number: QTBUG-46206
Change-Id: I8da530eedcef937f5dea6c7507be74cbe26186a1
Reviewed-by: Heikki Halmet <heikki.halmet@qt.io>
2020-05-13 09:34:56 +00:00
Friedemann Kleint
b35551a75e Document Qt for Python usage of the resource system
Pick-to: 5.15
Change-Id: I77576d0e6464099a7595a74f4bf8ef18ec515ec2
Reviewed-by: hjk <hjk@qt.io>
Reviewed-by: Venugopal Shivashankar <Venugopal.Shivashankar@qt.io>
Reviewed-by: Cristian Maureira-Fredes <cristian.maureira-fredes@qt.io>
2020-05-13 11:04:59 +02:00
Shawn Rutledge
e8057600b6 Remove deprecated functionality in QMetaObjectBuilder
Change-Id: Ic0539aed854b9bf9b335cbb8030d5a4eb4296a7d
Reviewed-by: Fabian Kosmale <fabian.kosmale@qt.io>
2020-05-13 08:51:56 +00:00
Assam Boudjelthia
4ff5c8b954 Android: add ANDROID_ABIS qmake snippet
Add snippet on how to use ANDROID_ABIS inside the .pro file.

Pick-to: 5.15
Task-number: QTCREATORBUG-24014
Change-Id: Id3bb960c04ea6480b06878121ad33f89c03e5642
Reviewed-by: BogDan Vatra <bogdan@kdab.com>
Reviewed-by: Leena Miettinen <riitta-leena.miettinen@qt.io>
2020-05-13 08:35:41 +00:00
Peter Seiderer
2246f270ff ARM neon draw helper only available for little endian
Fixes:

  qimage_conversions.cpp:(.text+0x2598): undefined reference to `storeRGB32FromARGB32PM_neon(unsigned char*, unsigned int const*, int, int, QVector<unsigned int> const*, QDitherInfo*)'

Task-number: QTBUG-84110
Change-Id: I61ac697dd1d0b2c793fd5751209cd041ee30ad38
Pick-to: 5.15
Reviewed-by: Allan Sandfeld Jensen <allan.jensen@qt.io>
2020-05-13 09:11:48 +02:00
Marc Mutz
b49314af1d Port qt_check_std3rules() to QStringView
Also port its callees. These functions scream to the QStringView-ified...

Change-Id: I13c95d65941eb8d02223306d80efd1437b4bd9b7
Reviewed-by: Lars Knoll <lars.knoll@qt.io>
2020-05-13 08:04:40 +02:00
Giuseppe D'Angelo
554ef6c39c Drop the deprecated QStringViewLiteral
Change-Id: I6e6643b4c7cbcd43c495c332de0d6874856b9528
Reviewed-by: Marc Mutz <marc.mutz@kdab.com>
2020-05-13 03:53:16 +02:00