After the fixes in the previous commits conversions between wchar_t containing
surrogates and UTF-{8,16,32} work correctly, so add a test ensuring that this
is the case.
Notice that other conversions are still broken in presence of surrogates.
See #17070.
UTF-32 conversions use a useful optimization by avoiding the extra scan of the
input wchar_t string in their FromWChar() and cWC2MB() implementation when
they are only asked to compute the required buffer size without actually doing
the conversion. However this means that for an input string containing UTF-16
surrogates (which is possible under MSW where wchar_t is 16 bits) the actual
size of the output string can be smaller than that returned by FromWChar(NULL).
Document that this may happen and avoid relying on the exact equality in the
tests.
See #17070.
Correctly fail if the wide string being converted is UTF-16 encoded (which can
only happen on platforms using 16 bit wchar_t, i.e. MSW) and ends in the
middle of a surrogate pair.
Notice that other conversions still wrongly encode invalid wchar_t sequences
such as 0xd800 not followed by anything, this will need to be fixed in the
future, but for now at least make it work for the most commonly used
conversion.
See #17070.
This keyword is not expanded by Git which means it's not replaced with the
correct revision value in the releases made using git-based scripts and it's
confusing to have lines with unexpanded "$Id$" in the released files. As
expanding them with Git is not that simple (it could be done with git archive
and export-subst attribute) and there are not many benefits in having them in
the first place, just remove all these lines.
If nothing else, this will make an eventual transition to Git simpler.
Closes#14487.
git-svn-id: https://svn.wxwidgets.org/svn/wx/wxWidgets/trunk@74602 c3d73ce0-8a6f-49c7-b76d-6d57e0e08775
Disable unit tests involving operations (such as conversions between UTF and
anything but plain ASCII) not available in ANSI build.
This fixes the test suite for non-Unicode build under Unix.
git-svn-id: https://svn.wxwidgets.org/svn/wx/wxWidgets/trunk@66103 c3d73ce0-8a6f-49c7-b76d-6d57e0e08775
Don't assign wxNO_LEN to int variable, this results in gcc warnings about
overflow in implicit constant conversion.
Use size_t for the variable containing string length to fix it.
git-svn-id: https://svn.wxwidgets.org/svn/wx/wxWidgets/trunk@65838 c3d73ce0-8a6f-49c7-b76d-6d57e0e08775
This test fails for MinGW, probably because its CRT doesn't use the same
locale names as MSVC CRT. Just disable it for now, as it was already disabled
for non-MSW platforms.
Also use LocaleSetter class (extracted from CLocaleSetter) to change the
locale for this test duration only to avoid affecting any tests running after
it.
git-svn-id: https://svn.wxwidgets.org/svn/wx/wxWidgets/trunk@65743 c3d73ce0-8a6f-49c7-b76d-6d57e0e08775
Remove unnecessary casts and initialize the variable with its contents
directly instead of using strange looking wxWCharBuffer ctor from size_t (even
if it did work, it was unnecessary).
git-svn-id: https://svn.wxwidgets.org/svn/wx/wxWidgets/trunk@65741 c3d73ce0-8a6f-49c7-b76d-6d57e0e08775
We didn't handle the case when the length of the input buffer was not
specified correctly and wxConvAuto::DetectBOM() could read beyond the end of
input. Moreover, the unit test actually relied on this as it didn't pass the
correct length for the literal strings with embedded NULs. This somehow worked
with MSVC but failed with MinGW (see #10713).
Correct the code to handle wxNO_LEN case correctly and fix the unit test to
pass the correct lengths.
git-svn-id: https://svn.wxwidgets.org/svn/wx/wxWidgets/trunk@65739 c3d73ce0-8a6f-49c7-b76d-6d57e0e08775
wxWidgets requires wchar_t for some time now; wx/chartype.h has a check
to fail complation without it. Simplify code by removing now-dead code
for the !wxUSE_WCHAR_T case.
git-svn-id: https://svn.wxwidgets.org/svn/wx/wxWidgets/trunk@63991 c3d73ce0-8a6f-49c7-b76d-6d57e0e08775
wxConvAuto implicitly supposed that the chunk of data passed to it for
translation was big enough to allow it to at least detect the BOM from it.
However this isn't necessarily the case and never is with wxTextInputStream
which reads the bytes one by one.
Fix this by waiting until we have enough data to be able to detect the BOM.
This still doesn't fix the problem with streams without BOM and the
corresponding unit test still fails -- it will need to be fixed at the level
of wxTextInputStream itself later but handling correctly the cases when a BOM
is present is already better than before.
See #11570.
git-svn-id: https://svn.wxwidgets.org/svn/wx/wxWidgets/trunk@63064 c3d73ce0-8a6f-49c7-b76d-6d57e0e08775
The variable "lenChunk" was incorrectly used as the length of the wide string
chunk which could result in wrong output.
Worse, the output buffer could be overflown for the final chunk because it
didn't have to have enough space for the trailing NUL(s) in it.
Fix both bugs and added unit tests for them.
Based on patch by Kuang-che Wu.
Closes#11486.
git-svn-id: https://svn.wxwidgets.org/svn/wx/wxWidgets/trunk@62793 c3d73ce0-8a6f-49c7-b76d-6d57e0e08775
