Qt on macOS has traditionally not included a BOM in the UTF-16 data,
but due to iOS requiring it it was changed in 4e196159. This had the
unfortunate side effect of breaking macOS applications that were not
prepared for the BOM, even if the public.utf16-plain-text UTI can have
an optional BOM, most notably Microsoft Excel. It also resulted in the
public.utf8-plain-text having a BOM, as that's automatically generated
by macOS based on the UTF-16 content we give it. Having a BOM in UTF-8
is technically fine, but not required, and recommended against.
The fact that iOS requires a BOM is a bit dubious, and most likely a
result of applications or system frameworks decoding the data using
NSUTF16StringEncoding, which assumes big-ending byte ordering if there
is no BOM, as opposed to public.utf16-plain-text which assumes native
byte ordering. Since we can't fix iOS our best bet is to include a BOM.
For macOS though, we revert back to the old behavior of not including
a BOM, since that seems to surprise macOS frameworks and applications
the least, even if having a BOM in public.utf16-plain-text should be
fully supported.
Longer term we should look at what kind of UTIs we generate. Most apps
on macOS do not generate public.utf16-plain-text, but instead generate
public.utf16-external-plain-text, which differs from the former in that
it assumes big-endian byte-ordering when there's no BOM. On iOS apps
seem to generate public.utf8-plain-text, and do not generate any UTF-16
UTIs. Moving Qt over to these UTIs would fix the problem as well, but
is a larger change that needs more research.
Change-Id: I4769c8b7d09daef7e3012e99cacc3237f7b0fc1a
Fixes: QTBUG-61562
Reviewed-by: Tor Arne Vestbø <tor.arne.vestbo@qt.io>
387691498a