Q(Color|File)Dialog: Fix several UBs (invalid cast/member call) in Private::canBeNativeDialog()

Found by UBSan:

  qcolordialog.cpp:86:5: runtime error: downcast of address 0x7ffdf50c1ec0 which does not point to an object of type 'QColorDialog'
  0x7ffdf50c1ec0: note: object is of type 'QDialog'
   fd 7f 00 00  d8 6e c7 23 b7 2a 00 00  50 c1 af 01 00 00 00 00  b0 70 c7 23 b7 2a 00 00  00 00 1a 1e
                ^~~~~~~~~~~~~~~~~~~~~~~
                vptr for 'QDialog'
    #0 0x2ab720e4ec97 in QColorDialogPrivate::q_func() const qcolordialog.cpp:86
    #1 0x2ab720e4ec97 in QColorDialogPrivate::canBeNativeDialog() const qcolordialog.cpp:1865
    #2 0x2ab720e84ed6 in QDialog::setVisible(bool) qdialog.cpp:696
    #3 0x2ab720e6c1fa in QDialog::~QDialog() qdialog.cpp:357
    #4 0x2ab720e2b276 in QColorDialog::~QColorDialog() qcolordialog.cpp:2187
    #5 0x2ab720e5e2c6 in QColorDialog::getColor(QColor const&, QWidget*, QString const&, QFlags<QColorDialog::ColorDialogOption>) qcolordialog.cpp:2148
    #6 0x2ab720e5e473 in QColorDialog::getRgba(unsigned int, bool*, QWidget*) qcolordialog.cpp:2176
    #7 0x407180 in tst_QColorDialog::testGetRgba() tst_qcolordialog.cpp:118

  qfiledialog_p.h:112:5: runtime error: downcast of address 0x7ffd6858cc60 which does not point to an object of type 'QFileDialog'
  0x7ffd6858cc60: note: object is of type 'QDialog'
   a1 2b 00 00  d8 1e 5e 0c a1 2b 00 00  b0 af 01 20 a1 2b 00 00  b0 20 5e 0c a1 2b 00 00  00 00 46 00
                ^~~~~~~~~~~~~~~~~~~~~~~
                vptr for 'QDialog'
    #0 0x2ba10980a9e7 in QFileDialogPrivate::q_func() const qfiledialog_p.h:112
    #1 0x2ba10980a9e7 in QFileDialogPrivate::canBeNativeDialog() const qfiledialog.cpp:695
    #2 0x2ba1097efe36 in QDialog::setVisible(bool) qdialog.cpp:696
    #3 0x2ba1097d715a in QDialog::~QDialog() qdialog.cpp:357
    #4 0x2ba109854c4b in QFileDialog::~QFileDialog() qfiledialog.cpp:380
    #5 0x4179dc in tst_QFiledialog::init() tst_qfiledialog.cpp:175

Fix by replacing Q_Q with the the equivalent expression for QDialog.

We can't re-use QDialogPrivate::q_func() here, since that is private,
and probably should stay like that.

Also fix an invalid member call in
QColorDialogPrivate::canBeNativeDialog():

  qcolordialog.cpp:2050:5: runtime error: member call on address 0x7ffdf50c1ec0 which does not point to an object of type 'QColorDialog'
  0x7ffdf50c1ec0: note: object is of type 'QDialog'
   fd 7f 00 00  d8 6e c7 23 b7 2a 00 00  50 c1 af 01 00 00 00 00  b0 70 c7 23 b7 2a 00 00  00 00 1a 1e
                ^~~~~~~~~~~~~~~~~~~~~~~
                vptr for 'QDialog'
    #0 0x2ab720e4e5ea in QColorDialog::options() const qcolordialog.cpp:2050
    #1 0x2ab720e4e8c8 in QColorDialogPrivate::canBeNativeDialog() const qcolordialog.cpp:1870
    #2 0x2ab720e84ed6 in QDialog::setVisible(bool) qdialog.cpp:696
    #3 0x2ab720e6c1fa in QDialog::~QDialog() qdialog.cpp:357
    #4 0x2ab720e2b276 in QColorDialog::~QColorDialog() qcolordialog.cpp:2187
    #5 0x2ab720e5e2c6 in QColorDialog::getColor(QColor const&, QWidget*, QString const&, QFlags<QColorDialog::ColorDialogOption>) qcolordialog.cpp:2148
    #6 0x2ab720e5e473 in QColorDialog::getRgba(unsigned int, bool*, QWidget*) qcolordialog.cpp:2176
    #7 0x407180 in tst_QColorDialog::testGetRgba() tst_qcolordialog.cpp:118

by accessing the data member directly instead of through the Public API.

Fix the same code in QFileDialog, even though the autotest coverage is
too limited for UBSan to point that one out explicitly.

This commit amends abe8b4ab9b, in which
it should have been included in the first place...

Change-Id: Iff0538eba61d2381359f0b61f35918d643f7aa0c
Reviewed-by: Thiago Macieira <thiago.macieira@intel.com>
This commit is contained in:
Marc Mutz 2016-09-21 15:21:00 +02:00
parent 6a7b6c376b
commit 82eb7d1537
2 changed files with 8 additions and 4 deletions

View File

@ -1862,12 +1862,14 @@ void QColorDialogPrivate::retranslateStrings()
bool QColorDialogPrivate::canBeNativeDialog() const
{
Q_Q(const QColorDialog);
// Don't use Q_Q here! This function is called from ~QDialog,
// so Q_Q calling q_func() invokes undefined behavior (invalid cast in q_func()).
const QDialog * const q = static_cast<const QDialog*>(q_ptr);
if (nativeDialogInUse)
return true;
if (q->testAttribute(Qt::WA_DontShowOnScreen))
return false;
if (q->options() & QColorDialog::DontUseNativeDialog)
if (options->options() & QColorDialog::DontUseNativeDialog)
return false;
QLatin1String staticName(QColorDialog::staticMetaObject.className());

View File

@ -695,12 +695,14 @@ void QFileDialogPrivate::emitFilesSelected(const QStringList &files)
bool QFileDialogPrivate::canBeNativeDialog() const
{
Q_Q(const QFileDialog);
// Don't use Q_Q here! This function is called from ~QDialog,
// so Q_Q calling q_func() invokes undefined behavior (invalid cast in q_func()).
const QDialog * const q = static_cast<const QDialog*>(q_ptr);
if (nativeDialogInUse)
return true;
if (q->testAttribute(Qt::WA_DontShowOnScreen))
return false;
if (q->options() & QFileDialog::DontUseNativeDialog)
if (options->options() & QFileDialog::DontUseNativeDialog)
return false;
QLatin1String staticName(QFileDialog::staticMetaObject.className());