AuroraMiddleware/qt5base-lts
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity

testlib: Add Test Anything Protocol (TAP) reporter

Browse Source 
The Test Anything Protocol (TAP), was originally Perl's simple text-based
interface between testing modules and test harnesses, but has since been
adopted by a large number of producers and consumers in many different
languages, which allows colorizing and summarizing test results.

The format is very simple:

TAP version 13
ok 1 - test description
not ok 2 - test description
  ---
  message: 'Failure message'
  severity: fail
  expected: 123
  actual: 456
  ...
ok 3 - test description # SKIP
1..3

The specification [1] is very brief, so the implementation has been
based on how typical consumers behave, especially when it comes to
the undefined diagnostics block.

[1] http://testanything.org/tap-version-13-specification.html

Change-Id: I616e802ea380165c678510e940ddc6607d39c92d
Reviewed-by: Simon Hausmann <simon.hausmann@qt.io>
This commit is contained in:
Tor Arne Vestbø 2018-03-01 17:06:23 +01:00
parent 8864aca9e5
commit 3b42e098ef
47 changed files with 4356 additions and 9 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

8
src/testlib/doc/src/qttestlib-manual.qdoc
View File

@ -175,7 +175,7 @@
\list \list
\li \c -o \e{filename,format} \br \li \c -o \e{filename,format} \br
Writes output to the specified file, in the specified format (one of Writes output to the specified file, in the specified format (one of
\c txt, \c xml, \c lightxml or \c xunitxml). The special filename \c - \c txt, \c xml, \c lightxml, \c xunitxml or \c tap). The special filename \c -
may be used to log to standard output. may be used to log to standard output.
\li \c -o \e filename \br \li \c -o \e filename \br
Writes output to the specified file. Writes output to the specified file.
@ -192,6 +192,8 @@
benchmarks, since it suppresses normal pass/fail messages. benchmarks, since it suppresses normal pass/fail messages.
\li \c -teamcity \br \li \c -teamcity \br
Outputs results in TeamCity format. Outputs results in TeamCity format.
\li \c -tap \br
Outputs results in Test Anything Protocol (TAP) format.
\endlist \endlist
The first version of the \c -o option may be repeated in order to log The first version of the \c -o option may be repeated in order to log
@ -199,8 +201,8 @@
option can log test results to standard output. option can log test results to standard output.
If the first version of the \c -o option is used, neither the second version If the first version of the \c -o option is used, neither the second version
of the \c -o option nor the \c -txt, \c -xml, \c -lightxml, \c -teamcity of the \c -o option nor the \c -txt, \c -xml, \c -lightxml, \c -teamcity,
or \c -xunitxml options should be used. \c -xunitxml or \c -tap options should be used.
If neither version of the \c -o option is used, test results will be logged to If neither version of the \c -o option is used, test results will be logged to
standard output. If no format option is used, test results will be logged in standard output. If no format option is used, test results will be logged in

3
src/testlib/qabstracttestlogger_p.h
View File

@ -58,6 +58,7 @@
QT_BEGIN_NAMESPACE QT_BEGIN_NAMESPACE
class QBenchmarkResult; class QBenchmarkResult;
class QTestData;
class QAbstractTestLogger class QAbstractTestLogger
{ {
@ -91,6 +92,8 @@ public:
virtual void enterTestFunction(const char *function) = 0; virtual void enterTestFunction(const char *function) = 0;
virtual void leaveTestFunction() = 0; virtual void leaveTestFunction() = 0;
virtual void enterTestData(QTestData *) {}
virtual void addIncident(IncidentTypes type, const char *description, virtual void addIncident(IncidentTypes type, const char *description,
const char *file = 0, int line = 0) = 0; const char *file = 0, int line = 0) = 0;
virtual void addBenchmarkResult(const QBenchmarkResult &result) = 0; virtual void addBenchmarkResult(const QBenchmarkResult &result) = 0;

254
src/testlib/qtaptestlogger.cpp Normal file
View File

@ -0,0 +1,254 @@
/****************************************************************************
**
** Copyright (C) 2018 The Qt Company Ltd.
** Contact: https://www.qt.io/licensing/
**
** This file is part of the QtTest module of the Qt Toolkit.
**
** $QT_BEGIN_LICENSE:LGPL$
** Commercial License Usage
** Licensees holding valid commercial Qt licenses may use this file in
** accordance with the commercial license agreement provided with the
** Software or, alternatively, in accordance with the terms contained in
** a written agreement between you and The Qt Company. For licensing terms
** and conditions see https://www.qt.io/terms-conditions. For further
** information use the contact form at https://www.qt.io/contact-us.
**
** GNU Lesser General Public License Usage
** Alternatively, this file may be used under the terms of the GNU Lesser
** General Public License version 3 as published by the Free Software
** Foundation and appearing in the file LICENSE.LGPL3 included in the
** packaging of this file. Please review the following information to
** ensure the GNU Lesser General Public License version 3 requirements
** will be met: https://www.gnu.org/licenses/lgpl-3.0.html.
**
** GNU General Public License Usage
** Alternatively, this file may be used under the terms of the GNU
** General Public License version 2.0 or (at your option) the GNU General
** Public license version 3 or any later version approved by the KDE Free
** Qt Foundation. The licenses are as published by the Free Software
** Foundation and appearing in the file LICENSE.GPL2 and LICENSE.GPL3
** included in the packaging of this file. Please review the following
** information to ensure the GNU General Public License requirements will
** be met: https://www.gnu.org/licenses/gpl-2.0.html and
** https://www.gnu.org/licenses/gpl-3.0.html.
**
** $QT_END_LICENSE$
**
****************************************************************************/
#include "qtaptestlogger_p.h"
#include "qtestlog_p.h"
#include "qtestresult_p.h"
#include "qtestassert.h"
#include <QtCore/qregularexpression.h>
QT_BEGIN_NAMESPACE
QTapTestLogger::QTapTestLogger(const char *filename)
: QAbstractTestLogger(filename)
, m_wasExpectedFail(false)
{
}
QTapTestLogger::~QTapTestLogger()
{
}
void QTapTestLogger::startLogging()
{
QAbstractTestLogger::startLogging();
QTestCharBuffer preamble;
QTest::qt_asprintf(&preamble, "TAP version 13\n"
// By convention, test suite names are output as diagnostics lines
// This is a pretty poor convention, as consumers will then treat
// actual diagnostics, e.g. qDebug, as test suite names o_O
"# %s\n", QTestResult::currentTestObjectName());
outputString(preamble.data());
}
void QTapTestLogger::stopLogging()
{
const int total = QTestLog::totalCount();
QTestCharBuffer testPlanAndStats;
QTest::qt_asprintf(&testPlanAndStats,
"1..%d\n"
"# tests %d\n"
"# pass %d\n"
"# fail %d\n",
total, total, QTestLog::passCount(), QTestLog::failCount());
outputString(testPlanAndStats.data());
QAbstractTestLogger::stopLogging();
}
void QTapTestLogger::enterTestFunction(const char *function)
{
Q_UNUSED(function);
m_wasExpectedFail = false;
}
void QTapTestLogger::enterTestData(QTestData *data)
{
Q_UNUSED(data);
m_wasExpectedFail = false;
}
using namespace QTestPrivate;
void QTapTestLogger::outputTestLine(bool ok, int testNumber, QTestCharBuffer &directive)
{
QTestCharBuffer testIdentifier;
QTestPrivate::generateTestIdentifier(&testIdentifier, TestFunction | TestDataTag);
QTestCharBuffer testLine;
QTest::qt_asprintf(&testLine, "%s %d - %s%s\n",
ok ? "ok" : "not ok", testNumber, testIdentifier.data(), directive.data());
outputString(testLine.data());
}
void QTapTestLogger::addIncident(IncidentTypes type, const char *description,
const char *file, int line)
{
if (m_wasExpectedFail && type == Pass) {
// XFail comes with a corresponding Pass incident, but we only want
// to emit a single test point for it, so skip the this pass.
return;
}
bool ok = type == Pass || type == XPass || type == BlacklistedPass;
QTestCharBuffer directive;
if (type == XFail || type == XPass || type == BlacklistedFail || type == BlacklistedPass)
// We treat expected or blacklisted failures/passes as TODO-failures/passes,
// which should be treated as soft issues by consumers. Not all do though :/
QTest::qt_asprintf(&directive, " # TODO %s", description);
int testNumber = QTestLog::totalCount();
if (type == XFail) {
// The global test counter hasn't been updated yet for XFail
testNumber += 1;
}
outputTestLine(ok, testNumber, directive);
if (!ok) {
// All failures need a diagnostics sections to not confuse consumers
// The indent needs to be two spaces for maximum compatibility
#define YAML_INDENT " "
outputString(YAML_INDENT "---\n");
if (type != XFail) {
// This is fragile, but unfortunately testlib doesn't plumb
// the expected and actual values to the loggers (yet).
static QRegularExpression verifyRegex(
QLatin1Literal("^'(?<actualexpression>.*)' returned (?<actual>\\w+).+\\((?<message>.*)\\)$"));
static QRegularExpression comparRegex(
QLatin1Literal("^(?<message>.*)\n"
"\\s*Actual\\s+\\((?<actualexpression>.*)\\)\\s*: (?<actual>.*)\n"
"\\s*Expected\\s+\\((?<expectedexpresssion>.*)\\)\\s*: (?<expected>.*)$"));
QString descriptionString = QString::fromUtf8(description);
QRegularExpressionMatch match = verifyRegex.match(descriptionString);
if (!match.hasMatch())
match = comparRegex.match(descriptionString);
if (match.hasMatch()) {
bool isVerify = match.regularExpression() == verifyRegex;
QString message = match.captured(QLatin1Literal("message"));
QString expected;
QString actual;
if (isVerify) {
QString expression = QLatin1Literal(" (")
% match.captured(QLatin1Literal("actualexpression")) % QLatin1Char(')') ;
actual = match.captured(QLatin1Literal("actual")).toLower() % expression;
expected = (actual.startsWith(QLatin1Literal("true")) ? QLatin1Literal("false") : QLatin1Literal("true")) % expression;
if (message.isEmpty())
message = QLatin1Literal("Verification failed");
} else {
expected = match.captured(QLatin1Literal("expected"))
% QLatin1Literal(" (") % match.captured(QLatin1Literal("expectedexpresssion")) % QLatin1Char(')');
actual = match.captured(QLatin1Literal("actual"))
% QLatin1Literal(" (") % match.captured(QLatin1Literal("actualexpression")) % QLatin1Char(')');
}
QTestCharBuffer diagnosticsYamlish;
QTest::qt_asprintf(&diagnosticsYamlish,
YAML_INDENT "type: %s\n"
YAML_INDENT "message: %s\n"
// Some consumers understand 'wanted/found', while others need
// 'expected/actual', so we do both for maximum compatibility.
YAML_INDENT "wanted: %s\n"
YAML_INDENT "found: %s\n"
YAML_INDENT "expected: %s\n"
YAML_INDENT "actual: %s\n",
isVerify ? "QVERIFY" : "QCOMPARE",
qPrintable(message),
qPrintable(expected), qPrintable(actual),
qPrintable(expected), qPrintable(actual)
);
outputString(diagnosticsYamlish.data());
} else {
QTestCharBuffer unparsableDescription;
QTest::qt_asprintf(&unparsableDescription,
YAML_INDENT "# %s\n", description);
outputString(unparsableDescription.data());
}
}
if (file) {
QTestCharBuffer location;
QTest::qt_asprintf(&location,
// The generic 'at' key is understood by most consumers.
YAML_INDENT "at: %s::%s() (%s:%d)\n"
// The file and line keys are for consumers that are able
// to read more granular location info.
YAML_INDENT "file: %s\n"
YAML_INDENT "line: %d\n",
QTestResult::currentTestObjectName(),
QTestResult::currentTestFunction(),
file, line, file, line
);
outputString(location.data());
}
outputString(YAML_INDENT "...\n");
}
m_wasExpectedFail = type == XFail;
}
void QTapTestLogger::addMessage(MessageTypes type, const QString &message,
const char *file, int line)
{
Q_UNUSED(file);
Q_UNUSED(line);
if (type == Skip) {
QTestCharBuffer directive;
QTest::qt_asprintf(&directive, " # SKIP %s", message.toUtf8().constData());
outputTestLine(/* ok = */ true, QTestLog::totalCount(), directive);
return;
}
QTestCharBuffer diagnostics;
QTest::qt_asprintf(&diagnostics, "# %s\n", qPrintable(message));
outputString(diagnostics.data());
}
QT_END_NAMESPACE

85
src/testlib/qtaptestlogger_p.h Normal file
View File

@ -0,0 +1,85 @@
/****************************************************************************
**
** Copyright (C) 2018 The Qt Company Ltd.
** Contact: https://www.qt.io/licensing/
**
** This file is part of the QtTest module of the Qt Toolkit.
**
** $QT_BEGIN_LICENSE:LGPL$
** Commercial License Usage
** Licensees holding valid commercial Qt licenses may use this file in
** accordance with the commercial license agreement provided with the
** Software or, alternatively, in accordance with the terms contained in
** a written agreement between you and The Qt Company. For licensing terms
** and conditions see https://www.qt.io/terms-conditions. For further
** information use the contact form at https://www.qt.io/contact-us.
**
** GNU Lesser General Public License Usage
** Alternatively, this file may be used under the terms of the GNU Lesser
** General Public License version 3 as published by the Free Software
** Foundation and appearing in the file LICENSE.LGPL3 included in the
** packaging of this file. Please review the following information to
** ensure the GNU Lesser General Public License version 3 requirements
** will be met: https://www.gnu.org/licenses/lgpl-3.0.html.
**
** GNU General Public License Usage
** Alternatively, this file may be used under the terms of the GNU
** General Public License version 2.0 or (at your option) the GNU General
** Public license version 3 or any later version approved by the KDE Free
** Qt Foundation. The licenses are as published by the Free Software
** Foundation and appearing in the file LICENSE.GPL2 and LICENSE.GPL3
** included in the packaging of this file. Please review the following
** information to ensure the GNU General Public License requirements will
** be met: https://www.gnu.org/licenses/gpl-2.0.html and
** https://www.gnu.org/licenses/gpl-3.0.html.
**
** $QT_END_LICENSE$
**
****************************************************************************/
#ifndef QTAPTESTLOGGER_P_H
#define QTAPTESTLOGGER_P_H
//
// W A R N I N G
// -------------
//
// This file is not part of the Qt API. It exists purely as an
// implementation detail. This header file may change from version to
// version without notice, or even be removed.
//
// We mean it.
//
#include <QtTest/private/qabstracttestlogger_p.h>
QT_BEGIN_NAMESPACE
class QTapTestLogger : public QAbstractTestLogger
{
public:
QTapTestLogger(const char *filename);
~QTapTestLogger();
void startLogging() override;
void stopLogging() override;
void enterTestFunction(const char *) override;
void leaveTestFunction() override {}
void enterTestData(QTestData *data) override;
void addIncident(IncidentTypes type, const char *description,
const char *file = 0, int line = 0) override;
void addMessage(MessageTypes type, const QString &message,
const char *file = 0, int line = 0) override;
void addBenchmarkResult(const QBenchmarkResult &) override {};
private:
void outputTestLine(bool ok, int testNumber, QTestCharBuffer &directive);
bool m_wasExpectedFail;
};
QT_END_NAMESPACE
#endif // QTAPTESTLOGGER_P_H

8
src/testlib/qtestcase.cpp
View File

@ -521,6 +521,7 @@ Q_TESTLIB_EXPORT void qtest_qParseArgs(int argc, char *argv[], bool qml)
" xml : XML document\n" " xml : XML document\n"
" lightxml : A stream of XML tags\n" " lightxml : A stream of XML tags\n"
" teamcity : TeamCity format\n" " teamcity : TeamCity format\n"
" tap : Test Anything Protocol\n"
"\n" "\n"
" *** Multiple loggers can be specified, but at most one can log to stdout.\n" " *** Multiple loggers can be specified, but at most one can log to stdout.\n"
"\n" "\n"
@ -532,6 +533,7 @@ Q_TESTLIB_EXPORT void qtest_qParseArgs(int argc, char *argv[], bool qml)
" -xml : Output results as XML document\n" " -xml : Output results as XML document\n"
" -lightxml : Output results as stream of XML tags\n" " -lightxml : Output results as stream of XML tags\n"
" -teamcity : Output results in TeamCity format\n" " -teamcity : Output results in TeamCity format\n"
" -tap : Output results in Test Anything Protocol format\n"
"\n" "\n"
" *** If no output file is specified, stdout is assumed.\n" " *** If no output file is specified, stdout is assumed.\n"
" *** If no output format is specified, -txt is assumed.\n" " *** If no output format is specified, -txt is assumed.\n"
@ -619,6 +621,8 @@ Q_TESTLIB_EXPORT void qtest_qParseArgs(int argc, char *argv[], bool qml)
logFormat = QTestLog::LightXML; logFormat = QTestLog::LightXML;
} else if (strcmp(argv[i], "-teamcity") == 0) { } else if (strcmp(argv[i], "-teamcity") == 0) {
logFormat = QTestLog::TeamCity; logFormat = QTestLog::TeamCity;
} else if (strcmp(argv[i], "-tap") == 0) {
logFormat = QTestLog::TAP;
} else if (strcmp(argv[i], "-silent") == 0) { } else if (strcmp(argv[i], "-silent") == 0) {
QTestLog::setVerboseLevel(-1); QTestLog::setVerboseLevel(-1);
} else if (strcmp(argv[i], "-v1") == 0) { } else if (strcmp(argv[i], "-v1") == 0) {
@ -653,8 +657,10 @@ Q_TESTLIB_EXPORT void qtest_qParseArgs(int argc, char *argv[], bool qml)
logFormat = QTestLog::XunitXML; logFormat = QTestLog::XunitXML;
else if (strcmp(format, "teamcity") == 0) else if (strcmp(format, "teamcity") == 0)
logFormat = QTestLog::TeamCity; logFormat = QTestLog::TeamCity;
else if (strcmp(format, "tap") == 0)
logFormat = QTestLog::TAP;
else { else {
fprintf(stderr, "output format must be one of txt, csv, lightxml, xml, teamcity or xunitxml\n"); fprintf(stderr, "output format must be one of txt, csv, lightxml, xml, tap, teamcity or xunitxml\n");
exit(1); exit(1);
} }
if (strcmp(filename, "-") == 0 && QTestLog::loggerUsingStdout()) { if (strcmp(filename, "-") == 0 && QTestLog::loggerUsingStdout()) {

20
src/testlib/qtestlog.cpp
View File

@ -47,6 +47,7 @@
#include <QtTest/private/qxunittestlogger_p.h> #include <QtTest/private/qxunittestlogger_p.h>
#include <QtTest/private/qxmltestlogger_p.h> #include <QtTest/private/qxmltestlogger_p.h>
#include <QtTest/private/qteamcitylogger_p.h> #include <QtTest/private/qteamcitylogger_p.h>
#include <QtTest/private/qtaptestlogger_p.h>
#if defined(HAVE_XCTEST) #if defined(HAVE_XCTEST)
#include <QtTest/private/qxctestlogger_p.h> #include <QtTest/private/qxctestlogger_p.h>
#endif #endif
@ -213,6 +214,11 @@ namespace QTest {
FOREACH_LOGGER(logger->leaveTestFunction()); FOREACH_LOGGER(logger->leaveTestFunction());
} }
static void enterTestData(QTestData *data)
{
FOREACH_LOGGER(logger->enterTestData(data));
}
static void addIncident(QAbstractTestLogger::IncidentTypes type, const char *description, static void addIncident(QAbstractTestLogger::IncidentTypes type, const char *description,
const char *file = 0, int line = 0) const char *file = 0, int line = 0)
{ {
@ -341,6 +347,12 @@ void QTestLog::enterTestFunction(const char* function)
QTest::TestLoggers::enterTestFunction(function); QTest::TestLoggers::enterTestFunction(function);
} }
void QTestLog::enterTestData(QTestData *data)
{
QTEST_ASSERT(data);
QTest::TestLoggers::enterTestData(data);
}
int QTestLog::unhandledIgnoreMessages() int QTestLog::unhandledIgnoreMessages()
{ {
int i = 0; int i = 0;
@ -500,6 +512,9 @@ void QTestLog::addLogger(LogMode mode, const char *filename)
case QTestLog::TeamCity: case QTestLog::TeamCity:
logger = new QTeamCityLogger(filename); logger = new QTeamCityLogger(filename);
break; break;
case QTestLog::TAP:
logger = new QTapTestLogger(filename);
break;
#if defined(HAVE_XCTEST) #if defined(HAVE_XCTEST)
case QTestLog::XCTest: case QTestLog::XCTest:
logger = new QXcodeTestLogger; logger = new QXcodeTestLogger;
@ -602,6 +617,11 @@ int QTestLog::blacklistCount()
return QTest::blacklists; return QTest::blacklists;
} }
int QTestLog::totalCount()
{
return passCount() + failCount() + skipCount() + blacklistCount();
}
void QTestLog::resetCounters() void QTestLog::resetCounters()
{ {
QTest::passes = 0; QTest::passes = 0;

6
src/testlib/qtestlog_p.h
View File

@ -57,12 +57,13 @@ QT_BEGIN_NAMESPACE
class QBenchmarkResult; class QBenchmarkResult;
class QRegularExpression; class QRegularExpression;
class QTestData;
class Q_TESTLIB_EXPORT QTestLog class Q_TESTLIB_EXPORT QTestLog
{ {
public: public:
enum LogMode { enum LogMode {
Plain = 0, XML, LightXML, XunitXML, CSV, TeamCity, Plain = 0, XML, LightXML, XunitXML, CSV, TeamCity, TAP,
#if defined(HAVE_XCTEST) #if defined(HAVE_XCTEST)
XCTest XCTest
#endif #endif
@ -71,6 +72,8 @@ public:
static void enterTestFunction(const char* function); static void enterTestFunction(const char* function);
static void leaveTestFunction(); static void leaveTestFunction();
static void enterTestData(QTestData *data);
static void addPass(const char *msg); static void addPass(const char *msg);
static void addFail(const char *msg, const char *file, int line); static void addFail(const char *msg, const char *file, int line);
static void addXFail(const char *msg, const char *file, int line); static void addXFail(const char *msg, const char *file, int line);
@ -110,6 +113,7 @@ public:
static int failCount(); static int failCount();
static int skipCount(); static int skipCount();
static int blacklistCount(); static int blacklistCount();
static int totalCount();
static void resetCounters(); static void resetCounters();

2
src/testlib/qtestresult.cpp
View File

@ -110,6 +110,8 @@ void QTestResult::setCurrentTestData(QTestData *data)
{ {
QTest::currentTestData = data; QTest::currentTestData = data;
QTest::failed = false; QTest::failed = false;
if (data)
QTestLog::enterTestData(data);
} }
void QTestResult::setCurrentTestFunction(const char *func) void QTestResult::setCurrentTestFunction(const char *func)

6
src/testlib/testlib.pro
View File

@ -40,7 +40,8 @@ HEADERS = \
qtestsystem.h \ qtestsystem.h \
qtesttouch.h \ qtesttouch.h \
qtestblacklist_p.h \ qtestblacklist_p.h \
qtesthelpers_p.h qtesthelpers_p.h \
qtaptestlogger_p.h
SOURCES = \ SOURCES = \
qabstractitemmodeltester.cpp \ qabstractitemmodeltester.cpp \
@ -67,7 +68,8 @@ SOURCES = \
qtestmouse.cpp \ qtestmouse.cpp \
qtestxunitstreamer.cpp \ qtestxunitstreamer.cpp \
qxunittestlogger.cpp \ qxunittestlogger.cpp \
qtestblacklist.cpp qtestblacklist.cpp \
qtaptestlogger.cpp
DEFINES *= QT_NO_CAST_TO_ASCII \ DEFINES *= QT_NO_CAST_TO_ASCII \
QT_NO_CAST_FROM_ASCII \ QT_NO_CAST_FROM_ASCII \

16
tests/auto/testlib/selftests/expected_assert.tap Normal file
View File

@ -0,0 +1,16 @@
TAP version 13
# tst_Assert
ok 1 - initTestCase()
ok 2 - testNumber1()
# ASSERT: "false" in file qtbase/tests/auto/testlib/selftests/assert/tst_assert.cpp, line 0
not ok 3 - testNumber2()
---
# Received a fatal error.
at: tst_Assert::testNumber2() (Unknown file:0)
file: Unknown file
line: 0
...
1..3
# tests 3
# pass 2
# fail 1

61
tests/auto/testlib/selftests/expected_badxml.tap Normal file
View File

@ -0,0 +1,61 @@
TAP version 13
# tst_BadXml
ok 1 - initTestCase()
# a message
not ok 2 - badDataTag(fail end cdata ]]> text ]]> more text)
---
# a failure
at: tst_BadXml::badDataTag() (qtbase/tests/auto/testlib/selftests/badxml/tst_badxml.cpp:101)
file: qtbase/tests/auto/testlib/selftests/badxml/tst_badxml.cpp
line: 101
...
# a message
ok 3 - badDataTag(pass end cdata ]]> text ]]> more text)
# a message
not ok 4 - badDataTag(fail quotes " text" more text)
---
# a failure
at: tst_BadXml::badDataTag() (qtbase/tests/auto/testlib/selftests/badxml/tst_badxml.cpp:101)
file: qtbase/tests/auto/testlib/selftests/badxml/tst_badxml.cpp
line: 101
...
# a message
ok 5 - badDataTag(pass quotes " text" more text)
# a message
not ok 6 - badDataTag(fail xml close > open < tags < text)
---
# a failure
at: tst_BadXml::badDataTag() (qtbase/tests/auto/testlib/selftests/badxml/tst_badxml.cpp:101)
file: qtbase/tests/auto/testlib/selftests/badxml/tst_badxml.cpp
line: 101
...
# a message
ok 7 - badDataTag(pass xml close > open < tags < text)
# a message
not ok 8 - badDataTag(fail all > " mixed ]]> up > " in < the ]]> hopes < of triggering "< ]]> bugs)
---
# a failure
at: tst_BadXml::badDataTag() (qtbase/tests/auto/testlib/selftests/badxml/tst_badxml.cpp:101)
file: qtbase/tests/auto/testlib/selftests/badxml/tst_badxml.cpp
line: 101
...
# a message
ok 9 - badDataTag(pass all > " mixed ]]> up > " in < the ]]> hopes < of triggering "< ]]> bugs)
# end cdata ]]> text ]]> more text
ok 10 - badMessage(string 0)
# quotes " text" more text
ok 11 - badMessage(string 1)
# xml close > open < tags < text
ok 12 - badMessage(string 2)
# all > " mixed ]]> up > " in < the ]]> hopes < of triggering "< ]]> bugs
ok 13 - badMessage(string 3)
not ok 14 - failWithNoFile()
---
# failure message
...
ok 15 - encoding() # SKIP Skipped for text due to unpredictable console encoding.
ok 16 - cleanupTestCase()
1..16
# tests 16
# pass 10
# fail 5

17
tests/auto/testlib/selftests/expected_benchlibcounting.tap Normal file
View File

@ -0,0 +1,17 @@
TAP version 13
# tst_BenchlibCounting
ok 1 - initTestCase()
ok 2 - passingBenchmark()
ok 3 - skippingBenchmark() # SKIP This is a skipping benchmark
not ok 4 - failingBenchmark()
---
# This is a failing benchmark
at: tst_BenchlibCounting::failingBenchmark() (qtbase/tests/auto/testlib/selftests/benchlibcounting/tst_benchlibcounting.cpp:58)
file: qtbase/tests/auto/testlib/selftests/benchlibcounting/tst_benchlibcounting.cpp
line: 58
...
ok 5 - cleanupTestCase()
1..5
# tests 5
# pass 3
# fail 1

15
tests/auto/testlib/selftests/expected_benchlibeventcounter.tap Normal file
View File

@ -0,0 +1,15 @@
TAP version 13
# tst_BenchlibEventCounter
ok 1 - initTestCase()
ok 2 - events(0)
ok 3 - events(1)
ok 4 - events(10)
ok 5 - events(100)
ok 6 - events(500)
ok 7 - events(5000)
ok 8 - events(100000)
ok 9 - cleanupTestCase()
1..9
# tests 9
# pass 9
# fail 0

9
tests/auto/testlib/selftests/expected_benchlibtickcounter.tap Normal file
View File

@ -0,0 +1,9 @@
TAP version 13
# tst_BenchlibTickCounter
ok 1 - initTestCase()
ok 2 - threeBillionTicks()
ok 3 - cleanupTestCase()
1..3
# tests 3
# pass 3
# fail 0

11
tests/auto/testlib/selftests/expected_benchlibwalltime.tap Normal file
View File

@ -0,0 +1,11 @@
TAP version 13
# tst_BenchlibWalltime
ok 1 - initTestCase()
ok 2 - waitForOneThousand()
ok 3 - waitForFourThousand()
ok 4 - qbenchmark_once()
ok 5 - cleanupTestCase()
1..5
# tests 5
# pass 5
# fail 0

456
tests/auto/testlib/selftests/expected_cmptest.tap Normal file
View File

@ -0,0 +1,456 @@
TAP version 13
# tst_Cmptest
ok 1 - initTestCase()
not ok 2 - compare_unregistered_enums()
---
# Compared values are not the same
at: tst_Cmptest::compare_unregistered_enums() (qtbase/tests/auto/testlib/selftests/cmptest/tst_cmptest.cpp:171)
file: qtbase/tests/auto/testlib/selftests/cmptest/tst_cmptest.cpp
line: 171
...
not ok 3 - compare_registered_enums()
---
type: QCOMPARE
message: Compared values are not the same
wanted: Sunday (Qt::Sunday)
found: Monday (Qt::Monday)
expected: Sunday (Qt::Sunday)
actual: Monday (Qt::Monday)
at: tst_Cmptest::compare_registered_enums() (qtbase/tests/auto/testlib/selftests/cmptest/tst_cmptest.cpp:178)
file: qtbase/tests/auto/testlib/selftests/cmptest/tst_cmptest.cpp
line: 178
...
not ok 4 - compare_class_enums()
---
type: QCOMPARE
message: Compared values are not the same
wanted: MyClassEnumValue2 (MyClassEnum::MyClassEnumValue2)
found: MyClassEnumValue1 (MyClassEnum::MyClassEnumValue1)
expected: MyClassEnumValue2 (MyClassEnum::MyClassEnumValue2)
actual: MyClassEnumValue1 (MyClassEnum::MyClassEnumValue1)
at: tst_Cmptest::compare_class_enums() (qtbase/tests/auto/testlib/selftests/cmptest/tst_cmptest.cpp:184)
file: qtbase/tests/auto/testlib/selftests/cmptest/tst_cmptest.cpp
line: 184
...
ok 5 - test_windowflags(pass)
not ok 6 - test_windowflags(fail1)
---
type: QCOMPARE
message: Compared values are not the same
wanted: Window|FramelessWindowHint|WindowSystemMenuHint|WindowStaysOnBottomHint (expectedWindowFlags)
found: Window|WindowSystemMenuHint|WindowStaysOnBottomHint (actualWindowFlags)
expected: Window|FramelessWindowHint|WindowSystemMenuHint|WindowStaysOnBottomHint (expectedWindowFlags)
actual: Window|WindowSystemMenuHint|WindowStaysOnBottomHint (actualWindowFlags)
at: tst_Cmptest::test_windowflags() (qtbase/tests/auto/testlib/selftests/cmptest/tst_cmptest.cpp:209)
file: qtbase/tests/auto/testlib/selftests/cmptest/tst_cmptest.cpp
line: 209
...
not ok 7 - test_windowflags(fail2)
---
type: QCOMPARE
message: Compared values are not the same
wanted: Window|FramelessWindowHint (expectedWindowFlags)
found: Window (actualWindowFlags)
expected: Window|FramelessWindowHint (expectedWindowFlags)
actual: Window (actualWindowFlags)
at: tst_Cmptest::test_windowflags() (qtbase/tests/auto/testlib/selftests/cmptest/tst_cmptest.cpp:209)
file: qtbase/tests/auto/testlib/selftests/cmptest/tst_cmptest.cpp
line: 209
...
ok 8 - test_unregistered_flags(pass)
not ok 9 - test_unregistered_flags(fail1)
---
type: QCOMPARE
message: Compared values are not the same
wanted: 0x5 (expectedFlags)
found: 0x3 (actualFlags)
expected: 0x5 (expectedFlags)
actual: 0x3 (actualFlags)
at: tst_Cmptest::test_unregistered_flags() (qtbase/tests/auto/testlib/selftests/cmptest/tst_cmptest.cpp:242)
file: qtbase/tests/auto/testlib/selftests/cmptest/tst_cmptest.cpp
line: 242
...
not ok 10 - test_unregistered_flags(fail2)
---
type: QCOMPARE
message: Compared values are not the same
wanted: 0x5 (expectedFlags)
found: 0x1 (actualFlags)
expected: 0x5 (expectedFlags)
actual: 0x1 (actualFlags)
at: tst_Cmptest::test_unregistered_flags() (qtbase/tests/auto/testlib/selftests/cmptest/tst_cmptest.cpp:242)
file: qtbase/tests/auto/testlib/selftests/cmptest/tst_cmptest.cpp
line: 242
...
ok 11 - compare_boolfuncs()
ok 12 - compare_to_nullptr()
ok 13 - compare_pointerfuncs()
not ok 14 - compare_tostring(int, string)
---
type: QCOMPARE
message: Compared values are not the same
wanted: QVariant(QString,hi) (expected)
found: QVariant(int,123) (actual)
expected: QVariant(QString,hi) (expected)
actual: QVariant(int,123) (actual)
at: tst_Cmptest::compare_tostring() (qtbase/tests/auto/testlib/selftests/cmptest/tst_cmptest.cpp:331)
file: qtbase/tests/auto/testlib/selftests/cmptest/tst_cmptest.cpp
line: 331
...
ok 15 - compare_tostring(both invalid)
not ok 16 - compare_tostring(null hash, invalid)
---
type: QCOMPARE
message: Compared values are not the same
wanted: QVariant() (expected)
found: QVariant(QVariantHash) (actual)
expected: QVariant() (expected)
actual: QVariant(QVariantHash) (actual)
at: tst_Cmptest::compare_tostring() (qtbase/tests/auto/testlib/selftests/cmptest/tst_cmptest.cpp:331)
file: qtbase/tests/auto/testlib/selftests/cmptest/tst_cmptest.cpp
line: 331
...
not ok 17 - compare_tostring(string, null user type)
---
type: QCOMPARE
message: Compared values are not the same
wanted: QVariant(PhonyClass) (expected)
found: QVariant(QString,A simple string) (actual)
expected: QVariant(PhonyClass) (expected)
actual: QVariant(QString,A simple string) (actual)
at: tst_Cmptest::compare_tostring() (qtbase/tests/auto/testlib/selftests/cmptest/tst_cmptest.cpp:331)
file: qtbase/tests/auto/testlib/selftests/cmptest/tst_cmptest.cpp
line: 331
...
not ok 18 - compare_tostring(both non-null user type)
---
type: QCOMPARE
message: Compared values are not the same
wanted: QVariant(PhonyClass,<value not representable as string>) (expected)
found: QVariant(PhonyClass,<value not representable as string>) (actual)
expected: QVariant(PhonyClass,<value not representable as string>) (expected)
actual: QVariant(PhonyClass,<value not representable as string>) (actual)
at: tst_Cmptest::compare_tostring() (qtbase/tests/auto/testlib/selftests/cmptest/tst_cmptest.cpp:331)
file: qtbase/tests/auto/testlib/selftests/cmptest/tst_cmptest.cpp
line: 331
...
ok 19 - compareQStringLists(empty lists)
ok 20 - compareQStringLists(equal lists)
not ok 21 - compareQStringLists(last item different)
---
type: QCOMPARE
message: Compared lists differ at index 2.
wanted: "DIFFERS" (opB)
found: "string3" (opA)
expected: "DIFFERS" (opB)
actual: "string3" (opA)
at: tst_Cmptest::compareQStringLists() (qtbase/tests/auto/testlib/selftests/cmptest/tst_cmptest.cpp:425)
file: qtbase/tests/auto/testlib/selftests/cmptest/tst_cmptest.cpp
line: 425
...
not ok 22 - compareQStringLists(second-last item different)
---
type: QCOMPARE
message: Compared lists differ at index 2.
wanted: "DIFFERS" (opB)
found: "string3" (opA)
expected: "DIFFERS" (opB)
actual: "string3" (opA)
at: tst_Cmptest::compareQStringLists() (qtbase/tests/auto/testlib/selftests/cmptest/tst_cmptest.cpp:425)
file: qtbase/tests/auto/testlib/selftests/cmptest/tst_cmptest.cpp
line: 425
...
not ok 23 - compareQStringLists(prefix)
---
# Compared lists have different sizes.
Actual (opA) size: 2
Expected (opB) size: 1
at: tst_Cmptest::compareQStringLists() (qtbase/tests/auto/testlib/selftests/cmptest/tst_cmptest.cpp:425)
file: qtbase/tests/auto/testlib/selftests/cmptest/tst_cmptest.cpp
line: 425
...
not ok 24 - compareQStringLists(short list second)
---
# Compared lists have different sizes.
Actual (opA) size: 12
Expected (opB) size: 1
at: tst_Cmptest::compareQStringLists() (qtbase/tests/auto/testlib/selftests/cmptest/tst_cmptest.cpp:425)
file: qtbase/tests/auto/testlib/selftests/cmptest/tst_cmptest.cpp
line: 425
...
not ok 25 - compareQStringLists(short list first)
---
# Compared lists have different sizes.
Actual (opA) size: 1
Expected (opB) size: 12
at: tst_Cmptest::compareQStringLists() (qtbase/tests/auto/testlib/selftests/cmptest/tst_cmptest.cpp:425)
file: qtbase/tests/auto/testlib/selftests/cmptest/tst_cmptest.cpp
line: 425
...
not ok 26 - compareQListInt()
---
type: QCOMPARE
message: Compared lists differ at index 2.
wanted: 4 (int2)
found: 3 (int1)
expected: 4 (int2)
actual: 3 (int1)
at: tst_Cmptest::compareQListInt() (qtbase/tests/auto/testlib/selftests/cmptest/tst_cmptest.cpp:432)
file: qtbase/tests/auto/testlib/selftests/cmptest/tst_cmptest.cpp
line: 432
...
not ok 27 - compareQListDouble()
---
type: QCOMPARE
message: Compared lists differ at index 0.
wanted: 1 (double2)
found: 1.5 (double1)
expected: 1 (double2)
actual: 1.5 (double1)
at: tst_Cmptest::compareQListDouble() (qtbase/tests/auto/testlib/selftests/cmptest/tst_cmptest.cpp:439)
file: qtbase/tests/auto/testlib/selftests/cmptest/tst_cmptest.cpp
line: 439
...
ok 28 - compareQColor(Qt::yellow vs "yellow")
not ok 29 - compareQColor(Qt::yellow vs Qt::green)
---
type: QCOMPARE
message: Compared values are not the same
wanted: #ff00ff00 (colorB)
found: #ffffff00 (colorA)
expected: #ff00ff00 (colorB)
actual: #ffffff00 (colorA)
at: tst_Cmptest::compareQColor() (qtbase/tests/auto/testlib/selftests/cmptest/tst_cmptest.cpp:458)
file: qtbase/tests/auto/testlib/selftests/cmptest/tst_cmptest.cpp
line: 458
...
not ok 30 - compareQColor(0x88ff0000 vs 0xffff0000)
---
type: QCOMPARE
message: Compared values are not the same
wanted: #ffff0000 (colorB)
found: #88ff0000 (colorA)
expected: #ffff0000 (colorB)
actual: #88ff0000 (colorA)
at: tst_Cmptest::compareQColor() (qtbase/tests/auto/testlib/selftests/cmptest/tst_cmptest.cpp:458)
file: qtbase/tests/auto/testlib/selftests/cmptest/tst_cmptest.cpp
line: 458
...
ok 31 - compareQPixmaps(both null)
not ok 32 - compareQPixmaps(one null)
---
type: QCOMPARE
message: Compared QPixmaps differ.
wanted: 0 (opB).isNull()
found: 1 (opA).isNull()
expected: 0 (opB).isNull()
actual: 1 (opA).isNull()
at: tst_Cmptest::compareQPixmaps() (qtbase/tests/auto/testlib/selftests/cmptest/tst_cmptest.cpp:483)
file: qtbase/tests/auto/testlib/selftests/cmptest/tst_cmptest.cpp
line: 483
...
not ok 33 - compareQPixmaps(other null)
---
type: QCOMPARE
message: Compared QPixmaps differ.
wanted: 1 (opB).isNull()
found: 0 (opA).isNull()
expected: 1 (opB).isNull()
actual: 0 (opA).isNull()
at: tst_Cmptest::compareQPixmaps() (qtbase/tests/auto/testlib/selftests/cmptest/tst_cmptest.cpp:483)
file: qtbase/tests/auto/testlib/selftests/cmptest/tst_cmptest.cpp
line: 483
...
ok 34 - compareQPixmaps(equal)
not ok 35 - compareQPixmaps(different size)
---
type: QCOMPARE
message: Compared QPixmaps differ in size.
wanted: 20x20 (opB)
found: 11x20 (opA)
expected: 20x20 (opB)
actual: 11x20 (opA)
at: tst_Cmptest::compareQPixmaps() (qtbase/tests/auto/testlib/selftests/cmptest/tst_cmptest.cpp:483)
file: qtbase/tests/auto/testlib/selftests/cmptest/tst_cmptest.cpp
line: 483
...
not ok 36 - compareQPixmaps(different pixels)
---
# Compared values are not the same
at: tst_Cmptest::compareQPixmaps() (qtbase/tests/auto/testlib/selftests/cmptest/tst_cmptest.cpp:483)
file: qtbase/tests/auto/testlib/selftests/cmptest/tst_cmptest.cpp
line: 483
...
ok 37 - compareQImages(both null)
not ok 38 - compareQImages(one null)
---
type: QCOMPARE
message: Compared QImages differ.
wanted: 0 (opB).isNull()
found: 1 (opA).isNull()
expected: 0 (opB).isNull()
actual: 1 (opA).isNull()
at: tst_Cmptest::compareQImages() (qtbase/tests/auto/testlib/selftests/cmptest/tst_cmptest.cpp:510)
file: qtbase/tests/auto/testlib/selftests/cmptest/tst_cmptest.cpp
line: 510
...
not ok 39 - compareQImages(other null)
---
type: QCOMPARE
message: Compared QImages differ.
wanted: 1 (opB).isNull()
found: 0 (opA).isNull()
expected: 1 (opB).isNull()
actual: 0 (opA).isNull()
at: tst_Cmptest::compareQImages() (qtbase/tests/auto/testlib/selftests/cmptest/tst_cmptest.cpp:510)
file: qtbase/tests/auto/testlib/selftests/cmptest/tst_cmptest.cpp
line: 510
...
ok 40 - compareQImages(equal)
not ok 41 - compareQImages(different size)
---
type: QCOMPARE
message: Compared QImages differ in size.
wanted: 20x20 (opB)
found: 11x20 (opA)
expected: 20x20 (opB)
actual: 11x20 (opA)
at: tst_Cmptest::compareQImages() (qtbase/tests/auto/testlib/selftests/cmptest/tst_cmptest.cpp:510)
file: qtbase/tests/auto/testlib/selftests/cmptest/tst_cmptest.cpp
line: 510
...
not ok 42 - compareQImages(different format)
---
type: QCOMPARE
message: Compared QImages differ in format.
wanted: 3 (opB)
found: 6 (opA)
expected: 3 (opB)
actual: 6 (opA)
at: tst_Cmptest::compareQImages() (qtbase/tests/auto/testlib/selftests/cmptest/tst_cmptest.cpp:510)
file: qtbase/tests/auto/testlib/selftests/cmptest/tst_cmptest.cpp
line: 510
...
not ok 43 - compareQImages(different pixels)
---
# Compared values are not the same
at: tst_Cmptest::compareQImages() (qtbase/tests/auto/testlib/selftests/cmptest/tst_cmptest.cpp:510)
file: qtbase/tests/auto/testlib/selftests/cmptest/tst_cmptest.cpp
line: 510
...
ok 44 - compareQRegion(equal-empty)
not ok 45 - compareQRegion(1-empty)
---
type: QCOMPARE
message: Compared values are not the same
wanted: QRegion(null) (rB)
found: QRegion(200x50+10+10) (rA)
expected: QRegion(null) (rB)
actual: QRegion(200x50+10+10) (rA)
at: tst_Cmptest::compareQRegion() (qtbase/tests/auto/testlib/selftests/cmptest/tst_cmptest.cpp:533)
file: qtbase/tests/auto/testlib/selftests/cmptest/tst_cmptest.cpp
line: 533
...
ok 46 - compareQRegion(equal)
not ok 47 - compareQRegion(different lists)
---
type: QCOMPARE
message: Compared values are not the same
wanted: QRegion(2 rectangles, 50x200+100+200, 200x50+10+10) (rB)
found: QRegion(200x50+10+10) (rA)
expected: QRegion(2 rectangles, 50x200+100+200, 200x50+10+10) (rB)
actual: QRegion(200x50+10+10) (rA)
at: tst_Cmptest::compareQRegion() (qtbase/tests/auto/testlib/selftests/cmptest/tst_cmptest.cpp:533)
file: qtbase/tests/auto/testlib/selftests/cmptest/tst_cmptest.cpp
line: 533
...
not ok 48 - compareQVector2D()
---
type: QCOMPARE
message: Compared values are not the same
wanted: QVector2D(1, 3) (v2b)
found: QVector2D(1, 2) (v2a)
expected: QVector2D(1, 3) (v2b)
actual: QVector2D(1, 2) (v2a)
at: tst_Cmptest::compareQVector2D() (qtbase/tests/auto/testlib/selftests/cmptest/tst_cmptest.cpp:542)
file: qtbase/tests/auto/testlib/selftests/cmptest/tst_cmptest.cpp
line: 542
...
not ok 49 - compareQVector3D()
---
type: QCOMPARE
message: Compared values are not the same
wanted: QVector3D(1, 3, 3) (v3b)
found: QVector3D(1, 2, 3) (v3a)
expected: QVector3D(1, 3, 3) (v3b)
actual: QVector3D(1, 2, 3) (v3a)
at: tst_Cmptest::compareQVector3D() (qtbase/tests/auto/testlib/selftests/cmptest/tst_cmptest.cpp:551)
file: qtbase/tests/auto/testlib/selftests/cmptest/tst_cmptest.cpp
line: 551
...
not ok 50 - compareQVector4D()
---
type: QCOMPARE
message: Compared values are not the same
wanted: QVector4D(1, 3, 3, 4) (v4b)
found: QVector4D(1, 2, 3, 4) (v4a)
expected: QVector4D(1, 3, 3, 4) (v4b)
actual: QVector4D(1, 2, 3, 4) (v4a)
at: tst_Cmptest::compareQVector4D() (qtbase/tests/auto/testlib/selftests/cmptest/tst_cmptest.cpp:560)
file: qtbase/tests/auto/testlib/selftests/cmptest/tst_cmptest.cpp
line: 560
...
not ok 51 - verify()
---
type: QVERIFY
message: Verification failed
wanted: true (opaqueFunc() < 2)
found: false (opaqueFunc() < 2)
expected: true (opaqueFunc() < 2)
actual: false (opaqueFunc() < 2)
at: tst_Cmptest::verify() (qtbase/tests/auto/testlib/selftests/cmptest/tst_cmptest.cpp:572)
file: qtbase/tests/auto/testlib/selftests/cmptest/tst_cmptest.cpp
line: 572
...
not ok 52 - verify2()
---
type: QVERIFY
message: 42
wanted: true (opaqueFunc() < 2)
found: false (opaqueFunc() < 2)
expected: true (opaqueFunc() < 2)
actual: false (opaqueFunc() < 2)
at: tst_Cmptest::verify2() (qtbase/tests/auto/testlib/selftests/cmptest/tst_cmptest.cpp:578)
file: qtbase/tests/auto/testlib/selftests/cmptest/tst_cmptest.cpp
line: 578
...
not ok 53 - tryVerify()
---
type: QVERIFY
message: Verification failed
wanted: true (opaqueFunc() < 2)
found: false (opaqueFunc() < 2)
expected: true (opaqueFunc() < 2)
actual: false (opaqueFunc() < 2)
at: tst_Cmptest::tryVerify() (qtbase/tests/auto/testlib/selftests/cmptest/tst_cmptest.cpp:584)
file: qtbase/tests/auto/testlib/selftests/cmptest/tst_cmptest.cpp
line: 584
...
not ok 54 - tryVerify2()
---
type: QVERIFY
message: 42
wanted: true (opaqueFunc() < 2)
found: false (opaqueFunc() < 2)
expected: true (opaqueFunc() < 2)
actual: false (opaqueFunc() < 2)
at: tst_Cmptest::tryVerify2() (qtbase/tests/auto/testlib/selftests/cmptest/tst_cmptest.cpp:590)
file: qtbase/tests/auto/testlib/selftests/cmptest/tst_cmptest.cpp
line: 590
...
ok 55 - verifyExplicitOperatorBool()
ok 56 - cleanupTestCase()
1..56
# tests 56
# pass 18
# fail 38

20
tests/auto/testlib/selftests/expected_commandlinedata.tap Normal file
View File

@ -0,0 +1,20 @@
TAP version 13
# tst_DataTable
ok 1 - initTestCase()
# QVERIFY(test)
ok 2 - fiveTablePasses(fiveTablePasses_data1)
# QVERIFY(test)
ok 3 - fiveTablePasses(fiveTablePasses_data2)
# QVERIFY(test)
ok 4 - fiveTablePasses(fiveTablePasses_data3)
# QVERIFY(test)
ok 5 - fiveTablePasses(fiveTablePasses_data4)
# QVERIFY(test)
ok 6 - fiveTablePasses(fiveTablePasses_data5)
# QVERIFY(test)
ok 7 - fiveTablePasses(fiveTablePasses_data1)
ok 8 - cleanupTestCase()
1..8
# tests 8
# pass 8
# fail 0

118
tests/auto/testlib/selftests/expected_counting.tap Normal file
View File

@ -0,0 +1,118 @@
TAP version 13
# tst_Counting
ok 1 - initTestCase()
ok 2 - testPassPass(row 1)
ok 3 - testPassPass(row 2)
ok 4 - testPassSkip(row 1)
ok 5 - testPassSkip(row 2) # SKIP Skipping
ok 6 - testPassFail(row 1)
not ok 7 - testPassFail(row 2)
---
type: QVERIFY
message: Verification failed
wanted: true (false)
found: false (false)
expected: true (false)
actual: false (false)
at: tst_Counting::testPassFail() (qtbase/tests/auto/testlib/selftests/counting/tst_counting.cpp:102)
file: qtbase/tests/auto/testlib/selftests/counting/tst_counting.cpp
line: 102
...
ok 8 - testSkipPass(row 1) # SKIP Skipping
ok 9 - testSkipPass(row 2)
ok 10 - testSkipSkip(row 1) # SKIP Skipping
ok 11 - testSkipSkip(row 2) # SKIP Skipping
ok 12 - testSkipFail(row 1) # SKIP Skipping
not ok 13 - testSkipFail(row 2)
---
type: QVERIFY
message: Verification failed
wanted: true (false)
found: false (false)
expected: true (false)
actual: false (false)
at: tst_Counting::testSkipFail() (qtbase/tests/auto/testlib/selftests/counting/tst_counting.cpp:102)
file: qtbase/tests/auto/testlib/selftests/counting/tst_counting.cpp
line: 102
...
not ok 14 - testFailPass(row 1)
---
type: QVERIFY
message: Verification failed
wanted: true (false)
found: false (false)
expected: true (false)
actual: false (false)
at: tst_Counting::testFailPass() (qtbase/tests/auto/testlib/selftests/counting/tst_counting.cpp:102)
file: qtbase/tests/auto/testlib/selftests/counting/tst_counting.cpp
line: 102
...
ok 15 - testFailPass(row 2)
not ok 16 - testFailSkip(row 1)
---
type: QVERIFY
message: Verification failed
wanted: true (false)
found: false (false)
expected: true (false)
actual: false (false)
at: tst_Counting::testFailSkip() (qtbase/tests/auto/testlib/selftests/counting/tst_counting.cpp:102)
file: qtbase/tests/auto/testlib/selftests/counting/tst_counting.cpp
line: 102
...
ok 17 - testFailSkip(row 2) # SKIP Skipping
not ok 18 - testFailFail(row 1)
---
type: QVERIFY
message: Verification failed
wanted: true (false)
found: false (false)
expected: true (false)
actual: false (false)
at: tst_Counting::testFailFail() (qtbase/tests/auto/testlib/selftests/counting/tst_counting.cpp:102)
file: qtbase/tests/auto/testlib/selftests/counting/tst_counting.cpp
line: 102
...
not ok 19 - testFailFail(row 2)
---
type: QVERIFY
message: Verification failed
wanted: true (false)
found: false (false)
expected: true (false)
actual: false (false)
at: tst_Counting::testFailFail() (qtbase/tests/auto/testlib/selftests/counting/tst_counting.cpp:102)
file: qtbase/tests/auto/testlib/selftests/counting/tst_counting.cpp
line: 102
...
ok 20 - testFailInInit(before)
not ok 21 - testFailInInit(fail)
---
# Fail in init()
at: tst_Counting::testFailInInit() (qtbase/tests/auto/testlib/selftests/counting/tst_counting.cpp:221)
file: qtbase/tests/auto/testlib/selftests/counting/tst_counting.cpp
line: 221
...
ok 22 - testFailInInit(after)
ok 23 - testFailInCleanup(before)
# This test function should execute and then QFAIL in cleanup()
not ok 24 - testFailInCleanup(fail)
---
# Fail in cleanup()
at: tst_Counting::testFailInCleanup() (qtbase/tests/auto/testlib/selftests/counting/tst_counting.cpp:229)
file: qtbase/tests/auto/testlib/selftests/counting/tst_counting.cpp
line: 229
...
ok 25 - testFailInCleanup(after)
ok 26 - testSkipInInit(before)
ok 27 - testSkipInInit(skip) # SKIP Skip in init()
ok 28 - testSkipInInit(after)
ok 29 - testSkipInCleanup(before)
# This test function should execute and then QSKIP in cleanup()
ok 30 - testSkipInCleanup(skip) # SKIP Skip in cleanup()
ok 31 - testSkipInCleanup(after)
ok 32 - cleanupTestCase()
1..32
# tests 32
# pass 16
# fail 8

183
tests/auto/testlib/selftests/expected_datatable.tap Normal file
View File

@ -0,0 +1,183 @@
TAP version 13
# tst_DataTable
ok 1 - initTestCase()
ok 2 - singleTestFunction1()
ok 3 - singleTestFunction2()
ok 4 - fiveTablePasses(fiveTablePasses_data 1)
ok 5 - fiveTablePasses(fiveTablePasses_data 2)
ok 6 - fiveTablePasses(fiveTablePasses_data 3)
ok 7 - fiveTablePasses(fiveTablePasses_data 4)
ok 8 - fiveTablePasses(fiveTablePasses_data 5)
not ok 9 - fiveTableFailures(fiveTableFailures_data 1)
---
type: QVERIFY
message: Verification failed
wanted: true (test)
found: false (test)
expected: true (test)
actual: false (test)
at: tst_DataTable::fiveTableFailures() (qtbase/tests/auto/testlib/selftests/datatable/tst_datatable.cpp:78)
file: qtbase/tests/auto/testlib/selftests/datatable/tst_datatable.cpp
line: 78
...
not ok 10 - fiveTableFailures(fiveTableFailures_data 2)
---
type: QVERIFY
message: Verification failed
wanted: true (test)
found: false (test)
expected: true (test)
actual: false (test)
at: tst_DataTable::fiveTableFailures() (qtbase/tests/auto/testlib/selftests/datatable/tst_datatable.cpp:78)
file: qtbase/tests/auto/testlib/selftests/datatable/tst_datatable.cpp
line: 78
...
not ok 11 - fiveTableFailures(fiveTableFailures_data 3)
---
type: QVERIFY
message: Verification failed
wanted: true (test)
found: false (test)
expected: true (test)
actual: false (test)
at: tst_DataTable::fiveTableFailures() (qtbase/tests/auto/testlib/selftests/datatable/tst_datatable.cpp:78)
file: qtbase/tests/auto/testlib/selftests/datatable/tst_datatable.cpp
line: 78
...
not ok 12 - fiveTableFailures(fiveTableFailures_data 4)
---
type: QVERIFY
message: Verification failed
wanted: true (test)
found: false (test)
expected: true (test)
actual: false (test)
at: tst_DataTable::fiveTableFailures() (qtbase/tests/auto/testlib/selftests/datatable/tst_datatable.cpp:78)
file: qtbase/tests/auto/testlib/selftests/datatable/tst_datatable.cpp
line: 78
...
not ok 13 - fiveTableFailures(fiveTableFailures_data 5)
---
type: QVERIFY
message: Verification failed
wanted: true (test)
found: false (test)
expected: true (test)
actual: false (test)
at: tst_DataTable::fiveTableFailures() (qtbase/tests/auto/testlib/selftests/datatable/tst_datatable.cpp:78)
file: qtbase/tests/auto/testlib/selftests/datatable/tst_datatable.cpp
line: 78
...
not ok 14 - startsWithFailure(startsWithFailure_data 1)
---
type: QVERIFY
message: Verification failed
wanted: true (test)
found: false (test)
expected: true (test)
actual: false (test)
at: tst_DataTable::startsWithFailure() (qtbase/tests/auto/testlib/selftests/datatable/tst_datatable.cpp:78)
file: qtbase/tests/auto/testlib/selftests/datatable/tst_datatable.cpp
line: 78
...
ok 15 - startsWithFailure(startsWithFailure_data 2)
ok 16 - startsWithFailure(startsWithFailure_data 3)
ok 17 - startsWithFailure(startsWithFailure_data 4)
ok 18 - startsWithFailure(startsWithFailure_data 5)
ok 19 - endsWithFailure(endsWithFailure 1)
ok 20 - endsWithFailure(endsWithFailure 2)
ok 21 - endsWithFailure(endsWithFailure 3)
ok 22 - endsWithFailure(endsWithFailure 4)
not ok 23 - endsWithFailure(endsWithFailure 5)
---
type: QVERIFY
message: Verification failed
wanted: true (test)
found: false (test)
expected: true (test)
actual: false (test)
at: tst_DataTable::endsWithFailure() (qtbase/tests/auto/testlib/selftests/datatable/tst_datatable.cpp:78)
file: qtbase/tests/auto/testlib/selftests/datatable/tst_datatable.cpp
line: 78
...
ok 24 - failureInMiddle(failureInMiddle_data 1)
ok 25 - failureInMiddle(failureInMiddle_data 2)
not ok 26 - failureInMiddle(failureInMiddle_data 3)
---
type: QVERIFY
message: Verification failed
wanted: true (test)
found: false (test)
expected: true (test)
actual: false (test)
at: tst_DataTable::failureInMiddle() (qtbase/tests/auto/testlib/selftests/datatable/tst_datatable.cpp:78)
file: qtbase/tests/auto/testlib/selftests/datatable/tst_datatable.cpp
line: 78
...
ok 27 - failureInMiddle(failureInMiddle_data 4)
ok 28 - failureInMiddle(failureInMiddle_data 5)
not ok 29 - fiveIsolatedFailures(fiveIsolatedFailures_data 1)
---
type: QVERIFY
message: Verification failed
wanted: true (!test)
found: false (!test)
expected: true (!test)
actual: false (!test)
at: tst_DataTable::fiveIsolatedFailures() (qtbase/tests/auto/testlib/selftests/datatable/tst_datatable.cpp:160)
file: qtbase/tests/auto/testlib/selftests/datatable/tst_datatable.cpp
line: 160
...
not ok 30 - fiveIsolatedFailures(fiveIsolatedFailures_data 2)
---
type: QVERIFY
message: Verification failed
wanted: true (!test)
found: false (!test)
expected: true (!test)
actual: false (!test)
at: tst_DataTable::fiveIsolatedFailures() (qtbase/tests/auto/testlib/selftests/datatable/tst_datatable.cpp:160)
file: qtbase/tests/auto/testlib/selftests/datatable/tst_datatable.cpp
line: 160
...
not ok 31 - fiveIsolatedFailures(fiveIsolatedFailures_data 3)
---
type: QVERIFY
message: Verification failed
wanted: true (!test)
found: false (!test)
expected: true (!test)
actual: false (!test)
at: tst_DataTable::fiveIsolatedFailures() (qtbase/tests/auto/testlib/selftests/datatable/tst_datatable.cpp:160)
file: qtbase/tests/auto/testlib/selftests/datatable/tst_datatable.cpp
line: 160
...
not ok 32 - fiveIsolatedFailures(fiveIsolatedFailures_data 4)
---
type: QVERIFY
message: Verification failed
wanted: true (!test)
found: false (!test)
expected: true (!test)
actual: false (!test)
at: tst_DataTable::fiveIsolatedFailures() (qtbase/tests/auto/testlib/selftests/datatable/tst_datatable.cpp:160)
file: qtbase/tests/auto/testlib/selftests/datatable/tst_datatable.cpp
line: 160
...
not ok 33 - fiveIsolatedFailures(fiveIsolatedFailures_data 5)
---
type: QVERIFY
message: Verification failed
wanted: true (!test)
found: false (!test)
expected: true (!test)
actual: false (!test)
at: tst_DataTable::fiveIsolatedFailures() (qtbase/tests/auto/testlib/selftests/datatable/tst_datatable.cpp:160)
file: qtbase/tests/auto/testlib/selftests/datatable/tst_datatable.cpp
line: 160
...
ok 34 - cleanupTestCase()
1..34
# tests 34
# pass 21
# fail 13

46
tests/auto/testlib/selftests/expected_datetime.tap Normal file
View File

@ -0,0 +1,46 @@
TAP version 13
# tst_DateTime
ok 1 - initTestCase()
not ok 2 - dateTime()
---
type: QCOMPARE
message: Compared values are not the same
wanted: 2000/05/03 04:03:04.000[UTC] (utc)
found: 2000/05/03 04:03:04.000[UTC+00:02] (local)
expected: 2000/05/03 04:03:04.000[UTC] (utc)
actual: 2000/05/03 04:03:04.000[UTC+00:02] (local)
at: tst_DateTime::dateTime() (qtbase/tests/auto/testlib/selftests/datetime/tst_datetime.cpp:52)
file: qtbase/tests/auto/testlib/selftests/datetime/tst_datetime.cpp
line: 52
...
ok 3 - qurl(empty urls)
not ok 4 - qurl(empty rhs)
---
type: QCOMPARE
message: Compared values are not the same
wanted: Invalid URL: (operandB)
found: http://example.com (operandA)
expected: Invalid URL: (operandB)
actual: http://example.com (operandA)
at: tst_DateTime::qurl() (qtbase/tests/auto/testlib/selftests/datetime/tst_datetime.cpp:60)
file: qtbase/tests/auto/testlib/selftests/datetime/tst_datetime.cpp
line: 60
...
not ok 5 - qurl(empty lhs)
---
type: QCOMPARE
message: Compared values are not the same
wanted: http://example.com (operandB)
found: Invalid URL: (operandA)
expected: http://example.com (operandB)
actual: Invalid URL: (operandA)
at: tst_DateTime::qurl() (qtbase/tests/auto/testlib/selftests/datetime/tst_datetime.cpp:60)
file: qtbase/tests/auto/testlib/selftests/datetime/tst_datetime.cpp
line: 60
...
ok 6 - qurl(same urls)
ok 7 - cleanupTestCase()
1..7
# tests 7
# pass 4
# fail 3

14
tests/auto/testlib/selftests/expected_exceptionthrow.tap Normal file
View File

@ -0,0 +1,14 @@
TAP version 13
# tst_Exception
ok 1 - initTestCase()
not ok 2 - throwException()
---
# Caught unhandled exception
at: tst_Exception::throwException() (qtbase/src/testlib/qtestcase.cpp:1846)
file: qtbase/src/testlib/qtestcase.cpp
line: 1846
...
1..2
# tests 2
# pass 1
# fail 1

96
tests/auto/testlib/selftests/expected_expectfail.tap Normal file
View File

@ -0,0 +1,96 @@
TAP version 13
# tst_ExpectFail
ok 1 - initTestCase()
# begin
not ok 2 - xfailAndContinue() # TODO This should xfail
---
at: tst_ExpectFail::xfailAndContinue() (qtbase/tests/auto/testlib/selftests/expectfail/tst_expectfail.cpp:65)
file: qtbase/tests/auto/testlib/selftests/expectfail/tst_expectfail.cpp
line: 65
...
# after
# begin
not ok 3 - xfailAndAbort() # TODO This should xfail
---
at: tst_ExpectFail::xfailAndAbort() (qtbase/tests/auto/testlib/selftests/expectfail/tst_expectfail.cpp:73)
file: qtbase/tests/auto/testlib/selftests/expectfail/tst_expectfail.cpp
line: 73
...
not ok 4 - xfailTwice()
---
# Already expecting a fail
at: tst_ExpectFail::xfailTwice() (qtbase/tests/auto/testlib/selftests/expectfail/tst_expectfail.cpp:83)
file: qtbase/tests/auto/testlib/selftests/expectfail/tst_expectfail.cpp
line: 83
...
not ok 5 - xfailWithQString() # TODO A string
---
at: tst_ExpectFail::xfailWithQString() (qtbase/tests/auto/testlib/selftests/expectfail/tst_expectfail.cpp:92)
file: qtbase/tests/auto/testlib/selftests/expectfail/tst_expectfail.cpp
line: 92
...
not ok 5 - xfailWithQString() # TODO Bug 5 (The message)
---
at: tst_ExpectFail::xfailWithQString() (qtbase/tests/auto/testlib/selftests/expectfail/tst_expectfail.cpp:97)
file: qtbase/tests/auto/testlib/selftests/expectfail/tst_expectfail.cpp
line: 97
...
ok 6 - xfailDataDrivenWithQVerify(Pass 1)
ok 7 - xfailDataDrivenWithQVerify(Pass 2)
not ok 8 - xfailDataDrivenWithQVerify(Abort) # TODO This test should xfail
---
at: tst_ExpectFail::xfailDataDrivenWithQVerify() (qtbase/tests/auto/testlib/selftests/expectfail/tst_expectfail.cpp:126)
file: qtbase/tests/auto/testlib/selftests/expectfail/tst_expectfail.cpp
line: 126
...
not ok 9 - xfailDataDrivenWithQVerify(Continue) # TODO This test should xfail
---
at: tst_ExpectFail::xfailDataDrivenWithQVerify() (qtbase/tests/auto/testlib/selftests/expectfail/tst_expectfail.cpp:126)
file: qtbase/tests/auto/testlib/selftests/expectfail/tst_expectfail.cpp
line: 126
...
ok 10 - xfailDataDrivenWithQCompare(Pass 1)
ok 11 - xfailDataDrivenWithQCompare(Pass 2)
not ok 12 - xfailDataDrivenWithQCompare(Abort) # TODO This test should xfail
---
at: tst_ExpectFail::xfailDataDrivenWithQCompare() (qtbase/tests/auto/testlib/selftests/expectfail/tst_expectfail.cpp:160)
file: qtbase/tests/auto/testlib/selftests/expectfail/tst_expectfail.cpp
line: 160
...
not ok 13 - xfailDataDrivenWithQCompare(Continue) # TODO This test should xfail
---
at: tst_ExpectFail::xfailDataDrivenWithQCompare() (qtbase/tests/auto/testlib/selftests/expectfail/tst_expectfail.cpp:160)
file: qtbase/tests/auto/testlib/selftests/expectfail/tst_expectfail.cpp
line: 160
...
ok 14 - xfailOnWrongRow(right row)
not ok 15 - xfailOnAnyRow(first row) # TODO This test should xfail
---
at: tst_ExpectFail::xfailOnAnyRow() (qtbase/tests/auto/testlib/selftests/expectfail/tst_expectfail.cpp:195)
file: qtbase/tests/auto/testlib/selftests/expectfail/tst_expectfail.cpp
line: 195
...
not ok 16 - xfailOnAnyRow(second row) # TODO This test should xfail
---
at: tst_ExpectFail::xfailOnAnyRow() (qtbase/tests/auto/testlib/selftests/expectfail/tst_expectfail.cpp:195)
file: qtbase/tests/auto/testlib/selftests/expectfail/tst_expectfail.cpp
line: 195
...
not ok 17 - xfailWithoutVerify(first row)
---
# QEXPECT_FAIL was called without any subsequent verification statements
...
not ok 18 - xfailWithoutVerify(second row)
---
# QEXPECT_FAIL was called without any subsequent verification statements
...
ok 19 - xpass() # TODO 'true' returned TRUE unexpectedly. ()
ok 20 - xpassDataDrivenWithQVerify(XPass) # TODO 'true' returned TRUE unexpectedly. ()
ok 21 - xpassDataDrivenWithQVerify(Pass)
ok 22 - xpassDataDrivenWithQCompare(XPass) # TODO QCOMPARE(1, 1) returned TRUE unexpectedly.
ok 23 - xpassDataDrivenWithQCompare(Pass)
ok 24 - cleanupTestCase()
1..24
# tests 24
# pass 18
# fail 6

20
tests/auto/testlib/selftests/expected_failcleanup.tap Normal file
View File

@ -0,0 +1,20 @@
TAP version 13
# tst_FailCleanup
ok 1 - initTestCase()
ok 2 - aTestFunction()
not ok 3 - cleanupTestCase()
---
type: QVERIFY
message: Fail inside cleanupTestCase
wanted: true (false)
found: false (false)
expected: true (false)
actual: false (false)
at: tst_FailCleanup::cleanupTestCase() (qtbase/tests/auto/testlib/selftests/failcleanup/tst_failcleanup.cpp:46)
file: qtbase/tests/auto/testlib/selftests/failcleanup/tst_failcleanup.cpp
line: 46
...
1..3
# tests 3
# pass 2
# fail 1

19
tests/auto/testlib/selftests/expected_failinit.tap Normal file
View File

@ -0,0 +1,19 @@
TAP version 13
# tst_FailInit
not ok 1 - initTestCase()
---
type: QVERIFY
message: Verification failed
wanted: true (false)
found: false (false)
expected: true (false)
actual: false (false)
at: tst_FailInit::initTestCase() (qtbase/tests/auto/testlib/selftests/failinit/tst_failinit.cpp:42)
file: qtbase/tests/auto/testlib/selftests/failinit/tst_failinit.cpp
line: 42
...
ok 2 - cleanupTestCase()
1..2
# tests 2
# pass 1
# fail 1

18
tests/auto/testlib/selftests/expected_failinitdata.tap Normal file
View File

@ -0,0 +1,18 @@
TAP version 13
# tst_FailInitData
not ok 1 - initTestCase()
---
type: QVERIFY
message: Verification failed
wanted: true (false)
found: false (false)
expected: true (false)
actual: false (false)
at: tst_FailInitData::initTestCase() (qtbase/tests/auto/testlib/selftests/failinitdata/tst_failinitdata.cpp:43)
file: qtbase/tests/auto/testlib/selftests/failinitdata/tst_failinitdata.cpp
line: 43
...
1..1
# tests 1
# pass 0
# fail 1

15
tests/auto/testlib/selftests/expected_fetchbogus.tap Normal file
View File

@ -0,0 +1,15 @@
TAP version 13
# tst_FetchBogus
ok 1 - initTestCase()
# QFETCH: Requested testdata 'bubu' not available, check your _data function.
not ok 2 - fetchBogus(foo)
---
# Received a fatal error.
at: tst_FetchBogus::fetchBogus() (Unknown file:0)
file: Unknown file
line: 0
...
1..2
# tests 2
# pass 1
# fail 1

10
tests/auto/testlib/selftests/expected_findtestdata.tap Normal file
View File

@ -0,0 +1,10 @@
TAP version 13
# FindTestData
ok 1 - initTestCase()
# testdata testfile could not be located!
ok 2 - paths()
ok 3 - cleanupTestCase()
1..3
# tests 3
# pass 3
# fail 0

82
tests/auto/testlib/selftests/expected_float.tap Normal file
View File

@ -0,0 +1,82 @@
TAP version 13
# tst_float
ok 1 - initTestCase()
ok 2 - floatComparisons(should SUCCEED 1)
not ok 3 - floatComparisons(should FAIL 1)
---
type: QCOMPARE
message: Compared floats are not the same (fuzzy compare)
wanted: 3 (operandRight)
found: 1 (operandLeft)
expected: 3 (operandRight)
actual: 1 (operandLeft)
at: tst_float::floatComparisons() (qtbase/tests/auto/testlib/selftests/float/tst_float.cpp:48)
file: qtbase/tests/auto/testlib/selftests/float/tst_float.cpp
line: 48
...
not ok 4 - floatComparisons(should FAIL 2)
---
type: QCOMPARE
message: Compared floats are not the same (fuzzy compare)
wanted: 3e-07 (operandRight)
found: 1e-07 (operandLeft)
expected: 3e-07 (operandRight)
actual: 1e-07 (operandLeft)
at: tst_float::floatComparisons() (qtbase/tests/auto/testlib/selftests/float/tst_float.cpp:48)
file: qtbase/tests/auto/testlib/selftests/float/tst_float.cpp
line: 48
...
not ok 5 - floatComparisons(should FAIL 3)
---
type: QCOMPARE
message: Compared floats are not the same (fuzzy compare)
wanted: 99999 (operandRight)
found: 99998 (operandLeft)
expected: 99999 (operandRight)
actual: 99998 (operandLeft)
at: tst_float::floatComparisons() (qtbase/tests/auto/testlib/selftests/float/tst_float.cpp:48)
file: qtbase/tests/auto/testlib/selftests/float/tst_float.cpp
line: 48
...
ok 6 - floatComparisons(should SUCCEED 2)
not ok 7 - compareFloatTests(1e0)
---
type: QCOMPARE
message: Compared floats are not the same (fuzzy compare)
wanted: 3 (t3)
found: 1 (t1)
expected: 3 (t3)
actual: 1 (t1)
at: tst_float::compareFloatTests() (qtbase/tests/auto/testlib/selftests/float/tst_float.cpp:96)
file: qtbase/tests/auto/testlib/selftests/float/tst_float.cpp
line: 96
...
not ok 8 - compareFloatTests(1e-7)
---
type: QCOMPARE
message: Compared floats are not the same (fuzzy compare)
wanted: 3e-07 (t3)
found: 1e-07 (t1)
expected: 3e-07 (t3)
actual: 1e-07 (t1)
at: tst_float::compareFloatTests() (qtbase/tests/auto/testlib/selftests/float/tst_float.cpp:96)
file: qtbase/tests/auto/testlib/selftests/float/tst_float.cpp
line: 96
...
not ok 9 - compareFloatTests(1e+7)
---
type: QCOMPARE
message: Compared floats are not the same (fuzzy compare)
wanted: 3e+07 (t3)
found: 1e+07 (t1)
expected: 3e+07 (t3)
actual: 1e+07 (t1)
at: tst_float::compareFloatTests() (qtbase/tests/auto/testlib/selftests/float/tst_float.cpp:96)
file: qtbase/tests/auto/testlib/selftests/float/tst_float.cpp
line: 96
...
ok 10 - cleanupTestCase()
1..10
# tests 10
# pass 4
# fail 6

52
tests/auto/testlib/selftests/expected_globaldata.tap Normal file
View File

@ -0,0 +1,52 @@
TAP version 13
# tst_globaldata
# initTestCase initTestCase (null)
ok 1 - initTestCase()
# init testGlobal local 1
# global: false
# local: false
# cleanup testGlobal local 1
ok 2 - testGlobal(1:local 1)
# init testGlobal local 2
# global: false
# local: true
# cleanup testGlobal local 2
ok 3 - testGlobal(1:local 2)
# init testGlobal local 1
# global: true
# local: false
# cleanup testGlobal local 1
ok 4 - testGlobal(2:local 1)
# init testGlobal local 2
# global: true
# local: true
# cleanup testGlobal local 2
ok 5 - testGlobal(2:local 2)
ok 6 - skip(1) # SKIP skipping
# init skipLocal local 1
ok 7 - skipLocal(1:local 1) # SKIP skipping
# cleanup skipLocal local 1
# init skipLocal local 2
ok 8 - skipLocal(1:local 2) # SKIP skipping
# cleanup skipLocal local 2
# init skipSingle local 1
# global: false local: false
# cleanup skipSingle local 1
ok 9 - skipSingle(1:local 1)
# init skipSingle local 2
# global: false local: true
# cleanup skipSingle local 2
ok 10 - skipSingle(1:local 2)
# init skipSingle local 1
ok 11 - skipSingle(2:local 1) # SKIP skipping
# cleanup skipSingle local 1
# init skipSingle local 2
# global: true local: true
# cleanup skipSingle local 2
ok 12 - skipSingle(2:local 2)
# cleanupTestCase cleanupTestCase (null)
ok 13 - cleanupTestCase()
1..13
# tests 13
# pass 9
# fail 0

23
tests/auto/testlib/selftests/expected_longstring.tap Normal file
View File

@ -0,0 +1,23 @@
TAP version 13
# tst_LongString
ok 1 - initTestCase()
not ok 2 - failWithLongString()
---
# Lorem ipsum dolor sit amet, consectetuer adipiscing elit. Aenean commodo ligula eget dolor. Aenean massa. Cum sociis natoque penatibus et magnis dis parturient montes, nascetur ridiculus mus. Donec quam felis, ultricies nec, pellentesque eu, pretium quis, sem. Nulla consequat massa quis enim. Donec pede justo, fringilla vel, aliquet nec, vulputate eget, arcu. In enim justo, rhoncus ut, imperdiet a, venenatis vitae, justo. Nullam dictum felis eu pede mollis pretium. Integer tincidunt. Cras dapibus. Vivamus elementum semper nisi. Aenean vulputate eleifend tellus. Aenean leo ligula, porttitor eu, consequat vitae, eleifend ac, enim. Aliquam lorem ante, dapibus in, viverra quis, feugiat a, tellus. Phasellus viverra nulla ut metus varius laoreet. Quisque rutrum. Aenean imperdiet. Etiam ultricies nisi vel augue. Curabitur ullamcorper ultricies nisi. Nam eget dui.
Etiam rhoncus. Maecenas tempus, tellus eget condimentum rhoncus, sem quam semper libero, sit amet adipiscing sem neque sed ipsum. Nam quam nunc, blandit vel, luctus pulvinar, hendrerit id, lorem. Maecenas nec odio et ante tincidunt tempus. Donec vitae sapien ut libero venenatis faucibus. Nullam quis ante. Etiam sit amet orci eget eros faucibus tincidunt. Duis leo. Sed fringilla mauris sit amet nibh. Donec sodales sagittis magna. Sed consequat, leo eget bibendum sodales, augue velit cursus nunc, quis gravida magna mi a libero. Fusce vulputate eleifend sapien. Vestibulum purus quam, scelerisque ut, mollis sed, nonummy id, metus. Nullam accumsan lorem in dui. Cras ultricies mi eu turpis hendrerit fringilla. Vestibulum ante ipsum primis in faucibus orci luctus et ultrices posuere cubilia Curae; In ac dui quis mi consectetuer lacinia.
Nam pretium turpis et arcu. Duis arcu tortor, suscipit eget, imperdiet nec, imperdiet iaculis, ipsum. Sed aliquam ultrices mauris. Integer ante arcu, accumsan a, consectetuer eget, posuere ut, mauris. Praesent adipiscing. Phasellus ullamcorper ipsum rutrum nunc. Nunc nonummy metus. Vestibulum volutpat pretium libero. Cras id dui. Aenean ut eros et nisl sagittis vestibulum. Nullam nulla eros, ultricies sit amet, nonummy id, imperdiet feugiat, pede. Sed lectus. Donec mollis hendrerit risus. Phasellus nec sem in justo pellentesque facilisis. Etiam imperdiet imperdiet orci. Nunc nec neque. Phasellus leo dolor, tempus non, auctor et, hendrerit quis, nisi.
Curabitur ligula sapien, tincidunt non, euismod vitae, posuere imperdiet, leo. Maecenas malesuada. Praesent congue erat at massa. Sed cursus turpis vitae tortor. Donec posuere vulputate arcu. Phasellus accumsan cursus velit. Vestibulum ante ipsum primis in faucibus orci luctus et ultrices posuere cubilia Curae; Sed aliquam, nisi quis porttitor congue, elit erat euismod orci, ac placerat dolor lectus quis orci. Phasellus consectetuer vestibulum elit. Aenean tellus metus, bibendum sed, posuere ac, mattis non, nunc. Vestibulum fringilla pede sit amet augue. In turpis. Pellentesque posuere. Praesent turpis.
Aenean posuere, tortor sed cursus feugiat, nunc augue blandit nunc, eu sollicitudin urna dolor sagittis lacus. Donec elit libero, sodales nec, volutpat a, suscipit non, turpis. Nullam sagittis. Suspendisse pulvinar, augue ac venenatis condimentum, sem libero volutpat nibh, nec pellentesque velit pede quis nunc. Vestibulum ante ipsum primis in faucibus orci luctus et ultrices posuere cubilia Curae; Fusce id purus. Ut varius tincidunt libero. Phasellus dolor. Maecenas vestibulum mollis diam. Pellentesque ut neque. Pellentesque habitant morbi tristique senectus et netus et malesuada fames ac turpis egestas.
at: tst_LongString::failWithLongString() (qtbase/tests/auto/testlib/selftests/longstring/tst_longstring.cpp:54)
file: qtbase/tests/auto/testlib/selftests/longstring/tst_longstring.cpp
line: 54
...
ok 3 - cleanupTestCase()
1..3
# tests 3
# pass 2
# fail 1

2011
tests/auto/testlib/selftests/expected_maxwarnings.tap Normal file
View File

File diff suppressed because it is too large Load Diff

32
tests/auto/testlib/selftests/expected_pairdiagnostics.tap Normal file
View File

@ -0,0 +1,32 @@
TAP version 13
# tst_PairDiagnostics
ok 1 - initTestCase()
not ok 2 - testQPair()
---
type: QCOMPARE
message: Compared values are not the same
wanted: "QPair(1,2)" (pair2)
found: "QPair(1,1)" (pair1)
expected: "QPair(1,2)" (pair2)
actual: "QPair(1,1)" (pair1)
at: tst_PairDiagnostics::testQPair() (qtbase/tests/auto/testlib/selftests/pairdiagnostics/tst_pairdiagnostics.cpp:51)
file: qtbase/tests/auto/testlib/selftests/pairdiagnostics/tst_pairdiagnostics.cpp
line: 51
...
not ok 3 - testStdPair()
---
type: QCOMPARE
message: Compared values are not the same
wanted: "std::pair(1,2)" (pair2)
found: "std::pair(1,1)" (pair1)
expected: "std::pair(1,2)" (pair2)
actual: "std::pair(1,1)" (pair1)
at: tst_PairDiagnostics::testStdPair() (qtbase/tests/auto/testlib/selftests/pairdiagnostics/tst_pairdiagnostics.cpp:58)
file: qtbase/tests/auto/testlib/selftests/pairdiagnostics/tst_pairdiagnostics.cpp
line: 58
...
ok 4 - cleanupTestCase()
1..4
# tests 4
# pass 2
# fail 2

9
tests/auto/testlib/selftests/expected_singleskip.tap Normal file
View File

@ -0,0 +1,9 @@
TAP version 13
# tst_SingleSkip
ok 1 - initTestCase()
ok 2 - myTest() # SKIP skipping test
ok 3 - cleanupTestCase()
1..3
# tests 3
# pass 2
# fail 0

13
tests/auto/testlib/selftests/expected_skip.tap Normal file
View File

@ -0,0 +1,13 @@
TAP version 13
# tst_Skip
ok 1 - initTestCase()
ok 2 - test() # SKIP skipping all
ok 3 - emptytest() # SKIP skipping all
ok 4 - singleSkip(local 1) # SKIP skipping one
# this line should only be reached once (true)
ok 5 - singleSkip(local 2)
ok 6 - cleanupTestCase()
1..6
# tests 6
# pass 3
# fail 0

9
tests/auto/testlib/selftests/expected_skipcleanup.tap Normal file
View File

@ -0,0 +1,9 @@
TAP version 13
# tst_SkipCleanup
ok 1 - initTestCase()
ok 2 - aTestFunction()
ok 3 - cleanupTestCase() # SKIP Skip inside cleanupTestCase.
1..3
# tests 3
# pass 2
# fail 0

8
tests/auto/testlib/selftests/expected_skipinit.tap Normal file
View File

@ -0,0 +1,8 @@
TAP version 13
# tst_SkipInit
ok 1 - initTestCase() # SKIP Skip inside initTestCase. This should skip all tests in the class.
ok 2 - cleanupTestCase()
1..2
# tests 2
# pass 1
# fail 0

7
tests/auto/testlib/selftests/expected_skipinitdata.tap Normal file
View File

@ -0,0 +1,7 @@
TAP version 13
# tst_SkipInitData
ok 1 - initTestCase() # SKIP Skip inside initTestCase_data. This should skip all tests in the class.
1..1
# tests 1
# pass 0
# fail 0

9
tests/auto/testlib/selftests/expected_sleep.tap Normal file
View File

@ -0,0 +1,9 @@
TAP version 13
# tst_Sleep
ok 1 - initTestCase()
ok 2 - sleep()
ok 3 - cleanupTestCase()
1..3
# tests 3
# pass 3
# fail 0

87
tests/auto/testlib/selftests/expected_strcmp.tap Normal file
View File

@ -0,0 +1,87 @@
TAP version 13
# tst_StrCmp
ok 1 - initTestCase()
ok 2 - compareCharStars()
not ok 3 - compareByteArray() # TODO Next test should fail
---
at: tst_StrCmp::compareByteArray() (qtbase/tests/auto/testlib/selftests/strcmp/tst_strcmp.cpp:75)
file: qtbase/tests/auto/testlib/selftests/strcmp/tst_strcmp.cpp
line: 75
...
not ok 3 - compareByteArray() # TODO Next test should fail
---
at: tst_StrCmp::compareByteArray() (qtbase/tests/auto/testlib/selftests/strcmp/tst_strcmp.cpp:82)
file: qtbase/tests/auto/testlib/selftests/strcmp/tst_strcmp.cpp
line: 82
...
not ok 3 - compareByteArray() # TODO Next test should fail
---
at: tst_StrCmp::compareByteArray() (qtbase/tests/auto/testlib/selftests/strcmp/tst_strcmp.cpp:89)
file: qtbase/tests/auto/testlib/selftests/strcmp/tst_strcmp.cpp
line: 89
...
not ok 3 - compareByteArray()
---
type: QCOMPARE
message: Compared values are not the same
wanted: "AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA"... (b)
found: "BBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBB"... (a)
expected: "AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA"... (b)
actual: "BBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBB"... (a)
at: tst_StrCmp::compareByteArray() (qtbase/tests/auto/testlib/selftests/strcmp/tst_strcmp.cpp:96)
file: qtbase/tests/auto/testlib/selftests/strcmp/tst_strcmp.cpp
line: 96
...
not ok 4 - failByteArray()
---
type: QCOMPARE
message: Compared values are not the same
wanted: "cba" (QByteArray("cba"))
found: "abc" (QByteArray("abc"))
expected: "cba" (QByteArray("cba"))
actual: "abc" (QByteArray("abc"))
at: tst_StrCmp::failByteArray() (qtbase/tests/auto/testlib/selftests/strcmp/tst_strcmp.cpp:102)
file: qtbase/tests/auto/testlib/selftests/strcmp/tst_strcmp.cpp
line: 102
...
not ok 5 - failByteArrayNull()
---
type: QCOMPARE
message: Compared values are not the same
wanted: "" (QByteArray())
found: "foo" (QByteArray("foo"))
expected: "" (QByteArray())
actual: "foo" (QByteArray("foo"))
at: tst_StrCmp::failByteArrayNull() (qtbase/tests/auto/testlib/selftests/strcmp/tst_strcmp.cpp:108)
file: qtbase/tests/auto/testlib/selftests/strcmp/tst_strcmp.cpp
line: 108
...
not ok 6 - failByteArrayEmpty()
---
type: QCOMPARE
message: Compared values are not the same
wanted: "foo" (QByteArray("foo"))
found: "" (QByteArray(""))
expected: "foo" (QByteArray("foo"))
actual: "" (QByteArray(""))
at: tst_StrCmp::failByteArrayEmpty() (qtbase/tests/auto/testlib/selftests/strcmp/tst_strcmp.cpp:113)
file: qtbase/tests/auto/testlib/selftests/strcmp/tst_strcmp.cpp
line: 113
...
not ok 7 - failByteArraySingleChars()
---
type: QCOMPARE
message: Compared values are not the same
wanted: "7" (QByteArray("7"))
found: "6" (QByteArray("6"))
expected: "7" (QByteArray("7"))
actual: "6" (QByteArray("6"))
at: tst_StrCmp::failByteArraySingleChars() (qtbase/tests/auto/testlib/selftests/strcmp/tst_strcmp.cpp:120)
file: qtbase/tests/auto/testlib/selftests/strcmp/tst_strcmp.cpp
line: 120
...
ok 8 - cleanupTestCase()
1..8
# tests 8
# pass 3
# fail 5

68
tests/auto/testlib/selftests/expected_subtest.tap Normal file
View File

@ -0,0 +1,68 @@
TAP version 13
# tst_Subtest
# initTestCase initTestCase (null)
ok 1 - initTestCase()
# init test1 (null)
# test1 test1 (null)
# cleanup test1 (null)
ok 2 - test1()
# test2_data test2 (null)
# test2_data end
# init test2 data0
# test2 test2 data0
# test2 end
# cleanup test2 data0
ok 3 - test2(data0)
# init test2 data1
# test2 test2 data1
# test2 end
# cleanup test2 data1
ok 4 - test2(data1)
# init test2 data2
# test2 test2 data2
# test2 end
# cleanup test2 data2
ok 5 - test2(data2)
# test3_data test3 (null)
# test3_data end
# init test3 data0
# test2 test3 data0
# test2 end
# cleanup test3 data0
ok 6 - test3(data0)
# init test3 data1
# test2 test3 data1
not ok 7 - test3(data1)
---
type: QCOMPARE
message: Compared values are not the same
wanted: "hello0" (QString("hello0"))
found: "hello1" (str)
expected: "hello0" (QString("hello0"))
actual: "hello1" (str)
at: tst_Subtest::test3() (qtbase/tests/auto/testlib/selftests/subtest/tst_subtest.cpp:141)
file: qtbase/tests/auto/testlib/selftests/subtest/tst_subtest.cpp
line: 141
...
# cleanup test3 data1
# init test3 data2
# test2 test3 data2
not ok 8 - test3(data2)
---
type: QCOMPARE
message: Compared values are not the same
wanted: "hello0" (QString("hello0"))
found: "hello2" (str)
expected: "hello0" (QString("hello0"))
actual: "hello2" (str)
at: tst_Subtest::test3() (qtbase/tests/auto/testlib/selftests/subtest/tst_subtest.cpp:141)
file: qtbase/tests/auto/testlib/selftests/subtest/tst_subtest.cpp
line: 141
...
# cleanup test3 data2
# cleanupTestCase cleanupTestCase (null)
ok 9 - cleanupTestCase()
1..9
# tests 9
# pass 7
# fail 2

118
tests/auto/testlib/selftests/expected_verbose1.tap Normal file
View File

@ -0,0 +1,118 @@
TAP version 13
# tst_Counting
ok 1 - initTestCase()
ok 2 - testPassPass(row 1)
ok 3 - testPassPass(row 2)
ok 4 - testPassSkip(row 1)
ok 5 - testPassSkip(row 2) # SKIP Skipping
ok 6 - testPassFail(row 1)
not ok 7 - testPassFail(row 2)
---
type: QVERIFY
message: Verification failed
wanted: true (false)
found: false (false)
expected: true (false)
actual: false (false)
at: tst_Counting::testPassFail() (qtbase/tests/auto/testlib/selftests/counting/tst_counting.cpp:102)
file: qtbase/tests/auto/testlib/selftests/counting/tst_counting.cpp
line: 102
...
ok 8 - testSkipPass(row 1) # SKIP Skipping
ok 9 - testSkipPass(row 2)
ok 10 - testSkipSkip(row 1) # SKIP Skipping
ok 11 - testSkipSkip(row 2) # SKIP Skipping
ok 12 - testSkipFail(row 1) # SKIP Skipping
not ok 13 - testSkipFail(row 2)
---
type: QVERIFY
message: Verification failed
wanted: true (false)
found: false (false)
expected: true (false)
actual: false (false)
at: tst_Counting::testSkipFail() (qtbase/tests/auto/testlib/selftests/counting/tst_counting.cpp:102)
file: qtbase/tests/auto/testlib/selftests/counting/tst_counting.cpp
line: 102
...
not ok 14 - testFailPass(row 1)
---
type: QVERIFY
message: Verification failed
wanted: true (false)
found: false (false)
expected: true (false)
actual: false (false)
at: tst_Counting::testFailPass() (qtbase/tests/auto/testlib/selftests/counting/tst_counting.cpp:102)
file: qtbase/tests/auto/testlib/selftests/counting/tst_counting.cpp
line: 102
...
ok 15 - testFailPass(row 2)
not ok 16 - testFailSkip(row 1)
---
type: QVERIFY
message: Verification failed
wanted: true (false)
found: false (false)
expected: true (false)
actual: false (false)
at: tst_Counting::testFailSkip() (qtbase/tests/auto/testlib/selftests/counting/tst_counting.cpp:102)
file: qtbase/tests/auto/testlib/selftests/counting/tst_counting.cpp
line: 102
...
ok 17 - testFailSkip(row 2) # SKIP Skipping
not ok 18 - testFailFail(row 1)
---
type: QVERIFY
message: Verification failed
wanted: true (false)
found: false (false)
expected: true (false)
actual: false (false)
at: tst_Counting::testFailFail() (qtbase/tests/auto/testlib/selftests/counting/tst_counting.cpp:102)
file: qtbase/tests/auto/testlib/selftests/counting/tst_counting.cpp
line: 102
...
not ok 19 - testFailFail(row 2)
---
type: QVERIFY
message: Verification failed
wanted: true (false)
found: false (false)
expected: true (false)
actual: false (false)
at: tst_Counting::testFailFail() (qtbase/tests/auto/testlib/selftests/counting/tst_counting.cpp:102)
file: qtbase/tests/auto/testlib/selftests/counting/tst_counting.cpp
line: 102
...
ok 20 - testFailInInit(before)
not ok 21 - testFailInInit(fail)
---
# Fail in init()
at: tst_Counting::testFailInInit() (qtbase/tests/auto/testlib/selftests/counting/tst_counting.cpp:221)
file: qtbase/tests/auto/testlib/selftests/counting/tst_counting.cpp
line: 221
...
ok 22 - testFailInInit(after)
ok 23 - testFailInCleanup(before)
# This test function should execute and then QFAIL in cleanup()
not ok 24 - testFailInCleanup(fail)
---
# Fail in cleanup()
at: tst_Counting::testFailInCleanup() (qtbase/tests/auto/testlib/selftests/counting/tst_counting.cpp:229)
file: qtbase/tests/auto/testlib/selftests/counting/tst_counting.cpp
line: 229
...
ok 25 - testFailInCleanup(after)
ok 26 - testSkipInInit(before)
ok 27 - testSkipInInit(skip) # SKIP Skip in init()
ok 28 - testSkipInInit(after)
ok 29 - testSkipInCleanup(before)
# This test function should execute and then QSKIP in cleanup()
ok 30 - testSkipInCleanup(skip) # SKIP Skip in cleanup()
ok 31 - testSkipInCleanup(after)
ok 32 - cleanupTestCase()
1..32
# tests 32
# pass 16
# fail 8

136
tests/auto/testlib/selftests/expected_verbose2.tap Normal file
View File

@ -0,0 +1,136 @@
TAP version 13
# tst_Counting
ok 1 - initTestCase()
# QVERIFY(true)
# QCOMPARE(2 + 1, 3)
ok 2 - testPassPass(row 1)
# QVERIFY(true)
# QCOMPARE(2 + 1, 3)
ok 3 - testPassPass(row 2)
# QVERIFY(true)
# QCOMPARE(2 + 1, 3)
ok 4 - testPassSkip(row 1)
ok 5 - testPassSkip(row 2) # SKIP Skipping
# QVERIFY(true)
# QCOMPARE(2 + 1, 3)
ok 6 - testPassFail(row 1)
# QVERIFY(false)
not ok 7 - testPassFail(row 2)
---
type: QVERIFY
message: Verification failed
wanted: true (false)
found: false (false)
expected: true (false)
actual: false (false)
at: tst_Counting::testPassFail() (qtbase/tests/auto/testlib/selftests/counting/tst_counting.cpp:102)
file: qtbase/tests/auto/testlib/selftests/counting/tst_counting.cpp
line: 102
...
ok 8 - testSkipPass(row 1) # SKIP Skipping
# QVERIFY(true)
# QCOMPARE(2 + 1, 3)
ok 9 - testSkipPass(row 2)
ok 10 - testSkipSkip(row 1) # SKIP Skipping
ok 11 - testSkipSkip(row 2) # SKIP Skipping
ok 12 - testSkipFail(row 1) # SKIP Skipping
# QVERIFY(false)
not ok 13 - testSkipFail(row 2)
---
type: QVERIFY
message: Verification failed
wanted: true (false)
found: false (false)
expected: true (false)
actual: false (false)
at: tst_Counting::testSkipFail() (qtbase/tests/auto/testlib/selftests/counting/tst_counting.cpp:102)
file: qtbase/tests/auto/testlib/selftests/counting/tst_counting.cpp
line: 102
...
# QVERIFY(false)
not ok 14 - testFailPass(row 1)
---
type: QVERIFY
message: Verification failed
wanted: true (false)
found: false (false)
expected: true (false)
actual: false (false)
at: tst_Counting::testFailPass() (qtbase/tests/auto/testlib/selftests/counting/tst_counting.cpp:102)
file: qtbase/tests/auto/testlib/selftests/counting/tst_counting.cpp
line: 102
...
# QVERIFY(true)
# QCOMPARE(2 + 1, 3)
ok 15 - testFailPass(row 2)
# QVERIFY(false)
not ok 16 - testFailSkip(row 1)
---
type: QVERIFY
message: Verification failed
wanted: true (false)
found: false (false)
expected: true (false)
actual: false (false)
at: tst_Counting::testFailSkip() (qtbase/tests/auto/testlib/selftests/counting/tst_counting.cpp:102)
file: qtbase/tests/auto/testlib/selftests/counting/tst_counting.cpp
line: 102
...
ok 17 - testFailSkip(row 2) # SKIP Skipping
# QVERIFY(false)
not ok 18 - testFailFail(row 1)
---
type: QVERIFY
message: Verification failed
wanted: true (false)
found: false (false)
expected: true (false)
actual: false (false)
at: tst_Counting::testFailFail() (qtbase/tests/auto/testlib/selftests/counting/tst_counting.cpp:102)
file: qtbase/tests/auto/testlib/selftests/counting/tst_counting.cpp
line: 102
...
# QVERIFY(false)
not ok 19 - testFailFail(row 2)
---
type: QVERIFY
message: Verification failed
wanted: true (false)
found: false (false)
expected: true (false)
actual: false (false)
at: tst_Counting::testFailFail() (qtbase/tests/auto/testlib/selftests/counting/tst_counting.cpp:102)
file: qtbase/tests/auto/testlib/selftests/counting/tst_counting.cpp
line: 102
...
ok 20 - testFailInInit(before)
not ok 21 - testFailInInit(fail)
---
# Fail in init()
at: tst_Counting::testFailInInit() (qtbase/tests/auto/testlib/selftests/counting/tst_counting.cpp:221)
file: qtbase/tests/auto/testlib/selftests/counting/tst_counting.cpp
line: 221
...
ok 22 - testFailInInit(after)
ok 23 - testFailInCleanup(before)
# This test function should execute and then QFAIL in cleanup()
not ok 24 - testFailInCleanup(fail)
---
# Fail in cleanup()
at: tst_Counting::testFailInCleanup() (qtbase/tests/auto/testlib/selftests/counting/tst_counting.cpp:229)
file: qtbase/tests/auto/testlib/selftests/counting/tst_counting.cpp
line: 229
...
ok 25 - testFailInCleanup(after)
ok 26 - testSkipInInit(before)
ok 27 - testSkipInInit(skip) # SKIP Skip in init()
ok 28 - testSkipInInit(after)
ok 29 - testSkipInCleanup(before)
# This test function should execute and then QSKIP in cleanup()
ok 30 - testSkipInCleanup(skip) # SKIP Skip in cleanup()
ok 31 - testSkipInCleanup(after)
ok 32 - cleanupTestCase()
1..32
# tests 32
# pass 16
# fail 8

53
tests/auto/testlib/selftests/expected_verifyexceptionthrown.tap Normal file
View File

@ -0,0 +1,53 @@
TAP version 13
# tst_VerifyExceptionThrown
ok 1 - initTestCase()
ok 2 - testCorrectStdTypes()
ok 3 - testCorrectStdExceptions()
ok 4 - testCorrectMyExceptions()
not ok 5 - testFailInt()
---
# Expected exception of type double to be thrown but unknown exception caught
at: tst_VerifyExceptionThrown::testFailInt() (qtbase/tests/auto/testlib/selftests/verifyexceptionthrown/tst_verifyexceptionthrown.cpp:115)
file: qtbase/tests/auto/testlib/selftests/verifyexceptionthrown/tst_verifyexceptionthrown.cpp
line: 115
...
not ok 6 - testFailStdString()
---
# Expected exception of type char* to be thrown but unknown exception caught
at: tst_VerifyExceptionThrown::testFailStdString() (qtbase/tests/auto/testlib/selftests/verifyexceptionthrown/tst_verifyexceptionthrown.cpp:120)
file: qtbase/tests/auto/testlib/selftests/verifyexceptionthrown/tst_verifyexceptionthrown.cpp
line: 120
...
not ok 7 - testFailStdRuntimeError()
---
# Expected exception of type std::runtime_error to be thrown but std::exception caught with message: logic error
at: tst_VerifyExceptionThrown::testFailStdRuntimeError() (qtbase/tests/auto/testlib/selftests/verifyexceptionthrown/tst_verifyexceptionthrown.cpp:125)
file: qtbase/tests/auto/testlib/selftests/verifyexceptionthrown/tst_verifyexceptionthrown.cpp
line: 125
...
not ok 8 - testFailMyException()
---
# Expected exception of type MyBaseException to be thrown but std::exception caught with message: logic error
at: tst_VerifyExceptionThrown::testFailMyException() (qtbase/tests/auto/testlib/selftests/verifyexceptionthrown/tst_verifyexceptionthrown.cpp:130)
file: qtbase/tests/auto/testlib/selftests/verifyexceptionthrown/tst_verifyexceptionthrown.cpp
line: 130
...
not ok 9 - testFailMyDerivedException()
---
# Expected exception of type std::runtime_error to be thrown but std::exception caught with message: MyDerivedException
at: tst_VerifyExceptionThrown::testFailMyDerivedException() (qtbase/tests/auto/testlib/selftests/verifyexceptionthrown/tst_verifyexceptionthrown.cpp:135)
file: qtbase/tests/auto/testlib/selftests/verifyexceptionthrown/tst_verifyexceptionthrown.cpp
line: 135
...
not ok 10 - testFailNoException()
---
# Expected exception of type std::exception to be thrown but no exception caught
at: tst_VerifyExceptionThrown::testFailNoException() (qtbase/tests/auto/testlib/selftests/verifyexceptionthrown/tst_verifyexceptionthrown.cpp:140)
file: qtbase/tests/auto/testlib/selftests/verifyexceptionthrown/tst_verifyexceptionthrown.cpp
line: 140
...
ok 11 - cleanupTestCase()
1..11
# tests 11
# pass 5
# fail 6

42
tests/auto/testlib/selftests/expected_warnings.tap Normal file
View File

@ -0,0 +1,42 @@
TAP version 13
# tst_Warnings
ok 1 - initTestCase()
# Warning
# Warning
# Debug
# Debug
# Info
# Info
# Baba
# Baba
# Bubublabla
# Babablabla
ok 2 - testWarnings()
# Did not receive message: "Warning0"
# Did not receive message: "Warning1"
not ok 3 - testMissingWarnings()
---
# Not all expected messages were received
...
# Did not receive any message matching: "Warning\s\d"
not ok 4 - testMissingWarningsRegularExpression()
---
# Not all expected messages were received
...
# Did not receive message: "Warning0"
# Did not receive message: "Warning1"
not ok 5 - testMissingWarningsWithData(first row)
---
# Not all expected messages were received
...
# Did not receive message: "Warning0"
# Did not receive message: "Warning1"
not ok 6 - testMissingWarningsWithData(second row)
---
# Not all expected messages were received
...
ok 7 - cleanupTestCase()
1..7
# tests 7
# pass 3
# fail 4

44
tests/auto/testlib/selftests/expected_xunit.tap Normal file
View File

@ -0,0 +1,44 @@
TAP version 13
# tst_Xunit
ok 1 - initTestCase()
# just a QWARN() !
ok 2 - testFunc1()
# a qDebug() call with comment-ending stuff -->
not ok 3 - testFunc2()
---
type: QCOMPARE
message: Compared values are not the same
wanted: 3 (3)
found: 2 (2)
expected: 3 (3)
actual: 2 (2)
at: tst_Xunit::testFunc2() (qtbase/tests/auto/testlib/selftests/xunit/tst_xunit.cpp:61)
file: qtbase/tests/auto/testlib/selftests/xunit/tst_xunit.cpp
line: 61
...
ok 4 - testFunc3() # SKIP skipping this function!
not ok 5 - testFunc4()
---
# a forced failure!
at: tst_Xunit::testFunc4() (qtbase/tests/auto/testlib/selftests/xunit/tst_xunit.cpp:71)
file: qtbase/tests/auto/testlib/selftests/xunit/tst_xunit.cpp
line: 71
...
not ok 6 - testFunc5() # TODO this failure is expected
---
at: tst_Xunit::testFunc5() (qtbase/tests/auto/testlib/selftests/xunit/tst_xunit.cpp:85)
file: qtbase/tests/auto/testlib/selftests/xunit/tst_xunit.cpp
line: 85
...
not ok 7 - testFunc6() # TODO this failure is also expected
---
at: tst_Xunit::testFunc6() (qtbase/tests/auto/testlib/selftests/xunit/tst_xunit.cpp:91)
file: qtbase/tests/auto/testlib/selftests/xunit/tst_xunit.cpp
line: 91
...
ok 8 - testFunc7() # TODO 'true' returned TRUE unexpectedly. ()
ok 9 - cleanupTestCase()
1..9
# tests 9
# pass 5
# fail 3

2
tests/auto/testlib/selftests/generate_expected_output.py
View File

@ -212,7 +212,7 @@ class Scanner (object):
del re del re
def generateTestData(testname, clean, def generateTestData(testname, clean,
formats = ('xml', 'txt', 'xunitxml', 'lightxml', 'teamcity'), formats = ('xml', 'txt', 'xunitxml', 'lightxml', 'teamcity', 'tap'),
extraArgs = { extraArgs = {
"commandlinedata": "fiveTablePasses fiveTablePasses:fiveTablePasses_data1 -v2", "commandlinedata": "fiveTablePasses fiveTablePasses:fiveTablePasses_data1 -v2",
"benchlibcallgrind": "-callgrind", "benchlibcallgrind": "-callgrind",

24
tests/auto/testlib/selftests/tst_selftests.cpp
View File

@ -310,6 +310,14 @@ QList<LoggerSet> tst_Selftests::allLoggerSets() const
QStringList() << "teamcity", QStringList() << "teamcity",
QStringList() << "-teamcity" << "-o" << logName("teamcity") QStringList() << "-teamcity" << "-o" << logName("teamcity")
) )
<< LoggerSet("old stdout tap",
QStringList() << "stdout tap",
QStringList() << "-tap"
)
<< LoggerSet("old tap",
QStringList() << "tap",
QStringList() << "-tap" << "-o" << logName("tap")
)
// Test with new-style options for a single logger // Test with new-style options for a single logger
<< LoggerSet("new stdout txt", << LoggerSet("new stdout txt",
QStringList() << "stdout txt", QStringList() << "stdout txt",
@ -357,6 +365,14 @@ QList<LoggerSet> tst_Selftests::allLoggerSets() const
QStringList() << "teamcity", QStringList() << "teamcity",
QStringList() << "-o" << logName("teamcity")+",teamcity" QStringList() << "-o" << logName("teamcity")+",teamcity"
) )
<< LoggerSet("new stdout tap",
QStringList() << "stdout tap",
QStringList() << "-o" << "-,tap"
)
<< LoggerSet("new tap",
QStringList() << "tap",
QStringList() << "-o" << logName("tap")+",tap"
)
// Test with two loggers (don't test all 32 combinations, just a sample) // Test with two loggers (don't test all 32 combinations, just a sample)
<< LoggerSet("stdout txt + txt", << LoggerSet("stdout txt + txt",
QStringList() << "stdout txt" << "txt", QStringList() << "stdout txt" << "txt",
@ -380,13 +396,14 @@ QList<LoggerSet> tst_Selftests::allLoggerSets() const
) )
// All loggers at the same time (except csv) // All loggers at the same time (except csv)
<< LoggerSet("all loggers", << LoggerSet("all loggers",
QStringList() << "txt" << "xml" << "lightxml" << "stdout txt" << "xunitxml", QStringList() << "txt" << "xml" << "lightxml" << "stdout txt" << "xunitxml" << "tap",
QStringList() << "-o" << logName("txt")+",txt" QStringList() << "-o" << logName("txt")+",txt"
<< "-o" << logName("xml")+",xml" << "-o" << logName("xml")+",xml"
<< "-o" << logName("lightxml")+",lightxml" << "-o" << logName("lightxml")+",lightxml"
<< "-o" << "-,txt" << "-o" << "-,txt"
<< "-o" << logName("xunitxml")+",xunitxml" << "-o" << logName("xunitxml")+",xunitxml"
<< "-o" << logName("teamcity")+",teamcity" << "-o" << logName("teamcity")+",teamcity"
<< "-o" << logName("tap")+",tap"
) )
; ;
} }
@ -851,6 +868,11 @@ bool tst_Selftests::compareOutput(const QString &logger, const QString &subdir,
expectedLine.replace(teamcityLocRegExp, teamCityLocation()); expectedLine.replace(teamcityLocRegExp, teamCityLocation());
} }
if (logger.endsWith(QLatin1String("tap"))) {
if (expectedLine.contains(QLatin1String("at:")) || expectedLine.contains(QLatin1String("file:")))
actualLine = expectedLine;
}
if (!compareLine(logger, subdir, benchmark, actualLine, if (!compareLine(logger, subdir, benchmark, actualLine,
expectedLine, errorMessage)) { expectedLine, errorMessage)) {
errorMessage->prepend(QLatin1String("Line ") + QString::number(i + 1) errorMessage->prepend(QLatin1String("Line ") + QString::number(i + 1)