Commit Graph

11 Commits

Author SHA1 Message Date
Thomas Van Lenten
df4e4d8717 Fix up the conformance tests
- Update all the new runner cases to handle a skip result as a skip and not a failure.
- Make ObjC use the new serialize_error result if it fails to generate the data.
2016-01-05 16:31:40 -05:00
Thomas Van Lenten
de3c56a0d1 Fix conformance test to only report skips in verbose mode.
commit e841bac4fc seems to have mis-merged the
change to reporting skipped tests.

I didn't check if there were other merge issues.
2016-01-05 15:00:51 -05:00
Feng Xiao
a0cecfd31a Make conformance tests build for C++ and Java.
Change-Id: Ibb3fe6f919cc7ca0df91da5e1697ba33d259e433
2015-12-16 13:38:10 -08:00
Feng Xiao
e841bac4fc Down-integrate from internal code base. 2015-12-11 17:10:28 -08:00
Josh Haberman
325392dd61 Conformance test implementation for Python. 2015-12-02 12:53:42 -08:00
Josh Haberman
b0500b37b2 Added support for Json and valid input to conformance tests.
This was enabled by the recent open-sourcing of JSON
support and MessageDifferencer.

MessageDifferencer allows the conformance suite to expand
because it allows us to write tests for payloads that parse
successfully.  To verify the testee's output payload, we
need to parse it back into a message and compare the message
instances.  Comparing output bytes vs. a golden message is
*not* valid, because protobufs do not have a canonical
encoding (especially in the presence of maps, which have
no prescribed serialization order).

We only add one small JSON test for now, but with the
framework in place we now have the foundation to dramatically
expand the coverage of the conformance test suite.

Also added the ability for the testee to skip tests that
exercise features that are unimplemented.  This allows
Java (which currently has no JSON support) to skip tests
involving JSON.

Change-Id: I697b4363da432b61ae3b638b4287c4cda1af4deb
2015-07-10 16:36:59 -07:00
Josh Haberman
23bf3b566f Removed test_name from conformance.proto.
Change-Id: I382dcda97fa123a6da4ff5faad5d7ece95853f33
2015-06-04 15:04:00 -07:00
Josh Haberman
d2b67389b3 Conformance tests can now be excluded based on their names.
This allows us to enable conformance tests even when we know
that some tests are failing and need to be fixed.

Change-Id: I372f43663008747db6f2b2cf06e6ffa4c6d85b2d
2015-06-03 12:08:08 -07:00
Bo Yang
5db217305f down-integrate internal changes 2015-05-21 19:32:02 -07:00
Josh Haberman
4e63b52088 Factored conformance tests so they can run in-process.
This is necessary for testing on iOS.

Change-Id: I54ec1e3aa2e9fbfff9a6cd6580920a6a62069b63
2015-04-15 14:56:05 -07:00
Josh Haberman
35a1cc7a7c Added first version of conformance tests.
Change-Id: Ib75664194491643f8e4f1503a2ed942a2d1e1655
2015-04-08 13:14:10 -07:00