Commit Graph

6 Commits

Author SHA1 Message Date
Josh Haberman
ef7894e2dc Make conformance tests more strict about the failure list.
The failure lists were falling out of date because the
tests would pass even if people forgot to remove failures
from the list.
2016-06-03 09:39:38 -07:00
Josh Haberman
e891c29f9b Allow conformance test runner to tolerate crashes, and re-enable conformance tests. 2016-01-11 16:15:46 -08:00
Feng Xiao
e841bac4fc Down-integrate from internal code base. 2015-12-11 17:10:28 -08:00
Josh Haberman
b0500b37b2 Added support for Json and valid input to conformance tests.
This was enabled by the recent open-sourcing of JSON
support and MessageDifferencer.

MessageDifferencer allows the conformance suite to expand
because it allows us to write tests for payloads that parse
successfully.  To verify the testee's output payload, we
need to parse it back into a message and compare the message
instances.  Comparing output bytes vs. a golden message is
*not* valid, because protobufs do not have a canonical
encoding (especially in the presence of maps, which have
no prescribed serialization order).

We only add one small JSON test for now, but with the
framework in place we now have the foundation to dramatically
expand the coverage of the conformance test suite.

Also added the ability for the testee to skip tests that
exercise features that are unimplemented.  This allows
Java (which currently has no JSON support) to skip tests
involving JSON.

Change-Id: I697b4363da432b61ae3b638b4287c4cda1af4deb
2015-07-10 16:36:59 -07:00
Josh Haberman
23bf3b566f Removed test_name from conformance.proto.
Change-Id: I382dcda97fa123a6da4ff5faad5d7ece95853f33
2015-06-04 15:04:00 -07:00
Josh Haberman
d2b67389b3 Conformance tests can now be excluded based on their names.
This allows us to enable conformance tests even when we know
that some tests are failing and need to be fixed.

Change-Id: I372f43663008747db6f2b2cf06e6ffa4c6d85b2d
2015-06-03 12:08:08 -07:00