protobuf/conformance
Joshua Haberman 9abf6e2ab0
Ported Ruby extension to upb_msg (#8184)
* WIP.

* WIP.

* WIP.

* WIP.

* WIP.

* WIP.

* Added some missing files.

* WIP.

* WIP.

* Updated upb.

* Extension loads, but crashes immediately.

* Gets through the test suite without SEGV!

Still a lot of bugs to fix, but it is a major step!

214 tests, 378 assertions, 37 failures, 147 errors, 0 pendings, 0 omissions, 0 notifications
14.0187% passed

* Test and build for Ruby 3.0

* Fixed a few more bugs, efficient #inspect is almost done.

214 tests, 134243 assertions, 30 failures, 144 errors, 0 pendings, 0 omissions, 0 notifications
18.6916% passed

* Fixed message hash initialization and encode depth checking.

214 tests, 124651 assertions, 53 failures, 70 errors, 0 pendings, 0 omissions, 0 notifications
42.5234% passed

* A bunch of fixes to failing tests, now 70% passing.

214 tests, 202091 assertions, 41 failures, 23 errors, 0 pendings, 0 omissions, 0 notifications
70.0935% passed

* More than 80% of tests are passing now.

214 tests, 322331 assertions, 30 failures, 9 errors, 0 pendings, 0 omissions, 0 notifications
81.7757% passed

Unfortunately there is also a sporadic bug/segfault hanging around
that appears to be GC-related.

* Add linux/ruby30 and macos/ruby30

* Use rvm master for 3.0.0-preview2

* Over 90% of tests are passing!

214 tests, 349898 assertions, 15 failures, 1 errors, 0 pendings, 0 omissions, 0 notifications
92.5234% passed

* Passes all tests!

214 tests, 369388 assertions, 0 failures, 0 errors, 0 pendings, 0 omissions, 0 notifications
100% passed

* A bunch of cleanup.

1. Removed a bunch of internal-only symbols from headers.
2. Required a frozen check to get a non-const pointer to a map or array.
3. De-duplicated the code to get a type argument for Map/RepeatedField.

* Removed a bunch more stuff from protobuf.h.  There is an intermittent assert failure.

Intermittent failure:

ruby: ../../../../ext/google/protobuf_c/protobuf.c:263: ObjectCache_Add: Assertion `rb_funcall(obj_cache2, (__builtin_constant_p("[]") ? __extension__ ({ static ID rb_intern_id_cache; if (!rb_intern_id_cache) rb_intern_id_cache = rb_intern2((("[]")
), (long)strlen(("[]"))); (ID) rb_intern_id_cache; }) : rb_intern("[]")), 1, key_rb) == val' failed

* Removed a few more things from protobuf.h.

* Ruby 3.0.0-preview2 to 3.0.0

* Require rake-compiler-dock >= 1.1.0

* More progress, fighting with the object cache.

* Passes on all Ruby versions!

* Updated and clarified comment regarding WeakMap.

* Fixed the wyhash compile.

* Fixed conformance tests for Ruby.

Conformance results now look like:

RUBYLIB=../ruby/lib:. ./conformance-test-runner --enforce_recommended --failure_list failure_list_ruby.txt --text_format_failure_list text_format_failure_list_ruby.txt ./conformance_ruby.rb

CONFORMANCE TEST BEGIN ====================================

CONFORMANCE SUITE PASSED: 1955 successes, 0 skipped, 58 expected failures, 0 unexpected failures.

CONFORMANCE TEST BEGIN ====================================

CONFORMANCE SUITE PASSED: 0 successes, 111 skipped, 8 expected failures, 0 unexpected failures.

Fixes include:

- Changed Ruby compiler to no longer reject proto2 maps.
- Changed Ruby compiler to emit a warning when proto2 extensions are
  present instead of rejecting the .proto file completely.
- Fixed conformance tests to allow proto2 and look up message by name
  instead of hardcoding a specific list of messages.
- Fixed conformance test to support the "ignore unknown" option for
  JSON.
- Fixed conformance test to properly report serialization errors.

* Removed debug printf and fixed #inspect for floats.

* Fixed compatibility test to have proper semantics for #to_json.

* Updated Makefile.am with new file list.

* Don't try to copy wyhash when inside Docker.

* Fixed bug where we would forget that a sub-object is frozen in Ruby >=2.7.

* Avoid exporting unneeded symbols and refactored a bit of code.

* Some more refactoring.

* Simplified and added more comments.

* Some more comments and simplification. Added a missing license block.

Co-authored-by: Masaki Hara <hara@wantedly.com>
2021-01-13 12:16:25 -08:00
..
third_party/jsoncpp Sync from Piper @327110949 2020-08-17 15:26:13 -07:00
autoload.php Fix php well known type conformance tests (#3828) (#3840) 2017-11-04 09:33:56 -07:00
binary_json_conformance_suite.cc Sync from Piper @317197168 2020-06-18 15:47:49 -07:00
binary_json_conformance_suite.h Sync from Piper @310457838 2020-05-07 16:21:50 -07:00
conformance_cpp.cc Sync from Piper @343521567 2020-11-20 10:44:58 -08:00
conformance_nodejs.js fix conformance test for text format 2019-02-26 13:46:30 -08:00
conformance_objc.m Down integrate to GitHub 2019-08-22 17:33:01 -07:00
conformance_php.php Set execute bit on files if and only if they begin with (#!). (#7347) 2020-04-01 15:28:25 -07:00
conformance_python.py Sync from Piper @305053102 2020-04-06 09:57:03 -07:00
conformance_ruby.rb Ported Ruby extension to upb_msg (#8184) 2021-01-13 12:16:25 -08:00
conformance_test_main.cc down integration from internal 2019-02-22 18:13:33 +08:00
conformance_test_runner.cc Down Integrate Internal Changes 2019-11-12 13:38:58 -08:00
conformance_test.cc Down Integrate Internal Changes 2019-11-12 13:38:58 -08:00
conformance_test.h Sync from Piper @308829107 2020-04-28 08:40:38 -07:00
conformance.proto Fix lots of spelling errors (#7751) 2020-08-10 11:08:25 -07:00
ConformanceJava.java Sync from Piper @311396324 2020-05-13 13:46:15 -07:00
ConformanceJavaLite.java Down Integrate Internal Changes 2019-11-12 13:38:58 -08:00
failure_list_cpp.txt Sync from Piper @310858019 2020-05-11 00:12:20 -07:00
failure_list_csharp.txt Sync from Piper @316511779 2020-06-15 11:48:47 -07:00
failure_list_java.txt Sync from Piper @311396324 2020-05-13 13:46:15 -07:00
failure_list_jruby.txt Set up testing configuration for jruby 2020-12-11 16:02:41 -08:00
failure_list_js.txt Sync from Piper @334206209 2020-09-28 11:54:54 -07:00
failure_list_objc.txt Fix enum writing. 2019-08-26 17:17:32 -04:00
failure_list_php_c.txt Sync from Piper @343521567 2020-11-20 10:44:58 -08:00
failure_list_php.txt Sync from Piper @327110949 2020-08-17 15:26:13 -07:00
failure_list_python_cpp.txt Sync from Piper @316511779 2020-06-15 11:48:47 -07:00
failure_list_python-post26.txt Down-integrate from google3. 2018-08-08 17:00:41 -07:00
failure_list_python.txt Sync from Piper @316511779 2020-06-15 11:48:47 -07:00
failure_list_ruby.txt Ported Ruby extension to upb_msg (#8184) 2021-01-13 12:16:25 -08:00
Makefile.am Ported Ruby extension to upb_msg (#8184) 2021-01-13 12:16:25 -08:00
README.md Replace repo links. 2018-08-22 11:55:30 -07:00
text_format_conformance_suite.cc Fixed build failure in conformance tests. 2020-09-28 14:40:09 -07:00
text_format_conformance_suite.h Sync from Piper @308829107 2020-04-28 08:40:38 -07:00
text_format_failure_list_cpp.txt Sync from Piper @334206209 2020-09-28 11:54:54 -07:00
text_format_failure_list_csharp.txt Update conformance test failures 2019-03-20 14:43:42 -07:00
text_format_failure_list_java.txt Sync from Piper @338284531 2020-10-21 10:04:14 -07:00
text_format_failure_list_jruby.txt Set up testing configuration for jruby 2020-12-11 16:02:41 -08:00
text_format_failure_list_php.txt Update conformance test failures 2019-03-20 14:43:42 -07:00
text_format_failure_list_python_2.7.txt Added text format failure list for Python 2.7. 2020-10-28 10:05:07 -07:00
text_format_failure_list_python_cpp_2.7.txt Added Python 2.7-specific failure list for Python/C++ also. 2020-10-28 10:35:06 -07:00
text_format_failure_list_python_cpp.txt Sync from Piper @334206209 2020-09-28 11:54:54 -07:00
text_format_failure_list_python.txt Sync from Piper @334206209 2020-09-28 11:54:54 -07:00
text_format_failure_list_ruby.txt Update conformance test failures 2019-03-20 14:43:42 -07:00
update_failure_list.py file() was removed in Python 3, use open() instead 2017-08-18 01:04:17 +02:00

Protocol Buffers - Google's data interchange format

Copyright 2008 Google Inc.

This directory contains conformance tests for testing completeness and correctness of Protocol Buffers implementations. These tests are designed to be easy to run against any Protocol Buffers implementation.

This directory contains the tester process conformance-test, which contains all of the tests themselves. Then separate programs written in whatever language you want to test communicate with the tester program over a pipe.

Before running any of these tests, make sure you run make in the base directory to build protoc, since all the tests depend on it.

$ make

Running the tests for C++

To run the tests against the C++ implementation, run:

$ cd conformance && make test_cpp

Running the tests for JavaScript (Node.js)

To run the JavaScript tests against Node.js, make sure you have "node" on your path and then run:

$ cd conformance && make test_nodejs

Running the tests for Ruby (MRI)

To run the Ruby tests against MRI, first build the C extension:

$ cd ruby && rake

Then run the tests like so:

$ cd conformance && make test_ruby

Running the tests for other languages

Most of the languages in the Protobuf source tree are set up to run conformance tests. However some of them are more tricky to set up properly. See tests.sh in the base of the repository to see how Travis runs the tests.

Testing other Protocol Buffer implementations

To run these tests against a new Protocol Buffers implementation, write a program in your language that uses the protobuf implementation you want to test. This program should implement the testing protocol defined in conformance.proto. This is designed to be as easy as possible: the C++ version is only 150 lines and is a good example for what this program should look like (see conformance_cpp.cc). The program only needs to be able to read from stdin and write to stdout.

Portability

Note that the test runner currently does not work on Windows. Patches to fix this are welcome! (But please get in touch first to settle on a general implementation strategy).