Commit Graph

136 Commits

Author SHA1 Message Date
Derek Perez
5df472af15
use our java tests target instead of recursive walk (#8777) 2021-06-28 14:31:56 -07:00
Derek Perez
e9ffe09c8f
avoid using Bazel latest. (#8753) 2021-06-22 09:35:30 -07:00
Jan Tattermusch
b4a06a193b print benchmarks JSON results to build job log 2021-05-31 11:07:33 +02:00
Jan Tattermusch
0b8d13a1d4
Merge pull request #8665 from jtattermusch/protobuf_fix_benchmark_script
Fix OSS benchmarking script
2021-05-31 11:02:20 +02:00
Jan Tattermusch
ceafbf9132 more cleanup 2021-05-27 12:29:29 +02:00
Jan Tattermusch
75afc88eae disable php benchmark and other cleanup 2021-05-27 11:32:50 +02:00
Jan Tattermusch
6135c916a4 fix benchmark run.sh script 2021-05-27 10:30:12 +02:00
Adam Cozzette
4e0a1119c0
Fix the Docker build for our javascript Docker image (#8661)
I noticed that our JavaScript Docker image is a couple years old and
seems to have an old NPM version that may be causing problems on #8610,
so I went ahead and rebuilt the image. To do that successfully, I had to
set some environment variables in the Dockerfile so that it can build
successfully without having to ask for input about timezone
configuration. This seems to be a known issue described here:
https://serverfault.com/questions/949991/how-to-install-tzdata-on-a-ubuntu-docker-image
I also updated it to explicitly install Python since that is required
for part of our JavaScript build.
2021-05-26 08:56:40 -07:00
Jan Tattermusch
95f5995cce add kokoro job configs for php aarch64 linux test 2021-05-21 09:53:54 +02:00
Jan Tattermusch
2d2ebf5e97 add php aarch64 linux emulated tests 2021-05-21 09:51:22 +02:00
Joshua Haberman
b0d90e3abe
Fixed PHP memory leaks and arginfo errors (#8614)
* Fixed a bunch of incorrect arginfo and a few incorrect error messages.

* Passes mem check test with no leaks!

* WIP.

* Fix build warning that was causing Bazel build to fail.

* Added compatibility code for PHP <8.0.

* Added test_valgrind target and made tests Valgrind-clean.

* Updated Valgrind test to fail if memory leaks are detected.

* Removed intermediate shell script so commands are easier to cut, paste, and modify.

* Passing all Valgrind tests!

* Hoist addref into ObjCache_Get().

* Removed special case of map descriptors by keying object map on upb_msgdef.

* Removed all remaining RETURN_ZVAL() macros.

* Removed all explicit reference add/del operations.

* Added REFCOUNTING.md to Makefile.am.
2021-05-13 23:16:49 -07:00
Adam Cozzette
7bbfd14b7f
Merge pull request #8603 from acozzette/merge-3.17.x
Merge 3.17.x into master
2021-05-13 08:59:12 -07:00
Adam Cozzette
ca295506c5
Make the dist_install test use Java 8 (#8609)
This is necessary because the Kotlin support requires Java 8 or
higher.

Making this test use Java 8 required switching it to the java_stretch
Docker image and updating that image to include some Python
dependencies.
2021-05-12 14:42:15 -07:00
Joshua Haberman
45e9707871
Some more updates to PHP testing infrastructure (#8576)
* WIP.

* Added build config for all of the tests.

* Use ../src/protoc if it is available, for cases where Bazel isn't available.

* Added test_php.sh.

* Fix for the broken macOS tests.

* Move all jobs to use php80 instead of lots of separate jobs.

* Only pass -t flag if we are running in a terminal.

* Updated php_all job to use new Docker stuff.
2021-05-12 10:42:16 -07:00
Jan Tattermusch
5e84a6169c
Merge pull request #8544 from jtattermusch/test_javascript_aarch64
Test javascript on linux aarch64 (under qemu emulator)
2021-04-30 07:11:09 +02:00
Jan Tattermusch
f3411a29da node tests require java to be available 2021-04-29 19:00:14 +02:00
Jan Tattermusch
37ba0bcfa2
Merge pull request #8391 from jtattermusch/cpp_aarch64_emulated_tests
run basic C++ aarch64 tests under qemu emulator
2021-04-29 18:20:55 +02:00
Jan Tattermusch
36243917c1 add testing jobs for javascript_aarch64 2021-04-29 16:46:12 +02:00
Jan Tattermusch
7124f2893d add testing scripts for protobuf javascript on aarch64 qemu emulator 2021-04-29 16:45:34 +02:00
Jan Tattermusch
1834e8cb4d fix typo in test_csharp_aarch64.sh 2021-04-26 20:17:09 +02:00
Jan Tattermusch
4536154a8a
Merge pull request #8495 from jtattermusch/run_ruby_aarch64_test
Run ruby aarch64 tests on the CI (with an emulator)
2021-04-26 18:37:28 +02:00
Jan Tattermusch
78a8d67f87
Merge pull request #8485 from jtattermusch/run_java_aarch64_test
Run java aarch64 tests on the CI (with an emulator)
2021-04-26 18:24:55 +02:00
Jan Tattermusch
b2c71571b3 add kokoro configs for csharp aarch64 2021-04-23 12:21:43 +02:00
Jan Tattermusch
47f6283e7f run csharp tests under aarch64 emulator 2021-04-23 12:21:43 +02:00
Jan Tattermusch
7c876925c7 add kokoro configs for ruby aarch64 2021-04-23 12:09:23 +02:00
Jan Tattermusch
e0267f6c0e run ruby tests under aarch64 emulator 2021-04-23 12:09:23 +02:00
Jan Tattermusch
4342e417a5 add protoc_crosscompile_aarch64.sh (same as for java aarch64 tests) 2021-04-23 12:09:23 +02:00
Jan Tattermusch
b284beecc2 add kokoro job config to test java aarch64 2021-04-23 12:03:06 +02:00
Jan Tattermusch
4939dd91fe allow running emulated java aarch64 tests 2021-04-23 12:03:06 +02:00
Jan Tattermusch
7956ad20d6
Merge pull request #8479 from jtattermusch/run_python_aarch64_tests
Run python aarch64 tests on the CI
2021-04-23 11:59:47 +02:00
Derek Perez
d1b2eff440
Cleanup kokoro usages of java_compatibility and eliminate java/compatibility_tests. (#8504) 2021-04-20 09:36:06 -07:00
Jan Tattermusch
c3ce10e49c use multiarch/qemu-user-static docker image for registering qemu 2021-04-14 15:45:01 +02:00
Jan Tattermusch
5d9a5b4118 fix incorrectly copypasted comment 2021-04-13 15:01:51 +02:00
Jan Tattermusch
c8239319e1 add cfg for python aarch64 tests 2021-04-13 14:25:55 +02:00
Jan Tattermusch
b4531870b4 add prepare_qemu.sh script 2021-04-13 14:25:55 +02:00
Jan Tattermusch
a0b1b70819 fixups in python_crosscompile_aarch64.sh
add --host=aarch64 to ./configure invocation
use protoc_qemu_wrapper.sh
2021-04-13 14:25:55 +02:00
Jan Tattermusch
6aa19d9145 be clever about docker "-it" args 2021-04-13 14:25:55 +02:00
Jan Tattermusch
64d0b2940d add C++ aarch64 kokoro configuration 2021-04-13 13:52:00 +02:00
Jan Tattermusch
b3d497f6a5 run basic C++ tests under qemu emulator 2021-04-13 13:52:00 +02:00
Jan Tattermusch
053966b495
Emulated python aarch64 tests (#8392)
This runs basic python aarch64 tests under qemu emulator, but uses artifacts that can be cross-built by a host compiler.
2021-04-05 14:12:54 -07:00
David L. Jones
f82e268ed7 Remove references to stale benchmark data sources. 2021-04-01 12:21:02 -07:00
Adam Cozzette
0533d042b3 Delete root-owned files in Kokoro builds
Some of our Kokoro builds have been failing because Kokoro is unable to
copy root-owned files when the build is complete. This commit fixes the
problem by deleting these files at the end.
2021-02-18 16:53:24 -08:00
Jan Tattermusch
a1f96fffc7
Merge pull request #8147 from JamesNK/jamesnk/writestring
Add .NET 5 target and improve WriteString performance with SIMD
2021-02-16 12:15:47 +01:00
Jan Tattermusch
818c2ba077 add libicu63 to C# dockerfile 2021-02-15 11:02:41 +01:00
David L. Jones
a25474ed45
Remove the (unused) Python all-versions Kokoro config. (#8288) 2021-02-11 16:35:00 -08:00
James Newton-King
d9889a4a15
Fix 5.0 SDK Linux restore issue 2021-02-06 16:32:48 +13:00
Paul Yang
525be941e0
Add test for php5 for pure php implementation (#8251)
* Add test for php5 for pure php implementation

* Add php5.6 to docker image

* Add php5.6 to php 32-bit docker image
2021-02-04 09:50:28 -08:00
James Newton-King
8932446d9e
Improve WriteString perf with SIMD 2021-02-04 07:26:34 +13:00
gerben-s
10ecb080cb
Merge pull request #8121 from busunkim96/add-39
add python 3.9
2021-02-01 19:34:02 -08:00
Joshua Haberman
9abf6e2ab0
Ported Ruby extension to upb_msg (#8184)
* WIP.

* WIP.

* WIP.

* WIP.

* WIP.

* WIP.

* Added some missing files.

* WIP.

* WIP.

* Updated upb.

* Extension loads, but crashes immediately.

* Gets through the test suite without SEGV!

Still a lot of bugs to fix, but it is a major step!

214 tests, 378 assertions, 37 failures, 147 errors, 0 pendings, 0 omissions, 0 notifications
14.0187% passed

* Test and build for Ruby 3.0

* Fixed a few more bugs, efficient #inspect is almost done.

214 tests, 134243 assertions, 30 failures, 144 errors, 0 pendings, 0 omissions, 0 notifications
18.6916% passed

* Fixed message hash initialization and encode depth checking.

214 tests, 124651 assertions, 53 failures, 70 errors, 0 pendings, 0 omissions, 0 notifications
42.5234% passed

* A bunch of fixes to failing tests, now 70% passing.

214 tests, 202091 assertions, 41 failures, 23 errors, 0 pendings, 0 omissions, 0 notifications
70.0935% passed

* More than 80% of tests are passing now.

214 tests, 322331 assertions, 30 failures, 9 errors, 0 pendings, 0 omissions, 0 notifications
81.7757% passed

Unfortunately there is also a sporadic bug/segfault hanging around
that appears to be GC-related.

* Add linux/ruby30 and macos/ruby30

* Use rvm master for 3.0.0-preview2

* Over 90% of tests are passing!

214 tests, 349898 assertions, 15 failures, 1 errors, 0 pendings, 0 omissions, 0 notifications
92.5234% passed

* Passes all tests!

214 tests, 369388 assertions, 0 failures, 0 errors, 0 pendings, 0 omissions, 0 notifications
100% passed

* A bunch of cleanup.

1. Removed a bunch of internal-only symbols from headers.
2. Required a frozen check to get a non-const pointer to a map or array.
3. De-duplicated the code to get a type argument for Map/RepeatedField.

* Removed a bunch more stuff from protobuf.h.  There is an intermittent assert failure.

Intermittent failure:

ruby: ../../../../ext/google/protobuf_c/protobuf.c:263: ObjectCache_Add: Assertion `rb_funcall(obj_cache2, (__builtin_constant_p("[]") ? __extension__ ({ static ID rb_intern_id_cache; if (!rb_intern_id_cache) rb_intern_id_cache = rb_intern2((("[]")
), (long)strlen(("[]"))); (ID) rb_intern_id_cache; }) : rb_intern("[]")), 1, key_rb) == val' failed

* Removed a few more things from protobuf.h.

* Ruby 3.0.0-preview2 to 3.0.0

* Require rake-compiler-dock >= 1.1.0

* More progress, fighting with the object cache.

* Passes on all Ruby versions!

* Updated and clarified comment regarding WeakMap.

* Fixed the wyhash compile.

* Fixed conformance tests for Ruby.

Conformance results now look like:

RUBYLIB=../ruby/lib:. ./conformance-test-runner --enforce_recommended --failure_list failure_list_ruby.txt --text_format_failure_list text_format_failure_list_ruby.txt ./conformance_ruby.rb

CONFORMANCE TEST BEGIN ====================================

CONFORMANCE SUITE PASSED: 1955 successes, 0 skipped, 58 expected failures, 0 unexpected failures.

CONFORMANCE TEST BEGIN ====================================

CONFORMANCE SUITE PASSED: 0 successes, 111 skipped, 8 expected failures, 0 unexpected failures.

Fixes include:

- Changed Ruby compiler to no longer reject proto2 maps.
- Changed Ruby compiler to emit a warning when proto2 extensions are
  present instead of rejecting the .proto file completely.
- Fixed conformance tests to allow proto2 and look up message by name
  instead of hardcoding a specific list of messages.
- Fixed conformance test to support the "ignore unknown" option for
  JSON.
- Fixed conformance test to properly report serialization errors.

* Removed debug printf and fixed #inspect for floats.

* Fixed compatibility test to have proper semantics for #to_json.

* Updated Makefile.am with new file list.

* Don't try to copy wyhash when inside Docker.

* Fixed bug where we would forget that a sub-object is frozen in Ruby >=2.7.

* Avoid exporting unneeded symbols and refactored a bit of code.

* Some more refactoring.

* Simplified and added more comments.

* Some more comments and simplification. Added a missing license block.

Co-authored-by: Masaki Hara <hara@wantedly.com>
2021-01-13 12:16:25 -08:00