Commit Graph

200 Commits

Author SHA1 Message Date
Adam Cozzette
ca295506c5
Make the dist_install test use Java 8 (#8609)
This is necessary because the Kotlin support requires Java 8 or
higher.

Making this test use Java 8 required switching it to the java_stretch
Docker image and updating that image to include some Python
dependencies.
2021-05-12 14:42:15 -07:00
Adam Cozzette
3546d29a00
Add the kotlin and kotlin-lite directories to the main pom.xml (#8585)
* Add the kotlin and kotlin-lite directories to the main pom.xml

* Updated Kotlin versions to 3.17.0-rc1

* Fix Mac CI tests by resolving brew package conflict
2021-05-10 13:36:33 -07:00
Dennis Shao
b0f4462ef2
Merge pull request #8536 from shaod2/update-python-docs-script
Update kokoro/docs/publish-python.sh
2021-05-05 16:47:31 -05:00
Jan Tattermusch
5e84a6169c
Merge pull request #8544 from jtattermusch/test_javascript_aarch64
Test javascript on linux aarch64 (under qemu emulator)
2021-04-30 07:11:09 +02:00
Jan Tattermusch
f3411a29da node tests require java to be available 2021-04-29 19:00:14 +02:00
Jan Tattermusch
37ba0bcfa2
Merge pull request #8391 from jtattermusch/cpp_aarch64_emulated_tests
run basic C++ aarch64 tests under qemu emulator
2021-04-29 18:20:55 +02:00
Jan Tattermusch
36243917c1 add testing jobs for javascript_aarch64 2021-04-29 16:46:12 +02:00
Jan Tattermusch
7124f2893d add testing scripts for protobuf javascript on aarch64 qemu emulator 2021-04-29 16:45:34 +02:00
Dennis Shao
8424042534
Update kokoro/docs/publish-python.sh
Update kokoro/docs/publish-python.sh to use the latest version of protoc and python.

This will make [the python doc-publishing job](https://fusion.corp.google.com/projectanalysis/summary/KOKORO/prod:cloud-devrel%2Fclient-libraries%2Fprotobuf%2Fdocs%2Fpython) pass again.
2021-04-26 14:59:14 -05:00
Jan Tattermusch
2a11e892ed
Merge pull request #8530 from jtattermusch/python_aarch64_manylinux_release
Start publishing python aarch64 manylinux wheels normally
2021-04-26 20:29:59 +02:00
Jan Tattermusch
1834e8cb4d fix typo in test_csharp_aarch64.sh 2021-04-26 20:17:09 +02:00
Jan Tattermusch
4536154a8a
Merge pull request #8495 from jtattermusch/run_ruby_aarch64_test
Run ruby aarch64 tests on the CI (with an emulator)
2021-04-26 18:37:28 +02:00
Jan Tattermusch
78a8d67f87
Merge pull request #8485 from jtattermusch/run_java_aarch64_test
Run java aarch64 tests on the CI (with an emulator)
2021-04-26 18:24:55 +02:00
Jan Tattermusch
0a2ddaebc9 start publishing python aarch64 manylinux wheels normally 2021-04-24 06:15:59 +02:00
Jan Tattermusch
b2c71571b3 add kokoro configs for csharp aarch64 2021-04-23 12:21:43 +02:00
Jan Tattermusch
47f6283e7f run csharp tests under aarch64 emulator 2021-04-23 12:21:43 +02:00
Jan Tattermusch
7c876925c7 add kokoro configs for ruby aarch64 2021-04-23 12:09:23 +02:00
Jan Tattermusch
e0267f6c0e run ruby tests under aarch64 emulator 2021-04-23 12:09:23 +02:00
Jan Tattermusch
4342e417a5 add protoc_crosscompile_aarch64.sh (same as for java aarch64 tests) 2021-04-23 12:09:23 +02:00
Jan Tattermusch
b284beecc2 add kokoro job config to test java aarch64 2021-04-23 12:03:06 +02:00
Jan Tattermusch
4939dd91fe allow running emulated java aarch64 tests 2021-04-23 12:03:06 +02:00
Jan Tattermusch
7956ad20d6
Merge pull request #8479 from jtattermusch/run_python_aarch64_tests
Run python aarch64 tests on the CI
2021-04-23 11:59:47 +02:00
Derek Perez
d1b2eff440
Cleanup kokoro usages of java_compatibility and eliminate java/compatibility_tests. (#8504) 2021-04-20 09:36:06 -07:00
Jan Tattermusch
c3ce10e49c use multiarch/qemu-user-static docker image for registering qemu 2021-04-14 15:45:01 +02:00
Jan Tattermusch
5d9a5b4118 fix incorrectly copypasted comment 2021-04-13 15:01:51 +02:00
Jan Tattermusch
c8239319e1 add cfg for python aarch64 tests 2021-04-13 14:25:55 +02:00
Jan Tattermusch
b4531870b4 add prepare_qemu.sh script 2021-04-13 14:25:55 +02:00
Jan Tattermusch
a0b1b70819 fixups in python_crosscompile_aarch64.sh
add --host=aarch64 to ./configure invocation
use protoc_qemu_wrapper.sh
2021-04-13 14:25:55 +02:00
Jan Tattermusch
6aa19d9145 be clever about docker "-it" args 2021-04-13 14:25:55 +02:00
Jan Tattermusch
64d0b2940d add C++ aarch64 kokoro configuration 2021-04-13 13:52:00 +02:00
Jan Tattermusch
b3d497f6a5 run basic C++ tests under qemu emulator 2021-04-13 13:52:00 +02:00
Jan Tattermusch
053966b495
Emulated python aarch64 tests (#8392)
This runs basic python aarch64 tests under qemu emulator, but uses artifacts that can be cross-built by a host compiler.
2021-04-05 14:12:54 -07:00
David L. Jones
f82e268ed7 Remove references to stale benchmark data sources. 2021-04-01 12:21:02 -07:00
Jan Tattermusch
0ebbd7d7c3
Enable crosscompiling aarch64 python wheels under dockcross manylinux docker image (#8280)
This uses the dockcross manylinux docker image to enable crosscompiling aarch64 python wheels. The wheels built for aarch64 linux are excluded from the release for now, pending testing (pending, in PR#8392).
2021-03-29 19:04:16 -07:00
salamani
a4694f9b66 correcting arch name for s390x 2021-03-10 08:09:44 -08:00
Adam Cozzette
ea2412e6ed Updated collect_all_artifacts.sh for Ubuntu Xenial 2021-02-18 16:53:24 -08:00
Adam Cozzette
0533d042b3 Delete root-owned files in Kokoro builds
Some of our Kokoro builds have been failing because Kokoro is unable to
copy root-owned files when the build is complete. This commit fixes the
problem by deleting these files at the end.
2021-02-18 16:53:24 -08:00
Adam Cozzette
8aa0063bef Updated collect_all_artifacts.sh to avoid installing apt-transport-https
The installation of apt-transport-https is failing and causing this
script to exit early. I suspect the package is no longer needed since
recent versions have moved it into the apt package, so this change
updates the script to stop trying to install apt-transport-https.
2021-02-18 16:53:24 -08:00
Jan Tattermusch
a1f96fffc7
Merge pull request #8147 from JamesNK/jamesnk/writestring
Add .NET 5 target and improve WriteString performance with SIMD
2021-02-16 12:15:47 +01:00
Jan Tattermusch
818c2ba077 add libicu63 to C# dockerfile 2021-02-15 11:02:41 +01:00
David L. Jones
a25474ed45
Remove the (unused) Python all-versions Kokoro config. (#8288) 2021-02-11 16:35:00 -08:00
James Newton-King
d9889a4a15
Fix 5.0 SDK Linux restore issue 2021-02-06 16:32:48 +13:00
Paul Yang
525be941e0
Add test for php5 for pure php implementation (#8251)
* Add test for php5 for pure php implementation

* Add php5.6 to docker image

* Add php5.6 to php 32-bit docker image
2021-02-04 09:50:28 -08:00
James Newton-King
8932446d9e
Improve WriteString perf with SIMD 2021-02-04 07:26:34 +13:00
David L. Jones
ac63e80bbb
Enhance explanation in kokoro/README.md. (#8253) 2021-02-02 16:49:58 -08:00
gerben-s
10ecb080cb
Merge pull request #8121 from busunkim96/add-39
add python 3.9
2021-02-01 19:34:02 -08:00
gerben-s
fe47e99a4f
Add cmake to the path to fix the build on kokoro (#8246) 2021-02-01 15:48:50 -08:00
Jan Tattermusch
94c3916dcf
fix problem when restoring C# dependencies on win2016 (#8243) 2021-02-01 09:25:56 -08:00
Joshua Haberman
9abf6e2ab0
Ported Ruby extension to upb_msg (#8184)
* WIP.

* WIP.

* WIP.

* WIP.

* WIP.

* WIP.

* Added some missing files.

* WIP.

* WIP.

* Updated upb.

* Extension loads, but crashes immediately.

* Gets through the test suite without SEGV!

Still a lot of bugs to fix, but it is a major step!

214 tests, 378 assertions, 37 failures, 147 errors, 0 pendings, 0 omissions, 0 notifications
14.0187% passed

* Test and build for Ruby 3.0

* Fixed a few more bugs, efficient #inspect is almost done.

214 tests, 134243 assertions, 30 failures, 144 errors, 0 pendings, 0 omissions, 0 notifications
18.6916% passed

* Fixed message hash initialization and encode depth checking.

214 tests, 124651 assertions, 53 failures, 70 errors, 0 pendings, 0 omissions, 0 notifications
42.5234% passed

* A bunch of fixes to failing tests, now 70% passing.

214 tests, 202091 assertions, 41 failures, 23 errors, 0 pendings, 0 omissions, 0 notifications
70.0935% passed

* More than 80% of tests are passing now.

214 tests, 322331 assertions, 30 failures, 9 errors, 0 pendings, 0 omissions, 0 notifications
81.7757% passed

Unfortunately there is also a sporadic bug/segfault hanging around
that appears to be GC-related.

* Add linux/ruby30 and macos/ruby30

* Use rvm master for 3.0.0-preview2

* Over 90% of tests are passing!

214 tests, 349898 assertions, 15 failures, 1 errors, 0 pendings, 0 omissions, 0 notifications
92.5234% passed

* Passes all tests!

214 tests, 369388 assertions, 0 failures, 0 errors, 0 pendings, 0 omissions, 0 notifications
100% passed

* A bunch of cleanup.

1. Removed a bunch of internal-only symbols from headers.
2. Required a frozen check to get a non-const pointer to a map or array.
3. De-duplicated the code to get a type argument for Map/RepeatedField.

* Removed a bunch more stuff from protobuf.h.  There is an intermittent assert failure.

Intermittent failure:

ruby: ../../../../ext/google/protobuf_c/protobuf.c:263: ObjectCache_Add: Assertion `rb_funcall(obj_cache2, (__builtin_constant_p("[]") ? __extension__ ({ static ID rb_intern_id_cache; if (!rb_intern_id_cache) rb_intern_id_cache = rb_intern2((("[]")
), (long)strlen(("[]"))); (ID) rb_intern_id_cache; }) : rb_intern("[]")), 1, key_rb) == val' failed

* Removed a few more things from protobuf.h.

* Ruby 3.0.0-preview2 to 3.0.0

* Require rake-compiler-dock >= 1.1.0

* More progress, fighting with the object cache.

* Passes on all Ruby versions!

* Updated and clarified comment regarding WeakMap.

* Fixed the wyhash compile.

* Fixed conformance tests for Ruby.

Conformance results now look like:

RUBYLIB=../ruby/lib:. ./conformance-test-runner --enforce_recommended --failure_list failure_list_ruby.txt --text_format_failure_list text_format_failure_list_ruby.txt ./conformance_ruby.rb

CONFORMANCE TEST BEGIN ====================================

CONFORMANCE SUITE PASSED: 1955 successes, 0 skipped, 58 expected failures, 0 unexpected failures.

CONFORMANCE TEST BEGIN ====================================

CONFORMANCE SUITE PASSED: 0 successes, 111 skipped, 8 expected failures, 0 unexpected failures.

Fixes include:

- Changed Ruby compiler to no longer reject proto2 maps.
- Changed Ruby compiler to emit a warning when proto2 extensions are
  present instead of rejecting the .proto file completely.
- Fixed conformance tests to allow proto2 and look up message by name
  instead of hardcoding a specific list of messages.
- Fixed conformance test to support the "ignore unknown" option for
  JSON.
- Fixed conformance test to properly report serialization errors.

* Removed debug printf and fixed #inspect for floats.

* Fixed compatibility test to have proper semantics for #to_json.

* Updated Makefile.am with new file list.

* Don't try to copy wyhash when inside Docker.

* Fixed bug where we would forget that a sub-object is frozen in Ruby >=2.7.

* Avoid exporting unneeded symbols and refactored a bit of code.

* Some more refactoring.

* Simplified and added more comments.

* Some more comments and simplification. Added a missing license block.

Co-authored-by: Masaki Hara <hara@wantedly.com>
2021-01-13 12:16:25 -08:00
Adam Cozzette
635156e406 Fix Mac tests by making an unshallow copy of Homebrew repos
It seems that updating shallow Git clones is expensive, and as a result
Homebrew recently started refusing to update shallow clones (see
https://github.com/Homebrew/discussions/discussions/226). This commit
tries to fix the problem by making these repos into full clones before
running "brew update".

I also came across another error about there being a conflicting version
of pip3 in /usr/local. I suspect that is related to the other Python
binaries that the script has to delete, so I added pip3 to the list and
that seemed to solve the problem.
2020-12-23 11:52:19 -08:00