Commit Graph

115 Commits

Author SHA1 Message Date
Jan Tattermusch
1834e8cb4d fix typo in test_csharp_aarch64.sh 2021-04-26 20:17:09 +02:00
Jan Tattermusch
4536154a8a
Merge pull request #8495 from jtattermusch/run_ruby_aarch64_test
Run ruby aarch64 tests on the CI (with an emulator)
2021-04-26 18:37:28 +02:00
Jan Tattermusch
78a8d67f87
Merge pull request #8485 from jtattermusch/run_java_aarch64_test
Run java aarch64 tests on the CI (with an emulator)
2021-04-26 18:24:55 +02:00
Jan Tattermusch
b2c71571b3 add kokoro configs for csharp aarch64 2021-04-23 12:21:43 +02:00
Jan Tattermusch
47f6283e7f run csharp tests under aarch64 emulator 2021-04-23 12:21:43 +02:00
Jan Tattermusch
7c876925c7 add kokoro configs for ruby aarch64 2021-04-23 12:09:23 +02:00
Jan Tattermusch
e0267f6c0e run ruby tests under aarch64 emulator 2021-04-23 12:09:23 +02:00
Jan Tattermusch
4342e417a5 add protoc_crosscompile_aarch64.sh (same as for java aarch64 tests) 2021-04-23 12:09:23 +02:00
Jan Tattermusch
b284beecc2 add kokoro job config to test java aarch64 2021-04-23 12:03:06 +02:00
Jan Tattermusch
4939dd91fe allow running emulated java aarch64 tests 2021-04-23 12:03:06 +02:00
Jan Tattermusch
7956ad20d6
Merge pull request #8479 from jtattermusch/run_python_aarch64_tests
Run python aarch64 tests on the CI
2021-04-23 11:59:47 +02:00
Derek Perez
d1b2eff440
Cleanup kokoro usages of java_compatibility and eliminate java/compatibility_tests. (#8504) 2021-04-20 09:36:06 -07:00
Jan Tattermusch
c3ce10e49c use multiarch/qemu-user-static docker image for registering qemu 2021-04-14 15:45:01 +02:00
Jan Tattermusch
5d9a5b4118 fix incorrectly copypasted comment 2021-04-13 15:01:51 +02:00
Jan Tattermusch
c8239319e1 add cfg for python aarch64 tests 2021-04-13 14:25:55 +02:00
Jan Tattermusch
b4531870b4 add prepare_qemu.sh script 2021-04-13 14:25:55 +02:00
Jan Tattermusch
a0b1b70819 fixups in python_crosscompile_aarch64.sh
add --host=aarch64 to ./configure invocation
use protoc_qemu_wrapper.sh
2021-04-13 14:25:55 +02:00
Jan Tattermusch
6aa19d9145 be clever about docker "-it" args 2021-04-13 14:25:55 +02:00
Jan Tattermusch
053966b495
Emulated python aarch64 tests (#8392)
This runs basic python aarch64 tests under qemu emulator, but uses artifacts that can be cross-built by a host compiler.
2021-04-05 14:12:54 -07:00
David L. Jones
f82e268ed7 Remove references to stale benchmark data sources. 2021-04-01 12:21:02 -07:00
Adam Cozzette
0533d042b3 Delete root-owned files in Kokoro builds
Some of our Kokoro builds have been failing because Kokoro is unable to
copy root-owned files when the build is complete. This commit fixes the
problem by deleting these files at the end.
2021-02-18 16:53:24 -08:00
Jan Tattermusch
a1f96fffc7
Merge pull request #8147 from JamesNK/jamesnk/writestring
Add .NET 5 target and improve WriteString performance with SIMD
2021-02-16 12:15:47 +01:00
Jan Tattermusch
818c2ba077 add libicu63 to C# dockerfile 2021-02-15 11:02:41 +01:00
David L. Jones
a25474ed45
Remove the (unused) Python all-versions Kokoro config. (#8288) 2021-02-11 16:35:00 -08:00
James Newton-King
d9889a4a15
Fix 5.0 SDK Linux restore issue 2021-02-06 16:32:48 +13:00
Paul Yang
525be941e0
Add test for php5 for pure php implementation (#8251)
* Add test for php5 for pure php implementation

* Add php5.6 to docker image

* Add php5.6 to php 32-bit docker image
2021-02-04 09:50:28 -08:00
James Newton-King
8932446d9e
Improve WriteString perf with SIMD 2021-02-04 07:26:34 +13:00
gerben-s
10ecb080cb
Merge pull request #8121 from busunkim96/add-39
add python 3.9
2021-02-01 19:34:02 -08:00
Joshua Haberman
9abf6e2ab0
Ported Ruby extension to upb_msg (#8184)
* WIP.

* WIP.

* WIP.

* WIP.

* WIP.

* WIP.

* Added some missing files.

* WIP.

* WIP.

* Updated upb.

* Extension loads, but crashes immediately.

* Gets through the test suite without SEGV!

Still a lot of bugs to fix, but it is a major step!

214 tests, 378 assertions, 37 failures, 147 errors, 0 pendings, 0 omissions, 0 notifications
14.0187% passed

* Test and build for Ruby 3.0

* Fixed a few more bugs, efficient #inspect is almost done.

214 tests, 134243 assertions, 30 failures, 144 errors, 0 pendings, 0 omissions, 0 notifications
18.6916% passed

* Fixed message hash initialization and encode depth checking.

214 tests, 124651 assertions, 53 failures, 70 errors, 0 pendings, 0 omissions, 0 notifications
42.5234% passed

* A bunch of fixes to failing tests, now 70% passing.

214 tests, 202091 assertions, 41 failures, 23 errors, 0 pendings, 0 omissions, 0 notifications
70.0935% passed

* More than 80% of tests are passing now.

214 tests, 322331 assertions, 30 failures, 9 errors, 0 pendings, 0 omissions, 0 notifications
81.7757% passed

Unfortunately there is also a sporadic bug/segfault hanging around
that appears to be GC-related.

* Add linux/ruby30 and macos/ruby30

* Use rvm master for 3.0.0-preview2

* Over 90% of tests are passing!

214 tests, 349898 assertions, 15 failures, 1 errors, 0 pendings, 0 omissions, 0 notifications
92.5234% passed

* Passes all tests!

214 tests, 369388 assertions, 0 failures, 0 errors, 0 pendings, 0 omissions, 0 notifications
100% passed

* A bunch of cleanup.

1. Removed a bunch of internal-only symbols from headers.
2. Required a frozen check to get a non-const pointer to a map or array.
3. De-duplicated the code to get a type argument for Map/RepeatedField.

* Removed a bunch more stuff from protobuf.h.  There is an intermittent assert failure.

Intermittent failure:

ruby: ../../../../ext/google/protobuf_c/protobuf.c:263: ObjectCache_Add: Assertion `rb_funcall(obj_cache2, (__builtin_constant_p("[]") ? __extension__ ({ static ID rb_intern_id_cache; if (!rb_intern_id_cache) rb_intern_id_cache = rb_intern2((("[]")
), (long)strlen(("[]"))); (ID) rb_intern_id_cache; }) : rb_intern("[]")), 1, key_rb) == val' failed

* Removed a few more things from protobuf.h.

* Ruby 3.0.0-preview2 to 3.0.0

* Require rake-compiler-dock >= 1.1.0

* More progress, fighting with the object cache.

* Passes on all Ruby versions!

* Updated and clarified comment regarding WeakMap.

* Fixed the wyhash compile.

* Fixed conformance tests for Ruby.

Conformance results now look like:

RUBYLIB=../ruby/lib:. ./conformance-test-runner --enforce_recommended --failure_list failure_list_ruby.txt --text_format_failure_list text_format_failure_list_ruby.txt ./conformance_ruby.rb

CONFORMANCE TEST BEGIN ====================================

CONFORMANCE SUITE PASSED: 1955 successes, 0 skipped, 58 expected failures, 0 unexpected failures.

CONFORMANCE TEST BEGIN ====================================

CONFORMANCE SUITE PASSED: 0 successes, 111 skipped, 8 expected failures, 0 unexpected failures.

Fixes include:

- Changed Ruby compiler to no longer reject proto2 maps.
- Changed Ruby compiler to emit a warning when proto2 extensions are
  present instead of rejecting the .proto file completely.
- Fixed conformance tests to allow proto2 and look up message by name
  instead of hardcoding a specific list of messages.
- Fixed conformance test to support the "ignore unknown" option for
  JSON.
- Fixed conformance test to properly report serialization errors.

* Removed debug printf and fixed #inspect for floats.

* Fixed compatibility test to have proper semantics for #to_json.

* Updated Makefile.am with new file list.

* Don't try to copy wyhash when inside Docker.

* Fixed bug where we would forget that a sub-object is frozen in Ruby >=2.7.

* Avoid exporting unneeded symbols and refactored a bit of code.

* Some more refactoring.

* Simplified and added more comments.

* Some more comments and simplification. Added a missing license block.

Co-authored-by: Masaki Hara <hara@wantedly.com>
2021-01-13 12:16:25 -08:00
Paul Yang
823f351448
Port for php8 (#8105)
* Port for php8

* Port php c extension for php8

* Update composer.json

* Drop php7.0 support

* Update phpunit for php7.1 in c extension test

* Add back support for php7.0

* Add badge for php8 continuous build
2020-12-04 13:46:34 -08:00
Bu Sun Kim
51c80ce1d5 add python 3.9 2020-12-04 21:16:47 +00:00
Paul Yang
70531697bc
Update docker file for php 32bit tests (#8116) 2020-12-02 16:44:56 -08:00
Paul Yang
e8906e4ecd
Update docker file for php7 (#8112) 2020-12-02 10:10:18 -08:00
Paul Yang
9432c99f37
Update docker image for php8 (#8103) 2020-11-30 10:19:50 -08:00
David L. Jones
b774da0d42
Change Python kokoro configs to test only their specified version. (#7927)
Using non-versioned scripts for `build_file` means every Python version is
tested (under Tox), and this is multiplied by each Python version running Tox.

For example, when the `python38` config is used to run Tox, the
Tox layer runs on Python 3.8. This then tests all of the Python versions
individually, including, for example, python27 tests.

This change fixes the `build_file` paths to point to the same-pathed build
script. For example, `kokoro/linux/python27/presubmit.cfg` now uses
`kokoro/linux/python27/build.sh`.

Some additional fixes:

*  Use `python -m tox` in tests.sh instead of just `tox`. This helps non-site
   installations of tox, where the `tox` script may not be on `$PATH`.
*  Ensure tox (and other Python build-related packages) are available in
   Python testing images. (New images have been pushed.)
*  Disable `--warnings_as_errors` due to deprecated function.
*  Remove apt lists per [Docker best practices][1].

[1]: https://docs.docker.com/develop/develop-images/dockerfile_best-practices/
2020-09-30 15:56:17 -07:00
Paul Yang
d4ca92962c
Port php c extension to php8 (#7793)
* Only ported c extension to php8.
* Didn't fixed the issue of throwing warnings for missing arginfo in bundled files.
* Tests not fixed, because syntax of phpunit (<7 vs >9.3) are not compatible.
* In next release, needs to drop php5 and php7.0 support (in order to use phpunit > 7)
2020-08-11 19:30:46 -07:00
Bo Yang
8947701be5 Add docker file for php8.0 2020-08-10 13:51:06 -07:00
Jan Tattermusch
fed7e18193 upgrade dotnet SDK in the CI to 3.1 2020-06-30 08:53:31 -04:00
Stanley Cheung
ef22086980
Fix :protobuf_objc bazel target (#7538)
* Potential 3.12 ObjC bazel patch

* Add test targets to bazel test

* Remove :protobuf_objc test target
2020-05-21 12:46:35 -07:00
Masaki Hara
f494cb222d
Test in Ruby 2.7 (#7386)
* Test in Ruby 2.7

* Turn off GC.stress for 'require' in Ruby 2.7

* Backport 2c8364b to compatibility_tests/v3.0.0
2020-04-24 09:50:27 -07:00
Paul Yang
f15ac4091f
Drop 3.3, 3.4 and use single version docker images for all python tests (#7396)
* Create a new docker image for python 3.6

The previous one cannot build, because python3.6 was removed from
stretch.

* Drop support for 3.3 and 3.4

Also update all docker file to be single version
2020-04-18 14:39:52 -07:00
Masaki Hara
64f6c59ccb
Build extensions for Ruby 2.7 (#7027)
* Build extensions for Ruby 2.7

* Try installing bundler 2.x

* Try bumping rake-compiler-dock

* Use standard RCD images

* Avoid 'rake cross native' with rake-compiler-dock

* Use Ruby 2.5 for building Ruby <= 2.6

* Use rake-compiler 1.1.0

* Specify target

* Don't update Ruby test image for now
2020-04-16 23:31:47 -07:00
Joshua Haberman
c649397029
Set execute bit on files if and only if they begin with (#!). (#7347)
* Set execute bit on files if and only if they begin with (#!).

Git only tracks the 'x' (executable) bit on each file. Prior to this
CL, our files were a random mix of executable and non-executable.
This change imposes some order by making files executable if and only
if they have shebang (#!) lines at the beginning.

We don't have any executable binaries checked into the repo, so
we shouldn't need to worry about that case.

* Added fix_permissions.sh script to set +x iff a file begins with (#!).
2020-04-01 15:28:25 -07:00
Yannic
f0cb9cdb95
[bazel] Move Java runtime/toolchains into //java (#7190)
* [bazel] Move Java runtime/toolchains into //java

This change moves `java_library` targets from the top-level BUILD file
into `//java/{core,lite,util}` and declares `alias` targets to point to
their new locations (hence, this is not a breaking change).

This will allow users that don't use Java to stop depending on
`@rules_java` (e.g. as requested in
https://github.com/bazelbuild/rules_scala/pull/989#issuecomment-583405161).

Note that there is no intention to deprecate + remove the top-level
targets in the foreseeable future.

* Add BUILD files in //java to java_EXTRA_DIST
2020-02-13 13:04:14 -08:00
Adam Cozzette
802e1b99b2 Attempt to mitigate "docker pull" failures
Lots of CI tests have been failing recently with error messages like
this from "docker pull":

  Error response from daemon: unexpected EOF

This seems to be an issue affecting other projects as well:
https://forums.docker.com/t/docker-pull-causes-error-response-from-daemon-unexpected-eof/88608

This commit tries to mitigate the problem by periodically retrying
failed Docker pulls.
2020-02-11 12:59:49 -08:00
Yannic
948740bc9d [bazel] Fix blacklisted_protos in cc_toolchain and add test (#7075) 2020-01-15 10:27:35 -08:00
Rafi Kamal
4500f31a6a Merge branch 3.11.x to master 2020-01-07 11:52:29 -08:00
Jie Luo
7faab5eeeb add python38 (#7009)
* add python38

* add python3.8

* fix tox
2019-12-13 15:49:18 -08:00
Paul Yang
7bb8b108d1
Add php 7.4 to docker image (#6971) (#6975)
* Add php 7.4 to docker image

* Use https links
2019-12-02 16:05:55 -08:00
Paul Yang
ef210164aa
Add php 7.4 to docker image (#6971)
* Add php 7.4 to docker image

* Use https links
2019-12-02 15:33:05 -08:00