Commit Graph

189 Commits

Author SHA1 Message Date
Jan Tattermusch
2a11e892ed
Merge pull request #8530 from jtattermusch/python_aarch64_manylinux_release
Start publishing python aarch64 manylinux wheels normally
2021-04-26 20:29:59 +02:00
Jan Tattermusch
1834e8cb4d fix typo in test_csharp_aarch64.sh 2021-04-26 20:17:09 +02:00
Jan Tattermusch
4536154a8a
Merge pull request #8495 from jtattermusch/run_ruby_aarch64_test
Run ruby aarch64 tests on the CI (with an emulator)
2021-04-26 18:37:28 +02:00
Jan Tattermusch
78a8d67f87
Merge pull request #8485 from jtattermusch/run_java_aarch64_test
Run java aarch64 tests on the CI (with an emulator)
2021-04-26 18:24:55 +02:00
Jan Tattermusch
0a2ddaebc9 start publishing python aarch64 manylinux wheels normally 2021-04-24 06:15:59 +02:00
Jan Tattermusch
b2c71571b3 add kokoro configs for csharp aarch64 2021-04-23 12:21:43 +02:00
Jan Tattermusch
47f6283e7f run csharp tests under aarch64 emulator 2021-04-23 12:21:43 +02:00
Jan Tattermusch
7c876925c7 add kokoro configs for ruby aarch64 2021-04-23 12:09:23 +02:00
Jan Tattermusch
e0267f6c0e run ruby tests under aarch64 emulator 2021-04-23 12:09:23 +02:00
Jan Tattermusch
4342e417a5 add protoc_crosscompile_aarch64.sh (same as for java aarch64 tests) 2021-04-23 12:09:23 +02:00
Jan Tattermusch
b284beecc2 add kokoro job config to test java aarch64 2021-04-23 12:03:06 +02:00
Jan Tattermusch
4939dd91fe allow running emulated java aarch64 tests 2021-04-23 12:03:06 +02:00
Jan Tattermusch
7956ad20d6
Merge pull request #8479 from jtattermusch/run_python_aarch64_tests
Run python aarch64 tests on the CI
2021-04-23 11:59:47 +02:00
Derek Perez
d1b2eff440
Cleanup kokoro usages of java_compatibility and eliminate java/compatibility_tests. (#8504) 2021-04-20 09:36:06 -07:00
Jan Tattermusch
c3ce10e49c use multiarch/qemu-user-static docker image for registering qemu 2021-04-14 15:45:01 +02:00
Jan Tattermusch
5d9a5b4118 fix incorrectly copypasted comment 2021-04-13 15:01:51 +02:00
Jan Tattermusch
c8239319e1 add cfg for python aarch64 tests 2021-04-13 14:25:55 +02:00
Jan Tattermusch
b4531870b4 add prepare_qemu.sh script 2021-04-13 14:25:55 +02:00
Jan Tattermusch
a0b1b70819 fixups in python_crosscompile_aarch64.sh
add --host=aarch64 to ./configure invocation
use protoc_qemu_wrapper.sh
2021-04-13 14:25:55 +02:00
Jan Tattermusch
6aa19d9145 be clever about docker "-it" args 2021-04-13 14:25:55 +02:00
Jan Tattermusch
053966b495
Emulated python aarch64 tests (#8392)
This runs basic python aarch64 tests under qemu emulator, but uses artifacts that can be cross-built by a host compiler.
2021-04-05 14:12:54 -07:00
David L. Jones
f82e268ed7 Remove references to stale benchmark data sources. 2021-04-01 12:21:02 -07:00
Jan Tattermusch
0ebbd7d7c3
Enable crosscompiling aarch64 python wheels under dockcross manylinux docker image (#8280)
This uses the dockcross manylinux docker image to enable crosscompiling aarch64 python wheels. The wheels built for aarch64 linux are excluded from the release for now, pending testing (pending, in PR#8392).
2021-03-29 19:04:16 -07:00
salamani
a4694f9b66 correcting arch name for s390x 2021-03-10 08:09:44 -08:00
Adam Cozzette
ea2412e6ed Updated collect_all_artifacts.sh for Ubuntu Xenial 2021-02-18 16:53:24 -08:00
Adam Cozzette
0533d042b3 Delete root-owned files in Kokoro builds
Some of our Kokoro builds have been failing because Kokoro is unable to
copy root-owned files when the build is complete. This commit fixes the
problem by deleting these files at the end.
2021-02-18 16:53:24 -08:00
Adam Cozzette
8aa0063bef Updated collect_all_artifacts.sh to avoid installing apt-transport-https
The installation of apt-transport-https is failing and causing this
script to exit early. I suspect the package is no longer needed since
recent versions have moved it into the apt package, so this change
updates the script to stop trying to install apt-transport-https.
2021-02-18 16:53:24 -08:00
Jan Tattermusch
a1f96fffc7
Merge pull request #8147 from JamesNK/jamesnk/writestring
Add .NET 5 target and improve WriteString performance with SIMD
2021-02-16 12:15:47 +01:00
Jan Tattermusch
818c2ba077 add libicu63 to C# dockerfile 2021-02-15 11:02:41 +01:00
David L. Jones
a25474ed45
Remove the (unused) Python all-versions Kokoro config. (#8288) 2021-02-11 16:35:00 -08:00
James Newton-King
d9889a4a15
Fix 5.0 SDK Linux restore issue 2021-02-06 16:32:48 +13:00
Paul Yang
525be941e0
Add test for php5 for pure php implementation (#8251)
* Add test for php5 for pure php implementation

* Add php5.6 to docker image

* Add php5.6 to php 32-bit docker image
2021-02-04 09:50:28 -08:00
James Newton-King
8932446d9e
Improve WriteString perf with SIMD 2021-02-04 07:26:34 +13:00
David L. Jones
ac63e80bbb
Enhance explanation in kokoro/README.md. (#8253) 2021-02-02 16:49:58 -08:00
gerben-s
10ecb080cb
Merge pull request #8121 from busunkim96/add-39
add python 3.9
2021-02-01 19:34:02 -08:00
gerben-s
fe47e99a4f
Add cmake to the path to fix the build on kokoro (#8246) 2021-02-01 15:48:50 -08:00
Jan Tattermusch
94c3916dcf
fix problem when restoring C# dependencies on win2016 (#8243) 2021-02-01 09:25:56 -08:00
Joshua Haberman
9abf6e2ab0
Ported Ruby extension to upb_msg (#8184)
* WIP.

* WIP.

* WIP.

* WIP.

* WIP.

* WIP.

* Added some missing files.

* WIP.

* WIP.

* Updated upb.

* Extension loads, but crashes immediately.

* Gets through the test suite without SEGV!

Still a lot of bugs to fix, but it is a major step!

214 tests, 378 assertions, 37 failures, 147 errors, 0 pendings, 0 omissions, 0 notifications
14.0187% passed

* Test and build for Ruby 3.0

* Fixed a few more bugs, efficient #inspect is almost done.

214 tests, 134243 assertions, 30 failures, 144 errors, 0 pendings, 0 omissions, 0 notifications
18.6916% passed

* Fixed message hash initialization and encode depth checking.

214 tests, 124651 assertions, 53 failures, 70 errors, 0 pendings, 0 omissions, 0 notifications
42.5234% passed

* A bunch of fixes to failing tests, now 70% passing.

214 tests, 202091 assertions, 41 failures, 23 errors, 0 pendings, 0 omissions, 0 notifications
70.0935% passed

* More than 80% of tests are passing now.

214 tests, 322331 assertions, 30 failures, 9 errors, 0 pendings, 0 omissions, 0 notifications
81.7757% passed

Unfortunately there is also a sporadic bug/segfault hanging around
that appears to be GC-related.

* Add linux/ruby30 and macos/ruby30

* Use rvm master for 3.0.0-preview2

* Over 90% of tests are passing!

214 tests, 349898 assertions, 15 failures, 1 errors, 0 pendings, 0 omissions, 0 notifications
92.5234% passed

* Passes all tests!

214 tests, 369388 assertions, 0 failures, 0 errors, 0 pendings, 0 omissions, 0 notifications
100% passed

* A bunch of cleanup.

1. Removed a bunch of internal-only symbols from headers.
2. Required a frozen check to get a non-const pointer to a map or array.
3. De-duplicated the code to get a type argument for Map/RepeatedField.

* Removed a bunch more stuff from protobuf.h.  There is an intermittent assert failure.

Intermittent failure:

ruby: ../../../../ext/google/protobuf_c/protobuf.c:263: ObjectCache_Add: Assertion `rb_funcall(obj_cache2, (__builtin_constant_p("[]") ? __extension__ ({ static ID rb_intern_id_cache; if (!rb_intern_id_cache) rb_intern_id_cache = rb_intern2((("[]")
), (long)strlen(("[]"))); (ID) rb_intern_id_cache; }) : rb_intern("[]")), 1, key_rb) == val' failed

* Removed a few more things from protobuf.h.

* Ruby 3.0.0-preview2 to 3.0.0

* Require rake-compiler-dock >= 1.1.0

* More progress, fighting with the object cache.

* Passes on all Ruby versions!

* Updated and clarified comment regarding WeakMap.

* Fixed the wyhash compile.

* Fixed conformance tests for Ruby.

Conformance results now look like:

RUBYLIB=../ruby/lib:. ./conformance-test-runner --enforce_recommended --failure_list failure_list_ruby.txt --text_format_failure_list text_format_failure_list_ruby.txt ./conformance_ruby.rb

CONFORMANCE TEST BEGIN ====================================

CONFORMANCE SUITE PASSED: 1955 successes, 0 skipped, 58 expected failures, 0 unexpected failures.

CONFORMANCE TEST BEGIN ====================================

CONFORMANCE SUITE PASSED: 0 successes, 111 skipped, 8 expected failures, 0 unexpected failures.

Fixes include:

- Changed Ruby compiler to no longer reject proto2 maps.
- Changed Ruby compiler to emit a warning when proto2 extensions are
  present instead of rejecting the .proto file completely.
- Fixed conformance tests to allow proto2 and look up message by name
  instead of hardcoding a specific list of messages.
- Fixed conformance test to support the "ignore unknown" option for
  JSON.
- Fixed conformance test to properly report serialization errors.

* Removed debug printf and fixed #inspect for floats.

* Fixed compatibility test to have proper semantics for #to_json.

* Updated Makefile.am with new file list.

* Don't try to copy wyhash when inside Docker.

* Fixed bug where we would forget that a sub-object is frozen in Ruby >=2.7.

* Avoid exporting unneeded symbols and refactored a bit of code.

* Some more refactoring.

* Simplified and added more comments.

* Some more comments and simplification. Added a missing license block.

Co-authored-by: Masaki Hara <hara@wantedly.com>
2021-01-13 12:16:25 -08:00
Adam Cozzette
635156e406 Fix Mac tests by making an unshallow copy of Homebrew repos
It seems that updating shallow Git clones is expensive, and as a result
Homebrew recently started refusing to update shallow clones (see
https://github.com/Homebrew/discussions/discussions/226). This commit
tries to fix the problem by making these repos into full clones before
running "brew update".

I also came across another error about there being a conflicting version
of pip3 in /usr/local. I suspect that is related to the other Python
binaries that the script has to delete, so I added pip3 to the list and
that seemed to solve the problem.
2020-12-23 11:52:19 -08:00
Paul Yang
823f351448
Port for php8 (#8105)
* Port for php8

* Port php c extension for php8

* Update composer.json

* Drop php7.0 support

* Update phpunit for php7.1 in c extension test

* Add back support for php7.0

* Add badge for php8 continuous build
2020-12-04 13:46:34 -08:00
Bu Sun Kim
51c80ce1d5 add python 3.9 2020-12-04 21:16:47 +00:00
Paul Yang
70531697bc
Update docker file for php 32bit tests (#8116) 2020-12-02 16:44:56 -08:00
Paul Yang
e8906e4ecd
Update docker file for php7 (#8112) 2020-12-02 10:10:18 -08:00
Paul Yang
9432c99f37
Update docker image for php8 (#8103) 2020-11-30 10:19:50 -08:00
Paul Yang
4eea2b00a3
Remove php 5 tests on mac (#7956) 2020-10-09 15:53:12 -07:00
David L. Jones
b774da0d42
Change Python kokoro configs to test only their specified version. (#7927)
Using non-versioned scripts for `build_file` means every Python version is
tested (under Tox), and this is multiplied by each Python version running Tox.

For example, when the `python38` config is used to run Tox, the
Tox layer runs on Python 3.8. This then tests all of the Python versions
individually, including, for example, python27 tests.

This change fixes the `build_file` paths to point to the same-pathed build
script. For example, `kokoro/linux/python27/presubmit.cfg` now uses
`kokoro/linux/python27/build.sh`.

Some additional fixes:

*  Use `python -m tox` in tests.sh instead of just `tox`. This helps non-site
   installations of tox, where the `tox` script may not be on `$PATH`.
*  Ensure tox (and other Python build-related packages) are available in
   Python testing images. (New images have been pushed.)
*  Disable `--warnings_as_errors` due to deprecated function.
*  Remove apt lists per [Docker best practices][1].

[1]: https://docs.docker.com/develop/develop-images/dockerfile_best-practices/
2020-09-30 15:56:17 -07:00
Adam Cozzette
58f4260e8e Merged 3.13.x into master 2020-08-17 11:06:11 -07:00
Paul Yang
d4ca92962c
Port php c extension to php8 (#7793)
* Only ported c extension to php8.
* Didn't fixed the issue of throwing warnings for missing arginfo in bundled files.
* Tests not fixed, because syntax of phpunit (<7 vs >9.3) are not compatible.
* In next release, needs to drop php5 and php7.0 support (in order to use phpunit > 7)
2020-08-11 19:30:46 -07:00
Bo Yang
8947701be5 Add docker file for php8.0 2020-08-10 13:51:06 -07:00
Peter Newman
e2cc2de304
Fix lots of spelling errors (#7751)
* Fix a typo

* Fix lots of spelling errors

* Fix a few more spelling mistakes

* s/parsable/parseable/

* Don't touch the third party files

* Cloneable is the preferred C# term

* Copyable is the preferred C++ term

* Revert "s/parsable/parseable/"

This reverts commit 534ecf7675.

* Revert unparseable->unparsable corrections
2020-08-10 11:08:25 -07:00