Commit Graph

195 Commits

Author SHA1 Message Date
Thomas Van Lenten
0a73ec7e88 Remove the support for the kokoro cocoapods integration tests. 2022-05-03 15:34:36 -04:00
Elliotte Rusty Harold
e22db322e4
Fail JDK 17 build when unit tests fail (#9878) 2022-04-28 19:10:12 +00:00
Adam Cozzette
860f8ad171
Remove the JavaScript implementation (#9874)
This implementation is about to be split out into a separate Git repo.
2022-04-28 12:08:26 -07:00
Elliotte Rusty Harold
1927520278
Include JDK11 in error message on first run (#9866)
* update error message on first run
* sync braces
* grammar
* spelling
2022-04-28 15:55:28 +00:00
Elliotte Rusty Harold
868ab1469f
build_java without conformance tests (#9873)
* build_java without conformance tests
2022-04-28 11:09:39 +00:00
David L. Jones
5283ef4b82
Add jdk11 test. (#9843) 2022-04-22 16:44:09 -07:00
Elliotte Rusty Harold
cc8ae4c154
Add JDK 17 target (#9802)
* add JDK 17 target
2022-04-18 19:54:42 +00:00
Elliotte Rusty Harold
46ae74c55c
Remove LAST_RELEASED variable we haven't been updating (#9803)
* Remove LAST_RELEASED variable we haven't been updating
2022-04-18 19:22:22 +00:00
Elliotte Rusty Harold
b27bf8b437
LAST_RELEASED=3.20.0 (#9741)
@acozzette We might want to add this to the release process since it seems this hasn't been updated for quite a while.
2022-04-18 17:52:12 +00:00
Elliotte Rusty Harold
bbc4f8dff8
add JDK 8 target (#9743) 2022-04-18 12:47:56 -04:00
Jason Lunn
6b2627718b Add JRuby 9.3.4.0 to Dockerfile and use it for 9.3.x testing. 2022-03-29 19:01:05 +00:00
Jon Skeet
9f597a4c59 Add regression test for https://github.com/protocolbuffers/protobuf/issues/9526 2022-03-29 16:38:30 +01:00
Marco Concetto Rudilosso
abdfd093af
Allow pre-compiled binaries for ruby 3.1.0 (#9566)
* Allow pre-compiled binaries for ruby 3.1.1

* add comment

* fix build and use ruby 3.1.0

* add ruby31 to build CI for tests and release

* trying to fix ci

* install ruby 3.1.0 in ruby_build_environment.sh

* use head for rvm to install 3.1.0

* just install master version of rvm in prepare_build_macos_rc

* force install of master rvm in ruby_build_environment.sh

* Use coroutine=universal when compiling ruby31

* use ucontext

* fix filename

* fix coroutine name

* use git head for rake-compiler-dock

* use newest rake-compiler-dock version
2022-03-17 09:38:36 -07:00
Adam Cozzette
388df0041a Update tests.sh to locally install all Java packages, not just util
The JRuby test runs appear to be fetching the util package from Maven
Central because it is not being installed locally. This causes the tests
to fail when we upgrade to a new version that has not yet been published
to Maven Central. This commit fixes the problem by locally installing
all the protobuf packages, not just util.
2022-03-04 09:52:19 -08:00
Jason Lunn
df0999141e Update JRuby 9.2.19.0 -> 9.2.20.1 and 9.3.0.0 to 9.3.3.0. 2022-02-25 14:11:12 +00:00
Joshua Haberman
301d315dc4
Dropped support for Python < 3.7 (#9480)
* Drop Python versions <3.7.

* Updated README to clarify that Python 3.7 is the minimum.

* Removed more Python 3.5-specific code.

Also changed tests to skip missing interpreters.

* Invoke tox directly instead of through Python.

Hopefully this will pick up python3.

* Updated java_stretch image to bullseye to get Python >= 3.7.

* Use jdk11 instead of jdk8.

* Installed python2 for gtest.

* Use "python3 -m venv" instead of "virtualenv."

* Install python3-venv.
2022-02-08 18:31:50 -08:00
Brett McBride
4c03fcf8fb php8.1 testing 2022-01-27 15:45:20 +11:00
Elliotte Rusty Harold
5711cab189
show less log junk from Maven
@perezd turn off de bugging messages such as

```
[DEBUG]   Imported: org.codehaus.plexus.configuration < plexus.core
```
2021-12-14 23:40:36 +00:00
Bu Sun Kim
c01cd6ec79
Add python 3.10 (#9034)
* Add python 3.10

* Update setup.py and tox.ini

* fix: fix 3.9 -> 3.10

* fix: py310-cpp
2021-10-15 10:24:49 -07:00
Jason Lunn
2ccc5df358 Install core rather than util 2021-10-11 19:10:09 +00:00
Jason Lunn
728878e98f Add JRuby test targets for JRuby 9.2.x and 9.3.x.
Standardize on JRuby 9.2.19.0 for building and for testing the 9.2.x branch.
2021-10-07 18:45:38 -04:00
Jason Lunn
0d3f95d7ae Narrow the scope of the install 2021-10-06 17:27:43 -04:00
Jason Lunn
2ed8173481 Bug fix for oracle7 regression 2021-10-06 16:40:09 -04:00
Jason Lunn
ddd7f46a7d Create an explicit build dependency from JRuby on Java.
Should prevent future flakiness if unreleased JRuby features are dependent on unreleased Java features by populating the local Maven repository with the artifacts needed when running `tests.sh jruby`.
2021-10-06 13:06:27 -04:00
Adam Cozzette
0e7a35f24b Update dist_install test to work around Python issue
googletest uses a Python script in its build which is not compatible
with Python 3. Unfortunately we can't easily upgrade googletest right
now, so this commit works around the problem by putting a python symlink
pointing to Python 2 in the $PATH.
2021-09-30 11:53:26 -07:00
Adam Cozzette
151e632bac Fix dist install test by ensuring that we use Python 3 (#9011)
* Fix dist install test by ensuring that we use Python 3

Now that we have dropped Python 2 support, we need to make sure this
install test uses Python 3.

* Update Docker image to install Python 3 version of setuptools

* Run pip3 instead of pip
2021-09-24 12:30:58 -07:00
Joshua Haberman
e5775e4fb6 Updated tests.sh to remove testing of old Python. 2021-09-09 19:06:50 -07:00
Joshua Haberman
d339e1748e Updated Ruby 3.0 test to 3.0.2. 2021-08-01 12:07:15 -07:00
Deanna Garcia
1e8457f8f8 Adding util tests 2021-06-24 22:06:55 +00:00
Deanna Garcia
41e89c7d5c Don't test kotlin if not in jdk8 2021-06-24 18:21:49 +00:00
Adam Cozzette
ca295506c5
Make the dist_install test use Java 8 (#8609)
This is necessary because the Kotlin support requires Java 8 or
higher.

Making this test use Java 8 required switching it to the java_stretch
Docker image and updating that image to include some Python
dependencies.
2021-05-12 14:42:15 -07:00
Adam Cozzette
5a9367ac57 Update tests.sh to skip the Kotlin tests on Oracle 7 2021-05-12 09:58:30 -07:00
Adam Cozzette
f98b287125
Remove Python compatibility tests (#8570) 2021-05-05 13:02:38 -07:00
Joshua Haberman
8b87075924
Updated and simplified PHP testing structure (#8558)
* Simplified PHP testing setup.

- Consolidated on a single autoloader, created by composer.
- Consolidated on a single phpunit invocation strategy: we run
  phpunit on a directory, which will run all tests matching *Test.php
  in that directory.
- We now rely on autoloading to import all test protos. require_once()
  calls for test protos are removed.
- For now the valgrind tests are removed. A follow-up PR will re-enable
  them in a more robust way.

* More improvements to PHP testing.

1. Replace custom PHPUnit-selection logic in test.sh with generic
   composer version selection.
2. Optimized both test proto generation and the custom extension
   build to avoid unnecessary work when the files are already up
   to date.

* Added assertions to verify that the C test doesn't use PHP sources.

* Updated tests.sh for the new PHP testing commands.

* Removed obsolete rules from tests.sh.

* Fixed generate_test_protos.sh for when tmp does not exist.

Also removed undefined_test.php and fixed Makefile.am.

* Added php8.0_all again which is still used.

* Added missing file to Makefile.am.

* Re-added php_all_32 rule which is also still used.

* Updated testing commands for macOS and download composer.

* Use /usr/local/bin on mac instead of /usr/bin, since the latter is not writable.
2021-05-04 10:19:22 -07:00
Derek Perez
d1b2eff440
Cleanup kokoro usages of java_compatibility and eliminate java/compatibility_tests. (#8504) 2021-04-20 09:36:06 -07:00
Paul Yang
525be941e0
Add test for php5 for pure php implementation (#8251)
* Add test for php5 for pure php implementation

* Add php5.6 to docker image

* Add php5.6 to php 32-bit docker image
2021-02-04 09:50:28 -08:00
gerben-s
10ecb080cb
Merge pull request #8121 from busunkim96/add-39
add python 3.9
2021-02-01 19:34:02 -08:00
Joshua Haberman
9abf6e2ab0
Ported Ruby extension to upb_msg (#8184)
* WIP.

* WIP.

* WIP.

* WIP.

* WIP.

* WIP.

* Added some missing files.

* WIP.

* WIP.

* Updated upb.

* Extension loads, but crashes immediately.

* Gets through the test suite without SEGV!

Still a lot of bugs to fix, but it is a major step!

214 tests, 378 assertions, 37 failures, 147 errors, 0 pendings, 0 omissions, 0 notifications
14.0187% passed

* Test and build for Ruby 3.0

* Fixed a few more bugs, efficient #inspect is almost done.

214 tests, 134243 assertions, 30 failures, 144 errors, 0 pendings, 0 omissions, 0 notifications
18.6916% passed

* Fixed message hash initialization and encode depth checking.

214 tests, 124651 assertions, 53 failures, 70 errors, 0 pendings, 0 omissions, 0 notifications
42.5234% passed

* A bunch of fixes to failing tests, now 70% passing.

214 tests, 202091 assertions, 41 failures, 23 errors, 0 pendings, 0 omissions, 0 notifications
70.0935% passed

* More than 80% of tests are passing now.

214 tests, 322331 assertions, 30 failures, 9 errors, 0 pendings, 0 omissions, 0 notifications
81.7757% passed

Unfortunately there is also a sporadic bug/segfault hanging around
that appears to be GC-related.

* Add linux/ruby30 and macos/ruby30

* Use rvm master for 3.0.0-preview2

* Over 90% of tests are passing!

214 tests, 349898 assertions, 15 failures, 1 errors, 0 pendings, 0 omissions, 0 notifications
92.5234% passed

* Passes all tests!

214 tests, 369388 assertions, 0 failures, 0 errors, 0 pendings, 0 omissions, 0 notifications
100% passed

* A bunch of cleanup.

1. Removed a bunch of internal-only symbols from headers.
2. Required a frozen check to get a non-const pointer to a map or array.
3. De-duplicated the code to get a type argument for Map/RepeatedField.

* Removed a bunch more stuff from protobuf.h.  There is an intermittent assert failure.

Intermittent failure:

ruby: ../../../../ext/google/protobuf_c/protobuf.c:263: ObjectCache_Add: Assertion `rb_funcall(obj_cache2, (__builtin_constant_p("[]") ? __extension__ ({ static ID rb_intern_id_cache; if (!rb_intern_id_cache) rb_intern_id_cache = rb_intern2((("[]")
), (long)strlen(("[]"))); (ID) rb_intern_id_cache; }) : rb_intern("[]")), 1, key_rb) == val' failed

* Removed a few more things from protobuf.h.

* Ruby 3.0.0-preview2 to 3.0.0

* Require rake-compiler-dock >= 1.1.0

* More progress, fighting with the object cache.

* Passes on all Ruby versions!

* Updated and clarified comment regarding WeakMap.

* Fixed the wyhash compile.

* Fixed conformance tests for Ruby.

Conformance results now look like:

RUBYLIB=../ruby/lib:. ./conformance-test-runner --enforce_recommended --failure_list failure_list_ruby.txt --text_format_failure_list text_format_failure_list_ruby.txt ./conformance_ruby.rb

CONFORMANCE TEST BEGIN ====================================

CONFORMANCE SUITE PASSED: 1955 successes, 0 skipped, 58 expected failures, 0 unexpected failures.

CONFORMANCE TEST BEGIN ====================================

CONFORMANCE SUITE PASSED: 0 successes, 111 skipped, 8 expected failures, 0 unexpected failures.

Fixes include:

- Changed Ruby compiler to no longer reject proto2 maps.
- Changed Ruby compiler to emit a warning when proto2 extensions are
  present instead of rejecting the .proto file completely.
- Fixed conformance tests to allow proto2 and look up message by name
  instead of hardcoding a specific list of messages.
- Fixed conformance test to support the "ignore unknown" option for
  JSON.
- Fixed conformance test to properly report serialization errors.

* Removed debug printf and fixed #inspect for floats.

* Fixed compatibility test to have proper semantics for #to_json.

* Updated Makefile.am with new file list.

* Don't try to copy wyhash when inside Docker.

* Fixed bug where we would forget that a sub-object is frozen in Ruby >=2.7.

* Avoid exporting unneeded symbols and refactored a bit of code.

* Some more refactoring.

* Simplified and added more comments.

* Some more comments and simplification. Added a missing license block.

Co-authored-by: Masaki Hara <hara@wantedly.com>
2021-01-13 12:16:25 -08:00
Bu Sun Kim
523b109666
Update tests.sh 2020-12-15 17:08:56 -07:00
Rob Widmer
24b2094f8a Set up testing configuration for jruby 2020-12-11 16:02:41 -08:00
Paul Yang
823f351448
Port for php8 (#8105)
* Port for php8

* Port php c extension for php8

* Update composer.json

* Drop php7.0 support

* Update phpunit for php7.1 in c extension test

* Add back support for php7.0

* Add badge for php8 continuous build
2020-12-04 13:46:34 -08:00
Bu Sun Kim
51c80ce1d5 add python 3.9 2020-12-04 21:16:47 +00:00
Joshua Haberman
f5ab3b166c PHP: Removed 32-bit-specific PHP failure list.
32 and 64-bit PHP should have the same set of failures now.
2020-11-02 10:45:10 -08:00
Paul Yang
4eea2b00a3
Remove php 5 tests on mac (#7956) 2020-10-09 15:53:12 -07:00
David L. Jones
b774da0d42
Change Python kokoro configs to test only their specified version. (#7927)
Using non-versioned scripts for `build_file` means every Python version is
tested (under Tox), and this is multiplied by each Python version running Tox.

For example, when the `python38` config is used to run Tox, the
Tox layer runs on Python 3.8. This then tests all of the Python versions
individually, including, for example, python27 tests.

This change fixes the `build_file` paths to point to the same-pathed build
script. For example, `kokoro/linux/python27/presubmit.cfg` now uses
`kokoro/linux/python27/build.sh`.

Some additional fixes:

*  Use `python -m tox` in tests.sh instead of just `tox`. This helps non-site
   installations of tox, where the `tox` script may not be on `$PATH`.
*  Ensure tox (and other Python build-related packages) are available in
   Python testing images. (New images have been pushed.)
*  Disable `--warnings_as_errors` due to deprecated function.
*  Remove apt lists per [Docker best practices][1].

[1]: https://docs.docker.com/develop/develop-images/dockerfile_best-practices/
2020-09-30 15:56:17 -07:00
David L. Jones
ff06e23d23
Add a setuptools extension for generating Python protobufs (#7783)
This extension allows Python code to be generated from setup.py files, so they
are created as part of a normal Python build. The extension uses an already-existing
protoc binary, which can be explicitly specified if needed.
2020-08-31 10:40:57 -07:00
Paul Yang
d4ca92962c
Port php c extension to php8 (#7793)
* Only ported c extension to php8.
* Didn't fixed the issue of throwing warnings for missing arginfo in bundled files.
* Tests not fixed, because syntax of phpunit (<7 vs >9.3) are not compatible.
* In next release, needs to drop php5 and php7.0 support (in order to use phpunit > 7)
2020-08-11 19:30:46 -07:00
David L. Jones
0f9ae07fe0 Override pip and virtualenv installation for macOS. (#7704)
This commit cleans up our environment setup for macOS builds.

-  Always use `python -m {pip | virtualenv}` form of commands.
-  Only upgrade in-place, and never use `--ignore-installed`.
-  Use latest tox on macos, and test 3.6 on macos presubmit.

These changes ensure that multibuilder won't break the system-installed Python on the Kokoro macOS builder, and thus won't resolve the ancient system version. For Tox, make sure we test with Python 3.6 (the version of Python in current macOS).
2020-08-07 14:44:49 -07:00
Joshua Haberman
093faebcdb
New PHP C Extension based on upb_msg (#7614)
* Added code for the new PHP extension.

* Removed a bunch of unused variables shown by compiler warnings.

* Test protobuf2 in the tests.

* Updated upb version to fix a goldenfile test.

* Added pure-PHP changes. Extension now passes all tests.

* Enabled protobuf2 for all C extension tests.

* Fixed pure=PHP lib: full names no longer start with '.'.

* Added files for new extension to Makefile.am.

* Downgraded make-preload.py to python 2, since python3 isn't available in the php_all Kokoro image.

* Disable tests of new C extension with PHP 5.x.

* Also do not compile the extension for PHP5.

* Accept version 5.*.*, and use /usr/bin/python.

* Addressed PR comments.

* Addressed PR comments.

* Added "const" to a parameter and fixed a memory leak seen in Valgrind.

* Stop testing the C extension for PHP5.

The next release of protobuf will deprecate the C extension
for PHP5, see:
https://github.com/protocolbuffers/protobuf/issues/7525

* Made the PHP5.6 Mac test only exercise pure-PHP.

* Build protoc for PHP 5.6 test.

* Rewrote bundling script in PHP to avoid dependency on Python.

* A few more fixes.

* Fixed int32/int64 behavior for 32-bit builds.

* Match more PHP versions in testing script.

* Use phpize --clean before building extension.

* Force-delete configure.in to avoid phpize problems cross-version.

* Delete both configure.ac and configure.in.
2020-06-25 12:21:01 -07:00
Joshua Haberman
f1ce8663ac
De-duplicated code to generate PHP protos and install phpunit. (#7605)
* De-duplicated code to generate PHP protos and install phpunit.

* Removed all references to generate_php_test_proto.

* Replaced with internal_build_cpp.

* Make Timestamp::__construct() static to avoid conflicts with MongoDB.

* Replicated PHPUnit versions and added new script to Makefile.am.

* Fixed filename in Makefile.am.

* Disabled test that SEGV's on macOS.

* Removed extraneous "set -e".

* Make sure generate_protos.sh happens on every test path.

* Removed stray '$' chars.

* Added proper support for aggregate_metadata tests. But now I get a stack overflow.

Stack overflow:
/Users/haberman/code/protobuf/php/tests/generated/GPBMetadata/Proto/TestDescriptors.php:16
/Users/haberman/code/protobuf/php/tests/generated/GPBMetadata/Proto/TestDescriptors.php:16
/Users/haberman/code/protobuf/php/tests/generated/GPBMetadata/Proto/TestDescriptors.php:16
/Users/haberman/code/protobuf/php/tests/generated/GPBMetadata/Proto/TestDescriptors.php:16
/Users/haberman/code/protobuf/php/tests/generated/GPBMetadata/Proto/TestDescriptors.php:16
/Users/haberman/code/protobuf/php/tests/generated/GPBMetadata/Proto/TestDescriptors.php:16
/Users/haberman/code/protobuf/php/tests/generated/GPBMetadata/Proto/TestDescriptors.php:16
/Users/haberman/code/protobuf/php/tests/generated/GPBMetadata/Proto/TestDescriptors.php:16

namespace GPBMetadata\Proto;

class TestDescriptors
{
    public static $is_initialized = false;

    public static function initOnce() {
        $pool = \Google\Protobuf\Internal\DescriptorPool::getGeneratedPool();
        if (static::$is_initialized == true) {
          return;
        }
        \GPBMetadata\Proto\TestDescriptors::initOnce();
        $pool->internalAddGeneratedFile(hex2bin(
            ""
        ), true);
        static::$is_initialized = true;
    }
}

* Fixed and verified metadata aggregation testing.
2020-06-09 15:40:29 -07:00