Commit Graph

296 Commits

Author SHA1 Message Date
Joshua Haberman
96307d28f6 Sync from Piper @364389596
PROTOBUF_SYNC_PIPER
2021-03-22 13:05:55 -07:00
Jan Tattermusch
aa13fde017
remove obsolete scripts for python wheels (#8250)
Nowadays, this seems to be script where python wheels are being built:
https://github.com/protocolbuffers/protobuf/blob/ac63e80/kokoro/release/python/linux/build_artifacts.sh

Removing the obsolete scripts to avoid confusion.
2021-02-02 16:53:26 -08:00
Rob Widmer
48234f5f01 Make the new multi-level ruby tests fit the existing structure better and add them to the makefile 2021-01-14 12:26:04 -08:00
Joshua Haberman
9abf6e2ab0
Ported Ruby extension to upb_msg (#8184)
* WIP.

* WIP.

* WIP.

* WIP.

* WIP.

* WIP.

* Added some missing files.

* WIP.

* WIP.

* Updated upb.

* Extension loads, but crashes immediately.

* Gets through the test suite without SEGV!

Still a lot of bugs to fix, but it is a major step!

214 tests, 378 assertions, 37 failures, 147 errors, 0 pendings, 0 omissions, 0 notifications
14.0187% passed

* Test and build for Ruby 3.0

* Fixed a few more bugs, efficient #inspect is almost done.

214 tests, 134243 assertions, 30 failures, 144 errors, 0 pendings, 0 omissions, 0 notifications
18.6916% passed

* Fixed message hash initialization and encode depth checking.

214 tests, 124651 assertions, 53 failures, 70 errors, 0 pendings, 0 omissions, 0 notifications
42.5234% passed

* A bunch of fixes to failing tests, now 70% passing.

214 tests, 202091 assertions, 41 failures, 23 errors, 0 pendings, 0 omissions, 0 notifications
70.0935% passed

* More than 80% of tests are passing now.

214 tests, 322331 assertions, 30 failures, 9 errors, 0 pendings, 0 omissions, 0 notifications
81.7757% passed

Unfortunately there is also a sporadic bug/segfault hanging around
that appears to be GC-related.

* Add linux/ruby30 and macos/ruby30

* Use rvm master for 3.0.0-preview2

* Over 90% of tests are passing!

214 tests, 349898 assertions, 15 failures, 1 errors, 0 pendings, 0 omissions, 0 notifications
92.5234% passed

* Passes all tests!

214 tests, 369388 assertions, 0 failures, 0 errors, 0 pendings, 0 omissions, 0 notifications
100% passed

* A bunch of cleanup.

1. Removed a bunch of internal-only symbols from headers.
2. Required a frozen check to get a non-const pointer to a map or array.
3. De-duplicated the code to get a type argument for Map/RepeatedField.

* Removed a bunch more stuff from protobuf.h.  There is an intermittent assert failure.

Intermittent failure:

ruby: ../../../../ext/google/protobuf_c/protobuf.c:263: ObjectCache_Add: Assertion `rb_funcall(obj_cache2, (__builtin_constant_p("[]") ? __extension__ ({ static ID rb_intern_id_cache; if (!rb_intern_id_cache) rb_intern_id_cache = rb_intern2((("[]")
), (long)strlen(("[]"))); (ID) rb_intern_id_cache; }) : rb_intern("[]")), 1, key_rb) == val' failed

* Removed a few more things from protobuf.h.

* Ruby 3.0.0-preview2 to 3.0.0

* Require rake-compiler-dock >= 1.1.0

* More progress, fighting with the object cache.

* Passes on all Ruby versions!

* Updated and clarified comment regarding WeakMap.

* Fixed the wyhash compile.

* Fixed conformance tests for Ruby.

Conformance results now look like:

RUBYLIB=../ruby/lib:. ./conformance-test-runner --enforce_recommended --failure_list failure_list_ruby.txt --text_format_failure_list text_format_failure_list_ruby.txt ./conformance_ruby.rb

CONFORMANCE TEST BEGIN ====================================

CONFORMANCE SUITE PASSED: 1955 successes, 0 skipped, 58 expected failures, 0 unexpected failures.

CONFORMANCE TEST BEGIN ====================================

CONFORMANCE SUITE PASSED: 0 successes, 111 skipped, 8 expected failures, 0 unexpected failures.

Fixes include:

- Changed Ruby compiler to no longer reject proto2 maps.
- Changed Ruby compiler to emit a warning when proto2 extensions are
  present instead of rejecting the .proto file completely.
- Fixed conformance tests to allow proto2 and look up message by name
  instead of hardcoding a specific list of messages.
- Fixed conformance test to support the "ignore unknown" option for
  JSON.
- Fixed conformance test to properly report serialization errors.

* Removed debug printf and fixed #inspect for floats.

* Fixed compatibility test to have proper semantics for #to_json.

* Updated Makefile.am with new file list.

* Don't try to copy wyhash when inside Docker.

* Fixed bug where we would forget that a sub-object is frozen in Ruby >=2.7.

* Avoid exporting unneeded symbols and refactored a bit of code.

* Some more refactoring.

* Simplified and added more comments.

* Some more comments and simplification. Added a missing license block.

Co-authored-by: Masaki Hara <hara@wantedly.com>
2021-01-13 12:16:25 -08:00
Rob Widmer
351a34f666 Added some missing jruby files to the Makefile 2020-12-11 16:02:41 -08:00
James Newton-King
79f5bad83c
Change ByteString to use memory and support unsafe create without copy 2020-11-18 21:58:32 +13:00
Joshua Haberman
f65bcca31d Added extra files to Makefile.am and fixed spelling errors. 2020-11-02 09:44:46 -08:00
Joshua Haberman
e0151addd3 Added missing files to Makefile.am. 2020-10-29 11:04:54 -07:00
Joshua Haberman
a29f792bae Updated Makefile.am for changed files. 2020-10-29 10:57:04 -07:00
Adam Cozzette
58f4260e8e Merged 3.13.x into master 2020-08-17 11:06:11 -07:00
Paul Yang
d4ca92962c
Port php c extension to php8 (#7793)
* Only ported c extension to php8.
* Didn't fixed the issue of throwing warnings for missing arginfo in bundled files.
* Tests not fixed, because syntax of phpunit (<7 vs >9.3) are not compatible.
* In next release, needs to drop php5 and php7.0 support (in order to use phpunit > 7)
2020-08-11 19:30:46 -07:00
Joshua Haberman
ad5a215359 Added proto3 presence support for PHP (#7724)
* WIP.

* Added proto3 presence support for PHP.

* Added compatibility code for old generated code.
2020-08-07 14:44:49 -07:00
Yannic Bonenberger
6b1d0f1149 Add objectivec/BUILD to objectivec_EXTRA_DIST files 2020-07-20 16:51:30 -07:00
Joshua Haberman
ffb2b53834
Remove old PHP extension (#7695)
* Removed references to legacy class names from generated code.

* Removed old PHP extension in favor of new extension. This drops PHP5 compat.

* Updated failure lists for things fixed by the new extension.

* Updated Makefile.am for new file list.

* Fix distcheck.
2020-07-09 11:06:24 -07:00
Jan Tattermusch
206b973afd
Merge pull request #7576 from jtattermusch/protobuf_buffer_serialization
New Span-based serialization logic (followup for #7351)
2020-06-29 09:26:25 +02:00
Joshua Haberman
093faebcdb
New PHP C Extension based on upb_msg (#7614)
* Added code for the new PHP extension.

* Removed a bunch of unused variables shown by compiler warnings.

* Test protobuf2 in the tests.

* Updated upb version to fix a goldenfile test.

* Added pure-PHP changes. Extension now passes all tests.

* Enabled protobuf2 for all C extension tests.

* Fixed pure=PHP lib: full names no longer start with '.'.

* Added files for new extension to Makefile.am.

* Downgraded make-preload.py to python 2, since python3 isn't available in the php_all Kokoro image.

* Disable tests of new C extension with PHP 5.x.

* Also do not compile the extension for PHP5.

* Accept version 5.*.*, and use /usr/bin/python.

* Addressed PR comments.

* Addressed PR comments.

* Added "const" to a parameter and fixed a memory leak seen in Valgrind.

* Stop testing the C extension for PHP5.

The next release of protobuf will deprecate the C extension
for PHP5, see:
https://github.com/protocolbuffers/protobuf/issues/7525

* Made the PHP5.6 Mac test only exercise pure-PHP.

* Build protoc for PHP 5.6 test.

* Rewrote bundling script in PHP to avoid dependency on Python.

* A few more fixes.

* Fixed int32/int64 behavior for 32-bit builds.

* Match more PHP versions in testing script.

* Use phpize --clean before building extension.

* Force-delete configure.in to avoid phpize problems cross-version.

* Delete both configure.ac and configure.in.
2020-06-25 12:21:01 -07:00
Jan Tattermusch
9d6969bec4 Merge remote-tracking branch 'upstream/master' into protobuf_buffer_serialization 2020-06-11 16:40:27 +02:00
Jan Tattermusch
e967fac23a update Makefile.am 2020-06-11 09:30:21 +02:00
Jan Tattermusch
2350e90a85 add new C# files to Makefile.am 2020-06-10 16:58:22 +02:00
Joshua Haberman
f1ce8663ac
De-duplicated code to generate PHP protos and install phpunit. (#7605)
* De-duplicated code to generate PHP protos and install phpunit.

* Removed all references to generate_php_test_proto.

* Replaced with internal_build_cpp.

* Make Timestamp::__construct() static to avoid conflicts with MongoDB.

* Replicated PHPUnit versions and added new script to Makefile.am.

* Fixed filename in Makefile.am.

* Disabled test that SEGV's on macOS.

* Removed extraneous "set -e".

* Make sure generate_protos.sh happens on every test path.

* Removed stray '$' chars.

* Added proper support for aggregate_metadata tests. But now I get a stack overflow.

Stack overflow:
/Users/haberman/code/protobuf/php/tests/generated/GPBMetadata/Proto/TestDescriptors.php:16
/Users/haberman/code/protobuf/php/tests/generated/GPBMetadata/Proto/TestDescriptors.php:16
/Users/haberman/code/protobuf/php/tests/generated/GPBMetadata/Proto/TestDescriptors.php:16
/Users/haberman/code/protobuf/php/tests/generated/GPBMetadata/Proto/TestDescriptors.php:16
/Users/haberman/code/protobuf/php/tests/generated/GPBMetadata/Proto/TestDescriptors.php:16
/Users/haberman/code/protobuf/php/tests/generated/GPBMetadata/Proto/TestDescriptors.php:16
/Users/haberman/code/protobuf/php/tests/generated/GPBMetadata/Proto/TestDescriptors.php:16
/Users/haberman/code/protobuf/php/tests/generated/GPBMetadata/Proto/TestDescriptors.php:16

namespace GPBMetadata\Proto;

class TestDescriptors
{
    public static $is_initialized = false;

    public static function initOnce() {
        $pool = \Google\Protobuf\Internal\DescriptorPool::getGeneratedPool();
        if (static::$is_initialized == true) {
          return;
        }
        \GPBMetadata\Proto\TestDescriptors::initOnce();
        $pool->internalAddGeneratedFile(hex2bin(
            ""
        ), true);
        static::$is_initialized = true;
    }
}

* Fixed and verified metadata aggregation testing.
2020-06-09 15:40:29 -07:00
Jan Tattermusch
7cb5597013
Overcome the ref struct limitation for pre-roslyn compilers by introducing GOOGLE_PROTOBUF_REFSTRUCT_COMPATIBILITY_MODE for generated code (#7490)
* enable compatibility mode in codegen

* regenerate protos

* improve readability

* more robust way of figuring out path to old C# compiler

* add recent C# changes
2020-06-02 00:26:25 -07:00
David Ostrovsky
6bbd56dfd9 Bazel: Discontinue JDK 7 compatibility support
The primary reason to drop that compatibility is because Bazel is
adding unconditionally -parameters option that is not compatible
with JDK 7. This pollutes the build log with annoying warnings:

  warning: -parameters is not supported for target value 1.7. \
  Use 1.8 or later.

Second reason is that nobody in the wild relies on JDK 7 any more
because it was discontinued years ago.

Also remove JDK9 config_setting rule that is not used any more.

Test Plan:

Build :protobuf_java and confirm, that there are no warnings any more
and that major byte version 52 is produced, that corresponds to Java 8:

  $ bazel build protobuf_java
  $ javap -v -cp bazel-bin/java/core/libcore.jar com.google.protobuf.Any | grep major
  major version: 52
2020-05-11 10:01:00 -07:00
Adam Cozzette
fa9ebaf807 Fixed the cpp_distcheck test 2020-05-11 08:59:35 -07:00
Jan Tattermusch
0757d8ee5c update Makefile.am 2020-05-11 11:24:48 +02:00
Jan Tattermusch
7134d25eb8 Merge remote-tracking branch 'upstream/master' into protobuf_csharp_new_parsing 2020-04-27 13:18:07 +02:00
Jon Skeet
81c9b85dee Add new files into Makefile.am 2020-04-24 17:39:14 +01:00
Jan Tattermusch
eac2a6a510 update C# distrib files in Makefile.am 2020-04-22 11:18:16 +02:00
Jan Tattermusch
50e03cdde3
Merge pull request #7360 from jtattermusch/refactor_googlebenchmark
Refactor some Protobuf C# microbenchmarks
2020-04-13 17:34:22 +02:00
Joshua Haberman
b7742c51fd Sync from Piper @305505267
PROTOBUF_SYNC_PIPER
2020-04-08 10:30:17 -07:00
Jan Tattermusch
c762a3bb2a update Makefile.am 2020-04-07 12:01:29 +02:00
Rafi Kamal
53128559ae
Update missing/renamed kernel files (#7310) 2020-03-17 12:16:12 -07:00
Jan Tattermusch
447b6bbad8 address review comments 2020-03-11 07:08:46 -04:00
David L. Jones
73f3a7209b Fix typo in Makefile.am.
This is a fix for a typo in commit 743a4322ba.

The missing newline continuation causes the re-generated Makefile to yield a parse error.
2020-02-27 11:26:39 -08:00
Artem Kustikov
743a4322ba
Fix js message pivot selection (#6813)
* fix javascript setFieldIgnoringDefault_ logic

* remove package-lock.json

* fix build script to include new UT asset file

Co-authored-by: Daniel Kurka <kurka.daniel@gmail.com>
2020-02-27 09:10:16 -08:00
Yannic
f0cb9cdb95
[bazel] Move Java runtime/toolchains into //java (#7190)
* [bazel] Move Java runtime/toolchains into //java

This change moves `java_library` targets from the top-level BUILD file
into `//java/{core,lite,util}` and declares `alias` targets to point to
their new locations (hence, this is not a breaking change).

This will allow users that don't use Java to stop depending on
`@rules_java` (e.g. as requested in
https://github.com/bazelbuild/rules_scala/pull/989#issuecomment-583405161).

Note that there is no intention to deprecate + remove the top-level
targets in the foreseeable future.

* Add BUILD files in //java to java_EXTRA_DIST
2020-02-13 13:04:14 -08:00
Thomas Van Lenten
56c48ae592 Revisit how the WKTs are bundled with ObjC.
There are have been a few issues around people using case sensitive file systems
what Xcode/clang does when looking at the paths. In attempts to solve one set of
warnings, new warnings/errors happened in different setup. So, to hopefully put
these problem away for got, move the WKTs to be at the same level as the other
headers.

- Revert "Override CocoaPods module to lowercase (#6464)"
  This reverts commit 479ba8226b.
- Move WKTs to the objectivec directory and make the old headers shim back to
  the new locations.
- Update objectivec/generate_well_known_types.sh to check them one at a time
  and to deal with the new locations for them.

Fixes #6803
2020-02-10 12:20:05 -05:00
Rafi Kamal
7e0a081e78
Add js files added in #7176 to Makefiles (#7189) 2020-02-07 17:14:49 -08:00
Dave MacLachlan
ab48ecf140 Update pddm to work with clang-format
Add clang-format off/on directives around pddm expansions to simplify formatting other code.
2020-01-21 13:14:01 -05:00
Dave MacLachlan
74956e1c15 Use references to Objective C classes instead of looking classes up by name.
This should reduce binary size slightly, small performance improvement, and improve linkage by forcing references to all used classes.

Note that this maintains backwards compatibility for sources generated by older protoc for the time being. If you want the benefits
you will need to recompile your protos with the newer protoc.
2020-01-15 15:01:53 -05:00
Rafi Kamal
4500f31a6a Merge branch 3.11.x to master 2020-01-07 11:52:29 -08:00
Paul Yang
fe1790ca0d
Fix Multiple Request for PHP (#7008)
* Add scripts to test multirequest

* chmod ug+x multirequest.sh

* Add continuous test

* Compile c extension

* Class entry is obsolete in the second request
1) Needes to use class name in persistent map
2) Invalidate class entry stored in descriptor

* Add new files to dist

* Fix compile_extension

* Cleanup outputs for phpize
2019-12-12 13:59:51 -08:00
Sydney Acksman
367fd27732 Rename files and revert changes to generate protos script 2019-12-03 10:52:24 -06:00
Sydney Acksman
2b0b838528 Add tests 2019-11-24 14:22:50 -06:00
Sydney Acksman
cbafc68d9e Add project to Makefile.am 2019-11-11 10:29:40 -06:00
Sydney Acksman
b84929f7fb Rename .CodeGen to .TestProtos in Makefile and script 2019-11-08 08:41:26 -06:00
Sydney Acksman
7e4be173ad Move generated files in Makefile.am 2019-11-08 07:23:50 -06:00
Jan Tattermusch
3f810569b3 update Makefile.am 2019-11-04 18:21:22 +01:00
Jon Skeet
aa22aa32f3 Fix Makefile.am by adding generated files 2019-11-04 16:35:38 +00:00
Jon Skeet
7581fd5ea6 Tests for issue 6822
(The implementation to fix this is in the following commit.)
2019-10-30 09:46:24 +00:00
Sydney Acksman
6da14fa94d Add GeneratedMessageTest.Proto2.cs to EXTRA_DIST 2019-08-23 17:29:59 -05:00