Commit Graph

328 Commits

Author SHA1 Message Date
Adam Cozzette
9638a5e531 Improve performance of parsing unknown fields in Java (#9371)
Credit should go to @elharo for most of these Java changes--I am just
cherry-picking them from our internal codebase. The one thing I did
change was to give the UTF-8 validation tests their own Bazel test
target. This makes it possible to give the other tests a shorter
timeout, which is important for UnknownFieldSetPerformanceTest in
particular.
2022-01-05 10:05:11 -08:00
Adam Cozzette
67c2a92f82
Ensure that release archives contain everything needed for Bazel (#9131)
This change adds some files to EXTRA_DIST in Makefile.am so that our
published tar and zip files will have everything needed for Bazel
builds. I also added a basic test for this so that next time we should
find out sooner if we're missing any important files.

This should fix #9129.
2021-10-21 06:47:17 -07:00
Adam Cozzette
f8a22b45c0
Remove references to six (#9096)
We no longer support Python 2 and therefore no longer need to depend on
six. I'm hoping this will make it possible to merge #8214.
2021-10-13 13:51:09 -07:00
Jason Lunn
3581d85ced Update JRuby to leverage pure-Ruby DSL.
JRuby unit and conformance test fixes.
2021-10-03 18:25:43 -04:00
Joshua Haberman
a3d24e34a9 Added new files to Makefile.am. 2021-09-28 22:38:35 +00:00
Joshua Haberman
2128ce074f Merge branch '3.18.x' into merge-release 2021-09-27 15:29:06 -07:00
Joshua Haberman
66a3e1a670 Added missing file to EXTRA_DIST. 2021-09-10 17:40:31 -07:00
Adam Cozzette
877e7b6472 Add new Python-C++ source files to Makefile.am 2021-08-25 13:43:50 -07:00
Adam Cozzette
8a3c4948a4
Merge pull request #8858 from acozzette/sync-stage
Integrate from Piper for C++, Java, and Python
2021-08-09 15:55:00 -07:00
Adam Cozzette
2fd5c96f4d Add map_test.inc to build configs and new Kotlin files to java_EXTRA_DIST 2021-08-03 14:13:43 -07:00
Joshua Haberman
fa8acc8e4c Removed wyhash files from the file list. 2021-08-02 22:23:03 -07:00
Joshua Haberman
d13afed563 Revert "Removed obsolete file from Makefile.am."
This reverts commit e9b368655a.
2021-08-02 20:47:40 -07:00
Joshua Haberman
726b7c2fed Removed obsolete file from Makefile.am. 2021-08-02 20:44:07 -07:00
Joshua Haberman
e9b368655a Removed obsolete file from Makefile.am. 2021-08-02 20:39:33 -07:00
Joshua Haberman
2538273fee Add new file to Makefile.am. 2021-07-31 13:13:45 -07:00
deannagarcia
611a08ee00
Merge pull request #8752 from deannagarcia/rulesJvmExternal
Create java_export targets
2021-06-25 13:49:30 -07:00
Deanna Garcia
c02b5824d0 Adding new pom template files to Makefile. 2021-06-22 17:04:03 +00:00
Adam Cozzette
aedf4e6316
Remove TimeUtil.java and TimeUtilTest.java from Makefile.am (#8750)
This should fix the failing distcheck test.
2021-06-18 16:58:40 -07:00
Benjamin Peterson
fa31183250
Add internal.bzl to EXTRA_DIST. (#8663)
internal.bzl is required to load java/core/BUILD and java/lite/BUILD, so it must be distributed.
2021-05-26 13:01:46 -07:00
Joshua Haberman
b0d90e3abe
Fixed PHP memory leaks and arginfo errors (#8614)
* Fixed a bunch of incorrect arginfo and a few incorrect error messages.

* Passes mem check test with no leaks!

* WIP.

* Fix build warning that was causing Bazel build to fail.

* Added compatibility code for PHP <8.0.

* Added test_valgrind target and made tests Valgrind-clean.

* Updated Valgrind test to fail if memory leaks are detected.

* Removed intermediate shell script so commands are easier to cut, paste, and modify.

* Passing all Valgrind tests!

* Hoist addref into ObjCache_Get().

* Removed special case of map descriptors by keying object map on upb_msgdef.

* Removed all remaining RETURN_ZVAL() macros.

* Removed all explicit reference add/del operations.

* Added REFCOUNTING.md to Makefile.am.
2021-05-13 23:16:49 -07:00
Joshua Haberman
8b87075924
Updated and simplified PHP testing structure (#8558)
* Simplified PHP testing setup.

- Consolidated on a single autoloader, created by composer.
- Consolidated on a single phpunit invocation strategy: we run
  phpunit on a directory, which will run all tests matching *Test.php
  in that directory.
- We now rely on autoloading to import all test protos. require_once()
  calls for test protos are removed.
- For now the valgrind tests are removed. A follow-up PR will re-enable
  them in a more robust way.

* More improvements to PHP testing.

1. Replace custom PHPUnit-selection logic in test.sh with generic
   composer version selection.
2. Optimized both test proto generation and the custom extension
   build to avoid unnecessary work when the files are already up
   to date.

* Added assertions to verify that the C test doesn't use PHP sources.

* Updated tests.sh for the new PHP testing commands.

* Removed obsolete rules from tests.sh.

* Fixed generate_test_protos.sh for when tmp does not exist.

Also removed undefined_test.php and fixed Makefile.am.

* Added php8.0_all again which is still used.

* Added missing file to Makefile.am.

* Re-added php_all_32 rule which is also still used.

* Updated testing commands for macOS and download composer.

* Use /usr/local/bin on mac instead of /usr/bin, since the latter is not writable.
2021-05-04 10:19:22 -07:00
Derek Perez
bbd6999c76
add missing files to Makefile.am for java bazel (#8522) 2021-04-21 17:50:12 -07:00
deannagarcia
76188afb94
Merge pull request #8272 from deannagarcia/kotlinProtos
Opensourcing kotlin protos
2021-04-21 10:29:26 -07:00
Yuriy Chernyshov
c784b65b2a
Remove unused python_protobuf.{cc,h} (#8513)
* Remove unused python_protobuf.{cc,h} #8513

* Fix automake input
2021-04-19 15:58:33 -07:00
Deanna Garcia
9aaaf70154 updating makefile 2021-03-31 17:23:02 +00:00
Deanna Garcia
b0acba39c9 cleaning up 2021-03-31 16:54:01 +00:00
Deanna Garcia
2c9aca5382 Adding another file to the makefile 2021-03-23 22:24:49 +00:00
Deanna Garcia
0454655fb0 updating typo in makefile 2021-03-23 22:16:06 +00:00
Deanna Garcia
6f5c6eb7ce Merge branch 'kotlinProtos' of https://github.com/deannagarcia/protobuf into kotlinProtos 2021-03-23 20:58:14 +00:00
Deanna Garcia
e263c7e9d6 Updating the makefile with new files 2021-03-23 20:57:58 +00:00
deannagarcia
87faccd18e
Merge branch 'master' into kotlinProtos 2021-03-23 13:47:07 -07:00
Deanna Garcia
1949746e9c Commit with --kotlin_out working 2021-03-23 20:10:53 +00:00
Joshua Haberman
96307d28f6 Sync from Piper @364389596
PROTOBUF_SYNC_PIPER
2021-03-22 13:05:55 -07:00
Jan Tattermusch
aa13fde017
remove obsolete scripts for python wheels (#8250)
Nowadays, this seems to be script where python wheels are being built:
https://github.com/protocolbuffers/protobuf/blob/ac63e80/kokoro/release/python/linux/build_artifacts.sh

Removing the obsolete scripts to avoid confusion.
2021-02-02 16:53:26 -08:00
Rob Widmer
48234f5f01 Make the new multi-level ruby tests fit the existing structure better and add them to the makefile 2021-01-14 12:26:04 -08:00
Joshua Haberman
9abf6e2ab0
Ported Ruby extension to upb_msg (#8184)
* WIP.

* WIP.

* WIP.

* WIP.

* WIP.

* WIP.

* Added some missing files.

* WIP.

* WIP.

* Updated upb.

* Extension loads, but crashes immediately.

* Gets through the test suite without SEGV!

Still a lot of bugs to fix, but it is a major step!

214 tests, 378 assertions, 37 failures, 147 errors, 0 pendings, 0 omissions, 0 notifications
14.0187% passed

* Test and build for Ruby 3.0

* Fixed a few more bugs, efficient #inspect is almost done.

214 tests, 134243 assertions, 30 failures, 144 errors, 0 pendings, 0 omissions, 0 notifications
18.6916% passed

* Fixed message hash initialization and encode depth checking.

214 tests, 124651 assertions, 53 failures, 70 errors, 0 pendings, 0 omissions, 0 notifications
42.5234% passed

* A bunch of fixes to failing tests, now 70% passing.

214 tests, 202091 assertions, 41 failures, 23 errors, 0 pendings, 0 omissions, 0 notifications
70.0935% passed

* More than 80% of tests are passing now.

214 tests, 322331 assertions, 30 failures, 9 errors, 0 pendings, 0 omissions, 0 notifications
81.7757% passed

Unfortunately there is also a sporadic bug/segfault hanging around
that appears to be GC-related.

* Add linux/ruby30 and macos/ruby30

* Use rvm master for 3.0.0-preview2

* Over 90% of tests are passing!

214 tests, 349898 assertions, 15 failures, 1 errors, 0 pendings, 0 omissions, 0 notifications
92.5234% passed

* Passes all tests!

214 tests, 369388 assertions, 0 failures, 0 errors, 0 pendings, 0 omissions, 0 notifications
100% passed

* A bunch of cleanup.

1. Removed a bunch of internal-only symbols from headers.
2. Required a frozen check to get a non-const pointer to a map or array.
3. De-duplicated the code to get a type argument for Map/RepeatedField.

* Removed a bunch more stuff from protobuf.h.  There is an intermittent assert failure.

Intermittent failure:

ruby: ../../../../ext/google/protobuf_c/protobuf.c:263: ObjectCache_Add: Assertion `rb_funcall(obj_cache2, (__builtin_constant_p("[]") ? __extension__ ({ static ID rb_intern_id_cache; if (!rb_intern_id_cache) rb_intern_id_cache = rb_intern2((("[]")
), (long)strlen(("[]"))); (ID) rb_intern_id_cache; }) : rb_intern("[]")), 1, key_rb) == val' failed

* Removed a few more things from protobuf.h.

* Ruby 3.0.0-preview2 to 3.0.0

* Require rake-compiler-dock >= 1.1.0

* More progress, fighting with the object cache.

* Passes on all Ruby versions!

* Updated and clarified comment regarding WeakMap.

* Fixed the wyhash compile.

* Fixed conformance tests for Ruby.

Conformance results now look like:

RUBYLIB=../ruby/lib:. ./conformance-test-runner --enforce_recommended --failure_list failure_list_ruby.txt --text_format_failure_list text_format_failure_list_ruby.txt ./conformance_ruby.rb

CONFORMANCE TEST BEGIN ====================================

CONFORMANCE SUITE PASSED: 1955 successes, 0 skipped, 58 expected failures, 0 unexpected failures.

CONFORMANCE TEST BEGIN ====================================

CONFORMANCE SUITE PASSED: 0 successes, 111 skipped, 8 expected failures, 0 unexpected failures.

Fixes include:

- Changed Ruby compiler to no longer reject proto2 maps.
- Changed Ruby compiler to emit a warning when proto2 extensions are
  present instead of rejecting the .proto file completely.
- Fixed conformance tests to allow proto2 and look up message by name
  instead of hardcoding a specific list of messages.
- Fixed conformance test to support the "ignore unknown" option for
  JSON.
- Fixed conformance test to properly report serialization errors.

* Removed debug printf and fixed #inspect for floats.

* Fixed compatibility test to have proper semantics for #to_json.

* Updated Makefile.am with new file list.

* Don't try to copy wyhash when inside Docker.

* Fixed bug where we would forget that a sub-object is frozen in Ruby >=2.7.

* Avoid exporting unneeded symbols and refactored a bit of code.

* Some more refactoring.

* Simplified and added more comments.

* Some more comments and simplification. Added a missing license block.

Co-authored-by: Masaki Hara <hara@wantedly.com>
2021-01-13 12:16:25 -08:00
Rob Widmer
351a34f666 Added some missing jruby files to the Makefile 2020-12-11 16:02:41 -08:00
James Newton-King
79f5bad83c
Change ByteString to use memory and support unsafe create without copy 2020-11-18 21:58:32 +13:00
Joshua Haberman
f65bcca31d Added extra files to Makefile.am and fixed spelling errors. 2020-11-02 09:44:46 -08:00
Joshua Haberman
e0151addd3 Added missing files to Makefile.am. 2020-10-29 11:04:54 -07:00
Joshua Haberman
a29f792bae Updated Makefile.am for changed files. 2020-10-29 10:57:04 -07:00
Adam Cozzette
58f4260e8e Merged 3.13.x into master 2020-08-17 11:06:11 -07:00
Paul Yang
d4ca92962c
Port php c extension to php8 (#7793)
* Only ported c extension to php8.
* Didn't fixed the issue of throwing warnings for missing arginfo in bundled files.
* Tests not fixed, because syntax of phpunit (<7 vs >9.3) are not compatible.
* In next release, needs to drop php5 and php7.0 support (in order to use phpunit > 7)
2020-08-11 19:30:46 -07:00
Joshua Haberman
ad5a215359 Added proto3 presence support for PHP (#7724)
* WIP.

* Added proto3 presence support for PHP.

* Added compatibility code for old generated code.
2020-08-07 14:44:49 -07:00
Yannic Bonenberger
6b1d0f1149 Add objectivec/BUILD to objectivec_EXTRA_DIST files 2020-07-20 16:51:30 -07:00
Joshua Haberman
ffb2b53834
Remove old PHP extension (#7695)
* Removed references to legacy class names from generated code.

* Removed old PHP extension in favor of new extension. This drops PHP5 compat.

* Updated failure lists for things fixed by the new extension.

* Updated Makefile.am for new file list.

* Fix distcheck.
2020-07-09 11:06:24 -07:00
Jan Tattermusch
206b973afd
Merge pull request #7576 from jtattermusch/protobuf_buffer_serialization
New Span-based serialization logic (followup for #7351)
2020-06-29 09:26:25 +02:00
Joshua Haberman
093faebcdb
New PHP C Extension based on upb_msg (#7614)
* Added code for the new PHP extension.

* Removed a bunch of unused variables shown by compiler warnings.

* Test protobuf2 in the tests.

* Updated upb version to fix a goldenfile test.

* Added pure-PHP changes. Extension now passes all tests.

* Enabled protobuf2 for all C extension tests.

* Fixed pure=PHP lib: full names no longer start with '.'.

* Added files for new extension to Makefile.am.

* Downgraded make-preload.py to python 2, since python3 isn't available in the php_all Kokoro image.

* Disable tests of new C extension with PHP 5.x.

* Also do not compile the extension for PHP5.

* Accept version 5.*.*, and use /usr/bin/python.

* Addressed PR comments.

* Addressed PR comments.

* Added "const" to a parameter and fixed a memory leak seen in Valgrind.

* Stop testing the C extension for PHP5.

The next release of protobuf will deprecate the C extension
for PHP5, see:
https://github.com/protocolbuffers/protobuf/issues/7525

* Made the PHP5.6 Mac test only exercise pure-PHP.

* Build protoc for PHP 5.6 test.

* Rewrote bundling script in PHP to avoid dependency on Python.

* A few more fixes.

* Fixed int32/int64 behavior for 32-bit builds.

* Match more PHP versions in testing script.

* Use phpize --clean before building extension.

* Force-delete configure.in to avoid phpize problems cross-version.

* Delete both configure.ac and configure.in.
2020-06-25 12:21:01 -07:00
Jan Tattermusch
9d6969bec4 Merge remote-tracking branch 'upstream/master' into protobuf_buffer_serialization 2020-06-11 16:40:27 +02:00
Jan Tattermusch
e967fac23a update Makefile.am 2020-06-11 09:30:21 +02:00