Commit Graph

7846 Commits

Author SHA1 Message Date
Jan Tattermusch
f6da785731
Merge pull request #8143 from JamesNK/jamesnk/messageadapter
Optimize MapField serialization by removing MessageAdapter
2021-01-26 16:26:50 +01:00
James Newton-King
69223b8386
Clean up 2021-01-21 13:36:00 +13:00
James Newton-King
542c0cc9d3
Fix missing key and test 2021-01-21 13:34:08 +13:00
Rob Widmer
48234f5f01 Make the new multi-level ruby tests fit the existing structure better and add them to the makefile 2021-01-14 12:26:04 -08:00
Rob Widmer
1d39f78818 Fix jruby support to handle messages nested more than 1 level deep 2021-01-14 12:26:04 -08:00
Joshua Haberman
9abf6e2ab0
Ported Ruby extension to upb_msg (#8184)
* WIP.

* WIP.

* WIP.

* WIP.

* WIP.

* WIP.

* Added some missing files.

* WIP.

* WIP.

* Updated upb.

* Extension loads, but crashes immediately.

* Gets through the test suite without SEGV!

Still a lot of bugs to fix, but it is a major step!

214 tests, 378 assertions, 37 failures, 147 errors, 0 pendings, 0 omissions, 0 notifications
14.0187% passed

* Test and build for Ruby 3.0

* Fixed a few more bugs, efficient #inspect is almost done.

214 tests, 134243 assertions, 30 failures, 144 errors, 0 pendings, 0 omissions, 0 notifications
18.6916% passed

* Fixed message hash initialization and encode depth checking.

214 tests, 124651 assertions, 53 failures, 70 errors, 0 pendings, 0 omissions, 0 notifications
42.5234% passed

* A bunch of fixes to failing tests, now 70% passing.

214 tests, 202091 assertions, 41 failures, 23 errors, 0 pendings, 0 omissions, 0 notifications
70.0935% passed

* More than 80% of tests are passing now.

214 tests, 322331 assertions, 30 failures, 9 errors, 0 pendings, 0 omissions, 0 notifications
81.7757% passed

Unfortunately there is also a sporadic bug/segfault hanging around
that appears to be GC-related.

* Add linux/ruby30 and macos/ruby30

* Use rvm master for 3.0.0-preview2

* Over 90% of tests are passing!

214 tests, 349898 assertions, 15 failures, 1 errors, 0 pendings, 0 omissions, 0 notifications
92.5234% passed

* Passes all tests!

214 tests, 369388 assertions, 0 failures, 0 errors, 0 pendings, 0 omissions, 0 notifications
100% passed

* A bunch of cleanup.

1. Removed a bunch of internal-only symbols from headers.
2. Required a frozen check to get a non-const pointer to a map or array.
3. De-duplicated the code to get a type argument for Map/RepeatedField.

* Removed a bunch more stuff from protobuf.h.  There is an intermittent assert failure.

Intermittent failure:

ruby: ../../../../ext/google/protobuf_c/protobuf.c:263: ObjectCache_Add: Assertion `rb_funcall(obj_cache2, (__builtin_constant_p("[]") ? __extension__ ({ static ID rb_intern_id_cache; if (!rb_intern_id_cache) rb_intern_id_cache = rb_intern2((("[]")
), (long)strlen(("[]"))); (ID) rb_intern_id_cache; }) : rb_intern("[]")), 1, key_rb) == val' failed

* Removed a few more things from protobuf.h.

* Ruby 3.0.0-preview2 to 3.0.0

* Require rake-compiler-dock >= 1.1.0

* More progress, fighting with the object cache.

* Passes on all Ruby versions!

* Updated and clarified comment regarding WeakMap.

* Fixed the wyhash compile.

* Fixed conformance tests for Ruby.

Conformance results now look like:

RUBYLIB=../ruby/lib:. ./conformance-test-runner --enforce_recommended --failure_list failure_list_ruby.txt --text_format_failure_list text_format_failure_list_ruby.txt ./conformance_ruby.rb

CONFORMANCE TEST BEGIN ====================================

CONFORMANCE SUITE PASSED: 1955 successes, 0 skipped, 58 expected failures, 0 unexpected failures.

CONFORMANCE TEST BEGIN ====================================

CONFORMANCE SUITE PASSED: 0 successes, 111 skipped, 8 expected failures, 0 unexpected failures.

Fixes include:

- Changed Ruby compiler to no longer reject proto2 maps.
- Changed Ruby compiler to emit a warning when proto2 extensions are
  present instead of rejecting the .proto file completely.
- Fixed conformance tests to allow proto2 and look up message by name
  instead of hardcoding a specific list of messages.
- Fixed conformance test to support the "ignore unknown" option for
  JSON.
- Fixed conformance test to properly report serialization errors.

* Removed debug printf and fixed #inspect for floats.

* Fixed compatibility test to have proper semantics for #to_json.

* Updated Makefile.am with new file list.

* Don't try to copy wyhash when inside Docker.

* Fixed bug where we would forget that a sub-object is frozen in Ruby >=2.7.

* Avoid exporting unneeded symbols and refactored a bit of code.

* Some more refactoring.

* Simplified and added more comments.

* Some more comments and simplification. Added a missing license block.

Co-authored-by: Masaki Hara <hara@wantedly.com>
2021-01-13 12:16:25 -08:00
Brent Shaffer
468bc193ec
fix: convert native PHP floats to single precision (#8187)
* fix: convert native PHP floats to single precision

* fix unpack
2021-01-11 11:53:35 -08:00
Joshua Haberman
7165c34606
Merge pull request #8195 from HannahShiSFB/gRPC-Ruby-issue-on-Windows
[Ruby] Fix: (SIGSEGV) gRPC-Ruby issue on Windows. memory alloc infinite recursion/run out of memory
2021-01-08 14:39:41 -08:00
root
68f72f3f6c bug fix. in a 32-bit system, sizeof(mem_block) is only 8, _upb_arena_alignup(sizeof(mem_block)) is 16 2021-01-08 19:01:37 +00:00
Aaron Son
10599e6c8d
js/binary/utils.js: Fix jspb.utils.joinUnsignedDecimalString to work with negative bitsLow and low but non-zero bitsHigh parameter. (#8170) 2021-01-07 17:22:10 -08:00
Joshua Haberman
7089ff0ef2
Fix PhpDoc comments for message accessors to include "|null". (#8136)
Message accessors will return null when when the field is not
set, so this should be reflected in the PhpDoc.

Also updated the code generator for the well-known types to reflect
the edits made in https://github.com/protocolbuffers/protobuf/pull/8105.

Also explicitly check for upb_msg_has() in the oneof accessor, so
we are not implicitly relying on unset message fields returning NULL
at the upb level.
2021-01-07 14:58:38 -08:00
Gilles Vollant
9647a7c235 Fix somes warning when compiling with Visual Studio 2019 on x64 target
In visual studio 2019 x64 target, size_t are 64 bits and int are 32 bits
2020-12-30 14:10:09 -08:00
brucearctor
6c970b5a05 Update options.md - Added 'Astounding' Project 2020-12-28 13:44:56 -08:00
Carlos O'Ryan
54d9f6a1da fix: support deprecated enums for GCC < 6 2020-12-28 12:52:49 -08:00
Abhishek Jain
36aff6a526 Fix for compiler warning issue#8145
Corrected the == check condition.

Signed-off-by: Abhishek Jain <abhishek.jain3@huawei.com>
2020-12-28 11:31:57 -08:00
Abhishek Jain
9505222a24 Fix TextFormatMapTest.DynamicMessage issue#5136
Clean up a multi-line string to conform to Unix line endings.

Signed-off-by: Abhishek Jain <abhishek.jain3@huawei.com>
2020-12-28 11:31:18 -08:00
Rosen Penev
cb8ca280cd remove CMAKE_THREAD_LIBS_INIT from pkgconfig CFLAGS
This is a linker flag and does not belong in CFLAGS.

Fixes an issue with ola and protobuf.

Signed-off-by: Rosen Penev <rosenp@gmail.com>
2020-12-28 10:30:00 -08:00
Adam Cozzette
635156e406 Fix Mac tests by making an unshallow copy of Homebrew repos
It seems that updating shallow Git clones is expensive, and as a result
Homebrew recently started refusing to update shallow clones (see
https://github.com/Homebrew/discussions/discussions/226). This commit
tries to fix the problem by making these repos into full clones before
running "brew update".

I also came across another error about there being a conflicting version
of pip3 in /usr/local. I suspect that is related to the other Python
binaries that the script has to delete, so I added pip3 to the list and
that seemed to solve the problem.
2020-12-23 11:52:19 -08:00
James Newton-King
6cf068c439
Clean up 2020-12-15 22:09:07 +13:00
James Newton-King
e9cc0b0c23
Optimize MapField serialization by removing MessageAdapter 2020-12-15 21:41:39 +13:00
Paul Yang
64dd751868
Update kokoro badge bucket to protobuf-kokoro-badges (#8138) 2020-12-14 10:48:17 -08:00
Jan Tattermusch
e78984465a
Merge pull request #8140 from jtattermusch/no_public_sign
C#: do not use <PublicSign>true</PublicSign> on non-windows
2020-12-14 17:15:27 +01:00
Jan Tattermusch
930ab0f33d C#: do not use <PublicSign>true</PublicSign> on non-windows 2020-12-14 14:41:42 +01:00
Rob Widmer
53be4109c0 Fixed spelling error in JRuby Protobuf class 2020-12-11 16:02:41 -08:00
Rob Widmer
351a34f666 Added some missing jruby files to the Makefile 2020-12-11 16:02:41 -08:00
Rob Widmer
24b2094f8a Set up testing configuration for jruby 2020-12-11 16:02:41 -08:00
Rob Widmer
7b1d6abbe4 Fix some more failing tests 2020-12-11 16:02:41 -08:00
Rob Widmer
8313e577e5 Fix ruby gem to work with JRuby 2020-12-11 16:02:41 -08:00
Thomas BACCELLI
26c0fbc15b Add macOS big sur compatibility 2020-12-07 14:36:56 -08:00
Elliotte Rusty Harold
c17e3c134c
Guava 30 (#7991) 2020-12-04 13:51:39 -08:00
Paul Yang
823f351448
Port for php8 (#8105)
* Port for php8

* Port php c extension for php8

* Update composer.json

* Drop php7.0 support

* Update phpunit for php7.1 in c extension test

* Add back support for php7.0

* Add badge for php8 continuous build
2020-12-04 13:46:34 -08:00
Paul Yang
70531697bc
Update docker file for php 32bit tests (#8116) 2020-12-02 16:44:56 -08:00
Gianni Tedesco
a91adf173b Don't include sys/param.h for _BYTE_ORDER
It includes a bunch of other files including a lot of macros which can
reduce the namespace available for actual protobuf.

For example, create a protobuf with a member called SIGSEGV. Since
macros cannot be namespaced in C++ this results in code which cannot be
compiled.

Fix this by just directly including endian.h
2020-12-02 11:53:57 -08:00
Paul Yang
e8906e4ecd
Update docker file for php7 (#8112) 2020-12-02 10:10:18 -08:00
Nadav Samet
9637a3b57c Allocate extension 1089 to ScalaPB validate
ScalaPB validate is a sub-package of ScalaPB that allows user to generate data validation code for ScalaPB generated classes.
2020-12-01 15:53:51 -08:00
Biswapriyo Nath
c777ade07b Conformance/CMake: Remove non-existence binary_json_conformance_main.cc file. 2020-12-01 10:05:33 -08:00
Biswapriyo Nath
5cc03457e6 stubs/mutex: Fix compilation by initializing variable in WrappedMutex class.
This solves the following error in gcc:
explicitly defaulted function cannot be declared 'constexpr' because the implicit declaration is not 'constexpr'
2020-11-30 15:06:55 -08:00
Mike Lehan
9ee163a5d7
Update param type on offsetGet (#8033)
The param was typed as `object` which is invalid as when items are set in the container the key passes through `checkKey` where it enforces the key must be various classes of integer, bool or string. Thus I have replaced it with a union type (in future this can be a proper PHP level typehint, but for now only doctype)
2020-11-30 11:29:13 -08:00
Paul Yang
9432c99f37
Update docker image for php8 (#8103) 2020-11-30 10:19:50 -08:00
Jorg Brown
3f91c10f0d
Work around -Werror=type-limits under gcc 10.2 (#8092)
* Work around `-Werror=type-limits` under gcc 10.2

This is an error when tag is greater than 128 under gcc 10.2:   `if (tag < 128) return *ptr == tag;`
It's an error even though the comparison occurs in a branch of code we know won't be taken.  See https://godbolt.org/z/1eaP86
This works around the problem by casting `tag` to the same type as `*ptr`.
2020-11-25 17:31:49 -08:00
Antony Dovgal
51b620ac96
unregister INI entries and fix invalid read on shutdown (#8042) 2020-11-25 11:02:35 -08:00
Adam Cozzette
471317f03a
Merge pull request #8077 from acozzette/fix-8059
Updated ax_pthread.m4 to its most recent upstream version
2020-11-25 11:02:02 -08:00
Adam Cozzette
9e2d875dd6
Merge pull request #8089 from acozzette/fixes
Fixed some minor issues
2020-11-25 10:48:56 -08:00
Adam Cozzette
72fddb7272 Fixed the Codespell check 2020-11-24 15:02:25 -08:00
Adam Cozzette
551b440aaa Fixed some minor issues
This commit fixes some minor issues that came up when I tried to
integrate changes from Git into Google's internal codebase:
- Include stubs/strutil.h for access to Split() helper function
- Use emplace() instead of insert() so that the code still works if
  we're working with a string_view instead of a string
2020-11-24 14:57:28 -08:00
Adam Cozzette
977dde9ee3
Merge pull request #7180 from stac47/fix_deprecated_is_pod
Fix std::is_pod deprecated in C++20
2020-11-24 09:15:05 -08:00
Adam Cozzette
e65887ad0e
Merge branch 'master' into fix_deprecated_is_pod 2020-11-23 16:47:29 -08:00
Tongliang Liao
5a7a4a52a7 Install examples in /usr/local/share/protobuf/examples. 2020-11-23 15:58:38 -08:00
Joshua Haberman
bd9a7104e1
Merge pull request #8076 from haberman/sync-stage
Integrate from Piper for C++, Java, and Python
2020-11-23 09:09:53 -08:00
Joshua Haberman
7993c7456c Added note to CHANGES.txt about Arena::SpaceAllocated(). 2020-11-23 09:09:05 -08:00