Commit Graph

400 Commits

Author SHA1 Message Date
Adam Cozzette
052dc799d2 Update protobuf version 2021-02-19 15:12:24 -08:00
Joshua Haberman
3b3aac95a6 [Ruby] Fix for truncating behavior when converting Float to Duration. 2021-02-19 13:58:19 -08:00
Joshua Haberman
256f1327ea [Ruby] Bugfix for Message.[] for repeated or map fields. 2021-02-19 09:29:06 -08:00
Adam Cozzette
ae50d9b990 Update protobuf version 2021-02-18 11:50:15 -08:00
Adam Cozzette
8260126500 Update protobuf version 2021-02-17 10:51:33 -08:00
Adam Cozzette
e9f2ef308c Update protobuf version 2021-02-05 15:37:10 -08:00
Joshua Haberman
cf28f28347
Updated upb to allow nonzero offset minutes in JSON timestamps. (#8258)
* Updated upb to allow nonzero offset minutes in JSON timestamps.

This is to pick up https://github.com/protocolbuffers/upb/pull/367

* Reverted error message change.

* Fixed quote chars.
2021-02-03 17:59:10 -08:00
Rob Widmer
48234f5f01 Make the new multi-level ruby tests fit the existing structure better and add them to the makefile 2021-01-14 12:26:04 -08:00
Rob Widmer
1d39f78818 Fix jruby support to handle messages nested more than 1 level deep 2021-01-14 12:26:04 -08:00
Joshua Haberman
9abf6e2ab0
Ported Ruby extension to upb_msg (#8184)
* WIP.

* WIP.

* WIP.

* WIP.

* WIP.

* WIP.

* Added some missing files.

* WIP.

* WIP.

* Updated upb.

* Extension loads, but crashes immediately.

* Gets through the test suite without SEGV!

Still a lot of bugs to fix, but it is a major step!

214 tests, 378 assertions, 37 failures, 147 errors, 0 pendings, 0 omissions, 0 notifications
14.0187% passed

* Test and build for Ruby 3.0

* Fixed a few more bugs, efficient #inspect is almost done.

214 tests, 134243 assertions, 30 failures, 144 errors, 0 pendings, 0 omissions, 0 notifications
18.6916% passed

* Fixed message hash initialization and encode depth checking.

214 tests, 124651 assertions, 53 failures, 70 errors, 0 pendings, 0 omissions, 0 notifications
42.5234% passed

* A bunch of fixes to failing tests, now 70% passing.

214 tests, 202091 assertions, 41 failures, 23 errors, 0 pendings, 0 omissions, 0 notifications
70.0935% passed

* More than 80% of tests are passing now.

214 tests, 322331 assertions, 30 failures, 9 errors, 0 pendings, 0 omissions, 0 notifications
81.7757% passed

Unfortunately there is also a sporadic bug/segfault hanging around
that appears to be GC-related.

* Add linux/ruby30 and macos/ruby30

* Use rvm master for 3.0.0-preview2

* Over 90% of tests are passing!

214 tests, 349898 assertions, 15 failures, 1 errors, 0 pendings, 0 omissions, 0 notifications
92.5234% passed

* Passes all tests!

214 tests, 369388 assertions, 0 failures, 0 errors, 0 pendings, 0 omissions, 0 notifications
100% passed

* A bunch of cleanup.

1. Removed a bunch of internal-only symbols from headers.
2. Required a frozen check to get a non-const pointer to a map or array.
3. De-duplicated the code to get a type argument for Map/RepeatedField.

* Removed a bunch more stuff from protobuf.h.  There is an intermittent assert failure.

Intermittent failure:

ruby: ../../../../ext/google/protobuf_c/protobuf.c:263: ObjectCache_Add: Assertion `rb_funcall(obj_cache2, (__builtin_constant_p("[]") ? __extension__ ({ static ID rb_intern_id_cache; if (!rb_intern_id_cache) rb_intern_id_cache = rb_intern2((("[]")
), (long)strlen(("[]"))); (ID) rb_intern_id_cache; }) : rb_intern("[]")), 1, key_rb) == val' failed

* Removed a few more things from protobuf.h.

* Ruby 3.0.0-preview2 to 3.0.0

* Require rake-compiler-dock >= 1.1.0

* More progress, fighting with the object cache.

* Passes on all Ruby versions!

* Updated and clarified comment regarding WeakMap.

* Fixed the wyhash compile.

* Fixed conformance tests for Ruby.

Conformance results now look like:

RUBYLIB=../ruby/lib:. ./conformance-test-runner --enforce_recommended --failure_list failure_list_ruby.txt --text_format_failure_list text_format_failure_list_ruby.txt ./conformance_ruby.rb

CONFORMANCE TEST BEGIN ====================================

CONFORMANCE SUITE PASSED: 1955 successes, 0 skipped, 58 expected failures, 0 unexpected failures.

CONFORMANCE TEST BEGIN ====================================

CONFORMANCE SUITE PASSED: 0 successes, 111 skipped, 8 expected failures, 0 unexpected failures.

Fixes include:

- Changed Ruby compiler to no longer reject proto2 maps.
- Changed Ruby compiler to emit a warning when proto2 extensions are
  present instead of rejecting the .proto file completely.
- Fixed conformance tests to allow proto2 and look up message by name
  instead of hardcoding a specific list of messages.
- Fixed conformance test to support the "ignore unknown" option for
  JSON.
- Fixed conformance test to properly report serialization errors.

* Removed debug printf and fixed #inspect for floats.

* Fixed compatibility test to have proper semantics for #to_json.

* Updated Makefile.am with new file list.

* Don't try to copy wyhash when inside Docker.

* Fixed bug where we would forget that a sub-object is frozen in Ruby >=2.7.

* Avoid exporting unneeded symbols and refactored a bit of code.

* Some more refactoring.

* Simplified and added more comments.

* Some more comments and simplification. Added a missing license block.

Co-authored-by: Masaki Hara <hara@wantedly.com>
2021-01-13 12:16:25 -08:00
root
68f72f3f6c bug fix. in a 32-bit system, sizeof(mem_block) is only 8, _upb_arena_alignup(sizeof(mem_block)) is 16 2021-01-08 19:01:37 +00:00
Rob Widmer
53be4109c0 Fixed spelling error in JRuby Protobuf class 2020-12-11 16:02:41 -08:00
Rob Widmer
24b2094f8a Set up testing configuration for jruby 2020-12-11 16:02:41 -08:00
Rob Widmer
7b1d6abbe4 Fix some more failing tests 2020-12-11 16:02:41 -08:00
Rob Widmer
8313e577e5 Fix ruby gem to work with JRuby 2020-12-11 16:02:41 -08:00
Adam Cozzette
f3192d6128 Update protobuf version 2020-11-13 11:27:32 -08:00
Adam Cozzette
1b851b3fe6 Update protobuf version 2020-11-12 12:44:26 -08:00
Adam Cozzette
a8ab1fade9 Update protobuf version 2020-11-10 17:15:32 -08:00
Adam Cozzette
27100b40fd Update protobuf version 2020-11-05 14:30:53 -08:00
Adam Cozzette
58f4260e8e Merged 3.13.x into master 2020-08-17 11:06:11 -07:00
Adam Cozzette
fde7cf7358 Update protobuf version 2020-08-14 15:20:53 -07:00
Adam Cozzette
98c2e69f8c Update protobuf version 2020-08-12 14:49:20 -07:00
Peter Newman
e2cc2de304
Fix lots of spelling errors (#7751)
* Fix a typo

* Fix lots of spelling errors

* Fix a few more spelling mistakes

* s/parsable/parseable/

* Don't touch the third party files

* Cloneable is the preferred C# term

* Copyable is the preferred C++ term

* Revert "s/parsable/parseable/"

This reverts commit 534ecf7675.

* Revert unparseable->unparsable corrections
2020-08-10 11:08:25 -07:00
Adam Cozzette
1c587a8059 Updated version to 3.13.0-rc2 2020-08-07 14:44:49 -07:00
Adam Cozzette
8940150706 Updated version to 3.13.0-rc1. 2020-08-07 14:44:49 -07:00
Joshua Haberman
9ce8c330e7
Updated version to 3.12.3 and updated CHANGES.txt. (#7580)
* Updated version to 3.12.3 and updated CHANGES.txt.

* Re-ran generate_descriptor_protos.sh and made it more parallel.
2020-06-01 13:36:50 -07:00
Joshua Haberman
beeb621f50
Update protobuf version (#7557) 2020-05-26 15:46:39 -07:00
Joshua Haberman
6ad138d85a
Update protobuf version (#7535) 2020-05-20 10:54:29 -07:00
Daniel Azuma
cf81f38b5d
Restore binary builds of Ruby 2.3 and 2.4 (#7529) (#7531) 2020-05-19 15:52:59 -07:00
Joshua Haberman
e90b71cdee Update protobuf version 2020-05-15 13:28:07 -07:00
Joshua Haberman
f597a24627 Update protobuf version 2020-05-12 12:49:46 -07:00
Joshua Haberman
e19bfc0e28
Stop building binary gems for ruby <2.5. (#7453)
It is no longer possible to "bundle install" these versions:

$ bundle install
Fetching gem metadata from https://rubygems.org/.................
reline-0.1.3 requires ruby version >= 2.5, which is incompatible with the current version, ruby 2.4.9p362
2020-05-01 12:35:09 -07:00
Joshua Haberman
a9f11d7d64
Updated version to 3.12.0-rc1. (#7449) 2020-04-30 15:34:10 -07:00
Masaki Hara
f494cb222d
Test in Ruby 2.7 (#7386)
* Test in Ruby 2.7

* Turn off GC.stress for 'require' in Ruby 2.7

* Backport 2c8364b to compatibility_tests/v3.0.0
2020-04-24 09:50:27 -07:00
Joshua Haberman
6b759688a1
Implemented proto3 presence for Ruby. (#7406)
* WIP.

* WIP.

* Builds and runs. Tests need to be updated to test presence.

* Ruby: proto3 presence is passing all tests.

* Fixed a bug where empty messages has the wrong oneof count.
2020-04-23 12:54:25 -07:00
David L. Jones
94afb8ab36
Add Ruby tests for oneof cases. (#7385)
In Ruby, the oneof case returns a symbol with the name of the field that is set. These were not previously tested.
2020-04-23 12:49:04 -07:00
Joshua Haberman
18950451c7
Ruby: assigning 'nil' to submessage should clear the field. (#7397)
Previously if you assigned 'nil' to a submessage in proto2
the field would be set to 'nil' but would still have its hasbit
set. This was a clear bug so I'm fixing it outright, even though
it is an observable behavior change.
2020-04-20 15:51:05 -07:00
Eric Walker
2c8364bcd1
Rails2.7 segfaults (#7091)
* Add a test suite for ruby 2.7

* Call BigDecimal() instead of BigDecimal.new()

BigDecimal.new was deprecated in ruby 2.6

* Switch FrozenError expectation to a matcher

The error message for FrozenError changed to include more information
about the mutated object. Switch from an exact match to an aproximate
match (equal => match). This does not change the prefix.

* We can safely ignore newest array methods from ruby 2.7
2020-04-17 10:20:38 -07:00
Masaki Hara
64f6c59ccb
Build extensions for Ruby 2.7 (#7027)
* Build extensions for Ruby 2.7

* Try installing bundler 2.x

* Try bumping rake-compiler-dock

* Use standard RCD images

* Avoid 'rake cross native' with rake-compiler-dock

* Use Ruby 2.5 for building Ruby <= 2.6

* Use rake-compiler 1.1.0

* Specify target

* Don't update Ruby test image for now
2020-04-16 23:31:47 -07:00
Joshua Haberman
0d43ba41ee
Update to new upb version (#7372) 2020-04-11 09:46:20 -07:00
Penelope Phippen
c558aa75a3
Call "Class#new" over rb_class_new_instance in decoding (#7352)
This patch has almost no change in behaviour where users have not
patched the implementation of new on either a specific proto object
class, or `Google::Protobuf::MessageExts::ClassMethods`. The default
implementation of `new`, and `rb_class_new_instance` have the same
behaviour.

By default when we call `new` on a class in Ruby, it goes to the `Class`
class's implementation:

```ruby
class Foo
end

>> Foo.method(:new).owner
=> Class
```

the `Class` implementation of `new` is (pseudocode, it's actually in c):

```ruby
class Class
  def new(*args, &blk)
    instance = alloc
    instance.initialize(*args, &blk)
    instance
  end
end
```

`rb_class_new_instance` does the same thing, it calls down to
[`rb_class_s_new`](https://github.com/ruby/ruby/blob/v2_5_5/object.c#L2147),
which calls `rb_class_alloc`, then `rb_obj_call_init`.

`rb_funcall` is a variadic c function for calling a ruby method on an object,
it takes:

* A `VALUE` on to which the method should be called
* An `ID` which should be an ID of a method, usually created with `rb_intern`,
  to get an ID from a string
* An integer, the number of arguments calling the  method with,
* A number of `VALUE`s, to send to the method call.

`rb_funcall` is the same as calling a method directly in Ruby, and will perform
ancestor chain respecting method lookup.

This means that in C extensions, if nobody has defined the `new` method on any
classes or modules in a class's inheritance chain calling
`rb_class_new_instance` is the same as calling `rb_funcall(klass,
rb_intern("new"))`, *however* this removes the ability for users to define or
monkey patch their own constructors in to the objects created by the C
extension.

In Ads, we define [`new`](https://git.io/JvFC9) on
`Google::Protobuf::MessageExts::ClassMethods` to allow us to insert a
monkeypatch which makes it possible to assign primitive values to wrapper type
fields (e.g. Google::Protobuf::StringValue). The monkeypatch we apply works for
objects that we create for the user via the `new` method. Before this commit,
however, the patch does not work for the `decode` method, for the reasons
outlined above. Before this commit, protobuf uses `rb_class_new_instance`.

After this commit, we use `rb_funcall(klass, rb_intern("new"), 0);` to construct
protobuf objects during decoding. While I haven't measured it this will have
a very minor performance impact for decoding (`rb_funcall` will have to go to the
method cache, which `rb_class_new_instance` will not). This does however do
the "more rubyish" thing of respecting the protobuf object's inheritance chain
to construct them during decoding.

I have run both Ads and Cloud's test suites for Ruby libraries against this
patch, as well as the protobuf Ruby gem's test suite locally.
2020-04-06 08:33:50 -07:00
Joshua Haberman
c649397029
Set execute bit on files if and only if they begin with (#!). (#7347)
* Set execute bit on files if and only if they begin with (#!).

Git only tracks the 'x' (executable) bit on each file. Prior to this
CL, our files were a random mix of executable and non-executable.
This change imposes some order by making files executable if and only
if they have shebang (#!) lines at the beginning.

We don't have any executable binaries checked into the repo, so
we shouldn't need to worry about that case.

* Added fix_permissions.sh script to set +x iff a file begins with (#!).
2020-04-01 15:28:25 -07:00
Rafi Kamal
0eb476b696 Merge branch 3.11.x to master 2020-02-18 16:42:47 -08:00
Rafi Kamal
c74057267d
Update protobuf version (#7206) 2020-02-12 14:41:16 -08:00
Rafi Kamal
906e9c6897
Fix for JSON serialization of 0/empty-valued wrapper types (#7198) (#7204)
* Fixed Ruby JSON serialization of 0/empty wrapper fields.

* Removed newly-passing conformance tests from the failure list.

Co-authored-by: Joshua Haberman <jhaberman@gmail.com>
2020-02-12 13:27:36 -08:00
Rafi Kamal
37fc4327ef
Fix for wrappers with a zero value (#7195) (#7201)
* Add failing tests for issues with wrapped values where the value is the default

* Add test for wrapped values without a value set

* Bugfix for wrapper types with default values.

The previous optimizations for wrapper types had a bug that prevented
wrappers from registering as "present" if the "value" field was not
present on the wire.

In practice the "value" field will not be serialized when it is zero,
according to proto3 semantics, but due to the optimization this
prevented it from creating a new object to represent the presence of the
field.

The fix is to ensure that if the wrapper message is present on the wire,
we always initialize its value to zero.

Co-authored-by: Joshua Haberman <jhaberman@gmail.com>
Co-authored-by: Dan Quan <dan@quan.io>
2020-02-12 12:37:19 -08:00
Joshua Haberman
f2c5ee5117
Fix for JSON serialization of 0/empty-valued wrapper types (#7198)
* Fixed Ruby JSON serialization of 0/empty wrapper fields.

* Removed newly-passing conformance tests from the failure list.
2020-02-11 13:18:32 -08:00
Joshua Haberman
1a74ba4cb4
Fix for wrappers with a zero value (#7195)
* Add failing tests for issues with wrapped values where the value is the default

* Add test for wrapped values without a value set

* Bugfix for wrapper types with default values.

The previous optimizations for wrapper types had a bug that prevented
wrappers from registering as "present" if the "value" field was not
present on the wire.

In practice the "value" field will not be serialized when it is zero,
according to proto3 semantics, but due to the optimization this
prevented it from creating a new object to represent the presence of the
field.

The fix is to ensure that if the wrapper message is present on the wire,
we always initialize its value to zero.

Co-authored-by: Dan Quan <dan@quan.io>
2020-02-11 08:20:00 -08:00
Rafi Kamal
345df38dd3
Update protobuf version (#7143) 2020-01-31 13:47:09 -08:00
Paul Yang
e8016753e3
Maven requires https connection (#7110) (#7114) 2020-01-22 12:29:56 -08:00