Commit Graph

52 Commits

Author SHA1 Message Date
Deanna Garcia
a00125024e Sync from Piper @353127564
PROTOBUF_SYNC_PIPER
2021-01-22 00:24:30 +00:00
Joshua Haberman
32e5deb1ac Sync from Piper @308829107
PROTOBUF_SYNC_PIPER
2020-04-28 08:40:38 -07:00
Joshua Haberman
6b759688a1
Implemented proto3 presence for Ruby. (#7406)
* WIP.

* WIP.

* Builds and runs. Tests need to be updated to test presence.

* Ruby: proto3 presence is passing all tests.

* Fixed a bug where empty messages has the wrong oneof count.
2020-04-23 12:54:25 -07:00
Joshua Haberman
aae5c491f7 Merge branch 'master' into ruby-lazy-wrappers 2019-10-29 17:53:35 -07:00
Joshua Haberman
8393d4833f Nearly all known cases (map, repeated field, and top-level) have been addressed.
The only case that doesn't work is decoding a wrapper type from JSON
at the top level.  This doesn't make sense and probably no users do it
I changed it to throw.
2019-10-29 13:30:12 -07:00
Joshua Haberman
bd253f0130 Fixed equality, and extended to repeated fields and maps. 2019-10-28 18:03:25 -07:00
Joshua Haberman
9cfb12bf0a Tests pass for all common operations.
A few things that don't work or aren't tested yet:
- wrappers at the top level
- equality checking for not-yet-expanded wrappers.
2019-10-21 09:19:19 -07:00
Alan Wu
c1ba7c643c Remove unused argument to avoid UB
`OneOfDescriptor_each` is registered as a Ruby method which takes zero
parameters, which means it should take one argument.

When Ruby invokes `OneOfDescriptor_each`, it calls it with one parameter
only, which is one less than what `OneOfDescriptor_each` takes before
this commit. Calling a function with the wrong number of argument is
technically undefined behavior.

See also: §6.5.2.2, N1256
2019-10-11 20:32:35 -04:00
Joshua Haberman
cf07d3c1b2 layout_init() optimization works! 2019-08-21 17:58:37 -07:00
Joshua Haberman
78378dab22 Merge branch 'master' into layout_clear 2019-08-20 17:24:18 -07:00
Joshua Haberman
c02a6fbf2c Bugfix for GC mark of oneof fields. 2019-08-16 09:37:36 -07:00
Joshua Haberman
1c9fb9d45b WIP. 2019-08-16 06:42:13 -07:00
Joshua Haberman
013a0ea882 Optimized layout_mark to not iterate over the msgdef. 2019-08-15 06:39:38 -07:00
Joshua Haberman
0088a75ce5 Merge branch 'master' into layout_mark 2019-08-15 05:36:24 -07:00
Joshua Haberman
0f76f8a83b Put oneof case offset in separate oneof table. 2019-08-15 03:37:06 -07:00
Joshua Haberman
63f324a993 Roll forward Ruby upb changes now that protobuf Ruby build is fixed (#5866)
* Rolled forward again with "Updated upb from defcleanup branch..."

Revert "Revert "Updated upb from defcleanup branch and modified Ruby to use it (#5539)" (#5848)"

This reverts commit 1568deab40.

* A few more merge fixes.

* Updated for defcleanup2 branch.

* Fixed upb to define upb_decode().

* Fixed names of nested messages.

* Revert submodule.

* Set -std=gnu90 and fixed warnings/errors.

Some of our Kokoro tests seem to run with this level of warnings,
and the source strives to be gnu90 compatible.  Enforcing it for
every build removes the possibility of some errors showing up in
Kokoro/Travis tests only.

* Fixed remaining warnings with gnu90 mode.

I tried to match warning flags with what Ruby appears to do
in our Kokoro tests.

* Initialize values registered by rb_gc_register_address().

* Fixed subtle GC bug.

We need to initialize this marked value before creating the instance.

* Truly fix the GC bug.

* Updated upb for mktime() fix.

* Removed XOPEN_SOURCE as we are not using strptime().

* Removed fixed tests from the conformance failure list for Ruby.

* Fixed memory error related to oneof def names.

* Picked up new upb changes re: JSON printing.

* Uncomment concurrent decoding test.
2019-08-14 14:41:37 -07:00
Joshua Haberman
1e37a94bb5 Optimized away the creation of empty string objects.
Prior to this CL, creating an empty message object would create
two empty string objects for every declared field.  First we
created a unique string object for the field's default.  Then
we created yet another string object when we assigned the
default value into the message: we called #encode to ensure
that the string would have the correct encoding and be frozen.

I optimized these unnecessary objects away with two fixes:

1. Memoize the empty string so that we don't create a new empty
   string for every field's default.
2. If we are assigning a string to a message object, avoid creating
   a new string if the assigned string has the correct encoding and
   is already frozen.
2019-08-13 04:54:11 -07:00
Joshua Haberman
d57581348d
Revert "Convert Google::Protobuf.deep_copy to pure Ruby" 2019-05-14 14:25:11 -07:00
Aaron Patterson
3b67455319
Convert Google::Protobuf.deep_copy to pure Ruby
In general, I think it will help us to debug issues if we have less C
code and more Ruby code.  This method can be implemented in pure Ruby,
so this commit converts it to pure Ruby.
2019-05-13 15:37:13 -07:00
Joshua Haberman
1568deab40
Revert "Updated upb from defcleanup branch and modified Ruby to use it (#5539)" (#5848)
This reverts commit 37581380fb.
2019-03-06 15:44:38 -08:00
Joshua Haberman
37581380fb
Updated upb from defcleanup branch and modified Ruby to use it (#5539) 2019-03-06 10:20:18 -08:00
Joe Bolinger
64eb9b6e85 Add more descriptive error messages to init methods in Ruby (#5659)
* add more descriptive error messages to init methods

* add type errors test to Makefile.am
2019-02-06 07:57:13 -08:00
Paul Yang
37a0ab7d83
Fix Any json encoding/decoding for ruby. (#5592)
* Fix Any json encoding/decoding for ruby.

* Revert unnecessary changes
2019-01-17 10:24:41 -08:00
Paul Yang
aa5c12e882
Revert "Revert "Enable the ignore_unknown_field option in the Ruby unmarshal options" (#5511)" (#5533)
* Revert "Revert "Enable the ignore_unknown_field option in the Ruby unmarshal options" (#5511)"

This reverts commit be1716a6d0.

* Separate ruby conformance test on Mac

* Fix shell syntax

* Fix test
2019-01-08 10:53:34 -08:00
Paul Yang
be1716a6d0
Revert "Enable the ignore_unknown_field option in the Ruby unmarshal options" (#5511) 2018-12-26 12:26:24 -08:00
Erik Benoist
9ac11326df Adds the ability to ignore unknown fields on parse
This adds the ability for the MRI Ruby library to optionally pass in a
ignore_unknown_fields option when decoding JSON. The functionality was
added upstream in upb, this change exposes that option.
2018-12-21 16:06:21 -06:00
Harshit Chopra
d0535cc09e Adds support for proto2 syntax for Ruby gem.
This change only adds basic proto2 support without advanced features
like extensions, custom options, maps, etc.

The protoc binary now generates ruby code for proto2 syntax.
However, for now, it is restricted to proto2 files without advanced features
like extensions, in which case it still errors out.

This change also modifies the DSL to add proto messages to the DescriptorPool.
There is a new DSL Builder#add_file to create a new FileDescriptor. With this,
the generated ruby DSL looks something like:

Google::Protobuf::DescriptorPool.generated_pool.build do
  add_file "test.proto" do
    add_message "foo" do
      optional :val, :int32, 1
    end
  end
end
2018-09-27 14:21:16 -04:00
Erik Benoist
74f8e24232 Adds a base class for all explicitly raised TypeErrors (#4255)
* This allows for ruby code to catch and handle Protobuf
    TypeErrors separately from the standard Ruby TypeError

  * Maintains backwards compatibility by having the new
    Google::Protobuf::TypeError inherit from the base
    TypeError. Any code that was catching TypeError should
    continue to work.
2018-06-26 20:24:24 -07:00
Paul Yang
0e7b589566
Add discard unknown API in ruby. (#3990)
* Add discard unknown API in ruby.

* Add test for oneof message field.

* Add TestUnknown to represent unknown field data clearly.

* Only serialize the message with unknown fields itself in test.

* Move discard_unknown from Message to Google.Protobuf
2017-12-07 14:18:38 -08:00
Paul Yang
23adfeb003 Reserve unknown in Ruby (#3763)
* Reserve unknown in ruby

* Revert ruby tests. Wait for cpp impl for conformance test

* Add conformance test for preserving unknown

* Add unknown field conformance test to csharp failure list.

* Fix comments

* Fix comment

* Fix comments

* Fix typo

* Use stringsink_string directly

* Mark hd unused

* Remove unused encodeunknown_handlerfunc
2017-10-26 14:41:43 -07:00
Paul Yang
cd5f49d094 Fix ruby segment fault (#3708)
* Fix ruby segment fault

1) rb_ary_new cannot be called during allocate function. During allocate
fucntion, the containing object hasn't been marked and rb_ary_new may
invoke gc to collect containing object.
2) The global map should be marked before allocating it. Otherwise it
may be garbage collected.

* Add test

* Remove commented code

* Fix grammer error
2017-10-03 17:28:49 -07:00
Aaron Patterson
f55c6ec24f
Storing the frame on the map means we don't need the array 2017-08-30 11:38:09 -07:00
Aaron Patterson
d6152dd51c
Move parse frame array to the Map object
This makes the frame stack per-parser, and per-thread.  Fixes #3250
2017-08-30 11:38:04 -07:00
Josh Haberman
9c6b8cb9bf Ruby: fixed Message#to_h for map fields. 2017-03-14 14:27:16 -07:00
Josh Haberman
d4213d839f Ruby: make sure map parsing frames are GC-rooted. 2016-08-26 09:03:55 -07:00
Josh Haberman
ff7f68ae9f Ruby: encode and freeze strings when the are assigned or decoded. 2016-07-21 11:37:54 -07:00
Josh Haberman
194ad621bb Ruby JSON: always accept both camelCase and original field names.
For JSON encoding we provide a new option to decide at
encode time whether to use camelCase or original proto field
names:

  json = MapMessage.encode_json(m, :preserve_proto_fieldnames => true)
2016-04-14 18:33:17 -07:00
Josh Haberman
78da66698b Changed Ruby to properly camelCase its JSON by default. 2016-02-18 10:50:14 -08:00
Josh Haberman
181c7f2636 Added Ruby to conformance tests.
This involved fixing a few important bugs in the
Ruby implementation -- mostly cases of mixing
upb field types and descriptor types (upb field
types do not distinguish between int/sint/fixed/sfixed
like descriptor types do).

Also added protobuf-specific exceptions so parse
errors can be caught specifically.

Change-Id: Ib49d3db976900b2c6f3455c8b88af52cfb86e036
2015-07-16 12:25:55 -07:00
Chris Fallin
231886f632 Ruby C extension speedup: don't re-intern constant string needlessly.
Also fixed lines with > 80 char length.
2015-05-19 16:19:00 -07:00
Joshua Haberman
202f87f8de Merge pull request #387 from cfallin/ruby-upb-update
Update MRI C Ruby extension to use new version of upb (with upb_env).
2015-05-18 14:07:21 -07:00
Chris Fallin
d326277397 Update MRI C Ruby extension to use new version of upb.
- Alter encode/decode paths to use the `upb_env` (environment)
  abstraction.
- Update upb amalgamation to upstream `93791bfe`.
- Fix a compilation warning (void*->char* cast).
- Modify build flags so that upb doesn't produce warnings -- the Travis
  build logs were pretty cluttered previously.
2015-05-15 11:36:12 -07:00
Chris Fallin
a526605aec Merge pull request #338 from skippy/encode-decode-helpers
ruby: Encode decode cleanup and behavior normalization
2015-05-15 10:52:56 -07:00
Adam Greene
cd7ebbe54f make repeated_field quack like an array 2015-05-14 10:38:11 -07:00
Adam Greene
d1b52a00e0 adding and simplifying encoders/decoders
* make consistent between mri and jruby
* create a #to_h and have it use symbols for keys
* add #to_json and #to_proto helpers on the Google::Protobuf message classes
2015-05-13 10:03:56 -07:00
Chris Fallin
eb33f9d3d6 Updated based on code-review comments. 2015-02-02 13:03:08 -08:00
Chris Fallin
e1b7d38d9a Addressed code-review comments. 2015-01-14 17:14:05 -08:00
Chris Fallin
fcd8889d5b Support oneofs in MRI Ruby C extension. 2015-01-14 14:35:57 -08:00
Chris Fallin
4c92289766 Addressed code-review comments. 2015-01-09 15:37:55 -08:00
Chris Fallin
80276ac021 Addressed code-review comments. 2015-01-06 18:01:32 -08:00