Commit Graph

147 Commits

Author SHA1 Message Date
Joshua Haberman
367e4691d2
Fixed memory bug: properly root repeated/map field when assigning. (#8639)
* Fixed memory bug: properly root repeated/map field when assigning.

Previously the protobuf extension would not properly root
memory from a repeated field or map when assigning to a
message field (see the attached test case).  This could cause
crashes if the repeated field is subsequently accessed.

* Add accidentally-deleted Ruby test.
2021-05-21 23:04:09 -07:00
Joshua Haberman
fecf7e9340
Merge pull request #8356 from lfittl/support-json-name-for-ruby
Ruby: Add support for proto3 json_name in compiler and field definitions
2021-04-20 11:21:35 -07:00
Lukas Fittl
93c703e816 Add test for json_name functionality 2021-04-03 18:42:54 -07:00
Joshua Haberman
f3f8707ba5
Merge pull request #8434 from haberman/ruby-message-eq
Fixed message equality in cases where the message type is different.
2021-04-01 12:16:53 -07:00
Joshua Haberman
d69f482883 Fixed message equality in cases where the message type is different.
If the message types are different, equality comparison must return
false.
2021-03-29 17:37:36 -07:00
Joshua Haberman
bb322c2b39 [Ruby] Fixed bug in string comparison logic. 2021-03-05 08:23:42 -08:00
Joshua Haberman
dfa54577d6 [Ruby] Fixed SEGV when users pass nil messages. 2021-03-02 12:44:16 -08:00
Joshua Haberman
4e3ea74e42 [Ruby] Fix for FieldDescriptor.get(msg).
This fix is similar to the previous bug found in
Message.[]. The fix is the same: we need to handle
arrays and maps properly.

Fixes: https://github.com/protocolbuffers/protobuf/issues/8325
2021-02-22 17:14:46 -08:00
Joshua Haberman
3b3aac95a6 [Ruby] Fix for truncating behavior when converting Float to Duration. 2021-02-19 13:58:19 -08:00
Joshua Haberman
256f1327ea [Ruby] Bugfix for Message.[] for repeated or map fields. 2021-02-19 09:29:06 -08:00
Rob Widmer
48234f5f01 Make the new multi-level ruby tests fit the existing structure better and add them to the makefile 2021-01-14 12:26:04 -08:00
Rob Widmer
1d39f78818 Fix jruby support to handle messages nested more than 1 level deep 2021-01-14 12:26:04 -08:00
Joshua Haberman
9abf6e2ab0
Ported Ruby extension to upb_msg (#8184)
* WIP.

* WIP.

* WIP.

* WIP.

* WIP.

* WIP.

* Added some missing files.

* WIP.

* WIP.

* Updated upb.

* Extension loads, but crashes immediately.

* Gets through the test suite without SEGV!

Still a lot of bugs to fix, but it is a major step!

214 tests, 378 assertions, 37 failures, 147 errors, 0 pendings, 0 omissions, 0 notifications
14.0187% passed

* Test and build for Ruby 3.0

* Fixed a few more bugs, efficient #inspect is almost done.

214 tests, 134243 assertions, 30 failures, 144 errors, 0 pendings, 0 omissions, 0 notifications
18.6916% passed

* Fixed message hash initialization and encode depth checking.

214 tests, 124651 assertions, 53 failures, 70 errors, 0 pendings, 0 omissions, 0 notifications
42.5234% passed

* A bunch of fixes to failing tests, now 70% passing.

214 tests, 202091 assertions, 41 failures, 23 errors, 0 pendings, 0 omissions, 0 notifications
70.0935% passed

* More than 80% of tests are passing now.

214 tests, 322331 assertions, 30 failures, 9 errors, 0 pendings, 0 omissions, 0 notifications
81.7757% passed

Unfortunately there is also a sporadic bug/segfault hanging around
that appears to be GC-related.

* Add linux/ruby30 and macos/ruby30

* Use rvm master for 3.0.0-preview2

* Over 90% of tests are passing!

214 tests, 349898 assertions, 15 failures, 1 errors, 0 pendings, 0 omissions, 0 notifications
92.5234% passed

* Passes all tests!

214 tests, 369388 assertions, 0 failures, 0 errors, 0 pendings, 0 omissions, 0 notifications
100% passed

* A bunch of cleanup.

1. Removed a bunch of internal-only symbols from headers.
2. Required a frozen check to get a non-const pointer to a map or array.
3. De-duplicated the code to get a type argument for Map/RepeatedField.

* Removed a bunch more stuff from protobuf.h.  There is an intermittent assert failure.

Intermittent failure:

ruby: ../../../../ext/google/protobuf_c/protobuf.c:263: ObjectCache_Add: Assertion `rb_funcall(obj_cache2, (__builtin_constant_p("[]") ? __extension__ ({ static ID rb_intern_id_cache; if (!rb_intern_id_cache) rb_intern_id_cache = rb_intern2((("[]")
), (long)strlen(("[]"))); (ID) rb_intern_id_cache; }) : rb_intern("[]")), 1, key_rb) == val' failed

* Removed a few more things from protobuf.h.

* Ruby 3.0.0-preview2 to 3.0.0

* Require rake-compiler-dock >= 1.1.0

* More progress, fighting with the object cache.

* Passes on all Ruby versions!

* Updated and clarified comment regarding WeakMap.

* Fixed the wyhash compile.

* Fixed conformance tests for Ruby.

Conformance results now look like:

RUBYLIB=../ruby/lib:. ./conformance-test-runner --enforce_recommended --failure_list failure_list_ruby.txt --text_format_failure_list text_format_failure_list_ruby.txt ./conformance_ruby.rb

CONFORMANCE TEST BEGIN ====================================

CONFORMANCE SUITE PASSED: 1955 successes, 0 skipped, 58 expected failures, 0 unexpected failures.

CONFORMANCE TEST BEGIN ====================================

CONFORMANCE SUITE PASSED: 0 successes, 111 skipped, 8 expected failures, 0 unexpected failures.

Fixes include:

- Changed Ruby compiler to no longer reject proto2 maps.
- Changed Ruby compiler to emit a warning when proto2 extensions are
  present instead of rejecting the .proto file completely.
- Fixed conformance tests to allow proto2 and look up message by name
  instead of hardcoding a specific list of messages.
- Fixed conformance test to support the "ignore unknown" option for
  JSON.
- Fixed conformance test to properly report serialization errors.

* Removed debug printf and fixed #inspect for floats.

* Fixed compatibility test to have proper semantics for #to_json.

* Updated Makefile.am with new file list.

* Don't try to copy wyhash when inside Docker.

* Fixed bug where we would forget that a sub-object is frozen in Ruby >=2.7.

* Avoid exporting unneeded symbols and refactored a bit of code.

* Some more refactoring.

* Simplified and added more comments.

* Some more comments and simplification. Added a missing license block.

Co-authored-by: Masaki Hara <hara@wantedly.com>
2021-01-13 12:16:25 -08:00
Rob Widmer
7b1d6abbe4 Fix some more failing tests 2020-12-11 16:02:41 -08:00
Rob Widmer
8313e577e5 Fix ruby gem to work with JRuby 2020-12-11 16:02:41 -08:00
Masaki Hara
f494cb222d
Test in Ruby 2.7 (#7386)
* Test in Ruby 2.7

* Turn off GC.stress for 'require' in Ruby 2.7

* Backport 2c8364b to compatibility_tests/v3.0.0
2020-04-24 09:50:27 -07:00
Joshua Haberman
6b759688a1
Implemented proto3 presence for Ruby. (#7406)
* WIP.

* WIP.

* Builds and runs. Tests need to be updated to test presence.

* Ruby: proto3 presence is passing all tests.

* Fixed a bug where empty messages has the wrong oneof count.
2020-04-23 12:54:25 -07:00
David L. Jones
94afb8ab36
Add Ruby tests for oneof cases. (#7385)
In Ruby, the oneof case returns a symbol with the name of the field that is set. These were not previously tested.
2020-04-23 12:49:04 -07:00
Joshua Haberman
18950451c7
Ruby: assigning 'nil' to submessage should clear the field. (#7397)
Previously if you assigned 'nil' to a submessage in proto2
the field would be set to 'nil' but would still have its hasbit
set. This was a clear bug so I'm fixing it outright, even though
it is an observable behavior change.
2020-04-20 15:51:05 -07:00
Eric Walker
2c8364bcd1
Rails2.7 segfaults (#7091)
* Add a test suite for ruby 2.7

* Call BigDecimal() instead of BigDecimal.new()

BigDecimal.new was deprecated in ruby 2.6

* Switch FrozenError expectation to a matcher

The error message for FrozenError changed to include more information
about the mutated object. Switch from an exact match to an aproximate
match (equal => match). This does not change the prefix.

* We can safely ignore newest array methods from ruby 2.7
2020-04-17 10:20:38 -07:00
Joshua Haberman
0d43ba41ee
Update to new upb version (#7372) 2020-04-11 09:46:20 -07:00
Joshua Haberman
c649397029
Set execute bit on files if and only if they begin with (#!). (#7347)
* Set execute bit on files if and only if they begin with (#!).

Git only tracks the 'x' (executable) bit on each file. Prior to this
CL, our files were a random mix of executable and non-executable.
This change imposes some order by making files executable if and only
if they have shebang (#!) lines at the beginning.

We don't have any executable binaries checked into the repo, so
we shouldn't need to worry about that case.

* Added fix_permissions.sh script to set +x iff a file begins with (#!).
2020-04-01 15:28:25 -07:00
Joshua Haberman
f2c5ee5117
Fix for JSON serialization of 0/empty-valued wrapper types (#7198)
* Fixed Ruby JSON serialization of 0/empty wrapper fields.

* Removed newly-passing conformance tests from the failure list.
2020-02-11 13:18:32 -08:00
Joshua Haberman
1a74ba4cb4
Fix for wrappers with a zero value (#7195)
* Add failing tests for issues with wrapped values where the value is the default

* Add test for wrapped values without a value set

* Bugfix for wrapper types with default values.

The previous optimizations for wrapper types had a bug that prevented
wrappers from registering as "present" if the "value" field was not
present on the wire.

In practice the "value" field will not be serialized when it is zero,
according to proto3 semantics, but due to the optimization this
prevented it from creating a new object to represent the presence of the
field.

The fix is to ensure that if the wrapper message is present on the wire,
we always initialize its value to zero.

Co-authored-by: Dan Quan <dan@quan.io>
2020-02-11 08:20:00 -08:00
Joshua Haberman
781d6963c6 Fixed the case of multi-line strings in JSON. 2019-10-31 12:23:25 -07:00
Joshua Haberman
5f25400250 Fixed conformance test regression: empty string wrapper. 2019-10-29 18:16:04 -07:00
Joshua Haberman
e8c67e14ac Fixed the oneof case for lazy wrappers. 2019-10-29 15:44:51 -07:00
Joshua Haberman
8393d4833f Nearly all known cases (map, repeated field, and top-level) have been addressed.
The only case that doesn't work is decoding a wrapper type from JSON
at the top level.  This doesn't make sense and probably no users do it
I changed it to throw.
2019-10-29 13:30:12 -07:00
Joshua Haberman
bd253f0130 Fixed equality, and extended to repeated fields and maps. 2019-10-28 18:03:25 -07:00
Joshua Haberman
9cfb12bf0a Tests pass for all common operations.
A few things that don't work or aren't tested yet:
- wrappers at the top level
- equality checking for not-yet-expanded wrappers.
2019-10-21 09:19:19 -07:00
Joshua Haberman
969d245bd3 WIP: first steps towards lazily creating wrappers. 2019-10-21 07:27:36 -07:00
Joshua Haberman
61b6670a2a Fixed leap year handling by reworking upb_mktime() -> upb_timegm(). (#6695)
The new function name also better reflects the semantics of the
function.  Like timegm(), this function always converts to/from
UTC, not local time.
2019-09-24 17:10:24 -07:00
Joshua Haberman
63f324a993 Roll forward Ruby upb changes now that protobuf Ruby build is fixed (#5866)
* Rolled forward again with "Updated upb from defcleanup branch..."

Revert "Revert "Updated upb from defcleanup branch and modified Ruby to use it (#5539)" (#5848)"

This reverts commit 1568deab40.

* A few more merge fixes.

* Updated for defcleanup2 branch.

* Fixed upb to define upb_decode().

* Fixed names of nested messages.

* Revert submodule.

* Set -std=gnu90 and fixed warnings/errors.

Some of our Kokoro tests seem to run with this level of warnings,
and the source strives to be gnu90 compatible.  Enforcing it for
every build removes the possibility of some errors showing up in
Kokoro/Travis tests only.

* Fixed remaining warnings with gnu90 mode.

I tried to match warning flags with what Ruby appears to do
in our Kokoro tests.

* Initialize values registered by rb_gc_register_address().

* Fixed subtle GC bug.

We need to initialize this marked value before creating the instance.

* Truly fix the GC bug.

* Updated upb for mktime() fix.

* Removed XOPEN_SOURCE as we are not using strptime().

* Removed fixed tests from the conformance failure list for Ruby.

* Fixed memory error related to oneof def names.

* Picked up new upb changes re: JSON printing.

* Uncomment concurrent decoding test.
2019-08-14 14:41:37 -07:00
Joe Bolinger
41e12344a1 Support hashes for struct initializers (#5716)
* support hashes for struct initalizers

* convert hash keys to string

* update tests

* add extra asserts
2019-07-25 11:17:51 -07:00
Bo Yang
bc742640ef Merge remote-tracking branch 'origin/3.8.x' 2019-06-21 04:01:45 +00:00
Paul Yang
640423f157
Change int64 json encoding to be string for php and ruby (#6251)
* Change int64 json encoding to be string for php and ruby

* Fix ruby test

* Sync upb change
2019-06-17 13:14:27 -07:00
Sorah Fukumori
997bd354d5 Fix TypeError on decoding enum map values in Ruby (#6262)
value_field_typeclass should be a enum module, not EnumDescriptor
object.

Also expanding tests for enum valued maps.

Fixes #4580

Signed-off-by: Sorah Fukumori <her@sorah.jp>
2019-06-16 15:20:18 -07:00
Joe Bolinger
e4bbca1fc5 Add wrapper type helpers for Ruby (#5739)
* add wrapper type helpers

* add check for _as_value suffix
2019-03-27 09:44:43 -07:00
Joe Bolinger
d2daa38986 Add native type setters for Timestamp and Duration in Ruby (#5751)
* add implicit time conversion

* add duration

* add init test

* more tests

* add type check and alternative c type check

* add rational and bigdecimal

* use rb_obj_is_kind_of

* use native time check

* chain implicit conversions

* remove unused variable
2019-03-26 16:48:24 -07:00
Joe Bolinger
f4f32a5b68 make tests work on older rubies (#5823) 2019-03-06 18:58:58 -08:00
Joshua Haberman
1568deab40
Revert "Updated upb from defcleanup branch and modified Ruby to use it (#5539)" (#5848)
This reverts commit 37581380fb.
2019-03-06 15:44:38 -08:00
Joshua Haberman
37581380fb
Updated upb from defcleanup branch and modified Ruby to use it (#5539) 2019-03-06 10:20:18 -08:00
Joe Bolinger
ae85b457d2 Fix Ruby module name generation when the ruby_package option is used (again) (#5794)
* Revert "Revert "Fix Ruby module name generation when the ruby_package option is used (#5735)""

This reverts commit bb211e851e.

* add new files to Makefile.am
2019-03-02 10:38:10 -08:00
Joe Bolinger
76685c6fae Add frozen checks in Ruby (#5726)
* add frozen checks

* Use rb_check_frozen

* Correct assertion on frozen error message

The second argument for the method assert_raise is the message
to show when the assertion fails. It does not check the error
object's message.
Add an additional assertion that does check the error's message.

* do frozen check first
2019-03-02 10:37:37 -08:00
Joe Bolinger
a6e3ac0db1 Generate extra enum method helpers for Ruby (#5670)
* example with extra enum method

* update expected test output

* slight simplification

* add test for generated enum helpers

* move const helpers to c extension

* more explicit test

* more explicit test

* indent

* add foo test

* add check for _const suffix
2019-03-02 10:36:28 -08:00
Paul Yang
bb211e851e Revert "Fix Ruby module name generation when the ruby_package option is used (#5735)"
This reverts commit 9638dcc340.
2019-02-27 20:56:03 -08:00
Joe Bolinger
9638dcc340 Fix Ruby module name generation when the ruby_package option is used (#5735)
* fix module name generation and add tests

* fix existing tests

* support both package name styles
2019-02-27 20:26:59 -08:00
Joe Bolinger
7f14ea9f48 Raise error for JSON overflow encoding in Ruby (#5752)
* add check for overflow

* de-nestify

* break long lines
2019-02-27 20:26:14 -08:00
Joe Bolinger
e780681e35 Allow repeated fields in Ruby to accept multiple arguments on push (#5736)
* let repeated fields push arrays

* add varargs push

* better test
2019-02-27 18:05:06 -08:00
Joe Bolinger
bc929a3e82 add eql? method (#5730) 2019-02-24 09:53:24 -08:00