Commit Graph

389 Commits

Author SHA1 Message Date
Joshua Haberman
aae5c491f7 Merge branch 'master' into ruby-lazy-wrappers 2019-10-29 17:53:35 -07:00
Joshua Haberman
e8c67e14ac Fixed the oneof case for lazy wrappers. 2019-10-29 15:44:51 -07:00
Joshua Haberman
8393d4833f Nearly all known cases (map, repeated field, and top-level) have been addressed.
The only case that doesn't work is decoding a wrapper type from JSON
at the top level.  This doesn't make sense and probably no users do it
I changed it to throw.
2019-10-29 13:30:12 -07:00
Joshua Haberman
bd253f0130 Fixed equality, and extended to repeated fields and maps. 2019-10-28 18:03:25 -07:00
Joshua Haberman
9cfb12bf0a Tests pass for all common operations.
A few things that don't work or aren't tested yet:
- wrappers at the top level
- equality checking for not-yet-expanded wrappers.
2019-10-21 09:19:19 -07:00
Joshua Haberman
969d245bd3 WIP: first steps towards lazily creating wrappers. 2019-10-21 07:27:36 -07:00
Joshua Haberman
77646a45e2
Merge pull request #6758 from XrXr/ub
Remove unused argument to avoid UB
2019-10-15 23:41:29 -07:00
Rafi Kamal
bf36d04348
Merge pull request #6757 from haberman/ruby-empty-names-release
Ported names fix for Ruby to the release branch.
2019-10-13 22:24:38 -07:00
Alan Wu
c1ba7c643c Remove unused argument to avoid UB
`OneOfDescriptor_each` is registered as a Ruby method which takes zero
parameters, which means it should take one argument.

When Ruby invokes `OneOfDescriptor_each`, it calls it with one parameter
only, which is one less than what `OneOfDescriptor_each` takes before
this commit. Calling a function with the wrong number of argument is
technically undefined behavior.

See also: §6.5.2.2, N1256
2019-10-11 20:32:35 -04:00
Joshua Haberman
35da84bf64 Ported names fix for Ruby to the release branch. 2019-10-11 14:52:08 -07:00
Rafi Kamal
2ed41359fe Update version to 3.11.0-RC0 2019-10-03 14:24:27 -07:00
Rafi Kamal
97c7c3fc97 Merge 3.10.x to master 2019-10-03 14:03:38 -07:00
Joshua Haberman
61b6670a2a Fixed leap year handling by reworking upb_mktime() -> upb_timegm(). (#6695)
The new function name also better reflects the semantics of the
function.  Like timegm(), this function always converts to/from
UTC, not local time.
2019-09-24 17:10:24 -07:00
Jie Luo
2a4657971d
Update to 3.10.0-rc0 (#6660)
* Cherry pick changes on update_version.py #6643

* Update version to 3.10.0-rc-0

* Do not add rc0 to php packages.xml
2019-09-18 13:30:03 -07:00
Rafi Kamal
5021cd3419 Update protobuf version 2019-09-12 13:50:24 -07:00
Rafi Kamal
4e07b6347d
Merge pull request #6618 from TeBoring/3.10.x-cherrypick
Cherry Pick (#6614)
2019-09-05 09:53:25 -07:00
Rafi Kamal
56bf637221 Update protobuf version 2019-09-05 09:47:52 -07:00
Paul Yang
398d37f95f Sync upb (#6614)
* Sync upb

https://github.com/protocolbuffers/upb/pull/209

* Update upb

https://github.com/protocolbuffers/upb/pull/210
2019-09-05 09:10:01 -07:00
Paul Yang
7f520092d9
Sync upb (#6614)
* Sync upb

https://github.com/protocolbuffers/upb/pull/209

* Update upb

https://github.com/protocolbuffers/upb/pull/210
2019-09-04 14:41:27 -07:00
Joshua Haberman
d2d49bf56a
Merge pull request #6547 from haberman/layout_clear
Optimization for layout_init()
2019-08-28 15:08:30 -07:00
Joshua Haberman
671c2459fc Fixed crash bug and moved initialization into init method. 2019-08-28 13:15:07 -07:00
Paul Yang
659e799fb8
Sync upb (#6577)
* Sync upb

* https://github.com/protocolbuffers/upb/pull/208

* Fix php c extension compiling

Don't use macros defined by upb

* Update ruby conformance failure list
2019-08-27 13:14:57 -07:00
Paul Yang
893a7d31e1
No need to have a separate ruby conformance test on Mac (#6555)
The previous issue of mktime doesn't work on Mac has been fixed.
2019-08-22 16:17:37 -07:00
Joshua Haberman
780b050975 Fix for GC of Ruby map frames. (#6533)
We were creating a map decoding frame when starting the *map*,
but clearing the GC slot when finishing each *map entry*.  This
means that the decoding frame could be collected in the meantime.
2019-08-22 14:36:27 -07:00
Julio Monteiro
8c3a2ce198 Add source code URI to the Ruby gemspec file (#6466)
* Add source code URI to the Ruby gemspec file

As a developer, I would like to easily find the source code origin for this gem when browsing through https://rubygems.org/gems/google-protobuf.
In order to solve this, I've followed instructions at https://guides.rubygems.org/specification-reference/#metadata by adding the source_code_uri metadata key pair.

* Use git tag instead of fixed "master" string

* Substitute the string, not the Gem::Version
2019-08-22 14:32:06 -07:00
Joshua Haberman
3e3407af49 Re-add memset() that seemed redundant but is necessary in case of GC. 2019-08-21 18:39:25 -07:00
Joshua Haberman
b9131f0aab Removed commented-out code. 2019-08-21 18:05:20 -07:00
Joshua Haberman
cf07d3c1b2 layout_init() optimization works! 2019-08-21 17:58:37 -07:00
Joshua Haberman
78378dab22 Merge branch 'master' into layout_clear 2019-08-20 17:24:18 -07:00
Joshua Haberman
63e4a3ecc9
Merge pull request #6541 from haberman/ruby_race_fix
Fix for race in lazy initialization of handlers.
2019-08-20 15:17:14 -07:00
Joshua Haberman
b245551a61 Fix for race in lazy initialization of handlers.
This fixes https://github.com/protocolbuffers/protobuf/issues/6532.
2019-08-20 14:08:31 -07:00
Joshua Haberman
c02a6fbf2c Bugfix for GC mark of oneof fields. 2019-08-16 09:37:36 -07:00
Joshua Haberman
1c9fb9d45b WIP. 2019-08-16 06:42:13 -07:00
Joshua Haberman
013a0ea882 Optimized layout_mark to not iterate over the msgdef. 2019-08-15 06:39:38 -07:00
Joshua Haberman
0088a75ce5 Merge branch 'master' into layout_mark 2019-08-15 05:36:24 -07:00
Joshua Haberman
0f76f8a83b Put oneof case offset in separate oneof table. 2019-08-15 03:37:06 -07:00
Joshua Haberman
63f324a993 Roll forward Ruby upb changes now that protobuf Ruby build is fixed (#5866)
* Rolled forward again with "Updated upb from defcleanup branch..."

Revert "Revert "Updated upb from defcleanup branch and modified Ruby to use it (#5539)" (#5848)"

This reverts commit 1568deab40.

* A few more merge fixes.

* Updated for defcleanup2 branch.

* Fixed upb to define upb_decode().

* Fixed names of nested messages.

* Revert submodule.

* Set -std=gnu90 and fixed warnings/errors.

Some of our Kokoro tests seem to run with this level of warnings,
and the source strives to be gnu90 compatible.  Enforcing it for
every build removes the possibility of some errors showing up in
Kokoro/Travis tests only.

* Fixed remaining warnings with gnu90 mode.

I tried to match warning flags with what Ruby appears to do
in our Kokoro tests.

* Initialize values registered by rb_gc_register_address().

* Fixed subtle GC bug.

We need to initialize this marked value before creating the instance.

* Truly fix the GC bug.

* Updated upb for mktime() fix.

* Removed XOPEN_SOURCE as we are not using strptime().

* Removed fixed tests from the conformance failure list for Ruby.

* Fixed memory error related to oneof def names.

* Picked up new upb changes re: JSON printing.

* Uncomment concurrent decoding test.
2019-08-14 14:41:37 -07:00
Joshua Haberman
1e37a94bb5 Optimized away the creation of empty string objects.
Prior to this CL, creating an empty message object would create
two empty string objects for every declared field.  First we
created a unique string object for the field's default.  Then
we created yet another string object when we assigned the
default value into the message: we called #encode to ensure
that the string would have the correct encoding and be frozen.

I optimized these unnecessary objects away with two fixes:

1. Memoize the empty string so that we don't create a new empty
   string for every field's default.
2. If we are assigning a string to a message object, avoid creating
   a new string if the assigned string has the correct encoding and
   is already frozen.
2019-08-13 04:54:11 -07:00
Jie Luo
43156775be
Merge 3.9.x 201908071359 to master (#6484)
* Add changelog for 3.9.x

* Revert "Make php message class final to avoid mocking (#6277)" (#6324)

This reverts commit 7f84a94366.
This is just temporary. Eventually, we still want to roll forward this
change. Some users are complaining they need more time to clean up their
code.

* Update extract_includes.bat.in

File io_win32.h is not in directory google\protobuf\stubs under directory google\protobuf\io

* Set oneof case in array constructor (#6351)

Forgot to set it previously.

* Update protobuf version (#6366)

* Drop building wheel for python 3.4 (#6406)

https://github.com/matthew-brett/multibuild/pull/240

* Fix binary compatibility in FieldCodec factory methods (#6380) (#6424)

* Fix binary compatibility in FieldCodec factory messages

* Make default value parameter for current factories required

* Route old methods through default value overloads

* Remove ExtensionRegistry.Add(params) overload

* Rename ExtensionRegistry.Add(IEnumerable<Extension>) overload to AddRange

* Edit naming of parameters in Extension classes

* * Fix add API warnings to docs for extension APIs
* Rename internal ExtensionSet.GetValue to TryGetValue

* Disable javadoc error (#6371)

* Disable javadoc error

Actual fixes of the javadoc will be followed up

* Remove duplicated configuration

* Update javadoc plugin version

* Updated Bazel test script to use most recent Bazel version (#6413) (#6433)

I'm not exactly sure why, but this fixes the failing Bazel presubmit
test. Using the most recent version seems like a good idea anyway so
that we can make sure we're compatible with any new Bazel changes.

* [bazel] Add fixes for --incompatible_load_{cc,java,proto}_rules_from_bzl

* No need to update version in generated code (#6471)

generate_descriptor will handle that

* Update protobuf version (#6472)
2019-08-09 13:21:18 -07:00
Joe Bolinger
41e12344a1 Support hashes for struct initializers (#5716)
* support hashes for struct initalizers

* convert hash keys to string

* update tests

* add extra asserts
2019-07-25 11:17:51 -07:00
Joe Bolinger
180d3e3287 fix null terminated string (#6370) 2019-07-22 17:11:58 -07:00
Chris Gaffney
7da7bec441 ruby: Improve performance of Google::Protobuf::Timestamp#to_time (#6360)
This changes to_time to use Ruby's built in Time.at with nanos support
rather than calculating a float and passing it to Time.at. The new
version runs about 3 times faster than the original version and
allocates fewer objects.

Warming up --------------------------------------
    protobuf#to_time    57.296k i/100ms
      faster#to_time   133.229k i/100ms
Calculating -------------------------------------
    protobuf#to_time    635.361k (± 2.1%) i/s -      3.209M in   5.052169s
      faster#to_time      1.873M (± 3.3%) i/s -      9.459M in   5.055169s

Comparison:
      faster#to_time:  1873368.8 i/s
    protobuf#to_time:   635361.4 i/s - 2.95x  slower

Calculating -------------------------------------
    protobuf#to_time   326.000  memsize (   126.000  retained)
                         7.000  objects (     2.000  retained)
                         0.000  strings (     0.000  retained)
      faster#to_time    86.000  memsize (     0.000  retained)
                         1.000  objects (     0.000  retained)
                         0.000  strings (     0.000  retained)

Comparison:
      faster#to_time:         86 allocated
    protobuf#to_time:        326 allocated - 3.79x more
2019-07-10 09:41:11 -07:00
Bo Yang
d581c76063 Merge branch 'master' into 3.9.x
Conflicts:
	java/lite/pom.xml
	java/util/pom.xml
2019-06-21 20:31:00 +00:00
Bo Yang
bc742640ef Merge remote-tracking branch 'origin/3.8.x' 2019-06-21 04:01:45 +00:00
Paul Yang
640423f157
Change int64 json encoding to be string for php and ruby (#6251)
* Change int64 json encoding to be string for php and ruby

* Fix ruby test

* Sync upb change
2019-06-17 13:14:27 -07:00
Hao Nguyen
638720274b Update version to 3.9.0 2019-06-17 09:34:50 -07:00
Sorah Fukumori
997bd354d5 Fix TypeError on decoding enum map values in Ruby (#6262)
value_field_typeclass should be a enum module, not EnumDescriptor
object.

Also expanding tests for enum valued maps.

Fixes #4580

Signed-off-by: Sorah Fukumori <her@sorah.jp>
2019-06-16 15:20:18 -07:00
Joshua Haberman
8041dd4034
Merge pull request #6166 from blowmage/ruby-remove-to_hash
[Ruby] Remove to_hash methods
2019-06-10 16:02:19 -07:00
Mike Moore
565154af78
Remove Ruby to_hash methods 2019-05-22 13:21:51 -06:00
Hao Nguyen
b0d648664d Update protobuf version 2019-05-21 14:07:13 -07:00
Hao Nguyen
2352a30e66 Revert "Update protobuf version"
This reverts commit 6d96493b31.
2019-05-21 13:49:06 -07:00
Hao Nguyen
6d96493b31 Update protobuf version 2019-05-21 13:31:01 -07:00
Joshua Haberman
d57581348d
Revert "Convert Google::Protobuf.deep_copy to pure Ruby" 2019-05-14 14:25:11 -07:00
Aaron Patterson
3b67455319
Convert Google::Protobuf.deep_copy to pure Ruby
In general, I think it will help us to debug issues if we have less C
code and more Ruby code.  This method can be implemented in pure Ruby,
so this commit converts it to pure Ruby.
2019-05-13 15:37:13 -07:00
Adam Cozzette
531505ca13 Updated gemspec to require Ruby 2.3 or higher 2019-05-10 11:36:35 -07:00
Paul Yang
7597f8ad24
Custom mktime to fix issue on mac (#6118)
* Custom mktime to fix issue on mac

* Remove succeeded tests from failure list
2019-05-09 20:51:35 -07:00
Hao Nguyen
37a89f8e1a Update version to 3.8.0 2019-04-23 16:14:52 -07:00
Paul Yang
f425b9f059
Merge pull request #5953 from acozzette/merge-3-7-x
Merge 3.7.x into master
2019-03-28 17:28:22 -07:00
Bo Yang
33dd96c23f Use mktime 2019-03-28 06:38:20 +00:00
Bo Yang
262cc06075 Fix ruby conformance test on mac 2019-03-28 06:35:07 +00:00
Bo Yang
2d9507ec74 Update upb for ruby 2019-03-28 02:17:39 +00:00
Bo Yang
a268b63284 Update upb for ruby 2019-03-27 21:19:01 +00:00
Joe Bolinger
e4bbca1fc5 Add wrapper type helpers for Ruby (#5739)
* add wrapper type helpers

* add check for _as_value suffix
2019-03-27 09:44:43 -07:00
Joe Bolinger
d2daa38986 Add native type setters for Timestamp and Duration in Ruby (#5751)
* add implicit time conversion

* add duration

* add init test

* more tests

* add type check and alternative c type check

* add rational and bigdecimal

* use rb_obj_is_kind_of

* use native time check

* chain implicit conversions

* remove unused variable
2019-03-26 16:48:24 -07:00
Adam Cozzette
8645d893ab Merge branch '3.7.x' into merge-3-7-x 2019-03-26 14:32:20 -07:00
Paul Yang
6973c3a504
Update version number to 3.7.1 (#5947)
* Update version number to 3.7.1

* Update version number in port_def.inc
2019-03-26 09:30:12 -07:00
Adam Cozzette
077079573f Fixed CI failures with Ruby 2.3 on OSX (#5948)
OSX has deprecated OpenSSL, so we need to install it via Homebrew and
build against that version when we build Ruby.
2019-03-25 17:19:48 -07:00
Benjamin Peterson
5939bc3619 Fix "the the". 2019-03-25 13:38:06 -07:00
Paul Yang
ba42cb53d9
Allow bytes field to be longer than 16000 bytes (#5924)
* Allow bytes field to be longer than 16000 bytes

* Remove empty line
2019-03-25 10:51:18 -07:00
Paul Yang
57b6597f46
Replace strptime with custom implementation (#5906)
* Replace strptime with custom implementation

* Fix ruby strptime

* Fix test

* Fix ruby conformance test

* Use mktime

* Remove EmptyFieldMask from failed conformance test list
2019-03-19 22:27:13 -07:00
Joe Bolinger
580826cd97 cherry-pick test fix from #5853 2019-03-14 16:32:01 -07:00
Joe Bolinger
604121aaa4 fix tests for older rubies 2019-03-14 11:42:22 -07:00
Joe Bolinger
0114727cc6 add more descriptive error messages to init methods 2019-03-14 11:37:39 -07:00
Paul Yang
4b145b1c7f
Parse nested listvalue in json for ruby (#5867)
* Add conformance test for nested listvalue

* Fix upb for parsing repeated Value/ListValue

* Add failed repeated ListValue conformance test into php failure list
2019-03-12 10:56:58 -07:00
Paul Yang
a18680890b
Encode empty ListValue (#5857)
* Encode empty ListValue

* Update mac ruby conformance failure list
2019-03-10 17:33:46 -07:00
bmoyles0117
0b5111dbab Raise error for JSON overflow encoding in Ruby (#5752) (#5861)
* add check for overflow

* de-nestify

* break long lines
2019-03-08 12:50:33 -08:00
Paul Yang
e75a689f7c
Fix ruby continuous tests (#5853)
* Fix ruby continuous tests

* Fix compatibility test
2019-03-07 17:16:31 -08:00
Joe Bolinger
f4f32a5b68 make tests work on older rubies (#5823) 2019-03-06 18:58:58 -08:00
Joshua Haberman
1568deab40
Revert "Updated upb from defcleanup branch and modified Ruby to use it (#5539)" (#5848)
This reverts commit 37581380fb.
2019-03-06 15:44:38 -08:00
Joshua Haberman
37581380fb
Updated upb from defcleanup branch and modified Ruby to use it (#5539) 2019-03-06 10:20:18 -08:00
Joe Bolinger
8bdecd7430 remove return 0 (#5840) 2019-03-05 23:01:49 -08:00
Bo Yang
cb6fa92ee8 Merge branch '3.7.x' 2019-03-04 10:18:18 -08:00
Joe Bolinger
ae85b457d2 Fix Ruby module name generation when the ruby_package option is used (again) (#5794)
* Revert "Revert "Fix Ruby module name generation when the ruby_package option is used (#5735)""

This reverts commit bb211e851e.

* add new files to Makefile.am
2019-03-02 10:38:10 -08:00
Joe Bolinger
76685c6fae Add frozen checks in Ruby (#5726)
* add frozen checks

* Use rb_check_frozen

* Correct assertion on frozen error message

The second argument for the method assert_raise is the message
to show when the assertion fails. It does not check the error
object's message.
Add an additional assertion that does check the error's message.

* do frozen check first
2019-03-02 10:37:37 -08:00
Joe Bolinger
a6e3ac0db1 Generate extra enum method helpers for Ruby (#5670)
* example with extra enum method

* update expected test output

* slight simplification

* add test for generated enum helpers

* move const helpers to c extension

* more explicit test

* more explicit test

* indent

* add foo test

* add check for _const suffix
2019-03-02 10:36:28 -08:00
Paul Yang
582743bf40
Update version number to 3.7.0 (#5793)
* Update version number to 3.7.0

* Add php/releash.sh to Makefile.am

* Make cpp_distcheck exit on fail
2019-02-28 12:55:14 -08:00
Paul Yang
bb211e851e Revert "Fix Ruby module name generation when the ruby_package option is used (#5735)"
This reverts commit 9638dcc340.
2019-02-27 20:56:03 -08:00
Joe Bolinger
9638dcc340 Fix Ruby module name generation when the ruby_package option is used (#5735)
* fix module name generation and add tests

* fix existing tests

* support both package name styles
2019-02-27 20:26:59 -08:00
Joe Bolinger
7f14ea9f48 Raise error for JSON overflow encoding in Ruby (#5752)
* add check for overflow

* de-nestify

* break long lines
2019-02-27 20:26:14 -08:00
Joe Bolinger
e780681e35 Allow repeated fields in Ruby to accept multiple arguments on push (#5736)
* let repeated fields push arrays

* add varargs push

* better test
2019-02-27 18:05:06 -08:00
Joe Bolinger
bc929a3e82 add eql? method (#5730) 2019-02-24 09:53:24 -08:00
Bo Yang
b52754b719 Update version to 3.7.0rc3 2019-02-22 11:31:48 -08:00
Xiang Dai
e479410564 delete all duplicate empty blanks (#5758)
Signed-off-by: Xiang Dai <764524258@qq.com>
2019-02-20 19:28:50 -08:00
Paul Yang
c7b3392201
Update version to 3.7.0 (#5749)
* Update version to 3.7.0

* Update version number for java bom
2019-02-19 15:54:45 -08:00
Joe Bolinger
39c0947893 implement to_s for message types (#5710) 2019-02-12 09:50:57 -08:00
Joe Bolinger
64eb9b6e85 Add more descriptive error messages to init methods in Ruby (#5659)
* add more descriptive error messages to init methods

* add type errors test to Makefile.am
2019-02-06 07:57:13 -08:00
Paul Yang
7d9377e7f3
Use gnu for ruby build because strptime is provided by posix (#5660)
* Use gnu for ruby build because strptime is provided by posix

* Move option to extconf.rb

* Remove unused code in Rakefile

* Add config files for kokoro test

* Use gnu 11

* Define _XOPEN_SOURCE

* Add gnu11 option

* Remove XOPEN

* Try base_cc_flags

* Try config_options

* Move time.h to top

* Try -D_XOPEN_SOURCE=700

* Define XOPEN_SOURCE directly

* Try extconf.rb

* Try CFLAGS

* Try ext.cross_config_options

* Make mac ruby release job fail on error

* Try again

* Try define _XOPEN_SOURCE in extconf.rb

* Try again

* Define __USE_XOPEN

* Remove 2.6.0

* No mingw

* Throw error on mingw

* Remove XOPEN_SOURCE in upb

* Add back mingw

* Remove comment
2019-02-01 10:09:34 -08:00
Adam Cozzette
1ee15baefc Updated version to 3.7.0rc2 2019-01-30 21:08:04 -08:00
Hao Nguyen
d5be2baa46 Update version to 3.7.0 2019-01-24 10:23:02 -08:00
Paul Yang
37a0ab7d83
Fix Any json encoding/decoding for ruby. (#5592)
* Fix Any json encoding/decoding for ruby.

* Revert unnecessary changes
2019-01-17 10:24:41 -08:00