Commit Graph

14 Commits

Author SHA1 Message Date
Joshua Haberman
9d578a3a9c Ported Ruby to ABSL wyhash. 2021-08-02 15:32:01 -07:00
James D
68cb69ea68
Fix source gem compilation (#8471)
* Select appropriate protoc binary for gem build

This should allow gem installation when
`bundle config force_ruby_platform true` has been set.

It has the unfortunate side effect of printing the protoc version during
build, but a quiet cross-platform solution is more complicated.

* Add path to wyhash header for source gem compilation
2021-04-12 10:06:44 -07:00
Joshua Haberman
9abf6e2ab0
Ported Ruby extension to upb_msg (#8184)
* WIP.

* WIP.

* WIP.

* WIP.

* WIP.

* WIP.

* Added some missing files.

* WIP.

* WIP.

* Updated upb.

* Extension loads, but crashes immediately.

* Gets through the test suite without SEGV!

Still a lot of bugs to fix, but it is a major step!

214 tests, 378 assertions, 37 failures, 147 errors, 0 pendings, 0 omissions, 0 notifications
14.0187% passed

* Test and build for Ruby 3.0

* Fixed a few more bugs, efficient #inspect is almost done.

214 tests, 134243 assertions, 30 failures, 144 errors, 0 pendings, 0 omissions, 0 notifications
18.6916% passed

* Fixed message hash initialization and encode depth checking.

214 tests, 124651 assertions, 53 failures, 70 errors, 0 pendings, 0 omissions, 0 notifications
42.5234% passed

* A bunch of fixes to failing tests, now 70% passing.

214 tests, 202091 assertions, 41 failures, 23 errors, 0 pendings, 0 omissions, 0 notifications
70.0935% passed

* More than 80% of tests are passing now.

214 tests, 322331 assertions, 30 failures, 9 errors, 0 pendings, 0 omissions, 0 notifications
81.7757% passed

Unfortunately there is also a sporadic bug/segfault hanging around
that appears to be GC-related.

* Add linux/ruby30 and macos/ruby30

* Use rvm master for 3.0.0-preview2

* Over 90% of tests are passing!

214 tests, 349898 assertions, 15 failures, 1 errors, 0 pendings, 0 omissions, 0 notifications
92.5234% passed

* Passes all tests!

214 tests, 369388 assertions, 0 failures, 0 errors, 0 pendings, 0 omissions, 0 notifications
100% passed

* A bunch of cleanup.

1. Removed a bunch of internal-only symbols from headers.
2. Required a frozen check to get a non-const pointer to a map or array.
3. De-duplicated the code to get a type argument for Map/RepeatedField.

* Removed a bunch more stuff from protobuf.h.  There is an intermittent assert failure.

Intermittent failure:

ruby: ../../../../ext/google/protobuf_c/protobuf.c:263: ObjectCache_Add: Assertion `rb_funcall(obj_cache2, (__builtin_constant_p("[]") ? __extension__ ({ static ID rb_intern_id_cache; if (!rb_intern_id_cache) rb_intern_id_cache = rb_intern2((("[]")
), (long)strlen(("[]"))); (ID) rb_intern_id_cache; }) : rb_intern("[]")), 1, key_rb) == val' failed

* Removed a few more things from protobuf.h.

* Ruby 3.0.0-preview2 to 3.0.0

* Require rake-compiler-dock >= 1.1.0

* More progress, fighting with the object cache.

* Passes on all Ruby versions!

* Updated and clarified comment regarding WeakMap.

* Fixed the wyhash compile.

* Fixed conformance tests for Ruby.

Conformance results now look like:

RUBYLIB=../ruby/lib:. ./conformance-test-runner --enforce_recommended --failure_list failure_list_ruby.txt --text_format_failure_list text_format_failure_list_ruby.txt ./conformance_ruby.rb

CONFORMANCE TEST BEGIN ====================================

CONFORMANCE SUITE PASSED: 1955 successes, 0 skipped, 58 expected failures, 0 unexpected failures.

CONFORMANCE TEST BEGIN ====================================

CONFORMANCE SUITE PASSED: 0 successes, 111 skipped, 8 expected failures, 0 unexpected failures.

Fixes include:

- Changed Ruby compiler to no longer reject proto2 maps.
- Changed Ruby compiler to emit a warning when proto2 extensions are
  present instead of rejecting the .proto file completely.
- Fixed conformance tests to allow proto2 and look up message by name
  instead of hardcoding a specific list of messages.
- Fixed conformance test to support the "ignore unknown" option for
  JSON.
- Fixed conformance test to properly report serialization errors.

* Removed debug printf and fixed #inspect for floats.

* Fixed compatibility test to have proper semantics for #to_json.

* Updated Makefile.am with new file list.

* Don't try to copy wyhash when inside Docker.

* Fixed bug where we would forget that a sub-object is frozen in Ruby >=2.7.

* Avoid exporting unneeded symbols and refactored a bit of code.

* Some more refactoring.

* Simplified and added more comments.

* Some more comments and simplification. Added a missing license block.

Co-authored-by: Masaki Hara <hara@wantedly.com>
2021-01-13 12:16:25 -08:00
Joshua Haberman
c649397029
Set execute bit on files if and only if they begin with (#!). (#7347)
* Set execute bit on files if and only if they begin with (#!).

Git only tracks the 'x' (executable) bit on each file. Prior to this
CL, our files were a random mix of executable and non-executable.
This change imposes some order by making files executable if and only
if they have shebang (#!) lines at the beginning.

We don't have any executable binaries checked into the repo, so
we shouldn't need to worry about that case.

* Added fix_permissions.sh script to set +x iff a file begins with (#!).
2020-04-01 15:28:25 -07:00
Joshua Haberman
63f324a993 Roll forward Ruby upb changes now that protobuf Ruby build is fixed (#5866)
* Rolled forward again with "Updated upb from defcleanup branch..."

Revert "Revert "Updated upb from defcleanup branch and modified Ruby to use it (#5539)" (#5848)"

This reverts commit 1568deab40.

* A few more merge fixes.

* Updated for defcleanup2 branch.

* Fixed upb to define upb_decode().

* Fixed names of nested messages.

* Revert submodule.

* Set -std=gnu90 and fixed warnings/errors.

Some of our Kokoro tests seem to run with this level of warnings,
and the source strives to be gnu90 compatible.  Enforcing it for
every build removes the possibility of some errors showing up in
Kokoro/Travis tests only.

* Fixed remaining warnings with gnu90 mode.

I tried to match warning flags with what Ruby appears to do
in our Kokoro tests.

* Initialize values registered by rb_gc_register_address().

* Fixed subtle GC bug.

We need to initialize this marked value before creating the instance.

* Truly fix the GC bug.

* Updated upb for mktime() fix.

* Removed XOPEN_SOURCE as we are not using strptime().

* Removed fixed tests from the conformance failure list for Ruby.

* Fixed memory error related to oneof def names.

* Picked up new upb changes re: JSON printing.

* Uncomment concurrent decoding test.
2019-08-14 14:41:37 -07:00
Paul Yang
7d9377e7f3
Use gnu for ruby build because strptime is provided by posix (#5660)
* Use gnu for ruby build because strptime is provided by posix

* Move option to extconf.rb

* Remove unused code in Rakefile

* Add config files for kokoro test

* Use gnu 11

* Define _XOPEN_SOURCE

* Add gnu11 option

* Remove XOPEN

* Try base_cc_flags

* Try config_options

* Move time.h to top

* Try -D_XOPEN_SOURCE=700

* Define XOPEN_SOURCE directly

* Try extconf.rb

* Try CFLAGS

* Try ext.cross_config_options

* Make mac ruby release job fail on error

* Try again

* Try define _XOPEN_SOURCE in extconf.rb

* Try again

* Define __USE_XOPEN

* Remove 2.6.0

* No mingw

* Throw error on mingw

* Remove XOPEN_SOURCE in upb

* Add back mingw

* Remove comment
2019-02-01 10:09:34 -08:00
micw523
0038ff49af Remove Debug Statement for Win/MinGW (#5283) 2018-10-23 12:50:57 -07:00
Paul Yang
f7ada1280f
Build ruby gem on kokoro (#4819)
* Install rake compiler

* Add kokoro config to build ruby gem on linux

* Rename from linix to linux

* Fix prepare_build.sh name

* Clean up

* Install bundler

* Install bundler

* Use c99 in order to build gem on mingw-32 on ruby 2.0.0

See https://github.com/rake-compiler/rake-compiler-dock/issues/4

* Move c99 config to extcofig.rb
2018-06-25 11:14:49 -07:00
Adam Cozzette
9fa40314fc Ruby: wrap calls to memcpy so that gem is compatible with pre-2.14 glibc
This commit adds a __wrap_memcpy function and a linker flag to use that
in place of memcpy for our Ruby gem C extension. This allows us to
always use the 2.2.5 version of memcpy, making it possible to use the
gem on distributions with pre-2.14 versions of glibc.

Before this change:
$ objdump -T protobuf_c.so | grep memcpy
0000000000000000      DF *UND*  0000000000000000  GLIBC_2.3.4 __memcpy_chk
0000000000000000      DF *UND*  0000000000000000  GLIBC_2.14  memcpy

After:
$ objdump -T protobuf_c.so | grep memcpy
0000000000000000      DF *UND*  0000000000000000  GLIBC_2.2.5 memcpy
0000000000000000      DF *UND*  0000000000000000  GLIBC_2.3.4 __memcpy_chk
0000000000042450 g    DF .text  0000000000000005  Base __wrap_memcpy

This is based on gRPC's solution to a similar problem:
5098508d2d/src/core/lib/support/wrap_memcpy.c

This fixes issue #2783.
2017-03-09 10:03:22 -08:00
Josh Haberman
e3ce451b60 Fixed compiler warnings and added -std=c99.
upb no longer requires -std=c99 but the Ruby/C
code still uses C99 features.
2015-06-09 11:08:25 -07:00
Josh Haberman
e8ed021ee7 Updated upb to latest version (C89).
Since this version of upb supports C89, all of the
extra compiler flags are no longer required.
2015-06-08 17:56:03 -07:00
Chris Fallin
d326277397 Update MRI C Ruby extension to use new version of upb.
- Alter encode/decode paths to use the `upb_env` (environment)
  abstraction.
- Update upb amalgamation to upstream `93791bfe`.
- Fix a compilation warning (void*->char* cast).
- Modify build flags so that upb doesn't produce warnings -- the Travis
  build logs were pretty cluttered previously.
2015-05-15 11:36:12 -07:00
Chris Fallin
fd1a3ff11d Support for maps in the MRI C Ruby extension.
This adds the Map container and support for parsing and serializing maps
in the protobuf wire format (as defined by the C++ implementation, with
MapEntry submessages in a repeated field). JSON map
serialization/parsing are not yet supported as these will require some
changes to upb as well.
2015-01-06 15:44:09 -08:00
Chris Fallin
91473dcebf Rename protobuf Ruby module to google/protobuf and rework its build
system. The Ruby module build now uses an amalgamated distribution of
upb, and successfully builds a Ruby gem called 'google-protobuf' with
module 'google/protobuf'.
2014-12-12 15:58:26 -08:00