Commit Graph

8248 Commits

Author SHA1 Message Date
Deanna Garcia
c26e452e9e Adding PHP fix to changes for 3.17.2 2021-06-02 16:22:31 +00:00
Joshua Haberman
b42f2376e4
Merge pull request #8670 from haberman/php-wkt-submsg
Fixed SEGV in sub-message getters for well-known types when message is unset.
2021-05-27 15:36:36 -07:00
Joshua Haberman
75de6aa21a Fixed sub-message getters for well-known types when message is unset.
The well-known types generate C code into wkt.inc, and this C code was
not testing isset($msg->submsg_field) like the generated code does:

```php
// PHP generated getter: checks isset().
public function getOptions()
{
    return isset($this->options) ? $this->options : null;
}
```

```c
// C generated getter, does not check upb_msg_has()
static PHP_METHOD(google_protobuf_Value, getListValue) {
  Message* intern = (Message*)Z_OBJ_P(getThis());
  const upb_fielddef *f = upb_msgdef_ntofz(intern->desc->msgdef,
                                           "list_value");
  zval ret;
  Message_get(intern, f, &ret);
  RETURN_COPY_VALUE(&ret);
}
```

This led to an error where we wnuld try to get a sub-message field from upb
when it `upb_msg_has(msg, field) == false`, which is an error according to upb.

There are two possible fixes for this bug. A guiding principle is that we want
the generated C code in wkt.inc to have the same behavior as PHP generated
code. Following this principle, the two possible fixes are:

1. Change the code generator for wkt.inc to check upb_msg_has(f) before
   calling Message_get(). This would match the isset() check that the
   The PHP generated code does, and we would leave the PHP code unchanged.

2. Change Message_get() to itself perform the upb_msg_has(f) check for
   sub-message fields. This means that generated code would no longer need
   to perform an isset() check, so we would want to remove this check from
   the PHP generated code also to avoid a redundant check.

Both of these are reasonable fixes, and it is not immediately obvious which is
better. (1) has the benefit of resolving this case when we are in more
specialized code (a getter function that already knows this is a sub-message
field), and therefore avoids performing the check later in more generic code
that would have to test the type again. On the other hand, the isset() check is
not needed for the pure PHP implementation, as an unset PHP variable will
return `null` anyway. And for the C extension, we'd rather check upb_msg_has()
at the C level instead of PHP.

So this change implements (2). The generated code in wkt.inc remains unchanged,
and the PHP generated code for sub-message fields is changed to remove the
isset() check.
2021-05-27 10:45:18 -07:00
deannagarcia
769826e338
Merge pull request #8658 from deannagarcia/update-change-notes
Updating change logs for 3.17.2
2021-05-25 14:08:49 -07:00
Deanna Garcia
a693d723d4 Add PR number for kotlin fix 2021-05-25 21:07:28 +00:00
Deanna Garcia
d303bba1bb Updating change logs 2021-05-25 21:02:52 +00:00
deannagarcia
d3b71f21bb
Merge pull request #8657 from deannagarcia/3.17.x-202105251932
Update version to 3.17.2
2021-05-25 13:47:49 -07:00
Deanna Garcia
5b8208ecea Update protobuf version 2021-05-25 19:36:05 +00:00
deannagarcia
b87d364831
Merge pull request #8653 from deannagarcia/fixDuplicateClassErrors
Remove maven-bundle-plugin to fix duplicate class errors
2021-05-25 10:52:58 -07:00
Deanna Garcia
5fac12fff0 Reverting the Makefile spacing changes 2021-05-25 17:38:17 +00:00
Deanna Garcia
3f8f6e925d Removing more files that were added during rebasing 2021-05-25 17:35:03 +00:00
Deanna Garcia
713f2d0008 Removing files that were added during rebasing 2021-05-25 17:33:14 +00:00
Deanna Garcia
021329b7e7 resolving conflicts 2021-05-25 17:27:07 +00:00
Deanna Garcia
2f8d310048 Fixing objc tests 2021-05-25 17:25:04 +00:00
Deanna Garcia
eb57ffe807 Reverting last change since tests are still failing 2021-05-25 17:25:03 +00:00
Deanna Garcia
6319a23bff Run update_file_lists.sh 2021-05-25 17:25:03 +00:00
Deanna Garcia
f44755496f Updating makefile to pass the tests 2021-05-25 17:24:15 +00:00
Deanna Garcia
76bd17ca5b Add another file to Makefile 2021-05-25 17:24:15 +00:00
Deanna Garcia
f9a60fc8ae Add another header to Makefile 2021-05-25 17:24:15 +00:00
Deanna Garcia
eb3ab8c59e Add headerfiles to Makefile 2021-05-25 17:24:15 +00:00
Joshua Haberman
e635605003
Added missing release notes for 3.17.1. (#8649) 2021-05-24 11:15:46 -07:00
Joshua Haberman
367e4691d2
Fixed memory bug: properly root repeated/map field when assigning. (#8639)
* Fixed memory bug: properly root repeated/map field when assigning.

Previously the protobuf extension would not properly root
memory from a repeated field or map when assigning to a
message field (see the attached test case).  This could cause
crashes if the repeated field is subsequently accessed.

* Add accidentally-deleted Ruby test.
2021-05-21 23:04:09 -07:00
Adam Cozzette
52784ced2f
Update protobuf version (#8633) 2021-05-19 16:43:44 -07:00
Joshua Haberman
0b87475592
Php & Ruby Cherry Picks for 3.17.1 (#8632)
* Some more updates to PHP testing infrastructure (#8576)

* WIP.

* Added build config for all of the tests.

* Use ../src/protoc if it is available, for cases where Bazel isn't available.

* Added test_php.sh.

* Fix for the broken macOS tests.

* Move all jobs to use php80 instead of lots of separate jobs.

* Only pass -t flag if we are running in a terminal.

* Updated php_all job to use new Docker stuff.

* Fixed PHP memory leaks and arginfo errors (#8614)

* Fixed a bunch of incorrect arginfo and a few incorrect error messages.

* Passes mem check test with no leaks!

* WIP.

* Fix build warning that was causing Bazel build to fail.

* Added compatibility code for PHP <8.0.

* Added test_valgrind target and made tests Valgrind-clean.

* Updated Valgrind test to fail if memory leaks are detected.

* Removed intermediate shell script so commands are easier to cut, paste, and modify.

* Passing all Valgrind tests!

* Hoist addref into ObjCache_Get().

* Removed special case of map descriptors by keying object map on upb_msgdef.

* Removed all remaining RETURN_ZVAL() macros.

* Removed all explicit reference add/del operations.

* Added REFCOUNTING.md to Makefile.am.

* Updated upb version and fixed PHP to not get unset message field. (#8621)

* Updated upb version and fixed PHP to not get unset message field.

* Updated changelog.

* Fixed preproc test to handle old versions of Clang withot __has_attribute().

* A second try at fixing __has_attribute().

* Copy __has_attribute() fix to cc file also.

* Updated failure list for PHP for fixed test.

* Updated version of upb for Ruby (#8624)

* Updated upb.

* Preserve legacy behavior for unset messages.

* Updated failure list.

* Updated CHANGES.txt.

* Added erroneously-deleted test file.

* Fixed condition on compatibility code.

* Re-introduced deleted file again, and fixed Rakefile to not delete it.

* Fix generation of test protos.
2021-05-19 15:53:47 -07:00
Adam Cozzette
65abb64e73
Fixed syntax errors in bswap_64 (#8613)
It looks like some syntax errors were introduced by mistake because some
left parentheses were changed to curly braces without the right parens
being changed accordingly. This fixes #8611.
2021-05-17 10:38:40 -07:00
Adam Cozzette
652d99a8ee
Update protobuf version (#8606) 2021-05-12 16:20:18 -07:00
Adam Cozzette
ca295506c5
Make the dist_install test use Java 8 (#8609)
This is necessary because the Kotlin support requires Java 8 or
higher.

Making this test use Java 8 required switching it to the java_stretch
Docker image and updating that image to include some Python
dependencies.
2021-05-12 14:42:15 -07:00
Adam Cozzette
0aabf89420 Remove some redundant language prefixes from the changelog for 3.17 2021-05-12 09:58:30 -07:00
Adam Cozzette
5a9367ac57 Update tests.sh to skip the Kotlin tests on Oracle 7 2021-05-12 09:58:30 -07:00
Adam Cozzette
a4866d34f9 Updated .gitignore to include Kotlin directories 2021-05-12 09:58:30 -07:00
Yuriy Chernyshov
ca7f54aa1a
Fix -Wunused-parameter in java_names.h (#8601)
* Fix -Wunused-parameter in java_names.h

* Apply review comments
2021-05-11 14:58:29 -07:00
Adam Cozzette
f807c02226
Update protobuf version to 3.17.0-rc2 (#8586)
* Update protobuf version

* Updated Kotlin versions and updated script to handle them
2021-05-10 15:31:35 -07:00
Yuriy Chernyshov
90d4f31438
Fix newly appeared -Wunused-parameter (#8587)
This is #8584 applied to 3.17.x
2021-05-10 13:50:15 -07:00
Adam Cozzette
3546d29a00
Add the kotlin and kotlin-lite directories to the main pom.xml (#8585)
* Add the kotlin and kotlin-lite directories to the main pom.xml

* Updated Kotlin versions to 3.17.0-rc1

* Fix Mac CI tests by resolving brew package conflict
2021-05-10 13:36:33 -07:00
Adam Cozzette
e211ad3300
Updated CHANGES.txt and PHP changelog for 3.17.0 (#8583) 2021-05-07 11:34:04 -07:00
Adam Cozzette
7025f37cde
Update protobuf version (#8575) 2021-05-07 09:51:15 -07:00
Adam Cozzette
3b08c721b8
Comment out unused variable in repeated_field_unittest.cc (#8578)
The test assertion is commented out, but this is causing some warnings
about an unused variable. This commit comments out the variable to fix
that problem.
2021-05-07 08:32:35 -07:00
Daniel Azuma
7e3bbeda92
fix(ruby): Fix crash when calculating Message hash values on 64-bit Windows (#8565)
* fix(ruby): Fix crash when calculating Message hash values on 64-bit Windows

* Better mapping for values outside the fixnum range

* Simpler downcasting of hash values

* Fix precedence

* Fix bundle on Ruby 2.4
2021-05-06 09:45:11 -07:00
Yuriy Chernyshov
c8389f8ed5
Fix constness issue detected by MSVC standard conforming mode (#8568)
This is the continuation of #8344
2021-05-06 09:09:22 -07:00
Daniel Azuma
414aca5f13
fix(ruby): Fix various exceptions in Ruby on 64-bit Windows (#8563)
* fix(ruby): Fix various exceptions in Ruby on 64-bit Windows.

Activates the secondary ObjectCache map on this platform, to prevent weak keys from being garbage collected. This happened on 64-bit Windows because pointers don't necessarily fit in a Fixnum, and were being represented as GC-able Bignums on that platform.

* Removed extraneous code, and used VALUE instead of intptr_t

* Call the C function for new object instance rather than evaling a Ruby string
2021-05-05 15:42:25 -07:00
Dennis Shao
b0f4462ef2
Merge pull request #8536 from shaod2/update-python-docs-script
Update kokoro/docs/publish-python.sh
2021-05-05 16:47:31 -05:00
Adam Cozzette
f98b287125
Remove Python compatibility tests (#8570) 2021-05-05 13:02:38 -07:00
Joshua Haberman
5df4c2ec94
Merge pull request #8546 from haberman/sync-stage
Integrate from Piper for C++, Java, and Python
2021-05-04 17:14:33 -07:00
Joshua Haberman
2bf63f5983 Removed file that was mistakenly re-introduced. 2021-05-04 15:50:02 -07:00
Joshua Haberman
753a2b76ee Merge branch 'master' into sync-stage 2021-05-04 15:49:27 -07:00
Joshua Haberman
8b87075924
Updated and simplified PHP testing structure (#8558)
* Simplified PHP testing setup.

- Consolidated on a single autoloader, created by composer.
- Consolidated on a single phpunit invocation strategy: we run
  phpunit on a directory, which will run all tests matching *Test.php
  in that directory.
- We now rely on autoloading to import all test protos. require_once()
  calls for test protos are removed.
- For now the valgrind tests are removed. A follow-up PR will re-enable
  them in a more robust way.

* More improvements to PHP testing.

1. Replace custom PHPUnit-selection logic in test.sh with generic
   composer version selection.
2. Optimized both test proto generation and the custom extension
   build to avoid unnecessary work when the files are already up
   to date.

* Added assertions to verify that the C test doesn't use PHP sources.

* Updated tests.sh for the new PHP testing commands.

* Removed obsolete rules from tests.sh.

* Fixed generate_test_protos.sh for when tmp does not exist.

Also removed undefined_test.php and fixed Makefile.am.

* Added php8.0_all again which is still used.

* Added missing file to Makefile.am.

* Re-added php_all_32 rule which is also still used.

* Updated testing commands for macOS and download composer.

* Use /usr/local/bin on mac instead of /usr/bin, since the latter is not writable.
2021-05-04 10:19:22 -07:00
Paul Yang
114efc42d4
Fix php json parsing not throwing error for unknown field. (#8528) 2021-05-04 10:09:56 -07:00
Yuriy Chernyshov
28101c3e39
Stop using GG_ prefixed macros (#8539)
* Stop using GG_ prefixed macros

GG_ULONGLONG was renamed to PROTOBUF_ULONGLONG some time ago

* Better
2021-05-03 15:05:02 -07:00
Kenneth Lum
952cf14442
fix typo (#8548) 2021-05-03 11:21:59 -07:00
Joshua Haberman
67044a2820 Reinstate the Python change that was somehow reverted in the sync. 2021-04-30 16:28:12 -07:00