Commit Graph

7272 Commits

Author SHA1 Message Date
Paul Yang
537c5aa9e0
Aggregate Metadata Files (#7155) (#7194)
Instead of calling initOnce of dependencies, initialize metadata of dependencies in the same file.

Needs to pass aggregate_metadata option to protoc to trigger, e.g.:
--php_out=aggregate_metadata=foo#bar:generated_dir
For each input file, transitive dependencies (including itself), whose package name has the prefix of foo or bar, will be aggregated, in which their metadata string will be aggregated in the same internalAddGeneratedFile call. For other dependencies, initOnce is called as before.

This feature is EXPERIMENTAL. DO NOT USE!!!
2020-02-11 15:44:40 -08:00
Stephan Hartmann
32caf511c8 Fix GCC build with PROTOBUF_USE_DLLS enabled
GCC does not allow mixing __attribute__(()) syntax alignas()
syntax. Re-use approach from chromium base/compiler_specific.h
2020-02-11 14:30:57 -08:00
Joshua Haberman
f2c5ee5117
Fix for JSON serialization of 0/empty-valued wrapper types (#7198)
* Fixed Ruby JSON serialization of 0/empty wrapper fields.

* Removed newly-passing conformance tests from the failure list.
2020-02-11 13:18:32 -08:00
Adam Cozzette
802e1b99b2 Attempt to mitigate "docker pull" failures
Lots of CI tests have been failing recently with error messages like
this from "docker pull":

  Error response from daemon: unexpected EOF

This seems to be an issue affecting other projects as well:
https://forums.docker.com/t/docker-pull-causes-error-response-from-daemon-unexpected-eof/88608

This commit tries to mitigate the problem by periodically retrying
failed Docker pulls.
2020-02-11 12:59:49 -08:00
Tim Swast
d1f4e25e15
python: publish sphinx docs to googleapis.dev
**Background:**

This is a follow-up to the PR that adds sphinx docs.

Googleapis.dev hosts client library documentation for several languages as
well as documentation for core packages such as google-api-core. It is owned
and maintained by the Cloud DevRel Core team.

**About this change:**

The `kokoro/docs/publish-python.sh` script builds the docs with sphinx, then
uses the [googleapis/docuploader](https://github.com/googleapis/docuploader)
tool to publish to googleapis.dev. The publish script is triggered manually
with a Kokoro RELEASE type job. Googlers, see go link
protobuf-docs-python-design for additional internal references.
2020-02-11 13:55:20 -06:00
Tim Swast
29c83baecc
python: add sphinx docs (#6525)
* python: generate documentation with Sphinx and Read the Docs

Background:

Formerly, the Python protobuf reference documentation was built with
[Epydoc](http://epydoc.sourceforge.net/). This package has not been
updated since 2008, and it has inconsistent formatting (see internal
issue 131415575) with most Python documentation. Sphinx is used for the
official docs.python.org docs as well as most other Python packages,
including the Google client libraries and related packages, such as
https://googleapis.dev/python/google-api-core/latest/

To build the docs with Sphinx:

1. Install the needed packages (`sphinx`, `sphinxcontrib-napoleon` for
Google-style docstring support). I've created a conda environment file
to make this easier:

```
conda env create -f python/docs/environment.yml
```

2. (Optional) Generate reference docs files and regenerate index:

```
cd python
python generate_docs.py
cd ..
```

3. Run Sphinx.

```
cd python/docs
make html
```

About this change:

The script at `python/generate_docs.py` creates a ReStructured Text file
for each public module in the protobuf Python package. The script also
updates the table of contents in `python/docs/index.rst` to point to
these module references.

Future work:

Testing the docs build on PRs requires contributors to actually do some
setup work to configure builds on their fork. It'd be better if CI had a
docs build session to verify that the Sphinx docs generation at least
runs.

There are many warnings due to not-quite-correct docstrings in the
actual Python code itself. I'm choosing to ignore these errors to keep
the PR small, but I recommend you fix these and then enable "fail on
warnings" in the docs build on CI.

* add docs to EXTRA_DIST

* add instructions to build documentation to generate_docs.py

* exclude python/odcs from cpp_distcheck
2020-02-11 11:40:17 -08:00
Joshua Haberman
1a74ba4cb4
Fix for wrappers with a zero value (#7195)
* Add failing tests for issues with wrapped values where the value is the default

* Add test for wrapped values without a value set

* Bugfix for wrapper types with default values.

The previous optimizations for wrapper types had a bug that prevented
wrappers from registering as "present" if the "value" field was not
present on the wire.

In practice the "value" field will not be serialized when it is zero,
according to proto3 semantics, but due to the optimization this
prevented it from creating a new object to represent the presence of the
field.

The fix is to ensure that if the wrapper message is present on the wire,
we always initialize its value to zero.

Co-authored-by: Dan Quan <dan@quan.io>
2020-02-11 08:20:00 -08:00
Jan Tattermusch
f20be83927 enforce recursion depth checking for unknown fields 2020-02-10 15:30:47 -08:00
Paul Yang
ac70b7cfb7
Aggregate Metadata Files (#7155)
Instead of calling initOnce of dependencies, initialize metadata of dependencies in the same file.

Needs to pass aggregate_metadata option to protoc to trigger, e.g.:
--php_out=aggregate_metadata=foo#bar:generated_dir
For each input file, transitive dependencies (including itself), whose package name has the prefix of foo or bar, will be aggregated, in which their metadata string will be aggregated in the same internalAddGeneratedFile call. For other dependencies, initOnce is called as before.

This feature is EXPERIMENTAL. DO NOT USE!!!
2020-02-10 11:27:27 -08:00
Rafi Kamal
fedb2beee3
Merge pull request #6938 from ObsidianMinor/csharp/fix/6936 (#7188)
Fix latest ArgumentException for C# extensions

Co-authored-by: Jan Tattermusch <jtattermusch@users.noreply.github.com>
2020-02-10 10:46:19 -08:00
Thomas Van Lenten
56c48ae592 Revisit how the WKTs are bundled with ObjC.
There are have been a few issues around people using case sensitive file systems
what Xcode/clang does when looking at the paths. In attempts to solve one set of
warnings, new warnings/errors happened in different setup. So, to hopefully put
these problem away for got, move the WKTs to be at the same level as the other
headers.

- Revert "Override CocoaPods module to lowercase (#6464)"
  This reverts commit 479ba8226b.
- Move WKTs to the objectivec directory and make the old headers shim back to
  the new locations.
- Update objectivec/generate_well_known_types.sh to check them one at a time
  and to deal with the new locations for them.

Fixes #6803
2020-02-10 12:20:05 -05:00
Rafi Kamal
7e0a081e78
Add js files added in #7176 to Makefiles (#7189) 2020-02-07 17:14:49 -08:00
Protobuf Team
cc938ea39a Project import generated by Copybara.
PiperOrigin-RevId: 293139128
2020-02-05 17:09:37 -08:00
Rafi Kamal
498de9f761
Changed the maven URL to use https when building dist artifacts 2020-02-02 14:04:32 -08:00
Rafi Kamal
5f3d6599f6
Update CHANGES.txt for 3.11.3 release (#7162) 2020-01-31 16:00:28 -08:00
Rafi Kamal
345df38dd3
Update protobuf version (#7143) 2020-01-31 13:47:09 -08:00
Rafi Kamal
bc237eb672
Install RubyGem bundler version specified in Gemfile.lock to fix release failure (#7144) (#7156)
Ref: https://bundler.io/blog/2019/05/14/solutions-for-cant-find-gem-bundler-with-executable-bundle.html
2020-01-31 09:43:58 -08:00
Elliotte Rusty Harold
47434eb43d
deps: update errorprone to 2.3.4 (#7125) (#7157) 2020-01-31 09:43:24 -08:00
Luca Santarella
c5eada8eeb
Refactored ulong to zend_ulong for php7.4 compatibility (#7147) 2020-01-30 16:47:32 -08:00
Rafi Kamal
4cf5bfee95
Install RubyGem bundler version specified in Gemfile.lock to fix release failure (#7144)
* Update Gem version to fix Ruby release failure

Ref: https://bundler.io/blog/2019/05/14/solutions-for-cant-find-gem-bundler-with-executable-bundle.html
2020-01-30 13:08:31 -08:00
Paul Yang
85219578fa
Add continuous test for php7.4 on mac (#7153) 2020-01-30 12:45:47 -08:00
Dave MacLachlan
302f3ea1e5 Update GOOGLE_PROTOBUF_OBJC_VERSION to 30003 2020-01-30 13:08:32 -05:00
Elliotte Rusty Harold
5e014c2ddf
Update JUnit and Truth (#7149) 2020-01-29 12:51:46 -08:00
Elliotte Rusty Harold
a62ee83401 deps: update errorprone to 2.3.4 (#7125)
deps: update errorprone to 2.3.4
2020-01-28 12:16:17 -08:00
Dave MacLachlan
f4a6b99339 Block subclassing
Add GPB_FINAL macro to block subclassing of generated ObjC classes.
2020-01-24 11:51:37 -05:00
dmaclach
6f9d488149 Move min Xcode version to 10.3
Xcode 10.1 has a compiler bug that breaks on generated code.
Xcode 10.3+ appear to be fine.
2020-01-23 16:43:00 -05:00
Thomas Van Lenten
effb65d4e3 Move a few more size() > 0 calls over to !empty(). 2020-01-23 12:53:52 -05:00
Thomas Van Lenten
edf69e5858 Make the unittest proto file generation handle additions better.
Didn't really handle when a file was missing, but all the other were up to date,
this handles that case also.
2020-01-23 08:30:51 -05:00
Paul Yang
e8016753e3
Maven requires https connection (#7110) (#7114) 2020-01-22 12:29:56 -08:00
Dave MacLachlan
e3c2af349f Clean up some warning messages.
- size vs empty
- unused decls
2020-01-22 14:42:57 -05:00
Paul Yang
629ca244a1
Call register_class before getClass from desc (#7077)
* Call register_class before getClass from desc

* Remove debug code
2020-01-21 15:38:42 -08:00
Dave MacLachlan
ab48ecf140 Update pddm to work with clang-format
Add clang-format off/on directives around pddm expansions to simplify formatting other code.
2020-01-21 13:14:01 -05:00
Dave MacLachlan
aa1d7e7c48 Change Objective C class references to using macros.
Allows easy replacing of class names in generated code using #define to redefine class names at compile time.
2020-01-21 12:12:11 -05:00
Paul Yang
39f4240856
Maven requires https connection (#7110) 2020-01-20 19:23:06 -08:00
Andres Valdes
35febfc2e9 Correct @return in Any.unpack docblock (#7089)
As per https://docs.phpdoc.org/latest/references/phpdoc/tags/return.html, the first token after the @return tag should be the return type.
2020-01-20 16:32:11 -08:00
Maxime Guerreiro
158b269097 Cleanup the RPC Implementations section
Remove dead links and update the ones that were moved.

Use https when possible.

Isolate inactive projects (not updated in the last few years) in a subsection.

release notes: no
2020-01-17 16:08:02 -08:00
dmaclach
f4b6e4a561 Clean up use of size() > 0 to !empty() 2020-01-15 17:54:53 -05:00
dmaclach
836d81a826 Fix up parameter name so decl matches defn
className -> class_name
2020-01-15 17:53:56 -05:00
Dave MacLachlan
74956e1c15 Use references to Objective C classes instead of looking classes up by name.
This should reduce binary size slightly, small performance improvement, and improve linkage by forcing references to all used classes.

Note that this maintains backwards compatibility for sources generated by older protoc for the time being. If you want the benefits
you will need to recompile your protos with the newer protoc.
2020-01-15 15:01:53 -05:00
Yannic
948740bc9d [bazel] Fix blacklisted_protos in cc_toolchain and add test (#7075) 2020-01-15 10:27:35 -08:00
Yannic
29cd005ce1 [bazel] Fix blacklisted_protos in cc_toolchain and add test (#7096) 2020-01-15 10:22:16 -08:00
Scott Hart
80eca4c1cd bug: #7076 adds OUT and OPTIONAL to windows portability files (#7088) 2020-01-13 15:21:40 -08:00
Scott Hart
2d9b1788e7 bug: #7076 adds OUT and OPTIONAL to windows portability files (#7087) 2020-01-13 15:21:27 -08:00
salamaniibm
c3ffdabed5 correcting the s390x Arch name 2020-01-10 16:07:15 -08:00
Elliotte Rusty Harold
5a578d2a5c deps: update guava to 28.2-android 2020-01-10 10:52:48 -08:00
Jingwen Chen
abc5062a61 Migrate from maven_jar to jvm_maven_import_external to prepare for Bazel 2.0 2020-01-09 14:10:34 -08:00
Yannic
77f30b686b Blacklist .proto source files if Bazel allows us to (#7065)
* Blacklist .proto source files is Bazel allows us to

This is a partial revert of 7b28278c7d to unblock, e.g., https://github.com/grpc/grpc/pull/21590 or https://github.com/lyft/envoy-mobile/issues/617 until Bazel is fixed.

Note: this is a forward-compatible change that automatically switches to the behavior intended by 7b28278c7d when a compatible Bazel is released without requiring users to upgrade Protobuf. We will revert this change when Bazel is fixed.

* Remove trailing ,

* Update BUILD
2020-01-09 14:10:34 -08:00
Yannic
082cc23afd Blacklist .proto source files if Bazel allows us to (#7065)
* Blacklist .proto source files is Bazel allows us to

This is a partial revert of 7b28278c7d to unblock, e.g., https://github.com/grpc/grpc/pull/21590 or https://github.com/lyft/envoy-mobile/issues/617 until Bazel is fixed.

Note: this is a forward-compatible change that automatically switches to the behavior intended by 7b28278c7d when a compatible Bazel is released without requiring users to upgrade Protobuf. We will revert this change when Bazel is fixed.

* Remove trailing ,

* Update BUILD
2020-01-09 10:23:02 -08:00
Rafi Kamal
ed8688de72
Merge pull request #7064 from rafi-kamal/3.11.x-202001071136
Merge branch 3.11.x to master
2020-01-08 12:24:20 -08:00
Brian Wignall
a104dffcb6 Fix typos (#7050)
Uses https://en.wikipedia.org/wiki/Wikipedia:Lists_of_common_misspellings/For_machines to find likely typos, with https://github.com/bwignall/typochecker to help automate the checking.
2020-01-08 10:18:20 -08:00