Commit Graph

7456 Commits

Author SHA1 Message Date
Rafi Kamal
9bbc664671 Project import generated by Copybara
PiperOrigin-RevId: 297709554
2020-02-27 15:55:57 -08:00
David L. Jones
42903f87f0
Merge pull request #7154 from tswast/issue4498-googleapis.dev
python: publish sphinx docs to googleapis.dev
2020-02-27 14:30:54 -08:00
David L. Jones
73f3a7209b Fix typo in Makefile.am.
This is a fix for a typo in commit 743a4322ba.

The missing newline continuation causes the re-generated Makefile to yield a parse error.
2020-02-27 11:26:39 -08:00
Protobuf Team
d2727a2ce3 Project import generated by Copybara
PiperOrigin-RevId: 297637283
2020-02-27 10:55:04 -08:00
Artem Kustikov
743a4322ba
Fix js message pivot selection (#6813)
* fix javascript setFieldIgnoringDefault_ logic

* remove package-lock.json

* fix build script to include new UT asset file

Co-authored-by: Daniel Kurka <kurka.daniel@gmail.com>
2020-02-27 09:10:16 -08:00
Mark Schaller
88579a44fb Reserve extension for Bazel failure detail metadata
The Bazel project (https://github.com/bazelbuild/bazel) intends to
enrich its server app's (protobuf-defined) command service's status
message with a structured representation of what, if anything, went
wrong while processing a command. We wish to use custom options to add
metadata to the protobuf constructs involved.

release notes: no
2020-02-25 14:40:36 -08:00
Egor Pugin
c798ba18a9 bug: #7076 adds ERROR to windows portability files 2020-02-21 14:49:18 -08:00
wsw2016
b96241b1b7 Improves performance of json_stream_parser.cc by factor 1000
JsonStreamParser::GetNextTokenType() uses HasPrefixString a lot on StringPiece as input. For each call two std::strings are constructed, compared and destroyed. Parsing of json-files with 50-60 MB in debug mode takes minutes.
2020-02-19 11:33:46 -08:00
Rafi Kamal
1370c7303a
Merge pull request #7231 from protocolbuffers/3.11.x-202002181624
Merge branch 3.11.x to master
2020-02-18 17:39:41 -08:00
Rafi Kamal
0eb476b696 Merge branch 3.11.x to master 2020-02-18 16:42:47 -08:00
Rafi Kamal
df2bce345d
Remove 32-bit Mac protoc reference from collect_all_artifacts (#7228) 2020-02-18 12:55:01 -08:00
Rafi Kamal
147fd0b710
Update Google.Protobuf.Tools.nuspec to remove 32-bit Mac protoc reference (#7227) 2020-02-18 10:23:20 -08:00
Rafi Kamal
1f2d6bf4b5
Remove 32-bit Mac binary from csharp build tools (#7215) 2020-02-14 12:50:45 -08:00
Rafi Kamal
d0bfd52211
Skip building distribution tar files for 32 bit Mac (#7214) 2020-02-14 12:13:20 -08:00
Rafi Kamal
e9e4c687d6
Update CHANGES.txt to include the description of #7210 (#7212) 2020-02-14 10:19:40 -08:00
Jan Tattermusch
0e8f69e532
enforce recursion depth checking for unknown fields (#7210) 2020-02-14 10:17:06 -08:00
rafikamal
83b47e4c65 Project import generated by Copybara
PiperOrigin-RevId: 294283216
2020-02-13 15:48:49 -08:00
rafikamal
becf5afd34 Project import generated by Copybara
PiperOrigin-RevId: 293911829
2020-02-13 15:48:41 -08:00
rafikamal
46b0fd9cec Project import generated by Copybara
PiperOrigin-RevId: 293893031
2020-02-13 15:48:34 -08:00
Rafi Kamal
d314101531
Remove protoc release for 32-bit Macs (#7209)
* Remove protoc release for 32-bit Macs 

Apple has been removing the support for 32-bit Mac apps: https://support.apple.com/en-us/HT208436

Our release infrastructure no longer supports building for 32-bit architecture.

* Remove protoc artifact for MacOS 32-bit
2020-02-13 14:00:19 -08:00
Yannic
f0cb9cdb95
[bazel] Move Java runtime/toolchains into //java (#7190)
* [bazel] Move Java runtime/toolchains into //java

This change moves `java_library` targets from the top-level BUILD file
into `//java/{core,lite,util}` and declares `alias` targets to point to
their new locations (hence, this is not a breaking change).

This will allow users that don't use Java to stop depending on
`@rules_java` (e.g. as requested in
https://github.com/bazelbuild/rules_scala/pull/989#issuecomment-583405161).

Note that there is no intention to deprecate + remove the top-level
targets in the foreseeable future.

* Add BUILD files in //java to java_EXTRA_DIST
2020-02-13 13:04:14 -08:00
Rafi Kamal
7393bf12fa
Update CHANGES.txt for 3.11.4 release (#7205) 2020-02-12 14:41:31 -08:00
Rafi Kamal
c74057267d
Update protobuf version (#7206) 2020-02-12 14:41:16 -08:00
Rafi Kamal
906e9c6897
Fix for JSON serialization of 0/empty-valued wrapper types (#7198) (#7204)
* Fixed Ruby JSON serialization of 0/empty wrapper fields.

* Removed newly-passing conformance tests from the failure list.

Co-authored-by: Joshua Haberman <jhaberman@gmail.com>
2020-02-12 13:27:36 -08:00
Chuck Atkins
10c48c980f Intel compiler: silence noisy warning for incorrectly evaluated inlining 2020-02-12 13:08:59 -08:00
Chuck Atkins
8312c36567 Intel compiler: ifdef out an incorrectly evaluated is_pod type trait 2020-02-12 13:08:59 -08:00
Yannic Bonenberger
06ae5d8042 Use PROTOBUF_RTTI from port_def.inc macro instead of checking for absence of GOOGLE_PROTOBUF_NO_RTTI macro
These were the only 2 remaining occurences of this pattern.
2020-02-12 13:08:25 -08:00
Martin Pärtel
82a057596e Destruct CodedOutputStream instead of using Trim() 2020-02-12 13:06:52 -08:00
Martin Pärtel
f6902d6afe Made protoc --descriptor_set_out deterministic.
Repeatable protoc output is useful because it's sometimes
checked into version control, and rerunning protoc should
not generate unnecessary diffs.
2020-02-12 13:06:52 -08:00
Rafi Kamal
3c020b90a7
Update CHANGES.txt for 3.11.4 release 2020-02-12 12:51:32 -08:00
Rafi Kamal
37fc4327ef
Fix for wrappers with a zero value (#7195) (#7201)
* Add failing tests for issues with wrapped values where the value is the default

* Add test for wrapped values without a value set

* Bugfix for wrapper types with default values.

The previous optimizations for wrapper types had a bug that prevented
wrappers from registering as "present" if the "value" field was not
present on the wire.

In practice the "value" field will not be serialized when it is zero,
according to proto3 semantics, but due to the optimization this
prevented it from creating a new object to represent the presence of the
field.

The fix is to ensure that if the wrapper message is present on the wire,
we always initialize its value to zero.

Co-authored-by: Joshua Haberman <jhaberman@gmail.com>
Co-authored-by: Dan Quan <dan@quan.io>
2020-02-12 12:37:19 -08:00
Rafi Kamal
1440569231
Update the Xcode version number in Kokoro (#7202) (#7203)
Currently all of our MacOS tests are failing with the error: "autom4te: need GNU m4 1.4 or later: /usr/bin/m4". This is likely to be happening because Kokoro has updated their Xcode version to 11.3.
2020-02-12 11:34:51 -08:00
Rafi Kamal
a24e594833
Update the Xcode version number in Kokoro (#7202)
Currently all of our MacOS tests are failing with the error: "autom4te: need GNU m4 1.4 or later: /usr/bin/m4". This is likely to be happening because Kokoro has updated their Xcode version to 11.3.
2020-02-12 11:28:38 -08:00
Paul Yang
537c5aa9e0
Aggregate Metadata Files (#7155) (#7194)
Instead of calling initOnce of dependencies, initialize metadata of dependencies in the same file.

Needs to pass aggregate_metadata option to protoc to trigger, e.g.:
--php_out=aggregate_metadata=foo#bar:generated_dir
For each input file, transitive dependencies (including itself), whose package name has the prefix of foo or bar, will be aggregated, in which their metadata string will be aggregated in the same internalAddGeneratedFile call. For other dependencies, initOnce is called as before.

This feature is EXPERIMENTAL. DO NOT USE!!!
2020-02-11 15:44:40 -08:00
Stephan Hartmann
32caf511c8 Fix GCC build with PROTOBUF_USE_DLLS enabled
GCC does not allow mixing __attribute__(()) syntax alignas()
syntax. Re-use approach from chromium base/compiler_specific.h
2020-02-11 14:30:57 -08:00
Joshua Haberman
f2c5ee5117
Fix for JSON serialization of 0/empty-valued wrapper types (#7198)
* Fixed Ruby JSON serialization of 0/empty wrapper fields.

* Removed newly-passing conformance tests from the failure list.
2020-02-11 13:18:32 -08:00
Adam Cozzette
802e1b99b2 Attempt to mitigate "docker pull" failures
Lots of CI tests have been failing recently with error messages like
this from "docker pull":

  Error response from daemon: unexpected EOF

This seems to be an issue affecting other projects as well:
https://forums.docker.com/t/docker-pull-causes-error-response-from-daemon-unexpected-eof/88608

This commit tries to mitigate the problem by periodically retrying
failed Docker pulls.
2020-02-11 12:59:49 -08:00
Tim Swast
d1f4e25e15
python: publish sphinx docs to googleapis.dev
**Background:**

This is a follow-up to the PR that adds sphinx docs.

Googleapis.dev hosts client library documentation for several languages as
well as documentation for core packages such as google-api-core. It is owned
and maintained by the Cloud DevRel Core team.

**About this change:**

The `kokoro/docs/publish-python.sh` script builds the docs with sphinx, then
uses the [googleapis/docuploader](https://github.com/googleapis/docuploader)
tool to publish to googleapis.dev. The publish script is triggered manually
with a Kokoro RELEASE type job. Googlers, see go link
protobuf-docs-python-design for additional internal references.
2020-02-11 13:55:20 -06:00
Tim Swast
29c83baecc
python: add sphinx docs (#6525)
* python: generate documentation with Sphinx and Read the Docs

Background:

Formerly, the Python protobuf reference documentation was built with
[Epydoc](http://epydoc.sourceforge.net/). This package has not been
updated since 2008, and it has inconsistent formatting (see internal
issue 131415575) with most Python documentation. Sphinx is used for the
official docs.python.org docs as well as most other Python packages,
including the Google client libraries and related packages, such as
https://googleapis.dev/python/google-api-core/latest/

To build the docs with Sphinx:

1. Install the needed packages (`sphinx`, `sphinxcontrib-napoleon` for
Google-style docstring support). I've created a conda environment file
to make this easier:

```
conda env create -f python/docs/environment.yml
```

2. (Optional) Generate reference docs files and regenerate index:

```
cd python
python generate_docs.py
cd ..
```

3. Run Sphinx.

```
cd python/docs
make html
```

About this change:

The script at `python/generate_docs.py` creates a ReStructured Text file
for each public module in the protobuf Python package. The script also
updates the table of contents in `python/docs/index.rst` to point to
these module references.

Future work:

Testing the docs build on PRs requires contributors to actually do some
setup work to configure builds on their fork. It'd be better if CI had a
docs build session to verify that the Sphinx docs generation at least
runs.

There are many warnings due to not-quite-correct docstrings in the
actual Python code itself. I'm choosing to ignore these errors to keep
the PR small, but I recommend you fix these and then enable "fail on
warnings" in the docs build on CI.

* add docs to EXTRA_DIST

* add instructions to build documentation to generate_docs.py

* exclude python/odcs from cpp_distcheck
2020-02-11 11:40:17 -08:00
Joshua Haberman
1a74ba4cb4
Fix for wrappers with a zero value (#7195)
* Add failing tests for issues with wrapped values where the value is the default

* Add test for wrapped values without a value set

* Bugfix for wrapper types with default values.

The previous optimizations for wrapper types had a bug that prevented
wrappers from registering as "present" if the "value" field was not
present on the wire.

In practice the "value" field will not be serialized when it is zero,
according to proto3 semantics, but due to the optimization this
prevented it from creating a new object to represent the presence of the
field.

The fix is to ensure that if the wrapper message is present on the wire,
we always initialize its value to zero.

Co-authored-by: Dan Quan <dan@quan.io>
2020-02-11 08:20:00 -08:00
Jan Tattermusch
f20be83927 enforce recursion depth checking for unknown fields 2020-02-10 15:30:47 -08:00
Paul Yang
ac70b7cfb7
Aggregate Metadata Files (#7155)
Instead of calling initOnce of dependencies, initialize metadata of dependencies in the same file.

Needs to pass aggregate_metadata option to protoc to trigger, e.g.:
--php_out=aggregate_metadata=foo#bar:generated_dir
For each input file, transitive dependencies (including itself), whose package name has the prefix of foo or bar, will be aggregated, in which their metadata string will be aggregated in the same internalAddGeneratedFile call. For other dependencies, initOnce is called as before.

This feature is EXPERIMENTAL. DO NOT USE!!!
2020-02-10 11:27:27 -08:00
Rafi Kamal
fedb2beee3
Merge pull request #6938 from ObsidianMinor/csharp/fix/6936 (#7188)
Fix latest ArgumentException for C# extensions

Co-authored-by: Jan Tattermusch <jtattermusch@users.noreply.github.com>
2020-02-10 10:46:19 -08:00
Thomas Van Lenten
56c48ae592 Revisit how the WKTs are bundled with ObjC.
There are have been a few issues around people using case sensitive file systems
what Xcode/clang does when looking at the paths. In attempts to solve one set of
warnings, new warnings/errors happened in different setup. So, to hopefully put
these problem away for got, move the WKTs to be at the same level as the other
headers.

- Revert "Override CocoaPods module to lowercase (#6464)"
  This reverts commit 479ba8226b.
- Move WKTs to the objectivec directory and make the old headers shim back to
  the new locations.
- Update objectivec/generate_well_known_types.sh to check them one at a time
  and to deal with the new locations for them.

Fixes #6803
2020-02-10 12:20:05 -05:00
Rafi Kamal
7e0a081e78
Add js files added in #7176 to Makefiles (#7189) 2020-02-07 17:14:49 -08:00
Protobuf Team
cc938ea39a Project import generated by Copybara.
PiperOrigin-RevId: 293139128
2020-02-05 17:09:37 -08:00
Rafi Kamal
498de9f761
Changed the maven URL to use https when building dist artifacts 2020-02-02 14:04:32 -08:00
Rafi Kamal
5f3d6599f6
Update CHANGES.txt for 3.11.3 release (#7162) 2020-01-31 16:00:28 -08:00
Rafi Kamal
345df38dd3
Update protobuf version (#7143) 2020-01-31 13:47:09 -08:00
Rafi Kamal
bc237eb672
Install RubyGem bundler version specified in Gemfile.lock to fix release failure (#7144) (#7156)
Ref: https://bundler.io/blog/2019/05/14/solutions-for-cant-find-gem-bundler-with-executable-bundle.html
2020-01-31 09:43:58 -08:00