This involved fixing a few important bugs in the
Ruby implementation -- mostly cases of mixing
upb field types and descriptor types (upb field
types do not distinguish between int/sint/fixed/sfixed
like descriptor types do).
Also added protobuf-specific exceptions so parse
errors can be caught specifically.
Change-Id: Ib49d3db976900b2c6f3455c8b88af52cfb86e036
- We do still generate the message types, as otherwise reflection breaks, even though it doesn't actually use those types.
- JSON handling hasn't been implemented yet
Install google/protobuf/stubs/status.h, and google/protobuf/stubs/stringpiece.h -- these are required in order to include google/protobuf/util/type_resolver.h.
Install google/protobuf/stubs/bytestream.h -- this is required in order to include google/protobuf/util/json_util.h.
This involves:
- Specifying a namespace in each proto (including ones we'd previously missed)
- Updating the generation script
- Changing codegen to implement IReflectedMessage.Fields explicitly (a good thing anyway)
- Changing reflection tests to take account of the explicit interface implementation
Non-generated code in this commit; generated code to follow
Change the C# namespace in descriptor.proto to Google.Protobuf.Reflection.
This then means changing where the generated code lives, which means updating the project file...
It also involves regenerating the C++ - which has updated the well-known types as well,
for no terribly obvious reason...
- Add a partial method called by all constructors
- Generate internal classes for descriptor.proto (only)
- Forbid proto2 descriptors except for descriptor.proto
- FieldAccessorTable is now non-generic
- We don't have a static field per message type in the umbrella class. (Message descriptors are accessed via the file descriptor.)
- Removed the "descriptor assigner" complication from the descriptor fixup; without extensions, we don't need it
- MapField implements IDictionary (more tests would be good...)
- RepeatedField implements IList (more tests would be good)
- Use expression trees to build accessors. (Will need to test this on various platforms... probably need a fallback strategy just using reflection directly.)
- Added FieldDescriptor.IsMap
- Added tests for reflection with generated messages
Changes to generated code coming in next commit.
It's not possible to define "#error" inside a define.
It causes 'error: stray ‘#’ in program' compilation error.
Now the define GOOGLE_PROTOBUF_ATOMICOPS_ERROR is the error message
and it's used along the code together "#error".
- Remove some old proto2-based C#-only messages
- Remove the "build" directory which only contained out-of-date files
- Remove the csharp_namespace option from proto2 messages
- Change "Google.ProtocolBuffers" to "Google.Protobuf" in other messages
- Change the default message hash code to 1 to be consistent with other code
- Change the empty list/map hash code to 0 as "empty map" is equivalent to "no map"
- Removed map fields from unittest_proto3.proto
- Created map_unittest_proto3.proto which is like map_unittest.proto but proto3-only
- Fixed factory methods in FieldCodec highlighted by using all field types :)
- Added tests for map serialization:
- Extra fields within entries
- Entries with value then key
- Non-contiguous entries for the same map
- Multiple entries for the same key
Changes to generated code coming in next commit
- Make some members internal
- Remove a lot of FrameworkPortability that isn't required
- Start adding documentation comments
- Remove some more group-based members
- Not passing in "the last tag read" into Read*Array, g
Now the Build tool needs to define -DHAVE_ZLIB and -DHAVE-PTHREAD rather
than providing a config.h
- Make pbconfig.h a manually written file to handle hash conditions
according to platform related macros.
- Remove #include "config.h" from source code.
- Changed the configure.ac and Makefile.am to pass down the macros.
- Change cmake to pass down the the macros.
Change-Id: I537249d5df8fdeba189706aec436d1ab1104a4dc
Escape characters don't count for string literal size, no need to pre-generate escape string.
Added unit test to touch enormous cpp generated descriptor.
Updated makefile to include enormous_descriptor.proto
Fixed language compatibility error.
Remove ICodedInputStream and ICodedOutputStream, and rewrite CodedInputStream and CodedOutputStream to be specific to the binary format. If we want to support text-based formats, that can be a whole different serialization mechanism.
This makes repeated fields really awkward at the moment - but when we reimplement RepeatedField<T> to be backed by an array, we can cast the array directly...
Refactored the threadlocal logic for Android and IOS into logic in
platform_macro.h which computes a GOOGLE_PROTOBUF_NO_THREADLOCAL
define which is then used elsewhere. This allows new platforms
without THREADLOCAL to be easily defined.
- Add more to the ObjC dir readme.
- Merge the ExtensionField and ExtensionDescriptor to reduce overhead.
- Fix an initialization race.
- Clean up the Xcode schemes.
- Remove the class/enum filter.
- Remove some forced inline that were bloating things without proof of performance wins.
- Rename some internal types to avoid conflicts with the well know types protos.
- Drop the use of ApplyFunctions to the compiler/optimizer can do what it wants.
- Better document some possible future improvements.
- Add missing support for parsing repeated primitive fields in packed or unpacked forms.
- Improve -hash.
- Add *Count for repeated and map<> fields to avoid auto create when checking for them being set.
When compiling a protobuf with gcc 4.1.2 for powerpc, I ran into
another of the following warning message:
INFO: From Compiling my_proto.pb.cc powerpc-603e-linux-gcc:
bazel-out/local_linux-dbg/genfiles/my_proto.pb.cc: In member
function `virtual void MyProto::Clear()':
bazel-out/local_linux-dbg/genfiles/my_proto.pb.cc:223: warning: this
decimal constant is unsigned only in ISO C90
The line in the proto file that was triggering it was:
if (_has_bits_[24 / 32] & 4278190080) {
ZR_(field1_, field2_);
}
_has_bits_ is a uint32. The constant mask should therefore be
unsigned. This change updates the constant to be generated as
unsigned.
The GOOGLE_PREDICT_FALSE macro is both incorrect, and doesn't match
the macro definition in glog, which causes conflicts when including
both libraries. This commit fixes that by making it identical to
what is in glog.
RepeatedField::begin()/end() will return NULL when the content is empty.
Passing these NULL values to std::copy() will result in runtime complains
from some compilers (e.g., vs2010).
We use a custom allocator for map fields and this allocator must be
passed correctly to hash_map to make sure it's allocated properly
with our custom allocator.
Change-Id: Ie59fa24bf11ff28ffd0fa870e24e456c66b2b9c5
- Style fixups in the code.
- map<> serialization fixes and more tests.
- Autocreation of map<> fields (to match repeated fields).
- @@protoc_insertion_point(global_scope|imports).
- Fixup proto2 syntax extension support.
- Move all startup code to +initialize so it happen on class usage and not app startup.
- Have generated headers use forward declarations and move imports into generated code, reduces what is need at compile time to speed up compiled and avoid pointless rippling of rebuilds.
When compiling a protobuf with gcc 3.3.2 for powerpc, I ran into the
following warning message:
INFO: From Compiling my_proto.pb.cc powerpc-603e-linux-gcc:
bazel-out/local_linux-dbg/genfiles/my_proto.pb.cc: In member
function `virtual void MyProto::Clear()':
bazel-out/local_linux-dbg/genfiles/my_proto.pb.cc:223: warning: this
decimal constant is unsigned only in ISO C90
The line in the proto file that was triggering it was:
if (_has_bits_[24 / 32] & 4278190080) {
ZR_(field1_, field2_);
}
_has_bits_ is a uint32. The constant mask should therefore be
unsigned. This change updates the constant to be generated as
unsigned.
Remove the ClassList support (maybe bring it back in the future).
Trim the includes to hopefully get a working Window build.
Add some more returns after switches for compilers that warn even when all values of the enum are handled.
Use ghtonl instead of htonl.
Change the use of [u]int(8,32)_t within the ObjC generator code to [u]int(8,32) to match the rest of the compiler.
Add objective-c generator files to Visual Studio project.
This is the start of establishing a C# namespace of "Google.ProtocolBuffers.TestProtos.Proto3" for proto3-syntax protos.
We could optionally split the directory structure as well into Proto2 and Proto3 for clarity.
This is a follow up CL for df184fba00
(Id937e25bbb35968ee76c92bd4a8ce6247408c443), which added
#undef GOOGLE_PROTOBUF_MISSING_HASH
where GOOGLE_PROTOBUF_MISSING_HASH macro is never defined.
With this CL, GOOGLE_PROTOBUF_MISSING_HASH macro will be cleaned up
after it is used.
This is less ideal from a dex count perspective because it requires a
new variable for each message, and because most apps have proguard
rules that will ensure that CREATOR classes are retained.
However, it is required to be able to use nano protos inside of AIDL
files, as the autogenerated AIDL code fails to compile otherwise. This
is a substantial benefit as it allows for backwards-compatible
parameters and return types in AIDL methods along the lines of
safeparcel.
Bug: 19084705
Change-Id: I66a2c0424b96cf8ff6b631b186cc4f9407dfc1f4
It turns out dex (apparently) was inlining these protected final
methods from ExtendableMessageNano into every message class. Removing
these methods from the base class and inlining their code reduces
the method count by 2 methods / message when the store_unknown_fields
option is on.
Change-Id: I0aa09f2016d39939c4c8b8219601793b8fab301f
I wasn't able to get the clear() method to inline into the
constructor when optimizations are on in proguard. As a result,
every message has an extra superfluous kept method assuming the
app never uses clear() directly.
There are a couple of instances where setting this option false is
necessary in order to get code dexing successfully without hitting
the method limit, e.g. https://goto.google.com/tltzq
In this example, I tried turning on the method/inlining/unique and
method/inlining/short optimizations before resorting to adding the
generate_clear option, but the method count did not decrease. The
clear() methods were contributing over a thousand extra methods.
Change-Id: If6a9651d6a59cdf70b1040d8248779710ac73105
@IntDef is a support library annotation which allows build tools to
determine the valid set of values for a given integer field when that
field is intended to be restricted like an enum. This avoids the
overhead of enums while still allowing for compile-time type checking
in most circumstances.
Change-Id: Iee02e0b49a8e069f6456572f538e0a0d301fdfd5
https://android-review.googlesource.com/#/c/67890/ removed field
initialization from the ctor, making it just call clear() instead.
When I added the generate_clear option back (as part of the reftypes
compat mode) in https://android-review.googlesource.com/#/c/109530/,
I forgot to ensure that what clear() used to do was inlined in the
constructor.
This change fixes NPEs that are happening for users of
reftypes_compat_mode who rely on unset repeated fields being empty
arrays rather than null.
Change-Id: Idb58746c60f4a4054b7ebb5c3b0e76b16ff88184
Previously, extensions with field numbers greater than 268435455 would
result in a compile time error in generated code that looks something
like this:
Foo.java:3178: error: integer number too large: 3346754610
3346754610);
This is because we were trying to represent the tag number (an
unsigned int) using a java int constant, but java int constants are
signed, and can't exceed Integer.MAX_VALUE.
Fixed by declaring it as a long instead, and casting it down to an
int in the implementation. This is safe, because the tag value always
fits in 32 bis.
Change-Id: If2017bacb4e20af667eaeaf9b65ddc2c30a7709f
The file `SEBS` is a build file for a toy build system I was writing in my spare time six years ago, back when I maintained protobufs. Arguably it was never appropriate for me to have placed this file in the protobuf tree, but it was convenient at the time and didn't seem to hurt anything. It clearly doesn't belong here now, though.
The later is not supported by some versions of MSVC and the former is
more accurate for the use case.
Change-Id: I7096226b1ebfef8340d288bbaf2788540aa3e97f
* Rosy hack doesn't apply (that test should be removed
for the open-source release).
* Added our own copy of parameterized.py (the open-source
version of Google Apputils doesn't contain it).
* The C++ Descriptor object didn't implement extension_ranges.
* Had to implement a hack around returning EncodeError, to
work around the module-loading behavior of the test runner.
Previously, we supported map fields in the Ruby DSL. However, we never
connected the final link in the chain and generated `map` DSL commands
for map fields in `.proto` files. My apologies -- I had been testing
with the DSL directly so I missed this.
Also fixed a handlerdata-setup-infinite-loop when a map value field's
type is its containing message.
"--dependency_manifest_out=FILE", protoc will write dependencies of
input proto files into FILE. In FILE, the format will be
<full path to FILE>: <full path to 1st proto>\\\n <full path to 2nd proto> ...
This cl is based on https://github.com/google/protobuf/pull/178
Use --manifest-file=somefile.d to output the dependency manifest.
This file will contain a list of files which were read by protoc as part
of creating the output files. It doesn't include the plugin inputs if
plugins are used, that could be a later extension.
The manifest file is in the format <output file>: <input files>. The
manifest file format only allows you to specify one output file, which
isn't a problem as it's used to detect input changes in order to detect
when to rerun the protoc command. The output file used in the manifest
is the manifest filename itself; to use this in ninja you should declare
the manifest file as the first output as well as the depfile input.
- A golden-file test that ensures protoc produces known-valid output.
- A Ruby test that loads that golden file and ensures it actually works
with the extension.
This split strategy allows us to test end-to-end without needing to
integrate the Ruby gem build system and the protoc build system. This is
desirable because we do not want a gem build/install to depend on
building protoc, and we do not want building protoc to depend on
building and testing the gem.
Including <sys/param.h> on Mac/iOS doesn't define __BYTE_ORDER so
PROTOBUF_LITTLE_ENDIAN was never being defined. This commit adds a check
for the __LITTLE_ENDIAN__ macro which is defined by clang and Apple gcc on
little endian architectures.
system. The Ruby module build now uses an amalgamated distribution of
upb, and successfully builds a Ruby gem called 'google-protobuf' with
module 'google/protobuf'.
