There is an example at the top of the page which contains code that appears to show a user how to create and serialize a message to a file. However the flags to open the file lack the O_CREAT flag which allows creating the file if it doesn't exist. I was troubleshooting a situation where this snippet was used and compiled, but never created a file.
This patch adds support for atomic operations on Solaris, on any platform.
It makes use of the atomic functions made available in Solaris' atomic.h
header.
__atomic_store_n() cannot take a memory model argument of
__ATOMIC_ACQUIRE, and __atomic_load_n() cannot take a memory model
argument of __ATOMIC_RELEASE, per the GCC documentation:
https://gcc.gnu.org/onlinedocs/gcc-4.9.1/gcc/_005f_005fatomic-Builtins.html
On Clang this generates a -Watomic-memory-ordering warning.
Promote the fences in Acquire_Store() and Release_Load() to the stronger
__ATOMIC_SEQ_CST memory model, which ought to be safe.
Note that there are no actual uses of Acquire_Store() or Release_Load()
in protobuf, though.
This follows the TSAN atomicops implementation, which also uses SEQ_CST
fences for these functions.
(Fixes #25.)
We cannot use Clang's __has_extension macro unless we really are
compiling on Clang, which means we cannot use this expression:
#if (defined(__clang__) && __has_extension(c_atomic)))
// ...
#endif
On GCC, this generates the following errors:
In file included from ./google/protobuf/stubs/atomicops.h:59:0,
from google/protobuf/stubs/atomicops_internals_x86_gcc.cc:36:
./google/protobuf/stubs/platform_macros.h:67:41: error: missing binary operator before token "("
(defined(__clang__) && __has_extension(c_atomic)))
^
In file included from google/protobuf/stubs/atomicops_internals_x86_gcc.cc:36:0:
./google/protobuf/stubs/atomicops.h:196:40: error: missing binary operator before token "("
(defined(__clang__) && __has_extension(c_atomic))
^
Instead, we have to protect the __has_extension expression by only
executing it when __clang__ is defined:
#if defined(__clang__)
# if __has_extension(c_atomic)
// ...
# endif
#endif
In the current implementation, a message with the same amount of null or
equal-valued fields as a different message type will have the same
hashCode. This adds more variety by including the hashCode of the
class's name in the hashCode calculations.
Change-Id: I284e3e6d198ad8037815948d1f65686465ffd623
Signed-off-by: Jason Neufeld <jneufeld@google.com>
The generic atomicops implementation is only exposed if GCC >= 4.7 is
available, but Clang, where the underlying __atomic built-ins are also
available, typically only claims to be GCC 4.2. This causes build
failures when compiling protobuf or the output of protoc's C++ code
generator on an architecture that needs the generic atomicops
implementation with Clang.
Clang has a "c_atomic" extension which can be tested for which almost
does what we want:
C11 atomic operations
Use __has_feature(c_atomic) or __has_extension(c_atomic) to
determine if support for atomic types using _Atomic is enabled.
Clang also provides a set of builtins which can be used to implement
the <stdatomic.h> operations on _Atomic types.
I'm not sure if this guarantees that the GNU atomic builtins (the ones
with the __atomic prefix) are also available, but in practice this
should guarantee that Clang is new enough.
With this change in place, Clang generates several diagnostics when
compiling the generic atomicops implementation. These appear to be bugs
in the generic atomicops implementation and are not Clang-specific.
The macro GOOGLE_PROTOBUF_ARCH_PPC is not used anywhere in the protobuf
source; there is no Power-specific atomics implementation, etc.
Funnily enough, the macro __ppc__ is not actually defined on 32-bit
Power on GCC/Linux, according to the following webpage:
http://nadeausoftware.com/articles/2012/02/c_c_tip_how_detect_processor_type_using_compiler_predefined_macros#POWER
and verified on a 32-bit Debian sid 'powerpc' chroot:
(sid_powerpc-dchroot)edmonds@partch:~$ gcc -dM -E - < /dev/null | grep -c __ppc__
0
(sid_powerpc-dchroot)edmonds@partch:~$ gcc -dM -E - < /dev/null | grep -c __LP64__
0
The behavior of the string ctor is undefined when you pass NULL. This
is checked strictly in C++11, so fails to compile.
Change-Id: Id5e0984ad1d37f2d504f7c42ac23e52ed4a58903
Instead of publishing its class I chose to encapsulate the troublesome
references in equals()/hashCode() in the generated code into superclass
methods in ExtendableMessageNano.
Changed a couple of java packages in the test suite to catch this issue
easier in the future.
Change-Id: I43f88411f63bb6f3ffc8d63361f2f77bebf6220a
This CL adds the "parcelable_messages" option. When enabled, all
generated message classes will conform to the Android Parcelable
contract. This is achieved by introducing a new parent class for
generated classes which implements the required functionality.
Since the store_unknown_fields option also makes use of a superclass,
ExtendableMessageNano, we have two versions of the new Parcelable
superclass: one extending MessageNano, and one extending
ExtendableMessageNano. These classes are otherwise identical.
As these classes depend on Android framework jars, they are not
included in the host .jar build of the nanoproto library.
Finally, add a test suite for running tests of Android-specific
functionality, as this cannot be done on a desktop JVM.
Change-Id: Icc2a257f03317e947f7078dbb9857c3286857497
Nano proto compiler normally throws an error if any service is
defined. If --ignore-services=true is set, no error is thrown and the
service is simply skipped.
Change-Id: Id82583555085cc55550d03a485d3f0189885240b
This avoids a race-condition when cachedSize is momentarily set to 0
for non-empty messages if multiple threads call getSerializedSize
(e.g. during serialization).
This is a retry of https://android-review.googlesource.com/#/c/88570/.
getSerializedSize() has been kept non-final so that messages generated
with a previous version of the compiler will not break.
Change-Id: I8d8154a10938cde579ae19c55eae55b1e70e0bda
This avoids a race-condition when cachedSize is momentarily set to 0
for non-empty messages if multiple threads call getSerializedSize
(e.g. during serialization).
Change-Id: I15a8ded92edbf41bf1c8d787960c5bbbc8a323c5
Both gcc and clang defines the __aarch64__ macro when building
for the 64-bit AArch64 execution state of ARMv8 processors, so
use this to detect the architecture.
Invalid values from the wire are silently ignored.
Unlike full/lite, the invalid values are not stored into the
unknown fields, because there's no way to get them out from
Nano's unknown fields without a matching Extension.
Edited README and slightly moved it towards a standalone
section for Nano, independent of the Micro section.
Change-Id: I2c1eb07f4d6d8f3aea242b8ddd95b9c966f3f177
Special values for float and double make it inaccurate to test the equality with ==.
The main Java library uses the standard Object.equals() implementation for all fields,
which for floating point fields means Float.equals() or Double.equals(). They define
equality as bitwise equality, with all NaN representations normalized to the same bit
sequence (and therefore equal to each other). This test checks that the nano
implementation complies with Object.equals(), so NaN == NaN and +0.0 != -0.0.
Change-Id: I97bb4a3687223d8a212c70cd736436b9dd80c1d7
- Get rid of TypeLiteral<T>. It was introduced to read the component
type of a List<T> at runtime. But we use arrays everywhere else,
and we can always read the component type of an array type at
runtime.
- Properly read/write "minor" types (e.g. sint32, sfixed32). The old
implementation could only read/write data as the "typical" types
(one per Java type), e.g. java.lang.Integer -> int32, java.lang.Long
-> int64. So if e.g. an extension specifies sfixed32 as the type, it
would be read/written in the totally incompatible int32 format.
- Properly serialize repeated packed fields. The old implementation
doesn't do packed serialization. As an added bonus, and to be more
aligned with the rest of protobuf nano / main, repeated packable
extensions can deserialize both packed and non-packed data.
- Split Extension class into a hierarchy so under typical usage a
large chunk of code dealing with primitive type extensions can be
removed by ProGuard.
Bug: https://code.google.com/p/android/issues/detail?id=62586
Change-Id: I0d692f35cc2a8ad3a5a1cb3ce001282b2356b041
Class initializers prevent ProGuard from inlining any methods
because it thinks the class initializer may have side effects.
This is true for static methods, but instance methods can still
be inlined, because to have an instance you will have touched
the class and any class initializers would have run. But
ProGuard only starts inlining instance methods of classes with
class initializers from v4.11b6, and Android uses v4.4 now.
This change tries to avoid the class initializers as much as
possible, by delaying the initialization of the empty array and
some fields' saved defaults until when they're needed. However,
if the message hosts any extensions, they must be public static
final and therefore introducing the class initializer. In that
case we won't bother with lazy initialization.
Change-Id: I00d8296f6eb0023112b93ee135cdb28dbd52b0b8
For nested message objects, don't generate accessor methods because they have
a default value that is not a valid value (null), so there is no reason to have
get/set/has/clear methods for them. Clients and protos (while serializing) can
check against the invalid value to see if it's been set.
Change-Id: Ic63400889581271b8cbcd9c45c84519d4921fd4b
It is a requirement for parsing code to handle packed and unpacked
forms on the wire for repeated packable fields. This change aligns
the javanano's behavior with the java's.
Bonus: optimize array length calculation when parsing repeated
fixed-size-element-type fields.
Bonus 2: lose "xMemoizedSerializedSize" for repeated enum fields,
and make the serialized size calculation match that for repeated
int32 fields.
Change-Id: I8a06103d9290234adb46b0971b5ed155544fe86a
- Migrates getCachedSize to the MessageNano parent class to save one method per message.
- Create ExtendableMessageNano parent class for protos with extensions, this saves the
getExtension and setExtension methods on the relevant messages.
- getSerializedSize's default case (with no fields to serialize) also migrate to the
parent class, which saves methods on empty messages.
- Container classes become interfaces to save the constructor.
Change-Id: I81f1a1b6d6a660096835e9df3ea20456655aab4a
Strip the null elements out before serializing the array.
This is helpful in the cases where the user wants to construct
an array of an inexact size for serialization. For example:
User constructs array of size 5 because they anticipate adding
more than 1 element before serialization. Only 3 get added, so
the array looks like [Obj, Obj, Obj, null, null]. This would
curently crash without this CL.
All repeated fields of ref-type elements can contain null
elements: repeated strings, repeated bytes, and repeated
messages/groups.
Change-Id: I117391c868c9a436536d70d6151780e9cc7e8227
Conflicts:
src/google/protobuf/compiler/javanano/javanano_message_field.cc
The option is only called 'generate_equals' because:
- equals() is the main thing; hashCode() is there only to
complement equals();
- it's shorter;
- toString() should not be included in this option because
it's more for debugging and it's more likely to stop
ProGuard from working well.
Also shortened the "has bit" expression; was
((bitField & mask) == mask), now ((bitField & mask) != 0).
Both the Java code and the bytecode are slightly shorter.
Change-Id: Ic309a08a60883bf454eb6612679aa99611620e76
Also pre-inlines set() and has() in serialization code. This could
theoretically help ProGuard: the message class size is usually large,
and because of this only, it may refuse to inline an accessor into
the serialization code, and as a result keeps the accessor intact.
Chances are, after pre-inlining all accessor calls within the message
class, those accessors become unused or single-use, so there are more
reasons for ProGuard to inline and then remove them.
Change-Id: I57decbe0b2533c1be21439de0aad15f49c7024dd
- Blank line after opening a message class (but not an enum interface).
- Let all code blocks insert blank lines before themselves. This applies to
'package' statement, all message classes, enum classes or constant groups,
extensions, bitfields, proto fields (one block per field; i.e. accessors
don't have blank lines among them), and basic MessageNano methods. In this
case we don't need to guess what the next block is and create blank lines
for it.
- Fixed some newline/indent errors.
- Only one SuppressWarnings("hiding") per file.
Change-Id: I865f52ad4fb6ea3b3a98b97ac9d78d19fc46c858
The public doc states that repeated fields are simply concatenated
and doesn't impose a different semantics for packed fields. This
CL fixes this for packed fields and adds tests covering all cases.
Also fixed a bit of missed null-repeated-field treatments.
Change-Id: Ie35277bb1a9f0b8171dc9d07b6adf9b9d3308de2
There's no distinction between a repeated field being null and being
empty. In both cases, nothing is sent on the wire. Clients might for
whatever reason inadvertently set a repeated field to null, so
protect against that and treat it just as if the field was empty.
Change-Id: Ic3846f7f2189d6cfff6f8ef3ca217daecc3c8be7
The field initializers have basically caused the compiled <init> method
to inline the whole clear() method, which means if ProGuard is not used
or failed to inline or remove clear(), there are two big chunks of code
that do the same thing. So why not just call clear() from the ctor.
Change-Id: Ief71e2b03db2e059b3bfa98309649368089ffab0
Previously it looked like this:
public final class OuterClass {
[...]
public static final class InnerClass extends
com.google.protobuf.nano.MessageNano {
[...]
public void setId(java.lang.String value) {
if (value == null) {
throw new java.lang.NullPointerException();
}
id_ = value;
bitfield0_ |= 0x00000001;
[...]
}
[...]
}
Now it looks like this:
public final class OuterClass {
[...]
public static final class InnerClass extends
com.google.protobuf.nano.MessageNano {
[...]
public void setId(java.lang.String value) {
if (value == null) throw new java.lang.NullPointerException();
id_ = value;
bitfield0_ |= 0x00000001;
[...]
}
[...]
}
Change-Id: I2a9289b528f785c846210d558206d677aa13e9be
This option generates fields as reference types, and serializes
based on nullness.
Change-Id: Ic32e0eebff59d14016cc9a19e15a9bb08ae0bba5
Signed-off-by: Brian Duff <bduff@google.com>
When parsing a group, the group's end tag should not be stored within the
message's unknownFieldData. Not only does this waste space, it is also output
the next time the group is serialized, resulting in two end tags for that group.
The resulting bytes are not always a valid protocol buffer and may fail to
parse.
This change ensures that group end tags do not result in an unknownFieldData
entry, and that messages with groups can be roundtripped without corruption.
Change-Id: I240f858a7217a7652b756598c34aacad5dcc3363
Conflicts:
java/src/test/java/com/google/protobuf/NanoTest.java
