default values and testing if a field is set in v3

[ekg]

Forgive the confusion, but how does one now check if a field is set in version 3?

[alfredkcp]

All field presence check (except for an embedded message) has been removed in proto3.

[ekg]

I see, so does this mean that we need to use semantics such that a non-embedded message field (such as int32) will be 0 when "unset"? Now the default values (empty string, 0 integer) can indicate non-presence or presence and that the value was set to the default.

[cfallin]

Yep, this was the design decision in proto3 -- primitive (non-message) fields are no longer nullable. It's better/more accurate to not even think of "unset"; i.e., a new message object already has every primitive field set. The intent, as I understand it, was to make a message more like a plain struct.

Nullable/optional fields can still be built by wrapping the primitives in submessages. There are predefined types for this in src/google/protobuf/wrappers.proto.

[daicoden]

I realize this is too late, but some feedback FWIW.

This makes APIs complicated for synchronizing objects between systems. You can't implement something like http PATCH semantics. Clients have to now read, echo, and detect true conflicts vs contention.

Given a contact with name & email, if client A wants to change the name, and client B wants to change the email, you have to push the smarts of dealing with concurrent updates to the clients for edge cases.

Before, you could simply look at the intent of the client by being able to tell what the client wanted changed to detect if there was a conflict. The responsibility of conflict handling in the client used to be simpler because it was only initiated when there was actually a conflict.

The workaround works... but is cumbersome if it applies to every field in your domain. If you want developers interacting through other data formats (like JSON) you need to write custom marshallers.

Again, FWIW - thanks!

[jhump]

BTW, @daicoden, you can still effectively test for absence of values (e.g. intent). The real thing missing from proto3 is that you can't define defaults. Nested messages always default to null, enums always default to the first value which must be assigned number 0 (so you could use a convention like ABSENT as the first value in every enum), strings are empty, numerics are zero, and bools are false.

If you need to distinguish absent strings/numbers/bools from ""/zero/false, there are "box" types included in proto's "well-known types". These can be checked if null to determine presence (basically just like using java.lang.Integer vs. primitive int in Java).

The downside is that box types are more verbose on the wire and likely to result in a little more garbage in Java (e.g. proto2's use of a field bitmask and primitives produces less garbage than using boxed types). Also, it's opt-in: you have to explicitly choose a box type vs. the primitive alternative when defining your messages.

[zackangelo]

@jhump I don't think the wrapper types get you there. PATCH semantics still require the ability to distinguish between a value that is absent and a value that is null (otherwise, how do you explicitly set a value to null?).

[daicoden]

@zackangelo good point. I guess you really need true tri-state fields for PATCH (omitted, clear, set). Proto2 had the null issue too, it only supported (omitted, set) because there is no null in proto.

I would be happy with (omitted, set) to allow partial document updates... I couldn't find a good way to do partial updates in proto3 aside from box-types which is really verbose in the code and makes others on the project cranky about using protos... but we will survive! ;) Thanks @jhump

[kskalski]

Yap, wrapper is of little help for PATCH/OVERRIDE semantics, since the problem just moves inside the nested message.
When you want to express request "set X to V" basically V cannot be a default value (receiver will not know if X wasn't set or was set to V==default).
With boxed values e.g. BoolValue(false) could be considered as "set X to false", however if you want to use MergeFrom it will still fail terribly, since BoolValue(false) merged on top of BoolValue(true) will still result in BoolValue(true)

In this scenario either default values chosen in proto3 should be excluded from domain of values ever used by your system, or as can easily be the case with e.g. "bool" you need to use some other type like:
enum NullableBool {
NullableBoolAbsent = 0; // unique name to not pollute namespace
True = 1;
False = 2;
}
which is acceptable, but makes user code more ugly (value() == True, value() == False, can have bugs when doing if (value()))

[adematte]

just stumbled on this old thread. It seems like the preferred way to handle PATH type requests is to use a field mask as provided by the well known types:
https://developers.google.com/protocol-buffers/docs/reference/google.protobuf#google.protobuf.FieldMask
Still would love a better alternative.

[house1909]

BTW, @daicoden, you can still effectively test for absence of values (e.g. intent). The real thing missing from proto3 is that you can't define defaults. Nested messages always default to null, enums always default to the first value which must be assigned number 0 (so you could use a convention like ABSENT as the first value in every enum), strings are empty, numerics are zero, and bools are false.
[...]

its not allowed to have the same value for Enums in one File

Protofile.proto:96:9: "NULLVALUE" is already defined in "<package/scope>".
Protofile.proto:96:9: Note that enum values use C++ scoping rules, meaning that enum values are siblings of their type, not children of it. Therefore, "NULLVALUE" must be unique within "<package/scope>", not just within "".