Skip to content

Fix wkt dupes#8942

Merged
acozzette merged 2 commits intoprotocolbuffers:masterfrom
perezd:fix-wkt-dupes
Oct 21, 2021
Merged

Fix wkt dupes#8942
acozzette merged 2 commits intoprotocolbuffers:masterfrom
perezd:fix-wkt-dupes

Conversation

@perezd
Copy link
Copy Markdown
Contributor

@perezd perezd commented Aug 31, 2021

We already pre-compile the well known types into the runtimes so they shouldn't be re-compiled. #8925.

Derek Perez added 2 commits August 31, 2021 16:01
We already pre-compile the well known types into the runtimes so they shouldn't be re-compiled. protocolbuffers#8925
command_line = "--java_out=lite:$(OUT)",
runtime = ":lite",
visibility = ["//visibility:public"],
# keep this in sync w/ LITE_WELL_KNOWN_PROTO_MAP in //:BUILD
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

compiler and descriptor are missing here but present above.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

They aren't available in lite.

@acozzette
Copy link
Copy Markdown

I'm going to close this since it appears to be unnecessary now that #8925 has been resolved, but please let me know if I am mistaken.

@acozzette acozzette closed this Oct 12, 2021
@eikemeier
Copy link
Copy Markdown
Contributor

eikemeier commented Oct 13, 2021

I'm going to close this since it appears to be unnecessary now that #8925 has been resolved, but please let me know if I am mistaken.

@acozzette It's valid. It's a minor issue, since the well known types don't take up much space and are probably identical anyways, but you get duplicates which waste space and introduce uncertainty which version is used.

That said, it's a long standing bug and probably not easy to fix, so it's not high priority. But IMHO the issue shouldn't be closed.

eikemeier added a commit to eikemeier/proto-java that referenced this pull request Oct 13, 2021
protobuf
/
protobuf8942

Signed-off-by: Oliver Eikemeier <eikemeier@fillmore-labs.com>
@eikemeier
Copy link
Copy Markdown
Contributor

See also the demonstration in brach 8942 : https://github.com/eikemeier/proto-java/tree/8942

$ bazel run //:main    
...
Duplicate resource in .../bazel-out/darwin-fastbuild/bin/external/com_google_protobuf/libtimestamp_proto-speed.jar:
 -> com/google/protobuf/Timestamp$1.class
 -> com/google/protobuf/Timestamp$Builder.class
 -> com/google/protobuf/Timestamp.class
 -> com/google/protobuf/TimestampOrBuilder.class
 -> com/google/protobuf/TimestampProto.class
Duplicate resource in .../bazel-out/darwin-fastbuild/bin/external/com_google_protobuf/java/core/libcore.jar:
 -> com/google/protobuf/Timestamp$1.class
 -> com/google/protobuf/Timestamp$Builder.class
 -> com/google/protobuf/Timestamp.class
 -> com/google/protobuf/TimestampOrBuilder.class
 -> com/google/protobuf/TimestampProto.class
Check finished

@acozzette acozzette reopened this Oct 13, 2021
@acozzette
Copy link
Copy Markdown

@eikemeier Thanks for letting me know. Do you know if this pull request solves this problem? Based on the comments on #8925, it sounded like we were just waiting on a Bazel release.

@acozzette acozzette merged commit adc1f93 into protocolbuffers:master Oct 21, 2021
@eikemeier
Copy link
Copy Markdown
Contributor

@eikemeier Thanks for letting me know. Do you know if this pull request solves this problem? Based on the comments on #8925, it sounded like we were just waiting on a Bazel release.

The issue is still present in Bazel 5.0rc1, Protobuf 3.19.1, see #9192. Should I try the latest HEAD?

@perezd
Copy link
Copy Markdown
Contributor Author

perezd commented Nov 5, 2021 via email

@eikemeier
Copy link
Copy Markdown
Contributor

Try head as well.

With HEAD (+PR #9195) and Bazel 5.0rc1 I see no duplicates any more, see the latest version of https://github.com/eikemeier/proto-java2

Fine so far, is there any documentation that you're supposed to do

maven_install(
    artifacts = [
...
    ] + PROTOBUF_MAVEN_ARTIFACTS,
    override_targets = {
        "com.google.protobuf:protobuf-java": "@com_google_protobuf//:protobuf_java",
        "com.google.protobuf:protobuf-java-util": "@com_google_protobuf//:protobuf_java_util",
        "com.google.protobuf:protobuf-javalite": "@com_google_protobuf//:protobuf_javalite",
    },
...
)

or should I open a PR for that, too?

@perezd
Copy link
Copy Markdown
Contributor Author

perezd commented Nov 5, 2021

good catch, please do!

plobsing added a commit to plobsing/rules_proto that referenced this pull request Jul 25, 2023
The runtime already supplies pre-compiled implementations for these.

Compiling the well-known types is always redundant work, but this
duplication really becomes a problem when we resolve a proto runtime
from Maven that's newer than the `protoc` used by `rules_proto`. When
that happens, we get many compilation errors of the form

```
 error: name clash: clearExtension(GeneratedExtension<ExtensionRangeOptions,?>) in Builder and clearExtension(GeneratedExtension<ExtensionRangeOptions,T>) in ExtendableBuilder have the same erasure, yet neither overrides the other
      public <Type> Builder clearExtension(
                            ^
  where T,MessageT,BuilderT are type-variables:
    T extends Object declared in method <T>clearExtension(GeneratedExtension<MessageT,T>)
    MessageT extends ExtendableMessage<MessageT> declared in class ExtendableBuilder
    BuilderT extends ExtendableBuilder<MessageT,BuilderT> declared in class ExtendableBuilder
```

This particular error was produced by introducing
`build.buf:protovalidate:0.0.1` (depends on
`com.google.protobuf:protobuf-java:3.23.4`) to the Maven deps of a
workspace that uses `rules_proto-4.0.0-3.20.0`, as exhibited by the
repo contained in this
[gist](https://gist.github.com/plobsing/290f099fbc98e797013c990957f17f56)
when using an unpatched `rules_proto`.

I had expected that using a protoc version of 3.20.0 and a runtime
version of 3.23.4 should work based on the
[overall protobuf compatibility guarantee](https://protobuf.dev/support/cross-version-runtime-guarantee/)
and the [java-specific statement](https://github.com/protocolbuffers/protobuf/blob/main/java/README.md#compatibility-notice).
And, aside from this compilation issue, it does seem to.

This fix mirrors protocolbuffers/protobuf#8942
made to the protobuf repo.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants