After updating to v0.7 with prost-types v0.10 "google.protobuf.rs" is not available

[g0hl1n]

Bug Report

Version

$ cargo tree | grep -e tonic -e prost
├── prost v0.10.0
│   └── prost-derive v0.10.0 (proc-macro)
├── prost-types v0.10.0
│   └── prost v0.10.0 (*)
└── tonic v0.7.1
    ├── prost v0.10.0 (*)
    ├── prost-derive v0.10.0 (proc-macro) (*)
├── tonic-build v0.7.0
│   ├── prost-build v0.10.0
│   │   ├── prost v0.10.0
│   │   │   └── prost-derive v0.10.0 (proc-macro) (*)
│   │   ├── prost-types v0.10.0
│   │   │   └── prost v0.10.0 (*)

Platform

Fedora release 35 (Thirty Five)

Linux 5.16.16-200.fc35.x86_64

Crates

• tonic-build

Description

After updating to tonic{,-build} v0.7 and prost{,-types} v0.10 cargo build fails with

error: couldn't read /.../target/debug/build/.../out/google.protobuf.rs: No such file or directory (os error 2)
 --> /.../target/debug/build/.../out/grpc_api_mod.rs:3:9
  |
3 |         include!("google.protobuf.rs");
  |         ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  |
  = note: this error originates in the macro `include` (in Nightly builds, run with -Z macro-backtrace for more info)

error: could not compile `...` due to previous error

build.rs calls:

fn gen_grpc_srv(include_file: &str, protos: Vec<String>) -> Result<(), Box<dyn std::error::Error>> {
    let proto_path = env::var("OUT_DIR").unwrap() + "/proto";
    tonic_build::configure()
        .build_server(true)
        .include_file(&include_file)
        .compile(&protos, &[proto_path])?;
    Ok(())
}

The google.protobuf.rs isn't available in the target directory:

$ find target -name google.protobuf.rs | wc -l
0

Using tonic-build's .compile_well_known_types(true) instead of prost-types works.

[LucioFranco]

Looks related to tokio-rs/prost@c577d3f

[LucioFranco]

@g0hl1n do you have the protobuf you're trying to build? I'd like to see if I can reproduce this bug.

[g0hl1n]

It seems it is related to include_file() of the Builder as building my proto with following build.rs works fine:

tonic_build::configure()
    .build_server(true)
    .compile(&["proto/test.proto"], &["proto"])?;

Where building with the following returns the described include error:

tonic_build::configure()
    .build_server(true)
    .include_file("gen_test.rs")
    .compile(&["proto/test.proto"], &["proto"])?;

The used proto is:

syntax = "proto3";

/*******************************************************************************
 * 1. Package */
package com.skidata.test.v1;

/*******************************************************************************
 * 2. Imports */
import "google/protobuf/empty.proto";

/*******************************************************************************
 * 3. File Options */

/*******************************************************************************
 * 4. service */

/* test service for https://github.com/hyperium/tonic/issues/964 */
service Test {
  /* test method */
  rpc GetTest(google.protobuf.Empty) returns (GetTestResponse);
}

/******************************************************************************
 * 5. resource "message" definitions */

/* Test application configuration */
message TestConfig {
}

/******************************************************************************
 * 6. request & response "message" definitions */

/* Test response */
message GetTestResponse {
  /* Test config */
  TestConfig conf = 1;
}

/******************************************************************************
 * 7. enum */

[g0hl1n]

@LucioFranco were you able to reproduce the problem or do you need more information from me?

[LucioFranco]

I have not had a chance to full debug it and its not exactly obvious to me. I would suggest as a work around for the moment I would swap out the use of Empty for your own message Empty {} that should solve the issue for now.

[g0hl1n]

Replacing Empty in an existing API isn't that easy. Furthermore it also affects other well-known-types like Duration and Wrappers. 😞

[LucioFranco]

You're totally right, let me try to take a look at this soon. Even though there is a work around I think we should solve this.

[g0hl1n]

fyi, i tried reverting tokio-rs/prost@c577d3f on prost's v0.10.1 tag for the prost and prost-types dependencies but it didn't change anything.

[g0hl1n]

@LucioFranco I've added a test for this problem in my repository and bisected it through the history (git bisect start v0.7.0 v0.6.2). This results in c78274e being identified as the first bad commit:

c78274e3fe5763cba291a605979cd7175ad6c38f is the first bad commit
commit c78274e3fe5763cba291a605979cd7175ad6c38f
Author: Jesse Szwedko <jesse@szwedko.me>
Date:   Wed Mar 30 10:43:06 2022 -0400

    feat: Update prost to 0.10 (#948)

 examples/Cargo.toml                         | 4 ++--
 interop/Cargo.toml                          | 4 ++--
 tests/ambiguous_methods/Cargo.toml          | 2 +-
 tests/compression/Cargo.toml                | 2 +-
 tests/extern_path/my_application/Cargo.toml | 4 ++--
 tests/extern_path/uuid/Cargo.toml           | 4 ++--
 tests/included_service/Cargo.toml           | 2 +-
 tests/integration_tests/Cargo.toml          | 2 +-
 tests/root-crate-path/Cargo.toml            | 2 +-
 tests/same_name/Cargo.toml                  | 2 +-
 tests/service_named_service/Cargo.toml      | 2 +-
 tests/stream_conflict/Cargo.toml            | 2 +-
 tests/wellknown-compiled/Cargo.toml         | 4 ++--
 tests/wellknown/Cargo.toml                  | 4 ++--
 tonic-build/Cargo.toml                      | 2 +-
 tonic-health/Cargo.toml                     | 2 +-
 tonic-reflection/Cargo.toml                 | 4 ++--
 tonic-types/Cargo.toml                      | 6 +++---
 tonic-web/tests/integration/Cargo.toml      | 2 +-
 tonic/Cargo.toml                            | 4 ++--
 20 files changed, 30 insertions(+), 30 deletions(-)

Nonetheless it must be somehow related to tonic as running the same test with prost/prost-types v0.9.0 on current master also fails.

[g0hl1n]

Thanks for the fix @LucioFranco!

When now adding the patch to my Cargo.toml it works 🥳

[patch.crates-io]
prost-build = { git = "https://github.com/tokio-rs/prost", rev = "b0cbe9c0a3bcb195e61fdfde32ee6cff9ced6de4" }

When do you expect a prost bugfix release (v0.10.2) to happen?

[LucioFranco]

Its out now 😄 thanks for the help debugging this