diff --git a/src/csharp/BUILD-INTEGRATION.md b/src/csharp/BUILD-INTEGRATION.md index 32024a8564b..237a63a430d 100644 --- a/src/csharp/BUILD-INTEGRATION.md +++ b/src/csharp/BUILD-INTEGRATION.md @@ -1,442 +1,499 @@ -Protocol Buffers/gRPC Codegen Integration Into .NET Build -================================================= +# Protocol Buffers/gRPC Codegen Integration Into .NET Build -With Grpc.Tools package version 1.17 we made it easier to compile .proto files -in your project using the `dotnet build` command, Visual Studio, or command-line -MSBuild. You need to configure the .csproj project according to the way you want -to integrate Protocol Buffer files into your build. +The [Grpc.Tools](https://www.nuget.org/packages/Grpc.Tools) NuGet package provides C# tooling support for generating C# code from `.proto` files in `.csproj` projects: +* It contains protocol buffers compiler and gRPC plugin to generate C# code. +* It can be used in building both grpc-dotnet projects and legacy c-core C# projects. -There is also a Reference section at the end of the file. +Using `Grpc.Tools` in `.csproj` files is described below. Other packages providing the runtime libraries for gRPC are described elsewhere. -Common scenarios ----------------- +## Getting Started -### I just want to compile .proto files into my library +The package [Grpc.Tools](https://www.nuget.org/packages/Grpc.Tools) is used automatically to generate the C# code for protocol buffer messages and gRPC service stubs from +`.proto` files. These files: +* are generated on an as-needed basis each time the project is built. +* aren't added to the project or checked into source control. +* are a build artifact usually contained in the obj directory. -This is the approach taken by the examples in the `csharp/examples` directory. -Protoc output files (for example, `Helloworld.cs` and `HelloworldGrpc.cs` -compiled from `helloworld.proto`) are placed among *object* and other temporary -files of your project, and automatically provided as inputs to the C# compiler. -As with other automatically generated .cs files, they are included in the source -and symbols NuGet package, if you build one. +This package is optional. You may instead choose to generate the C# source files from +`.proto` files by running the `protoc` compiler manually or from a script. +However this package helps to simplify generating the C# source files by +integrating the code generation into the build process. It can be used when building both +the server and client projects, and by both c-core C# projects and grpc-dotnet projects: +* The `Grpc.AspNetCore` metapackage already includes a reference to `Grpc.Tools`. +* gRPC for .NET client projects and projects using `Grpc.Core` need to reference `Grpc.Tools` explicity if you want code generation for those projects -Simply reference your .proto files in a `` item group. The following -example will add all .proto files in a project and all its subdirectories -(excluding special directories such as `bin` and `obj`): +`Grpc.Tools` is only used at build-time and has no runtime components. +It should be marked with `PrivateAssets="All"` to prevent it from being included at runtime, e.g. +```xml + + runtime; build; native; contentfiles; analyzers; buildtransitive + all + +``` + +Support is provided for the following platforms: +* Windows (x86, x64, and arm64 via using the x86 binaries) +* MacOS (x64 and arm64 via using the x64 binaries) +* Linux (x86, x64, and arm64) + +## Adding `.proto` files to a project + +To add `.proto` files to a project edit the project’s `.csproj` file and add an item group with a `` element that refers to the `.proto` file, e.g. ```xml - - - + + + ``` -You must add a reference to the NuGet packages Grpc.Tools and Grpc (the latter -is a meta-package, in turn referencing Grpc.Core and Google.Protobuf packages). -It is **very important** to mark Grpc.Tools as a development-only dependency, so -that the *users* of your library do not fetch the tools package: +Wildcards can be used to select several `.proto` files, e.g. -* "dotnet SDK" .csproj (Visual Studio, `dotnet new`): Add an attribute - `PrivateAssets="All"` to the Grpc.Tools package reference. See an example in the - [Greeter.csproj](../../examples/csharp/Helloworld/Greeter/Greeter.csproj#L10) - example project in this repository. If adding a package reference in Visual - Studio, edit the project file and add this attribute. [This is a bug in NuGet - client](https://github.com/NuGet/Home/issues/4125). +```xml + + + +``` -* "Classic" .csproj with `packages.config` (Visual Studio before 2017, old versions of Mono): - This is handled automatically by NuGet after you set the `developmentDependency="true"` - attribute on the `` tag in `packages.config`. +By default, a `` reference generates gRPC client and a service base class from the `service` definitions in the `.proto` files. +The `GrpcServices` attribute can be used to limit C# asset generation. See the reference section below for all +options. E.g. to only generate client code: -If building a NuGet package from your library with the nuget command line tool -from a .nuspec file, then the spec file may (and probably should) reference the -Grpc metapackage, but **do not add a reference to Grpc.Tools** to it. "dotnet SDK" -projects handle this automatically when called from `dotnet pack` by excluding -any packages with private assets, such as thus marked Grpc.Tools. +```xml + + + +``` -#### Per-file options that can be set in Visual Studio +For `.proto` files that are outside of the project directory a link can be added so that the files are visible in Visual Studio. E.g. -For a "dotnet SDK" project, you have more control of some frequently used options. -**You may need to open and close Visual Studio** for this form to appear in the -properties window after adding a reference to Grpc.Tools package (we do not know -whether this is a bug or by design, but it looks like a bug): +```xml + + + + + +``` -![Properties in an SDK project](doc/integration.md-fig.2-sdk.png) +For more examples see the example project files in GitHub: https://github.com/grpc/grpc-dotnet/tree/master/examples -You can also change options of multiple files at once by selecting them in the -Project Explorer together. +# Reference -For a "classic" project, you can only add .proto files with all options set to -default (if you find it necessary to modify these options, then hand-edit the -.csproj file). Click on the "show all files" button, add files to project, then -change file type of the .proto files to "Protobuf" in the Properties window -drop-down. This menu item will appear after you import the Grpc.Tools package: +## Protobuf item metadata reference -![Properties in a classic project](doc/integration.md-fig.1-classic.png) +The following metadata are recognized on the `` items. -See the Reference section at end of this file for options that can be set -per-file by modifying the source .csproj directly. +| Name | Default | Value | Synopsis | +|----------------|-----------|----------------------|----------------------------------| +| Access | `public` | `public`, `internal` | Generated class access | +| AdditionalProtocArguments | | arbitrary cmdline arguments | Extra command line flags passed to `protoc` command. To specify multiple arguments use semi-colons (;) to separate them. See example below | +| ProtoCompile | `true` | `true`, `false` | If `false`, don't invoke `protoc` to generate code. | +| ProtoRoot | See notes | A directory | Common root for set of files | +| CompileOutputs | `true` | `true`, `false` | If `false`, C# code will be generated, but it won't be included in the C# build. | +| OutputDir | See notes | A directory | Directory for generated C# files with protobuf messages | +| OutputOptions | | arbitrary options | Extra options passed to C# codegen as `--csharp_opt=opt1,opt2` | +| GrpcOutputDir | See notes | A directory | Directory for generated gRPC stubs | +| GrpcOutputOptions | | arbitrary options | Extra options passed to gRPC codegen as `--grpc_opt=opt1,opt2` | +| GrpcServices | `Both` | `None`, `Client`, `Server`, `Both` | Generated gRPC stubs | +| AdditionalImportDirs | See notes | Directories | Specify additional directories in which to search for imports `.proto` files | -#### My .proto files are in a directory outside the project +__Notes__ -Refer to the example files -[RouteGuide.csproj](../../examples/csharp/RouteGuide/RouteGuide/RouteGuide.csproj#L58-L60) -and [Greeter.csproj](../../examples/csharp/Helloworld/Greeter/Greeter.csproj#L11) -in this repository. For the files to show up in Visual Studio properly, add a -`Link` attribute with just a filename to the `` item. This will be the -display name of the file. In the `Include` attribute, specify the complete path -to file. A relative path is based off the project directory. +* __ProtoRoot__ +For files _inside_ the project directory or its subdirectories, `ProtoRoot` is set by default to the +project directory. -Or, if using Visual Studio, add files _as links_ from outside directory. In the -Add Files dialog, there is a little [down arrow near the Open -button](https://stackoverflow.com/a/9770061). Click on it, and choose "Add as -link". If you do not select this option, Visual Studio will copy files to the -project directory instead. + For files _outside_ of the project directory, the value +is set to the file's containing directory name, individually per file. If you +include a subtree of `.proto` files that lies outside of the project directory, you +need to set `ProtoRoot`. There is an example of this below. The path in +this variable is relative to the project directory. -Alternatively, you can also specify `AdditionalImportDirs` and provide a list of directories to search for imported .proto files. The directories are searched in the order given. +* __OutputDir__ +The default value is the value of the property +`Protobuf_OutputPath`. This property, in turn, unless you set it in your +project, will be set to the value of the standard MSBuild property +`IntermediateOutputPath`, which points to the location of compilation object +outputs, such as `"obj/Release/netstandard1.5/"`. The path in this property is +considered relative to the project directory. -Eg. +* __GrpcOutputDir__ +Unless explicitly set, will follow `OutputDir` for any given file. -```xml - -``` +* __Access__ +Sets generated class access on _both_ generated message and gRPC stub classes. -#### My .proto files have same filename in different folders +* __AdditionalProtocArguments__ +Pass additional commandline arguments to the `protoc` command being invoked. +Normally this option should not be used, but it exists for scenarios when you need to pass +otherwise unsupported (e.g. experimental) flags to protocol buffer compiler. -Starting from Grpc.Tools version 2.31, protocol buffers compilation preserves original folder structure for generated files. Eg. +* __OutputOptions__ +Pass additional C# code generation options to `protoc` in the form `--csharp_opt=opt1,opt2`. See [C#-specific options](https://protobuf.dev/reference/csharp/csharp-generated/#compiler_options) for possible values. -- `../ProjectFolder/Protos/v2/http.proto` -- `../ProjectFolder/Protos/v3/http.proto` +* __GrpcOutputOptions__ +Pass additional options to the `grpc_csharp_plugin` in form of the `--grpc_opt` flag. +Normally this option should not be used as its values are already controlled by `Access` +and `GrpcServices` metadata, but it might be useful in situations where you want +to explicitly pass some otherwise unsupported (e.g. experimental) options to the +`grpc_csharp_plugin`. -Will result in: +* __AdditionalImportDirs__ +Specify additional directories in which to search for imports in `.proto` files. The directories are searched in the order given. You may specify directories _outside_ of the +project directory. The directories are passed to the `protoc` code generator via the `-I/--proto_path` option +together with `Protobuf_StandardImportsPath` and `ProtoRoot` directories. -- `../ProjectFolder/obj/CONFIGURATION/FRAMEWORK/Protos/v2/Greet.cs` -- `../ProjectFolder/obj/CONFIGURATION/FRAMEWORK/Protos/v2/GreetGrpc.cs` -- `../ProjectFolder/obj/CONFIGURATION/FRAMEWORK/Protos/v3/Greet.cs` -- `../ProjectFolder/obj/CONFIGURATION/FRAMEWORK/Protos/v3/GreetGrpc.cs` +__Specifying multiple values in properties__ -This feature resolves problems we have faced in large projects. Moreover, There is now also a project-wide new option Protobuf_ProtoRoot to define the fallback ProtoRoot. If the ProtoRoot is set, this also reduces the amount of problems that lead to duplicates. Eg. +Some properties allow you to specify multiple values in a list. The items in a list need to +be separated by semi-colons (;). This is the syntax that MsBuild uses for lists. -```xml - - - -``` +The properties that can have lists of items are: `OutputOptions`, `AdditionalProtocArguments`, + `GrpcOutputOptions`, `AdditionalImportDirs` -Before Grpc.Tools version 2.31 all .proto files were compiled into `obj` directory, flattening relative paths. For proto files with duplicated names it cause following errors `NETSDK1022 Duplicate 'Compile' items were included. [...]` or `MSB3105 [...] Duplicate items are not supported by the "Sources" parameter`. The workaround for this problem was introducing relative paths in your `obj` folder, by manipulating output path. Eg. +Example: to specify two additional arguments: `--plugin=protoc-gen-myplugin=D:\myplugin.exe --myplugin_out=.` ```xml - - + ``` -__Note, this was a workaround approach, we recommend updating Grpc.Tools to the latest version.__ +## `grpc_csharp_plugin` command line options -### I just want to generate proto and gRPC C# sources from my .proto files (no C# compile) +Under the hood, the `Grpc.Tools` build integration invokes the `protoc` and `grpc_csharp_plugin` binaries +to perform code generation. Here is an overview of the available `grpc_csharp_plugin` options: -Suppose you want to place generated files right beside each respective source -.proto file. Create a .csproj library file in the common root of your .proto -tree, and add a reference to Grpc.Tools package (this works in Windows too, `$` -below stands for a command prompt in either platform): +| Name | Default | Synopsis | +|---------------- |-----------|----------------------------------------------------------| +| no_client | off | Don't generate the client stub | +| no_server | off | Don't generate the server-side stub | +| internal_access | off | Generate classes with "internal" visibility | +Note that the protocol buffer compiler has a special commandline syntax for plugin options. +Example: ``` -/myproject/myprotofiles$ dotnet new classlib - . . . - Restoring packages for /myproject/myprotofiles/myprotofiles.csproj... - . . . -/myproject/myprotofiles$ rm *.cs <-- remove all *.cs files from template; -C:\myproject\myprotofiles> del *.cs /y <-- on Windows, use the del command instead. -/myproject/myprotofiles$ dotnet add package Grpc.Tools +protoc --plugin=protoc-gen-grpc=grpc_csharp_plugin --csharp_out=OUT_DIR \ + --grpc_out=OUT_DIR --grpc_opt=lite_client,no_server \ + -I INCLUDE_DIR foo.proto ``` - -(the latter command also accepts an optional `--version X.Y` switch for a -specific version of package, should you need one). Next open the generated -.csproj file in a text editor. - -Since you are not building a package, you may not worry about adding -`PrivateAssets="All"` attribute, but it will not hurt, in case you are -repurposing the project at some time later. The important part is (1) tell the -gRPC tools to select the whole directory of files; (2) order placement of each -output besides its source, and (3) not compile the generated .cs files. Add the -following stanza under the `` xml node: - -```xml - - - +## Environment Variables + +Environment variables can be set to change the behavior of `Grpc.Tools` - setting the CPU architecture or operating system, or using custom built protocol buffers compiler and gRPC plugin. + +| Name | Synopsis | +|---------------------|-------------------------------------------------------------------------------| +|`PROTOBUF_TOOLS_OS` | Operating system version of the tools to use: `linux`, `macosx`, or `windows` | +|`PROTOBUF_TOOLS_CPU` | CPU architecture version of the tools to use: `x86`, `x64`, or `arm64` | +|`PROTOBUF_PROTOC` | Full path to the protocol buffers compiler | +|`GRPC_PROTOC_PLUGIN` | Full path to the grpc_csharp_plugin | + +For example, to use a custom built protoc compiler and grpc_csharp_plugin: +```bash +export PROTOBUF_PROTOC=$my_custom_build/protoc +export GRPC_PROTOC_PLUGIN=$my_custom_build/grpc_csharp_plugin +dotnet build myproject.csproj ``` -The `Include` tells the build system to recursively examine project directory -and its subdirectories (`**`) include all files matching the wildcard `*.proto`. -You can instead selectively include your files or selectively exclude files from -the glob pattern; [MSBuild documentation explains -that](https://docs.microsoft.com/visualstudio/msbuild/msbuild-items). The -`OutputDir="%(RelativeDir)"` orders the output directory for each .cs file be -same as the corresponding .proto directory. Finally, `CompileOutputs="false"` -prevents compiling the generated files into an assembly. +## MSBuild Properties -Note that an empty assembly is still generated, but you should ignore it. As -with any build system, it is used to detect out-of-date dependencies and -recompile them. +You can set some Properties in your project file or on the MSBuild command line. The +following properties change the behavior of `Grpc.Tools`: -#### I am getting a warning about a missing expected file! +| Name | Synopsis | +|---------------------|-------------------------------------------------------------------------------| +| `Protobuf_ToolsOs` | Same as `PROTOBUF_TOOLS_OS` environment variable | +| `Protobuf_ToolsCpu` | Same as `PROTOBUF_TOOLS_CPU` environment variable | +| `Protobuf_ProtocFullPath` | Same as `PROTOBUF_PROTOC` environment variable | +| `gRPC_PluginFullPath` | Same as `GRPC_PROTOC_PLUGIN` environment variable | +| `Protobuf_NoWarnMissingExpected` | Default: `false`. If `true` then no warnings are given if expected files not generated. See example below for an explanation. | +| `Protobuf_OutputPath`| Default: `IntermediateOutputPath` - ususally the `obj` directory. Sets the default value for `OutputDir` on `` items.| +| `EnableDefaultProtobufItems` | Default: `false`. If `true` then `.proto` files under the project are automatically included without the need to specify any `` items. | +| `Protobuf_StandardImportsPath` | The path for protobuf's [well known types](https://protobuf.dev/reference/protobuf/google.protobuf/) included in the NuGet package. It is automcatically passed to `protoc` via the `-I/--proto_path` option. | -When we are preparing compile, there is no way to know whether a given proto -file will produce a *Grpc.cs output or not. If the proto file has a `service` -clause, it will; otherwise, it won't, but the build script cannot know that in -advance. When we are treating generated .cs files as temporary, this is ok, but -when generating them for you, creating empty files is probably not. You need to -tell the compiler which files should be compiled with gRPC services, and which -only contain protobuffer message definitions. +# Scenarios and Examples -One option is just ignore the warning. Another is quench it by setting the -property `Protobuf_NoWarnMissingExpected` to `true`: +For other examples see also the `.csproj` files in the examples in GitHub: +* [grpc-dotnet examples](https://github.com/grpc/grpc-dotnet/tree/master/examples) +* [`Grpc.Core` examples](https://github.com/grpc/grpc/tree/v1.46.x/examples/csharp) -```xml - - true - -``` +Quick links to the examples below: -For a small to medium projects this is sufficient. But because of a missing -output dependency, the corresponding .proto file will be recompiled on every -build. If your project is large, or if other large builds depend on generated -files, and are also needlessly recompiled, you'll want to prevent these rebuilds -when files have not in fact changed, as follows: +* [ProtoRoot - Common root for one or more `.proto` files](#ProtoRoot) +* [AdditionalImportDirs - Setting location of imported `.proto` files](#AdditionalImportDirs) +* [GrpcServices - Generating gRPC services and protocol buffers messages](#grpcservices) +* [Automatically including `.proto` files](#autoinclude) +* [Generate proto and gRPC C# sources from .proto files (no C# compile)](#nocompile) +* [Visual Studio: setting per-file `.proto` file options](#visualstudio) +* [Bypassing Grpc.Tools to run the protocol buffers compiler explicitly](#compiler) -##### Explicitly tell protoc for which files it should use the gRPC plugin +--- +## ProtoRoot - Common root for one or more `.proto` files -You need to set the `Protobuf` item property `GrpcServices` to `None` for those -.proto inputs which do not have a `service` declared (or, optionally, those -which do but you do not want a service/client stub for). The default value for -the `GrpcServices` is `Both` (both client and server stub are generated). This -is easy enough to do with glob patterns if your files are laid out in -directories according to their service use, for example: +`ProtoRoot` specifies a common directory that is an ancestor for a set of `.proto` files. -```xml - - - - -``` +It has two purposes: +* working out relative directories to preserve the structure when generating `.cs` files +* adding a directory to be searched for imported `.proto` files -In this sample, all .proto files are compiled with `GrpcServices="None"`, except -for .proto files in subdirectories on any tree level named `hello/` and `bye`, -which will take `GrpcServices="Both"` Note the use of the `Update` attribute -instead of `Include`. If you write `Include` by mistake, the files will be added -to compile *twice*, once with, and once without GrpcServices. Pay attention not -to do that! +These are explained in an example below. -Another example would be the use of globbing if your service .proto files are -named according to a pattern, for example `*_services.proto`. In this case, The -`Update` attribute can be written as `Update="**/*_service.proto"`, to set the -attribute `GrpcServices="Both"` only on these files. +For `.proto` files under the project directory `ProtoRoot` is by default set to `.`. +It can also be explicitly set. -But what if no patterns work, and you cannot sort a large set of .proto file -into those containing a service and those not? As a last resort, +For `.proto` files outside of the project the value is set to the file's containing directory name. +If you include a subtree of `.proto` files then you must set `ProtoRoot` to give the +parent of the directory tree. -##### Force creating empty .cs files for missing outputs. +In either case if you are importing a `.proto` file from within another file then you should set +`ProtoRoot` so that the import paths can be found. (See also `AdditionalImportDirs` below.) -Naturally, this results in a dirtier compiler output tree, but you may clean it -using other ways (for example, by not copying zero-length .cs files to their -final destination). Remember, though, that the files are still important to keep -in their output locations to prevent needless recompilation. You may force -generating empty files by setting the property `Protobuf_TouchMissingExpected` -to `true`: +Generated files in the output directory will have the same directory structure as the +`.proto` files under `ProtoRoot`. +By default the output directory for generated files is `obj\CONFIGURATION\FRAMEWORK\` (e.g. `obj\Debug\net6.0\`) unless `OutputDir` or `GrpcOutputDir` are specified. +### Example use of `ProtoRoot` +Specifying: ```xml - - true - + + + ``` +for files: +``` + ProjectFolder\Protos\Services\v1\hello.proto + ProjectFolder\Protos\Services\v2\hello.proto + ProjectFolder\Protos\Messages\v1\message.proto + ..\OutsideProjectProtos\MyApi\alpha.proto + ..\OutsideProjectProtos\MyApi\beta.proto +``` +will generate files: +``` + ProjectFolder\obj\Debug\net6.0\Services\v1\Hello.cs + ProjectFolder\obj\Debug\net6.0\Services\v1\HelloGrpc.cs + ProjectFolder\obj\Debug\net6.0\Services\v2\Hello.cs + ProjectFolder\obj\Debug\net6.0\Services\v2\HelloGrpc.cs + ProjectFolder\obj\Debug\net6.0\Messages\v1\Message.cs + ProjectFolder\obj\Debug\net6.0\MyApi\Alpha.cs + ProjectFolder\obj\Debug\net6.0\MyApi\AlphaGrpc.cs + ProjectFolder\obj\Debug\net6.0\MyApi\Beta.cs + ProjectFolder\obj\Debug\net6.0\MyApi\BetaGrpc.cs -#### But I do not use gRPC at all, I need only protobuffer messages compiled - -Set `GrpcServices="None"` on all proto files: - -```xml - - - ``` +Things to notes: +* the directory structures under `ProjectFolder\Protos\` and `..\OutsideProjectProtos\` are mirrored in the output directory. +* the import search paths passed to `protoc` via `-I/--proto_path` option will include + `ProjectFolder\Protos` and `..\OutsideProjectProtos` +--- +## AdditionalImportDirs - Setting location of imported `.proto` files -#### That's good so far, but I do not want the `bin` and `obj` directories in my tree +In addition to specifying `ProtoRoot` other import directories can be specified for +directories to search when importing `.proto` files by specifying `AdditionalImportDirs` +and provide a list of directories. The directories are searched in the order given. -You may create the project in a subdirectory of the root of your files, such as, -for example, `.build`. In this case, you want to refer to the proto files -relative to that `.build/` directory as +You would use this when you want to import `.proto` files that you don't need to +separately compile as they are only used in import statements. E.g.: ```xml - - - + ``` -Pay attention to the `ProtoRoot` property. It needs to be set to the directory -where `import` declarations in the .proto files are looking for files, since the -project root is no longer the same as the proto root. - -Alternatively, you may place the project in a directory *above* your proto root, -and refer to the files with a subdirectory name: +Note: The path for protobuf's [well known types](https://protobuf.dev/reference/protobuf/google.protobuf/) is automatically included. E.g. the `import` below will work without having to explicity specifying the path in `AdditionalImportDirs`: +```protobuf +import "google/protobuf/wrappers.proto"; +``` +--- +## GrpcServices - Generating gRPC services and protocol buffers messages +The protocol buffers files (`.proto` files) define both the service interface and the +structure of the payload messages. + +Two `.cs` file can be generated from a `.proto` file. For example, if the `.proto` file +is `myfile.proto` then the two possible files are: +* `Myfile.cs` - contains the generated code for protocol buffers messages +* `MyfileGrpc.cs` - contains the generated code for gRPC client and/or server + +When a `.proto` file contains service definitions the protocol buffers compiler calls +the gRPC plugin to generate gRPC client and/or server stub code. Whether or not the `*Grpc.cs` file +is generated and what it contains is controlled by the `GrpcServices` metadata +on the `` item. + +* `GrpcServices="Both"` (the default) - `Myfile.cs` and `MyfileGrpc.cs` generated +* `GrpcServices="None"` - just `Myfile.cs` generated +* `GrpcServices="Client"` - `Myfile.cs` and `MyfileGrpc.cs` (just client code) +* `GrpcServices="Server"` - `Myfile.cs` and `MyfileGrpc.cs` (just server code) + +However when a `.proto` **does not file contains any service definitions** but only contains +message definitions then an empty (zero length) `MyfileGrpc.cs` may still be created +by `Grpc.Tools` unless the `.proto` file is specified with `GrpcServices="None"` in the project file. + +This is because `Grpc.Tools` has no way of knowing in advanced of running the protocol buffers +compiler whether a `.proto` file has a service clause. It creates the empty files as a marker +for incremental builds so that the `.proto` files are not unnecessarily recompiled. Empty files +are not a problem on a small project but you may wish to avoid them on a larger project. + +Therefore it is better to explicitly mark files with the correct `GrpcServices` metadata if you can. For +example: ```xml - - - + + + + +``` +In the above example all `.proto` files are compiled with `GrpcServices="None"`, except for `.proto` +files in subdirectories on any tree level named `hello` and `bye`, which will take +`GrpcServices="Both"`. Note the use of the `Update` attribute instead of `Include` - otherwise +the files would be added twice. + +Another example would be the use of globbing if your service `.proto` files are named according +to a pattern, for example `*_services.proto`. In this case the `Update` attribute can be written +as `Update="**/*_service.proto"` to set the attribute `GrpcServices="Both"` only on these files. + +### Seeing a warning about a missing expected file + +You will see the warning message: + ``` + Some expected protoc outputs were not generated + ``` + if all these are true: + * the location for the generated files is configured to a directory outside of the project, e.g. `OutputDir="..\outside-project\"` + * `*Grpc.cs` files have not been created because the `.proto` file does not contain a +service definintion +* you have not specified `GrpcServices="None"` + +This is because `Grpc.Tools` only creates empty `*Grpc.cs` files in directories +within the project (such as the intermediate `obj` directory). Empty files are not created +outside of the project directory so as not to pollute non-project directories. + +This warning can be suppressed by setting the MSBuild property: +```xml + + true + ``` +however it is better to set `GrpcServices="None"` on the `.proto` files affected to avoid +unnecessary rebuilds. -### Alas, this all is nice, but my scenario is more complex, -OR- -### I'll investigate that when I have time. I just want to run protoc as I did before. -One option is examine our [.targets and .props files](Grpc.Tools/build/) and see -if you can create your own build sequence from the provided targets so that it -fits your needs. Also please open an issue (and tag @kkm000 in it!) with your -scenario. We'll try to support it if it appears general enough. +--- +## Automatically including `.proto` files -But if you just want to run `protoc` using MsBuild `` task, as you -probably did before the version 1.17 of Grpc.Tools, we have a few build -variables that point to resolved names of tools and common protoc imports. -You'll have to roll your own dependency checking (or go with a full -recompilation each time, if that works for you), but at the very least each -version of the Tools package will point to the correct location of the files, -and resolve the compiler and plugin executables appropriate for the host system. -These property variables are: +For SDK projects it is possible to automatically include `.proto` files found in the project +directory or sub-directories, without having to specify them with a `` item. +To do this the property `EnableDefaultProtobufItems` has be set to `true` in the project file or on the MSBuild command line. -* `Protobuf_ProtocFullPath` points to the full path and filename of protoc executable, e. g., - "C:\Users\kkm\.nuget\packages\grpc.tools\1.17.0\build\native\bin\windows\protoc.exe". +It is recommended that you do not rely on automatic inclusion of `.proto` files except for +the simplest of projects since it does not allow you to control other settings such as +`GrpcServices`. -* `gRPC_PluginFullPath` points to the full path and filename of gRPC plugin, such as - "C:\Users\kkm\.nuget\packages\grpc.tools\1.17.0\build\native\bin\windows\grpc_csharp_plugin.exe" +By default `EnableDefaultProtobufItems` is not set and `` items must be included +in the project for the `.proto` files to be compiled. -* `Protobuf_StandardImportsPath` points to the standard proto import directory, for example, - "C:\Users\kkm\.nuget\packages\grpc.tools\1.17.0\build\native\include". This is - the directory where a declaration such as `import "google/protobuf/wrappers.proto";` - in a proto file would find its target. +--- -Use MSBuild property expansion syntax `$(VariableName)` in your protoc command -line to substitute these variables, for instance, +## Generate proto and gRPC C# sources from .proto files (no C# compile) -```xml - - - $(Protobuf_ProtocFullPath) --plugin=protoc-gen-grpc=$(gRPC_PluginFullPath) -I $(Protobuf_StandardImportsPath) ....rest of your command.... - - - - +If you just want to generate the C# sources from `.proto` files without compiling the C# files + (e.g. for use in other projects) then you can do something similar to this to a `.csproj` file: + ```xml + + + ``` -Also make sure *not* to include any file names to the `Protobuf` item -collection, otherwise they will be compiled by default. If, by any chance, you -used that name for your build scripting, you must rename it. - -### What about C++ projects? +* `Include` tells the build system to recursively examine project directory and its +subdirectories (**) include all files matching the wildcard `*.proto`. +* `OutputDir="%(RelativeDir)"` makes the output directory for each `.cs` file to be +same as the corresponding `.proto` directory. +* `CompileOutputs="false"` prevents compiling the generated files into an assembly. -This is in the works. Currently, the same variables as above are set to point to -the protoc binary, C++ gRPC plugin and the standard imports, but nothing else. -Do not use the `Protobuf` item collection name so that your project remains -future-proof. We'll use it for C++ projects too. +Note that an empty assembly is still generated which can be ignored. -Reference ---------- +NOTE: To start with an empty project to add your `.proto` files to you can do the following +at a command prompt: +```bash +dotnet new classlib +rm *.cs # remove .cs files - for Windows the command is: del *.cs /y +dotnet add package Grpc.Tools +``` -### Protobuf item metadata reference +--- +## Visual Studio: setting per-file `.proto` file options -The following metadata are recognized on the `` items. +In Visual Studio it is possible to set some frequently used per-file options on `.proto` files +without editing the `.csproj` file directly. However editing the `.csproj` gives you more +flexibilty. -| Name | Default | Value | Synopsis | -|----------------|-----------|----------------------|----------------------------------| -| Access | `public` | `public`, `internal` | Generated class access | -| AdditionalProtocArguments | | arbitrary cmdline arguments | Extra command line flags passed to `protoc` command. To specify multiple arguments use semi-colons (;) to separate them. See example below | -| ProtoCompile | `true` | `true`, `false` | Pass files to protoc? | -| ProtoRoot | See notes | A directory | Common root for set of files | -| CompileOutputs | `true` | `true`, `false` | C#-compile generated files? | -| OutputDir | See notes | A directory | Directory for generated C# files with protobuf messages | -| OutputOptions | | arbitrary options | Extra options passed to C# codegen as `--csharp_opt=opt1,opt2` | -| GrpcOutputDir | See notes | A directory | Directory for generated gRPC stubs | -| GrpcOutputOptions | | arbitrary options | Extra options passed to gRPC codegen as `--grpc_opt=opt1,opt2` | -| GrpcServices | `both` | `none`, `client`, `server`, `both` | Generated gRPC stubs | -| AdditionalImportDirs | See notes | Directories | Specify additional directories in which to search for imports .proto files | +### "dotnet SDK" projects -__Notes__ +For a "dotnet SDK" project, you have more control of some frequently used options. +**You may need to open and close Visual Studio** for this form to appear in the +properties window after adding a reference to `Grpc.Tools` package: -* __ProtoRoot__ -For files _inside_ the project cone, `ProtoRoot` is set by default to the -project directory. For every file _outside_ of the project directory, the value -is set to this file's containing directory name, individually per file. If you -include a subtree of proto files that lies outside of the project directory, you -need to set this metadatum. There is an example in this file above. The path in -this variable is relative to the project directory. +![Properties in an SDK project](doc/integration.md-fig.2-sdk.png) -* __OutputDir__ -The default value for this metadatum is the value of the property -`Protobuf_OutputPath`. This property, in turn, unless you set it in your -project, will be set to the value of the standard MSBuild property -`IntermediateOutputPath`, which points to the location of compilation object -outputs, such as "obj/Release/netstandard1.5/". The path in this property is -considered relative to the project directory. +You can also change options of multiple files at once by selecting them in the +Project Explorer together. -* __GrpcOutputDir__ -Unless explicitly set, will follow `OutputDir` for any given file. +### "classic" projects -* __Access__ -Sets generated class access on _both_ generated message and gRPC stub classes. +For a "classic" project, you can only add `.proto` files with all options set to +default. Click on the "show all files" button, add files to project, then +change file type of the `.proto` files to "Protobuf" in the Properties window +drop-down. This menu item will appear after you import the `Grpc.Tools` package: -* __AdditionalProtocArguments__ -Pass additional commandline arguments to the `protoc` command being invoked. -Normally this option should not be used, but it exists for scenarios when you need to pass -otherwise unsupported (e.g. experimental) flags to protocol buffer compiler. +![Properties in a classic project](doc/integration.md-fig.1-classic.png) -* __GrpcOutputOptions__ -Pass additional options to the `grpc_csharp_plugin` in form of the `--grpc_opt` flag. -Normally this option should not be used as it's values are already controlled by "Access" -and "GrpcServices" metadata, but it might be useful in situations where you want -to explicitly pass some otherwise unsupported (e.g. experimental) options to the -`grpc_csharp_plugin`. +--- +## Bypassing Grpc.Tools to run the protocol buffers compiler explicitly -* __AdditionalImportDirs__ -Specify additional directories in which to search for imports in .proto files. The directories are searched in the order given. You may specify directories _outside_ of the -project directory. The directories are passed to the `protoc` code generator via the `-I/--proto_path` option -together with `Protobuf_StandardImportsPath` and `ProtoRoot` directories. +It is possible to bypass all the build logic in `Grpc.Tools` and run the protocol buffers compiler +explicitly in your project file, and just use the `Grpc.Tools` as a means of getting the compiler. +**This is not recommended** but there may be situations where you want to do this. -__Specifying multiple values in properties__ +You can use the following Properties: +* `Protobuf_ProtocFullPath` points to the full path and filename of protoc executable, e.g. + `"...\.nuget\packages\grpc.tools\2.51.0\build\native\bin\windows\protoc.exe"` -Some properties allow you to specify multiple values in a list. The items in a list need to -be separated by semi-colons (;). This is the syntax that MsBuild uses for lists. +* `gRPC_PluginFullPath` points to the full path and filename of gRPC plugin, e.g. + `"...\.nuget\packages\grpc.tools\2.51.0\build\native\bin\windows\grpc_csharp_plugin.exe"` -The properties that can have lists of items are: __OutputOptions__, __AdditionalProtocArguments__, __GrpcOutputOptions__, __AdditionalImportDirs__ +* `Protobuf_StandardImportsPath` points to the standard proto import directory, e.g. + `"...\.nuget\packages\grpc.tools\2.51.0\build\native\include"`. This is + the directory where a declaration such as `import "google/protobuf/wrappers.proto";` + in a proto file would find its target. -Example: to specify two additional arguments: ```--plugin=protoc-gen-myplugin=D:\myplugin.exe --myplugin_out=.``` +then in your project file: ```xml - - - + + + $(Protobuf_ProtocFullPath) --plugin=protoc-gen-grpc=$(gRPC_PluginFullPath) -I $(Protobuf_StandardImportsPath) ....rest of your command.... + + + + ``` +Do not include any `` items in the project file as that will invoke the `Grpc.Tools` build and your files will be compiled twice. -`grpc_csharp_plugin` command line options ---------- +--- -Under the hood, the `Grpc.Tools` build integration invokes the `protoc` and `grpc_csharp_plugin` binaries -to perform code generation. Here is an overview of the available `grpc_csharp_plugin` options: +## See also -| Name | Default | Synopsis | -|---------------- |-----------|----------------------------------------------------------| -| no_client | off | Don't generate the client stub | -| no_server | off | Don't generate the server-side stub | -| internal_access | off | Generate classes with "internal" visibility | +gRPC project documentation: +* [gRPC for .NET](https://github.com/grpc/grpc-dotnet) +* [gRPC C# (legacy implementation using gRPC Core library)](https://github.com/grpc/grpc/blob/master/src/csharp/README.md) +* [Grpc.Tools internals](Grpc.Tools/implementation_notes.md) - implementation notes for how `Grpc.Tools` works -Note that the protocol buffer compiler has a special commandline syntax for plugin options. -Example: -``` -protoc --plugin=protoc-gen-grpc=grpc_csharp_plugin --csharp_out=OUT_DIR \ - --grpc_out=OUT_DIR --grpc_opt=lite_client,no_server \ - -I INCLUDE_DIR foo.proto -``` +Microsoft documentation: +* [Microsoft: Overview for gRPC on .NET](https://learn.microsoft.com/en-us/aspnet/core/grpc/) +* [Microsoft: C# Tooling support for .proto files](https://learn.microsoft.com/en-us/aspnet/core/grpc/basics#c-tooling-support-for-proto-files) diff --git a/src/csharp/Grpc.Tools/Grpc.Tools.csproj b/src/csharp/Grpc.Tools/Grpc.Tools.csproj index 30a11eaf8b0..bf1352b10fc 100644 --- a/src/csharp/Grpc.Tools/Grpc.Tools.csproj +++ b/src/csharp/Grpc.Tools/Grpc.Tools.csproj @@ -1,4 +1,4 @@ - + Protobuf.MSBuild $(GrpcCsharpVersion) @@ -35,20 +35,17 @@ Grpc.Tools The gRPC Authors Copyright 2018 The gRPC Authors - gRPC and Protocol Buffer compiler for managed C# and native C++ projects. - -Add this package to a project that contains .proto files to be compiled to code. -It contains the compilers, include files and project system integration for gRPC -and Protocol buffer service description files necessary to build them on Windows, -Linux and MacOS. Managed runtime is supplied separately in the Grpc.Core package. + gRPC and Protocol Buffer compiler for C# projects packageIcon.png Apache-2.0 https://github.com/grpc/grpc gRPC RPC HTTP/2 + README.md + diff --git a/src/csharp/Grpc.Tools/README.md b/src/csharp/Grpc.Tools/README.md new file mode 100644 index 00000000000..14d8b089efe --- /dev/null +++ b/src/csharp/Grpc.Tools/README.md @@ -0,0 +1,26 @@ +# Grpc.Tools - Protocol Buffers/gRPC C# Code Generation Build Integration + +This package provides C# tooling support for generating C# code from `.proto` files in `.csproj` projects: +* It contains protocol buffers compiler and gRPC plugin to generate C# code. +* It can be used in building both grpc-dotnet projects and legacy c-core C# projects. + +The package is used to automatically generate the C# code for protocol buffer messages +and gRPC service stubs from `.proto` files. These files: +* are generated on an as-needed basis each time the project is built. +* aren't added to the project or checked into source control. +* are a build artifact usually contained in the `obj` directory. + +This package is optional. You may instead choose to generate the C# source files from +`.proto` files by running the `protoc` compiler manually or from a script. + +## Simple example + +To add a `.proto` file to a project edit the project’s `.csproj` file and add an item group with a `` element that refers to the `.proto` file, e.g. + +```xml + + + +``` + +For more complex examples and detailed information on how to use this package see: [BUILD-INTEGRATION.md](https://github.com/grpc/grpc/blob/master/src/csharp/BUILD-INTEGRATION.md)