46 Commits

Author SHA1 Message Date
Johan Brandhorst
153d36db8c Remove golang/protobuf/jsonpbtest dep (#145)
With 1.4.0 the golang/protobuf json testing protofiles
are being moved to [an internal package][1]. While this is
technically a breaking change, I doubt the maintainers
are going to reintroduce it, so I've copied the necessary
parts to our own testing protofile.

This allows users of this package and, more importantly,
grpcui to try 1.4.0 without this package having to migrate yet.
When it comes to migrating to 1.4.0, this should also make the
transition easier.

[1]: https://github.com/golang/protobuf/tree/v1.4.0-rc.4/internal/testprotos
2020-04-03 12:45:05 -04:00
Joshua Humphries
2af40876fc drop Go 1.9 and 1.10 from CI; add 1.13 and 1.14 (#144) 2020-04-01 09:02:40 -04:00
Joshua Humphries
0218a7db67 add script and readme for creating new releases (#140) 2020-03-18 08:51:18 -04:00
Richard Belleville
96cfd48e32 Activate xDS support in grpcurl (#137) 2020-03-13 16:29:41 -04:00
Richard Belleville
d30f3a01b7 Bump grpc to 1.28 (#136) 2020-03-12 18:55:09 -04:00
Joshua Humphries
0d669e78d0 use wrapped TransportCredentials instead of handling handshake in Dialer (#130)
* use wrapped TransportCredentials instead of handling handshake in Dialer, so that grpc library will use correct :scheme
* support -authority for TLS conns; now effectively supercedes -servername flag
2020-01-27 12:15:47 -05:00
Blake Williams
9572bd4525 Register gzip decoder (#124) 2019-11-26 08:55:50 -05:00
Joshua Humphries
ccc9007156 add -protoset-out option (#120) 2019-09-30 09:50:17 -04:00
J M
9248ea0963 Add Expand Headers Feature (#117) 2019-09-26 17:26:38 -04:00
Joshua Humphries
4054d1d115 fix go.mod for Go 1.13 (#110) 2019-08-09 14:17:53 -04:00
Joshua Humphries
5631bba117 add official Dockerfile (#104) 2019-07-03 15:57:24 -04:00
Joshua Humphries
80425d1b17 use protoreflect 1.4.4 (#109) 2019-07-03 15:56:51 -04:00
Joshua Humphries
7e4045565f update all deps; use new ResolveFilenames method (#103) 2019-05-24 10:26:38 -04:00
Joshua Humphries
e5b4fc6cc0 add API to expose AnyResolver implementations backed by a DescriptorSource (#102) 2019-05-22 21:38:46 -04:00
Joshua Humphries
09c3d1d69e use just-released v1.3.0 protoreflect (#101) 2019-05-22 16:20:41 -04:00
Joshua Humphries
5d6316f470 Adds support for showing error details (#98)
To better support printing of google.protobuf.Any messages (error details), this
also makes a few other changes:
1. Allows printing of unresolvable Any messages using an "@value" field in JSON output
   that has the base64-encoded embedded message data.
2. Improves support for "-format text" to show expanded Any messages if possible.
   (Due to limitations in underlying proto package, this will usually *not* be
   that helpful. But this should greatly improve with v2 of the go protobuf API.)
3. Addresses a TODO in existing AnyResolver code to lazily fetch descriptors
   as needed instead of having to download all files eagerly.
2019-04-09 09:34:39 -04:00
Joshua Humphries
f0723c6273 fix flaky test (#93) 2019-03-25 10:34:32 -04:00
Joshua Humphries
fe97274a1b staticcheck no longer runs on Go 1.10 (#92)
* staticcheck no longer runs on Go 1.10, so run it on Go 1.11 in CI
* be explicit about default make target in .travis.yml
2019-03-22 14:35:01 -04:00
CodeLingo Team
1bbf8dae71 Fix function comments based on best practices from Effective Go (#87)
Signed-off-by: CodeLingo Bot <bot@codelingo.io>
2019-03-13 10:58:13 -04:00
Joshua Humphries
0fcd3253f6 latest protoreflect fixes some bugs in JSON marshaling/unmarshaling (#86) 2019-03-07 13:35:49 -05:00
Joshua Humphries
4c9c82cec3 use custom flagset (#85) 2019-02-28 10:58:53 -05:00
Joshua Humphries
5082a1dc68 add -max-msg-sz flag (#84) 2019-02-28 08:35:50 -05:00
Joshua Humphries
d641a66208 fix flaky test where code can be 'cancelled' unexpectedly, instead of some error code provided by the server (#83) 2019-02-27 20:28:43 -05:00
Joshua Humphries
ce84976d3c fix typo in readme
fixes #79
2019-02-27 18:16:55 -05:00
Joshua Humphries
b292d5aef8 add Go 1.12 to travis config (#82) 2019-02-27 17:31:54 -05:00
Joshua Humphries
5516a45602 update proto and grpc deps (#81)
Fixes the build by updating grpc (and deps) and using new, non-deprecated function
2019-02-27 17:30:44 -05:00
Andrew McCallum
4a329f3b13 add Homebrew installation method to README (#77) 2019-01-23 10:50:39 -05:00
Joshua Humphries
1c6532c060 fix minor issues caught by staticcheck (#76) 2019-01-03 10:09:24 -05:00
Joshua Humphries
0f9e76c978 ignore deprecated check for now (#75) 2018-12-17 10:47:17 -05:00
Joshua Humphries
9fa2fce63b make method invocation resilient to errors trying to load all files from server (#74) 2018-12-13 11:44:43 -05:00
Joshua Humphries
70e9bba1b8 fix typo in Makefile (#72) 2018-12-12 10:10:23 -05:00
Joshua Humphries
d86529bb4f don't use TLS 1.3, which isn't quite right yet in Go tip (#66) 2018-11-15 20:27:01 -05:00
Joshua Humphries
0dea37ee70 on error reading request, bidi stream would hang (#63) 2018-11-01 14:32:56 -04:00
Joshua Humphries
dfa06f4410 several Go libraries, including gRPC and golang.org/x/net/http2, no longer support Go 1.8 and older (#64) 2018-11-01 14:24:33 -04:00
Joshua Humphries
22ce2f04fd account type in OpenAccount RPC was ignored (#61) 2018-11-01 13:52:58 -04:00
Joshua Humphries
1e8e50f4f8 make MakeTemplate more robust (#60) 2018-10-22 22:59:11 -04:00
Joshua Humphries
7cabe7a9d0 move more stuff from cmd to package (#59) 2018-10-19 14:47:25 -04:00
Joshua Humphries
9a4bbacdd6 some pre-factoring and small fixes (#58)
* organize into multiple files
* make listing methods show fully-qualified names
* address small feedback from recent change (trim then check if empty)
2018-10-18 23:51:38 -04:00
Joshua Humphries
69ea782936 reconcile with recent change to "describe" output 2018-10-18 13:54:51 -04:00
Joshua Humphries
58cd93280e use proto syntax for describe (#57) 2018-10-18 13:50:44 -04:00
Joshua Humphries
a337c1afcf improve flag doc with go 1.10+ (#56) 2018-10-18 13:16:33 -04:00
Joshua Humphries
e00ef3eb7c add option to support text format (#54)
* augments grpcurl package API in order to handle multiple formats
* deprecates old signature for InvokeRpc
* add command-line flag to use protobuf text format instead of JSON
* use AnyResolver when marshaling to/from JSON
2018-10-16 21:26:16 -04:00
Joshua Humphries
397a8c18ca update README (#55)
* Revises the sections that talk about descriptor sources, making them more consistent.
* Adds a link to the Gophercon 2018 talk on grpcurl.
* Improve  installation section, mentioning versioned Go.
2018-10-16 12:56:40 -04:00
Joshua Humphries
554e69be2c use correct package name for golint (#53) 2018-10-12 09:06:35 -04:00
Ryo Nakao
2dd771c49e fix typo in InvokeRpc() comment (#50) 2018-09-26 22:02:34 -04:00
Leigh McCulloch
79a550b858 add goreleaser (#49)
* add goreleaser

Add a goreleaser configuration file that builds binaries for Linux,
MacOS and Windows, for 32bit (x86/386) and 64bit (x64/amd64). The
binaries will be archived into a tar.gz/zip file along with the LICENSE
file.

The `dist/` directory will be written to by goreleaser with the binaries
during the build process, so it's also been added to .gitignore.

To build all the binaries and release them to GitHub:
1. Tag the release. e.g. `git tag -a v1.0.0 -m 'First release'`
2. Generate a GitHub Personal Access Token. See https://github.com/settings/tokens.
3. Push the release to GitHub. e.g. `git push origin v1.0.0`
4. Make the release, which will publish binaries to the GitHub "Releases" page. e.g.  `GITHUB_TOKEN=xxxxxxx... make release`

* add -version flag

Run `grpcurl -version` to see the release version. Use `make install` to build a binary that shows the version based on current git hash (which will show a version number if HEAD is a release tag and otherwise uses `git describe` to summarize the version).
2018-09-25 09:34:02 -04:00
33 changed files with 3078 additions and 962 deletions

2
.gitignore vendored Normal file
View File

@@ -0,0 +1,2 @@
dist/
VERSION

24
.goreleaser.yml Normal file
View File

@@ -0,0 +1,24 @@
builds:
- binary: grpcurl
main: ./cmd/grpcurl
goos:
- linux
- darwin
- windows
goarch:
- amd64
- 386
ldflags:
- -s -w -X main.version=v{{.Version}}
archive:
format: tar.gz
format_overrides:
- goos: windows
format: zip
replacements:
amd64: x86_64
386: x86_32
darwin: osx
files:
- LICENSE

View File

@@ -3,16 +3,29 @@ sudo: false
matrix:
include:
- go: "1.7"
- go: "1.8"
- go: "1.9"
- go: "1.10"
env: VET=1
- go: "1.11"
env: GO111MODULE=off
- go: "1.11"
# TODO: update the version used to vet to Go 1.13
- go: 1.11.x
env:
- GO111MODULE=off
- VET=1
- go: 1.11.x
env: GO111MODULE=on
- go: 1.12.x
env: GO111MODULE=off
- go: 1.12.x
env: GO111MODULE=on
- go: 1.13.x
env:
- GO111MODULE=on
- GOPROXY=direct
- go: 1.14.x
env:
- GO111MODULE=on
- GOPROXY=direct
- go: tip
env:
- GO111MODULE=on
- GOPROXY=direct
script:
- if [[ "$VET" = 1 ]]; then make; else make deps test; fi
- if [[ "$VET" = 1 ]]; then make ci; else make deps test; fi

33
Dockerfile Normal file
View File

@@ -0,0 +1,33 @@
FROM golang:1.11.10-alpine as builder
MAINTAINER FullStory Engineering
# currently, a module build requires gcc (so Go tool can build
# module-aware versions of std library; it ships only w/ the
# non-module versions)
RUN apk update && apk add --no-cache ca-certificates git gcc g++ libc-dev
# create non-privileged group and user
RUN addgroup -S grpcurl && adduser -S grpcurl -G grpcurl
WORKDIR /tmp/fullstorydev/grpcurl
# copy just the files/sources we need to build grpcurl
COPY VERSION *.go go.* /tmp/fullstorydev/grpcurl/
COPY cmd /tmp/fullstorydev/grpcurl/cmd
# and build a completely static binary (so we can use
# scratch as basis for the final image)
ENV CGO_ENABLED=0
ENV GOOS=linux
ENV GOARCH=amd64
ENV GO111MODULE=on
RUN go build -o /grpcurl \
-ldflags "-w -extldflags \"-static\" -X \"main.version=$(cat VERSION)\"" \
./cmd/grpcurl
# New FROM so we have a nice'n'tiny image
FROM scratch
WORKDIR /
COPY --from=builder /etc/ssl/certs/ca-certificates.crt /etc/ssl/certs/ca-certificates.crt
COPY --from=builder /etc/passwd /etc/passwd
COPY --from=builder /grpcurl /bin/grpcurl
USER grpcurl
ENTRYPOINT ["/bin/grpcurl"]

View File

@@ -1,10 +1,12 @@
dev_build_version=$(shell git describe --tags --always --dirty)
# TODO: run golint and errcheck, but only to catch *new* violations and
# decide whether to change code or not (e.g. we need to be able to whitelist
# violations already in the code). They can be useful to catch errors, but
# they are just too noisy to be a requirement for a CI -- we don't even *want*
# to fix some of the things they consider to be violations.
.PHONY: ci
ci: deps checkgofmt vet staticcheck unused ineffassign predeclared test
ci: deps checkgofmt vet staticcheck ineffassign predeclared test
.PHONY: deps
deps:
@@ -16,7 +18,18 @@ updatedeps:
.PHONY: install
install:
go install ./...
go install -ldflags '-X "main.version=dev build $(dev_build_version)"' ./...
.PHONY: release
release:
@GO111MODULE=off go get github.com/goreleaser/goreleaser
goreleaser --rm-dist
.PHONY: docker
docker:
@echo $(dev_build_version) > VERSION
docker build -t fullstorydev/grpcurl:$(dev_build_version) .
@rm VERSION
.PHONY: checkgofmt
checkgofmt:
@@ -29,16 +42,19 @@ checkgofmt:
vet:
go vet ./...
# TODO: remove the ignored check; need it for now because it
# is complaining about a deprecated comment added to grpc,
# but it's not yet released. Once the new (non-deprecated)
# API is included in a release, we can move to that new
# version and fix the call site to no longer use deprecated
# method.
# This all works fine with Go modules, but without modules,
# CI is just getting latest master for dependencies like grpc.
.PHONY: staticcheck
staticcheck:
@go get honnef.co/go/tools/cmd/staticcheck
staticcheck ./...
.PHONY: unused
unused:
@go get honnef.co/go/tools/cmd/unused
unused ./...
.PHONY: ineffassign
ineffassign:
@go get github.com/gordonklaus/ineffassign
@@ -52,11 +68,11 @@ predeclared:
# Intentionally omitted from CI, but target here for ad-hoc reports.
.PHONY: golint
golint:
@go get github.com/golang/lint/golint
@go get golang.org/x/lint/golint
golint -min_confidence 0.9 -set_exit_status ./...
# Intentionally omitted from CI, but target here for ad-hoc reports.
.PHONY: errchack
.PHONY: errcheck
errcheck:
@go get github.com/kisielk/errcheck
errcheck ./...

View File

@@ -14,22 +14,22 @@ This program accepts messages using JSON encoding, which is much more friendly f
humans and scripts.
With this tool you can also browse the schema for gRPC services, either by querying
a server that supports [service reflection](https://github.com/grpc/grpc/blob/master/src/proto/grpc/reflection/v1alpha/reflection.proto),
by reading proto source files, or by loading in compiled "protoset" files (files that contain encoded file
[descriptor protos](https://github.com/google/protobuf/blob/master/src/google/protobuf/descriptor.proto)).
a server that supports [server reflection](https://github.com/grpc/grpc/blob/master/src/proto/grpc/reflection/v1alpha/reflection.proto),
by reading proto source files, or by loading in compiled "protoset" files (files that contain
encoded file [descriptor protos](https://github.com/google/protobuf/blob/master/src/google/protobuf/descriptor.proto)).
In fact, the way the tool transforms JSON request data into a binary encoded protobuf
is using that very same schema. So, if the server you interact with does not support
reflection, you will either need the proto source files that define the service or need
protoset files that `grpcurl` can use.
[Examples for how to set up server reflection can be found here.](https://github.com/grpc/grpc/blob/master/doc/server-reflection.md#known-implementations)
This repo also provides a library package, `github.com/fullstorydev/grpcurl`, that has
functions for simplifying the construction of other command-line tools that dynamically
invoke gRPC endpoints. This code is a great example of how to use the various packages of
the [protoreflect](https://godoc.org/github.com/jhump/protoreflect) library, and shows
off what they can do.
See also the [`grpcurl` talk at GopherCon 2018](https://www.youtube.com/watch?v=dDr-8kbMnaw).
## Features
`grpcurl` supports all kinds of RPC methods, including streaming methods. You can even
operate bi-directional streaming methods interactively by running `grpcurl` from an
@@ -44,6 +44,17 @@ service. If not, you can supply the `.proto` source files or you can supply prot
files (containing compiled descriptors, produced by `protoc`) to `grpcurl`.
## Installation
### Binaries
Download the binary from the [releases](https://github.com/fullstorydev/grpcurl/releases) page.
On macOS, `grpcurl` is available via Homebrew:
```shell
brew install grpcurl
```
### From Source
You can use the `go` tool to install `grpcurl`:
```shell
go get github.com/fullstorydev/grpcurl
@@ -59,7 +70,11 @@ If you have already pulled down this repo to a location that is not in your
run `make install`.
If you encounter compile errors, you could have out-dated versions of `grpcurl`'s
dependencies. You can update the dependencies by running `make updatedeps`.
dependencies. You can update the dependencies by running `make updatedeps`. You can
also use [`vgo`](https://github.com/golang/vgo) to install, which will use the right
versions of dependencies. Or, if you are using Go 1.11, you can add `GO111MODULE=on`
as a prefix to the commands above, which will also build using the right versions of
dependencies (vs. whatever you may already in your `GOPATH`).
## Usage
The usage doc for the tool explains the numerous options:
@@ -72,7 +87,7 @@ In the sections below, you will find numerous examples demonstrating how to use
### Invoking RPCs
Invoking an RPC on a trusted server (e.g. TLS without self-signed key or custom CA)
that requires no client certs and supports service reflection is the simplest thing to
that requires no client certs and supports server reflection is the simplest thing to
do with `grpcurl`. This minimal invocation sends an empty request body:
```shell
grpcurl grpc.server.com:443 my.custom.server.Service/Method
@@ -92,7 +107,7 @@ If you want to include `grpcurl` in a command pipeline, such as when using `jq`
create a request body, you can use `-d @`, which tells `grpcurl` to read the actual
request body from stdin:
```shell
grpcurl -d @ grpc.server.com:443 my.custom.server.Service/Method <<<EOM
grpcurl -d @ grpc.server.com:443 my.custom.server.Service/Method <<EOM
{
"id": 1234,
"tags": [
@@ -125,8 +140,10 @@ grpcurl localhost:8787 list my.custom.server.Service
### Describing Elements
The "describe" verb will print the type of any symbol that the server knows about
or that is found in a given protoset file and also print the full descriptor for the
symbol, in JSON.
or that is found in a given protoset file. It also prints a description of that
symbol, in the form of snippets of proto source. It won't necessarily be the
original source that defined the element, but it will be equivalent.
```shell
# Server supports reflection
grpcurl localhost:8787 describe my.custom.server.Service.MethodOne
@@ -138,7 +155,24 @@ grpcurl -protoset my-protos.bin describe my.custom.server.Service.MethodOne
grpcurl -import-path ../protos -proto my-stuff.proto describe my.custom.server.Service.MethodOne
```
## Proto Source Files
## Descriptor Sources
The `grpcurl` tool can operate on a variety of sources for descriptors. The descriptors
are required, in order for `grpcurl` to understand the RPC schema, translate inputs
into the protobuf binary format as well as translate responses from the binary format
into text. The sections below document the supported sources and what command-line flags
are needed to use them.
### Server Reflection
Without any additional command-line flags, `grpcurl` will try to use [server reflection](https://github.com/grpc/grpc/blob/master/src/proto/grpc/reflection/v1alpha/reflection.proto).
Examples for how to set up server reflection can be found [here](https://github.com/grpc/grpc/blob/master/doc/server-reflection.md#known-implementations).
When using reflection, the server address (host:port or path to Unix socket) is required
even for "list" and "describe" operations, so that `grpcurl` can connect to the server
and ask it for its descriptors.
### Proto Source Files
To use `grpcurl` on servers that do not support reflection, you can use `.proto` source
files.
@@ -151,14 +185,18 @@ location of the standard protos included with `protoc` (which contain various "w
types" with a package definition of `google.protobuf`). These files are "known" by `grpcurl`
as a snapshot of their descriptors is built into the `grpcurl` binary.
## Protoset Files
When using proto sources, you can omit the server address (host:port or path to Unix socket)
when using the "list" and "describe" operations since they only need to consult the proto
source files.
### Protoset Files
You can also use compiled protoset files with `grpcurl`. If you are scripting `grpcurl` and
need to re-use the same proto sources for many invocations, you will see better performance
by using protoset files (since it skips the parsing and compilation steps with each
invocation).
Protoset files contain binary encoded `google.protobuf.FileDescriptorSet` protos. To create
a protoset file, invoke `protoc` with the `*.proto` files that describe the service:
a protoset file, invoke `protoc` with the `*.proto` files that define the service:
```shell
protoc --proto_path=. \
--descriptor_set_out=myservice.protoset \
@@ -169,3 +207,8 @@ protoc --proto_path=. \
The `--descriptor_set_out` argument is what tells `protoc` to produce a protoset,
and the `--include_imports` argument is necessary for the protoset to contain
everything that `grpcurl` needs to process and understand the schema.
When using protosets, you can omit the server address (host:port or path to Unix socket)
when using the "list" and "describe" operations since they only need to consult the
protoset files.

9
cmd/grpcurl/go1_10.go Normal file
View File

@@ -0,0 +1,9 @@
// +build go1.10
package main
func indent() string {
// In Go 1.10 and up, the flag package automatically
// adds the right indentation.
return ""
}

9
cmd/grpcurl/go1_9.go Normal file
View File

@@ -0,0 +1,9 @@
// +build !go1.10
package main
func indent() string {
// In Go 1.9 and older, we need to add indentation
// after newlines in the flag doc strings.
return " \t"
}

View File

@@ -1,22 +1,20 @@
// Command grpcurl makes GRPC requests (a la cURL, but HTTP/2). It can use a supplied descriptor file or
// service reflection to translate JSON request data into the appropriate protobuf request data and vice
// versa for presenting the response contents.
// Command grpcurl makes gRPC requests (a la cURL, but HTTP/2). It can use a supplied descriptor
// file, protobuf sources, or service reflection to translate JSON or text request data into the
// appropriate protobuf messages and vice versa for presenting the response contents.
package main
import (
"encoding/json"
"flag"
"fmt"
"io"
"os"
"strconv"
"path/filepath"
"strings"
"time"
"github.com/golang/protobuf/jsonpb"
"github.com/golang/protobuf/proto"
"github.com/fullstorydev/grpcurl"
descpb "github.com/golang/protobuf/protoc-gen-go/descriptor"
"github.com/jhump/protoreflect/desc"
"github.com/jhump/protoreflect/dynamic"
"github.com/jhump/protoreflect/grpcreflect"
"golang.org/x/net/context"
"google.golang.org/grpc"
@@ -25,107 +23,151 @@ import (
"google.golang.org/grpc/keepalive"
"google.golang.org/grpc/metadata"
reflectpb "google.golang.org/grpc/reflection/grpc_reflection_v1alpha"
"google.golang.org/grpc/status"
"github.com/fullstorydev/grpcurl"
// Register gzip compressor so compressed responses will work:
_ "google.golang.org/grpc/encoding/gzip"
)
var version = "dev build <no version set>"
var (
exit = os.Exit
isUnixSocket func() bool // nil when run on non-unix platform
help = flag.Bool("help", false,
`Print usage instructions and exit.`)
plaintext = flag.Bool("plaintext", false,
`Use plain-text HTTP/2 when connecting to server (no TLS).`)
insecure = flag.Bool("insecure", false,
`Skip server certificate and domain verification. (NOT SECURE!). Not
valid with -plaintext option.`)
cacert = flag.String("cacert", "",
`File containing trusted root certificates for verifying the server.
Ignored if -insecure is specified.`)
cert = flag.String("cert", "",
`File containing client certificate (public key), to present to the
server. Not valid with -plaintext option. Must also provide -key option.`)
key = flag.String("key", "",
`File containing client private key, to present to the server. Not valid
with -plaintext option. Must also provide -cert option.`)
protoset multiString
protoFiles multiString
importPaths multiString
addlHeaders multiString
rpcHeaders multiString
reflHeaders multiString
authority = flag.String("authority", "",
":authority pseudo header value to be passed along with underlying HTTP/2 requests. It defaults to `host [ \":\" port ]` part of the target url.")
data = flag.String("d", "",
`JSON request contents. If the value is '@' then the request contents are
read from stdin. For calls that accept a stream of requests, the
contents should include all such request messages concatenated together
(optionally separated by whitespace).`)
connectTimeout = flag.String("connect-timeout", "",
`The maximum time, in seconds, to wait for connection to be established.
Defaults to 10 seconds.`)
keepaliveTime = flag.String("keepalive-time", "",
`If present, the maximum idle time in seconds, after which a keepalive
probe is sent. If the connection remains idle and no keepalive response
is received for this same period then the connection is closed and the
operation fails.`)
maxTime = flag.String("max-time", "",
`The maximum total time the operation can take. This is useful for
preventing batch jobs that use grpcurl from hanging due to slow or bad
network links or due to incorrect stream method usage.`)
emitDefaults = flag.Bool("emit-defaults", false,
`Emit default values from JSON-encoded responses.`)
msgTemplate = flag.Bool("msg-template", false,
`When describing messages, show a JSON template for the message type.`)
verbose = flag.Bool("v", false,
`Enable verbose output.`)
serverName = flag.String("servername", "", "Override servername when validating TLS certificate.")
flags = flag.NewFlagSet(os.Args[0], flag.ExitOnError)
help = flags.Bool("help", false, prettify(`
Print usage instructions and exit.`))
printVersion = flags.Bool("version", false, prettify(`
Print version.`))
plaintext = flags.Bool("plaintext", false, prettify(`
Use plain-text HTTP/2 when connecting to server (no TLS).`))
insecure = flags.Bool("insecure", false, prettify(`
Skip server certificate and domain verification. (NOT SECURE!) Not
valid with -plaintext option.`))
cacert = flags.String("cacert", "", prettify(`
File containing trusted root certificates for verifying the server.
Ignored if -insecure is specified.`))
cert = flags.String("cert", "", prettify(`
File containing client certificate (public key), to present to the
server. Not valid with -plaintext option. Must also provide -key option.`))
key = flags.String("key", "", prettify(`
File containing client private key, to present to the server. Not valid
with -plaintext option. Must also provide -cert option.`))
protoset multiString
protoFiles multiString
importPaths multiString
addlHeaders multiString
rpcHeaders multiString
reflHeaders multiString
expandHeaders = flags.Bool("expand-headers", false, prettify(`
If set, headers may use '${NAME}' syntax to reference environment
variables. These will be expanded to the actual environment variable
value before sending to the server. For example, if there is an
environment variable defined like FOO=bar, then a header of
'key: ${FOO}' would expand to 'key: bar'. This applies to -H,
-rpc-header, and -reflect-header options. No other expansion/escaping is
performed. This can be used to supply credentials/secrets without having
to put them in command-line arguments.`))
authority = flags.String("authority", "", prettify(`
The authoritative name of the remote server. This value is passed as the
value of the ":authority" pseudo-header in the HTTP/2 protocol. When TLS
is used, this will also be used as the server name when verifying the
server's certificate. It defaults to the address that is provided in the
positional arguments.`))
data = flags.String("d", "", prettify(`
Data for request contents. If the value is '@' then the request contents
are read from stdin. For calls that accept a stream of requests, the
contents should include all such request messages concatenated together
(possibly delimited; see -format).`))
format = flags.String("format", "json", prettify(`
The format of request data. The allowed values are 'json' or 'text'. For
'json', the input data must be in JSON format. Multiple request values
may be concatenated (messages with a JSON representation other than
object must be separated by whitespace, such as a newline). For 'text',
the input data must be in the protobuf text format, in which case
multiple request values must be separated by the "record separator"
ASCII character: 0x1E. The stream should not end in a record separator.
If it does, it will be interpreted as a final, blank message after the
separator.`))
connectTimeout = flags.Float64("connect-timeout", 0, prettify(`
The maximum time, in seconds, to wait for connection to be established.
Defaults to 10 seconds.`))
keepaliveTime = flags.Float64("keepalive-time", 0, prettify(`
If present, the maximum idle time in seconds, after which a keepalive
probe is sent. If the connection remains idle and no keepalive response
is received for this same period then the connection is closed and the
operation fails.`))
maxTime = flags.Float64("max-time", 0, prettify(`
The maximum total time the operation can take, in seconds. This is
useful for preventing batch jobs that use grpcurl from hanging due to
slow or bad network links or due to incorrect stream method usage.`))
maxMsgSz = flags.Int("max-msg-sz", 0, prettify(`
The maximum encoded size of a response message, in bytes, that grpcurl
will accept. If not specified, defaults to 4,194,304 (4 megabytes).`))
emitDefaults = flags.Bool("emit-defaults", false, prettify(`
Emit default values for JSON-encoded responses.`))
protosetOut = flags.String("protoset-out", "", prettify(`
The name of a file to be written that will contain a FileDescriptorSet
proto. With the list and describe verbs, the listed or described
elements and their transitive dependencies will be written to the named
file if this option is given. When invoking an RPC and this option is
given, the method being invoked and its transitive dependencies will be
included in the output file.`))
msgTemplate = flags.Bool("msg-template", false, prettify(`
When describing messages, show a template of input data.`))
verbose = flags.Bool("v", false, prettify(`
Enable verbose output.`))
serverName = flags.String("servername", "", prettify(`
Override server name when validating TLS certificate. This flag is
ignored if -plaintext or -insecure is used.
NOTE: Prefer -authority. This flag may be removed in the future. It is
an error to use both -authority and -servername (though this will be
permitted if they are both set to the same value, to increase backwards
compatibility with earlier releases that allowed both to be set).`))
)
func init() {
flag.Var(&addlHeaders, "H",
`Additional headers in 'name: value' format. May specify more than one
via multiple flags. These headers will also be included in reflection
requests requests to a server.`)
flag.Var(&rpcHeaders, "rpc-header",
`Additional RPC headers in 'name: value' format. May specify more than
one via multiple flags. These headers will *only* be used when invoking
the requested RPC method. They are excluded from reflection requests.`)
flag.Var(&reflHeaders, "reflect-header",
`Additional reflection headers in 'name: value' format. May specify more
than one via multiple flags. These headers will only be used during
reflection requests and will be excluded when invoking the requested RPC
method.`)
flag.Var(&protoset, "protoset",
`The name of a file containing an encoded FileDescriptorSet. This file's
contents will be used to determine the RPC schema instead of querying
for it from the remote server via the GRPC reflection API. When set: the
'list' action lists the services found in the given descriptors (vs.
those exposed by the remote server), and the 'describe' action describes
symbols found in the given descriptors. May specify more than one via
multiple -protoset flags. It is an error to use both -protoset and
-proto flags.`)
flag.Var(&protoFiles, "proto",
`The name of a proto source file. Source files given will be used to
determine the RPC schema instead of querying for it from the remote
server via the GRPC reflection API. When set: the 'list' action lists
the services found in the given files and their imports (vs. those
exposed by the remote server), and the 'describe' action describes
symbols found in the given files. May specify more than one via
multiple -proto flags. Imports will be resolved using the given
-import-path flags. Multiple proto files can be specified by specifying
multiple -proto flags. It is an error to use both -protoset and -proto
flags.`)
flag.Var(&importPaths, "import-path",
`The path to a directory from which proto sources can be imported,
for use with -proto flags. Multiple import paths can be configured by
specifying multiple -import-path flags. Paths will be searched in the
order given. If no import paths are given, all files (including all
imports) must be provided as -proto flags, and grpcurl will attempt to
resolve all import statements from the set of file names given.`)
flags.Var(&addlHeaders, "H", prettify(`
Additional headers in 'name: value' format. May specify more than one
via multiple flags. These headers will also be included in reflection
requests requests to a server.`))
flags.Var(&rpcHeaders, "rpc-header", prettify(`
Additional RPC headers in 'name: value' format. May specify more than
one via multiple flags. These headers will *only* be used when invoking
the requested RPC method. They are excluded from reflection requests.`))
flags.Var(&reflHeaders, "reflect-header", prettify(`
Additional reflection headers in 'name: value' format. May specify more
than one via multiple flags. These headers will *only* be used during
reflection requests and will be excluded when invoking the requested RPC
method.`))
flags.Var(&protoset, "protoset", prettify(`
The name of a file containing an encoded FileDescriptorSet. This file's
contents will be used to determine the RPC schema instead of querying
for it from the remote server via the gRPC reflection API. When set: the
'list' action lists the services found in the given descriptors (vs.
those exposed by the remote server), and the 'describe' action describes
symbols found in the given descriptors. May specify more than one via
multiple -protoset flags. It is an error to use both -protoset and
-proto flags.`))
flags.Var(&protoFiles, "proto", prettify(`
The name of a proto source file. Source files given will be used to
determine the RPC schema instead of querying for it from the remote
server via the gRPC reflection API. When set: the 'list' action lists
the services found in the given files and their imports (vs. those
exposed by the remote server), and the 'describe' action describes
symbols found in the given files. May specify more than one via multiple
-proto flags. Imports will be resolved using the given -import-path
flags. Multiple proto files can be specified by specifying multiple
-proto flags. It is an error to use both -protoset and -proto flags.`))
flags.Var(&importPaths, "import-path", prettify(`
The path to a directory from which proto sources can be imported, for
use with -proto flags. Multiple import paths can be configured by
specifying multiple -import-path flags. Paths will be searched in the
order given. If no import paths are given, all files (including all
imports) must be provided as -proto flags, and grpcurl will attempt to
resolve all import statements from the set of file names given.`))
}
type multiString []string
@@ -140,14 +182,30 @@ func (s *multiString) Set(value string) error {
}
func main() {
flag.CommandLine.Usage = usage
flag.Parse()
flags.Usage = usage
flags.Parse(os.Args[1:])
if *help {
usage()
os.Exit(0)
}
if *printVersion {
fmt.Fprintf(os.Stderr, "%s %s\n", filepath.Base(os.Args[0]), version)
os.Exit(0)
}
// Do extra validation on arguments and figure out what user asked us to do.
if *connectTimeout < 0 {
fail(nil, "The -connect-timeout argument must not be negative.")
}
if *keepaliveTime < 0 {
fail(nil, "The -keepalive-time argument must not be negative.")
}
if *maxTime < 0 {
fail(nil, "The -max-time argument must not be negative.")
}
if *maxMsgSz < 0 {
fail(nil, "The -max-msg-sz argument must not be negative.")
}
if *plaintext && *insecure {
fail(nil, "The -plaintext and -insecure arguments are mutually exclusive.")
}
@@ -160,8 +218,14 @@ func main() {
if (*key == "") != (*cert == "") {
fail(nil, "The -cert and -key arguments must be used together and both be present.")
}
if *format != "json" && *format != "text" {
fail(nil, "The -format option must be 'json' or 'text.")
}
if *emitDefaults && *format != "json" {
warn("The -emit-defaults is only used when using json format.")
}
args := flag.Args()
args := flags.Args()
if len(args) == 0 {
fail(nil, "Too few arguments.")
@@ -226,40 +290,28 @@ func main() {
}
ctx := context.Background()
if *maxTime != "" {
t, err := strconv.ParseFloat(*maxTime, 64)
if err != nil {
fail(nil, "The -max-time argument must be a valid number.")
}
timeout := time.Duration(t * float64(time.Second))
if *maxTime > 0 {
timeout := time.Duration(*maxTime * float64(time.Second))
ctx, _ = context.WithTimeout(ctx, timeout)
}
dial := func() *grpc.ClientConn {
dialTime := 10 * time.Second
if *connectTimeout != "" {
t, err := strconv.ParseFloat(*connectTimeout, 64)
if err != nil {
fail(nil, "The -connect-timeout argument must be a valid number.")
}
dialTime = time.Duration(t * float64(time.Second))
if *connectTimeout > 0 {
dialTime = time.Duration(*connectTimeout * float64(time.Second))
}
ctx, cancel := context.WithTimeout(ctx, dialTime)
defer cancel()
var opts []grpc.DialOption
if *keepaliveTime != "" {
t, err := strconv.ParseFloat(*keepaliveTime, 64)
if err != nil {
fail(nil, "The -keepalive-time argument must be a valid number.")
}
timeout := time.Duration(t * float64(time.Second))
if *keepaliveTime > 0 {
timeout := time.Duration(*keepaliveTime * float64(time.Second))
opts = append(opts, grpc.WithKeepaliveParams(keepalive.ClientParameters{
Time: timeout,
Timeout: timeout,
}))
}
if *authority != "" {
opts = append(opts, grpc.WithAuthority(*authority))
if *maxMsgSz > 0 {
opts = append(opts, grpc.WithDefaultCallOptions(grpc.MaxCallRecvMsgSize(*maxMsgSz)))
}
var creds credentials.TransportCredentials
if !*plaintext {
@@ -268,11 +320,27 @@ func main() {
if err != nil {
fail(err, "Failed to configure transport credentials")
}
if *serverName != "" {
if err := creds.OverrideServerName(*serverName); err != nil {
fail(err, "Failed to override server name as %q", *serverName)
// can use either -servername or -authority; but not both
if *serverName != "" && *authority != "" {
if *serverName == *authority {
warn("Both -servername and -authority are present; prefer only -authority.")
} else {
fail(nil, "Cannot specify different values for -servername and -authority.")
}
}
overrideName := *serverName
if overrideName == "" {
overrideName = *authority
}
if overrideName != "" {
if err := creds.OverrideServerName(overrideName); err != nil {
fail(err, "Failed to override server name as %q", overrideName)
}
}
} else if *authority != "" {
opts = append(opts, grpc.WithAuthority(*authority))
}
network := "tcp"
if isUnixSocket != nil && isUnixSocket() {
@@ -285,6 +353,22 @@ func main() {
return cc
}
if *expandHeaders {
var err error
addlHeaders, err = grpcurl.ExpandHeaders(addlHeaders)
if err != nil {
fail(err, "Failed to expand additional headers")
}
rpcHeaders, err = grpcurl.ExpandHeaders(rpcHeaders)
if err != nil {
fail(err, "Failed to expand rpc headers")
}
reflHeaders, err = grpcurl.ExpandHeaders(reflHeaders)
if err != nil {
fail(err, "Failed to expand reflection headers")
}
}
var cc *grpc.ClientConn
var descSource grpcurl.DescriptorSource
var refClient *grpcreflect.Client
@@ -339,6 +423,9 @@ func main() {
fmt.Printf("%s\n", svc)
}
}
if err := writeProtoset(descSource, svcs...); err != nil {
fail(err, "Failed to write protoset to %s", *protosetOut)
}
} else {
methods, err := grpcurl.ListMethods(descSource, symbol)
if err != nil {
@@ -351,6 +438,9 @@ func main() {
fmt.Printf("%s\n", m)
}
}
if err := writeProtoset(descSource, symbol); err != nil {
fail(err, "Failed to write protoset to %s", *protosetOut)
}
}
} else if describe {
@@ -378,76 +468,123 @@ func main() {
fail(err, "Failed to resolve symbol %q", s)
}
txt, err := grpcurl.GetDescriptorText(dsc, descSource)
if err != nil {
fail(err, "Failed to describe symbol %q", s)
}
switch dsc.(type) {
fqn := dsc.GetFullyQualifiedName()
var elementType string
switch d := dsc.(type) {
case *desc.MessageDescriptor:
fmt.Printf("%s is a message:\n", dsc.GetFullyQualifiedName())
elementType = "a message"
parent, ok := d.GetParent().(*desc.MessageDescriptor)
if ok {
if d.IsMapEntry() {
for _, f := range parent.GetFields() {
if f.IsMap() && f.GetMessageType() == d {
// found it: describe the map field instead
elementType = "the entry type for a map field"
dsc = f
break
}
}
} else {
// see if it's a group
for _, f := range parent.GetFields() {
if f.GetType() == descpb.FieldDescriptorProto_TYPE_GROUP && f.GetMessageType() == d {
// found it: describe the map field instead
elementType = "the type of a group field"
dsc = f
break
}
}
}
}
case *desc.FieldDescriptor:
fmt.Printf("%s is a field:\n", dsc.GetFullyQualifiedName())
elementType = "a field"
if d.GetType() == descpb.FieldDescriptorProto_TYPE_GROUP {
elementType = "a group field"
} else if d.IsExtension() {
elementType = "an extension"
}
case *desc.OneOfDescriptor:
fmt.Printf("%s is a one-of:\n", dsc.GetFullyQualifiedName())
elementType = "a one-of"
case *desc.EnumDescriptor:
fmt.Printf("%s is an enum:\n", dsc.GetFullyQualifiedName())
elementType = "an enum"
case *desc.EnumValueDescriptor:
fmt.Printf("%s is an enum value:\n", dsc.GetFullyQualifiedName())
elementType = "an enum value"
case *desc.ServiceDescriptor:
fmt.Printf("%s is a service:\n", dsc.GetFullyQualifiedName())
elementType = "a service"
case *desc.MethodDescriptor:
fmt.Printf("%s is a method:\n", dsc.GetFullyQualifiedName())
elementType = "a method"
default:
err = fmt.Errorf("descriptor has unrecognized type %T", dsc)
fail(err, "Failed to describe symbol %q", s)
}
txt, err := grpcurl.GetDescriptorText(dsc, descSource)
if err != nil {
fail(err, "Failed to describe symbol %q", s)
}
fmt.Printf("%s is %s:\n", fqn, elementType)
fmt.Println(txt)
if dsc, ok := dsc.(*desc.MessageDescriptor); ok && *msgTemplate {
// for messages, also show a template in JSON, to make it easier to
// create a request to invoke an RPC
tmpl := makeTemplate(dynamic.NewMessage(dsc))
fmt.Println("\nMessage template:")
jsm := jsonpb.Marshaler{Indent: " ", EmitDefaults: true}
err := jsm.Marshal(os.Stdout, tmpl)
tmpl := grpcurl.MakeTemplate(dsc)
_, formatter, err := grpcurl.RequestParserAndFormatterFor(grpcurl.Format(*format), descSource, true, false, nil)
if err != nil {
fail(err, "Failed to construct formatter for %q", *format)
}
str, err := formatter(tmpl)
if err != nil {
fail(err, "Failed to print template for message %s", s)
}
fmt.Println()
fmt.Println("\nMessage template:")
fmt.Println(str)
}
}
if err := writeProtoset(descSource, symbols...); err != nil {
fail(err, "Failed to write protoset to %s", *protosetOut)
}
} else {
// Invoke an RPC
if cc == nil {
cc = dial()
}
var dec *json.Decoder
var in io.Reader
if *data == "@" {
dec = json.NewDecoder(os.Stdin)
in = os.Stdin
} else {
dec = json.NewDecoder(strings.NewReader(*data))
in = strings.NewReader(*data)
}
h := &handler{dec: dec, descSource: descSource}
err := grpcurl.InvokeRpc(ctx, descSource, cc, symbol, append(addlHeaders, rpcHeaders...), h, h.getRequestData)
// if not verbose output, then also include record delimiters
// between each message, so output could potentially be piped
// to another grpcurl process
includeSeparators := !*verbose
rf, formatter, err := grpcurl.RequestParserAndFormatterFor(grpcurl.Format(*format), descSource, *emitDefaults, includeSeparators, in)
if err != nil {
fail(err, "Failed to construct request parser and formatter for %q", *format)
}
h := grpcurl.NewDefaultEventHandler(os.Stdout, descSource, formatter, *verbose)
err = grpcurl.InvokeRPC(ctx, descSource, cc, symbol, append(addlHeaders, rpcHeaders...), h, rf.Next)
if err != nil {
fail(err, "Error invoking method %q", symbol)
}
reqSuffix := ""
respSuffix := ""
if h.reqCount != 1 {
reqCount := rf.NumRequests()
if reqCount != 1 {
reqSuffix = "s"
}
if h.respCount != 1 {
if h.NumResponses != 1 {
respSuffix = "s"
}
if *verbose {
fmt.Printf("Sent %d request%s and received %d response%s\n", h.reqCount, reqSuffix, h.respCount, respSuffix)
fmt.Printf("Sent %d request%s and received %d response%s\n", reqCount, reqSuffix, h.NumResponses, respSuffix)
}
if h.stat.Code() != codes.OK {
fmt.Fprintf(os.Stderr, "ERROR:\n Code: %s\n Message: %s\n", h.stat.Code().String(), h.stat.Message())
if h.Status.Code() != codes.OK {
grpcurl.PrintStatus(os.Stderr, h.Status, formatter)
exit(1)
}
}
@@ -457,8 +594,8 @@ func usage() {
fmt.Fprintf(os.Stderr, `Usage:
%s [flags] [address] [list|describe] [symbol]
The 'host:port' is only optional when used with 'list' or 'describe' and a
protoset flag is provided.
The 'address' is only optional when used with 'list' or 'describe' and a
protoset or proto flag is provided.
If 'list' is indicated, the symbol (if present) should be a fully-qualified
service name. If present, all methods of that service are listed. If not
@@ -470,17 +607,37 @@ is given then the descriptors for all exposed or known services are shown.
If neither verb is present, the symbol must be a fully-qualified method name in
'service/method' or 'service.method' format. In this case, the request body will
be used to invoke the named method. If no body is given, an empty instance of
the method's request type will be sent.
be used to invoke the named method. If no body is given but one is required
(i.e. the method is unary or server-streaming), an empty instance of the
method's request type will be sent.
The address will typically be in the form "host:port" where host can be an IP
address or a hostname and port is a numeric port or service name. If an IPv6
address is given, it must be surrounded by brackets, like "[2001:db8::1]". For
Unix variants, if a -unix=true flag is present, then the address must be the
path to the domain socket.
`, os.Args[0])
flag.PrintDefaults()
Available flags:
`, os.Args[0])
flags.PrintDefaults()
}
func prettify(docString string) string {
parts := strings.Split(docString, "\n")
// cull empty lines and also remove trailing and leading spaces
// from each line in the doc string
j := 0
for _, part := range parts {
part = strings.TrimSpace(part)
if part == "" {
continue
}
parts[j] = part
j++
}
return strings.Join(parts[:j], "\n"+indent())
}
func warn(msg string, args ...interface{}) {
@@ -504,124 +661,14 @@ func fail(err error, msg string, args ...interface{}) {
}
}
type handler struct {
dec *json.Decoder
descSource grpcurl.DescriptorSource
reqCount int
respCount int
stat *status.Status
}
func (h *handler) OnResolveMethod(md *desc.MethodDescriptor) {
if *verbose {
txt, err := grpcurl.GetDescriptorText(md, h.descSource)
if err == nil {
fmt.Printf("\nResolved method descriptor:\n%s\n", txt)
}
func writeProtoset(descSource grpcurl.DescriptorSource, symbols ...string) error {
if *protosetOut == "" {
return nil
}
}
func (*handler) OnSendHeaders(md metadata.MD) {
if *verbose {
fmt.Printf("\nRequest metadata to send:\n%s\n", grpcurl.MetadataToString(md))
}
}
func (h *handler) getRequestData() ([]byte, error) {
// we don't use a mutex, though this methods will be called from different goroutine
// than other methods for bidi calls, because this method does not share any state
// with the other methods.
var msg json.RawMessage
if err := h.dec.Decode(&msg); err != nil {
return nil, err
}
h.reqCount++
return msg, nil
}
func (*handler) OnReceiveHeaders(md metadata.MD) {
if *verbose {
fmt.Printf("\nResponse headers received:\n%s\n", grpcurl.MetadataToString(md))
}
}
func (h *handler) OnReceiveResponse(resp proto.Message) {
h.respCount++
if *verbose {
fmt.Print("\nResponse contents:\n")
}
jsm := jsonpb.Marshaler{EmitDefaults: *emitDefaults, Indent: " "}
respStr, err := jsm.MarshalToString(resp)
f, err := os.Create(*protosetOut)
if err != nil {
fail(err, "failed to generate JSON form of response message")
return err
}
fmt.Println(respStr)
}
func (h *handler) OnReceiveTrailers(stat *status.Status, md metadata.MD) {
h.stat = stat
if *verbose {
fmt.Printf("\nResponse trailers received:\n%s\n", grpcurl.MetadataToString(md))
}
}
// makeTemplate fleshes out the given message so that it is a suitable template for creating
// an instance of that message in JSON. In particular, it ensures that any repeated fields
// (which include map fields) are not empty, so they will render with a single element (to
// show the types and optionally nested fields). It also ensures that nested messages are
// not nil by setting them to a message that is also fleshed out as a template message.
func makeTemplate(msg proto.Message) proto.Message {
dm, ok := msg.(*dynamic.Message)
if !ok {
return msg
}
// for repeated fields, add a single element with default value
// and for message fields, add a message with all default fields
// that also has non-nil message and non-empty repeated fields
for _, fd := range dm.GetMessageDescriptor().GetFields() {
if fd.IsRepeated() {
switch fd.GetType() {
case descpb.FieldDescriptorProto_TYPE_FIXED32,
descpb.FieldDescriptorProto_TYPE_UINT32:
dm.AddRepeatedField(fd, uint32(0))
case descpb.FieldDescriptorProto_TYPE_SFIXED32,
descpb.FieldDescriptorProto_TYPE_SINT32,
descpb.FieldDescriptorProto_TYPE_INT32,
descpb.FieldDescriptorProto_TYPE_ENUM:
dm.AddRepeatedField(fd, int32(0))
case descpb.FieldDescriptorProto_TYPE_FIXED64,
descpb.FieldDescriptorProto_TYPE_UINT64:
dm.AddRepeatedField(fd, uint64(0))
case descpb.FieldDescriptorProto_TYPE_SFIXED64,
descpb.FieldDescriptorProto_TYPE_SINT64,
descpb.FieldDescriptorProto_TYPE_INT64:
dm.AddRepeatedField(fd, int64(0))
case descpb.FieldDescriptorProto_TYPE_STRING:
dm.AddRepeatedField(fd, "")
case descpb.FieldDescriptorProto_TYPE_BYTES:
dm.AddRepeatedField(fd, []byte{})
case descpb.FieldDescriptorProto_TYPE_BOOL:
dm.AddRepeatedField(fd, false)
case descpb.FieldDescriptorProto_TYPE_FLOAT:
dm.AddRepeatedField(fd, float32(0))
case descpb.FieldDescriptorProto_TYPE_DOUBLE:
dm.AddRepeatedField(fd, float64(0))
case descpb.FieldDescriptorProto_TYPE_MESSAGE,
descpb.FieldDescriptorProto_TYPE_GROUP:
dm.AddRepeatedField(fd, makeTemplate(dynamic.NewMessage(fd.GetMessageType())))
}
} else if fd.GetMessageType() != nil {
dm.SetField(fd, makeTemplate(dynamic.NewMessage(fd.GetMessageType())))
}
}
return dm
defer f.Close()
return grpcurl.WriteProtoset(f, descSource, symbols...)
}

View File

@@ -0,0 +1,46 @@
package main
import (
"bytes"
"flag"
"testing"
)
func TestFlagDocIndent(t *testing.T) {
// Tests the prettify() and indent() function. The indent() function
// differs by Go version, due to differences in "flags" package across
// versions. Run with multiple versions of Go to ensure that doc output
// is properly indented, regardless of Go version.
var fs flag.FlagSet
var buf bytes.Buffer
fs.SetOutput(&buf)
fs.String("foo", "", prettify(`
This is a flag doc string.
It has multiple lines.
More than two, actually.`))
fs.Int("bar", 100, prettify(`This is a simple flag doc string.`))
fs.Bool("baz", false, prettify(`
This is another long doc string.
It also has multiple lines. But not as long as the first one.`))
fs.PrintDefaults()
expected :=
` -bar int
This is a simple flag doc string. (default 100)
-baz
This is another long doc string.
It also has multiple lines. But not as long as the first one.
-foo string
This is a flag doc string.
It has multiple lines.
More than two, actually.
`
actual := buf.String()
if actual != expected {
t.Errorf("Flag output had wrong indentation.\nExpecting:\n%s\nGot:\n%s", expected, actual)
}
}

View File

@@ -2,11 +2,9 @@
package main
import "flag"
var (
unix = flag.Bool("unix", false,
`Indicates that the server address is the path to a Unix domain socket.`)
unix = flags.Bool("unix", false, prettify(`
Indicates that the server address is the path to a Unix domain socket.`))
)
func init() {

304
desc_source.go Normal file
View File

@@ -0,0 +1,304 @@
package grpcurl
import (
"errors"
"fmt"
"io"
"io/ioutil"
"sync"
"github.com/golang/protobuf/proto"
descpb "github.com/golang/protobuf/protoc-gen-go/descriptor"
"github.com/jhump/protoreflect/desc"
"github.com/jhump/protoreflect/desc/protoparse"
"github.com/jhump/protoreflect/dynamic"
"github.com/jhump/protoreflect/grpcreflect"
"golang.org/x/net/context"
"google.golang.org/grpc/codes"
"google.golang.org/grpc/status"
)
// ErrReflectionNotSupported is returned by DescriptorSource operations that
// rely on interacting with the reflection service when the source does not
// actually expose the reflection service. When this occurs, an alternate source
// (like file descriptor sets) must be used.
var ErrReflectionNotSupported = errors.New("server does not support the reflection API")
// DescriptorSource is a source of protobuf descriptor information. It can be backed by a FileDescriptorSet
// proto (like a file generated by protoc) or a remote server that supports the reflection API.
type DescriptorSource interface {
// ListServices returns a list of fully-qualified service names. It will be all services in a set of
// descriptor files or the set of all services exposed by a gRPC server.
ListServices() ([]string, error)
// FindSymbol returns a descriptor for the given fully-qualified symbol name.
FindSymbol(fullyQualifiedName string) (desc.Descriptor, error)
// AllExtensionsForType returns all known extension fields that extend the given message type name.
AllExtensionsForType(typeName string) ([]*desc.FieldDescriptor, error)
}
// DescriptorSourceFromProtoSets creates a DescriptorSource that is backed by the named files, whose contents
// are encoded FileDescriptorSet protos.
func DescriptorSourceFromProtoSets(fileNames ...string) (DescriptorSource, error) {
files := &descpb.FileDescriptorSet{}
for _, fileName := range fileNames {
b, err := ioutil.ReadFile(fileName)
if err != nil {
return nil, fmt.Errorf("could not load protoset file %q: %v", fileName, err)
}
var fs descpb.FileDescriptorSet
err = proto.Unmarshal(b, &fs)
if err != nil {
return nil, fmt.Errorf("could not parse contents of protoset file %q: %v", fileName, err)
}
files.File = append(files.File, fs.File...)
}
return DescriptorSourceFromFileDescriptorSet(files)
}
// DescriptorSourceFromProtoFiles creates a DescriptorSource that is backed by the named files,
// whose contents are Protocol Buffer source files. The given importPaths are used to locate
// any imported files.
func DescriptorSourceFromProtoFiles(importPaths []string, fileNames ...string) (DescriptorSource, error) {
fileNames, err := protoparse.ResolveFilenames(importPaths, fileNames...)
if err != nil {
return nil, err
}
p := protoparse.Parser{
ImportPaths: importPaths,
InferImportPaths: len(importPaths) == 0,
IncludeSourceCodeInfo: true,
}
fds, err := p.ParseFiles(fileNames...)
if err != nil {
return nil, fmt.Errorf("could not parse given files: %v", err)
}
return DescriptorSourceFromFileDescriptors(fds...)
}
// DescriptorSourceFromFileDescriptorSet creates a DescriptorSource that is backed by the FileDescriptorSet.
func DescriptorSourceFromFileDescriptorSet(files *descpb.FileDescriptorSet) (DescriptorSource, error) {
unresolved := map[string]*descpb.FileDescriptorProto{}
for _, fd := range files.File {
unresolved[fd.GetName()] = fd
}
resolved := map[string]*desc.FileDescriptor{}
for _, fd := range files.File {
_, err := resolveFileDescriptor(unresolved, resolved, fd.GetName())
if err != nil {
return nil, err
}
}
return &fileSource{files: resolved}, nil
}
func resolveFileDescriptor(unresolved map[string]*descpb.FileDescriptorProto, resolved map[string]*desc.FileDescriptor, filename string) (*desc.FileDescriptor, error) {
if r, ok := resolved[filename]; ok {
return r, nil
}
fd, ok := unresolved[filename]
if !ok {
return nil, fmt.Errorf("no descriptor found for %q", filename)
}
deps := make([]*desc.FileDescriptor, 0, len(fd.GetDependency()))
for _, dep := range fd.GetDependency() {
depFd, err := resolveFileDescriptor(unresolved, resolved, dep)
if err != nil {
return nil, err
}
deps = append(deps, depFd)
}
result, err := desc.CreateFileDescriptor(fd, deps...)
if err != nil {
return nil, err
}
resolved[filename] = result
return result, nil
}
// DescriptorSourceFromFileDescriptors creates a DescriptorSource that is backed by the given
// file descriptors
func DescriptorSourceFromFileDescriptors(files ...*desc.FileDescriptor) (DescriptorSource, error) {
fds := map[string]*desc.FileDescriptor{}
for _, fd := range files {
if err := addFile(fd, fds); err != nil {
return nil, err
}
}
return &fileSource{files: fds}, nil
}
func addFile(fd *desc.FileDescriptor, fds map[string]*desc.FileDescriptor) error {
name := fd.GetName()
if existing, ok := fds[name]; ok {
// already added this file
if existing != fd {
// doh! duplicate files provided
return fmt.Errorf("given files include multiple copies of %q", name)
}
return nil
}
fds[name] = fd
for _, dep := range fd.GetDependencies() {
if err := addFile(dep, fds); err != nil {
return err
}
}
return nil
}
type fileSource struct {
files map[string]*desc.FileDescriptor
er *dynamic.ExtensionRegistry
erInit sync.Once
}
func (fs *fileSource) ListServices() ([]string, error) {
set := map[string]bool{}
for _, fd := range fs.files {
for _, svc := range fd.GetServices() {
set[svc.GetFullyQualifiedName()] = true
}
}
sl := make([]string, 0, len(set))
for svc := range set {
sl = append(sl, svc)
}
return sl, nil
}
// GetAllFiles returns all of the underlying file descriptors. This is
// more thorough and more efficient than the fallback strategy used by
// the GetAllFiles package method, for enumerating all files from a
// descriptor source.
func (fs *fileSource) GetAllFiles() ([]*desc.FileDescriptor, error) {
files := make([]*desc.FileDescriptor, len(fs.files))
i := 0
for _, fd := range fs.files {
files[i] = fd
i++
}
return files, nil
}
func (fs *fileSource) FindSymbol(fullyQualifiedName string) (desc.Descriptor, error) {
for _, fd := range fs.files {
if dsc := fd.FindSymbol(fullyQualifiedName); dsc != nil {
return dsc, nil
}
}
return nil, notFound("Symbol", fullyQualifiedName)
}
func (fs *fileSource) AllExtensionsForType(typeName string) ([]*desc.FieldDescriptor, error) {
fs.erInit.Do(func() {
fs.er = &dynamic.ExtensionRegistry{}
for _, fd := range fs.files {
fs.er.AddExtensionsFromFile(fd)
}
})
return fs.er.AllExtensionsForType(typeName), nil
}
// DescriptorSourceFromServer creates a DescriptorSource that uses the given gRPC reflection client
// to interrogate a server for descriptor information. If the server does not support the reflection
// API then the various DescriptorSource methods will return ErrReflectionNotSupported
func DescriptorSourceFromServer(_ context.Context, refClient *grpcreflect.Client) DescriptorSource {
return serverSource{client: refClient}
}
type serverSource struct {
client *grpcreflect.Client
}
func (ss serverSource) ListServices() ([]string, error) {
svcs, err := ss.client.ListServices()
return svcs, reflectionSupport(err)
}
func (ss serverSource) FindSymbol(fullyQualifiedName string) (desc.Descriptor, error) {
file, err := ss.client.FileContainingSymbol(fullyQualifiedName)
if err != nil {
return nil, reflectionSupport(err)
}
d := file.FindSymbol(fullyQualifiedName)
if d == nil {
return nil, notFound("Symbol", fullyQualifiedName)
}
return d, nil
}
func (ss serverSource) AllExtensionsForType(typeName string) ([]*desc.FieldDescriptor, error) {
var exts []*desc.FieldDescriptor
nums, err := ss.client.AllExtensionNumbersForType(typeName)
if err != nil {
return nil, reflectionSupport(err)
}
for _, fieldNum := range nums {
ext, err := ss.client.ResolveExtension(typeName, fieldNum)
if err != nil {
return nil, reflectionSupport(err)
}
exts = append(exts, ext)
}
return exts, nil
}
func reflectionSupport(err error) error {
if err == nil {
return nil
}
if stat, ok := status.FromError(err); ok && stat.Code() == codes.Unimplemented {
return ErrReflectionNotSupported
}
return err
}
// WriteProtoset will use the given descriptor source to resolve all of the given
// symbols and write a proto file descriptor set with their definitions to the
// given output. The output will include descriptors for all files in which the
// symbols are defined as well as their transitive dependencies.
func WriteProtoset(out io.Writer, descSource DescriptorSource, symbols ...string) error {
// compute set of file descriptors
filenames := make([]string, 0, len(symbols))
fds := make(map[string]*desc.FileDescriptor, len(symbols))
for _, sym := range symbols {
d, err := descSource.FindSymbol(sym)
if err != nil {
return fmt.Errorf("failed to find descriptor for %q: %v", sym, err)
}
fd := d.GetFile()
if _, ok := fds[fd.GetName()]; !ok {
fds[fd.GetName()] = fd
filenames = append(filenames, fd.GetName())
}
}
// now expand that to include transitive dependencies in topologically sorted
// order (such that file always appears after its dependencies)
expandedFiles := make(map[string]struct{}, len(fds))
allFilesSlice := make([]*descpb.FileDescriptorProto, 0, len(fds))
for _, filename := range filenames {
allFilesSlice = addFilesToSet(allFilesSlice, expandedFiles, fds[filename])
}
// now we can serialize to file
b, err := proto.Marshal(&descpb.FileDescriptorSet{File: allFilesSlice})
if err != nil {
return fmt.Errorf("failed to serialize file descriptor set: %v", err)
}
if _, err := out.Write(b); err != nil {
return fmt.Errorf("failed to write file descriptor set: %v", err)
}
return nil
}
func addFilesToSet(allFiles []*descpb.FileDescriptorProto, expanded map[string]struct{}, fd *desc.FileDescriptor) []*descpb.FileDescriptorProto {
if _, ok := expanded[fd.GetName()]; ok {
// already seen this one
return allFiles
}
expanded[fd.GetName()] = struct{}{}
// add all dependencies first
for _, dep := range fd.GetDependencies() {
allFiles = addFilesToSet(allFiles, expanded, dep)
}
return append(allFiles, fd.AsFileDescriptorProto())
}

62
desc_source_test.go Normal file
View File

@@ -0,0 +1,62 @@
package grpcurl
import (
"bytes"
"io/ioutil"
"testing"
"github.com/golang/protobuf/proto"
"github.com/golang/protobuf/protoc-gen-go/descriptor"
)
func TestWriteProtoset(t *testing.T) {
exampleProtoset, err := loadProtoset("./testing/example.protoset")
if err != nil {
t.Fatalf("failed to load example.protoset: %v", err)
}
testProtoset, err := loadProtoset("./testing/test.protoset")
if err != nil {
t.Fatalf("failed to load test.protoset: %v", err)
}
mergedProtoset := &descriptor.FileDescriptorSet{
File: append(exampleProtoset.File, testProtoset.File...),
}
descSrc, err := DescriptorSourceFromFileDescriptorSet(mergedProtoset)
if err != nil {
t.Fatalf("failed to create descriptor source: %v", err)
}
checkWriteProtoset(t, descSrc, exampleProtoset, "TestService")
checkWriteProtoset(t, descSrc, testProtoset, "grpc.testing.TestService")
checkWriteProtoset(t, descSrc, mergedProtoset, "TestService", "grpc.testing.TestService")
}
func loadProtoset(path string) (*descriptor.FileDescriptorSet, error) {
b, err := ioutil.ReadFile(path)
if err != nil {
return nil, err
}
var protoset descriptor.FileDescriptorSet
if err := proto.Unmarshal(b, &protoset); err != nil {
return nil, err
}
return &protoset, nil
}
func checkWriteProtoset(t *testing.T, descSrc DescriptorSource, protoset *descriptor.FileDescriptorSet, symbols ...string) {
var buf bytes.Buffer
if err := WriteProtoset(&buf, descSrc, symbols...); err != nil {
t.Fatalf("failed to write protoset: %v", err)
}
var result descriptor.FileDescriptorSet
if err := proto.Unmarshal(buf.Bytes(), &result); err != nil {
t.Fatalf("failed to unmarshal written protoset: %v", err)
}
if !proto.Equal(protoset, &result) {
t.Fatalf("written protoset not equal to input:\nExpecting: %s\nActual: %s", protoset, &result)
}
}

469
format.go Normal file
View File

@@ -0,0 +1,469 @@
package grpcurl
import (
"bufio"
"bytes"
"encoding/base64"
"encoding/json"
"fmt"
"io"
"reflect"
"strings"
"sync"
"github.com/golang/protobuf/jsonpb"
"github.com/golang/protobuf/proto"
"github.com/jhump/protoreflect/desc"
"github.com/jhump/protoreflect/dynamic"
"google.golang.org/grpc/codes"
"google.golang.org/grpc/metadata"
"google.golang.org/grpc/status"
)
// RequestParser processes input into messages.
type RequestParser interface {
// Next parses input data into the given request message. If called after
// input is exhausted, it returns io.EOF. If the caller re-uses the same
// instance in multiple calls to Next, it should call msg.Reset() in between
// each call.
Next(msg proto.Message) error
// NumRequests returns the number of messages that have been parsed and
// returned by a call to Next.
NumRequests() int
}
type jsonRequestParser struct {
dec *json.Decoder
unmarshaler jsonpb.Unmarshaler
requestCount int
}
// NewJSONRequestParser returns a RequestParser that reads data in JSON format
// from the given reader. The given resolver is used to assist with decoding of
// google.protobuf.Any messages.
//
// Input data that contains more than one message should just include all
// messages concatenated (though whitespace is necessary to separate some kinds
// of values in JSON).
//
// If the given reader has no data, the returned parser will return io.EOF on
// the very first call.
func NewJSONRequestParser(in io.Reader, resolver jsonpb.AnyResolver) RequestParser {
return &jsonRequestParser{
dec: json.NewDecoder(in),
unmarshaler: jsonpb.Unmarshaler{AnyResolver: resolver},
}
}
func (f *jsonRequestParser) Next(m proto.Message) error {
var msg json.RawMessage
if err := f.dec.Decode(&msg); err != nil {
return err
}
f.requestCount++
return f.unmarshaler.Unmarshal(bytes.NewReader(msg), m)
}
func (f *jsonRequestParser) NumRequests() int {
return f.requestCount
}
const (
textSeparatorChar = 0x1e
)
type textRequestParser struct {
r *bufio.Reader
err error
requestCount int
}
// NewTextRequestParser returns a RequestParser that reads data in the protobuf
// text format from the given reader.
//
// Input data that contains more than one message should include an ASCII
// 'Record Separator' character (0x1E) between each message.
//
// Empty text is a valid text format and represents an empty message. So if the
// given reader has no data, the returned parser will yield an empty message
// for the first call to Next and then return io.EOF thereafter. This also means
// that if the input data ends with a record separator, then a final empty
// message will be parsed *after* the separator.
func NewTextRequestParser(in io.Reader) RequestParser {
return &textRequestParser{r: bufio.NewReader(in)}
}
func (f *textRequestParser) Next(m proto.Message) error {
if f.err != nil {
return f.err
}
var b []byte
b, f.err = f.r.ReadBytes(textSeparatorChar)
if f.err != nil && f.err != io.EOF {
return f.err
}
// remove delimiter
if len(b) > 0 && b[len(b)-1] == textSeparatorChar {
b = b[:len(b)-1]
}
f.requestCount++
return proto.UnmarshalText(string(b), m)
}
func (f *textRequestParser) NumRequests() int {
return f.requestCount
}
// Formatter translates messages into string representations.
type Formatter func(proto.Message) (string, error)
// NewJSONFormatter returns a formatter that returns JSON strings. The JSON will
// include empty/default values (instead of just omitted them) if emitDefaults
// is true. The given resolver is used to assist with encoding of
// google.protobuf.Any messages.
func NewJSONFormatter(emitDefaults bool, resolver jsonpb.AnyResolver) Formatter {
marshaler := jsonpb.Marshaler{
EmitDefaults: emitDefaults,
Indent: " ",
AnyResolver: resolver,
}
return marshaler.MarshalToString
}
// NewTextFormatter returns a formatter that returns strings in the protobuf
// text format. If includeSeparator is true then, when invoked to format
// multiple messages, all messages after the first one will be prefixed with the
// ASCII 'Record Separator' character (0x1E).
func NewTextFormatter(includeSeparator bool) Formatter {
tf := textFormatter{useSeparator: includeSeparator}
return tf.format
}
type textFormatter struct {
useSeparator bool
numFormatted int
}
var protoTextMarshaler = proto.TextMarshaler{ExpandAny: true}
func (tf *textFormatter) format(m proto.Message) (string, error) {
var buf bytes.Buffer
if tf.useSeparator && tf.numFormatted > 0 {
if err := buf.WriteByte(textSeparatorChar); err != nil {
return "", err
}
}
// If message implements MarshalText method (such as a *dynamic.Message),
// it won't get details about whether or not to format to text compactly
// or with indentation. So first see if the message also implements a
// MarshalTextIndent method and use that instead if available.
type indentMarshaler interface {
MarshalTextIndent() ([]byte, error)
}
if indenter, ok := m.(indentMarshaler); ok {
b, err := indenter.MarshalTextIndent()
if err != nil {
return "", err
}
if _, err := buf.Write(b); err != nil {
return "", err
}
} else if err := protoTextMarshaler.Marshal(&buf, m); err != nil {
return "", err
}
// no trailing newline needed
str := buf.String()
if str[len(str)-1] == '\n' {
str = str[:len(str)-1]
}
tf.numFormatted++
return str, nil
}
type Format string
const (
FormatJSON = Format("json")
FormatText = Format("text")
)
// AnyResolverFromDescriptorSource returns an AnyResolver that will search for
// types using the given descriptor source.
func AnyResolverFromDescriptorSource(source DescriptorSource) jsonpb.AnyResolver {
return &anyResolver{source: source}
}
// AnyResolverFromDescriptorSourceWithFallback returns an AnyResolver that will
// search for types using the given descriptor source and then fallback to a
// special message if the type is not found. The fallback type will render to
// JSON with a "@type" property, just like an Any message, but also with a
// custom "@value" property that includes the binary encoded payload.
func AnyResolverFromDescriptorSourceWithFallback(source DescriptorSource) jsonpb.AnyResolver {
res := anyResolver{source: source}
return &anyResolverWithFallback{AnyResolver: &res}
}
type anyResolver struct {
source DescriptorSource
er dynamic.ExtensionRegistry
mu sync.RWMutex
mf *dynamic.MessageFactory
resolved map[string]func() proto.Message
}
func (r *anyResolver) Resolve(typeUrl string) (proto.Message, error) {
mname := typeUrl
if slash := strings.LastIndex(mname, "/"); slash >= 0 {
mname = mname[slash+1:]
}
r.mu.RLock()
factory := r.resolved[mname]
r.mu.RUnlock()
// already resolved?
if factory != nil {
return factory(), nil
}
r.mu.Lock()
defer r.mu.Unlock()
// double-check, in case we were racing with another goroutine
// that resolved this one
factory = r.resolved[mname]
if factory != nil {
return factory(), nil
}
// use descriptor source to resolve message type
d, err := r.source.FindSymbol(mname)
if err != nil {
return nil, err
}
md, ok := d.(*desc.MessageDescriptor)
if !ok {
return nil, fmt.Errorf("unknown message: %s", typeUrl)
}
// populate any extensions for this message, too
if exts, err := r.source.AllExtensionsForType(mname); err != nil {
return nil, err
} else if err := r.er.AddExtension(exts...); err != nil {
return nil, err
}
if r.mf == nil {
r.mf = dynamic.NewMessageFactoryWithExtensionRegistry(&r.er)
}
factory = func() proto.Message {
return r.mf.NewMessage(md)
}
if r.resolved == nil {
r.resolved = map[string]func() proto.Message{}
}
r.resolved[mname] = factory
return factory(), nil
}
// anyResolverWithFallback can provide a fallback value for unknown
// messages that will format itself to JSON using an "@value" field
// that has the base64-encoded data for the unknown message value.
type anyResolverWithFallback struct {
jsonpb.AnyResolver
}
func (r anyResolverWithFallback) Resolve(typeUrl string) (proto.Message, error) {
msg, err := r.AnyResolver.Resolve(typeUrl)
if err == nil {
return msg, err
}
// Try "default" resolution logic. This mirrors the default behavior
// of jsonpb, which checks to see if the given message name is registered
// in the proto package.
mname := typeUrl
if slash := strings.LastIndex(mname, "/"); slash >= 0 {
mname = mname[slash+1:]
}
mt := proto.MessageType(mname)
if mt != nil {
return reflect.New(mt.Elem()).Interface().(proto.Message), nil
}
// finally, fallback to a special placeholder that can marshal itself
// to JSON using a special "@value" property to show base64-encoded
// data for the embedded message
return &unknownAny{TypeUrl: typeUrl, Error: fmt.Sprintf("%s is not recognized; see @value for raw binary message data", mname)}, nil
}
type unknownAny struct {
TypeUrl string `json:"@type"`
Error string `json:"@error"`
Value string `json:"@value"`
}
func (a *unknownAny) MarshalJSONPB(jsm *jsonpb.Marshaler) ([]byte, error) {
if jsm.Indent != "" {
return json.MarshalIndent(a, "", jsm.Indent)
}
return json.Marshal(a)
}
func (a *unknownAny) Unmarshal(b []byte) error {
a.Value = base64.StdEncoding.EncodeToString(b)
return nil
}
func (a *unknownAny) Reset() {
a.Value = ""
}
func (a *unknownAny) String() string {
b, err := a.MarshalJSONPB(&jsonpb.Marshaler{})
if err != nil {
return fmt.Sprintf("ERROR: %v", err.Error())
}
return string(b)
}
func (a *unknownAny) ProtoMessage() {
}
var _ proto.Message = (*unknownAny)(nil)
// RequestParserAndFormatterFor returns a request parser and formatter for the
// given format. The given descriptor source may be used for parsing message
// data (if needed by the format). The flags emitJSONDefaultFields and
// includeTextSeparator are options for JSON and protobuf text formats,
// respectively. Requests will be parsed from the given in.
func RequestParserAndFormatterFor(format Format, descSource DescriptorSource, emitJSONDefaultFields, includeTextSeparator bool, in io.Reader) (RequestParser, Formatter, error) {
switch format {
case FormatJSON:
resolver := AnyResolverFromDescriptorSource(descSource)
return NewJSONRequestParser(in, resolver), NewJSONFormatter(emitJSONDefaultFields, anyResolverWithFallback{AnyResolver: resolver}), nil
case FormatText:
return NewTextRequestParser(in), NewTextFormatter(includeTextSeparator), nil
default:
return nil, nil, fmt.Errorf("unknown format: %s", format)
}
}
// DefaultEventHandler logs events to a writer. This is not thread-safe, but is
// safe for use with InvokeRPC as long as NumResponses and Status are not read
// until the call to InvokeRPC completes.
type DefaultEventHandler struct {
out io.Writer
descSource DescriptorSource
formatter func(proto.Message) (string, error)
verbose bool
// NumResponses is the number of responses that have been received.
NumResponses int
// Status is the status that was received at the end of an RPC. It is
// nil if the RPC is still in progress.
Status *status.Status
}
// NewDefaultEventHandler returns an InvocationEventHandler that logs events to
// the given output. If verbose is true, all events are logged. Otherwise, only
// response messages are logged.
func NewDefaultEventHandler(out io.Writer, descSource DescriptorSource, formatter Formatter, verbose bool) *DefaultEventHandler {
return &DefaultEventHandler{
out: out,
descSource: descSource,
formatter: formatter,
verbose: verbose,
}
}
var _ InvocationEventHandler = (*DefaultEventHandler)(nil)
func (h *DefaultEventHandler) OnResolveMethod(md *desc.MethodDescriptor) {
if h.verbose {
txt, err := GetDescriptorText(md, h.descSource)
if err == nil {
fmt.Fprintf(h.out, "\nResolved method descriptor:\n%s\n", txt)
}
}
}
func (h *DefaultEventHandler) OnSendHeaders(md metadata.MD) {
if h.verbose {
fmt.Fprintf(h.out, "\nRequest metadata to send:\n%s\n", MetadataToString(md))
}
}
func (h *DefaultEventHandler) OnReceiveHeaders(md metadata.MD) {
if h.verbose {
fmt.Fprintf(h.out, "\nResponse headers received:\n%s\n", MetadataToString(md))
}
}
func (h *DefaultEventHandler) OnReceiveResponse(resp proto.Message) {
h.NumResponses++
if h.verbose {
fmt.Fprint(h.out, "\nResponse contents:\n")
}
if respStr, err := h.formatter(resp); err != nil {
fmt.Fprintf(h.out, "Failed to format response message %d: %v\n", h.NumResponses, err)
} else {
fmt.Fprintln(h.out, respStr)
}
}
func (h *DefaultEventHandler) OnReceiveTrailers(stat *status.Status, md metadata.MD) {
h.Status = stat
if h.verbose {
fmt.Fprintf(h.out, "\nResponse trailers received:\n%s\n", MetadataToString(md))
}
}
// PrintStatus prints details about the given status to the given writer. The given
// formatter is used to print any detail messages that may be included in the status.
// If the given status has a code of OK, "OK" is printed and that is all. Otherwise,
// "ERROR:" is printed along with a line showing the code, one showing the message
// string, and each detail message if any are present. The detail messages will be
// printed as proto text format or JSON, depending on the given formatter.
func PrintStatus(w io.Writer, stat *status.Status, formatter Formatter) {
if stat.Code() == codes.OK {
fmt.Fprintln(w, "OK")
return
}
fmt.Fprintf(w, "ERROR:\n Code: %s\n Message: %s\n", stat.Code().String(), stat.Message())
statpb := stat.Proto()
if len(statpb.Details) > 0 {
fmt.Fprintf(w, " Details:\n")
for i, det := range statpb.Details {
prefix := fmt.Sprintf(" %d)", i+1)
fmt.Fprintf(w, "%s\t", prefix)
prefix = strings.Repeat(" ", len(prefix)) + "\t"
output, err := formatter(det)
if err != nil {
fmt.Fprintf(w, "Error parsing detail message: %v\n", err)
} else {
lines := strings.Split(output, "\n")
for i, line := range lines {
if i == 0 {
// first line is already indented
fmt.Fprintf(w, "%s\n", line)
} else {
fmt.Fprintf(w, "%s%s\n", prefix, line)
}
}
}
}
}
}

297
format_test.go Normal file
View File

@@ -0,0 +1,297 @@
package grpcurl
import (
"bytes"
"fmt"
"io"
"strings"
"testing"
"github.com/golang/protobuf/jsonpb"
"github.com/golang/protobuf/proto"
"github.com/golang/protobuf/ptypes/struct"
"github.com/jhump/protoreflect/desc"
"google.golang.org/grpc/metadata"
)
func TestRequestParser(t *testing.T) {
source, err := DescriptorSourceFromProtoSets("testing/example.protoset")
if err != nil {
t.Fatalf("failed to create descriptor source: %v", err)
}
msg, err := makeProto()
if err != nil {
t.Fatalf("failed to create message: %v", err)
}
testCases := []struct {
format Format
input string
expectedOutput []proto.Message
}{
{
format: FormatJSON,
input: "",
},
{
format: FormatJSON,
input: messageAsJSON,
expectedOutput: []proto.Message{msg},
},
{
format: FormatJSON,
input: messageAsJSON + messageAsJSON + messageAsJSON,
expectedOutput: []proto.Message{msg, msg, msg},
},
{
// unlike JSON, empty input yields one empty message (vs. zero messages)
format: FormatText,
input: "",
expectedOutput: []proto.Message{&structpb.Value{}},
},
{
format: FormatText,
input: messageAsText,
expectedOutput: []proto.Message{msg},
},
{
format: FormatText,
input: messageAsText + string(textSeparatorChar),
expectedOutput: []proto.Message{msg, &structpb.Value{}},
},
{
format: FormatText,
input: messageAsText + string(textSeparatorChar) + messageAsText + string(textSeparatorChar) + messageAsText,
expectedOutput: []proto.Message{msg, msg, msg},
},
}
for i, tc := range testCases {
name := fmt.Sprintf("#%d, %s, %d message(s)", i+1, tc.format, len(tc.expectedOutput))
rf, _, err := RequestParserAndFormatterFor(tc.format, source, false, false, strings.NewReader(tc.input))
if err != nil {
t.Errorf("Failed to create parser and formatter: %v", err)
continue
}
numReqs := 0
for {
var req structpb.Value
err := rf.Next(&req)
if err == io.EOF {
break
} else if err != nil {
t.Errorf("%s, msg %d: unexpected error: %v", name, numReqs, err)
}
if !proto.Equal(&req, tc.expectedOutput[numReqs]) {
t.Errorf("%s, msg %d: incorrect message;\nexpecting:\n%v\ngot:\n%v", name, numReqs, tc.expectedOutput[numReqs], &req)
}
numReqs++
}
if rf.NumRequests() != numReqs {
t.Errorf("%s: factory reported wrong number of requests: expecting %d, got %d", name, numReqs, rf.NumRequests())
}
}
}
// Handler prints response data (and headers/trailers in verbose mode).
// This verifies that we get the right output in both JSON and proto text modes.
func TestHandler(t *testing.T) {
source, err := DescriptorSourceFromProtoSets("testing/example.protoset")
if err != nil {
t.Fatalf("failed to create descriptor source: %v", err)
}
d, err := source.FindSymbol("TestService.GetFiles")
if err != nil {
t.Fatalf("failed to find method 'TestService.GetFiles': %v", err)
}
md, ok := d.(*desc.MethodDescriptor)
if !ok {
t.Fatalf("wrong kind of descriptor found: %T", d)
}
reqHeaders := metadata.Pairs("foo", "123", "bar", "456")
respHeaders := metadata.Pairs("foo", "abc", "bar", "def", "baz", "xyz")
respTrailers := metadata.Pairs("a", "1", "b", "2", "c", "3")
rsp, err := makeProto()
if err != nil {
t.Fatalf("failed to create response message: %v", err)
}
for _, format := range []Format{FormatJSON, FormatText} {
for _, numMessages := range []int{1, 3} {
for _, verbose := range []bool{true, false} {
name := fmt.Sprintf("%s, %d message(s)", format, numMessages)
if verbose {
name += ", verbose"
}
_, formatter, err := RequestParserAndFormatterFor(format, source, false, !verbose, nil)
if err != nil {
t.Errorf("Failed to create parser and formatter: %v", err)
continue
}
var buf bytes.Buffer
h := NewDefaultEventHandler(&buf, source, formatter, verbose)
h.OnResolveMethod(md)
h.OnSendHeaders(reqHeaders)
h.OnReceiveHeaders(respHeaders)
for i := 0; i < numMessages; i++ {
h.OnReceiveResponse(rsp)
}
h.OnReceiveTrailers(nil, respTrailers)
expectedOutput := ""
if verbose {
expectedOutput += verbosePrefix
}
for i := 0; i < numMessages; i++ {
if verbose {
expectedOutput += verboseResponseHeader
}
if format == "json" {
expectedOutput += messageAsJSON
} else {
if i > 0 && !verbose {
expectedOutput += string(textSeparatorChar)
}
expectedOutput += messageAsText
}
}
if verbose {
expectedOutput += verboseSuffix
}
out := buf.String()
if !compare(out, expectedOutput) {
t.Errorf("%s: Incorrect output. Expected:\n%s\nGot:\n%s", name, expectedOutput, out)
}
}
}
}
}
// compare checks that actual and expected are equal, returning true if so.
// A simple equality check (==) does not suffice because jsonpb formats
// structpb.Value strangely. So if that formatting gets fixed, we don't
// want this test in grpcurl to suddenly start failing. So we check each
// line and compare the lines after stripping whitespace (which removes
// the jsonpb format anomalies).
func compare(actual, expected string) bool {
actualLines := strings.Split(actual, "\n")
expectedLines := strings.Split(expected, "\n")
if len(actualLines) != len(expectedLines) {
return false
}
for i := 0; i < len(actualLines); i++ {
if strings.TrimSpace(actualLines[i]) != strings.TrimSpace(expectedLines[i]) {
return false
}
}
return true
}
func makeProto() (proto.Message, error) {
var rsp structpb.Value
err := jsonpb.UnmarshalString(`{
"foo": ["abc", "def", "ghi"],
"bar": { "a": 1, "b": 2 },
"baz": true,
"null": null
}`, &rsp)
if err != nil {
return nil, err
}
return &rsp, nil
}
var (
verbosePrefix = `
Resolved method descriptor:
rpc GetFiles ( .TestRequest ) returns ( .TestResponse );
Request metadata to send:
bar: 456
foo: 123
Response headers received:
bar: def
baz: xyz
foo: abc
`
verboseSuffix = `
Response trailers received:
a: 1
b: 2
c: 3
`
verboseResponseHeader = `
Response contents:
`
messageAsJSON = `{
"bar": {
"a": 1,
"b": 2
},
"baz": true,
"foo": [
"abc",
"def",
"ghi"
],
"null": null
}
`
messageAsText = `struct_value: <
fields: <
key: "bar"
value: <
struct_value: <
fields: <
key: "a"
value: <
number_value: 1
>
>
fields: <
key: "b"
value: <
number_value: 2
>
>
>
>
>
fields: <
key: "baz"
value: <
bool_value: true
>
>
fields: <
key: "foo"
value: <
list_value: <
values: <
string_value: "abc"
>
values: <
string_value: "def"
>
values: <
string_value: "ghi"
>
>
>
>
fields: <
key: "null"
value: <
null_value: NULL_VALUE
>
>
>
`
)

10
go.mod
View File

@@ -1,8 +1,10 @@
module github.com/fullstorydev/grpcurl
go 1.14
require (
github.com/golang/protobuf v1.1.0
github.com/jhump/protoreflect v1.0.0
golang.org/x/net v0.0.0-20180530234432-1e491301e022
google.golang.org/grpc v1.12.0
github.com/golang/protobuf v1.3.5
github.com/jhump/protoreflect v1.5.0
golang.org/x/net v0.0.0-20190311183353-d8887717615a
google.golang.org/grpc v1.28.0
)

70
go.sum
View File

@@ -1,13 +1,65 @@
github.com/golang/protobuf v1.1.0 h1:0iH4Ffd/meGoXqF2lSAhZHt8X+cPgkfn/cb6Cce5Vpc=
github.com/golang/protobuf v1.1.0/go.mod h1:6lQm79b+lXiMfvg/cZm0SGofjICqVBUtrP5yJMmIC1U=
github.com/jhump/protoreflect v1.0.0 h1:l94KtQ6gRI3ouKVcXNdofCQJWoHATzcI6tDizOgUaf0=
github.com/jhump/protoreflect v1.0.0/go.mod h1:kG/zRVeS2M91gYaCvvUbPkMjjtFQS4qqjcPFzFkh2zE=
golang.org/x/net v0.0.0-20180530234432-1e491301e022 h1:MVYFTUmVD3/+ERcvRRI+P/C2+WOUimXh+Pd8LVsklZ4=
cloud.google.com/go v0.26.0 h1:e0WKqKTd5BnrG8aKH3J3h+QvEIQtSUcf2n5UZ5ZgLtQ=
cloud.google.com/go v0.26.0/go.mod h1:aQUYkXzVsufM+DwF1aE+0xfcU+56JwCaLick0ClmMTw=
github.com/BurntSushi/toml v0.3.1/go.mod h1:xHWCNGjB5oqiDr8zfno3MHue2Ht5sIBksp03qcyfWMU=
github.com/census-instrumentation/opencensus-proto v0.2.1/go.mod h1:f6KPmirojxKA12rnyqOA5BBL4O983OfeGPqjHWSTneU=
github.com/client9/misspell v0.3.4/go.mod h1:qj6jICC3Q7zFZvVWo7KLAzC3yx5G7kyvSDkc90ppPyw=
github.com/cncf/udpa/go v0.0.0-20191209042840-269d4d468f6f h1:WBZRG4aNOuI15bLRrCgN8fCq8E5Xuty6jGbmSNEvSsU=
github.com/cncf/udpa/go v0.0.0-20191209042840-269d4d468f6f/go.mod h1:M8M6+tZqaGXZJjfX53e64911xZQV5JYwmTeXPW+k8Sc=
github.com/envoyproxy/go-control-plane v0.9.0/go.mod h1:YTl/9mNaCwkRvm6d1a2C3ymFceY/DCBVvsKhRF0iEA4=
github.com/envoyproxy/go-control-plane v0.9.4 h1:rEvIZUSZ3fx39WIi3JkQqQBitGwpELBIYWeBVh6wn+E=
github.com/envoyproxy/go-control-plane v0.9.4/go.mod h1:6rpuAdCZL397s3pYoYcLgu1mIlRU8Am5FuJP05cCM98=
github.com/envoyproxy/protoc-gen-validate v0.1.0 h1:EQciDnbrYxy13PgWoY8AqoxGiPrpgBZ1R8UNe3ddc+A=
github.com/envoyproxy/protoc-gen-validate v0.1.0/go.mod h1:iSmxcyjqTsJpI2R4NaDN7+kN2VEUnK/pcBlmesArF7c=
github.com/golang/glog v0.0.0-20160126235308-23def4e6c14b/go.mod h1:SBH7ygxi8pfUlaOkMMuAQtPIUF8ecWP5IEl/CR7VP2Q=
github.com/golang/mock v1.1.1/go.mod h1:oTYuIxOrZwtPieC+H1uAHpcLFnEyAGVDL/k47Jfbm0A=
github.com/golang/protobuf v1.2.0/go.mod h1:6lQm79b+lXiMfvg/cZm0SGofjICqVBUtrP5yJMmIC1U=
github.com/golang/protobuf v1.3.1/go.mod h1:6lQm79b+lXiMfvg/cZm0SGofjICqVBUtrP5yJMmIC1U=
github.com/golang/protobuf v1.3.2/go.mod h1:6lQm79b+lXiMfvg/cZm0SGofjICqVBUtrP5yJMmIC1U=
github.com/golang/protobuf v1.3.3/go.mod h1:vzj43D7+SQXF/4pzW/hwtAqwc6iTitCiVSaWz5lYuqw=
github.com/golang/protobuf v1.3.5 h1:F768QJ1E9tib+q5Sc8MkdJi1RxLTbRcTf8LJV56aRls=
github.com/golang/protobuf v1.3.5/go.mod h1:6O5/vntMXwX2lRkT1hjjk0nAC1IDOTvTlVgjlRvqsdk=
github.com/google/go-cmp v0.2.0 h1:+dTQ8DZQJz0Mb/HjFlkptS1FeQ4cWSnN941F8aEG4SQ=
github.com/google/go-cmp v0.2.0/go.mod h1:oXzfMopK8JAjlY9xF4vHSVASa0yLyX7SntLO5aqRK0M=
github.com/jhump/protoreflect v1.5.0 h1:NgpVT+dX71c8hZnxHof2M7QDK7QtohIJ7DYycjnkyfc=
github.com/jhump/protoreflect v1.5.0/go.mod h1:eaTn3RZAmMBcV0fifFvlm6VHNz3wSkYyXYWUh7ymB74=
github.com/prometheus/client_model v0.0.0-20190812154241-14fe0d1b01d4/go.mod h1:xMI15A0UPsDsEKsMN9yxemIoYk6Tm2C1GtYGdfGttqA=
golang.org/x/crypto v0.0.0-20190308221718-c2843e01d9a2/go.mod h1:djNgcEr1/C05ACkg1iLfiJU5Ep61QUkGW8qpdssI0+w=
golang.org/x/exp v0.0.0-20190121172915-509febef88a4/go.mod h1:CJ0aWSM057203Lf6IL+f9T1iT9GByDxfZKAQTCR3kQA=
golang.org/x/lint v0.0.0-20181026193005-c67002cb31c3/go.mod h1:UVdnD1Gm6xHRNCYTkRU2/jEulfH38KcIWyp/GAMgvoE=
golang.org/x/lint v0.0.0-20190227174305-5b3e6a55c961/go.mod h1:wehouNa3lNwaWXcvxsM5YxQ5yQlVC4a0KAMCusXpPoU=
golang.org/x/lint v0.0.0-20190313153728-d0100b6bd8b3/go.mod h1:6SW0HCj/g11FgYtHlgUYUwCkIfeOF89ocIRzGO/8vkc=
golang.org/x/net v0.0.0-20180530234432-1e491301e022/go.mod h1:mL1N/T3taQHkDXs73rZJwtUhF3w3ftmwwsq0BUmARs4=
golang.org/x/net v0.0.0-20180724234803-3673e40ba225/go.mod h1:mL1N/T3taQHkDXs73rZJwtUhF3w3ftmwwsq0BUmARs4=
golang.org/x/net v0.0.0-20180826012351-8a410e7b638d/go.mod h1:mL1N/T3taQHkDXs73rZJwtUhF3w3ftmwwsq0BUmARs4=
golang.org/x/net v0.0.0-20190213061140-3a22650c66bd/go.mod h1:mL1N/T3taQHkDXs73rZJwtUhF3w3ftmwwsq0BUmARs4=
golang.org/x/net v0.0.0-20190311183353-d8887717615a h1:oWX7TPOiFAMXLq8o0ikBYfCJVlRHBcsciT5bXOrH628=
golang.org/x/net v0.0.0-20190311183353-d8887717615a/go.mod h1:t9HGtf8HONx5eT2rtn7q6eTqICYqUVnKs3thJo3Qplg=
golang.org/x/oauth2 v0.0.0-20180821212333-d2e6202438be h1:vEDujvNQGv4jgYKudGeI/+DAX4Jffq6hpD55MmoEvKs=
golang.org/x/oauth2 v0.0.0-20180821212333-d2e6202438be/go.mod h1:N/0e6XlmueqKjAGxoOufVs8QHGRruUQn6yWY3a++T0U=
golang.org/x/sync v0.0.0-20180314180146-1d60e4601c6f/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM=
golang.org/x/sync v0.0.0-20181108010431-42b317875d0f/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM=
golang.org/x/sync v0.0.0-20190423024810-112230192c58/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM=
golang.org/x/sys v0.0.0-20180830151530-49385e6e1522/go.mod h1:STP8DvDyc/dI5b8T5hshtkjS+E42TnysNCUPdjciGhY=
golang.org/x/sys v0.0.0-20190215142949-d0b11bdaac8a h1:1BGLXjeY4akVXGgbC9HugT3Jv3hCI0z56oJR5vAMgBU=
golang.org/x/sys v0.0.0-20190215142949-d0b11bdaac8a/go.mod h1:STP8DvDyc/dI5b8T5hshtkjS+E42TnysNCUPdjciGhY=
golang.org/x/text v0.3.0 h1:g61tztE5qeGQ89tm6NTjjM9VPIm088od1l6aSorWRWg=
golang.org/x/text v0.3.0/go.mod h1:NqM8EUOU14njkJ3fqMW+pc6Ldnwhi/IjpwHt7yyuwOQ=
google.golang.org/genproto v0.0.0-20170818100345-ee236bd376b0 h1:jgaHBfsPDMBDKsth1hPtI1HcOyecWndWOFSGW21VgaM=
google.golang.org/genproto v0.0.0-20170818100345-ee236bd376b0/go.mod h1:JiN7NxoALGmiZfu7CAH4rXhgtRTLTxftemlI0sWmxmc=
golang.org/x/tools v0.0.0-20190114222345-bf090417da8b/go.mod h1:n7NCudcB/nEzxVGmLbDWY5pfWTLqBcC2KZ6jyYvM4mQ=
golang.org/x/tools v0.0.0-20190226205152-f727befe758c/go.mod h1:9Yl7xja0Znq3iFh3HoIrodX9oNMXvdceNzlUR8zjMvY=
golang.org/x/tools v0.0.0-20190311212946-11955173bddd/go.mod h1:LCzVGOaR6xXOjkQ3onu1FJEFr0SW1gC7cKk1uF8kGRs=
golang.org/x/tools v0.0.0-20190524140312-2c0ae7006135/go.mod h1:RgjU9mgBXZiqYHBnxXauZ1Gv1EHHAz9KjViQ78xBX0Q=
google.golang.org/appengine v1.1.0/go.mod h1:EbEs0AVv82hx2wNQdGPgUI5lhzA/G0D9YwlJXL52JkM=
google.golang.org/appengine v1.4.0 h1:/wp5JvzpHIxhs/dumFmF7BXTf3Z+dd4uXta4kVyO508=
google.golang.org/appengine v1.4.0/go.mod h1:xpcJRLb0r/rnEns0DIKYYv+WjYCduHsrkT7/EB5XEv4=
google.golang.org/genproto v0.0.0-20170818010345-ee236bd376b0/go.mod h1:JiN7NxoALGmiZfu7CAH4rXhgtRTLTxftemlI0sWmxmc=
google.golang.org/genproto v0.0.0-20180817151627-c66870c02cf8/go.mod h1:JiN7NxoALGmiZfu7CAH4rXhgtRTLTxftemlI0sWmxmc=
google.golang.org/genproto v0.0.0-20190819201941-24fa4b261c55 h1:gSJIx1SDwno+2ElGhA4+qG2zF97qiUzTM+rQ0klBOcE=
google.golang.org/genproto v0.0.0-20190819201941-24fa4b261c55/go.mod h1:DMBHOl98Agz4BDEuKkezgsaosCRResVns1a3J2ZsMNc=
google.golang.org/grpc v1.8.0/go.mod h1:yo6s7OP7yaDglbqo1J04qKzAhqBH6lvTonzMVmEdcZw=
google.golang.org/grpc v1.12.0 h1:Mm8atZtkT+P6R43n/dqNDWkPPu5BwRVu/1rJnJCeZH8=
google.golang.org/grpc v1.12.0/go.mod h1:yo6s7OP7yaDglbqo1J04qKzAhqBH6lvTonzMVmEdcZw=
google.golang.org/grpc v1.19.0/go.mod h1:mqu4LbDTu4XGKhr4mRzUsmM4RtVoemTSY81AxZiDr8c=
google.golang.org/grpc v1.23.0/go.mod h1:Y5yQAOtifL1yxbo5wqy6BxZv8vAUGQwXBOALyacEbxg=
google.golang.org/grpc v1.25.1/go.mod h1:c3i+UQWmh7LiEpx4sFZnkU36qjEYZ0imhYfXVyQciAY=
google.golang.org/grpc v1.28.0 h1:bO/TA4OxCOummhSf10siHuG7vJOiwh7SpRpFZDkOgl4=
google.golang.org/grpc v1.28.0/go.mod h1:rpkK4SK4GF4Ach/+MFLZUBavHOvF2JJB5uozKKal+60=
honnef.co/go/tools v0.0.0-20190102054323-c2f93a96b099/go.mod h1:rf3lG4BRIbNafJWhAfAdb/ePZxsR/4RtNHQocxwk9r4=
honnef.co/go/tools v0.0.0-20190523083050-ea95bdfd59fc/go.mod h1:rf3lG4BRIbNafJWhAfAdb/ePZxsR/4RtNHQocxwk9r4=

File diff suppressed because it is too large Load Diff

View File

@@ -1,6 +1,7 @@
package grpcurl_test
import (
"encoding/json"
"fmt"
"io"
"net"
@@ -25,6 +26,7 @@ import (
. "github.com/fullstorydev/grpcurl"
grpcurl_testing "github.com/fullstorydev/grpcurl/testing"
jsonpbtest "github.com/fullstorydev/grpcurl/testing/jsonpb_test_proto"
)
var (
@@ -44,6 +46,10 @@ type descSourceCase struct {
includeRefl bool
}
// NB: These tests intentionally use the deprecated InvokeRpc since that
// calls the other (non-deprecated InvokeRPC). That allows the tests to
// easily exercise both functions.
func TestMain(m *testing.M) {
var err error
sourceProtoset, err = DescriptorSourceFromProtoSets("testing/test.protoset")
@@ -197,12 +203,12 @@ func doTestListMethods(t *testing.T, source DescriptorSource, includeReflection
t.Fatalf("failed to list methods for TestService: %v", err)
}
expected := []string{
"EmptyCall",
"FullDuplexCall",
"HalfDuplexCall",
"StreamingInputCall",
"StreamingOutputCall",
"UnaryCall",
"grpc.testing.TestService.EmptyCall",
"grpc.testing.TestService.FullDuplexCall",
"grpc.testing.TestService.HalfDuplexCall",
"grpc.testing.TestService.StreamingInputCall",
"grpc.testing.TestService.StreamingOutputCall",
"grpc.testing.TestService.UnaryCall",
}
if !reflect.DeepEqual(expected, names) {
t.Errorf("ListMethods returned wrong results: wanted %v, got %v", expected, names)
@@ -214,7 +220,7 @@ func doTestListMethods(t *testing.T, source DescriptorSource, includeReflection
if err != nil {
t.Fatalf("failed to list methods for ServerReflection: %v", err)
}
expected = []string{"ServerReflectionInfo"}
expected = []string{"grpc.reflection.v1alpha.ServerReflection.ServerReflectionInfo"}
} else {
// without reflection, we see all services defined in the same test.proto file, which is the
// TestService as well as UnimplementedService
@@ -222,7 +228,7 @@ func doTestListMethods(t *testing.T, source DescriptorSource, includeReflection
if err != nil {
t.Fatalf("failed to list methods for ServerReflection: %v", err)
}
expected = []string{"UnimplementedCall"}
expected = []string{"grpc.testing.UnimplementedService.UnimplementedCall"}
}
if !reflect.DeepEqual(expected, names) {
t.Errorf("ListMethods returned wrong results: wanted %v, got %v", expected, names)
@@ -235,6 +241,145 @@ func doTestListMethods(t *testing.T, source DescriptorSource, includeReflection
}
}
func TestGetAllFiles(t *testing.T) {
expectedFiles := []string{"testing/test.proto"}
// server reflection picks up filename from linked in Go package,
// which indicates "grpc_testing/test.proto", not our local copy.
expectedFilesWithReflection := []string{"grpc_reflection_v1alpha/reflection.proto", "grpc_testing/test.proto"}
for _, ds := range descSources {
t.Run(ds.name, func(t *testing.T) {
files, err := GetAllFiles(ds.source)
if err != nil {
t.Fatalf("failed to get all files: %v", err)
}
names := fileNames(files)
expected := expectedFiles
if ds.includeRefl {
expected = expectedFilesWithReflection
}
if !reflect.DeepEqual(expected, names) {
t.Errorf("GetAllFiles returned wrong results: wanted %v, got %v", expected, names)
}
})
}
// try cases with more complicated set of files
otherSourceProtoset, err := DescriptorSourceFromProtoSets("testing/test.protoset", "testing/example.protoset")
if err != nil {
t.Fatal(err.Error())
}
otherSourceProtoFiles, err := DescriptorSourceFromProtoFiles(nil, "testing/test.proto", "testing/example.proto")
if err != nil {
t.Fatal(err.Error())
}
otherDescSources := []descSourceCase{
{"protoset[b]", otherSourceProtoset, false},
{"proto[b]", otherSourceProtoFiles, false},
}
expectedFiles = []string{
"google/protobuf/any.proto",
"google/protobuf/descriptor.proto",
"google/protobuf/empty.proto",
"google/protobuf/timestamp.proto",
"testing/example.proto",
"testing/example2.proto",
"testing/test.proto",
}
for _, ds := range otherDescSources {
t.Run(ds.name, func(t *testing.T) {
files, err := GetAllFiles(ds.source)
if err != nil {
t.Fatalf("failed to get all files: %v", err)
}
names := fileNames(files)
if !reflect.DeepEqual(expectedFiles, names) {
t.Errorf("GetAllFiles returned wrong results: wanted %v, got %v", expectedFiles, names)
}
})
}
}
func TestExpandHeaders(t *testing.T) {
inHeaders := []string{"key1: ${value}", "key2: bar", "key3: ${woo", "key4: woo}", "key5: ${TEST}",
"key6: ${TEST_VAR}", "${TEST}: ${TEST_VAR}", "key8: ${EMPTY}"}
os.Setenv("value", "value")
os.Setenv("TEST", "value5")
os.Setenv("TEST_VAR", "value6")
os.Setenv("EMPTY", "")
expectedHeaders := map[string]bool{"key1: value": true, "key2: bar": true, "key3: ${woo": true, "key4: woo}": true,
"key5: value5": true, "key6: value6": true, "value5: value6": true, "key8: ": true}
outHeaders, err := ExpandHeaders(inHeaders)
if err != nil {
t.Errorf("The ExpandHeaders function generated an unexpected error %s", err)
}
for _, expandedHeader := range outHeaders {
if _, ok := expectedHeaders[expandedHeader]; !ok {
t.Errorf("The ExpandHeaders function has returned an unexpected header. Received unexpected header %s", expandedHeader)
}
}
badHeaders := []string{"key: ${DNE}"}
_, err = ExpandHeaders(badHeaders)
if err == nil {
t.Errorf("The ExpandHeaders function should return an error for missing environment variables %q", badHeaders)
}
}
func fileNames(files []*desc.FileDescriptor) []string {
names := make([]string, len(files))
for i, f := range files {
names[i] = f.GetName()
}
return names
}
const expectKnownType = `{
"dur": "0s",
"ts": "1970-01-01T00:00:00Z",
"dbl": 0,
"flt": 0,
"i64": "0",
"u64": "0",
"i32": 0,
"u32": 0,
"bool": false,
"str": "",
"bytes": null,
"st": {"google.protobuf.Struct": "supports arbitrary JSON objects"},
"an": {"@type": "type.googleapis.com/google.protobuf.Empty", "value": {}},
"lv": [{"google.protobuf.ListValue": "is an array of arbitrary JSON values"}],
"val": {"google.protobuf.Value": "supports arbitrary JSON"}
}`
func TestMakeTemplateKnownTypes(t *testing.T) {
descriptor, err := desc.LoadMessageDescriptorForMessage((*jsonpbtest.KnownTypes)(nil))
if err != nil {
t.Fatalf("failed to load descriptor: %v", err)
}
message := MakeTemplate(descriptor)
jsm := jsonpb.Marshaler{EmitDefaults: true}
out, err := jsm.MarshalToString(message)
if err != nil {
t.Fatalf("failed to marshal to JSON: %v", err)
}
// make sure template JSON matches expected
var actual, expected interface{}
if err := json.Unmarshal([]byte(out), &actual); err != nil {
t.Fatalf("failed to parse actual JSON: %v", err)
}
if err := json.Unmarshal([]byte(expectKnownType), &expected); err != nil {
t.Fatalf("failed to parse expected JSON: %v", err)
}
if !reflect.DeepEqual(actual, expected) {
t.Errorf("template message is not as expected; want:\n%s\ngot:\n%s", expectKnownType, out)
}
}
func TestDescribe(t *testing.T) {
for _, ds := range descSources {
t.Run(ds.name, func(t *testing.T) {

389
invoke.go Normal file
View File

@@ -0,0 +1,389 @@
package grpcurl
import (
"bytes"
"fmt"
"io"
"strings"
"sync"
"sync/atomic"
"github.com/golang/protobuf/jsonpb"
"github.com/golang/protobuf/proto"
"github.com/jhump/protoreflect/desc"
"github.com/jhump/protoreflect/dynamic"
"github.com/jhump/protoreflect/dynamic/grpcdynamic"
"github.com/jhump/protoreflect/grpcreflect"
"golang.org/x/net/context"
"google.golang.org/grpc"
"google.golang.org/grpc/codes"
"google.golang.org/grpc/metadata"
"google.golang.org/grpc/status"
)
// InvocationEventHandler is a bag of callbacks for handling events that occur in the course
// of invoking an RPC. The handler also provides request data that is sent. The callbacks are
// generally called in the order they are listed below.
type InvocationEventHandler interface {
// OnResolveMethod is called with a descriptor of the method that is being invoked.
OnResolveMethod(*desc.MethodDescriptor)
// OnSendHeaders is called with the request metadata that is being sent.
OnSendHeaders(metadata.MD)
// OnReceiveHeaders is called when response headers have been received.
OnReceiveHeaders(metadata.MD)
// OnReceiveResponse is called for each response message received.
OnReceiveResponse(proto.Message)
// OnReceiveTrailers is called when response trailers and final RPC status have been received.
OnReceiveTrailers(*status.Status, metadata.MD)
}
// RequestMessageSupplier is a function that is called to retrieve request
// messages for a GRPC operation. This type is deprecated and will be removed in
// a future release.
//
// Deprecated: This is only used with the deprecated InvokeRpc. Instead, use
// RequestSupplier with InvokeRPC.
type RequestMessageSupplier func() ([]byte, error)
// InvokeRpc uses the given gRPC connection to invoke the given method. This function is deprecated
// and will be removed in a future release. It just delegates to the similarly named InvokeRPC
// method, whose signature is only slightly different.
//
// Deprecated: use InvokeRPC instead.
func InvokeRpc(ctx context.Context, source DescriptorSource, cc *grpc.ClientConn, methodName string,
headers []string, handler InvocationEventHandler, requestData RequestMessageSupplier) error {
return InvokeRPC(ctx, source, cc, methodName, headers, handler, func(m proto.Message) error {
// New function is almost identical, but the request supplier function works differently.
// So we adapt the logic here to maintain compatibility.
data, err := requestData()
if err != nil {
return err
}
return jsonpb.Unmarshal(bytes.NewReader(data), m)
})
}
// RequestSupplier is a function that is called to populate messages for a gRPC operation. The
// function should populate the given message or return a non-nil error. If the supplier has no
// more messages, it should return io.EOF. When it returns io.EOF, it should not in any way
// modify the given message argument.
type RequestSupplier func(proto.Message) error
// InvokeRPC uses the given gRPC channel to invoke the given method. The given descriptor source
// is used to determine the type of method and the type of request and response message. The given
// headers are sent as request metadata. Methods on the given event handler are called as the
// invocation proceeds.
//
// The given requestData function supplies the actual data to send. It should return io.EOF when
// there is no more request data. If the method being invoked is a unary or server-streaming RPC
// (e.g. exactly one request message) and there is no request data (e.g. the first invocation of
// the function returns io.EOF), then an empty request message is sent.
//
// If the requestData function and the given event handler coordinate or share any state, they should
// be thread-safe. This is because the requestData function may be called from a different goroutine
// than the one invoking event callbacks. (This only happens for bi-directional streaming RPCs, where
// one goroutine sends request messages and another consumes the response messages).
func InvokeRPC(ctx context.Context, source DescriptorSource, ch grpcdynamic.Channel, methodName string,
headers []string, handler InvocationEventHandler, requestData RequestSupplier) error {
md := MetadataFromHeaders(headers)
svc, mth := parseSymbol(methodName)
if svc == "" || mth == "" {
return fmt.Errorf("given method name %q is not in expected format: 'service/method' or 'service.method'", methodName)
}
dsc, err := source.FindSymbol(svc)
if err != nil {
if isNotFoundError(err) {
return fmt.Errorf("target server does not expose service %q", svc)
}
return fmt.Errorf("failed to query for service descriptor %q: %v", svc, err)
}
sd, ok := dsc.(*desc.ServiceDescriptor)
if !ok {
return fmt.Errorf("target server does not expose service %q", svc)
}
mtd := sd.FindMethodByName(mth)
if mtd == nil {
return fmt.Errorf("service %q does not include a method named %q", svc, mth)
}
handler.OnResolveMethod(mtd)
// we also download any applicable extensions so we can provide full support for parsing user-provided data
var ext dynamic.ExtensionRegistry
alreadyFetched := map[string]bool{}
if err = fetchAllExtensions(source, &ext, mtd.GetInputType(), alreadyFetched); err != nil {
return fmt.Errorf("error resolving server extensions for message %s: %v", mtd.GetInputType().GetFullyQualifiedName(), err)
}
if err = fetchAllExtensions(source, &ext, mtd.GetOutputType(), alreadyFetched); err != nil {
return fmt.Errorf("error resolving server extensions for message %s: %v", mtd.GetOutputType().GetFullyQualifiedName(), err)
}
msgFactory := dynamic.NewMessageFactoryWithExtensionRegistry(&ext)
req := msgFactory.NewMessage(mtd.GetInputType())
handler.OnSendHeaders(md)
ctx = metadata.NewOutgoingContext(ctx, md)
stub := grpcdynamic.NewStubWithMessageFactory(ch, msgFactory)
ctx, cancel := context.WithCancel(ctx)
defer cancel()
if mtd.IsClientStreaming() && mtd.IsServerStreaming() {
return invokeBidi(ctx, stub, mtd, handler, requestData, req)
} else if mtd.IsClientStreaming() {
return invokeClientStream(ctx, stub, mtd, handler, requestData, req)
} else if mtd.IsServerStreaming() {
return invokeServerStream(ctx, stub, mtd, handler, requestData, req)
} else {
return invokeUnary(ctx, stub, mtd, handler, requestData, req)
}
}
func invokeUnary(ctx context.Context, stub grpcdynamic.Stub, md *desc.MethodDescriptor, handler InvocationEventHandler,
requestData RequestSupplier, req proto.Message) error {
err := requestData(req)
if err != nil && err != io.EOF {
return fmt.Errorf("error getting request data: %v", err)
}
if err != io.EOF {
// verify there is no second message, which is a usage error
err := requestData(req)
if err == nil {
return fmt.Errorf("method %q is a unary RPC, but request data contained more than 1 message", md.GetFullyQualifiedName())
} else if err != io.EOF {
return fmt.Errorf("error getting request data: %v", err)
}
}
// Now we can actually invoke the RPC!
var respHeaders metadata.MD
var respTrailers metadata.MD
resp, err := stub.InvokeRpc(ctx, md, req, grpc.Trailer(&respTrailers), grpc.Header(&respHeaders))
stat, ok := status.FromError(err)
if !ok {
// Error codes sent from the server will get printed differently below.
// So just bail for other kinds of errors here.
return fmt.Errorf("grpc call for %q failed: %v", md.GetFullyQualifiedName(), err)
}
handler.OnReceiveHeaders(respHeaders)
if stat.Code() == codes.OK {
handler.OnReceiveResponse(resp)
}
handler.OnReceiveTrailers(stat, respTrailers)
return nil
}
func invokeClientStream(ctx context.Context, stub grpcdynamic.Stub, md *desc.MethodDescriptor, handler InvocationEventHandler,
requestData RequestSupplier, req proto.Message) error {
// invoke the RPC!
str, err := stub.InvokeRpcClientStream(ctx, md)
// Upload each request message in the stream
var resp proto.Message
for err == nil {
err = requestData(req)
if err == io.EOF {
resp, err = str.CloseAndReceive()
break
}
if err != nil {
return fmt.Errorf("error getting request data: %v", err)
}
err = str.SendMsg(req)
if err == io.EOF {
// We get EOF on send if the server says "go away"
// We have to use CloseAndReceive to get the actual code
resp, err = str.CloseAndReceive()
break
}
req.Reset()
}
// finally, process response data
stat, ok := status.FromError(err)
if !ok {
// Error codes sent from the server will get printed differently below.
// So just bail for other kinds of errors here.
return fmt.Errorf("grpc call for %q failed: %v", md.GetFullyQualifiedName(), err)
}
if respHeaders, err := str.Header(); err == nil {
handler.OnReceiveHeaders(respHeaders)
}
if stat.Code() == codes.OK {
handler.OnReceiveResponse(resp)
}
handler.OnReceiveTrailers(stat, str.Trailer())
return nil
}
func invokeServerStream(ctx context.Context, stub grpcdynamic.Stub, md *desc.MethodDescriptor, handler InvocationEventHandler,
requestData RequestSupplier, req proto.Message) error {
err := requestData(req)
if err != nil && err != io.EOF {
return fmt.Errorf("error getting request data: %v", err)
}
if err != io.EOF {
// verify there is no second message, which is a usage error
err := requestData(req)
if err == nil {
return fmt.Errorf("method %q is a server-streaming RPC, but request data contained more than 1 message", md.GetFullyQualifiedName())
} else if err != io.EOF {
return fmt.Errorf("error getting request data: %v", err)
}
}
// Now we can actually invoke the RPC!
str, err := stub.InvokeRpcServerStream(ctx, md, req)
if respHeaders, err := str.Header(); err == nil {
handler.OnReceiveHeaders(respHeaders)
}
// Download each response message
for err == nil {
var resp proto.Message
resp, err = str.RecvMsg()
if err != nil {
if err == io.EOF {
err = nil
}
break
}
handler.OnReceiveResponse(resp)
}
stat, ok := status.FromError(err)
if !ok {
// Error codes sent from the server will get printed differently below.
// So just bail for other kinds of errors here.
return fmt.Errorf("grpc call for %q failed: %v", md.GetFullyQualifiedName(), err)
}
handler.OnReceiveTrailers(stat, str.Trailer())
return nil
}
func invokeBidi(ctx context.Context, stub grpcdynamic.Stub, md *desc.MethodDescriptor, handler InvocationEventHandler,
requestData RequestSupplier, req proto.Message) error {
ctx, cancel := context.WithCancel(ctx)
defer cancel()
// invoke the RPC!
str, err := stub.InvokeRpcBidiStream(ctx, md)
var wg sync.WaitGroup
var sendErr atomic.Value
defer wg.Wait()
if err == nil {
wg.Add(1)
go func() {
defer wg.Done()
// Concurrently upload each request message in the stream
var err error
for err == nil {
err = requestData(req)
if err == io.EOF {
err = str.CloseSend()
break
}
if err != nil {
err = fmt.Errorf("error getting request data: %v", err)
cancel()
break
}
err = str.SendMsg(req)
req.Reset()
}
if err != nil {
sendErr.Store(err)
}
}()
}
if respHeaders, err := str.Header(); err == nil {
handler.OnReceiveHeaders(respHeaders)
}
// Download each response message
for err == nil {
var resp proto.Message
resp, err = str.RecvMsg()
if err != nil {
if err == io.EOF {
err = nil
}
break
}
handler.OnReceiveResponse(resp)
}
if se, ok := sendErr.Load().(error); ok && se != io.EOF {
err = se
}
stat, ok := status.FromError(err)
if !ok {
// Error codes sent from the server will get printed differently below.
// So just bail for other kinds of errors here.
return fmt.Errorf("grpc call for %q failed: %v", md.GetFullyQualifiedName(), err)
}
handler.OnReceiveTrailers(stat, str.Trailer())
return nil
}
type notFoundError string
func notFound(kind, name string) error {
return notFoundError(fmt.Sprintf("%s not found: %s", kind, name))
}
func (e notFoundError) Error() string {
return string(e)
}
func isNotFoundError(err error) bool {
if grpcreflect.IsElementNotFoundError(err) {
return true
}
_, ok := err.(notFoundError)
return ok
}
func parseSymbol(svcAndMethod string) (string, string) {
pos := strings.LastIndex(svcAndMethod, "/")
if pos < 0 {
pos = strings.LastIndex(svcAndMethod, ".")
if pos < 0 {
return "", ""
}
}
return svcAndMethod[:pos], svcAndMethod[pos+1:]
}

View File

@@ -7,14 +7,17 @@ cd "$(dirname $0)"
# Run this script to generate files used by tests.
echo "Creating protosets..."
protoc ../../../google.golang.org/grpc/interop/grpc_testing/test.proto \
-I../../../ --include_imports \
protoc testing/test.proto \
--include_imports \
--descriptor_set_out=testing/test.protoset
protoc testing/example.proto \
--include_imports \
--descriptor_set_out=testing/example.protoset
protoc testing/jsonpb_test_proto/test_objects.proto \
--go_out=paths=source_relative:.
echo "Creating certs for TLS testing..."
if ! hash certstrap 2>/dev/null; then
# certstrap not found: try to install it

90
releasing/README.md Normal file
View File

@@ -0,0 +1,90 @@
# Releases of gRPCurl
This document provides instructions for building a release of `grpcurl`.
The release process consists of a handful of tasks:
1. Drop a release tag in git.
2. Build binaries for various platforms. This is done using the local `go` tool and uses `GOOS` and `GOARCH` environment variables to cross-compile for supported platforms.
3. Creates a release in GitHub, uploads the binaries, and creates provisional release notes (in the form of a change log).
4. Build a docker image for the new release.
5. Push the docker image to Docker Hub, with both a version tag and the "latest" tag.
6. Submits a PR to update the [Homebrew](https://brew.sh/) recipe with the latest version.
Most of this is automated via a script in this same directory. The main thing you will need is a GitHub personal access token, which will be used for creating the release in GitHub (so you need write access to the fullstorydev/grpcurl repo) and to open a Homebrew pull request.
## Creating a new release
So, to actually create a new release, just run the script in this directory.
First, you need a version number for the new release, following sem-ver format: `v<Major>.<Minor>.<Patch>`. Second, you need a personal access token for GitHub.
We'll use `v2.3.4` as an example version and `abcdef0123456789abcdef` as an example GitHub token:
```sh
# from the root of the repo
GITHUB_TOKEN=abcdef0123456789abcd \
./releasing/do-release.sh v2.3.4
```
Wasn't that easy! There is one last step: update the release notes in GitHub. By default, the script just records a change log of commit descriptions. Use that log (and, if necessary, drill into individual PRs included in the release) to flesh out notes in the format of the `RELEASE_NOTES.md` file _in this directory_. Then login to GitHub, go to the new release, edit the notes, and paste in the markdown you just wrote.
That should be all there is to it! If things go wrong and you have to re-do part of the process, see the sections below.
----
### GitHub Releases
The GitHub release is the first step performed by the `do-release.sh` script. So generally, if there is an issue with that step, you can re-try the whole script.
Note, if running the script did something wrong, you may have to first login to GitHub and remove uploaded artifacts for a botched release attempt. In general, this is _very undesirable_. Releases should usually be considered immutable. Instead of removing uploaded assets and providing new ones, it is often better to remove uploaded assets (to make bad binaries no longer available) and then _release a new patch version_. (You can edit the release notes for the botched version explaining why there are no artifacts for it.)
The steps to do a GitHub-only release (vs. running the entire script) are the following:
```sh
# from the root of the repo
git tag v2.3.4
GITHUB_TOKEN=abcdef0123456789abcdef \
GO111MODULE=on \
make release
```
The `git tag ...` step is necessary because the release target requires that the current SHA have a sem-ver tag. That's the version it will use when creating the release.
This will create the release in GitHub with provisional release notes that just include a change log of commit messages. You still need to login to GitHub and revise those notes to adhere to the recommended format. (See `RELEASE_NOTES.md` in this directory.)
### Docker Hub Releases
To re-run only the Docker Hub release steps, we need to build an image with the right tag and then push to Docker Hub.
```sh
# from the root of the repo
echo v2.3.4 > VERSION
docker build -t fullstorydev/grpcurl:v2.3.4 .
# now that we have it built, push to Docker Hub
docker push fullstorydev/grpcurl:v2.3.4
# push "latest" tag, too
docker tag fullstorydev/grpcurl:v2.3.4 fullstorydev/grpcurl:latest
docker push fullstorydev/grpcurl:latest
```
If the `docker push ...` steps fail, you may need to run `docker login`, enter your Docker Hub login credentials, and then try to push again.
### Homebrew Releases
The last step is to update the Homebrew recipe to use the latest version. First, we need to compute the SHA256 checksum for the source archive:
```sh
# download the source archive from GitHub
URL=https://github.com/fullstorydev/grpcurl/archive/v2.3.4.tar.gz
curl -L -o tmp.tgz $URL
# and compute the SHA
SHA="$(sha256sum < tmp.tgz | awk '{ print $1 }')"
```
To actually create the brew PR, you need your GitHub personal access token again, as well as the URL and SHA from the previous step:
```sh
HOMEBREW_GITHUB_API_TOKEN=abcdef0123456789abcdef \
brew bump-formula-pr --url $URL --sha256 $SHA grpcurl
```
This creates a PR to bump the formula to the new version. When this PR is merged by brew maintainers, the new version becomes available!

View File

@@ -0,0 +1,11 @@
## Changes
### Command-line tool
* _In this list, describe the changes to the command-line tool._
* _Use one bullet per change. Include both bug-fixes and improvements. Omit this section if there are no changes that impact the command-line tool._
### Go package "github.com/fullstorydev/grpcurl"
* _In this list, describe the changes to exported API in the main package in this repo: "github.com/fullstorydev/grpcurl". These will often be closely related to changes to the command-line tool, though not always: changes that only impact the cmd/grpcurl directory of this repo do not impact exported API._
* _Use one bullet per change. Include both bug-fixes and improvements. Omit this section if there are no changes that impact the exported API._

56
releasing/do-release.sh Executable file
View File

@@ -0,0 +1,56 @@
#!/bin/bash
# strict mode
set -euo pipefail
IFS=$'\n\t'
if [[ -z ${DRY_RUN:-} ]]; then
PREFIX=""
else
PREFIX="echo"
fi
# input validation
if [[ -z ${GITHUB_TOKEN:-} ]]; then
echo "GITHUB_TOKEN environment variable must be set before running." >&2
exit 1
fi
if [[ $# -ne 1 || $1 == "" ]]; then
echo "This program requires one argument: the version number, in 'vM.N.P' format." >&2
exit 1
fi
VERSION=$1
# Change to root of the repo
cd "$(dirname "$0")/.."
# GitHub release
$PREFIX git tag "$VERSION"
# make sure GITHUB_TOKEN is exported, for the benefit of this next command
export GITHUB_TOKEN
GO111MODULE=on $PREFIX make release
# if that was successful, it could have touched go.mod and go.sum, so revert those
$PREFIX git checkout go.mod go.sum
# Docker release
# make sure credentials are valid for later push steps; this might
# be interactive since this will prompt for username and password
# if there are no valid current credentials.
$PREFIX docker login
echo "$VERSION" > VERSION
$PREFIX docker build -t "fullstorydev/grpcurl:${VERSION}" .
rm VERSION
# push to docker hub, both the given version as a tag and for "latest" tag
$PREFIX docker push "fullstorydev/grpcurl:${VERSION}"
$PREFIX docker tag "fullstorydev/grpcurl:${VERSION}" fullstorydev/grpcurl:latest
$PREFIX docker push fullstorydev/grpcurl:latest
# Homebrew release
URL="https://github.com/fullstorydev/grpcurl/archive/${VERSION}.tar.gz"
curl -L -o tmp.tgz "$URL"
SHA="$(sha256sum < tmp.tgz | awk '{ print $1 }')"
rm tmp.tgz
HOMEBREW_GITHUB_API_TOKEN="$GITHUB_TOKEN" $PREFIX brew bump-formula-pr --url "$URL" --sha256 "$SHA" grpcurl

View File

@@ -38,6 +38,7 @@ func (a *accounts) openAccount(customer string, accountType Account_Type, initia
a.AccountNumbersByCustomer[customer] = accountNums
var acct account
acct.AccountNumber = num
acct.Type = accountType
acct.BalanceCents = initialBalanceCents
acct.Transactions = append(acct.Transactions, &Transaction{
AccountNumber: num,

View File

@@ -120,7 +120,7 @@ func unaryLogger(ctx context.Context, req interface{}, info *grpc.UnaryServerInf
} else {
code = codes.Unknown
}
grpclog.Infof("completed <%d>: %v (%d) %v\n", i, code, code, time.Now().Sub(start))
grpclog.Infof("completed <%d>: %v (%d) %v\n", i, code, code, time.Since(start))
return rsp, err
}
@@ -135,7 +135,7 @@ func streamLogger(srv interface{}, ss grpc.ServerStream, info *grpc.StreamServer
} else {
code = codes.Unknown
}
grpclog.Infof("completed <%d>: %v(%d) %v\n", i, code, code, time.Now().Sub(start))
grpclog.Infof("completed <%d>: %v(%d) %v\n", i, code, code, time.Since(start))
return err
}

View File

@@ -3,9 +3,11 @@ syntax = "proto3";
import "google/protobuf/descriptor.proto";
import "google/protobuf/empty.proto";
import "google/protobuf/timestamp.proto";
import "testing/example2.proto";
message TestRequest {
repeated string file_names = 1;
repeated Extension extensions = 2;
}
message TestResponse {

Binary file not shown.

8
testing/example2.proto Normal file
View File

@@ -0,0 +1,8 @@
syntax = "proto3";
import "google/protobuf/any.proto";
message Extension {
uint64 id = 1;
google.protobuf.Any data = 2;
}

View File

@@ -0,0 +1,215 @@
// Code generated by protoc-gen-go. DO NOT EDIT.
// source: testing/jsonpb_test_proto/test_objects.proto
package jsonpb
import (
fmt "fmt"
proto "github.com/golang/protobuf/proto"
any "github.com/golang/protobuf/ptypes/any"
duration "github.com/golang/protobuf/ptypes/duration"
_struct "github.com/golang/protobuf/ptypes/struct"
timestamp "github.com/golang/protobuf/ptypes/timestamp"
wrappers "github.com/golang/protobuf/ptypes/wrappers"
math "math"
)
// Reference imports to suppress errors if they are not otherwise used.
var _ = proto.Marshal
var _ = fmt.Errorf
var _ = math.Inf
// This is a compile-time assertion to ensure that this generated file
// is compatible with the proto package it is being compiled against.
// A compilation error at this line likely means your copy of the
// proto package needs to be updated.
const _ = proto.ProtoPackageIsVersion3 // please upgrade the proto package
type KnownTypes struct {
An *any.Any `protobuf:"bytes,14,opt,name=an" json:"an,omitempty"`
Dur *duration.Duration `protobuf:"bytes,1,opt,name=dur" json:"dur,omitempty"`
St *_struct.Struct `protobuf:"bytes,12,opt,name=st" json:"st,omitempty"`
Ts *timestamp.Timestamp `protobuf:"bytes,2,opt,name=ts" json:"ts,omitempty"`
Lv *_struct.ListValue `protobuf:"bytes,15,opt,name=lv" json:"lv,omitempty"`
Val *_struct.Value `protobuf:"bytes,16,opt,name=val" json:"val,omitempty"`
Dbl *wrappers.DoubleValue `protobuf:"bytes,3,opt,name=dbl" json:"dbl,omitempty"`
Flt *wrappers.FloatValue `protobuf:"bytes,4,opt,name=flt" json:"flt,omitempty"`
I64 *wrappers.Int64Value `protobuf:"bytes,5,opt,name=i64" json:"i64,omitempty"`
U64 *wrappers.UInt64Value `protobuf:"bytes,6,opt,name=u64" json:"u64,omitempty"`
I32 *wrappers.Int32Value `protobuf:"bytes,7,opt,name=i32" json:"i32,omitempty"`
U32 *wrappers.UInt32Value `protobuf:"bytes,8,opt,name=u32" json:"u32,omitempty"`
Bool *wrappers.BoolValue `protobuf:"bytes,9,opt,name=bool" json:"bool,omitempty"`
Str *wrappers.StringValue `protobuf:"bytes,10,opt,name=str" json:"str,omitempty"`
Bytes *wrappers.BytesValue `protobuf:"bytes,11,opt,name=bytes" json:"bytes,omitempty"`
XXX_NoUnkeyedLiteral struct{} `json:"-"`
XXX_unrecognized []byte `json:"-"`
XXX_sizecache int32 `json:"-"`
}
func (m *KnownTypes) Reset() { *m = KnownTypes{} }
func (m *KnownTypes) String() string { return proto.CompactTextString(m) }
func (*KnownTypes) ProtoMessage() {}
func (*KnownTypes) Descriptor() ([]byte, []int) {
return fileDescriptor_ab4422ec10550c41, []int{0}
}
func (m *KnownTypes) XXX_Unmarshal(b []byte) error {
return xxx_messageInfo_KnownTypes.Unmarshal(m, b)
}
func (m *KnownTypes) XXX_Marshal(b []byte, deterministic bool) ([]byte, error) {
return xxx_messageInfo_KnownTypes.Marshal(b, m, deterministic)
}
func (m *KnownTypes) XXX_Merge(src proto.Message) {
xxx_messageInfo_KnownTypes.Merge(m, src)
}
func (m *KnownTypes) XXX_Size() int {
return xxx_messageInfo_KnownTypes.Size(m)
}
func (m *KnownTypes) XXX_DiscardUnknown() {
xxx_messageInfo_KnownTypes.DiscardUnknown(m)
}
var xxx_messageInfo_KnownTypes proto.InternalMessageInfo
func (m *KnownTypes) GetAn() *any.Any {
if m != nil {
return m.An
}
return nil
}
func (m *KnownTypes) GetDur() *duration.Duration {
if m != nil {
return m.Dur
}
return nil
}
func (m *KnownTypes) GetSt() *_struct.Struct {
if m != nil {
return m.St
}
return nil
}
func (m *KnownTypes) GetTs() *timestamp.Timestamp {
if m != nil {
return m.Ts
}
return nil
}
func (m *KnownTypes) GetLv() *_struct.ListValue {
if m != nil {
return m.Lv
}
return nil
}
func (m *KnownTypes) GetVal() *_struct.Value {
if m != nil {
return m.Val
}
return nil
}
func (m *KnownTypes) GetDbl() *wrappers.DoubleValue {
if m != nil {
return m.Dbl
}
return nil
}
func (m *KnownTypes) GetFlt() *wrappers.FloatValue {
if m != nil {
return m.Flt
}
return nil
}
func (m *KnownTypes) GetI64() *wrappers.Int64Value {
if m != nil {
return m.I64
}
return nil
}
func (m *KnownTypes) GetU64() *wrappers.UInt64Value {
if m != nil {
return m.U64
}
return nil
}
func (m *KnownTypes) GetI32() *wrappers.Int32Value {
if m != nil {
return m.I32
}
return nil
}
func (m *KnownTypes) GetU32() *wrappers.UInt32Value {
if m != nil {
return m.U32
}
return nil
}
func (m *KnownTypes) GetBool() *wrappers.BoolValue {
if m != nil {
return m.Bool
}
return nil
}
func (m *KnownTypes) GetStr() *wrappers.StringValue {
if m != nil {
return m.Str
}
return nil
}
func (m *KnownTypes) GetBytes() *wrappers.BytesValue {
if m != nil {
return m.Bytes
}
return nil
}
func init() {
proto.RegisterType((*KnownTypes)(nil), "jsonpb.KnownTypes")
}
func init() {
proto.RegisterFile("testing/jsonpb_test_proto/test_objects.proto", fileDescriptor_ab4422ec10550c41)
}
var fileDescriptor_ab4422ec10550c41 = []byte{
// 402 bytes of a gzipped FileDescriptorProto
0x1f, 0x8b, 0x08, 0x00, 0x00, 0x00, 0x00, 0x00, 0x02, 0xff, 0x7c, 0xd0, 0xdf, 0x6b, 0xd5, 0x30,
0x14, 0x07, 0x70, 0x96, 0xee, 0x4e, 0xcd, 0x44, 0x25, 0x88, 0x66, 0xd7, 0xa1, 0x22, 0x82, 0xc3,
0x1f, 0x2d, 0xb6, 0xa5, 0xef, 0x0e, 0x11, 0x44, 0x9f, 0xba, 0xe9, 0xeb, 0x48, 0xd6, 0xac, 0x74,
0x64, 0x49, 0x49, 0x4e, 0xee, 0xe8, 0xbf, 0xe6, 0x5f, 0x27, 0x49, 0x73, 0x45, 0x6e, 0xc9, 0xde,
0xee, 0xcd, 0xf7, 0x73, 0xbe, 0x9c, 0x53, 0xfc, 0x11, 0x84, 0x85, 0x41, 0xf5, 0xc5, 0xb5, 0xd5,
0x6a, 0xe4, 0x17, 0xfe, 0xef, 0xc5, 0x68, 0x34, 0xe8, 0x22, 0xfc, 0xd4, 0xfc, 0x5a, 0x5c, 0x82,
0xcd, 0xc3, 0x13, 0x39, 0x98, 0xd5, 0xfa, 0xa8, 0xd7, 0xba, 0x97, 0xa2, 0x08, 0xaf, 0xdc, 0x5d,
0x15, 0x4c, 0x4d, 0x33, 0x59, 0xbf, 0xdc, 0x8d, 0x3a, 0x67, 0x18, 0x0c, 0x5a, 0xc5, 0xfc, 0x78,
0x37, 0xb7, 0x60, 0xdc, 0x25, 0xc4, 0xf4, 0xd5, 0x6e, 0x0a, 0xc3, 0x8d, 0xb0, 0xc0, 0x6e, 0xc6,
0x54, 0xfd, 0xad, 0x61, 0xe3, 0x28, 0x4c, 0xdc, 0xf0, 0xcd, 0x9f, 0x15, 0xc6, 0x3f, 0x94, 0xbe,
0x55, 0xe7, 0xd3, 0x28, 0x2c, 0x79, 0x8b, 0x11, 0x53, 0xf4, 0xd1, 0xeb, 0xbd, 0x93, 0xc3, 0xf2,
0x69, 0x3e, 0xcf, 0xe6, 0xdb, 0xd9, 0xfc, 0x8b, 0x9a, 0x5a, 0xc4, 0x14, 0xf9, 0x80, 0xb3, 0xce,
0x19, 0xba, 0x17, 0xd8, 0xd1, 0x82, 0x7d, 0x8d, 0x17, 0xb4, 0x5e, 0x91, 0x77, 0x18, 0x59, 0xa0,
0x0f, 0x83, 0x7d, 0xbe, 0xb0, 0x67, 0xe1, 0x9a, 0x16, 0x59, 0x20, 0xef, 0x31, 0x02, 0x4b, 0x51,
0x80, 0xeb, 0x05, 0x3c, 0xdf, 0x1e, 0xd6, 0x22, 0xb0, 0xde, 0xca, 0x0d, 0x7d, 0x9c, 0xb0, 0x3f,
0x07, 0x0b, 0xbf, 0x99, 0x74, 0xa2, 0x45, 0x72, 0x43, 0x4e, 0x70, 0xb6, 0x61, 0x92, 0x3e, 0x09,
0xf8, 0xd9, 0x02, 0xcf, 0xd0, 0x13, 0x92, 0xe3, 0xac, 0xe3, 0x92, 0x66, 0x41, 0x1e, 0x2f, 0xef,
0xd2, 0x8e, 0x4b, 0x11, 0x7d, 0xc7, 0x25, 0xf9, 0x84, 0xb3, 0x2b, 0x09, 0x74, 0x3f, 0xf8, 0x17,
0x0b, 0xff, 0x4d, 0x6a, 0x16, 0xf7, 0xf0, 0xce, 0xf3, 0xa1, 0xa9, 0xe9, 0x2a, 0xc1, 0xbf, 0x2b,
0x68, 0xea, 0xc8, 0x87, 0xa6, 0xf6, 0xdb, 0xb8, 0xa6, 0xa6, 0x07, 0x89, 0x6d, 0x7e, 0xfd, 0xef,
0x5d, 0x53, 0x87, 0xfa, 0xaa, 0xa4, 0xf7, 0xd2, 0xf5, 0x55, 0xb9, 0xad, 0xaf, 0xca, 0x50, 0x5f,
0x95, 0xf4, 0xfe, 0x1d, 0xf5, 0xff, 0xbc, 0x0b, 0x7e, 0x9f, 0x6b, 0x2d, 0xe9, 0x83, 0xc4, 0x47,
0x3f, 0xd5, 0x5a, 0xce, 0x3c, 0x38, 0xdf, 0x6f, 0xc1, 0x50, 0x9c, 0xe8, 0x3f, 0x03, 0x33, 0xa8,
0x3e, 0xf6, 0x5b, 0x30, 0xe4, 0x33, 0x5e, 0xf1, 0x09, 0x84, 0xa5, 0x87, 0x89, 0x03, 0x4e, 0x7d,
0x3a, 0x0f, 0xcc, 0xf2, 0x6f, 0x00, 0x00, 0x00, 0xff, 0xff, 0xab, 0x65, 0x99, 0x5a, 0x8d, 0x03,
0x00, 0x00,
}

View File

@@ -0,0 +1,59 @@
// Go support for Protocol Buffers - Google's data interchange format
//
// Copyright 2015 The Go Authors. All rights reserved.
// https://github.com/golang/protobuf
//
// Redistribution and use in source and binary forms, with or without
// modification, are permitted provided that the following conditions are
// met:
//
// * Redistributions of source code must retain the above copyright
// notice, this list of conditions and the following disclaimer.
// * Redistributions in binary form must reproduce the above
// copyright notice, this list of conditions and the following disclaimer
// in the documentation and/or other materials provided with the
// distribution.
// * Neither the name of Google Inc. nor the names of its
// contributors may be used to endorse or promote products derived from
// this software without specific prior written permission.
//
// THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
// "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
// LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
// A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
// OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
// SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
// LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
// DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
// THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
// (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
// OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
syntax = "proto2";
import "google/protobuf/any.proto";
import "google/protobuf/duration.proto";
import "google/protobuf/struct.proto";
import "google/protobuf/timestamp.proto";
import "google/protobuf/wrappers.proto";
package jsonpb;
message KnownTypes {
optional google.protobuf.Any an = 14;
optional google.protobuf.Duration dur = 1;
optional google.protobuf.Struct st = 12;
optional google.protobuf.Timestamp ts = 2;
optional google.protobuf.ListValue lv = 15;
optional google.protobuf.Value val = 16;
optional google.protobuf.DoubleValue dbl = 3;
optional google.protobuf.FloatValue flt = 4;
optional google.protobuf.Int64Value i64 = 5;
optional google.protobuf.UInt64Value u64 = 6;
optional google.protobuf.Int32Value i32 = 7;
optional google.protobuf.UInt32Value u32 = 8;
optional google.protobuf.BoolValue bool = 9;
optional google.protobuf.StringValue str = 10;
optional google.protobuf.BytesValue bytes = 11;
}

Binary file not shown.

View File

@@ -109,24 +109,42 @@ func TestBrokenTLS_ClientPlainText(t *testing.T) {
}
// client connection (usually) succeeds since client is not waiting for TLS handshake
e, err := createTestServerAndClient(serverCreds, nil)
if err != nil {
if strings.Contains(err.Error(), "deadline exceeded") {
// It is possible that connection never becomes healthy:
// (we try several times, but if we never get a connection and the error message is
// a known/expected possibility, we'll just bail)
var e testEnv
failCount := 0
for {
e, err = createTestServerAndClient(serverCreds, nil)
if err == nil {
// success!
defer e.Close()
break
}
if strings.Contains(err.Error(), "deadline exceeded") ||
strings.Contains(err.Error(), "use of closed network connection") {
// It is possible that the connection never becomes healthy:
// 1) grpc connects successfully
// 2) grpc client tries to send HTTP/2 preface and settings frame
// 3) server, expecting handshake, closes the connection
// 4) in the client, the write fails, so the connection never
// becomes ready
// More often than not, the connection becomes ready (presumably
// the write to the socket succeeds before the server closes the
// connection). But when it does not, it is possible to observe
// timeouts when setting up the connection.
return
// The client will attempt to reconnect on transient errors, so
// may eventually bump into the connect time limit. This used to
// result in a "deadline exceeded" error, but more recent versions
// of the grpc library report any underlying I/O error instead, so
// we also check for "use of closed network connection".
failCount++
if failCount > 5 {
return // bail...
}
// we'll try again
} else {
// some other error occurred, so we'll consider that a test failure
t.Fatalf("failed to setup server and client: %v", err)
}
t.Fatalf("failed to setup server and client: %v", err)
}
defer e.Close()
// but request fails because server closes connection upon seeing request
// bytes that are not a TLS handshake
@@ -285,7 +303,7 @@ func simpleTest(t *testing.T, cc *grpc.ClientConn) {
cl := grpc_testing.NewTestServiceClient(cc)
ctx, cancel := context.WithTimeout(context.Background(), 3*time.Second)
defer cancel()
_, err := cl.UnaryCall(ctx, &grpc_testing.SimpleRequest{}, grpc.FailFast(false))
_, err := cl.UnaryCall(ctx, &grpc_testing.SimpleRequest{}, grpc.WaitForReady(true))
if err != nil {
t.Errorf("simple RPC failed: %v", err)
}