what i learned about apis in my first year at google
TRANSCRIPT
What I learned about APIs in my first year at Google(Protocol Buffers and gRPC)
10 billion+
API callsevery second
I was an “indie” iOS developer
What I knew about APIs before I joined Google:
APIs are essential.APIs must be tested to be sure they work.APIs must be designed appropriately for their
applications.
Google Cloud Platform
The Fallacies of Distributed Computing
The network is reliable
Latency is zero
Bandwidth is infinite
The network is secure
https://blogs.oracle.com/jag/resource/Fallacies.html
Topology doesn't change
There is one administrator
Transport cost is zero
The network is homogeneous
There’s this wall.
Conway’s Law: “organizations which design systems... are constrained to produce
designs which are copies of the communication structures of these organizations.”
Cloud platforms can help
us tear down that wall.
10 billion+
API callsevery second
When you do a lot of something, you want to pay attention to it.
You develop a language for describing it.You optimize it.You standardize it.You make it flexible, so it can adapt to changes.
Protocol buffers are a language-neutral, platform-neutral, extensible mechanism
for serializing structured data.
Implemented in many languages, sometimes many times...
“Protocol Buffers” means several things
1. A serialization mechanism
2. An interface description language
3. A methodology
Protocol Buffer Serialization
It’s just a stream of bytes
[field_number<<3 + wire_type] [length if necessary] [data]...
$ hexdump /tmp/request.pb
0000000 0a 05 68 65 6c 6c 6f
0a is “0000 1010”, sofield_number = 1
wire_type = 2
echo.proto
syntax = "proto3";
package echo;
service Echo {
rpc Get(EchoRequest) returns (EchoResponse) {}
rpc Update(stream EchoRequest) returns (stream EchoResponse) {}
}
message EchoRequest {
string text = 1;
}
message EchoResponse {
string text = 1;
}
The Protocol Buffer Language
The Protocol Buffer methodology
$ protoc echo.proto --go_out=.
$ more echo.pb.go
// Code generated by protoc-gen-go.
// source: echo.proto
// DO NOT EDIT!
...
A remote procedure call (RPC) framework layered on HTTP/2
Enables client and server applications to communicate transparently
Client “calls” methods on a Server (which may be on a different machine) as if it
were a local service
Makes it easy to build heterogeneous distributed systems
Separate full implementations in C, Java, and Go.
Google Cloud Platform
Interoperability
Java Service
Python Service
GoLang Service
C++ Service
gRPC Service
gRPC Stub
gRPC Stub
gRPC Stub
gRPC Stub
gRPC Service
gRPC Service
gRPC Service
gRPC Stub
Unary (nonstreaming) API
syntax = “proto3”;
message HelloRequest {string name = 1;
}
message HelloReply {string message = 1;
}
service HelloService {rpc SayHello(HelloRequest) returns (HelloReply);
}
Unary API: Client sends a
request, server sends a
response
Streaming APIs - 1 (of 3) Client Streaming
syntax = “proto3”;
message Latency {string name = 1;double val = 2;
}
message Histogram {string name = 1double p99 = 2;double p999 = 3;double avg = 4;
}
service MetricsService {rpc ReportLateny(stream Latency) returns Histogram;
}
Client Streaming API: Client
sends multiple messages;
Server sends one response
(Note: Server may choose to send the response
before all the client messages are received)
Streaming APIs - 2 (of 3) Server Streaming
syntax = “proto3”;
message StockRequest {string stocksymbol = 1;
}
message StockResponse {double price = 1;
}
service StockTickerService {rpc GetTicker(StockRequest) returns (stream StockResponse);
}
Server Streaming API: Client
sends one message; Server
sends multiple messages
Streaming APIs - 3 (of 3) Bidirectional Streaming
syntax = “proto3”;
message ChatMessage {string msg = 1;
}
service ChatService {rpc Chat(stream ChatMessage) returns (stream ChatMessage);
}
Bidi Streaming API: Client and
Server can send multiple
messages to each other
(Note: Client and server can independently send
messages to each other i.e they do not have to wait
for receiving a client message before sending a
message)
Google Cloud Platform
Some gRPC Adopters
Google Cloud Platform
Netflix deprecated its RPC framework in favor of gRPC:
“Our team has instead building an RPC solution on top of gRPC. We are doing this transition for two main reasons: multi-language support and better extensibility/composability through request interceptors. That’s our current plan moving forward.”
https://github.com/Netflix/ribbon
gRPC Adopters
We’re building Swift support for gRPC.
We are working to bring Protocol Buffers and gRPC support into OpenAPI.
