extending envoy with wasm from start to finish

35
#IstioCon Extending Envoy with Wasm from start to finish Ed Snible / @esnible / IBM Research With: The first dozen places I got stuck writing filters Includes LIVE DEMO

Upload: others

Post on 10-Nov-2021

29 views

Category:

Documents


0 download

TRANSCRIPT

Page 1: Extending Envoy with Wasm from start to finish

#IstioCon

Extending Envoy with Wasm from start to finish

Ed Snible / @esnible / IBM Research

With: The first dozen

places I got stuck

writing filters

Speaker instructions: https://docs.google.com/document/d/1sJwwE-cVPQSF7McChz9MW5myyrs1rfOkQy6Wn8T_X70/edit

Includes

LIVE DEMO

Page 2: Extending Envoy with Wasm from start to finish

#IstioCon

The trouble is that men very often resort to all sorts of devices in order not to think

because thinking is such hard work

—Thomas J. Watson

Page 3: Extending Envoy with Wasm from start to finish

Istio

ServiceEntry

Istio

VirtualService

docker.io/mycompany/a:latest

docker.io/mycompany/b:latest

Sprinkle a little Wasm

into your mesh

Istio

EnvoyFilter

Istio Architecture

Page 4: Extending Envoy with Wasm from start to finish

Why is WebAssembly for Proxies different and better?

The full power of

• a mainstream programming language

• running in an efficient engine

• potentially inspecting every communication between unmodified microservices

Page 5: Extending Envoy with Wasm from start to finish

Is it safe to let arbitrary code run on the sidecar?

• The ability to create EnvoyFilters is controlled by RBAC.

• Only trusted people should be adjusting mesh behavior.

• Extension code runs in a sandbox. The only networking available is HTTP and GRPC calls

• When making those calls, only clusters already known to Envoy can be used. No connecting to arbitrary hosts!

• Developers can cause trouble with unbounded recursion and memory exhaustion

• Admins with access to the pods can point at any storage, admins with access to the storage can replace expected binaries

• Currently no audit tools for .wasm in storage (that I know of).

��Yes! But…

��

Page 6: Extending Envoy with Wasm from start to finish

e.filters.network.metadata_exchange

e.corse.filters.http.wasm / envoy.wasm.metadata_exchange

istio_authn e.faulte.filters.http.wasm / my_filter

e.router

UML sequence diagram

“Connection”e.http_connection_manager

onNewConnection()Data arrives

Not shown: onResponseHeaders(), onResponseBody(), onDone()

Continueor

StopIteration

“The Filter Chain”

onRequestHeaders()onData() (aka onUpstreamData())

onRequestBody()

Reply sent

Envoy’s architecture

Continueor

StopIteration

myfile.wasm

app containe

r (real service)

Page 7: Extending Envoy with Wasm from start to finish

“convert GET /banana/nnn to GET /status/nnn”

FilterHeadersStatus RegexpRepl::onRequestHeaders( uint32_t hdr_count, bool eos) {

std::regex re("banana/([0-9]*)”); char const *replaceWith = "status/$1";

WasmDataPtr wdpGhmv = getRequestHeader(":path");

std::string result = std::regex_replace( wdpGhmv->toString(), re, replaceWith); WasmResult wrRrh = replaceRequestHeader( ":path", result);

return FilterHeadersStatus::Continue;}

AssemblyScript (JavaScript / TypeScript)C++

The stuff in blue comes from the SDKThe black stuff comes from your language

onRequestHeaders(hd_cnt: u32, eos: bool): FilterHeadersStatusValues {

// (TODO: Current AssemblyScript compiler refuses to compile RegExp) let re = new RegExp("banana/([0-9]*)"); let replaceWith = "status/$1";

let sValue = stream_context.headers.request.get(":path");

let result = sValue.replace(re, replaceWith); stream_context.headers.request.replace( ":path", result);

return FilterHeadersStatusValues.Continue;}

Example: URL rewrite to match client and server

Source code repo for this talk:

github.com/esnible/wasm-examples

Service A Httpbin Service

GET /banana/418 (expects GET /status/418)

Page 8: Extending Envoy with Wasm from start to finish

An Istio contributor’s first experiences

mycode.cc,.ts,.go

mycode.wasm

C++: docker run --workdir … --volume … wasmsdk:v2 /build_wasm.sh

JavaScript: npm run asbuild

details

Proxy

?WebAssemblyHub?EnvoyFilter yaml?Solo.io Wasme operator?

Go: docker run -it –workdir … --volume … tinygo/tinygo:0.16.0 \ tinygo build -o … -scheduler=none -target=wasi …/main.go

Rust: TODO

It is easy to find the SDKs and follow their tutorial steps to produce .wasm byte code

Compilation takes a few seconds!����

Another pod

AnotherpodAnother

Pod?

TODO: Bazel

Page 9: Extending Envoy with Wasm from start to finish

From a laptop to the sidecar

mycode.wasm

details-v1 app: details version: v1 annotations: userVol… uVolMnt…

docker.io/istio/examples-bookinfo-details-v1:1.16.2

gcr.io/istio-testing/proxyv2:latest

lr

ce

e

mycode.wasm

/var/local/wasm/new-filter.wasm

kubectl create configmap new-filter \ --from-file=new-filter.wasm=mycode.wasm

Kind: Deployment…spec: template: metadata: annotations: sidecar.istio.io/userVolume: '[{"name":"new-filter","configMap":{"name":"new-filter"}}]’I sidecar.istio.io/userVolumeMount: '[{"mountPath":"/var/local/wasm","name":"new-filter"}]’ …

Kind: ConfigMapbinary-data: new-filter.wasm=…

mycode.wasm

Add two annotations to your K8s Deployment

Istiod “Discovery”

IstioInjector

Envoy config

NOTE: Istio 1.9 includes an experimental feature to fetch Wasm bytecode from

URLs. This talk uses Kubernetes storage. Storage is simpler to set up and can be managed by RBAC.

https://istio.io/latest/docs/ops/configuration/extensibility/wasm-module-distribution/

ConfigMaps are good for experimentation.Production: Kubernetes volumes or (Istio

1.9) HTTP remote fetch of Wasm

Kubernetes YAML

Kubernetes YAML

1st2nd

Page 10: Extending Envoy with Wasm from start to finish

onNewConnection()Data arrives

Not shown: onResponseHeaders(), onResponseBody(), onDone()

Continueor

StopIteration

“The Filter Chain”

onRequestHeaders()onData()

onRequestBody()

Reply sentmyfile.wasm

Continueor

StopIteration

It isn’t enough just to put the code on the sidecar. We also need to tell Envoy where in the ”filter chain” to invoke the filter.

My filter runs under “HTTP Connection Manager”

onRequestHeaders()

onRequestBody()

e.filters.network.metadata_exchange

e.corse.filters.http.wasm / envoy.wasm.metadata_exchange

istio_authn e.faulte.filters.http.wasm / regex_filter

e.router“Connection”e.http_connection_manager

path-regexp.wasm

Page 11: Extending Envoy with Wasm from start to finish

kind: EnvoyFiltermetadata: name: httpbin-myfilterspec:workloadSelector: labels: app: httpbin configPatches: - applyTo: HTTP_FILTER match: context: SIDECAR_INBOUND listener: filterChain: filter: name: envoy.http_connection_manager subFilter: name: envoy.routerpatch: operation: INSERT_BEFORE value: name: myfilter typed_config: '@type': type.googleapis.com/udpa.type.v1.TypedStruct type_url: type.googleapis.com/envoy.extensions.filters.http.wasm.v3.Wasm

value: config: configuration: '@type': type.googleapis.com/google.protobuf.StringValue value: "{ "regex”: "banana/([0-9]*)", "replace”: "status/$1" }" root_id: ”regex_repl" vm_config: code: local: filename: /var/local/wasm/new-filter.wasm runtime: envoy.wasm.runtime.v8 vm_id: myexperiment-1.9

Which pods, where in Envoy, and your filter’s parameters

Pods to receive configuration

Some Envoy configuration

Where to attach the Envoy configuration

Same pods askubectl get pods –selector app=httpbin(Leave this blank if every pod should get it.)

• Config parameters for the filter• We named our extension “regex_repl” in the source code• We mounted it at /var/local/wasm/new-filter.wasm

To HTTP connection manager

{ "regex”: "banana/([0-9]*)", "replace”: "status/$1" }

EnvoyFilterIstio YAML

Page 12: Extending Envoy with Wasm from start to finish

Before: New .wasm unknown to Envoy

httpbin app: httpbin annotations: userVol… uVolMnt…

docker.io/kennethreitz/httpbin

gcr.io/istio-testing/proxyv2:latest

mycode.wasm

/var/local/wasm/new-filter.wasm

lr

ce

e

Istio

VirtualService

Istio

ServiceEntryl

rc

e

e

Envoy config

Kubernetes

Pod

Page 13: Extending Envoy with Wasm from start to finish

After: Running code on Envoy

httpbin app: httpbin annotations: userVol… uVolMnt…

docker.io/kennethreitz/httpbin

gcr.io/istio-testing/proxyv2:latest

IstioEnvoyFilter

Pods: app=httpbin

configuration: | { "regex”: "banana/([0-9]*)", "replace”: "status/$1" }

root_id: "regex_repl"code.local.filename: /var/local/wasm/new-filter.wasm

(all boilerplate)

Workload Selector

Value

ApplyTo

mycode.wasm

/var/local/wasm/new-filter.wasm

lr

ce

e

Istio

VirtualService

Istio

ServiceEntryl

rc

e

e

Envoy config

Kubernetes

Pod

Page 14: Extending Envoy with Wasm from start to finish

Thank You!

But wait, there’s more…

Page 15: Extending Envoy with Wasm from start to finish

The first dozen

places I got stuck

when I tried to write

my own filters

Page 16: Extending Envoy with Wasm from start to finish

Do I need to write a filter at all?

• Before you begin, check Envoy’s catalog to see if there is already a filter that does what you want!

• https://www.envoyproxy.io/docs/envoy/latest/api-v3/config/filter/filter

• If so, you can add use Istio’s EnvoyFilter to add it to your pods, without writing a new filter.

• Warning: Possible upgrade risk

• The functionality might be elsewhere.• The Regex filter I’m showing is not needed; Envoy exposes

regex_rewrite on config.route.v3.RouteAction

• Github might have what you need (in source format)• Start at

https://github.com/istio-ecosystem/wasm-extensions

• Solo.io’s WebAssembly Hub might already have what you need (as a binary)

Network FiltersEchoSNI ClusterClient TLS authenticationDirect responseDubbo ProxyDubbo Proxy Route ConfigurationEchoNetwork External AuthorizationHTTP connection managerKafka BrokerLocal rate limitMongo proxyMySQL proxyPostgres proxyRate limitRBACRedis ProxyRocketMQ ProxyRocketmq Proxy Route ConfigurationSNI Cluster FilterSNI dynamic forward proxyTCP ProxyThrift Proxy Route ConfigurationThrift ProxyWasmZooKeeper proxy

CORS processingAWS DynamoDBgRPC HTTP/1 bridgegRPC WebAdaptive ConcurrencyAWS LambdaAwsRequestSigningBufferCompressorCorsCSRFDecompressorDynamic forward proxyDynamoExternal AuthorizationFault InjectiongRPC HTTP/1.1 BridgegRPC HTTP/1.1 Reverse BridgegRPC-JSON transcodergRPC statisticsgRPC WebGzipHeader-To-Metadata FilterHealth checkIP taggingJWT AuthenticationKill RequestLocal Rate limit

LuaOnDemandOriginal Src FilterRate limitRBACRouterSquashTapWasm

HTTP Filters

Using Envoy’s own filters might create upgrade risk

��

Page 17: Extending Envoy with Wasm from start to finish

Can the filter be implemented?

Diagram: https://istio.io/latest/docs/concepts/wasm/

Istio authn

envoy.corsenvoy.router

Envoy “filter chain”

app container (real service)

Language restrictions• AssemblyScript has no JavaScript regular

expressions, no JSON.stringify()• JSON parsing in TinyGo works (but build env

currently requires Bazel)

• No access to operating system.• std::getenv("PATH") returns ""• You can’t do networking things• C++ sleep() doesn’t sleep, no

JavaScript setTimeout()• The only calls allowed are Envoy

functions and the SDK functions of your language.

Filters can• Read Envoy configuration for itself and the

bootstrap section of the Envoy config (for example the node.id, or node.metadata.ISTIO_VERSION)

• Limited HTTP and GRPC access via Envoy• onTick() can be used for background processing

Filters run in a “sandbox” with significant restrictions

Page 18: Extending Envoy with Wasm from start to finish

Coding style and SDK stumbling blocks

• Filters implement use a non-blocking “event-driven” flow, like JavaScript.

• It helps if you have coded in that style before.

• Descriptions of many constants you’ll need are only given in Envoy headers

• https://github.com/envoyproxy/envoy/blob/master/include/envoy/http/filter.h

• Where is main()?

• Read through the source of the SDK for your language• You need to know SDK-provided global methods• Your editor’s auto-completion isn’t enough

TODO: one thing I forgot to mention wrt your slides -> the programming paradigm is not “async” from JavaScript, it’s event-driven programming (https://en.wikipedia.org/wiki/Event-driven_programming) that’s common in high-performance servers, etc.

Page 19: Extending Envoy with Wasm from start to finish

RootContextonConfigure()onStart()onTick()onQueueReady()createContext()httpCall(cluster: string, headers:

Headers, body: ArrayBuffer, timeout_ms: u32, origin_context: BaseContext, callback(…))

BaseContextonDone()onDelete()

ContextonNewConnection()onDownstreamData()onUpstreamData()onDownstreamConnectionClose()onUpstreamConnectionClose()onRequestHeaders()onRequestMetadata()onRequestBody()onRequestTrailers()onResponseHeaders()onResponseMetadata()onResponseBody()onResponseTrailers()onLog()

RegexReplRootonConfigure()

RegexpReplonRequestHeaders()

UML model of SDKs

This particular example comes from Solo.io’s AssemblyScript SDK, but the C++ SDK is similar.

Your filter classes

SDK-provided base classes

Envoy has two kinds of filters, HTTP filters with access to things like headers and bodies, and Network filters that see bytes.

Warning: Some of these methods only have effect in one or the other kind of filter.

Page 20: Extending Envoy with Wasm from start to finish

RootContextonPluginStart()onVMStart()onTick()onQueueReady()onVMDone()onLog()

StreamContextonNewConnection()onDownstreamData()onUpstreamData()onDownstreamClose()onUpstreamClose()onStreamDone()onLog()

ContextonHttpRequestHeaders()onHttpRequestBody()onHttpRequestTrailers()onHttpResponseHeaders()onHttpResponseBody()onHttpResponseTrailers()onHttpStreamDone()onLog()

RegexReplRootonPluginStart()

RegexpReplonRequestHeaders()

UML model of Go SDK

The Go SDK is factored to make a distinction between HTTP and stream filters.

Your filter structs

Interfaces

optional

Page 21: Extending Envoy with Wasm from start to finish

How WASM code using the SDK starts• JavaScript

registerRootContext( () => { return RootContextHelper.wrap(new RegexReplRoot()); }, "regex_repl");

• C++• There’s no main()

static RegisterContextFactory register_AuthContext( CONTEXT_FACTORY(RegexRepl), ROOT_FACTORY(RegexReplRoot), "regex_repl");

• Go (TinyGo)• There is a main()

func newContext(rootContextID, contextID uint32) proxywasm.HttpContext { return &regexRepl{contextID: contextID} }func main() { proxywasm.SetNewHttpContext(newContext) // Go SDK doesn’t offer a string name!}

IstioEnvoyFilter

root_id: "add_header"configuration: …code.local: …

(all boilerplate)

Workload Selector

Value

ApplyTo

Envoy’s loading of your .wasm file triggers static constructors (C++), main() (Go), or code outside methods (AssemblyScript)

Envoy starts your code and calls onStart() three times – or more if Envoy’s --concurrency is set.

Page 22: Extending Envoy with Wasm from start to finish

Configuring filtersIstio

EnvoyFilter

… Kubernetes selector …

root_id: …configuration: | … JSON …code.local.filename: …

Workload Selector

Value

ApplyTo

Deployers/Admins: Pass serialized JSON

Yes, JSON inside YAML!root_id: regex_replconfiguration: | { "regex”: "banana/([0-9]*)", "replace”: "status/$1" }

C++: • WasmDataPtr wdp = getBufferBytes(WasmBufferType::PluginConfiguration, 0, configuration_size);AssemblyScript:• let s = this.root_context.getConfiguration();Go:• data, err := proxywasm.GetPluginConfiguration(pluginConfigurationSize)

Developers: Get bytes, explicitly deserialize

The Go and C++ versions can ONLY be called during onConfigure().Most global functions have that characteristic.

YAML

Page 23: Extending Envoy with Wasm from start to finish

The header handler isn’t supplied with the headers…

virtual FilterHeadersStatus onRequestHeaders(uint32_t, bool) { …}

Context

��

FilterHeadersStatus Context::onRequestHeaders( uint32_t hdr_cnt, bool eos) {

WasmDataPtr wdpUserAgent = getRequestHeader("User-Agent");std::string sStatus = wdpUserAgent->toString();

WasmDataPtr headerPairs = getRequestHeaderPairs();auto headerVec = headerPairs->pairs();

onXXXHeaders() parameters don’t provide access. Global functions fetch them.

AssemblyScript (JavaScript / TypeScript)C++

onRequestHeaders(a: u32, end_of_stream: bool): FilterHeadersStatusValues { …}

let headerPairs: Headers = stream_context.headers.request.get_headers()

let sStatus: string = stream_context.headers.request.get("User-Agent")

onRequestHeaders(hdr_count: u32, eos: bool): FilterHeadersStatusValues {

Page 24: Extending Envoy with Wasm from start to finish

httpCall()

WasmResult httpCall( std::string_view uri, const HeaderStringPairs &request_headers, std::string_view request_body, const HeaderStringPairs &request_trailers, uint32_t timeout_milliseconds, HttpCallCallback callback);

I will be publishing an entire blog on how to use this function. It’s also in the GitHub repo I mentioned earlier.

• “uri” isn’t a URI, it’s an Envoy “cluster name”• The HTTP Method, Path and Authority as treated as request “headers”

• Envoy is picky about parameters and miserly about error response info• The callback is async

• onXXX() methods invoke httpGet() then return StopIteration• Callbacks must call setEffectiveContext() and continueRequest()

Page 25: Extending Envoy with Wasm from start to finish

Debugging

• If you are coming from a browser-based Wasm environment, you may expect a debugger such as Chrome DevTools.

• I haven’t found anything like that. I’m using log statements!• istioctl proxy-config log deployment/httpbin --level wasm:trace• kubectl logs deploy/httpbin -c istio-proxy --follow

• Watch out for ”linker errors”:• error envoy wasm Failed to load Wasm module due to a missing import:

env._ZN14RegexpReplRoot6onTickEv

• To get Envoy to pick up changes after a recompile delete then recreate the Istio EnvoyFilter.

• It isn’t sufficient to just change the underlying mounted storage• Watch out for timing

• Kubernetes can be slow to update ConfigMap mounts (up to a minute)• Envoy can be slow to stop running plugins, also up to a minute, log statements from the earlier version

will appear during this time.

Page 26: Extending Envoy with Wasm from start to finish

https://github.com/proxy-wasm https://github.com/solo-io

proxy-runtimeAssemblyScrip

t SDK

wasmetooling

specABI

Specification

proxy-wasm-cpp-host

C++ for host

proxy-wasm-cpp-sdk

C++ for plugins

proxy-wasm-rust-sdk

Rust plugins

https://github.com/envoyproxy

envoy-wasmEnvoy itself w/WASM and

envoy_filter_http_wasm_example.cc

Envoy itself,many contributors, CNCF

https://github.com/istio

proxyWASM plugins used by Istio

(Stats and Metadata Exchange)Istio itself

Istio announcement

https://istio.io/latest/blog/2020/wasm-announce/

https://solo-io.slack.com/#web-assembly-hub

https://envoyproxy.slack.com/#envoy-wasm

Open Source Landscape

https://github.com/tetratelabs

proxy-wasm-go-sdkTinyGo SDK

https://github.com/istio-ecosystem

wasm-extensionsExamples

https://isto.slack.com/#extensions-telemetry

Page 27: Extending Envoy with Wasm from start to finish

Bibliography

• Documentation:• https://istio.io/latest/docs/concepts/wasm/

• Blogs:• Yaroslav Skopets, “Taming a Network Filter”

• https://blog.envoyproxy.io/taming-a-network-filter-44adcf91517• This video shows TCP filters, which I did not cover today!

• Videos:• Idit Levine and Scott Weiss’ IstioCon presentation

https://events.istio.io/istiocon-2021/sessions/developing-debugging-webassembly-filters/ earlier today!

• Daniel Grimm, “Hacking the Mesh: Extending Istio with WebAssembly Modules”, DevNation https://www.facebook.com/openshift/videos/375151853879233 (starts 1:45 minutes in)

• John Plevyak & Dhi Aurrahman, “Extending Envoy with WebAssembly”, KubeCon EU 2019, https://www.youtube.com/watch?v=XdWmm_mtVXI

• John Plevyak, starts 13 minutes in, gives a high-level overview

• Source code repo for this talk: github.com/esnible/wasm-examples https://s3.us.cloud-object-storage.appdomain.cloud/developer/default/series/os-academy-istio-2020/static/4-WASM.pdf

Page 28: Extending Envoy with Wasm from start to finish

Backup

• (Backup slides, if we have a Q&A and I am asked about specific surprises)

Page 29: Extending Envoy with Wasm from start to finish

Unpleasant Surprises

��WasmDataPtr headerPairs = getRequestHeaderPairs();auto headerVec = headerPairs->pairs();

This code only works in onRequestHeaders(). If you call it later, for example in onResponseHeaders(), it’s empty. (Save a copy of anything you’ll use beyond the current member function.)

This code almost works. It’s an easy mistake to make if you are coming from a garbage-collected language.

auto headerPairs = getRequestHeaderPairs() ->pairs();

��Tip: don’t use auto.Use std::vector<std::pair<std::string_view, std::string_view>>

C++ programmers know to be wary of implicit destruction and free() when they see string_view.

C++let headerPairs: Headers = stream_context.headers.request.get_headers()

AssemblyScript (JavaScript / TypeScript)

Page 30: Extending Envoy with Wasm from start to finish

Making httpCalls

WasmResult httpCall( std::string_view uri, const HeaderStringPairs &request_headers, std::string_view request_body, const HeaderStringPairs &request_trailers, uint32_t timeout_milliseconds, HttpCallCallback callback);

SDK API looks straightforward:

Three of the parameters confuse beginners

• uri isn’t a URI !!!• Is that a GET or a POST? Where to put HTTP PATH and Method?• How does my std::function<void(uint32_t, size_t, uint32_t)> callback get a this into my Context?������

��

After my callback has done, how do I “rejoin” or “rendezvous” with the onXXX() method?

Troubleshooting failures

No detailed error messages. Any mistakes receive WasmResult::BadArgument, an enum which is the digit 2

Page 31: Extending Envoy with Wasm from start to finish

http method and pathWasmResult httpCall(std::string_view uri, const HeaderStringPairs &request_headers,std::string_view request_body, const HeaderStringPairs &request_trailers,uint32_t timeout_milliseconds, HttpCallCallback callback);

std::vector<std::pair<std::string, std::string>> callHeaders;callHeaders.push_back(std::pair<std::string, std::string>(":method", "GET"));callHeaders.push_back(std::pair<std::string, std::string>(":path", "/"));callHeaders.push_back(std::pair<std::string, std::string>(":authority", "example.com"));

WasmResult wr = this->root()->httpCall( "outbound|80||example.com", callHeaders, "", HeaderStringPairs(), 5000, …);

��

��

Envoy uses “headers” for non-header things such as the HTTP method. Prepended with a colon. I figured this out from Lua examples!

“uri” must be a cluster name,Something you see at :15000/clusters when looking at the Envoy dashboard,And perhaps defined with a ServiceEntry.

Page 32: Extending Envoy with Wasm from start to finish

httpCall callback (C++)WasmResult httpCall(std::string_view uri, const HeaderStringPairs &request_headers,std::string_view request_body, const HeaderStringPairs &request_trailers,uint32_t timeout_milliseconds, HttpCallCallback callback);

WasmResult wr = this->root()->httpCall( "outbound|80||example.com", callHeaders, "", HeaderStringPairs(), 5000, std::bind(&MyClass::myHttpCallback, this, std::placeholders::_1, std::placeholders::_2, std::placeholders::_3));

��

��

C++ Use bind(), similar to what you would do with JavaScript, to make a regular function pointer out of a member function.

AssemblyScript Works well.

Page 33: Extending Envoy with Wasm from start to finish

httpCall callback (Go)

_, err := proxywasm.DispatchHttpCall( "outbound|80||example.com", headers, "", [][2]string{}, 5000, createMyHttpCallback(ctx))

func createMyHttpCallback(mc *myClass) proxywasm.HttpCalloutCallBack { return func(numHeaders, bodySize, numTrailers int) { mc.httpCallResponseCallback(numHeaders, bodySize, numTrailers) }}

��

��

Go: Write a function that returns a closure

DispatchHttpCall(upstream string,headers [][2]string, body string, trailers [][2]string,timeoutMillisecond uint32, callBack HttpCalloutCallBack) (calloutID uint32, err error)

Page 34: Extending Envoy with Wasm from start to finish

The httpCall doesn’t block

FilterHeadersStatus MyContext::onRequestHeaders(uint32_t header_count, bool end_of_stream) {

WasmResult wr = this->root()->httpCall(…) if (wr == WasmResult::Ok) { return FilterHeadersStatus::StopIteration; }

logWarn("context_id:" + strId_ + " MyContext ::onResponseHeaders() httpCall() failed. wr is " + toString(wr)); return FilterHeadersStatus::Continue;}

Ok doesn’t mean the HTTP succeeded with a 200 OK. It means the parameters were accepted and the callback will be invoked in the future.

StopIteration tells Envoy to stop working for now. Later we’ll rejoin the filter chain where we left off.

Continue tells Envoy to let the rest of the filters run. If we don’t return this, nothing happens and the caller may eventually time out.

If you’ve programmed in Node.js, this should make sense

Page 35: Extending Envoy with Wasm from start to finish

Handling the callbackThe callbacks receive a size and some counts. Get the real headers and body from global functions

void MyContext::myHttpCallback(uint32_t headerCount, size_t body_size, uint32_t trailerCount) { WasmDataPtr wdpGhmv = getHeaderMapValue(WasmHeaderMapType::HttpCallResponseHeaders, ":status"); uint32_t status = std::stoi(wdpGhmv->toString()); WasmDataPtr wdpBody = getBufferBytes(WasmBufferType::HttpCallResponseBody, 0, body_size);

WasmResult wrSec = setEffectiveContext(); if (wrSec != WasmResult::Ok) { logWarn("context_id:" + strId_ + " Experiment::myHttpCallback() setEffectiveContext() FAILED. wrSec is " + toString(wrSec)); }

… your code …

WasmResult wr = continueRequest(); if (wr != WasmResult::Ok) { logWarn("context_id:" + strId_ + " MyContext ::myHttpCallback() continueRequest() FAILED. wr is " + toString(wr)); }}

After setEffectiveContext() you can call setResponseHeaderPairs() and friends, continueRequest(), or even sendLocalResponse().If you don’t call continueRequest() or sendLocalResponse(), the caller will not receive anything and may eventually time out.

If you forget setEffectiveContext() nothing happens. continueRequest() returns Ok, yet nothing happens.

��e.corsistio_authne.fault

e.filters.http.wasm / my_filter

e.router…… …filter chain