概览
aws-sdk-go-v2包含了aws上百个服务的sdk封装，为了简化开发和维护，使用aws smithy进行代码生成。
各个服务的入口都在service目录下，均是生成的代码。
以s3vectors为例，aws底层提供了通用的http和签名能力，而专属于s3vectors的处理则通过中间件来实现。
中间件middleware模型：
middleware库也是在aws smithy仓库中。
调用链分析
type Stack struct {
// Initialize prepares the input, and sets any default parameters as
// needed, (e.g. idempotency token, and presigned URLs).
//
// Takes Input Parameters, and returns result or error.
//
// Receives result or error from Serialize step.
Initialize *InitializeStep
// Serialize serializes the prepared input into a data structure that can be consumed
// by the target transport's message, (e.g. REST-JSON serialization)
//
// Converts Input Parameters into a Request, and returns the result or error.
//
// Receives result or error from Build step.
Serialize *SerializeStep
// Build adds additional metadata to the serialized transport message
// (e.g. HTTP's Content-Length header, or body checksum). Decorations and
// modifications to the message should be copied to all message attempts.
//
// Takes Request, and returns result or error.
//
// Receives result or error from Finalize step.
Build *BuildStep
// Finalize performs final preparations needed before sending the message. The
// message should already be complete by this stage, and is only alternated
// to meet the expectations of the recipient (e.g. Retry and AWS SigV4
// request signing)
//
// Takes Request, and returns result or error.
//
// Receives result or error from Deserialize step.
Finalize *FinalizeStep
// Deserialize reacts to the handler's response returned by the recipient of the request
// message. Deserializes the response into a structured type or error above
// stacks can react to.
//
// Should only forward Request to underlying handler.
//
// Takes Request, and returns result or error.
//
// Receives raw response, or error from underlying handler.
Deserialize *DeserializeStep
id string
}
它们都实现了Middleware接口，通过next Handler可实现链式调用
// Middleware provides the interface to call handlers in a chain.
type Middleware interface {
// ID provides a unique identifier for the middleware.
ID() string
// Performs the middleware's handling of the input, returning the output,
// or error. The middleware can invoke the next Handler if handling should
// continue.
HandleMiddleware(ctx context.Context, input interface{}, next Handler) (
output interface{}, metadata Metadata, err error,
)
}
当然，这里的链式调用，指的是将Initialize、Serialize这几个Middleware连起来
这中间有些绕
-1. 首先在Client::invokeOperation中
stack := middleware.NewStack(opID, smithyhttp.NewStackRequest)
//然后通过传入的addOperationGetVectorBucketMiddlewares为stack的各个位置添加对应中间件
decorated := middleware.DecorateHandler(handler, stack)
result, metadata, err = decorated.Handle(ctx, params)
-2. DecorateHandler返回的是个decoratedHandler对象，实现了Handle方法
type decoratedHandler struct {
// The next handler to be called.
Next Handler
// The current middleware decorating the handler.
With Middleware
}
// Handle implements the Handler interface to handle a operation invocation.
func (m decoratedHandler) Handle(ctx context.Context, input interface{}) (
output interface{}, metadata Metadata, err error,
) {
return m.With.HandleMiddleware(ctx, input, m.Next)
}
-3. 这样，最终将调用Stack的HandleMiddleware方法
func (s *Stack) HandleMiddleware(ctx context.Context, input interface{}, next Handler) (
output interface{}, metadata Metadata, err error,
) {
h := DecorateHandler(next,
s.Initialize,
s.Serialize,
s.Build,
s.Finalize,
s.Deserialize,
)
return h.Handle(ctx, input)
}
func DecorateHandler(h Handler, with ...Middleware) Handler {
for i := len(with) - 1; i >= 0; i-- {
h = decoratedHandler{
Next: h,
With: with[i],
}
}
return h
}
-4. 这里的h.Handle第二次出现，但这次执行到底将会执行s.Initialize、s.Serialize等的HandleMiddleware方法。以Initialize为例：
type InitializeStep struct {
head *decoratedInitializeHandler
tail *decoratedInitializeHandler
}
type decoratedInitializeHandler struct {
Next InitializeHandler
With InitializeMiddleware
}
func (s *InitializeStep) HandleMiddleware(ctx context.Context, in interface{}, next Handler) (
out interface{}, metadata Metadata, err error,
) {
sIn := InitializeInput{
Parameters: in,
}
wh := &initializeWrapHandler{next}
if s.head == nil {
res, metadata, err := wh.HandleInitialize(ctx, sIn)
return res.Result, metadata, err
}
s.tail.Next = wh
res, metadata, err := s.head.HandleInitialize(ctx, sIn)
return res.Result, metadata, err
}
注意这里sIn := InitializeInput{Parameters: in}
InitializeStep也有一条链，最终执行什么则取决于给InitializeStep注册了哪些InitializeMiddleware中间件。
而initialize和serialize如何连接起来的呢，答案就在initializeWrapHandler上，它的HandleInitialize方法与head-tail链中的不同，调的是SerializeStep的handle。
而最后DeserializeStep后面则连接ClientHandler执行真正的请求。
下面先分析注册方式。
addOperationGetVectorBucketMiddlewares中有很多注册相关代码，举个例子：
func addlegacyEndpointContextSetter(stack *middleware.Stack, o Options) error {
return stack.Initialize.Add(&legacyEndpointContextSetter{
LegacyResolver: o.EndpointResolver,
}, middleware.Before)
}
这其实就是把&legacyEndpointContextSetter{LegacyResolver: o.EndpointResolver}这个InitializeMiddleware类型的中间件插入到head-tail链中。
Add是加到尾部，Insert是加到中间，Before和After决定是加到对应中间件前面还是后面。
输入输出定义
// 最外层：GetVectorBucketInput/GetVectorBucketOutput
GetVectorBucketInput
// 然后内部层次
InitializeInput{
Parameters: GetVectorBucketInput,
}
SerializeInput{
Parameters: GetVectorBucketInput,
Request: s.newRequest(),
}
BuildInput{
Request: Request,
}
FinalizeInput{
Request: Request,
}
DeserializeInput{
Request: Request,
}
// ClientHandler输入
Request
// ClientHandler输出
res = Response{
Response: resp,// http.Response
}
DeserializeOutput{
RawResponse: res,// smithyhttp.Response，内部其实就是http.Response
Result: nil,
}
// Result需要DeserializeStep中的中间件反序列化出来
response, ok := out.RawResponse.(*smithyhttp.Response)
output := &GetVectorBucketOutput{}
out.Result = output//body反序列结果放在这里
FinalizeOutput{
Result: GetVectorBucketOutput,
}
BuildOutput{
Result: GetVectorBucketOutput
}
SerializeOutput{
Result: GetVectorBucketOutput
}
InitializeOutput{
Result: GetVectorBucketOutput
}
以获取header信息为例，只能在DeserializeStep中注册中间件来获取原始信息。默认情况下，已经注册了部分中间件获取原始http响应、响应时间、requestid等信息，放在了metadata中
metadata是一个map:
map[interface{}]interface{}
中间件
aws-sdk-go-v2中间件注册
注册到config中，应用到全局
最终目的是调用stack.xxxStep.Add()或者stack.xxxStep.Insert()等，将中间件插入到链式调用中。
但是这个注册行为是延迟执行的，而stack并没有暴露在外面，所以需要提供一个类似：
先看invokeOperation暴露了什么参数：
func (c *Client) invokeOperation(
ctx context.Context, opID string, params interface{}, optFns []func(*Options), stackFns ...func(*middleware.Stack, Options) error,
)(xxx){
stack := middleware.NewStack(opID, smithyhttp.NewStackRequest)
...
for _, fn := range optFns {
fn(&options)
}
...
for _, fn := range options.APIOptions {
if err := fn(stack); err != nil {
return nil, metadata, err
}
}
...
}
而APIOptions是个闭包的列表
APIOptions []func(*middleware.Stack) error
那么目的就变成了向options.APIOptions中插入一个闭包，而在创建s3vectors的client时(NewFromConfig和New)提供了这样的机会。用户可以提供一个func(*Options)的闭包，这个闭包会在创建client时执行。那么用户就可以借助这个机会在该闭包中操作options.APIOptions，注册中间件。
这样为了注册中间件，就需要提供三层操作：
- 第一层在NewFromConfig中提供一个
func(*Options)的闭包A，借此机会插入一个func(*middleware.Stack) error的闭包 - 第二层在
func(s *middleware.Stack) error闭包B中注册中间件，如s.Initialize.Add(xx,xx) - 第三层是中间件的实现（结构体C），例如Initialize的中间件需要实现对应的接口，例如
ID()和HandleInitialize
第一层其实也可以省略，因为config的APIOptions成员是公开的，可以直接插入自己的中间件注册逻辑B
注册到特定接口调用
前面实质上是直接注册到config的APIOptions中，闭包A的执行是在NewFromConfig中发生的。
但也可以延迟放进去，原理是invokeOperation中也提供了optFns的执行机会。
func (c *Client) GetVectorBucket(ctx context.Context, params *GetVectorBucketInput, optFns ...func(*Options)) (*GetVectorBucketOutput, error)
可以看到，可以把闭包A作为optFns参数传进去
中间件实现
示例：
// ComputeContentLength provides a middleware to set the content-length
// header for the length of a serialize request body.
type ComputeContentLength struct {
}
// ID returns the identifier for the ComputeContentLength.
func (m *ComputeContentLength) ID() string { return "ComputeContentLength" }
// HandleBuild adds the length of the serialized request to the HTTP header
// if the length can be determined.
func (m *ComputeContentLength) HandleBuild(
ctx context.Context, in middleware.BuildInput, next middleware.BuildHandler,
) (
out middleware.BuildOutput, metadata middleware.Metadata, err error,
) {
req, ok := in.Request.(*Request)
if !ok {
return out, metadata, fmt.Errorf("unknown request type %T", req)
}
// do nothing if request content-length was set to 0 or above.
if req.ContentLength >= 0 {
return next.HandleBuild(ctx, in)
}
// attempt to compute stream length
if n, ok, err := req.StreamLength(); err != nil {
return out, metadata, fmt.Errorf(
"failed getting length of request stream, %w", err)
} else if ok {
req.ContentLength = n
}
return next.HandleBuild(ctx, in)
}
每一个这样的中间件在Handlexxxx方法中都要执行next.Handlexxxx，这是一个中间件所必须实现的。
aws 官方示例
这个时候再看aws给出的官方计时示例：SDK 操作计时 - 适用于 Go 的 AWS SDK v2
import (
"context"
"fmt"
"log"
"net/http"
"sync"
"time"
awsmiddleware "github.com/aws/aws-sdk-go-v2/aws/middleware"
awshttp "github.com/aws/aws-sdk-go-v2/aws/transport/http"
"github.com/aws/aws-sdk-go-v2/config"
"github.com/aws/aws-sdk-go-v2/service/sqs"
"github.com/aws/smithy-go/middleware"
smithyrand "github.com/aws/smithy-go/rand"
)
// WithOperationTiming instruments an SQS client to dump timing information for
// the following spans:
// - overall operation time
// - HTTPClient call time
//
// This instrumentation will also emit the request ID, service name, and
// operation name for each invocation.
//
// Accepts a message "handler" which is invoked with formatted messages to be
// handled externally, you can use the declared PrintfMSGHandler to simply dump
// these values to stdout.
func WithOperationTiming(msgHandler func(string)) func(*sqs.Options) {
return func(o *sqs.Options) {
o.APIOptions = append(o.APIOptions, addTimingMiddlewares(msgHandler))
o.HTTPClient = &timedHTTPClient{
client: awshttp.NewBuildableClient(),
msgHandler: msgHandler,
}
}
}
// PrintfMSGHandler writes messages to stdout.
func PrintfMSGHandler(msg string) {
fmt.Printf("%s\n", msg)
}
type invokeIDKey struct{}
func setInvokeID(ctx context.Context, id string) context.Context {
return middleware.WithStackValue(ctx, invokeIDKey{}, id)
}
func getInvokeID(ctx context.Context) string {
id, _ := middleware.GetStackValue(ctx, invokeIDKey{}).(string)
return id
}
// Records the current time, and returns a function to be called when the
// target span of events is completed. The return function will emit the given
// span name and time elapsed to the given message consumer.
func timeSpan(ctx context.Context, name string, consumer func(string)) func() {
start := time.Now()
return func() {
elapsed := time.Now().Sub(start)
consumer(fmt.Sprintf("[%s] %s: %s", getInvokeID(ctx), name, elapsed))
}
}
type timedHTTPClient struct {
client *awshttp.BuildableClient
msgHandler func(string)
}
func (c *timedHTTPClient) Do(r *http.Request) (*http.Response, error) {
defer timeSpan(r.Context(), "http", c.msgHandler)()
resp, err := c.client.Do(r)
if err != nil {
return nil, fmt.Errorf("inner client do: %v", err)
}
return resp, nil
}
type addInvokeIDMiddleware struct {
msgHandler func(string)
}
func (*addInvokeIDMiddleware) ID() string { return "addInvokeID" }
func (*addInvokeIDMiddleware) HandleInitialize(ctx context.Context, in middleware.InitializeInput, next middleware.InitializeHandler) (
out middleware.InitializeOutput, md middleware.Metadata, err error,
) {
id, err := smithyrand.NewUUID(smithyrand.Reader).GetUUID()
if err != nil {
return out, md, fmt.Errorf("new uuid: %v", err)
}
return next.HandleInitialize(setInvokeID(ctx, id), in)
}
type timeOperationMiddleware struct {
msgHandler func(string)
}
func (*timeOperationMiddleware) ID() string { return "timeOperation" }
func (m *timeOperationMiddleware) HandleInitialize(ctx context.Context, in middleware.InitializeInput, next middleware.InitializeHandler) (
middleware.InitializeOutput, middleware.Metadata, error,
) {
defer timeSpan(ctx, "operation", m.msgHandler)()
return next.HandleInitialize(ctx, in)
}
type emitMetadataMiddleware struct {
msgHandler func(string)
}
func (*emitMetadataMiddleware) ID() string { return "emitMetadata" }
func (m *emitMetadataMiddleware) HandleInitialize(ctx context.Context, in middleware.InitializeInput, next middleware.InitializeHandler) (
middleware.InitializeOutput, middleware.Metadata, error,
) {
out, md, err := next.HandleInitialize(ctx, in)
invokeID := getInvokeID(ctx)
requestID, _ := awsmiddleware.GetRequestIDMetadata(md)
service := awsmiddleware.GetServiceID(ctx)
operation := awsmiddleware.GetOperationName(ctx)
m.msgHandler(fmt.Sprintf(`[%s] requestID = "%s"`, invokeID, requestID))
m.msgHandler(fmt.Sprintf(`[%s] service = "%s"`, invokeID, service))
m.msgHandler(fmt.Sprintf(`[%s] operation = "%s"`, invokeID, operation))
return out, md, err
}
func addTimingMiddlewares(mh func(string)) func(*middleware.Stack) error {
return func(s *middleware.Stack) error {
if err := s.Initialize.Add(&timeOperationMiddleware{msgHandler: mh}, middleware.Before); err != nil {
return fmt.Errorf("add time operation middleware: %v", err)
}
if err := s.Initialize.Add(&addInvokeIDMiddleware{msgHandler: mh}, middleware.Before); err != nil {
return fmt.Errorf("add invoke id middleware: %v", err)
}
if err := s.Initialize.Insert(&emitMetadataMiddleware{msgHandler: mh}, "RegisterServiceMetadata", middleware.After); err != nil {
return fmt.Errorf("add emit metadata middleware: %v", err)
}
return nil
}
}
func main() {
cfg, err := config.LoadDefaultConfig(context.Background())
if err != nil {
log.Fatal(fmt.Errorf("load default config: %v", err))
}
svc := sqs.NewFromConfig(cfg, WithOperationTiming(PrintfMSGHandler))
var wg sync.WaitGroup
for i := 0; i < 6; i++ {
wg.Add(1)
go func() {
defer wg.Done()
_, err = svc.ListQueues(context.Background(), nil)
if err != nil {
fmt.Println(fmt.Errorf("list queues: %v", err))
}
}()
}
wg.Wait()
}
可以清晰地看出注册中间件的闭包是通过addTimingMiddlewares返回的，共注册了三个中间件。
这中间利用了middleware.WithStackValue和middleware.GetStackValue在调用栈中传递信息。
扫一扫
2307
被折叠的 条评论
为什么被折叠?
到【灌水乐园】发言
