参考博文

https://ewanvalentine.io/microservices-in-golang-part-1/
这个博文是作者微服务系统的第一篇，本学习笔记基于目前的5篇而成

part 1 利用gRPC,protobuf定义服务

本人在学习过程中没有严格按照博文中放在github目录，而是在主目录中创建一个wdyshippy的目录，目录中文件结构如下

.
├── consignment-cli
│   ├── cli.go
│   ├── consignment-cli
│   ├── consignment.json
│   ├── Dockerfile
│   └── Makefile
└── consignment-service
    ├── consignment-service
    ├── Dockerfile
    ├── main.go
    ├── Makefile
    └── proto
        └── consignment
            ├── consignment.pb.go
            └── consignment.proto

4 directories,11 files

执行的protoc命令

protoc -I. --go_out=plugins=grpc:/home/wdy/wdyshippy/consignment-service proto/consignment/consignment.proto

如果make build时报错

makefile:2: *** 遗漏分隔符

原因是在编写makefile文件时，命令前面应该是tab写为了空格

在这个阶段中，consignment-service的Makefile文件内容如下

build:
    protoc -I. --go_out=plugins=grpc:$(GOPATH)/src/github.com/ewanvalentine/shipper/consignment-service 
      proto/consignment/consignment.proto

part 2 - Docker and go-micro

创建Dockefiel

$ touch consignment-service/Dockerfile

写入如下内容

FROM alpine:latest

RUN mkdir /app
WORKDIR /app
ADD consignment-service /app/consignment-service

CMD ["./consignment-service"]

原作者提示如果是在linux机子上编译测试时，将alpine改为debian

FROM debian:latest

RUN mkdir /app
WORKDIR /app
ADD consignment-service /app/consignment-service

CMD ["./consignment-service"]

WORKDIR 表示 ／app目录作为上下文目录用来装载我们的程序consignment-service

接下来为Makefile文件增加内容用来生成docker image

build:
    ... 
    GOOS=linux GOARCH=amd64 go build
    docker build -t consignment-service .

除此之外再添加

run: 
    docker run -p 50051:50051 consignment-service

用来执行consignment-service

docker run -p 后面的参数，第一个端口号是对外的端口，第二个是内部的端口。

Go-micro

使用Go-micro来加入service discovery功能

首先安装

go get -u github.com/micro/protobuf/{proto,protoc-gen-go}

修改makefile文件中protoc的参数,将参数plugins=后面的grpc更改为micro

build:
    protoc -I. --go_out=plugins=micro:/home/wdy/wdyshippy/consignment-service proto/consignment/consignment.proto
    ...

...

代码也要相应修改

import中 要引入 micro “github.com/micro/go-micro”

// consignment-service/main.go
package main

import (

    // Import the generated protobuf code
    "fmt"

    pb "github.com/EwanValentine/shippy/consignment-service/proto/consignment"
    micro "github.com/micro/go-micro"
    "golang.org/x/net/context"
)

server接口实现时，response的位置改变为输入参数，输出参数只有error

func (s *service) CreateConsignment(ctx context.Context,req *pb.Consignment,res *pb.Response) error {
...
func (s *service) GetConsignments(ctx context.Context,req *pb.GetRequest,res *pb.Response) error {
  ...

还有就是server的初始化

func main() {

    repo := &Repository{}

    // Create a new service. Optionally include some options here.
    srv := micro.NewService(

        // This name must match the package name given in your protobuf definition
        micro.Name("go.micro.srv.consignment"),micro.Version("latest"),)

    // Init will parse the command line flags.
    srv.Init()

    // Register handler
    pb.RegisterShippingServiceHandler(srv.Server(),&service{repo})

    // Run the server
    if err := srv.Run(); err != nil {
        fmt.Println(err)
    }
}

最后就是不需要在代码中硬编码端口号，Go-micro通过环境变量或命令行参数来传递。

run:
    docker run -p 50051:50051 
        -e MICRO_SERVER_ADDRESS=:50051 
        -e MICRO_REGISTRY=mdns consignment-service

如上，通过 -e MICRO_SERVER_ADDRESS=:50051 指定服务地址，通过-e MICRO_REGISTRY=mdns指定service discovery功能使用mdns。在实际项目中mdns很少使用，大部分使用consul。

除了服务端代码更改外，客户端代码也需要更改

import (
    ...
    "github.com/micro/go-micro/cmd"
    microclient "github.com/micro/go-micro/client"

)

func main() {
    cmd.Init()

    // Create new greeter client
    client := pb.NewShippingServiceClient("go.micro.srv.consignment",microclient.DefaultClient)
    ...
}

通过docker启动consignment-service后，执行客户端程序会连接失败，原因在于server允许在docker中使用的是docker中的dmns，和客户端使用的不是同一个，所以发现不了server，解决方法就是把客户端程序也放入到docker 中，使用同一个dmns。

创建consignment-cli/Makefile文件

build:
    GOOS=linux GOARCH=amd64 go build
    docker build -t consignment-cli .

run:
    docker run -e MICRO_REGISTRY=mdns consignment-cli

Dockerfile文件

FROM alpine:latest

RUN mkdir -p /app
WORKDIR /app

ADD consignment.json /app/consignment.json
ADD consignment-cli /app/consignment-cli

CMD ["./consignment-cli"]

作者最后又提供了一个更加标准的Dockerfile文件，这个文件中包含了consignment-service的开发环境和生存环境

在开发环境中通过引用github.com/golang/dep/cmd/dep 来自动更新包依赖

RUN go get -u github.com/golang/dep/cmd/dep

# Create a dep project,and run `ensure`,which will pull in all 
# of the dependencies within this directory.
RUN dep init && dep ensure

完整文件如下

# consignment-service/Dockerfile

# We use the official golang image,which contains all the 
# correct build tools and libraries. Notice `as builder`,
# this gives this container a name that we can reference later on. 
FROM golang:1.9.0 as builder

# Set our workdir to our current service in the gopath
WORKDIR /go/src/github.com/EwanValentine/shippy/consignment-service

# Copy the current code into our workdir
COPY . .

# Here we're pulling in godep,which is a dependency manager tool,
# we're going to use dep instead of go get,to get around a few
# quirks in how go get works with sub-packages.
RUN go get -u github.com/golang/dep/cmd/dep

# Create a dep project,and run `ensure`,which will pull in all 
# of the dependencies within this directory.
RUN dep init && dep ensure

# Build the binary,with a few flags which will allow
# us to run this binary in Alpine. 
RUN CGO_ENABLED=0 GOOS=linux go build -a -installsuffix cgo .

# Here we're using a second FROM statement,which is strange,
# but this tells Docker to start a new build process with this
# image.
FROM alpine:latest

# Security related package,good to have.
RUN apk --no-cache add ca-certificates

# Same as before,create a directory for our app.
RUN mkdir /app
WORKDIR /app

# Here,instead of copying the binary from our host machine,
# we pull the binary from the container named `builder`,within
# this build context. This reaches into our previous image,finds
# the binary we built,and pulls it into this container. Amazing!
COPY --from=builder /go/src/github.com/EwanValentine/shippy/consignment-service/consignment-service .

# Run the binary as per usual! This time with a binary build in a
# separate container,with all of the correct dependencies and
# run time libraries.
CMD ["./consignment-service"]

part 3 - docker-compose and datastores

介绍了docker-compose的安装 Install docker-compose: https://docs.docker.com/compose/install/

还有 docker-compose的使用

docker-compose.yml的内容如下：

version: '3.1'

services: consignment-cli: build: ./consignment-cli environment: MICRO_REGISTRY: "mdns" consignment-service: build: ./consignment-service ports: - 50051:50051 environment: MICRO_ADDRESS: ":50051" MICRO_REGISTRY: "mdns" DB_HOST: "datastore:27017" vessel-service: build: ./vessel-service ports: - 50052:50051 environment: MICRO_ADDRESS: ":50051" MICRO_REGISTRY: "mdns"

还介绍了数据库包

http://jinzhu.me/gorm/

最后介绍了go-micro客户端另外一种编写方式

package main

import (
    "log"
    "os"

    pb "github.com/EwanValentine/shippy/user-service/proto/user"
    microclient "github.com/micro/go-micro/client"
    "github.com/micro/go-micro/cmd"
    "golang.org/x/net/context"
    "github.com/micro/cli"
    "github.com/micro/go-micro"
)


func main() {

    cmd.Init()

    // Create new greeter client
    client := pb.NewUserServiceClient("go.micro.srv.user",microclient.DefaultClient)

    // Define our flags
    service := micro.NewService(
        micro.Flags(
            cli.StringFlag{
                Name:  "name",Usage: "You full name",},cli.StringFlag{
                Name:  "email",Usage: "Your email",cli.StringFlag{
                Name:  "password",Usage: "Your password",cli.StringFlag{
                Name: "company",Usage: "Your company",),)

    // Start as service
    service.Init(

        micro.Action(func(c *cli.Context) {

            name := c.String("name")
            email := c.String("email")
            password := c.String("password")
            company := c.String("company")

            // Call our user service
            r,err := client.Create(context.TODO(),&pb.User{
                Name: name,Email: email,Password: password,Company: company,})
            if err != nil {
                log.Fatalf("Could not create: %v",err)
            }
            log.Printf("Created: %s",r.User.Id)

            getAll,err := client.GetAll(context.Background(),&pb.Request{})
            if err != nil {
                log.Fatalf("Could not list users: %v",err)
            }
            for _,v := range getAll.Users {
                log.Println(v)
            }

            os.Exit(0)
        }),)

    // Run the server
    if err := service.Run(); err != nil {
        log.Println(err)
    }
}

传递参数运行如下

$ docker-compose run user-cli command 
  --name="Ewan Valentine"    --email="ewan.valentine89@gmail.com"    --password="Testing123"    --company="BBC"

Part 4 - Authentication with JWT

运行两个数据库

$ docker run -d -p 5432:5432 postgres
$ docker run -d -p 27017:27017 mongo

JWT

https://jwt.io/

http://cryto.net/~joepie91/blog/2016/06/13/stop-using-jwt-for-sessions/

Cheat_Sheet_for_Java”>https://www.owasp.org/index.php/JSON_Web_Token(JWT)_Cheat_Sheet_for_Java

https://auth0.com/blog/json-web-token-signing-algorithms-overview/

https://tools.ietf.org/html/rfc7518#section-3

Go library for this: github.com/dgrijalva/jwt-go

User-service

负责用户信息的认证和token的发放校验

https://github.com/EwanValentine/shippy/blob/master/user-service/token_service.go

Token_service.go中代码负责jwt的编码(Encode)和解码(Decode)，用于server的调用

Encode

// Encode a claim into a JWT
func (srv *TokenService) Encode(user *pb.User) (string,error) {
    // Create the Claims
    claims := CustomClaims{
        user,jwt.StandardClaims{
            ExpiresAt: 15000,Issuer:    "go.micro.srv.user",}

    // Create token
    token := jwt.NewWithClaims(jwt.SigningMethodHS256,claims)

    // Sign token and return
    return token.SignedString(key)
}

上面代码有误，设置ExpiresAt 为15000，运行程序会报token过期，正确代码为

// Encode a claim into a JWT
func (srv *TokenService) Encode(user *User) (string,error) {

    expireToken := time.Now().Add(time.Hour * 72).Unix()

    // Create the Claims
    claims := CustomClaims{
        user,jwt.StandardClaims{
            ExpiresAt: expireToken,claims)

    // Sign token and return
    return token.SignedString(key)
}

Decode

func (srv *TokenService) Decode(token string) (*CustomClaims,error) {

    // Parse the token
    tokenType,err := jwt.ParseWithClaims(string(key),&CustomClaims{},func(token *jwt.Token) (interface{},error) {
        return key,nil
    })

    // Validate the token and return the custom claims
    if claims,ok := tokenType.Claims.(*CustomClaims); ok && tokenType.Valid {
        return claims,nil
    } else {
        return nil,err
    }
}

上面代码运行会panic

2018/01/17 01:39:31 panic recovered: runtime error: invalid memory address or nil pointer dereference
2018/01/17 01:39:31 goroutine 35 [running]:
runtime/debug.Stack(0xc420121920,0x2,0x2)
        /home/wdy/go/src/runtime/debug/stack.go:24 +0x79
github.com/micro/go-micro/server.(*rpcServer).accept.func1(0xb8dfe0,0xc4200e4f80)
        /home/wdy/svn/cloud_trunk/factory/branches/tob_material/main/tobmaterialsys/src/github.com/micro/go-micro/server/rpc_server.go:60 +0x124
panic(0x8d1720,0xbbe270)
        /home/wdy/go/src/runtime/panic.go:489 +0x2cf
main.(*TokenService).Decode(0xc420011560,0xc4202cf080,0x15b,0xc420266af0,0xc42011e780,0xc4200326b8)
        /home/wdy/wdyshippy/user-service/token_service.go:45 +0x10c
main.(*service).ValidateToken(0xc4200b1740,0x7f90d890c050,0xc420271140,0xc420271170,0xc420032738,0x41168c)
        /home/wdy/wdyshippy/user-service/handler.go:76 +0x11c
_/home/wdy/wdyshippy/user-service/proto/user.(*UserService).ValidateToken(0xc420011af0,0x0,0x0)
        /home/wdy/wdyshippy/user-service/proto/user/user.pb.go:310 +0x5b
reflect.Value.call(0xc42005ea00,0xc42000e1d0,0x13,0x95de61,0x4,0xc420032bb0,0x901c80,0x913a40,...)
        /home/wdy/go/src/reflect/value.go:434 +0x91f
reflect.Value.Call(0xc42005ea00,0x40,0x38)
        /home/wdy/go/src/reflect/value.go:302 +0xa4
github.com/micro/go-micro/server.(*service).call.func1(0x7f90d890c050,0xb8fdc0,0xc420132f00,0x0)

原因在于Decode中jwt.ParseWithClaims的第一个参数应该是token，正确代码为：

// Decode a token string into a token object
func (srv *TokenService) Decode(tokenString string) (*CustomClaims,error) {

    // Parse the token
    token,err := jwt.ParseWithClaims(tokenString,func(token *jwt.Token) (interface{},error) {
        return key,nil
    })

    // Validate the token and return the custom claims
    if claims,ok := token.Claims.(*CustomClaims); ok && token.Valid {
        return claims,nil
    } else {
        return nil,err
    }
}

https://github.com/EwanValentine/shippy/blob/master/user-service/handler.go

负责用户的创建，密码校验和token发放和校验

golang.org/x/crypto/bcrypt

用户创建

func (srv *service) Create(ctx context.Context,req *pb.User,res *pb.Response) error {

    // Generates a hashed version of our password
    hashedPass,err := bcrypt.GenerateFromPassword([]byte(req.Password),bcrypt.DefaultCost)
    if err != nil {
        return err
    }
    req.Password = string(hashedPass)
    if err := srv.repo.Create(req); err != nil {
        return err
    }
    res.User = req
    return nil
}

用户密码校验，校验成功则发放token

func (srv *service) Auth(ctx context.Context,req *pb.User,res *pb.Token) error {
    log.Println("Logging in with:",req.Email,req.Password)
    user,err := srv.repo.GetByEmail(req.Email)
    log.Println(user)
    if err != nil {
        return err
    }

    // Compares our given password against the hashed password
    // stored in the database
    if err := bcrypt.CompareHashAndPassword([]byte(user.Password),[]byte(req.Password)); err != nil {
        return err
    }

    token,err := srv.tokenService.Encode(user)
    if err != nil {
        return err
    }
    res.Token = token
    return nil
}

token校验

func (srv *service) ValidateToken(ctx context.Context,req *pb.Token,res *pb.Token) error {

    // Decode token
    claims,err := srv.tokenService.Decode(req.Token)
    if err != nil {
        return err
    }

    log.Println(claims)

    if claims.User.Id == "" {
        return errors.New("invalid user")
    }

    res.Valid = true

    return nil
}

consignment-cli

客户端请求的时候加上token

github.com/micro/go-micro/metadata

...
ctx := metadata.NewContext(context.Background(),map[string]string{
        "token": token,})

    r,err := client.CreateConsignment(ctx,consignment)
    if err != nil {
        log.Fatalf("Could not create: %v",err)
    }
    log.Printf("Created: %t",r.Created)

    getAll,err := client.GetConsignments(ctx,&pb.GetRequest{})
    if err != nil {
        log.Fatalf("Could not list consignments: %v",err)
    }
    for _,v := range getAll.Consignments {
        log.Println(v)
    }
...

consignment-server

https://github.com/EwanValentine/shippy/tree/master/consignment-service

// shippy-consignment-service/main.go
func main() {
    ... 
    // Create a new service. Optionally include some options here.
    srv := micro.NewService(

        // This name must match the package name given in your protobuf definition
        micro.Name("go.micro.srv.consignment"),micro.Version("latest"),// Our auth middleware
        micro.WrapHandler(AuthWrapper),)
    ...
}

... 

// AuthWrapper is a high-order function which takes a HandlerFunc
// and returns a function,which takes a context,request and response interface.
// The token is extracted from the context set in our consignment-cli,that
// token is then sent over to the user service to be validated.
// If valid,the call is passed along to the handler. If not,
// an error is returned.
func AuthWrapper(fn server.HandlerFunc) server.HandlerFunc {
    return func(ctx context.Context,req server.Request,resp interface{}) error {
        meta,ok := metadata.FromContext(ctx)
        if !ok {
            return errors.New("no auth meta-data found in request")
        }

        // Note this is now uppercase (not entirely sure why this is...)
        token := meta["Token"]
        log.Println("Authenticating with token: ",token)

        // Auth here
        authClient := userService.NewUserServiceClient("go.micro.srv.user",client.DefaultClient)
        _,err := authClient.ValidateToken(context.Background(),&userService.Token{
            Token: token,})
        if err != nil {
            return err
        }
        err = fn(ctx,req,resp)
        return err
    }
}

AuthWarpper是一个中间件，接受一个HandlerFunc，进行某种处理后 返回HandlerFunc。

该中间件从context提取token，然后发送到user-service中校验token是否有效，如果有效再继续执行真正的操作fn。

Part 5 - Event brokering with Go Micro

修改 user-service/main.go

func main() {
    ... 
    // Init will parse the command line flags.
    srv.Init()

    // Get instance of the broker using our defaults
    pubsub := srv.Server().Options().Broker

    // Register handler
    pb.RegisterUserServiceHandler(srv.Server(),&service{repo,tokenService,pubsub})
    ...
}

修改user-service/handler.go

const topic = "user.created"

type service struct {
    repo         Repository
    tokenService Authable
    PubSub       broker.Broker
}

func (srv *service) Create(ctx context.Context,bcrypt.DefaultCost)
    if err != nil {
        return err
    }
    req.Password = string(hashedPass)
    if err := srv.repo.Create(req); err != nil {
        return err
    }
    res.User = req
    if err := srv.publishEvent(req); err != nil {
        return err
    }
    return nil
}

func (srv *service) publishEvent(user *pb.User) error {
    // Marshal to JSON string
    body,err := json.Marshal(user)
    if err != nil {
        return err
    }

    // Create a broker message
    msg := &broker.Message{
        Header: map[string]string{
            "id": user.Id,Body: body,}

    // Publish message to broker
    if err := srv.PubSub.Publish(topic,msg); err != nil {
        log.Printf("[pub] failed: %v",err)
    }

    return nil
}

除此之外 user-service handler.go import添加

?```go
"github.com/micro/go-micro/broker"

_ "github.com/micro/go-plugins/broker/nats"

?```

user-service/Makefile中添加

-e MICRO_BROKER=nats 

        -e MICRO_BROKER_ADDRESS=0.0.0.0:4222 

email-service参见原作者的repo https://github.com/EwanValentine/shippy-email-service

srv.Init()
pubsub := srv.Server().Options().Broker

在go-micro中， srv.Init()会搜索所有的配置，包括plugin，环境变量以及命令行参数，然后把这些配置初始化为service的组成部分。为了使用这些配置实例，需要通过 srv.Server().Options(),在本项目例子中，通过指定

-e MICRO_BROKER=nats 
        -e MICRO_BROKER_ADDRESS=0.0.0.0:4222 

go-micro会找到NATS的broker plugin，创建对应的实例，用于之后的连接和使用。

在email-server中，用来订阅event

if err := pubsub.Connect(); err != nil {
        log.Fatal(err)
    }

    // Subscribe to messages on the broker
    _,err := pubsub.Subscribe(topic,func(p broker.Publication) error {
        var user *pb.User
        if err := json.Unmarshal(p.Message().Body,&user); err != nil {
            return err
        }
        log.Println(user)
        go sendEmail(user)
        return nil
    })

在user-service中，用来发布event

func (srv *service) publishEvent(user *pb.User) error {
    // Marshal to JSON string
    body,err)
    }

    return nil
}

遇到的问题以及解决方法

NATS配置

本人成功的方法如下

wdy@wdy:~$ docker pull nats
Using default tag: latest
latest: Pulling from library/nats
f169c9506d74: Pull complete
bb9eff5cafb0: Pull complete
Digest: sha256:61fcb1f40da2111434fc910b0865c54155cd6e5f7c42e56e031c3f35a9998075
Status: Downloaded newer image for nats:latest

wdy@wdy:~$ docker run -p 4222:4222 -p 8222:8222 -p 6222:6222 --name gnatsd -ti nats:latest
[1] 2018/01/17 05:45:07.167855 [INF] Starting nats-server version 1.0.4
[1] 2018/01/17 05:45:07.167935 [INF] Starting http monitor on 0.0.0.0:8222
[1] 2018/01/17 05:45:07.167961 [INF] Listening for client connections on 0.0.0.0:4222
[1] 2018/01/17 05:45:07.167964 [INF] Server is ready
[1] 2018/01/17 05:45:07.168111 [INF] Listening for route connections on 0.0.0.0:6222

user-service nats连接失败

User-service 执行make run之后报如下错误

nats: no servers available for connection

解决办法，将user_serice中的makefile的

run:
    docker run -p 50053:50051 
    -e MICRO_SERVER_ADDRESS=:50051 
    -e MICRO_REGISTRY=mdns 
    user-service

改为

run:
    docker run --net="host" 
    -p 50055 
    -e MICRO_SERVER_ADDRESS=:50055 
    -e MICRO_REGISTRY=mdns 
    -e MICRO_BROKER=nats 
    -e MICRO_BROKER_ADDRESS=0.0.0.0:4222 

//TODO 等作者6-11的更新后，本文再同步更新

（编辑：李大同）

【声明】本站内容均来自网络，其相关言论仅代表作者个人观点，不代表本站立场。若无意侵犯到您的权利，请及时与联系站长删除相关内容!