thaissa bueno sanches - consultant at avanade€¦ · ©2018 avanade inc. all rights reserved....
TRANSCRIPT
![Page 1: Thaissa Bueno Sanches - Consultant at Avanade€¦ · ©2018 Avanade Inc. All Rights Reserved. Thaissa Bueno Sanches - Consultant at Avanade Implantando modelos Deep Learning em cluster](https://reader036.vdocuments.mx/reader036/viewer/2022062602/5f01c2647e708231d400e679/html5/thumbnails/1.jpg)
©2018 Avanade Inc. All Rights Reserved.
Thaissa Bueno Sanches - Consultant at Avanade
Implantando modelos Deep Learning em cluster Kubernetes com GPU Ativada
![Page 2: Thaissa Bueno Sanches - Consultant at Avanade€¦ · ©2018 Avanade Inc. All Rights Reserved. Thaissa Bueno Sanches - Consultant at Avanade Implantando modelos Deep Learning em cluster](https://reader036.vdocuments.mx/reader036/viewer/2022062602/5f01c2647e708231d400e679/html5/thumbnails/2.jpg)
©2018 Avanade Inc. All Rights Reserved. <Highly Confidential> See Avanade’s Data Management Policy
• Noções básicas de implantação (carga útil, lotes, HTTP, Web Service)
• Comparação de GPU / CPU para inferência
• Kubernetes
• Etapas comuns
• Implantação no Kubernetes usando o Kubectl
• Implantação no Kubernetes usando o AzureML
• Implantação no Kubernetes usando Kubeflow e "TF serving"
Agenda
![Page 3: Thaissa Bueno Sanches - Consultant at Avanade€¦ · ©2018 Avanade Inc. All Rights Reserved. Thaissa Bueno Sanches - Consultant at Avanade Implantando modelos Deep Learning em cluster](https://reader036.vdocuments.mx/reader036/viewer/2022062602/5f01c2647e708231d400e679/html5/thumbnails/3.jpg)
©2018 Avanade Inc. All Rights Reserved. <Highly Confidential> See Avanade’s Data Management Policy
• O que é implantar?
Noções básicas de implantação (payload, batching, HTTP, Web Service)
![Page 4: Thaissa Bueno Sanches - Consultant at Avanade€¦ · ©2018 Avanade Inc. All Rights Reserved. Thaissa Bueno Sanches - Consultant at Avanade Implantando modelos Deep Learning em cluster](https://reader036.vdocuments.mx/reader036/viewer/2022062602/5f01c2647e708231d400e679/html5/thumbnails/4.jpg)
©2018 Avanade Inc. All Rights Reserved. <Highly Confidential> See Avanade’s Data Management Policy
• Payload
Noções básicas de implantação (payload, batching, HTTP, Web Service)
penultimate
layer
RGB Channels
of input imageConvolution layer
with KernelsPooling layer Fully connected layer
CatDogMouse
![Page 5: Thaissa Bueno Sanches - Consultant at Avanade€¦ · ©2018 Avanade Inc. All Rights Reserved. Thaissa Bueno Sanches - Consultant at Avanade Implantando modelos Deep Learning em cluster](https://reader036.vdocuments.mx/reader036/viewer/2022062602/5f01c2647e708231d400e679/html5/thumbnails/5.jpg)
©2018 Avanade Inc. All Rights Reserved. <Highly Confidential> See Avanade’s Data Management Policy
• Encode em base64
• Transfere arquivo
Noções básicas de implantação (payload, batching, HTTP, Web Service)
decoded_img = base64.b64decode(request.json["input"])img_buffer = BytesIO(decoded_img)pil_img = Image.open(img_buffer).convert("RGB")
Image.open(request.files['image']).convert("RGB")
![Page 6: Thaissa Bueno Sanches - Consultant at Avanade€¦ · ©2018 Avanade Inc. All Rights Reserved. Thaissa Bueno Sanches - Consultant at Avanade Implantando modelos Deep Learning em cluster](https://reader036.vdocuments.mx/reader036/viewer/2022062602/5f01c2647e708231d400e679/html5/thumbnails/6.jpg)
©2018 Avanade Inc. All Rights Reserved. <Highly Confidential> See Avanade’s Data Management Policy
• Batching
Noções básicas de implantação (payload, batching, HTTP, Web Service)
![Page 7: Thaissa Bueno Sanches - Consultant at Avanade€¦ · ©2018 Avanade Inc. All Rights Reserved. Thaissa Bueno Sanches - Consultant at Avanade Implantando modelos Deep Learning em cluster](https://reader036.vdocuments.mx/reader036/viewer/2022062602/5f01c2647e708231d400e679/html5/thumbnails/7.jpg)
©2018 Avanade Inc. All Rights Reserved. <Highly Confidential> See Avanade’s Data Management Policy
• HTTP
Noções básicas de implantação (payload, batching, HTTP, Web Service)
![Page 8: Thaissa Bueno Sanches - Consultant at Avanade€¦ · ©2018 Avanade Inc. All Rights Reserved. Thaissa Bueno Sanches - Consultant at Avanade Implantando modelos Deep Learning em cluster](https://reader036.vdocuments.mx/reader036/viewer/2022062602/5f01c2647e708231d400e679/html5/thumbnails/8.jpg)
©2018 Avanade Inc. All Rights Reserved. <Highly Confidential> See Avanade’s Data Management Policy
• Múltiplas conexões:
Uma conexão é aberta e somente é aberta uma nova conexão quando a anterior forfechada
Noções básicas de implantação (payload, batching, HTTP, Web Service)
![Page 9: Thaissa Bueno Sanches - Consultant at Avanade€¦ · ©2018 Avanade Inc. All Rights Reserved. Thaissa Bueno Sanches - Consultant at Avanade Implantando modelos Deep Learning em cluster](https://reader036.vdocuments.mx/reader036/viewer/2022062602/5f01c2647e708231d400e679/html5/thumbnails/9.jpg)
©2018 Avanade Inc. All Rights Reserved. <Highly Confidential> See Avanade’s Data Management Policy
• Conexões persistentes:
• Uma conexão é aberta e tem seu estado persistido até o termino de sua utilização.
Noções básicas de implantação (payload, batching, HTTP, Web Service)
![Page 10: Thaissa Bueno Sanches - Consultant at Avanade€¦ · ©2018 Avanade Inc. All Rights Reserved. Thaissa Bueno Sanches - Consultant at Avanade Implantando modelos Deep Learning em cluster](https://reader036.vdocuments.mx/reader036/viewer/2022062602/5f01c2647e708231d400e679/html5/thumbnails/10.jpg)
©2018 Avanade Inc. All Rights Reserved. <Highly Confidential> See Avanade’s Data Management Policy
• Pipelining:
Em apenas uma única solicitação TCP varias conexões HTTP são enviadas ao mesmo tempo.
Noções básicas de implantação (payload, batching, HTTP, Web Service)
![Page 11: Thaissa Bueno Sanches - Consultant at Avanade€¦ · ©2018 Avanade Inc. All Rights Reserved. Thaissa Bueno Sanches - Consultant at Avanade Implantando modelos Deep Learning em cluster](https://reader036.vdocuments.mx/reader036/viewer/2022062602/5f01c2647e708231d400e679/html5/thumbnails/11.jpg)
©2018 Avanade Inc. All Rights Reserved. <Highly Confidential> See Avanade’s Data Management Policy
• Head of line blocking:
Em apenas uma única solicitação TCP varias conexões HTTP são enviadas ao mesmo tempo porem uma ou mais solicitações podem ser bloqueada para aguardar o termino da anterior por ter excedido o limite de requests paralelos.
Noções básicas de implantação (payload, batching, HTTP, Web Service)
![Page 12: Thaissa Bueno Sanches - Consultant at Avanade€¦ · ©2018 Avanade Inc. All Rights Reserved. Thaissa Bueno Sanches - Consultant at Avanade Implantando modelos Deep Learning em cluster](https://reader036.vdocuments.mx/reader036/viewer/2022062602/5f01c2647e708231d400e679/html5/thumbnails/12.jpg)
©2018 Avanade Inc. All Rights Reserved. <Highly Confidential> See Avanade’s Data Management Policy
• Multiplexing:
A multiplexação é um método no HTTP / 2 pelo qual várias solicitações HTTP podem ser enviadas e as respostas podem ser recebidas de forma assíncrona por meio de uma única conexão TCP. A multiplexação é o coração do protocolo HTTP / 2
Noções básicas de implantação (payload, batching, HTTP, Web Service)
![Page 13: Thaissa Bueno Sanches - Consultant at Avanade€¦ · ©2018 Avanade Inc. All Rights Reserved. Thaissa Bueno Sanches - Consultant at Avanade Implantando modelos Deep Learning em cluster](https://reader036.vdocuments.mx/reader036/viewer/2022062602/5f01c2647e708231d400e679/html5/thumbnails/13.jpg)
©2018 Avanade Inc. All Rights Reserved. <Highly Confidential> See Avanade’s Data Management Policy
• Web Service:
• É uma solução para integração de sistemase comunicação com aplicações diferentes, podendo utilizar o protocolo de comunicação SOAP (Simple Object Access Protocol, em português Protocolo Simples de Acesso a Objetos) ou a arquitetura REST(Representational State Transfer em português Transferência de Estado Representacional).
Noções básicas de implantação (payload, batching, HTTP, Web Service)
Container Instances
![Page 14: Thaissa Bueno Sanches - Consultant at Avanade€¦ · ©2018 Avanade Inc. All Rights Reserved. Thaissa Bueno Sanches - Consultant at Avanade Implantando modelos Deep Learning em cluster](https://reader036.vdocuments.mx/reader036/viewer/2022062602/5f01c2647e708231d400e679/html5/thumbnails/14.jpg)
©2018 Avanade Inc. All Rights Reserved. <Highly Confidential> See Avanade’s Data Management Policy
• CPU VS GPU
Comparação de GPU / CPU para inferência
![Page 15: Thaissa Bueno Sanches - Consultant at Avanade€¦ · ©2018 Avanade Inc. All Rights Reserved. Thaissa Bueno Sanches - Consultant at Avanade Implantando modelos Deep Learning em cluster](https://reader036.vdocuments.mx/reader036/viewer/2022062602/5f01c2647e708231d400e679/html5/thumbnails/15.jpg)
©2018 Avanade Inc. All Rights Reserved. <Highly Confidential> See Avanade’s Data Management Policy
• CPU VS GPU
Comparação de GPU / CPU para inferência
![Page 16: Thaissa Bueno Sanches - Consultant at Avanade€¦ · ©2018 Avanade Inc. All Rights Reserved. Thaissa Bueno Sanches - Consultant at Avanade Implantando modelos Deep Learning em cluster](https://reader036.vdocuments.mx/reader036/viewer/2022062602/5f01c2647e708231d400e679/html5/thumbnails/16.jpg)
©2018 Avanade Inc. All Rights Reserved. <Highly Confidential> See Avanade’s Data Management Policy
• Kubernetes é um sistema de orquestração de contêiners open-source que automatiza a implantação, o dimensionamento e a gestão de aplicações em contêiners.
Kubernetes
![Page 17: Thaissa Bueno Sanches - Consultant at Avanade€¦ · ©2018 Avanade Inc. All Rights Reserved. Thaissa Bueno Sanches - Consultant at Avanade Implantando modelos Deep Learning em cluster](https://reader036.vdocuments.mx/reader036/viewer/2022062602/5f01c2647e708231d400e679/html5/thumbnails/17.jpg)
©2018 Avanade Inc. All Rights Reserved. <Highly Confidential> See Avanade’s Data Management Policy
Kubernetes
Node Node Node
Cluster
![Page 18: Thaissa Bueno Sanches - Consultant at Avanade€¦ · ©2018 Avanade Inc. All Rights Reserved. Thaissa Bueno Sanches - Consultant at Avanade Implantando modelos Deep Learning em cluster](https://reader036.vdocuments.mx/reader036/viewer/2022062602/5f01c2647e708231d400e679/html5/thumbnails/18.jpg)
©2018 Avanade Inc. All Rights Reserved. <Highly Confidential> See Avanade’s Data Management Policy
• Desenvolver modelo
• Desenvolver API de modelo
• Preparar contêiner docker para o serviço da web
• Implantar no Kubernetes
Etapas comuns
![Page 19: Thaissa Bueno Sanches - Consultant at Avanade€¦ · ©2018 Avanade Inc. All Rights Reserved. Thaissa Bueno Sanches - Consultant at Avanade Implantando modelos Deep Learning em cluster](https://reader036.vdocuments.mx/reader036/viewer/2022062602/5f01c2647e708231d400e679/html5/thumbnails/19.jpg)
©2018 Avanade Inc. All Rights Reserved. <Highly Confidential> See Avanade’s Data Management Policy
• Desenvolver modelo, modelo API, Flask App
• Crie uma imagem de contêiner com modelo, modelo API e Flask App
• Teste localmente e envie a imagem para o Docker Hub
• Provisionar o Cluster Kubernetes
• Conecte-se ao Kubernetes com kubectl
• Implantar aplicativo usando o manifesto (.yaml)
Implantação no Kubernetes usando o Kubectl
![Page 20: Thaissa Bueno Sanches - Consultant at Avanade€¦ · ©2018 Avanade Inc. All Rights Reserved. Thaissa Bueno Sanches - Consultant at Avanade Implantando modelos Deep Learning em cluster](https://reader036.vdocuments.mx/reader036/viewer/2022062602/5f01c2647e708231d400e679/html5/thumbnails/20.jpg)
©2018 Avanade Inc. All Rights Reserved. <Highly Confidential> See Avanade’s Data Management Policy
Implantação no Kubernetes usando o Kubectl
https://github.com/Microsoft/AKSDeploymentTutorial.git
![Page 21: Thaissa Bueno Sanches - Consultant at Avanade€¦ · ©2018 Avanade Inc. All Rights Reserved. Thaissa Bueno Sanches - Consultant at Avanade Implantando modelos Deep Learning em cluster](https://reader036.vdocuments.mx/reader036/viewer/2022062602/5f01c2647e708231d400e679/html5/thumbnails/21.jpg)
©2018 Avanade Inc. All Rights Reserved. <Highly Confidential> See Avanade’s Data Management Policy
• Configuração do AzureML, modelo de desenvolvimento, registro do modelo no espaço de trabalho do AzureML, desenvolvimento da API do modelo (script de pontuação)
• Crie imagens com o AzureML usando dependências do conda, requisitos de pip e outras dependências
• Capturar imagem do Azure Container Registry (ACR) e testar localmente
• Provisione o cluster do Kubernetes e implemente o serviço da Web com o AzureML.
Implantação no Kubernetes usando o AzureML
![Page 22: Thaissa Bueno Sanches - Consultant at Avanade€¦ · ©2018 Avanade Inc. All Rights Reserved. Thaissa Bueno Sanches - Consultant at Avanade Implantando modelos Deep Learning em cluster](https://reader036.vdocuments.mx/reader036/viewer/2022062602/5f01c2647e708231d400e679/html5/thumbnails/22.jpg)
©2018 Avanade Inc. All Rights Reserved. <Highly Confidential> See Avanade’s Data Management Policy
Implantação no Kubernetes usando o AzureML
https://github.com/Microsoft/AKSDeploymentTutorialAML.git
![Page 23: Thaissa Bueno Sanches - Consultant at Avanade€¦ · ©2018 Avanade Inc. All Rights Reserved. Thaissa Bueno Sanches - Consultant at Avanade Implantando modelos Deep Learning em cluster](https://reader036.vdocuments.mx/reader036/viewer/2022062602/5f01c2647e708231d400e679/html5/thumbnails/23.jpg)
©2018 Avanade Inc. All Rights Reserved. <Highly Confidential> See Avanade’s Data Management Policy
• Desenvolva modelo e salve como TensorFlow servable
• Puxe a imagem TensorFlow Serving do Docker hub, monte o caminho do modelo, abra a porta da API REST, teste localmente
• Crie um Kubernetes cluster, anexe o armazenamento blob no AKS, copie o modelo servable
• Instale o Ksonnet, Kubeflow e implante o webservice usando o componente KubeflowTensorFlow serving usando "ksonnet template".
Implantação no Kubernetes usando o AzureML
![Page 24: Thaissa Bueno Sanches - Consultant at Avanade€¦ · ©2018 Avanade Inc. All Rights Reserved. Thaissa Bueno Sanches - Consultant at Avanade Implantando modelos Deep Learning em cluster](https://reader036.vdocuments.mx/reader036/viewer/2022062602/5f01c2647e708231d400e679/html5/thumbnails/24.jpg)
©2018 Avanade Inc. All Rights Reserved. <Highly Confidential> See Avanade’s Data Management Policy
Implantação no Kubernetes usando o AzureML
https://github.com/Microsoft/DeployDLKubeflowAKS.git
![Page 25: Thaissa Bueno Sanches - Consultant at Avanade€¦ · ©2018 Avanade Inc. All Rights Reserved. Thaissa Bueno Sanches - Consultant at Avanade Implantando modelos Deep Learning em cluster](https://reader036.vdocuments.mx/reader036/viewer/2022062602/5f01c2647e708231d400e679/html5/thumbnails/25.jpg)
©2018 Avanade Inc. All Rights Reserved. <Highly Confidential> See Avanade’s Data Management Policy
©2018 Avanade Inc. All Rights Reserved.
Participe de um treinamento
GRATUITO de Azure Machine
Learning Service
bit.ly/azuremltdc
![Page 26: Thaissa Bueno Sanches - Consultant at Avanade€¦ · ©2018 Avanade Inc. All Rights Reserved. Thaissa Bueno Sanches - Consultant at Avanade Implantando modelos Deep Learning em cluster](https://reader036.vdocuments.mx/reader036/viewer/2022062602/5f01c2647e708231d400e679/html5/thumbnails/26.jpg)
Contatos
Site/Blog/Email:http://www.thaissasanches.com.brhttp://meetup.com/pt-BR/DevelopersBR/https://meetup.com/pt-BR/ai-brasil/
Redes Sociais:Linkedin: /in/thaissa-bueno-sanchesGithub: thayssa1186Twitter: thayssa1186
Vagas Avanade:https://careers.avanade.com/jobsenusurl/SearchJobs/?3_56_3=19753
©2018 Avanade Inc. All Rights Reserved.
![Page 27: Thaissa Bueno Sanches - Consultant at Avanade€¦ · ©2018 Avanade Inc. All Rights Reserved. Thaissa Bueno Sanches - Consultant at Avanade Implantando modelos Deep Learning em cluster](https://reader036.vdocuments.mx/reader036/viewer/2022062602/5f01c2647e708231d400e679/html5/thumbnails/27.jpg)
• Formada em tecnologia em rede de computadores pela UNIVEM/Marilia.
• Especialista em desenvolvimento .NET e Java.Pós Graduanda Machine Learning e Deep Learning na IGTI.
• Consultora de TI e Arquiteta
• Faço parte da coordenação do evento TDC – Trilha de IA
• Organizadora do AIFest 2018
• Uma das coordenadoras do Developers BR e IA Brasil além de varias comunidades de tecnologia que participo.
Quem sou eu?
©2018 Avanade Inc. All Rights Reserved.
![Page 28: Thaissa Bueno Sanches - Consultant at Avanade€¦ · ©2018 Avanade Inc. All Rights Reserved. Thaissa Bueno Sanches - Consultant at Avanade Implantando modelos Deep Learning em cluster](https://reader036.vdocuments.mx/reader036/viewer/2022062602/5f01c2647e708231d400e679/html5/thumbnails/28.jpg)
Thaissa Bueno Sanches
Consultant at Avanade
Obrigada