microservice monitoring and quality management for modern apps and infrastructures

Microservice Monitoring and Quality Management for Modern Apps and

Infrastructures

Speakers

Pavlo Baron, CTO, Instana

Ravi Yadav, Partnerships Architect, Mesosphere

Pedro Pacheco, Solutions Architect, Instana

Why Instana & DC/OS?

Pavlo Baron, CTO, Instana

Architectures Have Evolved

Host

Container Middlewa

reMicroservices

Business Service

Complexity + Scale Never Cease

Modern Architectures Have Evolved to a new Level of Distribution and

Scale

Mesosphere DC/OS

DC/OS is the Foundation for Modern Architecture Implementation

• Scheduling• Container lifecycle

management • Location transparent resource

management• Continuous deployment

DC/OS engineers and automates change

•Containers facilitating rapid development and deployment

•A container allocation from DC/OS change code, dependency, messaging structure

Feedback! What just happened!

• What changed?• Where did change occur?• Was there any quality impact?• What is the root cause?

In a system of constant change, operators need feedback!

Polling Question #1

© 2017 Mesosphere, Inc. All Rights Reserved. 9

MESOSPHERE DC/OS

Ravi Yadav, Partnerships Architect

PLATFORM SERVICES

CONTAINERORCHESTRATI

ONCI/CD BIG DATA

ANALYTICSMESSAGE

QUEUEDISTRIBUTED

DATABASE SEARCH

BUSINESS APPS

Distributed computing expertise.DevOps process & culture.

Cloud-native technologies.

OPERATIONAL EXPERTISE

INFRASTRUCTURE

PHYSICAL VIRTUAL CLOUD

BUILDING THE MODERN ENTERPRISE STACK

10

BUSINESS APPS

$

$ $

$$ $

$$$

$$

$$$ $$$ $$

$ $

$$$

$

$$

CLOUD LIMITATIONS

- LOCK-IN- HIGH COST- NO CONTROL

AWSPLATFORM SERVICES

OPERATIONS & TOOLS

ADMINISTRATORS OPERATIONAL PROCESSES PROPRIETARY TECHNOLOGIES


ONCI/CD BIG DATA

ANALYTICSMESSAGE

QUEUEDISTRIBUTED

DATABASE SEARCH

AWS INFRASTRUCTURE (EC2)

11

BUSINESS APPS

MESOSPHERE ENTERPRISE DC/OS


ONCI/CD BIG DATA

ANALYTICSMESSAGE

QUEUEDISTRIBUTED

DATABASE SEARCH

PLATFORM SERVICES

RESOURCEMANAGEMENT OPERATIONS SECURITY MULTI-TENANCY

DATACENTER-CLOUD OPERATING SYSTEM

APACHE MESOS

PRIVATE CLOUD AWS INFRASTRUCTURE (EC2)

GOOGLE CLOUD MICROSOFT AZURE

MESOSPHERE ENTERPRISE DC/OS

More than 100 services

CLOUD INDEPENDENCE

12

© 2017 Mesosphere, Inc. All Rights Reserved.

MESOSPHERE DC/OS: RUN YOUR DATACENTER AS A GIANT COMPUTER

Deploy & run complex services with an app store like experience

Manage your infrastructure as a single giant computing pool


NEW UIDC/OS

DC/OS 1.9


KEY THEMES - DC/OS 1.9

WORKLOADS

● Pods ● GPU based

scheduling

COMPLIANCEREPORTS

● NIST-800/53● ISO 27001● FISMA-Mod.● SOC-2● HIPAA

● Alluxio ● Couchbase● Datastax DSE ● Elastic (ELK)● Redis● Apache Flink

DATA SERVICES ECOSYSTEM

OPERATIONS

● Remote Container Shell

● Unified Metrics● Unified Logging● Deployment

Failure Debugging● Upgrades &

Configuration updates


WHY MESOSPHERE?

Unmatched Technology: DC/OS

● Containers in production at scale

● Stateful data services

● Maturity of technology

● Easy to install & run - anywhere (cloud, physical VM)

● Evergreen platform with DC/OS services

Experts on Distributed Systems● Experts & committers of open source

software (e.g., Mesos)

● Experience building & operating distributed systems at scale

● Experience helping enterprises solve real modern apps challenges

e

Polling Question #2

Host

Container

Middleware

Modern Architecture

MicroServices

Business Service

Modern Technology Stack

Business Service

Business Service

•Interconnected with multiple dependencies

•Many programming languages and frameworks

• Many container vendors• Variety of service types, micro

to macro• Legacy components

Challenge with Monitoring Modern Architectures

Host Middlewa

re

Microservice

Container

Custom Tracing or APM

Logging

Container monitoring

System monitoring

• Alerts come from technical silos

• No correlation between alerts and business services

• Alarms don’t map to problems• No context of situation• Sampling/statistical based

monitoring• Too much specialized

expertise needed to assure quality

No Alignment Between Tools and the Modern Stack

Business Service

Misalignment = Zero Visibility

Instana’s Dynamic APM: Bringing Order to Complexity and Chaos

Host

Container

Middleware

Code Exceptions/Errors

MicroService

Business Service

Business KPIs & SLAs

Automatic discovery of architecture with alignment of context:

• Vertical technical stack• Service dependency map• Measurement of service quality

Automatic Root Cause Analysis in 3 Seconds

Immediate exact problem identification with full context

Data Ingestion &Health

Calculation

Sensor Data

Real-time Stream Processing

Incident Detection

Alerting

Dependency

Health

Metrics

Dynamic Graph

API & CLI

Configuration

Instana Processing Pipeline

3 seconds from sensing to root cause identification

Quality of Service

3D Map

One Agent per Host

One Sensor per active component

Continuous discovery up the stack

Trace messages between microservices Sensor Repository

Agent

Knowledge Engine

Elasticsearch sensor

Tomcat sensor

JVM sensor

Linux sensor

Auto Discovery

Communication

LocalSensor Memory

&Contextual

Compression

Immediate, Automatic and Continuous Discovery of Components and Dependencies

Demo Use Cases

Continuous and Automatic Discovery

Application Mapping

Changes Detection and Marathon Events Correlation

Root Cause Understanding

Polling Question #3

microservice monitoring and quality management for modern apps and infrastructures

Software