highly scalable caching service on cloud - redis
TRANSCRIPT
HUAWEI TECHNOLOGIES CO., LTD.
www.huawei.com
Slide title :40-47pt
Slide subtitle :26-30pt
Color::white
Corporate Font :
FrutigerNext LT Medium
Font to be used by customers and
partners :
Arial
Highly Scalable Caching Service on Cloud Krishna Kumar & Irfan Rehman Huawei India Cloud Platforms Team
HUAWEI TECHNOLOGIES CO., LTD.
Slide title :32-35pt
Color: R153 G0 B0
Corporate Font :
FrutigerNext LT Medium
Font to be used by customers and
partners :
Arial
Slide text :20-22pt
Bullets level 2-5:
18pt
Color:Black
Corporate Font :
FrutigerNext LT Medium
Font to be used by customers and
partners :
Arial
Top right corner
for field-mark,
customer or
partner logotypes.
----------------
The following
nine groups of
colors are an
example of how
our design colors
can be used,
please take note
that you should
only use one
design color
group per slide.
For specific
usage details,
refer to the
“Typesetting
Standard”.
2
We will discuss
Introduction
In-memory caching
Industry use cases
Types of cloud service provisioning
The problem to scale..
Resourse scheduler - Solution 1
Resourse scheduler - Solution 2
Demo
Q&A
HUAWEI TECHNOLOGIES CO., LTD.
Slide title :32-35pt
Color: R153 G0 B0
Corporate Font :
FrutigerNext LT Medium
Font to be used by customers and
partners :
Arial
Slide text :20-22pt
Bullets level 2-5:
18pt
Color:Black
Corporate Font :
FrutigerNext LT Medium
Font to be used by customers and
partners :
Arial
Top right corner
for field-mark,
customer or
partner logotypes.
----------------
The following
nine groups of
colors are an
example of how
our design colors
can be used,
please take note
that you should
only use one
design color
group per slide.
For specific
usage details,
refer to the
“Typesetting
Standard”.
3
Introduction to Cloud Caching Service
Cache Component that stores data so future requests for that data can be served faster. Distributed Cache Extension of the traditional concept of cache used in a single locale. A distributed cache may span multiple servers so that it can grow in size and in transactional capacity. It is mainly used to store application data residing in database and web session data. The distributed cache architecture is built on scale-out, with inherent partitioning capabilities to spread the workload. Cloud Service Any resource that is provided over the Internet. The most common cloud service resources are Software as a Service (SaaS), Platform as a Service (PaaS) and Infrastructure as a Service (IaaS).
users cache
HUAWEI TECHNOLOGIES CO., LTD.
Slide title :32-35pt
Color: R153 G0 B0
Corporate Font :
FrutigerNext LT Medium
Font to be used by customers and
partners :
Arial
Slide text :20-22pt
Bullets level 2-5:
18pt
Color:Black
Corporate Font :
FrutigerNext LT Medium
Font to be used by customers and
partners :
Arial
Top right corner
for field-mark,
customer or
partner logotypes.
----------------
The following
nine groups of
colors are an
example of how
our design colors
can be used,
please take note
that you should
only use one
design color
group per slide.
For specific
usage details,
refer to the
“Typesetting
Standard”.
4
• We focus on in-memory caches and exclude
others (ssd, multi-tier, etc) from our discussion
HUAWEI TECHNOLOGIES CO., LTD.
Slide title :32-35pt
Color: R153 G0 B0
Corporate Font :
FrutigerNext LT Medium
Font to be used by customers and
partners :
Arial
Slide text :20-22pt
Bullets level 2-5:
18pt
Color:Black
Corporate Font :
FrutigerNext LT Medium
Font to be used by customers and
partners :
Arial
Top right corner
for field-mark,
customer or
partner logotypes.
----------------
The following
nine groups of
colors are an
example of how
our design colors
can be used,
please take note
that you should
only use one
design color
group per slide.
For specific
usage details,
refer to the
“Typesetting
Standard”.
5
Major in-Memory Cache Uses Cases
Database offloading in case of slow database or disk access Web content caching in case of high latency and narrow bandwidth to actual servers Low latency data access for time critical operations Pub/Sub mechanisms for multiple services and multiple consumers Message queues distributed network message passing mechanisms CDN caches in-memory caches especially for content index and ranking, not necessarily for content itself Caching and computation sorted sets, lists, queues, etc
HUAWEI TECHNOLOGIES CO., LTD.
Slide title :32-35pt
Color: R153 G0 B0
Corporate Font :
FrutigerNext LT Medium
Font to be used by customers and
partners :
Arial
Slide text :20-22pt
Bullets level 2-5:
18pt
Color:Black
Corporate Font :
FrutigerNext LT Medium
Font to be used by customers and
partners :
Arial
Top right corner
for field-mark,
customer or
partner logotypes.
----------------
The following
nine groups of
colors are an
example of how
our design colors
can be used,
please take note
that you should
only use one
design color
group per slide.
For specific
usage details,
refer to the
“Typesetting
Standard”.
6
Leading Industry Cache Offerings
Redis Memcached Apache Hbase CounchBase Basho Riak Oracle Coherence DataStax Cloudant AeroSpike
See other players in the domain of caching: https://www.g2crowd.com/products/redis/competitors/alternatives
HUAWEI TECHNOLOGIES CO., LTD.
Slide title :32-35pt
Color: R153 G0 B0
Corporate Font :
FrutigerNext LT Medium
Font to be used by customers and
partners :
Arial
Slide text :20-22pt
Bullets level 2-5:
18pt
Color:Black
Corporate Font :
FrutigerNext LT Medium
Font to be used by customers and
partners :
Arial
Top right corner
for field-mark,
customer or
partner logotypes.
----------------
The following
nine groups of
colors are an
example of how
our design colors
can be used,
please take note
that you should
only use one
design color
group per slide.
For specific
usage details,
refer to the
“Typesetting
Standard”.
7
• We chose redis for cloud integration evaluation..
HUAWEI TECHNOLOGIES CO., LTD.
Slide title :32-35pt
Color: R153 G0 B0
Corporate Font :
FrutigerNext LT Medium
Font to be used by customers and
partners :
Arial
Slide text :20-22pt
Bullets level 2-5:
18pt
Color:Black
Corporate Font :
FrutigerNext LT Medium
Font to be used by customers and
partners :
Arial
Top right corner
for field-mark,
customer or
partner logotypes.
----------------
The following
nine groups of
colors are an
example of how
our design colors
can be used,
please take note
that you should
only use one
design color
group per slide.
For specific
usage details,
refer to the
“Typesetting
Standard”.
8
Most popular in-memory cache in the world
Redis is an open source (BSD licensed), in-memory data structure store, used as database,
cache and message broker. It supports data structures such as strings, hashes, lists, sets,
sorted sets with range queries, bitmaps, hyperloglogs and geospatial indexes with radius
queries. Redis has built-in replication, Lua scripting, LRU eviction, transactions and
different levels of on-disk persistence, and provides high availability via Redis Sentinel and
automatic partitioning with Redis Cluster.
Web
App
DB
Web
App Web
App Web
App Volatile/temp data
Why Redis NoSQL?
No lock execution
Effectively uses individual core per
instance
Lot of data structure use cases
Blazingly fast for a single core
performer
HUAWEI TECHNOLOGIES CO., LTD.
Slide title :32-35pt
Color: R153 G0 B0
Corporate Font :
FrutigerNext LT Medium
Font to be used by customers and
partners :
Arial
Slide text :20-22pt
Bullets level 2-5:
18pt
Color:Black
Corporate Font :
FrutigerNext LT Medium
Font to be used by customers and
partners :
Arial
Top right corner
for field-mark,
customer or
partner logotypes.
----------------
The following
nine groups of
colors are an
example of how
our design colors
can be used,
please take note
that you should
only use one
design color
group per slide.
For specific
usage details,
refer to the
“Typesetting
Standard”.
9
• The problem to scale…
HUAWEI TECHNOLOGIES CO., LTD.
Slide title :32-35pt
Color: R153 G0 B0
Corporate Font :
FrutigerNext LT Medium
Font to be used by customers and
partners :
Arial
Slide text :20-22pt
Bullets level 2-5:
18pt
Color:Black
Corporate Font :
FrutigerNext LT Medium
Font to be used by customers and
partners :
Arial
Top right corner
for field-mark,
customer or
partner logotypes.
----------------
The following
nine groups of
colors are an
example of how
our design colors
can be used,
please take note
that you should
only use one
design color
group per slide.
For specific
usage details,
refer to the
“Typesetting
Standard”.
10
Redis Service in cloud – major Vendors 1) Amazon - Elastic Cache: it easy to deploy, operate, and scale an in-memory cache in the cloud. It
support Redis Master/Slavce replication and Multi AZ - https://aws.amazon.com/elasticache/
2) Microsoft - gives you Redis reliable hosting and monitoring - https://azure.microsoft.com/en-us/documentation/services/redis-cache/
3) Google – Deploy Redis on compute Engine. With Redis-as-a-Service, Redislabs will manage and monitor your instance on Google Cloud Platform. https://cloud.google.com/solutions/redis/
4) IBM - Support Redis service on SoftLayer - https://marketplace.ibmcloud.com/apps/87#!overview
5) RedHat OpenShift – Redis support for community-contributed cartridges - https://blog.openshift.com/enabling-redis-for-your-app/
6) Digital Ocean – Supports using Redis in the cloud - https://www.digitalocean.com/community/tutorials/how-to-install-and-use-redis
7) Alibaba Aliyun – Aliyun Key-Value Store (KVStore for short) is an online Key-Value storage service compatible with the open-source Redis protocol - http://www.aliyun.com/product/kvstore/?lang=en
8) Redis Labs – Enterprise cluster; offered as a fully managed service, also used as third party offering by other vendors.
HUAWEI TECHNOLOGIES CO., LTD.
Slide title :32-35pt
Color: R153 G0 B0
Corporate Font :
FrutigerNext LT Medium
Font to be used by customers and
partners :
Arial
Slide text :20-22pt
Bullets level 2-5:
18pt
Color:Black
Corporate Font :
FrutigerNext LT Medium
Font to be used by customers and
partners :
Arial
Top right corner
for field-mark,
customer or
partner logotypes.
----------------
The following
nine groups of
colors are an
example of how
our design colors
can be used,
please take note
that you should
only use one
design color
group per slide.
For specific
usage details,
refer to the
“Typesetting
Standard”.
11
One view of Redis service provisioning in cloud
Instance on Bare metal - Toughest to maintain, but provides best performance - A single instance wastes rest of the cpu resources (complicated
multi-tenancy)
Instance on Virtual Machine - Lesser performance - Scaling means provisioning another VM
Instance in a container - Lesser performance, given the network translations - Scaling is easy, but seamless is difficult if the redis cluster is non
uniform (server is different type, proxy is another, etc) • Eventually all the above is resources. • Service Instances could be placed anywhere in the pool. • In fact different types could coexist, and communicate, if the
network allows so.
HUAWEI TECHNOLOGIES CO., LTD.
Slide title :32-35pt
Color: R153 G0 B0
Corporate Font :
FrutigerNext LT Medium
Font to be used by customers and
partners :
Arial
Slide text :20-22pt
Bullets level 2-5:
18pt
Color:Black
Corporate Font :
FrutigerNext LT Medium
Font to be used by customers and
partners :
Arial
Top right corner
for field-mark,
customer or
partner logotypes.
----------------
The following
nine groups of
colors are an
example of how
our design colors
can be used,
please take note
that you should
only use one
design color
group per slide.
For specific
usage details,
refer to the
“Typesetting
Standard”.
12
• The problem to scale given resources..
needs resource scheduling…
HUAWEI TECHNOLOGIES CO., LTD.
Slide title :32-35pt
Color: R153 G0 B0
Corporate Font :
FrutigerNext LT Medium
Font to be used by customers and
partners :
Arial
Slide text :20-22pt
Bullets level 2-5:
18pt
Color:Black
Corporate Font :
FrutigerNext LT Medium
Font to be used by customers and
partners :
Arial
Top right corner
for field-mark,
customer or
partner logotypes.
----------------
The following
nine groups of
colors are an
example of how
our design colors
can be used,
please take note
that you should
only use one
design color
group per slide.
For specific
usage details,
refer to the
“Typesetting
Standard”.
13
Redis service instance topologies
When a customer requests for a redis instance from cloud it could be one for the following configuration.
Single Redis-Server •CPU = 1 •mem = variable •Disk = optional
Master/Slave Redis • Primarily for replication • CPU= 1 + N Slaves • mem = variable • Disk = optional
master
Slave 1 Slave 2
Proxy
Redis Cluster with Proxy • Only Using twem proxy or Codis for data shard • Popular with older version of redis eg:2.8 • CPU= Number of nodes
Redis 3.0 Cluster •Latest offering from Redis ; No proxy required. •Possible to horizontally scale ; CPU= Number of nodes
HUAWEI TECHNOLOGIES CO., LTD.
Slide title :32-35pt
Color: R153 G0 B0
Corporate Font :
FrutigerNext LT Medium
Font to be used by customers and
partners :
Arial
Slide text :20-22pt
Bullets level 2-5:
18pt
Color:Black
Corporate Font :
FrutigerNext LT Medium
Font to be used by customers and
partners :
Arial
Top right corner
for field-mark,
customer or
partner logotypes.
----------------
The following
nine groups of
colors are an
example of how
our design colors
can be used,
please take note
that you should
only use one
design color
group per slide.
For specific
usage details,
refer to the
“Typesetting
Standard”.
14
Proxy based sharding Twitter handled redis cluster by implementing twemproxy
Shard aware clients
External Cluster manager Wayfair an online furniture store managed redis cluster using zookeeper
Enterprise Cluster Redis Labs Enterprise cluster - https://redislabs.com/redis-cloud
Redis 3.0 http://redis.io/download
How traditionally Industry handled Redis so far…
HUAWEI TECHNOLOGIES CO., LTD.
Slide title :32-35pt
Color: R153 G0 B0
Corporate Font :
FrutigerNext LT Medium
Font to be used by customers and
partners :
Arial
Slide text :20-22pt
Bullets level 2-5:
18pt
Color:Black
Corporate Font :
FrutigerNext LT Medium
Font to be used by customers and
partners :
Arial
Top right corner
for field-mark,
customer or
partner logotypes.
----------------
The following
nine groups of
colors are an
example of how
our design colors
can be used,
please take note
that you should
only use one
design color
group per slide.
For specific
usage details,
refer to the
“Typesetting
Standard”.
15
• We tried couple of solutions using
mesos and kubernetes…
HUAWEI TECHNOLOGIES CO., LTD.
Slide title :32-35pt
Color: R153 G0 B0
Corporate Font :
FrutigerNext LT Medium
Font to be used by customers and
partners :
Arial
Slide text :20-22pt
Bullets level 2-5:
18pt
Color:Black
Corporate Font :
FrutigerNext LT Medium
Font to be used by customers and
partners :
Arial
Top right corner
for field-mark,
customer or
partner logotypes.
----------------
The following
nine groups of
colors are an
example of how
our design colors
can be used,
please take note
that you should
only use one
design color
group per slide.
For specific
usage details,
refer to the
“Typesetting
Standard”.
16
Mesos Kubernetes
Apache Mesos is a cluster manager that provides efficient
resource isolation and sharing across distributed
applications, or frameworks.
Apache Mesos abstracts CPU, memory, storage, and other
compute resources away from machines (physical or
virtual), enabling fault-tolerant and elastic distributed
systems.
Google Kubernetes brings 'Google style' cluster
management capabilities to the world of Docker
containers on the data center scenarios.
For production Google uses Omega/Borg, not Kubernetes
which is open sourced in June 2014.
• Mesos using same principle as Linux Kernel but higher
layer of abstraction.
• It can run Hadoop, MPI, Hypertable, Spark, and other
frameworks on a dynamically shared pool of nodes.
• Manage a cluster of Linux containers as a single system.
• There is a Kubernetes Framework for Mesos by
Mesosphere.
Was developed at the University of California, Berkeley
year 2009 developed by Benjamin Hindman, Andy
Konwinski, Matel Zaharia as well as professor Ion Stoica
Become top level Apache project from July 2013
Project Started at Google
Recently launched 1.0 production code (2nd half 2015)
Getting more popular day by day along with Docker.
Works with various Frameworks like Marathon, Cronos. You can
write your own. There is a core algorithm(DRF) to share
resources and you can define your own scheduling policy.
Frameworks does Fault Tolerance, Load balancing, Quotas,
Discovery, etc.
Kubernetes establishes robust declarative primitives for
maintaining the desired state requested by the user. Self-
healing mechanisms, such as auto-restarting, re-scheduling,
and replicating containers require active controllers, not just
imperative orchestration.
Introduction
HUAWEI TECHNOLOGIES CO., LTD.
Slide title :32-35pt
Color: R153 G0 B0
Corporate Font :
FrutigerNext LT Medium
Font to be used by customers and
partners :
Arial
Slide text :20-22pt
Bullets level 2-5:
18pt
Color:Black
Corporate Font :
FrutigerNext LT Medium
Font to be used by customers and
partners :
Arial
Top right corner
for field-mark,
customer or
partner logotypes.
----------------
The following
nine groups of
colors are an
example of how
our design colors
can be used,
please take note
that you should
only use one
design color
group per slide.
For specific
usage details,
refer to the
“Typesetting
Standard”.
17
Mesos Kubernetes
The Architecture
HUAWEI TECHNOLOGIES CO., LTD.
Slide title :32-35pt
Color: R153 G0 B0
Corporate Font :
FrutigerNext LT Medium
Font to be used by customers and
partners :
Arial
Slide text :20-22pt
Bullets level 2-5:
18pt
Color:Black
Corporate Font :
FrutigerNext LT Medium
Font to be used by customers and
partners :
Arial
Top right corner
for field-mark,
customer or
partner logotypes.
----------------
The following
nine groups of
colors are an
example of how
our design colors
can be used,
please take note
that you should
only use one
design color
group per slide.
For specific
usage details,
refer to the
“Typesetting
Standard”.
18
Solution 1 : Kubernetes (Cluster manager)
We could provide redis instance in docker containers
Kubernetes has a Mesos framework which can co-exist with
other frameworks/workloads in the same Mesos cluster
The concept of ‘POD’ can be used to club multiple containers
together. Kubernetes ensures that these containers are started
on the same slave
Kubernetes automatically provides fault- tolerance
Why kubernetes?
HUAWEI TECHNOLOGIES CO., LTD.
Slide title :32-35pt
Color: R153 G0 B0
Corporate Font :
FrutigerNext LT Medium
Font to be used by customers and
partners :
Arial
Slide text :20-22pt
Bullets level 2-5:
18pt
Color:Black
Corporate Font :
FrutigerNext LT Medium
Font to be used by customers and
partners :
Arial
Top right corner
for field-mark,
customer or
partner logotypes.
----------------
The following
nine groups of
colors are an
example of how
our design colors
can be used,
please take note
that you should
only use one
design color
group per slide.
For specific
usage details,
refer to the
“Typesetting
Standard”.
19
Mesos
Master
Kubernetes
Framework for
Mesos
Cloud service
broker Kube- Executor kube- Executor Kube- Executor Kube- Executor
Redis
Request
from cloud Each Service instance will be a POD, provisioned
for given type
Solution 1 : Kubernetes as fw of mesos
HUAWEI TECHNOLOGIES CO., LTD.
Slide title :32-35pt
Color: R153 G0 B0
Corporate Font :
FrutigerNext LT Medium
Font to be used by customers and
partners :
Arial
Slide text :20-22pt
Bullets level 2-5:
18pt
Color:Black
Corporate Font :
FrutigerNext LT Medium
Font to be used by customers and
partners :
Arial
Top right corner
for field-mark,
customer or
partner logotypes.
----------------
The following
nine groups of
colors are an
example of how
our design colors
can be used,
please take note
that you should
only use one
design color
group per slide.
For specific
usage details,
refer to the
“Typesetting
Standard”.
20 Page 20
Cannot add additional containers in a already running POD. Very
necessary for Redis 3.0 horizontal scalability. Below discussion in
google groups explains the implementation difficulties in
Kubernetes https://groups.google.com/forum/#!topic/google-
containers/_Gz9dH2hqg4
However it’s a popular demand from the kubernetes community
Limited Flexibility by using Kubernetes Framework
Solution 1 : Kubernetes (Cluster manager) Contd..
Limitations..
HUAWEI TECHNOLOGIES CO., LTD.
Slide title :32-35pt
Color: R153 G0 B0
Corporate Font :
FrutigerNext LT Medium
Font to be used by customers and
partners :
Arial
Slide text :20-22pt
Bullets level 2-5:
18pt
Color:Black
Corporate Font :
FrutigerNext LT Medium
Font to be used by customers and
partners :
Arial
Top right corner
for field-mark,
customer or
partner logotypes.
----------------
The following
nine groups of
colors are an
example of how
our design colors
can be used,
please take note
that you should
only use one
design color
group per slide.
For specific
usage details,
refer to the
“Typesetting
Standard”.
21 Page 21
Implement a simplistic Mesos framework for Long Running
Process
Use “mesos-go” an opensource framework library written in
golang for development
Check feasibility for vertical scaling of redis-instances
Collect statistics against each redis instance
Enable the fw to ensure instances can communicate to create a
cluster (user 3.0)
Solution 2: Mesos Framework dealing directly with Redis
Instances
Scope
HUAWEI TECHNOLOGIES CO., LTD.
Slide title :32-35pt
Color: R153 G0 B0
Corporate Font :
FrutigerNext LT Medium
Font to be used by customers and
partners :
Arial
Slide text :20-22pt
Bullets level 2-5:
18pt
Color:Black
Corporate Font :
FrutigerNext LT Medium
Font to be used by customers and
partners :
Arial
Top right corner
for field-mark,
customer or
partner logotypes.
----------------
The following
nine groups of
colors are an
example of how
our design colors
can be used,
please take note
that you should
only use one
design color
group per slide.
For specific
usage details,
refer to the
“Typesetting
Standard”.
22
New Redis Mesos Framework
Mesos
Master
Redis
Framework for
Mesos
Cloud service
broker Redis Executor Redis Executor Redis Executor Redis Executor
Redis
Request
from cloud
HUAWEI TECHNOLOGIES CO., LTD.
Slide title :32-35pt
Color: R153 G0 B0
Corporate Font :
FrutigerNext LT Medium
Font to be used by customers and
partners :
Arial
Slide text :20-22pt
Bullets level 2-5:
18pt
Color:Black
Corporate Font :
FrutigerNext LT Medium
Font to be used by customers and
partners :
Arial
Top right corner
for field-mark,
customer or
partner logotypes.
----------------
The following
nine groups of
colors are an
example of how
our design colors
can be used,
please take note
that you should
only use one
design color
group per slide.
For specific
usage details,
refer to the
“Typesetting
Standard”.
23
Redis Executor Redis Executor
POC : Design of Simple Redis Framework
Redis Scheduler
http server Monitor
Master
Select offer
&
Submit tasks
Collect Stats
Redis Executor
Monitor
pid=123
http server
UI
PID = 123
PID = 124
PID = 125
Monitor
pid=124
Monitor
pid=125
Launch Task
Mesos
Master
Mesos Slaves
HUAWEI TECHNOLOGIES CO., LTD.
Slide title :32-35pt
Color: R153 G0 B0
Corporate Font :
FrutigerNext LT Medium
Font to be used by customers and
partners :
Arial
Slide text :20-22pt
Bullets level 2-5:
18pt
Color:Black
Corporate Font :
FrutigerNext LT Medium
Font to be used by customers and
partners :
Arial
Top right corner
for field-mark,
customer or
partner logotypes.
----------------
The following
nine groups of
colors are an
example of how
our design colors
can be used,
please take note
that you should
only use one
design color
group per slide.
For specific
usage details,
refer to the
“Typesetting
Standard”.
24 Page 24
To build a Mesos framework we have to develop two components.
Scheduler – Talk to the mesos master and submit tasks
Executor – Execute your tasks at the slave
By Default Mesos considers each work load is a task not LRP (Long Running Process)
What is a Mesos framework (a developers perspective):
Usual Logic of Scheduler Usual Logic of Executor
Loop (Wait for offer from Mesos Master) if (offer.Resource > Task.Resource) then Select Offer else Reject Offer Repeat
Loop (wait for tasks to Launch ) Update task.State = STARTED execute task Update task.State = Finished / Error Repeat
Solution 2: Mesos Framework dealing directly with Redis
Instances
HUAWEI TECHNOLOGIES CO., LTD.
Slide title :32-35pt
Color: R153 G0 B0
Corporate Font :
FrutigerNext LT Medium
Font to be used by customers and
partners :
Arial
Slide text :20-22pt
Bullets level 2-5:
18pt
Color:Black
Corporate Font :
FrutigerNext LT Medium
Font to be used by customers and
partners :
Arial
Top right corner
for field-mark,
customer or
partner logotypes.
----------------
The following
nine groups of
colors are an
example of how
our design colors
can be used,
please take note
that you should
only use one
design color
group per slide.
For specific
usage details,
refer to the
“Typesetting
Standard”.
25 Page 25
Its easier to write a Mesos framework
We have better controller & flexibility over the running redis
instances
We can collect better statistics on the running tasks
We can plan migration of running tasks among different slaves
We can group bunch of running processes and allocate them in a
single slave (like kubernetes POD)
We would be able to offer Redis 3.0 and scale Horizontally
We could use any container technology of our choice like Docker
or RKT or RunC in bare-metal servers for additional instance
security
WHY and what more can we do?
Solution 2: Mesos Framework dealing directly with Redis
Instances
HUAWEI TECHNOLOGIES CO., LTD.
Slide title :32-35pt
Color: R153 G0 B0
Corporate Font :
FrutigerNext LT Medium
Font to be used by customers and
partners :
Arial
Slide text :20-22pt
Bullets level 2-5:
18pt
Color:Black
Corporate Font :
FrutigerNext LT Medium
Font to be used by customers and
partners :
Arial
Top right corner
for field-mark,
customer or
partner logotypes.
----------------
The following
nine groups of
colors are an
example of how
our design colors
can be used,
please take note
that you should
only use one
design color
group per slide.
For specific
usage details,
refer to the
“Typesetting
Standard”.
26
DEMO
HUAWEI TECHNOLOGIES CO., LTD.
Slide title :32-35pt
Color: R153 G0 B0
Corporate Font :
FrutigerNext LT Medium
Font to be used by customers and
partners :
Arial
Slide text :20-22pt
Bullets level 2-5:
18pt
Color:Black
Corporate Font :
FrutigerNext LT Medium
Font to be used by customers and
partners :
Arial
Top right corner
for field-mark,
customer or
partner logotypes.
----------------
The following
nine groups of
colors are an
example of how
our design colors
can be used,
please take note
that you should
only use one
design color
group per slide.
For specific
usage details,
refer to the
“Typesetting
Standard”.
27
Q&A
HUAWEI TECHNOLOGIES CO., LTD.
Slide title :32-35pt
Color: R153 G0 B0
Corporate Font :
FrutigerNext LT Medium
Font to be used by customers and
partners :
Arial
Slide text :20-22pt
Bullets level 2-5:
18pt
Color:Black
Corporate Font :
FrutigerNext LT Medium
Font to be used by customers and
partners :
Arial
Top right corner
for field-mark,
customer or
partner logotypes.
----------------
The following
nine groups of
colors are an
example of how
our design colors
can be used,
please take note
that you should
only use one
design color
group per slide.
For specific
usage details,
refer to the
“Typesetting
Standard”.
28
Astro Project – Spark SQL on Hbase Astro is fully distributed SQL on Hbase solution based on Spark Ecosystem
100% Open source Any contributions are welcome.
HBase
Spark
Astro
Spark Application
SQL ML Graph Stream Community
Package
•Astro query processing is fully distributed. •CRUD(Create, Read, Update and Delete) operations support •Fully utilizing Spark SQL query optimizer. •Advanced optimizations to push down computations close to storage. •Co-Location along with Hbase.
As Astro plugs into spark, Astro query results can be seamlessly used for further in-depth analysis using Spark SQL DataFrame, MLLib, GraphX etc…
Address: https://github.com/HuaweiBigData/astro
Compared to existing SQL on Hbase solution the major Advantages are: