bitworks cloudstack ui - cseuug 08 august 2017

CloudStack-UI

Ilya Zolotukhin, Product Team Leader, Bitworksmailto: zolotukhin_ii@bitworks.softwarehttps://bitworks.software/

CSEUUG Meetup, August 17, 2017

Apache CloudStack user self-service web interface for IaaS and application delivery services

Agenda

1. About the speaker2. Origins - how and why we decided to

develop the product3. Current problems of the native ACS UI as

our users and helpdesks see them4. Comparison - native vs CloudStack-UI5. UX Goals: Simplicity, Early Errors

Detection, Intuitive Navigation, Less Steps6. Domains7. Where the project moves8. The role of tags for the product

9. Pluggable infrastructurea. Shell In A Box pluginb. Pulse pluginc. Vault plugin

10. Application delivery (and CICD) development

11. Development process - work on features, bugs, release cycles

12. Development roadmap13. Product difficulties and ACS advices14. Team15. About Bitworks16. Thank you slide

2

1. Team Leader at Bitworksa. 3 Years of full-stack development b. 3 Years of front-end development for

AdTech Industry2. Front-End consultant

About the SpeakerWho is it?

3

Why We Develop CloudStack-UI

1. Educational “Incubator” for new teammates2. Our affiliate ISP company uses ACS and has collected broad

experience over 3 years of service delivery:a. We have a Project Sponsorb. We have an understanding of what shortcomings the

current UI has and what features and UX our competitors provide (AWS, DO, etc.)

c. Affiliate company plans to launch new ACS-based cloud3. As a software development company, we understand jQuery is

not a good choice any longer for current web UI and new adopters try to stay away from it (why not just change the current layout?).

4

Current Problems of Native ACSUI From User’s Perspective1. Difficult to learn and remember how to run operations (even for

Basic Zone). E.g. to make a template from a snapshot2. Old-fashioned (competes poorly against AWS, DO, GCP)3. First glance disappointments

a. rookie administrators (who deploy ACS for the first time)b. newbie users who come from AWS, DO, GCP, etc

5

Current Problems of Native ACSUI From HelpDesk Perspective1. Difficult to learn and remember how to run operations (even for

Basic Zone). E.g. to make a template from a snapshot2. Troublesome phone-based consultations (it’s difficult to explain

where to navigate, what input to find, and change)3. Troublesome ticket-based consultations (no way to remember

the state)

6

CloudStack-UI vs Native ACS UI (I)

Feature CS-UI Native

UI Credo 1. Better UX2. User’s needs first3. Simplify navigation

1. Reflect API fully2. ER-based navigation design3. Administrator needs first

Purpose Cloud Users Cloud AdministratorsCloud Users

Care Helpful, protective. Advanced client-side logic. Tries to avoid error situations and explains errors if possible.

Careless (let’s try it and see what happens). Almost server-side error detection.

Business logic

“Thinks” about cloud from the business logic point of view (e.g. default service offerings)

Just reflects technology to UI

7

CloudStack-UI vs Native ACS UI (II)

Feature CS-UI Native

Look & Feel Modern Material (Google) design, responsive (PC, Tablets), fits great for “tap-ish” devices.

Custom design, sometimes old-fashioned, designed for PC.

Redesign proposal (ACS 4.3 - 2013, not implemented):https://cwiki.apache.org/confluence/display/CLOUDSTACK/Update+UI+visual+appearance

Navigation State reflection to URL (History API) No reflection (makes life of HelpDesk difficult)

Technology 1. Angular v4 https://angular.io/2. TypeScript3. Material Design (Look & Feel)

https://material.angular.io/

1. No framework2. JavaScript3. jQuery

Deployment Dockerized SPA Embedded

8

CloudStack-UI vs Native ACS UI (III)

Feature CS-UI Native

Future Supportability

Uses modern Angular framework which is expected to be used for the next 6-7 years (angular.js - 2009-2016)

Custom jQuery-based framework

Maturity Rookie Mature

Adoption None High

License Apache License v2 Apache License v2

9

Native UI VM Creation

10

Every State is Represented with a Permalink

11

https://cs.ui/instances?zones=031a55bb-5d6b-4336-ab93-d5dead28a887&groups=www&states=Running&groupings=zones&groupings=groups

UX Goals: Simplicity, Early Errors Detection, Intuitive Navigation, Less Steps

12

UX Goals: Simplicity, Early Errors Detection, Intuitive Navigation, Less Steps

13

UX Goals: Simplicity, Early Errors Detection, Intuitive Navigation, Less Steps

14

UX Goals: Simplicity, Early Errors Detection, Intuitive Navigation, Less Steps

15

UX Goals: Simplicity, Early Errors Detection, Intuitive Navigation, Less Steps

16

UX Goals: Simplicity, Early Errors Detection, Intuitive Navigation, Less Steps

17

UX Goals: Simplicity, Early Errors Detection, Intuitive Navigation, Less Steps

18

UX Goals: Simplicity, Early Errors Detection, Intuitive Navigation, Less Steps

19

UX Goals: Simplicity, Early Errors Detection, Intuitive Navigation, Less Steps

20

UX Goals: Simplicity, Early Errors Detection, Intuitive Navigation, Less Steps

21

UX Goals: Simplicity, Early Errors Detection, Intuitive Navigation, Less Steps

22

UX Goals: Simplicity, Early Errors Detection, Intuitive Navigation, Less Steps

23

UX Goals: Simplicity, Early Errors Detection, Intuitive Navigation, Less Steps

24

Security Group Review: Single SG per VM - Useful For Average Cloud Users

25

Security Group Review: Single SG per VM - Useful For Average Cloud Users

26

Security Group Review: Single SG per VM - Useful For Average Cloud Users

27

Domain Improvements

28

3 ways to specify a default domain:

1. URL Query params2. Config (config.json)

(deployment)3. Type it on the Login Page

The Role of ACS Tags in CloudStack-UI

29

Tags are widely used

1. user - preferences, localization, theming, plugins2. vm - helpers, colors, description, grouping, plugins3. volume - marking for removal4. SG - distinguish between SG templates and actual SGs,

marking for removal5. template - helpers

Suffer from:1. tag values are too short (255 chars max)2. no account tags

The Role of ACS Tags in CloudStack-UIWhere do we use tags?

1. VM a. Instance groupb. UI-specific settings. i.e. color, description

2. Snapshot description3. Template download URL4. User

a. Languageb. Time formatc. Interface color

5. Volume description6. Plugins

a. WebShellb. Pulse

30

The Role of ACS Tags in CloudStack-UI[cloudstack-ui-cleaner]

31

Current cleaner implementation is naive. Need to reimplement with event subscription.

The Role of ACS Tags in CloudStack-UI:[cloudstack-ui-cleaner]

32

Lack of delete chain operations for VM:1. for SGs2. Snapshots3. Disks4. Anything else?

markResourceAutoExpunge?resource=sg&id=<sgid>&vm=<vmid>

Suggestion:

The Role of ACS Tags in CloudStack-UI. An Example of Tags Usage for VMs

33

Suggestion:Add option to API to copy tags from template to VM

Extensions Ecosystem: ACS Umbrella

34

Extension Plugin: Pulse

35

Extension Plugin: Pulse

36

Pulse:CPU/RAM

37

38

Disk: read/write/errors Network: read/write/errors/drops

Extension Plugin: Shell In A Box

39

Shell In A Box: Benefits

40

● High Interactivity● Copy & Paste● Automatic session termination on inactivity

timeout● No out-of-band access

TODO:

● Vault plugin integration (passwordless SSH)● Control-centre (SSH-sessions multiplexor)

Shell In A Box: Use of Tags

41

Shell In A Box: Console View

42

Extension (Vault) = Philosophy

43

Vault: Purpose

44

● CMDB (Configuration Management Database) implementation● Tag service replacement for native ACS (because native ACS

tags lack functionality)● Two-way secure communication for VM <-> CMDB (via Vault’s

tokens, scopes, ACLs)● One-time tokens for Webshell (private SSH keys storage,

passwordless SSH access)● The basis for Applications plugin

Vault: Technology

45

Thoughts on Project Direction

46

Project Direction - Traditional IaaS vs IaC (CCA). More and More Users Don’t Need VMs - They Need:

47

Applications

1. deployed (templates)2. configured (templates)3. upgraded (seamlessly)4. monitored (visually)5. troubleshot (visually)6. backed up and restored (automatically)7. optimized (performance / price optimization)

Decrease costs and simplify all mentioned above (DIY approach)

OK. Sure, traditional users are still here (corporate users)

Development Process

1. Agile + Scrum + ZenHub2. 2-week sprints3. peer-to-peer code review

48

Development Process

49

Development Process

50

Development Process

1. Public: a. Docker hub releases ( https://hub.docker.com/r/bwsw/cloudstack-ui )

■ docker pull bwsw/cloudstack-ui:latest■ docker pull bwsw/cloudstack-ui:[version]

b. Travis CI2. Private: Jenkins

a. Docker per-branch testing and deployment■ http://sandbox-url/[branch-name]

51

Difficulties: ACS Advices1. Missing fields in responses

a. RestoreVM - missing jobinstancetype and jobinstanceid2. Lowercase

a. <jobstatus>b. <jobprocstatus>c. <jobresultcode>d. <jobresulttype>

3. Some responses contain <success>true</success> , while others contain entity4. No Error codes5. Strange naming

a. UserVm instead of VirtualMachine (tags)b. getVMPassword , cleanVMReservationsCleanups

52

Difficulties: ACS Advices

1. Schedule format in createSnapshot is confusing2. Results don't contain all necessary fields:3. templatefilter is required in listTemplates, while isofilter is

optional in listIsos4. Tag operations are asynchronous (require polling)5. Arrays in "list" responses are named inconsistently. For example,

listVirtualMachines returns <virtualmachine>, while listAsyncJobs returns <asyncjobs>

6. Some of trues in { success: true } are strings “true” and some are booleans

53

Core Development Team

1. Ivan Kudryavtsev - Product Owner2. Elena Ershova - Project Manager3. Ilya Zolotukhin - me (technical consulting, review, PM)4. Developers:

a. Andrey Bentsb. Vladimir Shakhov

(3 more developers and a QA-engineer help when available)

54

About Bitworks Software

55

● Telecommunications (ISP, XaaS, HPC)

● AdTech● FinTech (NASDAQ

technologies)● Pharmaceutical (medical

data processing, machine learning models)

● IoT

● Backend (Scala, Java, Python, Go)

● Frontend (Angular.js, Angular, React)

● QA (automation, manual)● ML/ETL (Apache Spark,

TensorFlow, etc.)● Project Management

(Scrum)

About Bitworks Software

56

WWW:https://bitworks.software/en

Corporate Presentation: https://goo.gl/uYCzGu

Please, ask for a free USB flash stick with the presentation if you are interested (we have some)

E-mail: info@bitworks.software

Many Thanks to Organizers

57

The EndQuestions?

58