openstack 簡介
DESCRIPTION
這個投影片有三個主軸: 第一個主軸是簡單的介紹各個元件,與如何快速試用 第二個主軸是介紹 openstack 架構與元件溝通方式 第二個主軸是介紹追蹤 openstack source code 的心得TRANSCRIPT
Openstack 簡介果凍
簡介
● 任職於迎廣科技○ python○ openstack
● http://about.me/ya790206● http://blog.blackwhite.
tw/● https://github.
com/ya790206/call_seq
Openstack
● Build your own cloud.● RESTful api
Openstack
iScsi libvirt LVM Open vSwitch
other
trystack
● try openstack● use your facebook id to login.● http://trystack.org/
devstack
● build your development environment.● mini config:
ADMIN_PASSWORD=secreteDATABASE_PASSWORD=$ADMIN_PASSWORDRABBIT_PASSWORD=$ADMIN_PASSWORDSERVICE_PASSWORD=$ADMIN_PASSWORDSERVICE_TOKEN=a682f596-76f3-11e3-b3b2-e716f9080d50
SCREEN_LOGDIR=$DEST/logs/screen
Component
● keystone -- auth and service catalog ● nova -- VM● glance -- image service● cinder -- block storage service● swift -- object storage service● neutron -- network
glance service:● privide image● support backend:
○ s3○ swift○ file system
cinder service:● for instance
swift: ● object storage
other interesting project
● Trove -- database as a service● Sahara -- provision a Hadoop cluster on top
of OpenStack● taskflow● oslo messaging
Common Requirement
SQLAlchemyoslo.configPastePasteDeployiso8601greenleteventlet
RoutesWebObsix
Threading Model
● green thread model(implemented through using the Python eventlet and greenlet libraries.
Python VM
Openstack thread model
VM
M thread
M kernel thread
M thread M thread
1 kernel thread N kernel
threadkernel layer
application layer
XXX-api
YYY-api
ZZZ-api
python- XXXlib
python- YYYlib
python- ZZZlib
horizondashboard
REST api
REST api
REST api
REST apifunction call
function call
function call
REST api
XXX-aaa
XXX-bbb
amqp
rest a
pi
amqp
XXX-ccc
other
api.py: wsgi server(API or Controller)rpcapi.py < = > manager.py
api.pyrequest for RESTful api
python call
rpcapi.py
oslo.messagi
ng
manager.py
nova architecture
有什麼問題,中間加一層就對了
● Mediates interactions between nova-compute and the database.
● Security○ nova-compute never access database directly.
● Ease of Upgrade● Performance
○ Queries to the MySQL database will block the main thread of a service.
○ scales horizontally
nova-conductor
Component Communications
● REST api:○ nova-api -> keystone-api○ glance-api -> glance-registry
● oslo.messaging(AMQP)○ nova-api -> nova-compute○ nova-api -> cinder-volume
REST api
glance/api/v1/images.pyglance/registry/client/v1/client.py
glance/registry/api/v1/images.py
rest api
oslo.messaging
● support multi-endpoints.● RPC Client <-> Server:
○ call○ cast
● Notifier <-> Notification Listener○ ceilometer
example code:https://github.com/kgiusti/oslo-messaging-clients
openstack.taskflow
● workflow library.● make task execution easy, consistent, and
reliable.● flows can be stopped, resumed, and safely
reverted.
openstack.taskflow
● Tasks:○ retry○ revert
● Flows○ revert○ pattern:
■ Linear■ Unordered■ Directed acyclic graph
● Engines:○ serial○ parallel○ worker-based
openstack.taskflow -- use case
task1: create server || rollback by delete servertask2: create+attach volume || rollback by delete volume
class CallJim(task.Task):
def execute(self, jim_number, *args, **kwargs):
print("Calling jim %s." % jim_number)
class CallJoe(task.Task):
def execute(self, joe_number, *args, **kwargs):
print("Calling joe %s." % joe_number)
flow = lf.Flow('simple-linear').add(
CallJim(),
CallJoe()
)
taskflow.engines.run(flow, store=dict(joe_number=444,
jim_number=555))
Example
https://github.com/openstack/taskflow/blob/master/taskflow/examples/graph_flow.py
https://github.com/openstack/taskflow/blob/master/taskflow/examples/reverting_linear.py
Trace Code Tip
● Document● Database schema● Basic data struct● Entry point● Test code● Debug message.
Basic Data Struct
● for example:○ django model for django○ intobject.c for python intergrater.○ glance models for openstack glance.
pyreverse
● UML tool for Python● pyreverse -ASmy -k -o png main.py -p Main
Entry point
1. 由下而上(由樹見林)a. traceback.extract_stack.b. use debugger to show traceback.
2. 由上而下(由林見樹)a. call_seq.
call_seq
● written by me.● pip install git+https://github.
com/ya790206/call_seq● trace you program how to run and generate
a report with json format.
from call_seq import CallSeq
trail = CallSeq() trail.set_trace() # the code you want to trace. trail.unset_trace() trail.dump_to_file('output.json')
"arguments": {}, "code": "test1()", "file_name": "demo.py", "lineno": 14, "name": "test1", "return": "<int>: 2", "return_lineno": 10, "seq": [ { "arguments": {}, "code": "test2()", "file_name": "demo.py", "lineno": 9, "return": "<NoneType>", "return_lineno": 5, "seq": [] } ]
Test Code
● 看看 function test 和 unit testhttps://github.com.openstack.nova.blob.master.nova.tests.test_flavors.py
https://github.com.openstack.cinder.blob.master.cinder.tests.test_volume.py
總結
● openstack: build you own cloud.● communication:
○ REST api○ oslo.messaging
● Trace Code Tip
徵才
● 迎廣科技研發中心● 尋找 openstack, python 的愛好者● 聯絡方式: http://www.blackwhite.
tw/#/contact
Thank you
雜記
● 此張投影片後的投影片內容為演講後所新增內容,對前面的投影片內容做補充
為何要介紹openstack元件?
● 站在巨人的肩上, 才能看的更遠○ 利用現有軟體, 減少沒必要的程式開發○ 利用已經有軟體, 創造更多價值
為何要介紹 Common Requirement?
● 了解大型專案所採用的 library● 如果我們要做 api server,但是不想浪費效能在無意
義的運算上。○ 採用 django, bottle 對 api server 而言,太多餘○ 直接寫 wsgi server 太痛苦○ 可以模仿 openstack 用 paste, route, 和 webob 來
建立 api server。
● 為 nova conductor 鋪路
為何介紹 Threading Model ?
● 有問題,中間加一層就對了
● sql query 會影響 eventlet 效能(我不確定,文件寫的)
為何介紹 nova conductor ?
● 另一個 rpc library 的選擇● 相似專案: zerorpc
為何介紹 oslo.messaging?
● 類似 pipeline,或 sql transaction。● 當工作失敗後,可以 rollback 動作。或是隔幾
秒重試,或是換不同的參數看看。● 每個工作下可以擁有子任務,子任務可以循序
執行,平行執行等。
為何介紹 openstack.taskflow ?
● 當你使用由下往上看(show traceback)的方法時,這是一個很好的切入點。
為何要在 trace code tip 中提到 basic data struct?
● 因為茫茫程式碼海裡,不知道要從何下手
為何要由下往上看?
● 要觀其大略,識大局● 為何我要寫 call_seq:
○ 因為我懶得一個一個開檔案看程式碼
○ 既然程式都知道怎麼執行了,讓他告訴我也不是一件
過份的事
為何要由上往下看?