vdi storage and storage virtualization
TRANSCRIPT
![Page 1: VDI storage and storage virtualization](https://reader034.vdocuments.mx/reader034/viewer/2022050921/555c41e1d8b42a2c068b4e02/html5/thumbnails/1.jpg)
Storage Virtualization and VDI Storage
Storage Virtualization and VDI Storage
Sisimon Soman
![Page 2: VDI storage and storage virtualization](https://reader034.vdocuments.mx/reader034/viewer/2022050921/555c41e1d8b42a2c068b4e02/html5/thumbnails/2.jpg)
Agenda
• Datacenter Storage in General
• DAS, NAS and SAN
• Storage Virtualization
• VDI Storage– Issues with VDI Storage– Solutions for Performance issues
![Page 3: VDI storage and storage virtualization](https://reader034.vdocuments.mx/reader034/viewer/2022050921/555c41e1d8b42a2c068b4e02/html5/thumbnails/3.jpg)
Study by IDC
• In 2011 roughly 1.8 trillion GBs of data created.
• Total data doubled in last two years.
• The prediction is total data will grow by 50x in next decade.
• Too much of unstructured data.
• Disks are still in stone-age with motor, spindle and head.
Numbers collected from EMC.com which is available for public
![Page 4: VDI storage and storage virtualization](https://reader034.vdocuments.mx/reader034/viewer/2022050921/555c41e1d8b42a2c068b4e02/html5/thumbnails/4.jpg)
Types of Storage
• Classify into three according to the access mechanism between server and storage.– Direct Attached Storage (DAS)– Network Attached Storage (NAS)– Storage Area Network (SAN)
![Page 5: VDI storage and storage virtualization](https://reader034.vdocuments.mx/reader034/viewer/2022050921/555c41e1d8b42a2c068b4e02/html5/thumbnails/5.jpg)
Direct Attached Storage (DAS)
• Disk(s) directly connected to the machine.• Most simple/common storage.• Example: Our Laptops, Desk tops etc
SCSI
![Page 6: VDI storage and storage virtualization](https://reader034.vdocuments.mx/reader034/viewer/2022050921/555c41e1d8b42a2c068b4e02/html5/thumbnails/6.jpg)
Network Attached Storage
• Uses CIFS/NFS protocol to access files.• Similar to a remote shared folder.• Client side redirector forward the file requests to
NAS box.• Example: EMC Celerra, NetApp FAS
LAN LAN
Servers
NAS box
![Page 7: VDI storage and storage virtualization](https://reader034.vdocuments.mx/reader034/viewer/2022050921/555c41e1d8b42a2c068b4e02/html5/thumbnails/7.jpg)
NAS - Module diagramApplication
File System Redirector
CIFS Protocol Layer
TCP/IP Stack
NIC Driver
User Space
Kernel Space
LAN LAN
CIFS Protocol Layer
TCP/IP Stack
NIC Driver
File System
Volume Manager
Disk Driver
NAS Box
Server
![Page 8: VDI storage and storage virtualization](https://reader034.vdocuments.mx/reader034/viewer/2022050921/555c41e1d8b42a2c068b4e02/html5/thumbnails/8.jpg)
What is a SAN
Picture taken from wikipedia
EMC Symmetrix DMX 2000 EMC Symmetrix DMX 1000
![Page 9: VDI storage and storage virtualization](https://reader034.vdocuments.mx/reader034/viewer/2022050921/555c41e1d8b42a2c068b4e02/html5/thumbnails/9.jpg)
SAN Continued..
• Example : EMC Symmetrix, CLARiiON
Fibre Channel Switch
Servers
SAN
![Page 10: VDI storage and storage virtualization](https://reader034.vdocuments.mx/reader034/viewer/2022050921/555c41e1d8b42a2c068b4e02/html5/thumbnails/10.jpg)
SAN Architecture
Fibre Channel Switch
Controllers/Storage Processors (It can be ACTIVE-ACTIVE or ACTIVE-PASSIVE)
Disks
GBs of memory in each controller
![Page 11: VDI storage and storage virtualization](https://reader034.vdocuments.mx/reader034/viewer/2022050921/555c41e1d8b42a2c068b4e02/html5/thumbnails/11.jpg)
Difference between SAN and NAS
• In NAS ‘file streams’ are transferred thru wire.
• In SAN ‘disk blocks’ are read from storage.• In SAN Fibre Channel is the common
communication mechanism. (SAN support SCSI over TCP/IP also, which is called iSCSI)
• In NAS file streams transfer over TCP/IP stack.
![Page 12: VDI storage and storage virtualization](https://reader034.vdocuments.mx/reader034/viewer/2022050921/555c41e1d8b42a2c068b4e02/html5/thumbnails/12.jpg)
Common features of SAN
• Backup
• Replication
• Snapshot
• Features specific to SAN like SRDF for Symmetrix
SAN Specific data collected from wikipedia
![Page 13: VDI storage and storage virtualization](https://reader034.vdocuments.mx/reader034/viewer/2022050921/555c41e1d8b42a2c068b4e02/html5/thumbnails/13.jpg)
Storage Virtualization - Advantages
• Hide the internal complexity of storage system.
• Better Disk block usage – Study shows only 30-40% of disk space is used effectively.
• Better Performance.
• Scalability.
![Page 14: VDI storage and storage virtualization](https://reader034.vdocuments.mx/reader034/viewer/2022050921/555c41e1d8b42a2c068b4e02/html5/thumbnails/14.jpg)
Taxonomy of Storage Virtualization
• Virtualization at Host Operating system storage stack.
• Switch/Appliance based Virtualization.
• Virtualization at external storage array (SAN).
![Page 15: VDI storage and storage virtualization](https://reader034.vdocuments.mx/reader034/viewer/2022050921/555c41e1d8b42a2c068b4e02/html5/thumbnails/15.jpg)
Storage Stack
File System
Volume Manager
Disk Class Driver
Hardware Driver
ApplicationUser Space
Kernel Space
Host Operating system Storage stack
Fibre channel adapter
Fibre channel Switch
SAN
Fibre Channel cable
![Page 16: VDI storage and storage virtualization](https://reader034.vdocuments.mx/reader034/viewer/2022050921/555c41e1d8b42a2c068b4e02/html5/thumbnails/16.jpg)
Virtualization at Operating Storage Stack
File System
Volume Manager
Disk Class Driver
Hardware Driver
• A typical Windows Storage stack
IO Request flow thru each layer
![Page 17: VDI storage and storage virtualization](https://reader034.vdocuments.mx/reader034/viewer/2022050921/555c41e1d8b42a2c068b4e02/html5/thumbnails/17.jpg)
Virtualization at File System Layer
File System
Volume Manager
Disk Class Driver
Hardware Driver
File1.doc
Disk blocks
Virtual Cluster Number (VCN)
Logical Cluster Number (LCN)
• NTFS expose IOCTL FSCTL_GET_RETRIEVAL_POINTERS, so that any app can query VCN-LCN mapping. Usually the disk defragment app uses this IOCTL.
![Page 18: VDI storage and storage virtualization](https://reader034.vdocuments.mx/reader034/viewer/2022050921/555c41e1d8b42a2c068b4e02/html5/thumbnails/18.jpg)
HSM File System (Tiered Storage)
RAM ($50/MB)
SAN ($.5/MB)
TAPE($.05/MB)
Only 20-30% of actively used data
• Ex: EMC DiskXtender
![Page 19: VDI storage and storage virtualization](https://reader034.vdocuments.mx/reader034/viewer/2022050921/555c41e1d8b42a2c068b4e02/html5/thumbnails/19.jpg)
Virtualization at Volume Manager - RAID
• RAID 5 - Also know as Striped Volume with parity.• Fault tolerance is achieved by reserving an equivalent of
one disk for keeping parity information.• Rotate the parity stripe to all disks - Avoid the possibility
of parity disk become busy all time.
RAID
![Page 20: VDI storage and storage virtualization](https://reader034.vdocuments.mx/reader034/viewer/2022050921/555c41e1d8b42a2c068b4e02/html5/thumbnails/20.jpg)
Virtualization at Disk Controller
• Disk Controller convert Logical Block Address to Cylinder Head Sector address.
• Disk controller take care of damaged sectors also.
LBA 1
LBA 2
LBA 3
LBA 4
LBA 5
LBA n
![Page 21: VDI storage and storage virtualization](https://reader034.vdocuments.mx/reader034/viewer/2022050921/555c41e1d8b42a2c068b4e02/html5/thumbnails/21.jpg)
Putting it all together
Windows Storage Stack
![Page 22: VDI storage and storage virtualization](https://reader034.vdocuments.mx/reader034/viewer/2022050921/555c41e1d8b42a2c068b4e02/html5/thumbnails/22.jpg)
Switch based Virtualization
• Combine LUNs from one or more Arrays to a single virtual LUN and pass to host OS.
• Take one big LUN from one Array, divide it and give to different host OS.
• Security – One host can see certain LUNs only.• Vendors are adding more intelligence in switch
level like advanced volume management, caching, QoS functions…
• EMC Invista, IBM SAN Volume Controller
![Page 23: VDI storage and storage virtualization](https://reader034.vdocuments.mx/reader034/viewer/2022050921/555c41e1d8b42a2c068b4e02/html5/thumbnails/23.jpg)
Switch level Virtualization
IBM EMC HP HDX
Intelligent switch – Virtualize LUNs from different array
![Page 24: VDI storage and storage virtualization](https://reader034.vdocuments.mx/reader034/viewer/2022050921/555c41e1d8b42a2c068b4e02/html5/thumbnails/24.jpg)
Virtualization at SAN
Virtual Disk
Disk in the storage array
![Page 25: VDI storage and storage virtualization](https://reader034.vdocuments.mx/reader034/viewer/2022050921/555c41e1d8b42a2c068b4e02/html5/thumbnails/25.jpg)
VDI Storage
![Page 26: VDI storage and storage virtualization](https://reader034.vdocuments.mx/reader034/viewer/2022050921/555c41e1d8b42a2c068b4e02/html5/thumbnails/26.jpg)
Problems with VDI Storage
• Boot/Login Storm• App Storm• Virus scanning.• Many PoC fails or end up with more cost
because of storage array cost.• Some queries in XD/VDI-in-a box forum
– Bootup time of approx ~170 min.– Slow logon.– Do I need to put dedicated LUN to each server.
![Page 27: VDI storage and storage virtualization](https://reader034.vdocuments.mx/reader034/viewer/2022050921/555c41e1d8b42a2c068b4e02/html5/thumbnails/27.jpg)
Windows partition alignment issue
• Data stored in disk as blocks. Block size vary, usually 64K (multiple of OS page size).
• Windows XP/2003 write signature starting of partition and actual partition start at 63 sectors to make it aligned with disk cylinder boundaries.
• This can result in extra IOs.• The partition should aligned with SSD
cache/storage block.
![Page 28: VDI storage and storage virtualization](https://reader034.vdocuments.mx/reader034/viewer/2022050921/555c41e1d8b42a2c068b4e02/html5/thumbnails/28.jpg)
Windows 2003 partition
Block 1 Block 2 Block 3 Block 4 Block 5 Block 6
NTFS Volume 1Signature
Partition starting at less SSD friendly location
![Page 29: VDI storage and storage virtualization](https://reader034.vdocuments.mx/reader034/viewer/2022050921/555c41e1d8b42a2c068b4e02/html5/thumbnails/29.jpg)
IO Blender problem
• Sequential and Random IO.
• OS try to make the IO sequential – Windows Cache Mgr and SCSI/Storport driver, Linux Buffer cache and IO Scheduler.
• Hypervisor screw-up this optimization.
• OS think data from block storage, hypervisor convert it to VHD file (.vmdk)
![Page 30: VDI storage and storage virtualization](https://reader034.vdocuments.mx/reader034/viewer/2022050921/555c41e1d8b42a2c068b4e02/html5/thumbnails/30.jpg)
Read/Write IOPs
• Write IOPs are costly – Cache flushing, RAID cost..
• Windows Paging IO – Paging IOs are sensitive and slowing down it reduce system performance.
• Memory intensive App may increase Paging IO. Number of Paging writes may go higher than write IOs from App.
![Page 31: VDI storage and storage virtualization](https://reader034.vdocuments.mx/reader034/viewer/2022050921/555c41e1d8b42a2c068b4e02/html5/thumbnails/31.jpg)
What is new in VDI Storage
• More than a dozen Storage Startups
• Driver at guest OS stack to profile IOs.
• Module at Hypervisor storage stack which do actual IO scheduling.
• Merge random IOs, Dedupe and compression(make ssd cache effective).
• SSD cache
• Algorithm to analyze App IOPs(profiling) and scheduling is proprietary to vendors.
![Page 32: VDI storage and storage virtualization](https://reader034.vdocuments.mx/reader034/viewer/2022050921/555c41e1d8b42a2c068b4e02/html5/thumbnails/32.jpg)
What is new in VDI Storage
IO Profiler
OS 1
IO Profiler
OS 2
IO Profiler
OS 3
IO Scheduler
Hypervisor
SSD Cache/Storage
Hardware
![Page 33: VDI storage and storage virtualization](https://reader034.vdocuments.mx/reader034/viewer/2022050921/555c41e1d8b42a2c068b4e02/html5/thumbnails/33.jpg)
Questions ?Questions ?