storage network designs for oltp business continuity
DESCRIPTION
Storage Network Designs for OLTP Business Continuity. Marc Farley President, Building Storage Networks, Inc. Agenda. The Vendor Neutral Approach Overview of OLTP &High Availability I/O Redundancy Methods Storage Network Technologies Storage Networking for HA OLTP. Vendor Neutral Approach. - PowerPoint PPT PresentationTRANSCRIPT
![Page 1: Storage Network Designs for OLTP Business Continuity](https://reader036.vdocuments.mx/reader036/viewer/2022070411/56814716550346895db44d77/html5/thumbnails/1.jpg)
Hosted by
Storage Network Designs for OLTP Business Continuity
Marc FarleyPresident, Building Storage Networks, Inc.
![Page 2: Storage Network Designs for OLTP Business Continuity](https://reader036.vdocuments.mx/reader036/viewer/2022070411/56814716550346895db44d77/html5/thumbnails/2.jpg)
Hosted by
Agenda
The Vendor Neutral Approach
Overview of OLTP &High Availability
I/O Redundancy Methods
Storage Network Technologies
Storage Networking for HA OLTP
![Page 3: Storage Network Designs for OLTP Business Continuity](https://reader036.vdocuments.mx/reader036/viewer/2022070411/56814716550346895db44d77/html5/thumbnails/3.jpg)
Hosted by
Vendor Neutral Approach
Generic terms, not vendor terms
Assumed basic knowledge of SAN, NAS,
RAID
![Page 4: Storage Network Designs for OLTP Business Continuity](https://reader036.vdocuments.mx/reader036/viewer/2022070411/56814716550346895db44d77/html5/thumbnails/4.jpg)
Hosted by
And now, for something completely different…..
![Page 5: Storage Network Designs for OLTP Business Continuity](https://reader036.vdocuments.mx/reader036/viewer/2022070411/56814716550346895db44d77/html5/thumbnails/5.jpg)
Hosted by
OLTP Environments
Mission critical business applications• Business in real-time
Expensive equipment and software
Aggressive performance objectives
Highly skilled IT staff• Hands-on computing operations
![Page 6: Storage Network Designs for OLTP Business Continuity](https://reader036.vdocuments.mx/reader036/viewer/2022070411/56814716550346895db44d77/html5/thumbnails/6.jpg)
Hosted by
OLTP Database SoftwareOracle,
• 8i Oracle Parallel Server (OPS)• 9i Real Application Cluster (RAC)
IBM • DB2 UDB• Informix
MS SQL Server
Sybase, My SQL, others
![Page 7: Storage Network Designs for OLTP Business Continuity](https://reader036.vdocuments.mx/reader036/viewer/2022070411/56814716550346895db44d77/html5/thumbnails/7.jpg)
Hosted by
OLTP OS Platforms
IBM S/390 MVS
Unix Systems
Windows 2000+
HA Linux
![Page 8: Storage Network Designs for OLTP Business Continuity](https://reader036.vdocuments.mx/reader036/viewer/2022070411/56814716550346895db44d77/html5/thumbnails/8.jpg)
Hosted by
OLTP Requirements
99.999% uptime
Non-degrading response time
High transaction rates
Seamless scalability
Cost relief
![Page 9: Storage Network Designs for OLTP Business Continuity](https://reader036.vdocuments.mx/reader036/viewer/2022070411/56814716550346895db44d77/html5/thumbnails/9.jpg)
Hosted by
Database Storage Approaches
Raw parititions• Bypass OS I/O buffering
File system• Facilitates data management
NFS mounted• Offload DB server, NTAP + Oracle
![Page 10: Storage Network Designs for OLTP Business Continuity](https://reader036.vdocuments.mx/reader036/viewer/2022070411/56814716550346895db44d77/html5/thumbnails/10.jpg)
Hosted by
Atomicity – No partial transactions
Consistency – All tables are in a consistent state before and after a completed transaction
Isolation – One transaction cannot contaminate other transactions
Durability – Transactions are complete only when the database updates are written to disk storage
ACID Properties of OLTP
![Page 11: Storage Network Designs for OLTP Business Continuity](https://reader036.vdocuments.mx/reader036/viewer/2022070411/56814716550346895db44d77/html5/thumbnails/11.jpg)
Hosted by
Challenges of OLTP
Major systems integration effort• Intricate tuning and monitoring• Little tolerance for errors
Complex data structures & relationships
Time and sequence-sensitive processes• Must be adhered to for data integrity
Shifting workloads and bottlenecks
![Page 12: Storage Network Designs for OLTP Business Continuity](https://reader036.vdocuments.mx/reader036/viewer/2022070411/56814716550346895db44d77/html5/thumbnails/12.jpg)
Hosted by
OLTP Database Files
Data files• Database data, tablespaces
Redo log files, archive log files• Reconstruct or rollback transactions
Control files• File layout information
![Page 13: Storage Network Designs for OLTP Business Continuity](https://reader036.vdocuments.mx/reader036/viewer/2022070411/56814716550346895db44d77/html5/thumbnails/13.jpg)
Hosted by
OLTP Table Space Storage
Use many spindles to distribute hot spots
RAID 0+1 recommended
File system recommended over raw
partitions• Easier data management
![Page 14: Storage Network Designs for OLTP Business Continuity](https://reader036.vdocuments.mx/reader036/viewer/2022070411/56814716550346895db44d77/html5/thumbnails/14.jpg)
Hosted by
Striping for Performance
RAID Controller (Microsecond performance)
DiskDrive
DiskDrive
DiskDrive
DiskDrive
DiskDrive
DiskDrive
Disk Drives (Millesecond performance)From rotational latency and seek time
![Page 15: Storage Network Designs for OLTP Business Continuity](https://reader036.vdocuments.mx/reader036/viewer/2022070411/56814716550346895db44d77/html5/thumbnails/15.jpg)
Hosted by
My Personal Favorite, RAID 0+1
RAID Controller
DiskDrive
DiskDrive
DiskDrive
DiskDrive
1 2 3 4 5
DiskDrive
DiskDrive
DiskDrive
DiskDrive
DiskDrive
DiskDrive
Mirrored Pairs of Striped Members
![Page 16: Storage Network Designs for OLTP Business Continuity](https://reader036.vdocuments.mx/reader036/viewer/2022070411/56814716550346895db44d77/html5/thumbnails/16.jpg)
Hosted by
OLTP Redo Log StorageRaw partitions recommended
• Sequential high speed writes
Separate mirror pairs per log file group
Capacity for 30 – 60 minutes of data
Goal is to limit disk contention for current and active log files
![Page 17: Storage Network Designs for OLTP Business Continuity](https://reader036.vdocuments.mx/reader036/viewer/2022070411/56814716550346895db44d77/html5/thumbnails/17.jpg)
Hosted by
OLTP Archive Log Storage
File system or NFS mounting is required• NFS mounting is recommended
Mirroring or RAID
Goal is to have easy access in case they
are needed for reconstruction
![Page 18: Storage Network Designs for OLTP Business Continuity](https://reader036.vdocuments.mx/reader036/viewer/2022070411/56814716550346895db44d77/html5/thumbnails/18.jpg)
Hosted by
High Availability
The ability for a system or application to immediately
continue its mission after loss or damage to system
components, systems, facilities and data
![Page 19: Storage Network Designs for OLTP Business Continuity](https://reader036.vdocuments.mx/reader036/viewer/2022070411/56814716550346895db44d77/html5/thumbnails/19.jpg)
Hosted by
Availability Threats
Expected
• Scaling limitations
Processor
Storage capacity
Network
• Consolidations
• Product life cycles
Unexpected
• Failures
• Bugs
• Virus
• Operator errors
• Disasters
![Page 20: Storage Network Designs for OLTP Business Continuity](https://reader036.vdocuments.mx/reader036/viewer/2022070411/56814716550346895db44d77/html5/thumbnails/20.jpg)
Hosted by
HA Engages All Elements
Systems• Application
Network connections• Network services
Storage and I/O subsystems
![Page 21: Storage Network Designs for OLTP Business Continuity](https://reader036.vdocuments.mx/reader036/viewer/2022070411/56814716550346895db44d77/html5/thumbnails/21.jpg)
Hosted by
Scoping the Risks
System Network Storage
Component HBA Cable Disk drive
System Server Switch Subsystem
Pathological Virus attack
on platform
Service provider
outage
Environmental
media loss
Site Server rooms
gutted
All external
communications
Total data loss
![Page 22: Storage Network Designs for OLTP Business Continuity](https://reader036.vdocuments.mx/reader036/viewer/2022070411/56814716550346895db44d77/html5/thumbnails/22.jpg)
Hosted by
Managing the Risks
Local copies of data • Immediate availability
(Remote) Nearby• Immediate availability to several hours
Remote Far away• One to several days availability
![Page 23: Storage Network Designs for OLTP Business Continuity](https://reader036.vdocuments.mx/reader036/viewer/2022070411/56814716550346895db44d77/html5/thumbnails/23.jpg)
Hosted by
Disaster/Availability Radii
Local Remote Nearby
Remote Far Away
![Page 24: Storage Network Designs for OLTP Business Continuity](https://reader036.vdocuments.mx/reader036/viewer/2022070411/56814716550346895db44d77/html5/thumbnails/24.jpg)
Hosted by
Nobody Expects…..
Weird things to happen to them
Disintegration of media
Underground flooding through tunnels
Fires in Telco switching centers
![Page 25: Storage Network Designs for OLTP Business Continuity](https://reader036.vdocuments.mx/reader036/viewer/2022070411/56814716550346895db44d77/html5/thumbnails/25.jpg)
Hosted by
High Availability for OLTP
Duplication of functions• Without degrading performance• Without risking data integrity
Brute force techniques
Automation and efficiency
Cost is always an issue • And high availability DOES cost
![Page 26: Storage Network Designs for OLTP Business Continuity](https://reader036.vdocuments.mx/reader036/viewer/2022070411/56814716550346895db44d77/html5/thumbnails/26.jpg)
Hosted by
Redundancy. Again!
A Long Time Ago in a Job Not So Far Away…………….
Remember Marc, there is only one
concept:REDUNDANCY!
Marc Skyfaller Farley
Marc Skyfaller Farley
Jedi Jim GastJedi Jim Gast
WhateverGot it Jim. Let’s Eat!
You must learn the to be a master of redundancy it if
you are going to be a storage geek.
![Page 27: Storage Network Designs for OLTP Business Continuity](https://reader036.vdocuments.mx/reader036/viewer/2022070411/56814716550346895db44d77/html5/thumbnails/27.jpg)
Hosted by
Eventually, I Learned to Appreciate His Teachings……
•REDUNDANCYNSPoF (No Single Point of Failure)
•REDUNDANCYNSPoF (No Single Point of Failure)
Don’t get the giant spicy Polish for lunch – its too much for the digestion
Don’t get the giant spicy Polish for lunch – its too much for the digestion
![Page 28: Storage Network Designs for OLTP Business Continuity](https://reader036.vdocuments.mx/reader036/viewer/2022070411/56814716550346895db44d77/html5/thumbnails/28.jpg)
Hosted by
OLTP HA Requires Complete Redundancy Protection
Client network
Server systems and components
Application modules
I/O Channels and Networks
Storage subsystems and components
Data
![Page 29: Storage Network Designs for OLTP Business Continuity](https://reader036.vdocuments.mx/reader036/viewer/2022070411/56814716550346895db44d77/html5/thumbnails/29.jpg)
Hosted by
A Quick Look At Clustered Storage
Shared Nothing Shared Everything
Each server controls its own storage address space
Both servers share control of a common storage address space
![Page 30: Storage Network Designs for OLTP Business Continuity](https://reader036.vdocuments.mx/reader036/viewer/2022070411/56814716550346895db44d77/html5/thumbnails/30.jpg)
Hosted by
Examples of OLTP Clusters
Microsoft SQL Server Oracle 9.1 RAC
Data is exchanged between servers
Failoverpaths only
Data is accessed directly from
storage
![Page 31: Storage Network Designs for OLTP Business Continuity](https://reader036.vdocuments.mx/reader036/viewer/2022070411/56814716550346895db44d77/html5/thumbnails/31.jpg)
Hosted by
One more time, with subsystems…
Microsoft SQL Server Oracle 9.1 RAC
All storage is shared by all cluster nodes
Same subsystem but different address spaces
![Page 32: Storage Network Designs for OLTP Business Continuity](https://reader036.vdocuments.mx/reader036/viewer/2022070411/56814716550346895db44d77/html5/thumbnails/32.jpg)
Hosted by
I/O Redundancy
Host to subsystem• Mirroring: Host to independent targets
• Multi-pathing: Host to a single target
Subsystem to subsystem• Store and forward:
Local
Remote
![Page 33: Storage Network Designs for OLTP Business Continuity](https://reader036.vdocuments.mx/reader036/viewer/2022070411/56814716550346895db44d77/html5/thumbnails/33.jpg)
Hosted by
Disk Mirroring: Redundant storage targets
Independent, identically sized storage address spaces
One controller Two controllers
![Page 34: Storage Network Designs for OLTP Business Continuity](https://reader036.vdocuments.mx/reader036/viewer/2022070411/56814716550346895db44d77/html5/thumbnails/34.jpg)
Hosted by
Disk Mirroring: I/Os to 2 Targets
“Brute force” redundancy: fast and simple
Both read and write I/Os• Overlapped reads for performance
Local connections
Limited capacity*
I/O Bottlenecks* for random I/O activity
• * if targets are disk drives
![Page 35: Storage Network Designs for OLTP Business Continuity](https://reader036.vdocuments.mx/reader036/viewer/2022070411/56814716550346895db44d77/html5/thumbnails/35.jpg)
Hosted by
Disk Mirroring for Redo Log Files
Log files are a common bottleneck
Use raw partitions
Redundancy is required• Mirroring is adequate
Use highest RPM with lowest seek times
Put on a separate channel from database I/O
Use separate mirrored pairs per group
![Page 36: Storage Network Designs for OLTP Business Continuity](https://reader036.vdocuments.mx/reader036/viewer/2022070411/56814716550346895db44d77/html5/thumbnails/36.jpg)
Hosted by
Mirroring to Storage Subsystems
Independent, identically sized storage address spaces
Two controllers
StorageSubsystem
StorageSubsystem
![Page 37: Storage Network Designs for OLTP Business Continuity](https://reader036.vdocuments.mx/reader036/viewer/2022070411/56814716550346895db44d77/html5/thumbnails/37.jpg)
Hosted by
Mirroring to SubsystemsTargets are subsystems, not disks
• Separate address spaces
Capacity scales to subsystem max
Double level redundancy• Mirroring plus RAID
Multiple disk spindles reduces I/O bottlenecks
![Page 38: Storage Network Designs for OLTP Business Continuity](https://reader036.vdocuments.mx/reader036/viewer/2022070411/56814716550346895db44d77/html5/thumbnails/38.jpg)
Hosted by
Disk Mirroring Datafiles from Host to Storage Subsystems
Disk mirroring + subsystem RAID
Excellent capacity scaling
Adjacent and across campus/town• One subsystem outside site radius
Requires longer distance cabling
Reads and writes both transmitted
![Page 39: Storage Network Designs for OLTP Business Continuity](https://reader036.vdocuments.mx/reader036/viewer/2022070411/56814716550346895db44d77/html5/thumbnails/39.jpg)
Hosted by
Multi-Pathing: Redundant Paths Between a Host & Subsystem
Pathing software determines that a transmission error occurs & switches to a redundant path
Application data volume
![Page 40: Storage Network Designs for OLTP Business Continuity](https://reader036.vdocuments.mx/reader036/viewer/2022070411/56814716550346895db44d77/html5/thumbnails/40.jpg)
Hosted by
Multi-pathing vs Mirroring
Mirroring assumes independent, but
similar storage targets
Multi-pathing assumes multiple paths to
the exact same target
Mirroring can use a single HBA, multi-
pathing needs two HBAs
![Page 41: Storage Network Designs for OLTP Business Continuity](https://reader036.vdocuments.mx/reader036/viewer/2022070411/56814716550346895db44d77/html5/thumbnails/41.jpg)
Hosted by
Path Failures
1. HBA problem
2. Link, switch or network problem
3. Subsystem controller problem
Application data volume
![Page 42: Storage Network Designs for OLTP Business Continuity](https://reader036.vdocuments.mx/reader036/viewer/2022070411/56814716550346895db44d77/html5/thumbnails/42.jpg)
Hosted by
Transmission failures recognized after SCSI timeouts are exceeded
I/O sent to storage
No ack received
The I/Os is retried and eventually an error is passed back to the process that issued the I/O
![Page 43: Storage Network Designs for OLTP Business Continuity](https://reader036.vdocuments.mx/reader036/viewer/2022070411/56814716550346895db44d77/html5/thumbnails/43.jpg)
Hosted by
Path Failover for OLTP I/O
Redundant path resources take over activities for a failed
path to sustain operations without disrupting service or
risking data integrity
![Page 44: Storage Network Designs for OLTP Business Continuity](https://reader036.vdocuments.mx/reader036/viewer/2022070411/56814716550346895db44d77/html5/thumbnails/44.jpg)
Hosted by
Store and Forward
Host
BA
Independent, identically sized storage address spaces
![Page 45: Storage Network Designs for OLTP Business Continuity](https://reader036.vdocuments.mx/reader036/viewer/2022070411/56814716550346895db44d77/html5/thumbnails/45.jpg)
Hosted by
Store & Forward: One Host I/O and Two Copies of Data
Only real option for remote copies
Does not forward read I/Os
Proprietary protocols and methods• Standards are emerging ie. FC/IP
First step to storage snapshots
![Page 46: Storage Network Designs for OLTP Business Continuity](https://reader036.vdocuments.mx/reader036/viewer/2022070411/56814716550346895db44d77/html5/thumbnails/46.jpg)
Hosted by
Store and Forward: Acknowledgements
A
I/O ACK
B
Forward
Asynchronous
A
I/O ACK
B
Synchronous
Forward
ACK
![Page 47: Storage Network Designs for OLTP Business Continuity](https://reader036.vdocuments.mx/reader036/viewer/2022070411/56814716550346895db44d77/html5/thumbnails/47.jpg)
Hosted by
Trade-offs withAcknowledgement Handling
Synchronous• Always preferred• Slowest performance• State of copy is precise
Asynchronous: • Fastest performance• Least precise knowledge of copy status
![Page 48: Storage Network Designs for OLTP Business Continuity](https://reader036.vdocuments.mx/reader036/viewer/2022070411/56814716550346895db44d77/html5/thumbnails/48.jpg)
Hosted by
Store & Forward: Local and Remote Copies
Local & nearby copy techniques
• Synchronous
• Fiber optic cabling, optical/DWDM services
Remote-far away copy techniques
• Asynchronous
• ATM gateways, OC-12 or less, FC/IP
![Page 49: Storage Network Designs for OLTP Business Continuity](https://reader036.vdocuments.mx/reader036/viewer/2022070411/56814716550346895db44d77/html5/thumbnails/49.jpg)
Hosted by
Mirroring vs Synchronous Store and Forward for Local & Nearby Copies
Mirroring
• Async I/O
• Reads and writes
• No snapshot tie-in
• Uses more host slots
• Least costly
Store and Forward
• Async or Sync I/O
• Writes only
• Snapshot ready
• May conserve host I/O
slots
• Most costly
![Page 50: Storage Network Designs for OLTP Business Continuity](https://reader036.vdocuments.mx/reader036/viewer/2022070411/56814716550346895db44d77/html5/thumbnails/50.jpg)
Hosted by
Combining Mirroring with Store and Forward
LocalNearby Remote Far
AwayMirroring Radius
Store and Forward Radius
![Page 51: Storage Network Designs for OLTP Business Continuity](https://reader036.vdocuments.mx/reader036/viewer/2022070411/56814716550346895db44d77/html5/thumbnails/51.jpg)
Hosted by
Data Redundancy for OLTP
Backup
Snapshots
Delta (log files)
![Page 52: Storage Network Designs for OLTP Business Continuity](https://reader036.vdocuments.mx/reader036/viewer/2022070411/56814716550346895db44d77/html5/thumbnails/52.jpg)
Hosted by
Backup for OLTP
A whole subject unto itself
Disaster recovery primarily
Cold? Who can afford to do that anymore?
Hot – put DB in backup mode
Backup snapshot image of data
![Page 53: Storage Network Designs for OLTP Business Continuity](https://reader036.vdocuments.mx/reader036/viewer/2022070411/56814716550346895db44d77/html5/thumbnails/53.jpg)
Hosted by
Subsystem Snapshots for OLTP
DatabaseServer
DiskStorage
SubsystemA
DiskStorage
SubsystemB
DiskStorage
Subsystemc
1. Flush host buffers (sync, sync)
2. Create Snapshot
![Page 54: Storage Network Designs for OLTP Business Continuity](https://reader036.vdocuments.mx/reader036/viewer/2022070411/56814716550346895db44d77/html5/thumbnails/54.jpg)
Hosted by
Logical Snapshots for OLTP
v
1. The address space is mapped
2. First updates
3. Secondupdates
Overwritten data locations are not returned to the free space pool. (Undelete)
![Page 55: Storage Network Designs for OLTP Business Continuity](https://reader036.vdocuments.mx/reader036/viewer/2022070411/56814716550346895db44d77/html5/thumbnails/55.jpg)
Hosted by
Delta Redundancy with Log Files
Recording of all transaction activities
Roll forward, bring up to date
Roll Backward, go to known good state
Terrific tool for remote redundancy
Not HA
Process cannot have holes in it
![Page 56: Storage Network Designs for OLTP Business Continuity](https://reader036.vdocuments.mx/reader036/viewer/2022070411/56814716550346895db44d77/html5/thumbnails/56.jpg)
Hosted by
Remote Redundancy w/ Log Files
-1
d(x) = f(x) – f(x-1) f(x-1) f(x)
Latest Redo Log File Previous Instance Current to Log File Switch Checkpoint
![Page 57: Storage Network Designs for OLTP Business Continuity](https://reader036.vdocuments.mx/reader036/viewer/2022070411/56814716550346895db44d77/html5/thumbnails/57.jpg)
Hosted by
And now, some thoughts from our sponsor…..
How come I always end up doing all the work?
He never does anything
except eat and sleep
ManagingManagingRedundancy Redundancy is Hard Workis Hard Work
Redundancy is a way of life
![Page 58: Storage Network Designs for OLTP Business Continuity](https://reader036.vdocuments.mx/reader036/viewer/2022070411/56814716550346895db44d77/html5/thumbnails/58.jpg)
Hosted by
SAN Considerations
Fabrics and SAN Islands
Zoning
Switches and directors
Multiplexing (oversubscribing)
Security
![Page 59: Storage Network Designs for OLTP Business Continuity](https://reader036.vdocuments.mx/reader036/viewer/2022070411/56814716550346895db44d77/html5/thumbnails/59.jpg)
Hosted by
Fabrics ARE the SAN Environment
One size does not fit all applications
Larger fabrics carry more risks
VSANs are probably a good idea
Only use switches supporting hot, stateful
firmware upgrades
![Page 60: Storage Network Designs for OLTP Business Continuity](https://reader036.vdocuments.mx/reader036/viewer/2022070411/56814716550346895db44d77/html5/thumbnails/60.jpg)
Hosted by
SAN Islands May be Best for OLTP
Most risk averse approach
Dual fabrics, one fabric per I/O path
Switch problems do not cascade
But, higher management costs
![Page 61: Storage Network Designs for OLTP Business Continuity](https://reader036.vdocuments.mx/reader036/viewer/2022070411/56814716550346895db44d77/html5/thumbnails/61.jpg)
Hosted by
Zoning & OLTP
All ports defined to zones• No rogue ports and zombie zones
Restrict access to current servers• Need-to-access only
![Page 62: Storage Network Designs for OLTP Business Continuity](https://reader036.vdocuments.mx/reader036/viewer/2022070411/56814716550346895db44d77/html5/thumbnails/62.jpg)
Hosted by
Switches and DirectorsRedundancy eats slots and ports
• Pathing, mirroring
• Separate channels for data and logs
Avoid traversing ISLs, if possible• Added latency and blocking potential
• Trunking must have NSPoF
![Page 63: Storage Network Designs for OLTP Business Continuity](https://reader036.vdocuments.mx/reader036/viewer/2022070411/56814716550346895db44d77/html5/thumbnails/63.jpg)
Hosted by
Security
Admin security for an OLTP SAN should
be as strong as possible• No monkey business
No default passwords left
WAN encryption of log files
![Page 64: Storage Network Designs for OLTP Business Continuity](https://reader036.vdocuments.mx/reader036/viewer/2022070411/56814716550346895db44d77/html5/thumbnails/64.jpg)
Hosted by
Recommendations:Determine OLTP availability needs
• Where copies should be, time to access
Match storage network implementation to
DB file types
Develop availability-driven policies• Equipment
• Processes