mgangler virtualization
DESCRIPTION
TRANSCRIPT
![Page 1: Mgangler Virtualization](https://reader034.vdocuments.mx/reader034/viewer/2022051412/54b7fdf34a7959e95a8b45ce/html5/thumbnails/1.jpg)
Using Virtualization to setup Oracle QA/Development Database Environments
2/16/11 1
![Page 2: Mgangler Virtualization](https://reader034.vdocuments.mx/reader034/viewer/2022051412/54b7fdf34a7959e95a8b45ce/html5/thumbnails/2.jpg)
Using Virtualization to setup Oracle QA/Development Database
Environments
Rocky Mountain Training DaysFeb. 16, 2011
Mike Gangler – Ithaka / JSTOR
![Page 3: Mgangler Virtualization](https://reader034.vdocuments.mx/reader034/viewer/2022051412/54b7fdf34a7959e95a8b45ce/html5/thumbnails/3.jpg)
Presentation Agenda
• Who am I• Describe QA / Development Challenges• Pre-Virtual Oracle Environment layout• Solaris Zones and ZFS• ZFS Commands Used• Virtual Oracle Environment layout• Steps required – Zones• Scripts required• Demo if time / Wifi• Recap and QA
16-Feb-2011 3
![Page 4: Mgangler Virtualization](https://reader034.vdocuments.mx/reader034/viewer/2022051412/54b7fdf34a7959e95a8b45ce/html5/thumbnails/4.jpg)
Quick Survey
• How many people are using a form of Virtualization ?
16-Feb-2011
![Page 5: Mgangler Virtualization](https://reader034.vdocuments.mx/reader034/viewer/2022051412/54b7fdf34a7959e95a8b45ce/html5/thumbnails/5.jpg)
Quick Survey
• How many people are using a form of Virtualization ?– VMWARE ?
16-Feb-2011
![Page 6: Mgangler Virtualization](https://reader034.vdocuments.mx/reader034/viewer/2022051412/54b7fdf34a7959e95a8b45ce/html5/thumbnails/6.jpg)
Quick Survey
• How many people are using a form of Virtualization ?– VMWARE ?– Solaris ?
16-Feb-2011
![Page 7: Mgangler Virtualization](https://reader034.vdocuments.mx/reader034/viewer/2022051412/54b7fdf34a7959e95a8b45ce/html5/thumbnails/7.jpg)
Quick Survey
• How many people are using a form of Virtualization ?– VMWARE ?– Solaris ?– Oracle VM ?
16-Feb-2011
![Page 8: Mgangler Virtualization](https://reader034.vdocuments.mx/reader034/viewer/2022051412/54b7fdf34a7959e95a8b45ce/html5/thumbnails/8.jpg)
Quick Survey
• How many people are using a form of Virtualization ?– VMWARE ?– Solaris ?– Oracle VM ?– Other ?
16-Feb-2011
![Page 9: Mgangler Virtualization](https://reader034.vdocuments.mx/reader034/viewer/2022051412/54b7fdf34a7959e95a8b45ce/html5/thumbnails/9.jpg)
Quick Survey
• How many people are using a form of Virtualization ?– VMWARE ?– Solaris ?– Oracle VM ?– Other ?
• How many are running oracle instances on these VM machines ?
16-Feb-2011
![Page 10: Mgangler Virtualization](https://reader034.vdocuments.mx/reader034/viewer/2022051412/54b7fdf34a7959e95a8b45ce/html5/thumbnails/10.jpg)
Michael J. Gangler (Mike)
• Database Specialist – Ithaka– Working with Oracle, MSSQL, MySQL
• Working with Oracle since Oracle Version 5• IOUG
– Executive Vice President – Finance– Director of Membership services– www.ioug.org– April 10 – 14 – Collaborate –Orlando
• Southeast Michigan Oracle Professionals (SEMOP)– Past President and Current Director– www.semop.org
• Michigan Oracle User Summit (MOUS)– Database / Tech. director– www.mous.us– Wednesday November 30th – Schoolcraft College
16-Feb-2011
![Page 11: Mgangler Virtualization](https://reader034.vdocuments.mx/reader034/viewer/2022051412/54b7fdf34a7959e95a8b45ce/html5/thumbnails/11.jpg)
Ithaka’s mission
• ITHAKA is a not-for-profit organization dedicated to helping the academic community take full advantage of rapidly advancing information and networking technologies. We serve scholars, researchers, and students by providing the content, tools, and services needed to preserve the scholarly record and to advance research and teaching in sustainable ways. We are committed to working in collaboration with other organizations to maximize benefits to our stakeholders.
16-Feb-2011 11
![Page 12: Mgangler Virtualization](https://reader034.vdocuments.mx/reader034/viewer/2022051412/54b7fdf34a7959e95a8b45ce/html5/thumbnails/12.jpg)
Ithaka’s Services
• Ithaka S+R works with initiatives and organizations to develop sustainable business models and conducts research and analysis on the impact of digital media on the academic community as a whole. www.ithaka.org
• JSTOR helps scholars, researchers, and students discover, use, and build upon a wide range of content in a trusted digital archive of over 1,200 academic journals and other content. JSTOR uses information technology and tools to increase productivity and facilitate new forms of scholarship. www.jstor.org
• Portico preserves scholarly literature published in electronic form—more than 10,000 e-journals and 28,000 e-books—and ensures that these materials remains accessible to future scholars, researchers, and students. www.portico.org
16-Feb-2011 12
![Page 13: Mgangler Virtualization](https://reader034.vdocuments.mx/reader034/viewer/2022051412/54b7fdf34a7959e95a8b45ce/html5/thumbnails/13.jpg)
16-Feb-2011
![Page 14: Mgangler Virtualization](https://reader034.vdocuments.mx/reader034/viewer/2022051412/54b7fdf34a7959e95a8b45ce/html5/thumbnails/14.jpg)
Pre-Virtual Environment
16-Feb-2011
![Page 15: Mgangler Virtualization](https://reader034.vdocuments.mx/reader034/viewer/2022051412/54b7fdf34a7959e95a8b45ce/html5/thumbnails/15.jpg)
Database / Application Specifics
• Oracle 10.2.0.4• SGA – 11 gb• Oracle Dataguard / standby used• Data is replicated via application and Daemons• 1 schema – 160 – 200 gb• Holds only pointers versus actual files• Files are stored at each data center• Software is a 3rd party application
16-Feb-2011 15
![Page 16: Mgangler Virtualization](https://reader034.vdocuments.mx/reader034/viewer/2022051412/54b7fdf34a7959e95a8b45ce/html5/thumbnails/16.jpg)
QA / Development Challenges
• QA requires restoration from Production - Long export / Import times – (6-8 Hours)
• QA restore every 1 – 2 weeks.• Multi-Data Center requires additional work and
headaches• Development needs many restores due to changes
and testing • Application written data replication (Versus Database)
16-Feb-2011 16
![Page 17: Mgangler Virtualization](https://reader034.vdocuments.mx/reader034/viewer/2022051412/54b7fdf34a7959e95a8b45ce/html5/thumbnails/17.jpg)
Pre-Virtual QA Environment
16-Feb-2011 17
WebApp
Master
Web App
Slave
Web App
Slave
DataCenter
1
DataCenter
2
Data Center
3
Qa.jstor.org
Data Replication
Data Replication
![Page 18: Mgangler Virtualization](https://reader034.vdocuments.mx/reader034/viewer/2022051412/54b7fdf34a7959e95a8b45ce/html5/thumbnails/18.jpg)
Pre-Virtual QA Steps
• Export Production database – 2 hours• Zip production export – 30 minutes• Copy production export file to sites
– Manchester, England – 3 hours– Princeton , NJ – 1.5-2 hours– Ann Arbor – 1 hour
• Unzip export file – 30 minutes• Import export file – 3 – 5 hours
• Total Time for all 3 sites (approx) – 10 – 12 hours
16-Feb-2011 18
![Page 19: Mgangler Virtualization](https://reader034.vdocuments.mx/reader034/viewer/2022051412/54b7fdf34a7959e95a8b45ce/html5/thumbnails/19.jpg)
Virtual EnvironmentZones & ZFS
16-Feb-2011
![Page 20: Mgangler Virtualization](https://reader034.vdocuments.mx/reader034/viewer/2022051412/54b7fdf34a7959e95a8b45ce/html5/thumbnails/20.jpg)
20
Solaris Zones
16-Feb-2011
![Page 21: Mgangler Virtualization](https://reader034.vdocuments.mx/reader034/viewer/2022051412/54b7fdf34a7959e95a8b45ce/html5/thumbnails/21.jpg)
21
ZFS
16-Feb-2011
ZFS is a combined file system and logical volume manager designed by Sun Microsystems.
ZFS includes :data integrity (protection against bit rot, etc.)Support for high storage capacities Integration of the concepts of file system and volume
managementCan provide snapshots and copy-on-write clonesContinuous integrity checking and automatic repairRAID-Z and native NFSv4 ACLs. ZFS is implemented as open-source software
![Page 22: Mgangler Virtualization](https://reader034.vdocuments.mx/reader034/viewer/2022051412/54b7fdf34a7959e95a8b45ce/html5/thumbnails/22.jpg)
22
Solaris ZFS
16-Feb-2011
![Page 23: Mgangler Virtualization](https://reader034.vdocuments.mx/reader034/viewer/2022051412/54b7fdf34a7959e95a8b45ce/html5/thumbnails/23.jpg)
23
ZFS Commands Used
16-Feb-2011
“zfs snapshot” - performs a copy of the data “zoneadm” – halts and starts the local zones “zfs destroy” – removing a clone or zpool “zfs clone” - Writeable snapshots , resulting in
two independent file systems that share a set of blocks.
“zfs list” – lists out all the snapshots “zfs set mountpoint” – allocates an area data
to a mountpoint “zfs set zoned” - turns on/off the local zones
![Page 24: Mgangler Virtualization](https://reader034.vdocuments.mx/reader034/viewer/2022051412/54b7fdf34a7959e95a8b45ce/html5/thumbnails/24.jpg)
ZFS Snapshot
An advantage of copy-on-write is that when ZFS writes new data, the blocks containing the old data can be retained, allowing a snapshot version of the file system to be maintained. ZFS snapshots are created very quickly, since all the data composing the snapshot is already stored; they are also space efficient, since any unchanged data is shared among the file system and its snapshots.
• Runs from the Master site• Need Root password• All oracle processes need to be shutdown• User must be in NON- /u01 directory
16-Feb-2011 24
![Page 25: Mgangler Virtualization](https://reader034.vdocuments.mx/reader034/viewer/2022051412/54b7fdf34a7959e95a8b45ce/html5/thumbnails/25.jpg)
ZFS Snapshot - Example
• [oracle@aa2vps217]$ su• Password: • # zfs snapshot tank/zones/aa2vps217-u01@20100701• # zfs list• NAME USED AVAIL REFER MOUNTPOINT• tank 849G 624G 267K /tank• tank/zones 821G 624G 26K /tank/zones• tank/zones/aa2vps217-u01 677G 624G 400G /u01• tank/zones/aa2vps217-u01@20100602 133G - 162G -• tank/zones/aa2vps217-u01@20100701 0G - 447G -
16-Feb-2011
![Page 26: Mgangler Virtualization](https://reader034.vdocuments.mx/reader034/viewer/2022051412/54b7fdf34a7959e95a8b45ce/html5/thumbnails/26.jpg)
Virtual EnvironmentProcess and setup
16-Feb-2011
![Page 27: Mgangler Virtualization](https://reader034.vdocuments.mx/reader034/viewer/2022051412/54b7fdf34a7959e95a8b45ce/html5/thumbnails/27.jpg)
2716-Feb-2011
•Qa.jstor.org
•Web •Master
WebSlave
•Web•Slave
•ff•Db•Master
•Db•Slave
•Db•slave
1 physical Machine
1 Global Zone3 Zones
1 physical Machine1 Global Zone3 Zones
Virtual QA environment
![Page 28: Mgangler Virtualization](https://reader034.vdocuments.mx/reader034/viewer/2022051412/54b7fdf34a7959e95a8b45ce/html5/thumbnails/28.jpg)
Virtual QA Steps
• Export Production database – Not Required – Using Previous nights backup * cron job copies it to master site nightly
• Zip production export – N/A• Unzip Export File – 30 Minutes• Import Export File to MASTER ONLY ! - 2 – 3 Hours• Run ZFS SNAPSHOT Master Only – 5 Minutes• Run ZFS CLONE to slave sites (run from global Zone) - 10 Mins• Run script to change database names (Each site) – 5 Minutes• Total Time for all 3 sites (approx) – 3 – 4 Hours
16-Feb-2011 28
![Page 29: Mgangler Virtualization](https://reader034.vdocuments.mx/reader034/viewer/2022051412/54b7fdf34a7959e95a8b45ce/html5/thumbnails/29.jpg)
ZFS Cloning Process
• Runs from the Global Zone Site – Must be root user
• Cloning steps: Halt the zones
zoneadm –z <slave> halt Destroy the old u01 directories (slave machines only)
zfs destroy –r tank/zones/<slave>-u01 Clone the master u01 directory to the slaves
Zfs clone tank/zones/<master>-u01@`date +%Y%m%d` tank/zones/<slave1>-u01
16-Feb-2011 29
![Page 30: Mgangler Virtualization](https://reader034.vdocuments.mx/reader034/viewer/2022051412/54b7fdf34a7959e95a8b45ce/html5/thumbnails/30.jpg)
ZFS Cloning Process (Con’t)
List out Zone copiesZfs list –o name,origin
Set the mount pointsZfs set mountpoint=/u01 tank/zones/<slave>-u01
Set the zone back onZfs set zoned=on tank/zones/<slave>-u01
Reboot the zonesZoneadm –z <slave> boot
List out Zone copies – same as above
16-Feb-2011 30
![Page 31: Mgangler Virtualization](https://reader034.vdocuments.mx/reader034/viewer/2022051412/54b7fdf34a7959e95a8b45ce/html5/thumbnails/31.jpg)
3116-Feb-2011
Example Clone Script – Page 1
#!/usr/bin/env bash
export PATH=/usr/local/bin:$PATH
host=aa2x4240-04echo =====Must be run as SU======echo =====Cloning omega - 217, 218, 219 ======
echo =====listing out the zone copies=======echo zfs list -o name,originzfs list -o name,origin
echo =====halting zones=======zoneadm -z aa2vps218 haltzoneadm -z aa2vps219 halt
ZFS Clone Example
![Page 32: Mgangler Virtualization](https://reader034.vdocuments.mx/reader034/viewer/2022051412/54b7fdf34a7959e95a8b45ce/html5/thumbnails/32.jpg)
3216-Feb-2011
Example Clone Script – Page 2 (cont)
echo =====destroying old u01 directories======zfs destroy -r tank/zones/aa2vps218-u01zfs destroy -r tank/zones/aa2vps219-u01
echo =====cloning snapshots===========zfs clone tank/zones/aa2vps217-u01@`date +%Y%m%d` tank/zones/aa2vps218-u01zfs clone tank/zones/aa2vps217-u01@`date +%Y%m%d` tank/zones/aa2vps219-u01
echo =====listing out the new zone copies======echo zfs list -o name,originzfs list -o name,origin
ZFS Clone Example – Page 2
![Page 33: Mgangler Virtualization](https://reader034.vdocuments.mx/reader034/viewer/2022051412/54b7fdf34a7959e95a8b45ce/html5/thumbnails/33.jpg)
3316-Feb-2011
Example Clone Scripts – Page 3 (cont)
echo =====setting the mount points========zfs set mountpoint=/u01 tank/zones/aa2vps218-u01zfs set mountpoint=/u01 tank/zones/aa2vps219-u01
echo =====set zoned=on===========zfs set zoned=on tank/zones/aa2vps218-u01zfs set zoned=on tank/zones/aa2vps219-u01
echo =====boot machines=======zoneadm -z aa2vps218 bootzoneadm -z aa2vps219 boot
echo =====listing out the new zone copies======zfs list -o name,origin
ZFS Clone Example – Page 3
![Page 34: Mgangler Virtualization](https://reader034.vdocuments.mx/reader034/viewer/2022051412/54b7fdf34a7959e95a8b45ce/html5/thumbnails/34.jpg)
After Clone Script - Slaves
• NOTE – used if the database name needs to be the same
• Rename listener.ora, tnsnames.ora with new machine name
• Clean up Grid Control –
– Remove entries from sysman/emd directory– Change sysman/emd/targets.xml – new machine– Change sysman/emd/emd.proprties – new machine name
16-Feb-2011 34
![Page 35: Mgangler Virtualization](https://reader034.vdocuments.mx/reader034/viewer/2022051412/54b7fdf34a7959e95a8b45ce/html5/thumbnails/35.jpg)
After Clone Example Script - Slaves
• NOTE – used if the database name needs to be the same
• Runs on Slave Sites – as Oracle User• Needs to be in $HOME
• #!/bin/bash
• ### set -xv• oldmach=$1• newmach=$2
• for FILE in $ORACLE_HOME/network/admin/*.ora• do• sed -e 's/'"$oldmach"'/'"$newmach"'/g' $FILE > tmp.out• mv tmp.out $FILE• done
16-Feb-2011 35
![Page 36: Mgangler Virtualization](https://reader034.vdocuments.mx/reader034/viewer/2022051412/54b7fdf34a7959e95a8b45ce/html5/thumbnails/36.jpg)
After Clone Example Script – Slaves – part 2
rm -r -f $AGENT_HOME/sysman/emd/core*.*rm -r -f $AGENT_HOME/sysman/emd/state/*rm -r -f $AGENT_HOME/sysman/emd/upload/*rm -r -f $AGENT_HOME/sysman/emd/recv/*rm -r -f $AGENT_HOME/sysman/emd/collection/*rm -r -f $AGENT_HOME/sysman/emd/lastupld.xmlrm -r -f $AGENT_HOME/sysman/emd/agntstmp.txt
for FILE in $AGENT_HOME/sysman/emd/targets.xmldo sed -e 's/'"$oldmach"'/'"$newmach"'/g' $FILE > tmp.out mv tmp.out $FILEDonefor FILE in $AGENT_HOME/sysman/config/emd.propertiesdo sed -e 's/'"$oldmach"'/'"$newmach"'/g' $FILE > tmp.out mv tmp.out $FILEdone
exit 0
16-Feb-2011 36
![Page 37: Mgangler Virtualization](https://reader034.vdocuments.mx/reader034/viewer/2022051412/54b7fdf34a7959e95a8b45ce/html5/thumbnails/37.jpg)
3716-Feb-2011
# ./clone-presentation.sh=====Must be run as SU===========Cloning omega - 97, 98, 99 and ===========halting zones=======zoneadm -z aa2vps98 haltzoneadm -z aa2vps99 halt=====destroying old u01 directories======zfs destroy -r tank/zones/aa2vps98-u01zfs destroy -r tank/zones/aa2vps99-u01=====cloning snapshots===========zfs clone tank/zones/aa2vps97-u01@20101012 tank/zones/aa2vps98-u01zfs clone tank/zones/aa2vps97-u01@20101012 tank/zones/aa2vps99-u01=====setting the mount points========zfs set mountpoint=/u01 tank/zones/aa2vps98-u01zfs set mountpoint=/u01 tank/zones/aa2vps99-u01cannot mount '/u01': directory is not emptyproperty may be set but unable to remount filesystem=====set zoned=on===========zfs set zoned=on tank/zones/aa2vps98-u01zfs set zoned=on tank/zones/aa2vps99-u01=====boot machines=========zoneadm -z aa2vps98 bootzoneadm -z aa2vps99 boot
Example Run – ZFS Clone – Page 1
![Page 38: Mgangler Virtualization](https://reader034.vdocuments.mx/reader034/viewer/2022051412/54b7fdf34a7959e95a8b45ce/html5/thumbnails/38.jpg)
3816-Feb-2011
=====listing out the new zone copies======zfs list -o name,originNAME ORIGINtank -tank/snaps -tank/swap1 -tank/zones -tank/zones/aa2vps191 -tank/zones/aa2vps191-u01 -tank/zones/aa2vps247-v01 -tank/zones/aa2vps247-v01@20081106_detached -tank/zones/aa2vps247-v02 -tank/zones/aa2vps247-v02@20081105_detached -tank/zones/aa2vps248-v01 tank/zones/aa2vps247-v01@20081106_detachedtank/zones/aa2vps248-v02 tank/zones/aa2vps247-v02@20081105_detachedtank/zones/aa2vps249-v01 tank/zones/aa2vps247-v01@20081106_detachedtank/zones/aa2vps249-v02 tank/zones/aa2vps247-v02@20081105_detachedtank/zones/aa2vps97 tank/zones/aa2vps98@200909131322-migrationtank/zones/aa2vps97-u01 -tank/zones/aa2vps97-u01@20101008 -tank/zones/aa2vps98 tank/zones/aa2vps99@200909131307-migrationtank/zones/aa2vps98@200909131322-migration -tank/zones/aa2vps98-u01 tank/zones/aa2vps97-u01@20101008tank/zones/aa2vps99 -tank/zones/aa2vps99@200909131307-migration -tank/zones/aa2vps99-u01 tank/zones/aa2vps97-u01@20101008#
Example Run – ZFS Clone – Page 2
![Page 39: Mgangler Virtualization](https://reader034.vdocuments.mx/reader034/viewer/2022051412/54b7fdf34a7959e95a8b45ce/html5/thumbnails/39.jpg)
3916-Feb-2011
TIPS
o Keep the zone root, Oracle Binaries and Oracle data on the SANo Keep the resource management simple – Use dedicated CPU
feature in lieu resource poolso To Prevent potential problems in the global zone:
Create a separate file system for each local zone root Avoid using bock/raw devices in non-global zones
o For Greater Isolation of Environments: Keep the zone root, Oracle Binaries and Oracle data on
separate file systems and mounts. Install Oracle inside each non-global zone rather than
sharing binaries from the global zone.
![Page 40: Mgangler Virtualization](https://reader034.vdocuments.mx/reader034/viewer/2022051412/54b7fdf34a7959e95a8b45ce/html5/thumbnails/40.jpg)
40
Conclusion
16-Feb-2011
Virtualization can reduce down time Very Efficient with Test and QA
databases I haven’t tried it with Production
databases, but we do use it for Production Web Servers
Need to understand all the applications on the local zones
You can clone across local zones and global zones
Concepts can be used with any virtualization software (VMWARE, etc).