provisioning oracle fmw environments with chef & puppet con7629 mark nelson, edwin biemond...
TRANSCRIPT
Provisioning Oracle FMWEnvironments with Chef & PuppetCON7629
Mark Nelson, Edwin BiemondFusion Middleware Platform TeamOctober 02, 2014
Copyright © 2014, Oracle and/or its affiliates. All rights reserved. |
Copyright © 2014, Oracle and/or its affiliates. All rights reserved. | 3
• Mark Nelson– Architect, Oracle
Copyright © 2014, Oracle and/or its affiliates. All rights reserved. | 4
• Edwin Biemond– Consulting Member Technical Staff, Oracle
Copyright © 2014, Oracle and/or its affiliates. All rights reserved. | 5
Agenda
Current State
Experiences
Continuous Delivery
Testing
Wrap Up, Q & A
1
2
3
4
5
Copyright © 2014, Oracle and/or its affiliates. All rights reserved. | 6
Agenda
Current State
Experiences
Continuous Delivery
Testing
Wrap Up, Q & A
1
2
3
4
5
Copyright © 2014, Oracle and/or its affiliates. All rights reserved. |
What is possible Now!!!
Copyright © 2014, Oracle and/or its affiliates. All rights reserved. | 8
Focus on Puppet & Chef, which are the most popular provisioning tools
• > 10.000.000 nodes• Fastest growing companies in the US• Stand-alone (Client) & Master (Server)• Open Source & Enterprise / Premium– Free < 10 Nodes (Puppet)– Free < 5 Nodes (Chef)
• Manage Linux, Solaris & Windows• Cloud plugins, VMware, Google, Openstack,
Azure & Amazon
Copyright © 2014, Oracle and/or its affiliates. All rights reserved. | 9
Chef components overview
Copyright © 2014, Oracle and/or its affiliates. All rights reserved. | 10
Next major release• SOA architecture• Services• Easy to scale up• Own release lifecycle
• Runs in a JVM
• 3 times faster
• Build with clojure, same as PuppetDB
• JRuby
Puppet component overview
Copyright © 2014, Oracle and/or its affiliates. All rights reserved. | 11
• Administrator / Enterprise focus• Forge.puppetlabs.com > 2700
Modules• Language, Puppet DSL & Ruby• Manifest-> Some Puppet DSL Code • Puppet Modules• Facter• Type/Provider -> Ruby
• Developer focus• Supermarket.getchef.com> 1600
Cookbooks• Language, Chef DSL & Ruby• Recipe -> Some Chef DSL Code• Chef Cookbooks• Ohai• Resource/Provider– Light -> Chef DSL– Heavy -> Ruby
Copyright © 2014, Oracle and/or its affiliates. All rights reserved. | 12
file{'/etc/resolv.conf':
content => template('mod/my_resolv.conf.erb'),
owner => 'root',
group => 'root',
mode => '0644',
}
$files = ['bacon', 'eggs', 'sausage']
file{$files:
content => "#{title} is delicious!",
}
package{'bash':
ensure => latest,
}
template '/etc/resolv.conf' do
source 'my_resolv.conf.erb'
owner 'root'
group 'root'
mode '0644'
end
['bacon', 'eggs', 'sausage'].each do |type|
file "/tmp/#{type}" do
content "#{type} is delicious!"
end
end
package 'bash' do
action :upgrade
end
Some DSL Code
Copyright © 2014, Oracle and/or its affiliates. All rights reserved. | 13
So how does it work!!• We have to create some recipes/manifests which describes the To Be
situation & not how to achieve it • Execute chef-apply or puppet apply• Chef/Puppet retrieves the current state and determines what the necessary
actions should be• Executes the actions (apply only deltas)
Copyright © 2014, Oracle and/or its affiliates. All rights reserved. | 14
Agenda
Current State
Experiences
Continuous Delivery
Testing
Wrap Up, Q & A
1
2
3
4
5
15Copyright © 2014, Oracle and/or its affiliates. All rights reserved. |
ExperiencesChef & Puppet challenges & solutions
Copyright © 2014, Oracle and/or its affiliates. All rights reserved. | 16
WebLogic 12c Changes• All jars, no O.S. specific installer– Jars can be used on every Operating System
• Shared oracle_common which is part of WebLogic Infrastructure– ServiceBus installer +/- 400Mb
• Repository Creation Utility is now part of WebLogic Infrastructure– FMW Database repository tightly connected to the domain (because of OPSS)
• WebLogic Restful Management Services
Copyright © 2014, Oracle and/or its affiliates. All rights reserved. | 17
Oracle Tools like OPatch, RCU or BSU are not idempotent• 2nd Times always fails– It breaks Chef or Puppet– Hard to handle this with the standard Puppet/Chef exec resource– Slow, inventory command is always faster then apply and let it crash
• For example OPatch utility– Detect current state with opatch lsinventory• Every utility has a different output ( XML, CSV or text ouput )
– Determine the right action– Skip, Install or Remove the patch with opatch apply– Check the outcome
Copyright © 2014, Oracle and/or its affiliates. All rights reserved. | 18
One Time• Not really the Chef or Puppet way• One big or many WLST scripts• Use exec of Chef or Puppet, almost
impossible to control (Facter/Ohai)• Execute this once or need to catch
the BeanAlreadyExists exception• Re-use your old scripts• Easy to get out of sync
Model driven• WebLogic Chef/Puppet Resources• The Model is always up-to-date• Need to know some Ruby• Map WLST or the WLS Rest
Interface (12.1.3) to Chef or Puppet Attributes• 4 WLST scripts per WLS resource,
retrieve, create/update and destroy• Big effort to have all WLS resources
One time provisioning or Model driven
Copyright © 2014, Oracle and/or its affiliates. All rights reserved. | 19
• BSU, OPatch, AdminServer & ManagedServer Control• Machine, Domain, Server, Channels• (Dynamic) Cluster, Coherence,
Server Template• User, Group, Authentication
providers• Deployment
• Datasource• File Persistence• JMS server, JMS module,
Connection Factory, Queue, Topic, SubDeployment, Quota, Foreign Server• SAF agents, Imported Destination,
Remote Context, Error handler• Workmanagers• Virtualhost
Puppet WebLogic Resources ( Total of 37)
Copyright © 2014, Oracle and/or its affiliates. All rights reserved. | 20
Puppet apply Puppet resource wls_server SoaServer1 --edit
Model driven resource demo
Copyright © 2014, Oracle and/or its affiliates. All rights reserved. | 21
WLST Performance
• Startup of WLST takes times-> even with skipWLSModuleScanning
• Connect() or readDomain(‘xx’) is slow
• With > 10 WLST scripts or with multiple Domains you will have some bad performance
• Don’t schedule a Chef or Puppet run every XX minutes
Copyright © 2014, Oracle and/or its affiliates. All rights reserved. | 22
WLST Domain Daemon process
• For Every domain
• Is already connected to the Domain
• Auto start & destroy in every Puppet/Chef run
• More complexity -> need to know when it is finished or when it fails
• Can’t handle invalid WLST/Python scripts
• It’s Fast -> 2 Domains with 8 clusters and > 100 Queues in just a few minutes ( normal 30-60 minutes)
Copyright © 2014, Oracle and/or its affiliates. All rights reserved. | 23
Naming of WebLogic artifacts• In Chef or Puppet a resource should be unique. There can be only one
Firewall Service or one Package Bash• In WebLogic you can have multiple Domains and a Queue only has to
unique inside a JMS Module.• Need to have a WebLogic Title convention and implement Title pattern in
all the Chef/Puppet Resources
Copyright © 2014, Oracle and/or its affiliates. All rights reserved. | 24
Orchestration of FMW HA Environment• A High Available FMW environments needs at least– A Database configuration with Oracle RAC or Oracle Dataguard ( 10 a 30 min ) –WebLogic AdminServer node which depends on the Database ( 5 a 25 min )– 2 or more WebLogic nodes which depends on the AdminServer ( 5 a 10 min )
• Chef or Puppet can’t orchestrate multi-node environments• Can implement some try / catch together with a repeat & wait – AdminServer should wait for a Database with a FMW Repository–WebLogic Nodes should wait for a Domain pack on the AdminServer
• We need something on top of Chef/Puppet which can handle this
Copyright © 2014, Oracle and/or its affiliates. All rights reserved. | 25
Puppet & Chef Sandbox environments
Oracle VirtualBox
Vagrant
Desktop
Shared Folder Puppet / Chef code
VM
VMTemplate
Puppet/ChefClient
Oracle
JDK
WebLogic
Database
GithubForge
Supermarket
yum / wget
• Out of the box with Chef Test Kitchen• Puppet -> requires some Vagrant
configuration
• Easy to simulate environments• Destroy & Re-Create• Direct feedback, Change Log levels• Remote shell access• Only need some MB of configuration data• Retrieve the latest Modules or Cookbooks
Copyright © 2014, Oracle and/or its affiliates. All rights reserved. | 26
Oracle Big Files
• Download & Extract zip files requires a lot of memory or a big swap file
• Takes a lot of time to download plus extract & requires disk space
• It is slow in every run because Puppet/Chef file resource checks for differences (locally and remote)
• Use a Shared Drive
• Skip the File resource step with some Puppet/Chef Facts
Copyright © 2014, Oracle and/or its affiliates. All rights reserved. | 27
Agenda
Current State
Experiences
Continuous Delivery
Testing
Wrap Up, Q & A
1
2
3
4
5
28Copyright © 2014, Oracle and/or its affiliates. All rights reserved. |
Continuous Delivery
Copyright © 2014, Oracle and/or its affiliates. All rights reserved. | 29
Separate environment specific variables from your code• Puppet – Facter– Parameter manifest class– Hiera ( Multiple levels Node, Env, Common ) with a YAML, Database, JSON Backend
• CHEF–OHAI– Roles– Environments– Databags– Attribute
Copyright © 2014, Oracle and/or its affiliates. All rights reserved. | 30
Promotion of infrastructure• Versioning (Git Suite -> Gitbucket, Gitlab or Github) for infrastructure
configuration & Development code– Issues– Pull Requests
• Multi environment on the master (Development, Test, Staging, Prod)• Retrieve Modules/Cookbooks from Git or use Forge or Supermarket– Puppet -> Puppet Librarian, R10k– Chef -> Berkshelf, Chef Librarian
Copyright © 2014, Oracle and/or its affiliates. All rights reserved. | 31
Model your environments• Define Building blocks or Roles– Better to understand• just assign to nodes
– Easier to maintain• Consist of small modules• Optimal re-use
– Auto-adapt to the environment• Developer desktop• High Available production environment
Copyright © 2014, Oracle and/or its affiliates. All rights reserved. | 32
Standardize your VM Base ImagePacker.io is a tool for creating identical machine images for multiple platforms from a single source configuration–Works with PXE boot– Download ISO/Net install– Installs Chef or Puppet client– Different post-processors• Vagrant for VMware or Oracle Virtualbox• Amazon• Docker• OpenStack• Google
–Only have to provide a Kickstart file
Copyright © 2014, Oracle and/or its affiliates. All rights reserved. | 33
Continuous Environment Overview – Modules/Cookbooks
Git• Clone
Code• RVM (Parallel)• Chef version
• Lint, RSpec
Smoke Test• Vagrant• Librarian
Approve• Archive• Version
Distribute• Repository
Copyright © 2014, Oracle and/or its affiliates. All rights reserved. | 34
Continuous Environment Overview – Building blocks
Git• Clone
Smoke Test• Vagrant• Librarian
Approve• Archive• Version
Distribute• Repository
Copyright © 2014, Oracle and/or its affiliates. All rights reserved. | 35
Continuous Environment Overview – Test 2 Production
Test
VMs
Cookbooks
RecipesProductionStaging
Test Environment 1
Test Environment 2
Copyright © 2014, Oracle and/or its affiliates. All rights reserved. | 36
Agenda
Current State
Experiences
Continuous Delivery
Testing
Wrap Up, Q & A
1
2
3
4
5
Copyright © 2014, Oracle and/or its affiliates. All rights reserved. | 37
Test/Code Quality frameworks• Rubocup for Ruby coding style
• Chef– Foodcritic for Chef coding style– ChefSpec/Serverspec
• Puppet– Puppet-Lint for Puppet coding style– Rspec
• RVM/Bundler to test the different Ruby & Puppet/Chef combinations• Sublime or IntelliJ plugins
Copyright © 2014, Oracle and/or its affiliates. All rights reserved. | 38
Test against different versionof• Ruby• Chef/Puppet
Test for
• Lint• RSpec Tests• Code Coverage
Opensource
Test in the cloud with Travis CI
Copyright © 2014, Oracle and/or its affiliates. All rights reserved. | 39
Robot Framework• Test your FMW environment if it contains what you expect.– Can’t fully rely on Puppet or Chef output
• With Robot you can add tests in readable and understandable text• Libraries– Selenium for WebLogic Web applications tests– HTTP/Rest Requests for WebLogic Restful Management Services
Copyright © 2014, Oracle and/or its affiliates. All rights reserved. | 40
Test WebLogic applications with Robot
Copyright © 2014, Oracle and/or its affiliates. All rights reserved. | 41
Agenda
Current State
Experiences
Continuous Delivery
Testing
Wrap Up, Q & A
1
2
3
4
5
Copyright © 2014, Oracle and/or its affiliates. All rights reserved. | 42
• JDK7– URandom fix, JCE Security policy
• ORAWLS– Install, Patch, RCU, Domain, Cluster, WLS Native Types
• ORADB, ORA_RAC– Install, ASM, RAC, Patch, Instance, RCU 11g
• ORACLE– DB Native Types like Tablespace, Role, User & Init params
• > 10k Downloads
• Many Contributions/Pull requests
• Many Big Enterprise companies are using it
• Developers with Vagrant/Virtualbox
Download locations
• Forge.puppetlabs.com
• Github.com/biemond
• Github.com/hajee
Opensource Puppet Modules
Copyright © 2014, Oracle and/or its affiliates. All rights reserved. |
ConclusionFinally we have total control over FMW Environments without any human interaction.
And experience the provisioning of new FMW environments in minutes and push changes to DTAP
44Copyright © 2014, Oracle and/or its affiliates. All rights reserved. |
Q & [email protected]@oracle.com
Copyright © 2014, Oracle and/or its affiliates. All rights reserved. | 45