mage titans usa 2016 - mathew beane - edit fully stacked: less oops, more ops for magento...

Fully Stacked: Less Oops, More Ops for Magento Development

Magento Master Mover & Magento Certified Developer

Director of Systems Engineering @ Robofirm

Mathew Beane

Todays Plan

• Foundations: An overview of Magento Development and Operations.

• Team: Discussing the culture of processes, technology and people that are found throughout the Devops Lifecycle.

• Building Blocks: Examining the moving parts that make up Devops.

• Creating a Blueprint: A brief look into the setting up Devops.

Magento Development & OperationsFoundations, an overview of Devops.

What is Devops?

“Devops is the practice of operations and development engineers participating together in the entire service lifecycle, from design through the development process to production support.”From: http://bit.ly/1SVOYGU

Devops Goals

• Release on demand• Prepare technology and processes• Infrastructure as code• Eliminate technical debt • Fail without fear• Measure and document everything• Provide a knowledge and cultural feedback loop

Devops Foundation: CAMS Cycle

Term coined by John Willis in 2010• http://bit.ly/20cIRRq

• This is a feedback loop which can be built around simple building blocks

• Produces incremental growth of the Devops culture, which creates more automation, monitoring and sharing

CAMS: Culture

“People and process first. If you don’t have culture, all automation attempts will be fruitless.”

CAMS: Automation

“Start to stitch together an automation fabric for Devops. Tools for release management, provisioning, configuration management, systems integration, monitoring and control, and orchestration become important pieces in building a Devops fabric.”

CAMS: Measurement

“If you can’t measure, you can’t improve. A successful Devops implementation will measure everything it can as often as it can… performance metrics, process metrics, and even people metrics.”

CAMS: Sharing

“Sharing is the loopback in the CAMS cycle. Creating a culture where people share ideas and problems is critical.”

Team DevopsDiscussing the processes, technology and people that are found throughout the Devops Lifecycle.

Team Devops

Each Devops role has people, processes and technologies that are used to deliver the end product.

Devops Individual Roles

Roles may be more individualized for some of your “Star Players”• Automation Architect• Release Manager• Security Engineer• Experience/Quality Assurance• Disruptive Technology Research Technician

Business / Operations

Business• Sales• Support• Administration• Business Operations• Business Snowflake Logic

Operations• “The Store” and any hardware or

software that is providing it• Support for customers,

administrators and other users• Secondary and tertiary

applications that support “The Store”

Development / Engineering

• People who write and test code• Tools that are used for Development• Methods and Practices that are encouraged• The development and testing infrastructure

Testing / Monitoring

• A wide variety of technologies and tools exist for testing• Monitoring can turn up things tests missed• Responding to monitoring is a must for it to be effective• Testing and Monitoring should be throughout the Devops lifecycle

Customers / End Users

• Users of the store/website including administrators and customers• Any service that consumes data from the website• Processes that are related to “User Experience”• Feedback from this group is critical to operational success

Building BlocksExamining some of the moving parts that make up Devops.

Devops Infrastructure Overview

Example Setup of Development Environment

Example Setup of Testing Environment

Infrastructure and Configuration as Code

Automated builds using Vagrant and Terraform are a great way to treat infrastructure as code.

Configuration Management Tools:• Ansible• Chef• Puppet• Salt Stack• And many others…

AWS

AWS Continuous Full Coverage

http://fbrnc.net/blog/2016/03/continuous-load-testing

Continuous Integration and Delivery Tools

Automated build, deploy, test and release.

CI/CD Tools that work well with Magento• Amazon CodePipeline • Bamboo• Jenkins • TravisCI• And many others…

Continuous Testing

* Testing is a broad subject, deserving of more attention.

Continuous Integration tools like bamboo provides hooks to allow you to test using several different methods.

Magento 2: Testing is built in and easy to tie into your CI/CD

Magento 1: More difficult, it is possible with some effort

Continuous Monitoring

Nagios:

Continuous Stats

See metrics from all of your apps, tools & services in one place using Datadog.

This can be tied to a variety of inputs and outputs:• Newrelic• AWS Cloudwatch• Slack• VictorOps / Pagerduty• And Many More

Metrics

Logs

With the constant fragmentation of the infrastructure you end up with some pretty incredible logging requirements.

Data sources of interest:• Infrastructure metrics (CPU, Network IO)• Business metrics (Sign ups, logins, placed orders)• Events (deployments, auto-scaling activity)• Test metrics (response times, error rates)

Creating a Devops BlueprintA brief look into setting up Devops.

“The Devops Plan”

Define, Document & Empower Devops Culture1. Identify and Document Processes2. Find people within the Roles3. Focus on building the foundation for feedback loop

Start Automating4. Application build process5. Development environment builds6. UAT environments7. Production environments

Build in monitoring, metrics and create feedback loop.

Small Simple Building Blocks

• Start with the git workflow• Application build process• Testing and peer review• Release build process• Automate production deployments• Use operations and monitoring to

plan the next release

Keep it simple.

With so many choices for all of the technology, its important to focus on small solutions that fit within your existing infrastructure that are easy to implement and maintain.

Questions & Answers

Thanks!

• My Family• Robofirm• Mage Titans• The Magento Community• Fabrizio Branca

The people who have contributed to the meta-culture that inspired this talk.

Follow me on twitter:

@aepod

See my blog at http://aepod.com/ for a more detailed list of sources and inspiration for this talk.