OpenStack Freezer -101Trinath Somanchi

Session Outline• Why backups?

• OpenStack DRaaS – Freezer

• Components

• Architecture

• Backup types

• Storage types

• Data restore

• Freezer – Simplified workflow

• Agent – Backup Workflow

• Scheduler – Workflow

• Session JOB and JSON formats

Why Backups?

• Backups are needed to protect information from being lost. The causes for information loss can be : data deletion or corruption, theft, hacking, fire , etc ...

• In case of an incident, a backup needs to be restored quickly which means minimizing system downtime.

• Lack of a proper backup and restore system means long system downtime, which results in significant $$$ loss.

• Application bugs, Power failures, legal requirements and compliance.

OpenStack DRaaS – Freezer

Freezer is a Backup Restore DR as a Service platform that helps you to automate the data backup and restore process.

The following features are available:

▪ Backup file system using point-in-time snapshot▪ Strong encryption supported: AES-256-CFB▪ Backup file system tree directly (without volume snapshot)▪ Backup MySQL with lvm snapshot▪ Restore data from a specific date automatically to file system▪ Low storage consumption as the backup are uploaded as a stream▪ Flexible backup policy (incremental and differential)▪ Data is archived in GNU Tar format for file based incremental▪ Multiple compression algorithm support (zlib, bzip2, xz)▪ Remove old backup automatically according to the provided parameters▪ Multiple storage media support (Swift, local file system, or ssh)▪ Flush kernel buffered memory to disk▪ Multi-platform (Linux, Windows, *BSD, OSX)▪ Manage multiple jobs (I.e., multiple backups on the same node)▪ Synchronize backups and restore on multiple nodes▪ Web user interface integrated with OpenStack Horizon▪ Execute scripts/commands before or after a job execution

Freezer uses GNU Tar or Rsync algorithm under the hood to execute incremental backup and restore. When a key is provided, it uses OpenSSL or pycrypto module (OpenSSL compatible) to encrypt data. (AES-256-CFB).

Freezer - Components

Web GUI

• Web interface that interacts with the Freezer API to configure and change settings

• Provides most of the features from the Freezer Agent CLI, advanced scheduler settings such as multi-node backup synchronization, metrics, and reporting.

Scheduler – Client side component

• Client side component, running on the node where the data backup is to be executed.

• Consists of a daemon that retrieves the data from the freezer API and executes jobs (i.e. backups, restore, admin actions, info actions, pre and/or post job scripts) by running the Freezer Agent.

• Manages the execution and synchronization of multiple jobs executed on a single or multiple nodes.

Agent – Client side component

• Multiprocessing Python software that runs on the client side, where the data backup is to be executed.

• It can be executed standalone or by the Freezer Scheduler.

• The Freezer Agent provides a flexible way to execute backup, restore and other actions on a running system.

API

• Used to store and provide metadata to the Freezer Web UI and to the Freezer Scheduler.

• Used to store session information for multi node backup synchronization.

DB Elasticsearch

• Backend used by the API to store and retrieve metrics, metadata sessions information, job status, etc.

Freezer - Architecture

Backup typesDirect file system

•$> sudo freezer-agent --path-to-backup /data/dir/to/backup --container freezer_new-data-backup --backup-name my-backup-name

The command will generate a compressed tar gzip file of the directory /data/dir/to/backup. The generated file will be segmented in stream and uploaded in the Swift container called freezer_new-data-backup, with backup name my-backup-name.

•$> sudo freezer-agent --lvm-srcvol /dev/jenkins/jenkins-home --lvm-dirmount /var/snapshot-backup --lvm-volgroupjenkins --path-to-backup /var/snapshot-backup --container freezer_jenkins-backup-prod --exclude "\*.lock" --mode fs --backup-name jenkins-ops2

Cinder Backup•To make a cinder backup you should provide cinder-vol-id or cindernative-vol-id parameters in command line arguments. Freezer doesn't do any additional checks and assumes that making a backup of that image will be sufficient to restore your data in the future.

•Execute a cinder backup:

$ freezer-agent --mode cinder --cinder-vol-id 3ad7a62f-217a-48cd-a861-43ec0a04a78b

•Execute a MySQL backup with Cinder:

$ freezer-agent --mysql-conf /root/.freezer/freezer-mysql.conf --container freezer_mysql-backup-prod --mode mysql --backup-name mysql-ops002 --cinder-vol-id 3ad7a62f-217a-48cd-a861-43ec0a04a78b

Nova Backup• To make a Nova backup you should provide a Nova parameter in the arguments. Freezer doesn't do any additional checks and assumes that making a backup of that instance will be sufficient to restore your data in future.

•Execute a nova backup:

$ freezer-agent --mode nova --nova-inst-id 3ad7a62f-217a-48cd-a861-43ec0a04a78b

•Execute a MySQL backup with Nova:

$ freezer-agent --mysql-conf /root/.freezer/freezer-mysql.conf

--container freezer_mysql-backup-prod --mode mysql

--backup-name mysql-ops002

--nova-inst-id 3ad7a62f-217a-48cd-a861-43ec0a04a78b

Storage typesOpenStack - SwiftTo use swift storage specify "--storage swift" or omit "--storage" parameter altogether (Swift storage is the default). And use "--container <swift-container-name>"

Backup example:

$ sudo freezer-agent --path-to-backup /data/dir/to/backup --container freezer-container --backup-name my-backup-name --storage swift

Restore example:

$ sudo freezer-agent --action restore --restore-abs-path /data/dir/to/backup

--container freezer-container --backup-name my-backup-name --storage swift

Local StorageTo use local storage specify "--storage local" And use "--container <path-to-folder-with-backups>"

Backup example:

$ sudo freezer-agent --path-to-backup /data/dir/to/backup --container /tmp/my_backup_path/ --backup-name my-backup-name --storage local

Restore example:

$ sudo freezer-agent --action restore --restore-abs-path /data/dir/to/backup

--container /tmp/my_backup_path/ --backup-name my-backup-name

--storage local

SSH StorageTo use ssh storage specify "--storage ssh" And use "--container <path-to-folder-with-backups-on-remote-machine>" Also you should specify ssh-username, ssh-key and ssh-host parameters. ssh-port is optional parameter, default is 22.

Backup example:

$ sudo freezer-agent --path-to-backup /data/dir/to/backup --container /remote-machine-path/ --backup-name my-backup-name --storage ssh --ssh-username ubuntu --ssh-key ~/.ssh/id_rsa --ssh-host 8.8.8.8

Restore example:

$ sudo freezer-agent --action restore --restore-abs-pat /data/dir/to/backup --container /remote-machine-path/ --backup-name my-backup-name --storage ssh --ssh-username ubuntu --ssh-key ~/.ssh/id_rsa --ssh-host 8.8.8.8

Data RestoreAs a general rule, when you execute a restore, the application that writes or reads data should be stopped so that during the restore operation, the restored data is not inadvertently read or written by the application.

Local Storage Restore:

Cinder Restore:Cinder restore currently creates a volume with the contents of the saved one, but doesn't implement detachment of existing volume and attachment of the new one to the vm. You should implement these steps manually. To create a new volume from existing content run the next command:

Execute a cinder restore:$ freezer-agent --action restore --cinder-vol-id 3ad7a62f-217a-48cd-a861-43ec0a04a78b$ freezer-agent --action restore --cindernative-vol-id 3ad7a62f-217a-48cd-a861-43ec0a04a78b

sudo freezer-agent --action restore --container /local_backup_storage/ --backup-name adminui.git --hostname git-HP-DL380-host-001 --restore-abs-path /home/git/repositories/adminui.git/ --restore-from-date "2014-05-23T23:23:23" --storage local

Nova Restore:Nova restore currently creates an instance with the content of saved one, but the ip address of the vm will be different as well as its id.Execute a nova restore:

$ freezer-agent --action restore --nova-inst-id 3ad7a62f-217a-48cd-a861-43ec0a04a78b

Freezer – Simplified workflow

Freezer Agent backup work flow

Freezer Scheduler work flow

Freezer Job Session

Freezer Job JSON

cat test_job.json

{"job_actions": [

{"freezer_action": {

"action": "backup","mode": "fs","backup_name": "backup1","path_to_backup": "/home/me/datadir","container": "schedule_backups","log_file": "/home/me/.freezer/freezer.log"

},"max_retries": 3,"max_retries_interval": 60

}],"job_schedule": {

"schedule_interval": "4 hours","schedule_start_date": "2015-08-16T17:58:00"

},"description": "schedule_backups 6"

}

References

• OpenStack Backup, Restore, DR

(Freezer) -

https://www.slideshare.net/saad_asd49/o

penstack-backup-restore-dr-freezer

• Freezer API -

https://github.com/openstack/freezer-api

• Freezer - Vietnam OpenStack Technical

Meetup #12

https://www.slideshare.net/vietstack/free

zer-vietnam-openstack-technical-

meetup-12

• Freezer API JSON samples

https://github.com/openstack/freezer-

api/tree/master/api-

ref/source/v1/samples

• Freezer API documentation

https://github.com/openstack/freezer-

api/blob/master/README.rst

Thank you !