a close-up look at potential future enhancements in tivoli...

Tivoli Storage, IBM Software Group

© 2005 IBM Corporation

A Close-up Look at Potential Future Enhancements in Tivoli Storage Manager

Dave CannonTivoli Storage Manager ArchitectOxford University TSM SymposiumSeptember 2005


© 2005 IBM Corporation2 Potential Future Enhancements in Tivoli Storage Manager

Disclaimer

This presentation describes potential future enhancements to IBM Tivoli Storage Manager

Information in this presentation does not constitute a commitment to deliver the described enhancements or to do so in a particular timeframe

IBM reserves the right to change product plans, features, and delivery schedules according to business needs and requirements

This presentation uses the following designations regarding availability of potential product enhancements– Planned: GA is currently planned for 2005, but subject to change– Candidate: Will be a candidate for possible delivery in 2006 - 2007



Agenda

Management of data in server storage

Optimizing restore of client data

Backup set enhancements

TSM database

Snapshot exploitation

IBM TotalStorage N series (Network Appliance) integration



Simultaneous Migration and Storage Pool Backup

Can reduce client backup window as compared to simultaneous write to primary and copy storage pools during backup operation

Combines windows for migration and storage pool backup

Periodic storage pool backup required to ensure all data copied

Copy pool 2

Copy pool 1

Client

2. Storage pool backup

2. Storage pool backup

Server

2. Migration1. Backup

Data flow

Candidate



LAN-free Simultaneous Write to Copy Pools

LAN-free, simultaneous write to primary pool and copy pool(s)

Reduces LAN traffic as compared with LAN-based simultaneous write

Duplicates data while reducing time for storage pool backup

Periodic storage pool backup required to ensure all data copied

Copy pool 2

Primary pool

Client/Storage Agent

Server

Copy pool 1

LAN

SAN

Control

Data flow

Candidate



Management of Redundant FilesCommon files (operating system, application) are not duplicated in the TSM storage hierarchy for each node

Also known as single instance store

Client can transparently restore unique and common files

Benefits– Reduced bandwidth and/or time required for backup– Reduced space requirements in storage pools– Especially attractive for disk-only TSM configurations

Client 1 filesystem

File XClient 2 filesystem

Client 3 filesystem

File X

TSM Server

File X

File X

Only one instance of File X is stored on server, but each

node shows backup of this file

Candidate



Management of Redundant Files: Model NodeEach node can optionally be assigned to a model node with same operating system and preferably common application/project files (e.g., one model node per project)If file has already been stored by model node, same file is not resent by other nodes but unique database entry is created for each instanceAlthough multiple nodes share a common file, each node – Can have unique attributes (policy, insertion date, security) for the common file– Can query file information for its own objects, whether stored uniquely or in common

Efficient checking for previously stored common files because only the assigned model node is checkedData for model node can be stored on disk or collocated tape for faster restores (avoids need for mounting many tapes)

Node Name

MODEL_NODE

SMITH

JONES

JOHNSON

Cataloged Files for Node

Solid boxes indicate files stored in storage hierarchy by this node

Striped boxes indicate files which require database entries only (file is stored by model node)



MOVE NODEDATA for Off-site Volumes

DB

2. Copy files from on-site location to new

copy pool volumes

3. Update database to reference new location of files

TSM server location1. Use database to identify files for

designated nodes on off-site volumesOff-site volumes

Offsite location

4. Transporttapes off-site

5. Transporttapes on-site

Allows consolidation of data on off-site volumes for selected nodes

Supports specification of following options for data to be moved– On-site data only– Off-site data only– All data Candidate



Agenda




TSM database





Collocation of Active Data

Active-only storage pool allows collocation of active backup versions for fast restore – Active backup versions still reside on client machine (have not been modified or deleted)– Active-only pools will be sequential-access only (typically sequential-access disk)

Possible benefits- Optimized access to active versions - Reduced size of disk pools if only active versions are stored- Reduced data movement in preparation for restore of active data

Challenge is to collocate active data while also preserving performance benefits of file aggregation and correspondence to data in copy storage pools

Active data in disk pool for fast restore

Active and inactive data for point-in-time restore

Storage Hierarchy

Candidate



Collocation of Active Data: Example

Client

Active-only pool

A0 B0 C0 D0

A0 B0 C0 D0

A0 B0 C0 D0

1. Client backs up A0, B0, C0, D0 to active-only pool. Then these files are migrated to next pool.

Server

Active/inactive pool

Migration

B1A0 B0 C0 D0 E1

Client

Active-only pool

Server

Active/inactive pool

Migration

B1 E1

B1A0 B0 C0 D0 E1

B1A0 B0 C0 D0 E1

2. Client backs up B1, E1, which deactivates B0. Then B1, E1 are migrated to next pool.

B1A0 B0 C0 D0 E1

Client

Active-only pool

Server

Active/inactive pool B1A0 B0 C0 D0 E1

B1A0 C0 D0 E1

3. Reclamation removes inactive B0 from active-only pool.

B1A0 B0 C0 D0 E1

Client

Active-only pool

Server

Active/inactive pool B1A0 B0 C0 D0 E1

B1A0 C0 D0 E1

4. Client restores active files A0, C0, D0, B1, and E1 from active-only pool.

A0 C0 D0 B1 E1



Transfer from Active/Inactive to Active-Only Pool

Copy DataCopies active versions from active/inactive primary pool to active-only primary poolTransfer is performed incrementallyCan be restricted by node and file spaceAllows population of active-only pool with previously stored data

Restore to Active-Only PoolRestores active files – From copy pool containing both active

and inactive data – To active-only primary pool

Allows recovery of active-only primary pool from active/inactive copy pool



Copy data

Storage Hierarchy

Active/inactive copy pool

Restore

Candidate



Reclamation of active-only copy pool recovers space occupied by inactive and deleted files

Active-Only Copy Storage Pools

Reduced storage requirement for copy poolSimplified tape management because fewer tapesImproved recovery performance for active data

Incremental backup from active-only primary pool to active-only copy pool



Storage Hierarchy

Active-only copy pool

Restore

Backup

Reclamation

Inactive filesExpired files

Candidate



Restore of Individual Files from File System Image

Server

1. Client creates image and collects file metadata

2. Image and metadata sent to server

3. Server stores image and table of contents (TOC) in

storage hierarchyMetadata

Image

Image

DBTOC

ClientBackup

Server1. Client specifies images

to examine

3. Client queries file information and user

selects files to restore

2. Server loads TOC(s) into TSM database

Image

DBTOC

Client

4. Selected files restored

Restore

TOC

Candidate



Agenda




TSM database





Generation of Backup Sets to Point in Time

Allows generation of backup set to specified point in time– Files must not already be expired or deleted from the server– Default is date/time of backup set generation

Benefits– Increases operational flexibility because backup sets need not be generated

before next client backup– Allows retroactive generation of “synthetic full” backup if unanticipated needs

arise

TSM Server

GENERATE BACKUPSET nodename …PITDATE=mm/dd/yyyy PITTIME=hh:mm:ss

Storage Hierarchy

Backup Set

Candidate



File Selection During Backup Set RestoreServer1. Client specifies

backup sets

3. Client queries file information and user

selects files to restore

Client Server

2. Server locates backup set TOCs and loads into database

DB

TOC TOC

Backup Set

4. Server restores data to client

4. Server restores data to client

Server storage pool has table of contents (TOC) with backup set contents

On demand, server loads TOC into TSM database

Client queries TOC and displays directory tree structure and files

User expands directories and selects files for restore

Simplifies task of restoring selected directories/files from a backup setCandidate



Generation of Backup Sets for List of Nodes

Allows generation of multiple backup sets with a single command– A distinct backup set is generated for each node– Backup sets generated with a single command are written sequentially to the

same media

Improves media utilization

Physical security may be important for backup set restore at the client (without the TSM server)

GENERATE BACKUPSET bob,fred,sue

Distinct backup sets fornodes BOB, FRED, and SUE

TSM ServerStorage

Hierarchy

Candidate



Differential Backup SetsSupports full and differential backups sets– Full includes all files matching the backup set specification– Differential includes all matching files that were backed up from the client since

the last full backup set was generated

Restore requires base and differential backup sets

Benefits– Reduces media requirement – Faster generation of backup sets

Full Backup Set

Time

Differential 2Differential 1

Candidate



Backup Set Support for Application Data

Expand backup set support to include Data Protection products

Allows Data Protection client to restore from backup set via the TSM server

Candidate applications include– Microsoft Exchange– Microsoft SQL Server– Lotus Domino

Candidate



Possible Uses for Enhanced Backup Sets

Offsite vaulting of self-describing data

Traditional tape rotation (e.g., grandfather/father/son) for SMB or departmental environments

Long-term retention of point-in-time data– Individual files in backup set do not require entries in TSM database because

entire backup set is tracked as a single entity– Separate policies for retention of backup set and individual files stored by TSM – May be useful in environments with stringent regulatory requirements for data

retention



Agenda




TSM database





DB2 Database Engine for TSM

Conversion of existing TSM database

ConversionProprietary Database

DB2 Database

TSM Server

Before conversion

After conversion

TSM server unavailable during conversion

Installation of new TSM server

DB2 Database

TSM Server

DB2 included in install package

Candidate



Why a Proprietary TSM Database Today?

Portability to diverse server platforms- Proprietary TSM database portable to any server platform

- In early days of ADSM, DB2 had only limited platform support

TSM database optimized for storage-management operations- Database schema

- Customized, multilevel locking

- ARIES technology reduces overhead for logging and recovery

Bit vectors stored in database for tracking block allocation in random-access storage



Potential Benefits of DB2 as Database Engine

Scalability – Database – Recovery log

Database maintenance– Online database reorganization– Referential-integrity checking to eliminate or minimize need for database audit

Monitoring - Complete SQL function including UNION, INTERSECT, and correlated sub-

queries not available with proprietary database- Ability to optimize SQL query performance by defining secondary indices

Potential ability to support alternate configurations- Distributed databases- Enhanced LAN-free and library sharing



Design Goals for DB2 DatabaseNo extra expense for database product license

TSM administration should not become more complex– Preserve existing administrative interfaces to the extent possible– Provide virtualized tables equivalent to those available via SQL today

Automate conversion of existing database entries to DB2 with minimal downtime of the TSM server– Conversion time may be comparable to that for database unload/load– Test using large customer databases

Performance should be similar to that of current proprietary database engine

Support backup and recovery of the TSM database

Continue to support random-access disk storage pools with equivalent of bit-vector implementation in DB2 engine



Agenda




TSM database





Current Snapshot Support in TSM

Windows backup-archive client– LVSA for open-file support and image backup from TSM Windows client

– Snapshot image backup from Linux x86 / Linux IA64 client

– Microsoft Volume Shadow Copy Service (VSS) support for backup of Windows 2003 system state

TSM for Hardware (exploitation of snapshot for backup/restore)– ESS: Backup/restore of DB2, Oracle, and mySap data on AIX

– DS6000/DS8000: Oracle and mySAP (with Oracle or DB2) on AIX (Available 9/2005)

– SVC: Oracle and mySAP (with Oracle or DB2) on AIX (Available 9/2005)



Snapshot Exploitation Topology

TSM Server

Storage Hierarchy

ApplicationData

Local Snapshot Versions

Policy-based snapshot

management

Application System

Backup System

Backup to TSM Server

Restore can be performed fromLocal snapshot versionsTSM storage hierarchy

Snapshot Creation



Benefits of Planned TSM Snapshot Support

Support for multiple, persistent snapshots – Persistent snapshots retained locally after backup to TSM

– Allows extremely fast restore directly from the snapshot using “snapback” to the original location

Backup of snapshots to TSM server – Transfer is outboard of application server to minimize impact to application (if

snapshot provider allows access by another machine)

– Copies on TSM server provide long-term retention and disaster recovery

Policy-based management of local, persistent snapshots– Retention policies may be different for local snapshots and copies on TSM server

– Automatic reuse of local snapshot storage as older snapshot versions expire



Potential Snapshot Exploitation

Additional snapshot providers– DS6000/DS8000: DB2 on AIX (Planned GA 4Q2005)– SVC: DB2 on AIX (Planned GA 4Q2005)

Federated backup/restore of multi-partition DB2 on AIX (Planned GA 4Q2005)

Microsoft Volume Shadow Copy Service (VSS) exploitation– Data Protection for Exchange (Planned GA 4Q2005)– Data Protection for SQL Server (Candidate)

File system snapshot exploitation– Snapshot backup/restore on AIX JFS2 (Candidate)– Snapshot backup/restore on SAN FS (Candidate)



Agenda




TSM database





IBM-NetApp Agreement

IBM to OEM Network Appliance (NetApp) hardware offerings (as IBM TotalStorage N series)

IBM will have the right to rebrand/resell NetApp software

Promotes integration of TSM with NetApp products– TSM will be a preferred backup/recovery solution for NetApp products– System Storage Archive Manager (formerly TSM for Data Retention) will be a

preferred solution in environments with stringent requirements for retention management

– Current TSM-NetApp integration to be extended for additional customer value

NetApp to position IBM as preferred supplier of tape solutions



Planned N Series (NetApp) Integration

Enable TSM archive client for System Storage Archive Manager

VSS support for NetApp SnapDrive

Improved support for backup/archive using CIFS/NFS

NDMP V4 support

Disaster recovery / backend data movement for NDMP-generated data

NDMP 3-way configuration

NDMP filer-to-server configuration



System Storage Archive Manager

System Storage Archive Manager formerly known as TSM for Data RetentionPreviously, archive/retrieve could be performed only via the TSM APITSM command-line and GUI client will be able to – Archive to and retrieve from Archive Manager– Activate event-based retention for specific files– Request deletion hold/release for specific files

Storage Hierarchy

Software ApplicationTSM API

Archive Client


SnapLockStorage

(optional)


IBMDS4000

TotalStorage DR550

Planned GA 3Q2005



VSS Support for NetApp SnapDrive

Fast, low-impact backup of Exchange / SQL servers

Multiple versions of local, persistent snapshots

Offloaded backup to TSM server

N series/NetApp (iSCSI/FCP) device

NetApp SnapDrivestorage virtualization software

(VSS provider)

Exchange / SQL Server

(VSS writer)

Microsoft Volume Shadow Copy Service (VSS)

DP for Exchange

(VSS requester)

TSM Server

Exchange: Planned GA 4Q2005SQL Server: Candidate

Windows Host Machine



Backup/Archive of NetApp Using CIFS/NFS Today

NAS File Server

NAS file system

CIFS/NFSAccess TSM

ClientTSM

Server

Storage hierarchy

CIFS/NFS allows TSM client to access data on N series/NetApp devices, which do not support third-party backup software

BenefitsFile-level granularityProgressive incremental backupTSM storage hierarchy

LimitationsFile-access overheadFile attributes may not be preservedQ-trees not preservedData flow through TSM client/serverData flow over the network

Could be in same box



Improved CIFS/NFS Support

Preserve file attributes across backup/restore or archive/retrieve round tripHomogenous environments only – Windows data with Windows backup-archive client– UNIX/Linux data with UNIX/Linux backup-archive client

NetApp

Windows Data

UNIX/Linux Data

WindowsClient

UNIX/LinuxClient

TSM Server

CIFS access

NFS access

Planned GA 1Q2006



NDMP: Disaster Recovery / Backend Movement

Use NDMP tape-to-tape copy for backend movement of data stored on TSM server via NDMPSupported operations – Storage pool backup– Storage pool or volume restore– Move Data

• Intra-pool for space recovery• Inter-pool for migration to new device type

DRM support for NDMP dataRestore Node will use copy pool, if primary data not accessible

TSM Server

NDMP control (TCP/IP)

Data flow (SCSI/FC)

NAS Device Tape Library

Read

Write

Candidate



NDMP: 3-Way and Filer-to-Server Configurations

TSM Server


Data(TCP/IP)

Data

RoboticsControl

(TCP/IP)

RoboticsControl

(SCSI/FC)

Tape libraryTarget device could be integrated within

tape library

NAS file system

NAS Device Target Device

Data

Allows centralization of tape resourceExploits libraries with NDMP server

Data


Roboticscontrol

Tape libraryData(TCP/IP)

Data

NAS file system

TSM Server

NAS Device

Allows centralization of tape resourceFull capability of TSM storage hierarchy

Candidate



Summary

This presentation has described a sampling of potential future enhancements to Tivoli Storage Manager

Themes have included– Efficient management of data in server storage

– Optimized restore of client data

– Enhanced backup sets for off-site vaulting and long-term retention of point-in-time data

– Improved scalability, maintenance, and monitoring of the TSM database

– Exploitation of snapshots to minimize impact of backups and achieve faster restore

– Extended integration with N series / Network Appliance devices

a close-up look at potential future enhancements in tivoli...

Documents