oracle exadata recipes - gbv.de .contents chapter6: preparingfor exadata 139 6-1. planning...

Oracle Exadata Recipes

A Problem-Solution Approach

John Clarke

Apress-

Contents

About the Author xxxiii

About the Technical Reviewer xxxv

Acknowledgments xxxvii

Introduction xxxix

Part 1: Exadata Architecture 1

Chapter 1: Exadata Hardware 3

1-1. Identifying Exadata Database Machine Components 3

Problem 3

Solution 3

How It Works 7

1-2. Displaying Storage Server Architecture Details 10

Problem 10

Solution 10

How It Works 12

1-3. Displaying Compute Server Architecture Details 14

Problem 14

Solution 14

How It Works 16

1-4. Listing Disk Storage Details on the Exadata Storage Servers 17

Problem 17

Solution 17

How It Works 21

vii

CONTENTS

1-5. Listing Disk Storage Details on the Compute Servers 23

Problem 23

Solution 23

How It Works 27

1-6. Listing Flash Storage on the Exadata Storage Servers 27

Problem 27

Solution 27

How It Works 29

1-7. Gathering Configuration Information for the InfiniBandSwitches 29

Problem 29

Solution 29

How It Works 32

Chapter 2: Exadata Software 33

2-1. Understanding the Role of Exadata Storage Server Software 33

Problem 33

Solution 33

How It Works 35

2-2. Validating Oracle 11gR2 Databases on Exadata 35

Problem 35

Solution 35

How It Works 38

2-3. Validating Oracle 11gR2 Grid Infrastructure on Exadata 38

Problem 38

Solution 38

How It Works 41

2-4. Locating the Oracle Cluster Registry and Voting Disks on Exadata 42

Problem 42

Solution 42

How It Works 43

viii

CONTENTS

2-5. Validating Oracle 11gR2 Real Application Clusters Installation and Database

Storage on Exadata 44

Problem 44

Solution 44

How It Works 47

2-6. Validating Oracle 11gR2 Real Application Clusters Networking on Exadata 47

Problem 47

Solution 47

How It Works 52

a Chapter 3: How Oracle Works on Exadata 53

3-1. Mapping Physical Disks, LUNs, and Cell Disks on the Storage Servers 53

Problem 53

Solution 53

How It Works 56

3-2. Mapping ASM Disks, Grid Disks, and Cell Disks 58

Problem 58

Solution 58

How It Works 60

3-3. Mapping Flash Disks to Smart Flash Storage 61

Problem 61

Solution 61

How It Works 62

3-4. Identifying Cell Server Software Processes 62

Problem 62

Solution 63

How It Works 63

3-5. Tracing Oracle I/O Requests on Exadata Compute Nodes 65

Problem 65

Solution 65

How It Works 65

ix

CONTENTS

3-6. Validating That Your Oracle RAC Interconnect Is Using InfiniBand 68

Problem 68

Solution 68

How It Works 70

3-7. Tracing cellsrv on the Storage Servers 71

Problem 71

Solution 71

How It Works 73

Part 2: Preparing for Exadata 75

Chapter 4: Workload Qualification 77

4-1. Quantifying I/O Characteristics of Your Current Database 77

Problem 77

Solution 77

How It Works 80

4-2. Conducting a Smart Scan Fit Analysis Using AWR 81

Problem 81

Solution 81

How It Works 85

4-3. Conducting a Smart Scan Fit Analysis Using Exadata Simulation 87

Problem 87

Solution 87

How It Works 91

4-4. Performing a Hybrid Columnar Compression Fit Assessment 93

Problem 93

Solution 93

How It Works 95

Chapter 5: Sizing Exadata 97

5-1. Determining CPU Requirements 98

Problem 98

Solution 98

How It Works 107

x

CONTENTS

5-2. Determining lOPs Requirements 108

Problem 108

Solution 108

How It Works 112

5-3. Determining I/O Bandwidth Requirements 113

Problem 113

Solution 113

How It Works 118

5-4. Determining ASM Redundancy Requirements 119

Problem 119

Solution 119

How It Works 119

5-5. Forecasting Storage Capacity 122

Problem 122

Solution 122

How It Works 124

5-6. Planning for Database Growth 125

Problem 125

Solution 125

How It Works 133

5-7. Planning for Disaster Recovery 133

Problem 133

Solution 133

How It Works 133

5-8. Planning for Backups 134

Problem 134

Solution 134

How It Works 135

5-9. Determining Your Fast Recovery Area and RECO Disk Group Size Requirements 136

Problem 136

Solution 136

How It Works 138

xi

CONTENTS

Chapter 6: Preparing for Exadata 139

6-1. Planning and Understanding Exadata Networking 139

Problem 139

Solution 139

How It Works 141

6-2. Configuring DNS 143

Problem 143

Solution 143

How It Works 144

6-3. Running checkip.sh 144

Problem 144

Solution 144

How It Works 147

6-4. Customizing Your InfiniBand Network Configuration 147

Problem 147

Solution 147

How It Works 147

6-5. Determining Your DATA and RECO Storage Requirements 148

Problem 148

Solution 148

How It Works 149

6-6. Planning for ASM Disk Group Redundancy 149

Problem 149

Solution 149

How It Works 152

6-7. Planning Database and ASM Extent Sizes 152

Problem 152

Solution 152

How It Works 152

xii

CONTENTS

6-8. Completing the Pre-Delivery Survey 153

Problem 153

Solution 153

How It Works 154

6-9. Completing the Configuration Worksheet 154

Problem 154

Solution 154

How It Works 156

Part 3: Exadata Administration 157

Chapter 7: Administration and Diagnostics Utilities 159

7-1. Logging in to the Exadata Compute and Storage Cells Using SSH 159

Problem 159

Solution 159

How It Works 160

7-2. Configuring SSH Equivalency 161

Problem 161

Solution 161

How It Works 162

7-3. Locating Key Configuration Files and Directories on the Cell Servers 162

Problem 162

Solution 162

How It Works 163

7-4. Locating Key Configuration Files and Directories on the Compute Nodes 164

Problem 164

Solution 164

How It Works 164

7-5. Starting and Stopping Cell Server Processes 165

Problem 165

Solution 165

How It Works 166

xiii

CONTENTS

7-6. Administering Storage Cells Using CellCLI 167

Problem 167

Solution 167

How It Works 168

7-7. Administering Storage Cells Using deli 168

Problem 168

Solution 168

How It Works 170

7-8. Generating Diagnostics from the ILOM Interface 171

Problem 171

Solution 171

How It Works 173

7-9. Performing an Exadata Health Check Using exachk 173

Problem 173

Solution 173

How It Works 176

7-10. Collecting Compute and Cell Server Diagnostics Using the sundiag.sh Utility 176

Problem 176

Solution 176

How It Works 177

7-11. Collecting RAID Storage Information Using the MegaCLI utility 178

Problem 178

Solution 178

How It Works 179

7-12. Administering the Storage Cell Network Using ipconf 179

Problem 179

Solution 179

How It Works 180

7-13. Validating Your InfiniBand Switches with the CheckSWProfile.sh Utility 181

Problem 181

Solution 181

How It Works 182

xiv

CONTENTS

7-14. Verifying Your InfiniBand Network Topology 182

Problem 182

Solution 182

How It Works 183

7-15. Diagnosing Your InfiniBand Network 183

Problem 183

Solution 183

How It Works 187

7-16. Connecting to Your Cisco Catalyst 4948 Switch and Changing Switch Configuration 187

Problem 187

Solution 187

How It Works 188

Chapter 8: Backup and Recovery 189

8-1. Backing Up the Storage Servers 189

Problem 189

Solution 189

How It Works 190

8-2. Displaying the Contents of Your CELLBOOT USB Flash Drive 191

Problem 191

Solution 191

How It Works 192

8-3. Creating a Cell Boot Image on an External USB Drive 192

Problem 192

Solution 193

How It Works 194

8-4. Backing Up Your Compute Nodes Using Your Enterprise Backup Software 195

Problem 195

Solution 195

How It Works 196

XV

CONTENTS

8-5. Backing Up the Compute Servers Using LVM Snapshots 196

Problem 196

Solution 197

How It Works 200

8-6. Backing Up Your Oracle Databases with RMAN 201

Problem 201

Solution 201

How It Works 201

8-7. Backing Up the InfiniBand Switches 202

Problem 202

Solution 203

How It Works 204

8-8. Recovering Storage Cells from Loss of a Single Disk 205

Problem 205

Solution 205

How It Works 208

8-9. Recovering Storage Cells from Loss of a System Volume Using CELLBOOT Rescue 210

Problem 210

Solution 210

How It Works 221

8-10. Recovering from a Failed Storage Server Patch 222

Problem 222

Solution 222

How It Works 223

8-11. Recovering Compute Server Using LVM Snapshots 223

Problem 223

Solution 223

How It Works 228

8-12. Reimaging a Compute Node 229

Problem 229

Solution 229

How It Works 231

xvi

CONTENTS

8-13. Recovering Your InfiniBand Switch Configuration 232

Problem 232

Solution 232

How It Works 234

8-14. Recovering from Loss of Your Oracle Cluster Registry and Voting Disks 235

Problem 235

Solution 235

How It Works 237

Chapter 9: Storage Administration 239

9-1. Building ASM Disk