interstage shunsaku data manager operator's guide (advanced) · 2005-07-21 · preface purpose...

Interstage Shunsaku Data Manager Operator's Guide (Advanced)

Operator’s Guide (Advanced)

Trademarks Trademarks of other companies are used in this manual only to identify particular products or systems.

Product Trademark/Registered Trademark

Microsoft, Visual C++, Windows, and Windows Server

Registered trademark or trademarks of Microsoft Corporation in the United States and other countries.

Solaris Solaris and all other trademarks and logos related to Solaris are registered trademarks or trademarks of Sun Microsystems Inc. U.S.A. in the United States and other countries, and are used under license from the said company.

Linux Registered trademark or trademark of Linus Torvalds in the United States and other countries.

Java Java and other trademarks comprising Java are trademarks of Sun Microsystems Inc. U.S.A. in the United States and other countries.

Red Hat, RPM, and all trademarks and logos based on Red Hat

Trademarks or registered trademarks of Red Hat Inc. in the United States and other countries.

Interstage Interstage is a registered trademark of Fujitsu Limited.

Fujitsu documentation may contain specific technologies that are subject to the Foreign Exchange and Foreign Trade Control Law. When such specific technology is described in a document, and that document is either exported or provided to a non-resident, permission must be sought under the above law.

Fujitsu Limited

First Edition (May 2005) The contents of this manual may be revised without prior notice.

All Rights Reserved, Copyright © FUJITSU Limited 2005

ii

Preface

Purpose of This Manual This manual explains the functions provided by Interstage Shunsaku Data Manager (hereafter abbreviated as Shunsaku) to construct highly reliable large-scale systems. It also describes how to perform environment setup and operation.

Target Audience This manual is aimed at the following readers:

• Persons who construct and operate highly reliable large-scale Shunsaku systems

Requisite Knowledge This manual assumes that the reader has an understanding of the following topics:

• XML

• C language

• Java

• Internet basics

• Cluster systems

• Windows

• Linux

• Solaris OE

iii

Preface

Organization of This Manual The organization of this manual is as follows:

• Chapter 1 - Overview

This chapter provides an overview of the features and functions of Shunsaku.

• Chapter 2 - Design

This chapter explains the system configuration, file organization, and design procedures used with Shunsaku.

• Chapter 3 - Operation

This chapter explains the overall operation of Shunsaku.

• Chapter 4 - Transaction Function

This chapter provides an overview of the transaction function and explains its usage.

• Chapter 5 - Shunsaku File

This chapter provides an overview of the Shunsaku File function and explains its usage.

• Chapter 6 - HA Functions

This chapter provides an overview of the HA (High Availability) function and explains director server failover and search server failover.

• Chapter 7 - Backup and Recovery

This chapter provides an overview of the backup and recovery functions and explains their usage.

• Chapter 8 - Maintenance

This chapter explains the monitoring processes that occur while Shunsaku is running and what to do when an error occurs.

• Appendix A - Execution Parameters in Application Environment Files

Appendix A explains the application environment files that are used to set the execution parameters of Shunsaku.

• Appendix B - Allowable Values

Appendix B lists the allowable values that apply to Shunsaku.

• Appendix C - Estimating Resources

Appendix C explains how to estimate the various resources used by Shunsaku.

• Appendix D - Conflict Relationships

Appendix D explains the application-command conflicts and command-command conflicts that can occur.

• Appendix E - Sample Programs

Appendix E presents a number of sample programs.

• Appendix F - Java APIs Used in V6.0L20 and Earlier

Appendix F details the Java APIs for data searches and updates that were provided in V6.0L20 and earlier.

iv

Preface

• Appendix G - C APIs Used in V6.0L30 and Earlier

Appendix G details the APIs that Shunsaku provides for use by applications created with the C programming language. The APIs in this chapter were provided with V6.0L30 or earlier.

v

Preface

vi

Table of Contents

Chapter 1 Overview Features of Shunsaku.................................................................................................................1-2

A Highly Flexible and Scalable System Configuration...........................................................1-2 High Data Maintainability and Availability that Improve Reliability.........................................1-4

Functions of Shunsaku ...............................................................................................................1-6 Transaction ............................................................................................................................1-6 Backup and Recovery............................................................................................................1-7 High Availability (HA) .............................................................................................................1-8 Shunsaku File ......................................................................................................................1-10

Chapter 2 Design System Configuration..................................................................................................................2-2

Process Configuration ...........................................................................................................2-4 Server Configuration..............................................................................................................2-5 File Configuration.................................................................................................................2-10

Designing Shunsaku Systems ..................................................................................................2-15 Appointing an Administrator.................................................................................................2-15 Designing the Server Configuration.....................................................................................2-16

Determining the Number of Searchers...........................................................................2-16 Determining the Number of Directors.............................................................................2-16 Determining the Number of Director Servers .................................................................2-17 Determining the Number of Search Servers ..................................................................2-17 Determining the Number of Sort Servers .......................................................................2-17

Determining the Disk Allocations .........................................................................................2-17 Determining the Character Encoding ..................................................................................2-17 Estimating Resource Requirements ....................................................................................2-17 Designing HA.......................................................................................................................2-18 Determining Port Numbers ..................................................................................................2-18 Designing Backups ..............................................................................................................2-18

Designing Applications..............................................................................................................2-19 Installation .................................................................................................................................2-20

vii

Table of Contents

Chapter 3 Operation Operation Flow........................................................................................................................... 3-2 Starting and Stopping Shunsaku................................................................................................ 3-3

Starting Shunsaku................................................................................................................. 3-3 Stopping Shunsaku............................................................................................................... 3-5

Creating Lookup Data ................................................................................................................ 3-7 Data Searches and Updates...................................................................................................... 3-9 System Operation .................................................................................................................... 3-10

Redistributing Search Data..................................................................................................3-11 Optimizing Director Data Files ............................................................................................ 3-13 Adding Searchers and Search Servers .............................................................................. 3-15

Adding Search Servers to Increase Search Speed ....................................................... 3-15 Adding a Search Server to Handle Future Increases in Data Volumes ........................ 3-18

Adding Directors and Director Servers ............................................................................... 3-20 Adding Sorters and Sort Servers ........................................................................................ 3-23 Deleting Searchers ............................................................................................................. 3-25 Deleting Directors ............................................................................................................... 3-27 Deleting Sorters .................................................................................................................. 3-29 Changing the Data Insertion Destination Director .............................................................. 3-31

Chapter 4 Transaction Function Overview of Transactions........................................................................................................... 4-2

What is a Transaction? ......................................................................................................... 4-2 Starting and Stopping Transactions ...................................................................................... 4-4 Auto Commit and Manual Commit........................................................................................ 4-5 How Data Appears to the User ............................................................................................. 4-6 Controlling Transaction Concurrency ................................................................................... 4-6

Using Transactions with Java APIs............................................................................................ 4-7 Handling Errors ................................................................................................................... 4-10

Using Transactions with C APIs ............................................................................................... 4-12 Handling Errors ................................................................................................................... 4-15

Chapter 5 Shunsaku File Overview of Shunsaku File ........................................................................................................ 5-2 Shunsaku File Installation Guidelines........................................................................................ 5-3

Application Development ...................................................................................................... 5-3 Hardware Resource Allocation ............................................................................................. 5-3

Creating and Deleting Shunsaku Files ...................................................................................... 5-4 Creating a Shunsaku File and Adding a Director to the File................................................. 5-4 Deleting Shunsaku Files ....................................................................................................... 5-5 Creating, Deleting and Changing Domains .......................................................................... 5-6

viii

Table of Contents

Methods Used to Access Shunsaku File ....................................................................................5-7 Per-connection Specification .................................................................................................5-7

Specifying with the Java API.............................................................................................5-7 Specifying with the C API..................................................................................................5-8

Specification in the API Environment File ..............................................................................5-8 Setting the Access Range of an Application that does not Specify a Shunsaku File ............5-9

Notes on Using Shunsaku Files................................................................................................5-11

Chapter 6 HA Functions Overview of Shunsaku's HA Functions.......................................................................................6-2

Failover Function for Director Servers...................................................................................6-2 Failover Function for Search Servers ....................................................................................6-7

Failover Operations for Director Servers ....................................................................................6-9 Designing Director Server Failovers......................................................................................6-9

Shunsaku Cluster System Configuration..........................................................................6-9 Shunsaku File Allocation ................................................................................................6-12

Procedure for Setting up Director Server Failovers Using MSCS.......................................6-13 Making Preliminary MSCS Settings................................................................................6-14 Installing Shunsaku.........................................................................................................6-14 Setting Up Shunsaku......................................................................................................6-15 Creating an Area for Storing Director Data Files ............................................................6-16 Creating an Area for Storing Operation Log Files ..........................................................6-16 Registering Services.......................................................................................................6-16 Creating Resources........................................................................................................6-16 Varying Resources Online ..............................................................................................6-18 Testing Failover Operation..............................................................................................6-18 Varying Resources Offline ..............................................................................................6-18

Procedure for Setting up Director Server Failovers Using PRIMECLUSTER.....................6-18 Making Preliminary PRIMECLUSTER Settings..............................................................6-19 Installing Shunsaku.........................................................................................................6-20 Setting Up Shunsaku......................................................................................................6-20 Creating an Area for Storing Director Data Files ............................................................6-22 Creating an Area for Storing Operation Log Files ..........................................................6-22 Deactivating GDS Resources.........................................................................................6-22 Creating Cluster Applications .........................................................................................6-22 Starting Cluster Applications...........................................................................................6-24 Checking Cluster Application Operation.........................................................................6-24 Stopping the Cluster Applications...................................................................................6-24

Recovery Procedure for Shared Disk Faults .......................................................................6-24 Recovery Procedure for Director Data Files (MSCS).....................................................6-25 Recovery Procedure for Operation Log Files (MSCS) ...................................................6-27 Recovery Procedure for Director Data Files (PRIMECLUSTER)...................................6-29 Operation Log File Recovery Procedure (PRIMECLUSTER) ........................................6-31

ix

Table of Contents

Failover Operations for Search Servers .................................................................................. 6-33 Designing Failovers for Search Servers ............................................................................. 6-33 Setting Up Failovers for Search Servers ............................................................................ 6-35

Installing and Setting Up Shunsaku Systems Including Alternative Searchers............. 6-35 Adding Alternative Searchers ........................................................................................ 6-37

Failover Operations............................................................................................................. 6-39

Chapter 7 Backup and Recovery Overview of Backup and Recovery............................................................................................ 7-2

Relationship between Operation Log Files and Backup Data .............................................. 7-3 Backup Cycle ........................................................................................................................ 7-5 Backup Start Declaration and End Declaration .................................................................... 7-5 Confirming Backup Status..................................................................................................... 7-5

Backing Up and Restoring the Shunsaku Environment Files .................................................... 7-6 Backup .................................................................................................................................. 7-6 Recovery............................................................................................................................... 7-7

How to Recover to a Point Immediately Prior to a Malfunction ................................................. 7-8 Preparation............................................................................................................................ 7-9 Backup .................................................................................................................................. 7-9 Recovery..............................................................................................................................7-11

How to Recover to the Latest Backup ..................................................................................... 7-14 Preparation.......................................................................................................................... 7-14 Backup ................................................................................................................................ 7-15 Recovery............................................................................................................................. 7-17

How to Recover to a Specified Point ....................................................................................... 7-20 Preparation.......................................................................................................................... 7-20 Backup ................................................................................................................................ 7-20 Recovery............................................................................................................................. 7-21

Recovering the Operation Log Files ........................................................................................ 7-23

Chapter 8 Maintenance Monitoring .................................................................................................................................. 8-2

Monitoring System Status ..................................................................................................... 8-3 Monitoring System Operation Information ....................................................................... 8-3 Monitoring System Setup Information ............................................................................. 8-6

Monitoring Connection Status............................................................................................... 8-7 Monitoring Connection Information.................................................................................. 8-7

Monitoring Conductor Status ...............................................................................................8-11 Monitoring Conductor Operation Information .................................................................8-11 Monitoring Conductor Setup Information....................................................................... 8-14

Monitoring Sorter Status ..................................................................................................... 8-16 Monitoring Sorter Operation Information ....................................................................... 8-16 Monitoring Sorter Setup Information.............................................................................. 8-19

x

Table of Contents

Monitoring Director Status ...................................................................................................8-21 Monitoring Director Operation Information .....................................................................8-21 Monitoring Director Setup Information............................................................................8-25

Monitoring Searcher Status .................................................................................................8-28 Monitoring Searcher Operation Information ...................................................................8-28 Monitoring Searcher Setup Information..........................................................................8-30

Recreating Lookup Data ...........................................................................................................8-32 Deleting Lookup Data ..........................................................................................................8-32 Recreating Lookup Data ......................................................................................................8-33

Responding to Errors ................................................................................................................8-34 Handling Server Errors ........................................................................................................8-34

Detecting Errors and Assessing the Impact on Operations............................................8-34 Recovering from Errors ..................................................................................................8-35

Handling Application Server Errors......................................................................................8-36 Recovering from Errors ..................................................................................................8-37

Handling errors Produced during Command Execution......................................................8-38

Appendix A Execution Parameters in Environment Files System Environment Files ......................................................................................................... A-3 Conductor Environment Files .................................................................................................. A-12 Director Environment Files ...................................................................................................... A-20 Sorter Environment Files ......................................................................................................... A-28 Searcher Environment Files .................................................................................................... A-32 API Environment Files ............................................................................................................. A-36 Correspondence between V6.0 and V7.0 Execution Parameters ........................................... A-40

Appendix B Allowable Values

Appendix C Estimating Resources Estimating Resources for Each Server......................................................................................C-2 Estimating Resources for the Conductor...................................................................................C-3

The Size of Conductor Log Files ..........................................................................................C-3 Memory Requirements for Conductors.................................................................................C-3

Estimating Resources for Sorter................................................................................................C-6 The Size of Sorter Log Files .................................................................................................C-6 Memory Requirements for Sorters .......................................................................................C-6

Estimating Resources for the Director .......................................................................................C-9 The Size of Director Data Files.............................................................................................C-9 The Size of Operation Log Files .........................................................................................C-10 The Size of Director Log Files ............................................................................................C-10 Memory Requirements for Directors................................................................................... C-11

xi

Table of Contents

Estimating Resources for Searcher .........................................................................................C-14 The Size of Searcher Log Files ..........................................................................................C-14 Memory Requirements for Searchers.................................................................................C-14

Appendix D Conflict Relationships Conflict Relationships between Applications and Commands...................................................D-2

Leading Command/Following Application.............................................................................D-2 Leading Application/Following Command.............................................................................D-5

Conflict Relationships between Commands ..............................................................................D-9 Conflict Relationships for Conductor Commands.................................................................D-9 Conflict Relationships for Director Commands...................................................................D-10 Conflict Relationships for Sorter Commands......................................................................D-11 Conflict Relationships for Searcher Commands................................................................. D-11

Appendix E Sample Programs Sample Program Using the Java APIs....................................................................................... E-2 Sample Program Using the C APIs............................................................................................ E-7

Appendix F Java APIs Used in V6.0L20 and Earlier ShunSearch Class ..................................................................................................................... F-3

create .................................................................................................................................... F-3 executeSearch1 .................................................................................................................... F-5 executeSearch2 .................................................................................................................... F-6 executeSearch3 .................................................................................................................... F-8 getHitCount ........................................................................................................................... F-9 getRetNum.......................................................................................................................... F-10 getRecNo ............................................................................................................................ F-11 getRecData ......................................................................................................................... F-13

ShunBatch Class...................................................................................................................... F-14 ShunBatch........................................................................................................................... F-14 executeAdd ......................................................................................................................... F-16 executeDelete ..................................................................................................................... F-17

Character Encoding ................................................................................................................. F-19 Error Codes Output .................................................................................................................. F-20

xii

Table of Contents

Appendix G C APIs Used in V6.0L30 and Earlier C APIs ........................................................................................................................................G-2

shunadd ................................................................................................................................G-2 shundeletebyrecid.................................................................................................................G-4 shunsearch1 .........................................................................................................................G-6 shunsearch2 .........................................................................................................................G-8

Reply Data Storage Management Structure..................................................................G-10 shunsearch3 .......................................................................................................................G-13

Reply Data Storage Management Structure..................................................................G-16 shunsort ..............................................................................................................................G-18

Reply Data Storage Management Structure..................................................................G-20 Error Codes Output..................................................................................................................G-23

xiii

Table of Contents

xiv

Chapter 1 Overview

This chapter explains the Shunsaku features and functions that are useful for constructing highly reliable large-scale systems.

• Features of Shunsaku

• Functions of Shunsaku

Refer to the Operation Guide for more information on the basic functions of Shunsaku.

1-1

Chapter 1: Overview

Features of Shunsaku Shunsaku has two main features for developing reliable, large-scale systems:

• A Highly Flexible and Scalable System Configuration

• High Data Maintainability and Availability that Improve Reliability

A Highly Flexible and Scalable System Configuration When Shunsaku is running on a single server, increases in the amount of data that is handled can result in inadequate performance due to the limitations of the server hardware. In such cases, directors, searchers, and sorters can be relocated to separate servers to distribute the load evenly on each server and improve operating efficiency. This is referred to as a multi-server configuration.

The following figure shows the addition of a Director Server and a Search Server:

Figure 1-1 Adding a Search Server and a Director Server

1-2

Features of Shunsaku

The following figure shows the addition of a Sort Server:

Figure 1-2 Adding a Sort Server

As the scale of a business expands, performance may deteriorate due to an increase in the number of jobs that must be processed, and different jobs may be required to perform different types of operations. In such cases, Shunsaku can group data according to individual directors. This can improve the performance of individual jobs and provide flexibility to the operation design. This function is referred to as “Shunsaku File”.

Figure 1-3 Shunsaku File - Grouping Data by Director

1-3

Chapter 1: Overview

High Data Maintainability and Availability that Improve Reliability Systems that are required to provide a higher level of reliability must be able to reliably restore data in the event of an unexpected system failure or disk malfunction.

Shunsaku is equipped with backup and recovery functions that enable it to reliably restore data right up to the state that existed immediately prior to the problem.

Figure 1-4 Data Integrity by Backup and Recovery

Shunsaku also features a High Availability (HA) function that provides systems with a high level of availability.

If an active searcher malfunctions, the data that was being processed by that searcher is automatically transferred to another searcher that is functioning normally, allowing operations to continue uninterrupted.

It is also possible to place spare directors and searchers on standby. The data in a malfunctioning director or searcher can then be transferred to the standby director or searcher so that the malfunction does not have a detrimental effect on performance. This is referred to as a failover function.

1-4

Features of Shunsaku

Figure 1-5 Failover Function Overview

Shunsaku is also equipped with a transaction function that guarantees data integrity when simultaneous data updates occur.

1-5

Chapter 1: Overview

Functions of Shunsaku The following functions make Shunsaku suitable for reliable, large-scale operation:

• Transaction

• Backup and Recovery

• High Availability (HA)

• Shunsaku File

Each of these functions is explained in the following sections.

Transaction When multiple items of data are updated and the system needs to maintain data integrity between pre-update and post-update data, it is sometimes desirable to specify or cancel a series of update operations together.

Applications can use Shunsaku’s transaction function to guarantee data integrity in such situations. The decision on whether to apply to Shunsaku the results of updates that were performed after a transaction had started can be made at the time that transaction terminates.

Figure 1-6 Overview of Transaction Function

Refer to Chapter 4, Transaction Function for more information on transactions.

1-6

Functions of Shunsaku

Backup and Recovery If a disk error occurs while Shunsaku is operating, data that has been backed up, together with an update log, can be used to restore Shunsaku to the state in which it was before the problem occurred.

This capability is referred to as the backup and recovery function.

Figure 1-7 Overview of Backup and Recovery

Refer to Chapter 7, Backup and Recovery for more information on backup and recovery.

1-7

Chapter 1: Overview

High Availability (HA) Systems that are required to provide a high level of reliability must be able to provide high availability (HA), which minimizes the interruption to services caused by a server failure.

If a problem occurs in an active searcher, Shunsaku will automatically transfer the data from the faulty searcher to another (functioning) searcher so that operations can continue without interruption. This capability is referred to as the automatic degradation function.

Failover operation can also be performed so that even if a server fails, another server can immediately take over its duties and ensure that there is no interruption to services. Director Server failover reduces system downtime, and Search Server failover maintains search performance levels.

Shunsaku provides the following HA functions:

• Director Server failover

• Search Server failover

Figure 1-8 Configuration for Director Server Failover

1-8

Functions of Shunsaku

Figure 1-9 Configuration for Search Server Failover

Refer to Chapter 6, HA Function for more information on HA.

1-9

Chapter 1: Overview

Shunsaku File When multiple jobs are to be simultaneously created and operated using Shunsaku, it is sometimes desirable to determine the operating mode and hardware resources separately for each job.

Shunsaku File is used to ensure this kind of job independence. Shunsaku File makes it possible to manage the search target data maintained by each director in a separate group. This can be used to restrict the extent to which an application can access each grouped director.

Using Shunsaku File, it is possible to build applications that access only the data required for a particular job, and to individually allocate hardware resources to suit the performance requirements of each job.

Multiple files can also be gathered together into a group known as a domain. This makes it possible to create jobs that perform transverse searches across the data handled by more than one job.

Figure 1-10 Example of Shunsaku File Operation

Refer to Chapter 5, Shunsaku File for more information on Shunsaku File.

1-10

Chapter 2 Design

This chapter describes topics relating to Shunsaku system design.

• System Configuration

• Designing Shunsaku Systems

• Designing Applications

• Installation

2-1

Chapter 2: Design

System Configuration This section explains the following Shunsaku configurations:

• Process Configuration

• Server Configuration

• File Configuration

The servers that make up Shunsaku can be used with the following operating system combinations.

Table 2-1 Operating System and Shunsaku Server Compatibility

Product Admin server

Director server

Sort server

Search server Application server

Windows version of Shunsaku

Linux version of Shunsaku

Solaris OE version of Shunsaku

Refer to Operator’s Guide for details on the Admin Server.

2-2

System Configuration

Note the points below when designing server configurations for Shunsaku:

Figure 2-1 Shunsaku Server Configuration Design

• Place Shunsaku within the local Intranet to protect data from illegal accesses while these servers are connected to the Internet.

• Public Web servers (that make information available to the outside world) can be managed by the firewall and protected from illegal accesses by adding a third network called “DMZ” to the firewall and connecting the Web servers to that network.

• Use a communications method whereby accesses that pass over the firewall are routed via an application server.

• To protect networks against data leakage and data falsification, an encryption method such as SSL should be used in communications outside the application server.

2-3

Chapter 2: Design

Process Configuration The following diagrams show the configuration of Shunsaku processes:

Figure 2-2 Shunsaku Process Configuration

Shunsaku includes the following four processes. A system that is made up of these processes is referred to as a Shunsaku system.

Table 2-2 Shunsaku System Processes Process Role

Conductor Manages directors, receives requests from applications, and issues requests to directors and sorters.

Director Manages searchers and data. (Extracts data that matches search conditions.)

Sorter Sorts and aggregates search results.

Searcher Evaluates search conditions.

2-4


Server Configuration Shunsaku can be used to construct the following server configurations:

• Systems that operate using a single server

• Systems in which searchers run on separate servers

• Systems in which applications run on separate servers

• Systems that operate with additional director servers

• Systems that operate with additional sort servers

These configurations are defined using system environment files.

Examples of each configuration are shown below:

Figure 2-3 System that Operates Using a Single Server

2-5

Chapter 2: Design

Figure 2-4 System in which Searchers Run on Separate Servers

2-6


Figure 2-5 Systems in which Applications Run on Separate Servers

2-7

Chapter 2: Design

Figure 2-6 System that Operates with an Additional Director Server

2-8


Figure 2-7 System that Operates with an Additional Sort Server

2-9

Chapter 2: Design

File Configuration The following figure shows the files that make up Shunsaku.

Figure 2-8 Configuration of Shunsaku Files

2-10


The following table explains the files that are located on each server.

Director Server

Table 2-3 Director Server Files File type Number of

files Usage Location

System environment file

1 This file holds various types of information that define the Shunsaku system configuration.

Shunsaku installation folder \Shunsaku\etc\system\ Shunsaku system name.cfg

/etc/opt/FJSVshnsk/etc/system/ Shunsaku system name.cfg

Conductor environment file

1 This file holds various types of information that define the application environment for the conductor.

Shunsaku installation folder \Shunsaku\etc\conductor\conductor identifier.cfg

/etc/opt/FJSVshnsk/etc/conductor/conductor identifier.cfg

Director environment file

1

(*1)

This file holds various types of information that define the director environment.

Shunsaku installation folder\Shunsaku\etc\director\director identifier.cfg

/etc/opt/FJSVshnsk/etc/director/director identifier.cfg

Director data file

Multiple These files hold data that has been imported into the director.

The location of these files is specified in the DataFileFolder parameter in the director environment file.

Conductor environment file

2 These files store operation logs and error messages for the conductor.

The location of these files is specified in the LogFileFolder parameter in the conductor environment file.

Director log file

2 (*2) These files hold error messages and the operation log for the director.

The location of these files is specified in the LogFileFolder parameter in the director environment file.

Operation log file

Multiple These files store update operations as update log data.

The location of these files is specified in the OperationLogFolder parameter in the director environment file.

Text file Multiple These are input files for importing data into a director.

Arbitrary

2-11

Chapter 2: Design

File type Number of files

Usage Location

Conductor core file

Multiple These files contain information for investigating problems that occur in a conductor.

The location of these files is specified in the CoreFileFolder parameter in the conductor environment file.

Director core file

Multiple These files contain information for investigating problems that occur in a director.

The location of these files is specified in the CoreFileFolder parameter in the director environment file.

*1 One environment file is required for each director that is running.

*2 Two of these files are created for each director that is running.

Sort Server

Table 2-4 Sort Server Files File type Number of




Shunsaku installation folder \Shunsaku\etc\system\Shunsakusystem name.cfg

/etc/opt/FJSVshnsk/etc/system/Shunsaku system name.cfg

Sorter environment file

1 (*1) This file holds information that specifies the application environment for the sorter.

Shunsaku installation folder\Shunsaku\etc\sorter\sorter identifier.cfg

/etc/opt/FJSVshnsk/etc/sorter/sorter identifier.cfg

Sorter log file 2 (*2) These files store operation logs and error messages for the sorter.

The location of these files is specified in the LogFileFolder parameter in the sorter environment file.

Sorter core file Multiple These files contain information for investigating problems that occur in a sorter.

The location of these files is specified in the CoreFileFolder parameter in the sorter environment file.

*1 One environment file is required for each sorter that is running.

*2 Two of these files are created for each sorter that is running.

2-12


Search Server

Table 2-5 Search Server Files File type Number of




Shunsaku installation folder\Shunsaku\etc\system\ Shunsaku system name.cfg

/etc/opt/FJSVshnsk/etc/system/ Shunsaku system name.cfg

Searcher environment file

1 (*1) These files hold various types of information that define the application environment for the searcher.

Shunsaku installation folder\Shunsaku\etc\searcher\searcher identifier.cfg

/etc/opt/FJSVshnsk/etc/searcher/searcher identifier.cfg

Searcher log file

2

(*2)

These files hold error messages and the operation log for the searcher.

The location of these files is specified in the LogFileFolder parameter in the searcher log file.

Searcher core file

Multiple These files contain information for investigating problems that occur in a searcher.

The location of these files is specified in the CoreFileFolder parameter in the searcher environment file.

*1 One environment file is required for each searcher that is running.

*2 Two of these files are created for each searcher that is running.

2-13

Chapter 2: Design

Application Server

Table 2-6 Application Server Files File type Number of


API environment file (*1)

1 This file holds various types of information that define the application environment for the application server.

Shunsaku installation folder\Shunsaku\etc\api\api.cfg

/etc/opt/FJSVshnsk/etc/api/api.cfg

*1 The API environment file can be specified for each machine where the APIs are installed, or for each application that runs.

Note

Refer to Appendix A, Execution Parameters in Environment Files for more information on environment file priority.

2-14

Designing Shunsaku Systems

Designing Shunsaku Systems The following flowchart shows the design procedure that is followed when using Shunsaku:

Figure 2-9 Shunsaku Design Procedure

Appointing an Administrator Appoint an administrator for the Shunsaku system.

Select a member of the “Administrators” group as the administrator.

Select the superuser as the administrator.

2-15

Chapter 2: Design

Designing the Server Configuration Design the configuration of the servers where the conductors, directors, sorters, and searchers will reside.

Shunsaku can be operated using the following server configurations:

• Single server operations

• Operations with separate searcher servers

• Operations with applications on separate servers

• Operations with additional director servers

• Operations with additional sort servers

Determining the Number of Searchers Determine the number of searchers to use according to the amount of data that will be allocated to the searchers.

The data that is placed on a searcher is loaded into the searcher’s memory.

Determine the number of searchers based on the total amount of data to be stored in Shunsaku. Avoid too much data being placed on each searcher. Considering fluctuations of memory usage and search performance, it is recommended that no more than approximately 1 GB of data be placed on a single searcher.

Note also that during degradation, the data of the searcher experiencing a problem will be allocated to the remaining searchers and loaded into their memories. This means that the amount of data during degradation as well as fluctuations in memory usage, search performance, and other factors involved must be considered when determining the number of searchers.

Note

Searchers can be added while the system is in operation. Therefore, even if the volume of data exceeds the estimate made during the design phase, it is possible to simply add another searcher and continue operating normally.

Refer to Estimating Resources for Searcher in Appendix C, Estimating Resources for more information on searcher memory usage.

Determining the Number of Directors Determine the number of directors, taking into account the number of searchers, the amount of memory used by directors, and the Shunsaku operating method. If the number of searchers exceeds the upper limit, it becomes necessary to use more than one director.

The maximum number of searchers that can be registered with a single director is 20, so if the number of searchers is greater than this number, more than one director must be used.

Furthermore, there is a maximum limit on the amount of memory that can be used by a single director. Ensure that each director uses no more than approximately 1.5 GB of memory. If the director is likely to use more than 1.5 GB of memory, multiple directors are required. Refer to 'Estimating Resources for the Director' in Appendix C for more information on how to calculate the amount of memory used by directors

Note

Directors can be added while the Shunsaku system is operating.

Therefore, even if more searchers are added than the estimate made during the design phase, it is possible to simply add another director and continue operating normally.

2-16

Designing Shunsaku Systems

Determining the Number of Director Servers If placing more than one director on a single director server is likely to reach the limits of the hardware capacity, prepare more than one director server and install a director on each server.

Determining the Number of Search Servers If the amount of data stored in Shunsaku is likely to exceed the memory capacity for the search server, prepare more than one search server according to the amount of data stored.

To obtain the full performance of searchers, install search servers so that a single CPU is allocated to each searcher.

Determining the Number of Sort Servers If the number of concurrent sort processes is large and sort processes consume a large amount of CPU capacity, distribute the load by increasing the number of sort servers.

Note

Sorters can be added while the system is in operation.

Determining the Disk Allocations Determine the disks to be allocated, taking backup operations into consideration. For this reason, the disks to be allocated should be determined during the ‘Designing Backups’ phase described later in this chapter.

Refer to Designing Backups for more information on designing backup operations.

Refer to Chapter 7, Backup and Recovery for more information on backup.

Determining the Character Encoding Determine the character encoding to be used.

Use the same character encoding for the following:

• Character encoding specified in the CharacterCode parameter in the system environment

• Input file data for the shundimport command

• Search expressions, return expressions, and sort expressions specified with Shunsaku's C APIs

Note

Refer to 'CharacterCode' in System Environment Files in Appendix A for more information on character encodings that can be used by Shunsaku. The character encoding specified in the XML declaration is not enabled.

Estimating Resource Requirements Estimating server resources involves taking into account both file resources and memory resources. Refer to Appendix C, Estimating Resources for more information on how to estimate each type of resource.

2-17

Chapter 2: Design

Designing HA Determine whether to use the failover function. The failover function makes it possible to build a system with a high level of availability.

If director server failover is to be employed, design a Shunsaku cluster system configuration and determine the appropriate placement of the Shunsaku files.

If search server failover is to be employed, determine how many servers are to be prepared for alternative searchers.

Refer to Chapter 6, HA Function for more information.

Determining Port Numbers Specify the port number for receiving requests from Shunsaku APIs. Ensure that this port number is the same as the port number for receiving requests that is specified in the Conductor parameter in the system environment file.

Refer to 'Conductor' in System Environment Files in Appendix A for more information.

Designing Backups Design backups to be performed during Shunsaku operation.

Backups must be designed taking into account the recovery operation performed in the event of a disk malfunction. The key design points are explained below:

• Location of files on disks

The following files should be placed in the following locations to distribute risk in the event of a disk malfunction.

File name Location

Director data file

Environment files

Disk 1

Operation log file

Backup data (*1)

Disk 2

*1 Backup data of director data files and environment files

• Backup cycle

Determine how often backups should be performed.

• Operation log file use

Determine up to what point data should be recovered if an error occurs on the disk where a director data file is located. Use this recovery point to determine the use of the operation log file.

Refer to Chapter 7, Backup and Recovery for more information.

2-18

Designing Applications

Designing Applications Refer to the Application Development Guide for more information on how to design applications that search and update data using Shunsaku APIs.

2-19

Chapter 2: Design

Installation Install Shunsaku after the design phase is complete.

Refer to the Installation Guide for more information on Shunsaku installation and setup.

2-20

Chapter 3 Operation

This chapter describes how to operate Shunsaku systems.

• Operation Flow

• Starting and Stopping Shunsaku

• Creating Lookup Data

• Data Searches and Updates

• System Operation

3-1

Chapter 3: Operation

Operation Flow This section describes the flow of operations performed by Shunsaku.

Figure 3-1 Shunsaku Flow of Operations

The following tasks are performed during system operation:

• Redistributing Search Data

• Optimizing Director Data Files

• Adding Searchers and Search Servers

• Adding Directors and Director Servers

• Adding Sorters and Sort Servers

• Deleting Searchers

• Deleting Directors

• Deleting Sorters

• Changing the Data Insertion Destination Director

3-2

Starting and Stopping Shunsaku

Starting and Stopping Shunsaku This section explains how to start and stop Shunsaku.

Starting Shunsaku The procedure used to start Shunsaku is explained below.

Use the procedure to start sort servers, search servers, any additional director servers, and the director server where the conductor is located.

Figure 3-2 Shunsaku Start Procedure

3-3


Execute the shunsysstart command on sort servers, search servers, any additional director servers, and the director server where the conductor is located.

To start Shunsaku, execute the shunsysstart command on all servers in the same way. shunsysstart -n Shunsaku system name

Note

Note the following points when starting additional director servers, search servers, and the director server where the conductor is located:

• When directors start up, they become available only after all of the searchers defined by the Searcher parameter in the system environment file have finished starting up, and the distribution of director data has completed.

However, if degradation has been performed before a director starts up, the number of searchers that will be available when the director starts depends on how many searchers have produced abnormalities, as follows:

− If only some searchers have produced abnormalities The number of searchers that were connected last time the director stopped

− If all searchers have produced abnormalities The number specified by the Searcher parameter in the system environment file, which lists the director identifier of the director that the searchers connect to.

• When searchers start up, they access the director defined in the system environment file. If the director has not begun starting up, searchers will try to access the director at one-second intervals.

• Start the conductor after all of the directors defined in the system environment file have finished starting up.

• Data cannot be imported, searched, updated, sorted, or aggregated until all directors, searchers, sorters, and the conductor have finished starting up.

• Before the directors, searchers, sorters, and the conductor can be started, they must be registered with the service.

Refer to the Installation Guide for more information on registering directors, searchers, sorters, and conductors with the service.

3-4

Starting and Stopping Shunsaku

Stopping Shunsaku This section explains how to stop Shunsaku.

Use the following procedure to stop the director server, search servers, any additional director servers, and the sort servers where the conductor is located.

Figure 3-3 Shunsaku Stop Procedure

Execute the shunsysstop command on the director server, search servers, any additional director servers, and sort servers where the conductor is located.

3-5


To stop Shunsaku, execute the shunsysstop command on all servers in the same way.

shunsysstop -n Shunsaku system name

Note

If an abnormality occurs during the stop process and Shunsaku fails to stop, the shunsysstop command will simply terminate. Refer to the error message displayed, eliminate the cause of the error, and then execute the command again. Refer to the Messages for more information about errors.

If a transaction is running when Shunsaku is stopped, the behavior will depend on how Shunsaku is stopped.

Stop method Behavior when Shunsaku is stopped

Stopped normally Shunsaku will wait until the transaction finishes and then stop.

Stopped forcibly Shunsaku will roll the transaction back and then stop.

3-6

Creating Lookup Data

Creating Lookup Data "Lookup data" refers to the director data managed by directors and the search data that is distributed to each searcher.

Lookup data is created using the shundimport command to import text files that contain XML data.

The following figure shows the flow of operations for creating lookup data:

Figure 3-4 Flow of Operations for Creating Lookup Data

Use the following procedure to import data in XML format from a text file and create lookup data:

1. Execute the shundstate command, check the total number of records, and record the result. shundstate -s director identifier

3-7


2. Prepare a text file. If multiple text files exist, join them into a single file. Refer to 'XML Documents in Text Files',in Appendix C of the Operator's Guide for more information on the content of text files. Note

− Files on the local disk of the director server are used as input files for creating lookup data, so create the text files on the local disk of the director server.

− Using the Shunsaku XML conversion function, text files that store XML data and that are used as input files for importing data can be created from data stored in databases or CSV files. Refer to Chapter 5, XML Conversion Function in the Operator's Guide for more information on the XML conversion function.

3. Create lookup data by executing the shundimport command and importing XML data stored in text files. shundimport -s director identifier -f file name

4. Execute the shundstate command and confirm that the total number of records has changed. shundstate -s director identifier

Note

Refer to 'Monitoring Director Operation Information' in Chapter 8, Maintenance for more information on the content displayed by the shundstate command.

• Back up the director data files after executing the shundimport command with the -n option, or if update logs have not been collected. Refer to Backup in Chapter 7, Backup and Recovery for more information on backing up director data files.

• Refer to Recreating Lookup Data in Chapter 8, Maintenance for more information on how to replace or recreate lookup data after it has been created.

3-8

Data Searches and Updates

Data Searches and Updates Once lookup data has been created, it is possible to perform data searches and updates. Data searches and updates are conducted from applications using the Shunsaku APIs.

Refer to the Application Development Guide for more information on how to develop applications.

3-9


System Operation This section explains the following system operations that are performed with Shunsaku:

• Redistributing Search Data

• Optimizing Director Data Files

• Adding Searchers and Search Servers

• Adding Directors and Director Servers

• Adding Sorters and Sort Servers

• Deleting Searchers

• Deleting Directors

• Deleting Sorters

• Changing the Data Insertion Destination Director

This section also presents guides for executing each of the following operations: redistributing search data, optimizing director data files, adding search servers, director servers and sort servers.

Guide to Redistributing Search Data

Redistribute search data if search data is not evenly allocated between search servers. Use the information displayed by the shundstate command to compare the amount of data allocated to each connected searcher.

Guide to Optimizing Director Data Files

Optimize director data files if the amount of wasted space in the director data files has increased. Refer to the fragmentation rate displayed by the shundstate command.

Guide to Adding Searchers or Search Servers

Add extra searchers or search servers if search speed needs to be increased or if the amount of data is expected to increase.

Guide to Adding Directors or Director Servers

If there is a possibility that the limits of the hardware capacity will be reached if multiple directors are allocated to a single director server, prepare multiple director servers and allocate a director to each one.

Guide to Adding Sorters or Sort Servers

Perform load balancing by adding sort servers if a large number of sort processes are being performed concurrently and the CPU usage by sort processes is high.

3-10

System Operation

Redistributing Search Data Search data must be redistributed if a large imbalance develops in the amount of data allocated to each searcher due to data deletions, additional searchers, or some other reason.

Redistributing search data ensures that data is allocated evenly to each searcher.

Shunsaku's search speed depends on the searcher with the most data. If the amount of data allocated to each server is uneven, redistributing search data will equalize the data distribution and reduce the maximum amount of data allocated to any one searcher, thereby improving search performance.

The following diagram shows an example of an operation pattern where search data is redistributed.

Figure 3-5 Example of Operation Pattern where Search Data is Redistributed

3-11


Use the following procedure to redistribute search data.

Figure 3-6 Procedure for Redistributing Search Data

1. Execute the shundstate command to check the amount of data allocated to each searcher from the information for connected searchers. shundstate -s director identifier -p

2. If the amount of data allocated to each searcher is uneven, execute the shundresend command to redistribute search data. shundresend -s director identifier

3. Execute the shundstate command again to check that data is now evenly distributed to each searcher from the information for connected searchers. shundstate -s director identifier –p

Note


Refer to Appendix D, Conflict Relationships for more information on the shundresend command and on conflict relationships between applications and commands.

3-12

System Operation

Optimizing Director Data Files If there are a large number of search data deletions or update processes, the amount of wasted space in the director data files may increase, and search performance may drop. In this case, optimize the director data files.

Figure 3-7 Example of Optimizing Director Data Files

Check the fragmentation rate to determine whether the director data files should be optimized. Execute the shundstate command to check the fragmentation rate.

Note

• Fragmentation is where data is scattered. Access efficiency drops as the fragmentation rate increases.

The fragmentation rate indicates the ratio of the required data stored in the director data files (as specified in the DataFileFolder parameter in the director environment file) to the redundant data left over from processes that were canceled during deletions or updates. Larger values indicate that there is more redundant data.

The larger the fragmentation rate, the more disk space will be used. For example, a fragmentation rate of 75% means that there is three times (75/25) as much redundant data as required data, and so the amount of disk space being used is four times as large as the actual amount of data. Check the amount of free disk space and disk performance, and then decide whether to perform optimization.

3-13


Use the following procedure to optimize the director data files:

Figure 3-8 Procedure for Optimizing Director Data Files

1. Execute the shundstate command to check the fragmentation rate. shundstate -s director identifier -w

2. If the increase in the fragmentation rate indicates that the director data files need to be optimized, execute the shundcds command to optimize the director data files. shundcds -s director identifier

3. Execute the shundstate command to check that the fragmentation rate is 0% after the director data files have been optimized. shundstate -s director identifier -w

Note

• Refer to 'Monitoring Director Operation Information' in Chapter 8, Maintenance for more information on the content displayed by the shundstate command.

• Refer to Appendix D Conflict Relationships for more information on the shundcds command and on conflict relationships between applications and commands.

• While director data files are being optimized, the amount of disk space required temporarily is equal to twice the size of the director data files.

3-14

System Operation

Adding Searchers and Search Servers Search servers are added for the following two reasons:

• To increase search speed

• To handle future increases in amounts of data

Add search servers when it is necessary to meet one or both of these objectives.

Adding Search Servers to Increase Search Speed The following diagram shows an example of how to add a search server to increase search speed.

Figure 3-9 Example of Adding Search Server to Improve Search Speed

3-15


Use the following procedure to add a search server to improve search speeds:

Figure 3-10 Procedure for Adding a Search Server

1. Install and set up a searcher on each additional search server. Refer to 'Installation and Setup with Additional Search Servers' in Chapter 4, Shunsaku System Installation and Setup of the Installation Guide for more information on installing searchers.

2. Edit and distribute the system environment file. Edit the system environment file by adding the following execution parameter:

Execution parameter name Meaning Explanation/Note

Searcher Server information for searchers

Add all of the information for searchers on the search server being added

Note Edit the system environment file on the director server where the conductor is located.

• Distribute the system environment file to the "\etc\system" folder in the Shunsaku installation folder on all servers.

• Distribute the system environment file to the /etc/opt/FJSVshnsk/etc/system directory on all servers.

3-16

System Operation

3. If host names have been specified in the server information for searchers without using IP addresses, specify the IP addresses and host names of the additional search servers in the following file on the director server where the conductor is located.

Windows installation folder\system32\drivers\etc\hosts

/etc/hosts

4. Reflect the information for the newly registered searchers on the director by executing the shunsyscfgedit command on the director server that contains the director where the searchers have been registered. shunsyscfgedit -n Shunsaku system name

5. Execute the shunsysstart command on the additional search server to start the searchers. shunsysstart -n Shunsaku system name

6. Execute the shundstate command on the director server to check that the operating status of the additional searchers is ACTIVE. shundstate -s director identifier –p

7. Execute the shundresend command on the director server to reallocate search data. shundresend -s director identifier

8. Execute the shundstate command on the director server to check that the search target data has been allocated evenly to each searcher. shundstate -s director identifier –p

Note


• Simply starting the additional searchers does not reallocate search data. In order to improve search speed, reallocate search data by executing the shundresend command.

• Data cannot be updated until search data has been reallocated using the shundresend command.

3-17


Adding a Search Server to Handle Future Increases in Data Volumes The following diagram shows an example of how to add a search server to handle future increases in data volumes:

Figure 3-11 Example of Adding Search Servers to Respond to Increased Amounts of Data

3-18

System Operation

The following figure shows the flow for adding search servers in order to respond to future increases in amounts of data:

Figure 3-12 Flow for Adding Search Servers

1. Install and set up searchers on the additional search server.

Note Refer to 'Installation and Setup with Additional Search Servers' in Chapter 4, Shunsaku System Installation and Setup of the Installation Guide for more information on installing searchers.

2. Edit and distribute the system environment file. Edit the system environment file by adding the following execution parameter:




Note Edit the system environment file on the director server where the conductor is located.

− Distribute the system environment file to the "\etc\system" folder in the Shunsaku installation folder on all servers.

3-19


− Distribute the system environment file to the /etc/opt/FJSVshnsk/etc/system directory on all servers.

3. If host names have been specified in the server information for searchers without using IP addresses, specify the IP addresses and host names of the additional search servers in the following file on the director server where the conductor is located.


/etc/hosts

4. Reflect the information for the newly registered searchers on the director by executing the shunsyscfgedit command on the director server that contains the conductor where the searchers have been registered. shunsyscfgedit -n Shunsaku system name

5. Execute the shunsysstart command on the additional search serve to start the searchers. shunsysstart -n Shunsaku system name

6. Execute the shundstate command on the director server to check that the operating status of the additional searchers is ACTIVE. shundstate -s director identifier –p

Note Refer to 'Monitoring Director Operation Information' in Chapter 8, Maintenance for more information on the content displayed by the shundstate command.

Adding Directors and Director Servers Extra directors must be added if there are more searchers that can be managed by a single director.

Directors can be added to an existing director server. However, in order to handle large amounts of data and improve performance, it is also possible to add a new director server.

Director servers can be added while the Shunsaku system is operating.

Refer to 'Systems that operate with additional director servers' in Server Configurations in Chapter 2, Design for more information on server configurations.

3-20

System Operation

Figure 3-13 Procedure for Adding Director Servers

Use the following procedure to add director servers.

1. Install and set up the directors on the director server being added.

Note There is no need to install or set up a conductor or sorter when adding a director server. Refer to 'Installation and Setup with Additional Director Servers' in Chapter 4, Shunsaku System Installation and Setup of the Installation Guide for more information on installing directors.

2. Install and set up the searchers on the search server being added. Refer to 'Installation and Setup with Additional Search Servers' in Chapter 4, Shunsaku System Installation and Setup of the Installation Guide for more information on installing searchers.

3. Edit the system environment file on the director server where the conductor is located, and distribute it to the additional director servers and search servers. Edit the system environment file by adding the following execution parameters:


Director Server information for directors

Add information for directors on the director server being added



− Distribute the system environment file to the “\etc\system” folder in the Shunsaku installation folder on all servers.

3-21



4. Execute the shunsysstart command on the additional search server and director server to start the searchers and directors. shunsysstart -n Shunsaku system name

5. If Shunsaku Files are being used, edit the conductor environment file on the director server where the conductor is located. Add the following execution parameters:


File File information Add File information for the directors being added

Domain Domain information

Edit this information if a new File is being created and the newly created File is to be included in a Domain

InsertPoint

Information for the director where data is to be inserted

If multiple directors are to be used without using Files, specify the director where data from APIs is to be inserted. If Files are to be used, specify the director where data from APIs is to be inserted for each File on which multiple directors have been registered.

6. If host names have been specified in the server information for directors and server information for searchers without using IP addresses, specify the IP addresses and host names of the additional director servers in the following file on the director server where the conductor is located.


/etc/hosts

Note This procedure is not required when adding directors to the director server where the conductor is located.

7. Execute the shunsyscfgedit command on the director server where the conductor is located in order to reflect the additional director information on the conductor. shunsyscfgedit -n Shunsaku system name

3-22

System Operation

Adding Sorters and Sort Servers The objective of adding sort servers is to balance the load of sort processing.

Refer to 'Systems that operate with Additional Sort Servers' in Chapter 2, Design for more information on server configurations.

Use the following procedure to add sort servers and sorters:

Figure 3-14 Procedure for Adding Sort Servers and Sorters

1. If the sorter is to be allocated to a director server, install and set up the sorters on the additional sort server.

Note Refer to 'Installation and Setup with Additional Sort Servers' in Chapter 4, Shunsaku System Installation and Setup of the Installation Guide for more information on installing sorters. There is no need to install or set up conductors or directors when installing sort servers.

2. Edit the system environment file on the director server where the conductor is located, and distribute it to the additional director servers and sort servers Edit the system environment file by adding the following execution parameter.


Sorter Server information for sorters

Add information for sorters on the sort server being added

3-23


Note



3. If host names have been specified in the server information for sorters without using IP addresses, specify the IP addresses and host names of the additional sort servers in the following file on the director server where the conductor is located:


/etc/hosts

4. Execute the shunsysstart command on the additional sort servers to start the sorters. shunsysstart -n Shunsaku system name

5. Execute the shunsyscfgedit command on the director server where the conductor is located in order to reflect the additional sorter information on the conductor. shunsyscfgedit -n Shunsaku system name

3-24

System Operation

Deleting Searchers Use the following procedure to delete searchers.

Figure 3-15 Procedure for Deleting Searchers

1. Stop Shunsaku. Refer to Stopping Shunsaku for more information on stopping Shunsaku.

2. Edit and distribute the system environment file. Edit the system environment file on the director server where the conductor is located.



Delete the parameter for the searcher being deleted.



3-25


3. With the Windows edition of Shunsaku, the searcher service that was registered must be deleted in order to delete searchers. Delete the searcher service using the shunsservice command. Delete the searcher from the service by executing the shunsservice command on the search server where the searcher is located. shunsservice -s searcher identifier -n Shunsaku system name -d

4. Delete the searcher environment file.

Delete the following file: Shunsaku installation folder\etc\searcher\searcher identifier for the searcher to be deleted.cfg

Delete the following file: /etc/opt/FJSVshnsk/etc/searcher/searcher identifier for the searcher to be deleted.cfg

5 Delete the searcher port number specified in the following file on the server where the searcher is located.

Windows installation folder\system32\drivers\etc\services

/etc/services

6. Start Shunsaku. Refer to Starting Shunsaku for more information on starting Shunsaku.

3-26

System Operation

Deleting Directors Use the following procedure to delete directors.

Figure 3-16 Procedure for Deleting Directors

1. Stop Shunsaku. Refer to Stopping Shunsaku for more information on stopping Shunsaku.

3-27


2. Edit and distribute the system environment file. Edit the system environment file on the director server where the conductor is located. Delete the following execution parameters.


Director Server information for directors

Delete the parameter for the director being deleted.


Delete the parameters for the searchers connected to the director being deleted



3. With the Windows edition of Shunsaku, the director service that was registered must be deleted in order to delete directors. Delete the director service using the shundservice command. Delete the director from the service by executing the shundservice command on the director server where the director is located. shunsservice -s director identifier -n Shunsaku system name -d

4. Delete the director environment file.

Delete the following file: Shunsaku installation folder\etc\director\director identifier for the director to be deleted.cfg

Delete the following file: /etc/opt/FJSVshnsk/etc/director/director identifier for the director to be deleted.cfg

5. Delete all of the searchers connected to the director to be deleted. Refer to Deleting Searchers for more information on deleting searchers.

6. Delete director data files. Delete the director data files that have been used by the director to be deleted. Delete everything in the directory with the same name as the director identifier under the directory specified by the DataFileFolder parameter in the director environment file.

7. Delete operation log files. Delete any operation log files that have been used by the director to be deleted. Delete everything in the directory with the same name as the director identifier under the directory specified by the OperationLogFolder parameter in the director environment file.

3-28

System Operation

8. Delete the port number for the director that is specified in the following file on the server where the director is located.


/etc/services

The following ports defined as port numbers for the director must be deleted. − The search port number for searchers

− The update port number for searchers

− The port number for receiving requests

9. Start Shunsaku. Refer to Starting Shunsaku for more information on starting Shunsaku.

Deleting Sorters Use the following request to delete sorters.

Figure 3-17 Procedure for Deleting Sorters

3-29


1. Stop Shunsaku.

Note Refer to Stopping Shunsaku for more information on stopping Shunsaku.

2. Edit and distribute the system environment file. Edit the system environment file on the director server where the conductor is located. Delete the following execution parameter:


Sorter Server information for sorters Delete the parameter for the sorter being deleted.

Note



3. With the Windows edition of Shunsaku, the sorter service that was registered must be deleted in order to delete sorters. Delete the sorter service using the shunoservice command. Delete the sorter from the service by executing the shunoservice command on the server where the sorter is located. shunsservice -s sorter identifier -n Shunsaku system name -d

4. Delete the sorter environment file.

Delete the following file: Shunsaku installation folder\etc\sorter\sorter identifier for the sorter to be deleted.cfg

Delete the following file: /etc/opt/FJSVshnsk/etc/sorter/sorter identifier for the sorter to be deleted.cfg

5. Delete the port number for the sorter that is specified in the following file on the server where the sorter is located.


/etc/services

3-30

System Operation

6. Start Shunsaku.

Note Refer to Starting Shunsaku for more information on starting Shunsaku.

Changing the Data Insertion Destination Director If multiple directors are being used, it is possible to change which director is used to insert data from APIs.

This section explains how to change the director where data is inserted from APIs.

The following diagram shows an example of an operation pattern where data is managed so that data is allocated evenly to each director.

The operating conditions for this example area as follows:

• The system will monitor the amount of data on each director every day.

• When each day's jobs finish, the system will determine which director has the least data, and change the insertion destination for data from APIs to that director.

• There are three directors.

The data insertion destination director can be changed during operations.

3-31


Figure 3-18 Example of Operation Pattern for Distributing Data Evenly to Each Director

3-32

System Operation

The procedure for changing the data insertion destination director for this operation pattern is as follows:

Figure 3-19 Procedure for Changing the Data Insertion Destination Director

1. Execute the shundstate command, check the total number of records for each director, and record the result. shundstate -s director identifier -w

2. Specify the director to be used as the data insertion destination.

3. Change the value of the InsertPoint parameter in the conductor environment file to the director identifier of the director specified in Step 2 above.

4. Execute the shunsyscfgedit command to change the data insertion destination director to the value of the InsertPoint parameter specified in Step 3 above. shunsyscfgedit -n Shunsaku system name

Note Refer to 'Monitoring Director Operation Information' in Chapter 8, Maintenance for more information on the content displayed by the shundstate command.

3-33


3-34

Chapter 4 Transaction Function

This chapter explains the transaction function provided by Shunsaku and how to use transactions when developing applications.

• Overview of Transactions

• Using Transactions with Java APIs

• Using Transactions with C APIs

4-1

Chapter 4: Transaction Function

Overview of Transactions This section explains the following aspects of Shunsaku’s transaction function:

• What is a Transaction?

• Starting and Stopping Transactions

• Auto Commit and Manual Commit

• How Data Appears to the User

• Controlling Transaction Concurrency

Refer to Appendix E, Sample Programs for sample programs that use the transaction function.

What is a Transaction? A transaction is the smallest unit of work that has a meaning with respect to business rules. It cannot be further factored or broken down.

This section uses the example of participation in educational courses to explain transactions.

Assume that to attend the course, it is necessary to update the following two XML documents:

• An XML document for managing course information (the student capacity of the course, the number of individuals who have been enrolled, etc.)

• An XML document for managing student information (the course attended by each student, etc.)

XML Document for Managing Course Information

<course> <name>Business negotiation</name> <instructor> <first-name>Max</first-name> <last-name>Cameron</last-name> </instructor> . . . <capacity>40</capacity> <current-auditors>30</current-auditors> . . . </course> <course> . . .

4-2

Overview of Transactions

XML Document for Managing Student Information\

<student> <first-name>Jack</first-name> <last-name>Wilson</last-name> <e-mail>[email protected]</e-mail> <course>Japanese language</course> . . . <course>Business negotiation</course> . . . </student> <student> <first-name>Mary</first-name> <last-name>Thompson</last-name> <e-mail>[email protected]</e-mail> <course>Chinese language</course> . . . </student> <student> . . .

At this point, let us assume that the update process associated with Mary Thompson’s application to attend a course in business negotiation techniques is performed. The resulting XML documents are as follows:

XML Document for Managing Course Information

<course> <name>Business negotiation</name> <instructor> <first-name>Max</first-name> <last-name>Cameron</last-name> </instructor> . . . <capacity>40</capacity> <current-auditors>31</current-auditors> . . . </course> <course> . . .

4-3


XML Document for Managing Student Information

<student> <first-name>Jack</first-name> <last-name>Wilson</last-name> <e-mail>[email protected]</e-mail> <course>Japanese language</course> . . . <course>Business negotiation</course> . . . </student> <student> <first-name>Mary</first-name> <last-name>Thompson</last-name> <e-mail>[email protected]</e-mail> <course>Chinese language</course> . . . <course>Business negotiation</course> . . . </student> <student> . . .

In this example, the information that Mary is attending the course in business negotiation techniques is added to Mary’s student information, and the number of course participants in the business negotiation techniques course information is increased. These two data manipulations only have meaning as a pair; either manipulation on its own will not be a correct representation of the actual situation. If either data manipulation fails, the data will become logically inconsistent.

Transactions are used in situations such as these. Shunsaku transactions are made up of a series of data changes. Shunsaku ensures that the data changes making up the transaction are either all valid or else all invalid. In short, by making a meaningful series of data manipulations a single transaction, it becomes possible to maintain the logical consistency of the data as seen from the user.

Starting and Stopping Transactions A transaction is started automatically by the initial data update process following establishment of a connection, or after the previous transaction has finished. Subsequent data updates do not become part of the transaction until the transaction is terminated.

Transactions are terminated by a commit or rollback operation. A commit operation enables all changes, and a rollback operation cancels all changes.

4-4

Overview of Transactions

Commit and rollback operations use the following APIs. These APIs enable applications to stop transactions at any time.

Language type API name

commit Java

rollback

ShunCommit C

ShunRollback

Transactions will also terminate in the situations shown in the following table.

The table lists the causes of terminations and the resulting operations.

Main cause of termination Operation

An application process terminates.

The conductor judges that the application process has terminated because the machine where the application is running fails or a communication problem occurs.(*)

Transactions that are running when the application process terminates will be rolled back.

An error that does not allow the transaction to continue occurs.

The transaction will be rolled back and the rollback will be reported to the application as an API result.

(*) Refer to 'If Errors Occur with the Application Server' in Chapter 8, Maintenance for more information on restoration methods.

Auto Commit and Manual Commit Transactions can be committed either automatically or manually.

"Auto commit" refers to the way Shunsaku automatically starts and finishes transactions for each data manipulation (i.e., API call).

"Manual commit" refers to the way applications perform the commit operation by executing commit or ShunCommit.

Use manual commit to perform multiple update operations as a single transaction.

The following APIs can be used to switch between auto commit and manual commit. Auto commit is enabled by default when a connection is established.

Language type API name

Java language setAutoCommit

C language ShunSetConnectAttr

4-5


How Data Appears to the User Update results cannot be seen until they have been committed. Only the pre-update data is visible. This rule also applies to the lookup of data within a single transaction after an update.

For example, suppose that within a given transaction, an XML document is added and a search is performed for that document. There is a problem here because until that entire transaction has been committed, the XML document that was added will not appear in the search results. Once the transaction has been committed, the newly added XML document will appear in the search results.

Controlling Transaction Concurrency When multiple transactions that are running at the same time manipulate the same data, appropriate adjustments must be performed to prevent contradictions and inconsistencies from occurring. Shunsaku performs this control automatically according to the following rules:

• Lookup and updating operations with respect to the same XML document operate simultaneously. One operation will not force the other operation to wait.

• An error will occur if an attempt is made to update an XML document that is being updated by another transaction that has not yet completed.

• An XML document that is referenced by one application may sometimes be updated and committed by another transaction. In such cases, if the application references the same XML document again, the result will be different from what it was previously. This is true even if the application that referenced the XML document is currently executing a transaction.

In other words, regardless of whether a transaction is running, referencing the same XML document multiple times may result in different data being seen each time.

• If a search is performed twice using the same search expression while one transaction is running, and another transaction adds and commits data that matches that search condition between those two searches, the second search result will include one or more XML documents that are not contained in the first search result.

The following table shows if a conflict will occur when two concurrent transactions access the same XML document.

Search Add Delete Update

Search -

Add - - - -

Delete - x x

Update - x x

: No conflict. Operations will be executed simultaneously.

x: Conflict. The operation that was executed later will generate an error.

-: No such situation will occur.

Note

If a commit process and a search take place simultaneously, the search may use any partial results that have been committed by transactions that are performing the commit process.

4-6

Using Transactions with Java APIs

Using Transactions with Java APIs This section provides an example of creating an application that uses Java APIs.

The following example handles the deletion and updating of an XML document as a single transaction.

Figure 4-1 Example of Handling the Deletion and Updating of an XML Document as a Single Transaction

Sample Code

/* * This program assumes that the following variables have been declared * sQuery :Character string containing search expression * sReturn :Character string containing return expression * updateData :Character string containing update data */ ShunConnection con = null; ShunPreparedStatement searchStmt = null; ShunResultSet rs = null; ShunPreparedRecordID deleteRecId = null; ShunPreparedRecordID updateRecId = null;

4-7


try { // Create ShunConnection object con = new ShunConnection("DirSvr1", 33101); // Create ShunPreparedStatement object searchStmt = con.prepareSearch(sQuery, sReturn); // Set the number of items to return per request searchStmt.setRequest(1, 30); // Execute search and create ShunResultSet object rs = searchStmt.executeSearch(); // Obtain record ID String[] idList = new String[rs.getReturnCount()]; int count = 0; while (rs.next()) { idList[count] = rs.getRecordID(); count++; } rs.close(); searchStmt.close(); // Turn off auto commit con.setAutoCommit(false); (1) // Create a ShunPreparedRecordID object in order to delete records deleteRecId = con.prepareDeleteRecordID(); // Target the third item of data for deletion deleteRecId.add(idList[2]); // Delete data if (0 < deleteRecId.getCount()) { deleteRecId.deleteByRecordID(); System.out.println("Deletion complete"); } deleteRecId.close(); // Create a ShunPreparedRecordID object for updating data updateRecId = con.prepareUpdateRecordID(); // Target the fifth item of data to be updated updateRecId.add(idList[4], updateData); // Update data if (0 < updateRecId.getCount()) { updateRecId.updateByRecordID(); System.out.println("Update complete"); } updateRecId.close(); // Execute commit con.commit(); (2) con.close(); } catch (ShunException ex) { try { // Execute rollback if (!con.getAutoCommit()) { con.rollback(); (3) } } catch (ShunException e) { int errorLevel = ex.getErrLevel(); switch (errorLevel) { case ShunException.SHUN_ERROR: System.out.println("Error level: SHUN_ERROR");

4-8


break; case ShunException.SHUN_ERROR_CONNECTION_TERMINATED: System.out.println("Error level: SHUN_ERROR_CONNECTION_TERMINATED"); break; case ShunException.SHUN_ERROR_TRANSACTION_ROLLEDBACK: System.out.println(" Error level: SHUN_ERROR_TRANSACTION_ROLLEDBACK"); break; } System.out.println("Error message: " + ex.getMessage()); e.printStackTrace(); } // Obtain error information int errorLevel = ex.getErrLevel(); switch (errorLevel) { case ShunException.SHUN_ERROR: System.out.println(" Error level: SHUN_ERROR"); break; case ShunException.SHUN_ERROR_CONNECTION_TERMINATED: System.out.println(" Error level: SHUN_ERROR_CONNECTION_TERMINATED"); break; case ShunException.SHUN_ERROR_TRANSACTION_ROLLEDBACK: System.out.println("Error level: SHUN_ERROR_TRANSACTION_ROLLEDBACK"); break; } System.out.println("Error message: " + ex.getMessage()); ex.printStackTrace(); } // Recovery process in the event of an error finally { try { if (rs != null)rs.close(); if (searchStmt != null)searchStmt.close(); if (deleteRecId != null)deleteRecId.close(); if (updateRecId != null)updateRecId.close(); if (con != null)con.close(); } catch (ShunException ex) { int errorLevel = ex.getErrLevel(); switch (errorLevel) { case ShunException.SHUN_ERROR: System.out.println("Error level: SHUN_ERROR"); break; case ShunException.SHUN_ERROR_CONNECTION_TERMINATED: System.out.println("Error level: SHUN_ERROR_CONNECTION_TERMINATED"); break; case ShunException.SHUN_ERROR_TRANSACTION_ROLLEDBACK: System.out.println(" Error level: SHUN_ERROR_TRANSACTION_ROLLEDBACK"); break; } System.out.println("Error message: " + ex.getMessage()); ex.printStackTrace(); } }

4-9


(1) Switching auto commit to disabled state The setAutoCommit method of the ShunConnection class is used to disable auto commit.

Note Switching is necessary because auto commit is enabled when a connection is established. Set the setAutoCommit method to ‘true’ to enable auto commit again.

(2) Executing commit operation The commit method is executed to terminate a transaction and update Shunsaku with the operations that have been performed.

(3) Executing rollback If an error occurs, the rollback method is executed to roll back a transaction without updating Shunsaku with the operations that have been performed.

Note Refer to the Java API Reference for more information on Java APIs.

Handling Errors ShunException is thrown if an error occurs in an application that uses Java APIs. By using the ShunException class method, the following information can be obtained:

• Error level

• Error message

Sample Code

try { : } catch (ShunException ex) { int errorLevel = ex.getErrLevel(); (1) switch( errorLevel ) { case ShunException.SHUN_ERROR : System.out.println("Error level: SHUN_ERROR"); break; case ShunException.SHUN_ERROR_TRANSACTION_ROLLEDBACK : System.out.println("Error level: SHUN_ERROR_TRANSACTION_ROLLEDBACK"); break; case ShunException.SHUN_ERROR_CONNECTION_TERMINATED : System.out.println("Error level: SHUN_ERROR_CONNECTION_TERMINATED"); break; } System.out.println("Error message: " + ex.getMessage()); (2) }

4-10


(1) Error Level The getErrLevel method can be used to obtain the error level. An error may cause a connection to be forcibly closed. By examining the error level, the user can determine the state of an application after an error has occurred. The three error levels shown in the following table can be returned.

Constant Meaning

SHUN_ERROR A warning-level error has occurred due to an incorrect parameter, etc. Re-execute the process from the point at which the error occurred.

SHUN_ERROR_TRANSACTION_ ROLLEDBACK

An error has caused a transaction to be canceled. Re-execute the process from the start of the transaction.

SHUN_ERROR_CONNECTION_TERMINATED

An error has caused the connection to be forcibly closed. Close the ShunConnection object where the error occurred and then re-execute the process starting from establishing the connection.

(2) Error Message The getMessage method can be used to obtain the error message. Refer to Messages for more information on error messages.

4-11


Using Transactions with C APIs This section provides an example of creating an application that uses C APIs.

The following example handles the deletion and updating of an XML document as a single transaction.

Figure 4-2 Example of Handling the Deletion and Updating of an XML Document as a Single Transaction

Sample Code

/*This program assumes that the following variables have been declared */ /* query :Search expression */ /* returnQuery:Return expression */ /* updateData :Update data */ /* pResultId :Structure array holding record ID */ /* pDeleteId :Structure array for setting the record IDs to be deleted */ /* pUpdateId :Structure array for setting the record IDs to be updated */

4-12

Using Transactions with C APIs

/* Allocate a connection handle */ ret = ShunAllocHandle(NULL, &ConH); if(ret != SHUN_SUCCESS) { error("ShunAllocHandle ", ConH); return; } /* Establish a connection by specifying the host name and port number */ ret = ShunConnect(ConH, "DirSvr1", 33101, NULL); if(ret != SHUN_SUCCESS) { error("ShunConnect ", ConH); return; } /* Allocate a data manipulation handle for searching data */ ret = ShunAllocHandle(ConH, &searchStmtH); if(ret != SHUN_SUCCESS) { error("ShunAllocHandle ", ConH); return; } /* Execute search */ ret = ShunSearch(searchStmtH, 1, NULL, 0, 30, query, returnQuery, NULL, &Hit_Cnt, &Return_Cnt, &Returnable_Cnt, &Rec_Id_Out, &Data, &First_Pos, &Last_Pos); if(ret != SHUN_SUCCESS) { error("ShunSearch ", searchStmtH); return; } /* Retain record ID */ for(i=0; i< Return_Cnt; i++) { pResultId[i] = Rec_Id_Out[i]; } /* Release the data manipulation handle for searching data */ ret = ShunFreeHandle(searchStmtH); if(ret != SHUN_SUCCESS) { error("ShunFreeHandle ", searchStmtH); return; } /* Turn auto commit off */ ret = ShunSetConnectAttr(ConH, SHUN_ATTR_AUTOCOMMIT, SHUN_FALSE); (1) if(ret != SHUN_SUCCESS) { error("ShunSetConnectAttr ", ConH); return; } /* Allocate a data manipulation handle for deleting data */ ret = ShunAllocHandle(ConH, &deleteStmtH); if(ret != SHUN_SUCCESS) { error("ShunAllocHandle ", ConH); return; } /* Target the third item of data for deletion */ pDeleteId[0] = pResultId[2]; /* Execute deletion */ ret = ShunDeleteRecId(deleteStmtH, 1, pDeleteId); if(ret != SHUN_SUCCESS) { error("ShunDeleteRecId ", deleteStmtH); return; } /* Release the data manipulation handle for deleting data */ ret = ShunFreeHandle(deleteStmtH); if(ret != SHUN_SUCCESS) {

4-13


error("ShunFreeHandle ",deleteStmtH); return; } /* Allocate a data manipulation handle for updating data */ ret = ShunAllocHandle(ConH, &updateStmtH); if(ret != SHUN_SUCCESS) { error("ShunAllocHandle ", ConH); return; } /* Target the fifth item of data to be updated */ pUpdateId[0] = pResultId[4]; /* Update the data */ ret = ShunUpdateRecId(updateStmtH, 1, pUpdateId,updateData); if(ret != SHUN_SUCCESS) { error("ShunUpdateRecId", updateStmtH); return; } /* Release the data manipulation handle for updating data */ ret = ShunFreeHandle(updateStmtH); if(ret != SHUN_SUCCESS) { error("ShunFreeHandle ", updateStmtH); return; } /* End the transactions */ ret = ShunCommit(ConH); (2) if(ret != SHUN_SUCCESS) { error("ShunCommit ", ConH); return; } /* Close the connection */ ret = ShunDisconnect(ConH); if(ret != SHUN_SUCCESS) { error("ShunDisconnect", ConH); return; } /* Release the connection handle */ ret = ShunFreeHandle(ConH); if(ret != SHUN_SUCCESS) { error("ShunAllocHandle ", ConH); return; } /* Error processing */ void error(char* func, SHUNHANDLE handle) { printf( "%s:abnormal end\n", func ); ShunGetErrorMessage(handle, &Err_Level, &Err_Msg); printf("Error level = %d\n", Err_Level); printf("Error message = %s\n", Err_Msg); if(ConH != NULL) { ret = ShunRollback(ConH); (3) } ret = ShunFreeHandle(ConH); return; }

4-14

Using Transactions with C APIs

(1) Switching auto commit to disabled state The ShunSetConnectAttr function is used to disable auto commit after the connection handle has been allocated.

Note Switching is necessary because auto commit is enabled when a connection is established. Use the ShunSetConnectAttr function to set SHUN_ATTR_AUTOCOMMIT to SHUN_TRUE to enable auto commit again.

(2) Executing commit operation The ShunCommit function is executed to terminate a transaction and update Shunsaku with the operations that have been performed.

(3) Executing rollback If an error occurs, the ShunRollback function is executed to roll back a transaction without updating Shunsaku with the operations that have been performed.

Note Refer to the C API Reference for more information on C APIs.

Handling Errors If a warning or an error occurs while a C API is being executed, a value other than normal termination will be returned when each function executes. If a value other than normal termination is returned, the ShunGetErrMsgorMessage function can be used to obtain the following information:

• Error level

• Error message

Sample Code

if(ret != SHUN_SUCCESS) { /* Obtain error information */ ShunGetErrorMessage(StmtH, &Err_Level, &Err_Msg); switch ( Err_Level ) { case SHUN_ERROR_CONNECTION: printf("Error level = SHUN_ERROR_CONNECTION\n"); break; case SHUN_ERROR_TRANSACTION: printf("Error level = SHUN_ERROR_TRANSACTION\n"); break; case SHUN_ERROR_DATA: printf("Error level = SHUN_ERROR_DATA\n"); break; } printf("Error message = %s\n", Err_Msg); }

4-15


Obtaining Error Information Examine the value returned when each function executes and, if an error has occurred, obtain the error information.

Error Level By examining the error level, the user can determine the state of an application after an error has occurred.

The three error levels shown in the following table can be returned:

Constant Meaning

SHUN_ERROR_DATA A warning-level error has occurred due to an incorrect parameter, etc. Re-execute the process from the point at which the error occurred.

SHUN_ERROR_TRANSACTION An error has caused a transaction to be canceled. Re-execute the process from the start of the transaction.

SHUN_ERROR_CONNECTION An error has caused the connection to be forcibly closed. Discard and then recreate the connection handle where the error occurred, and then re-execute the process starting from establishing the connection.

Error messages Refer to Messages for more detailed information on error messages.

4-16

Chapter 5 Shunsaku File

This chapter explains the Shunsaku File function.

• Overview of Shunsaku File

• Shunsaku File Installation Guidelines

• Creating and Deleting Shunsaku Files

• Methods Used to Access Shunsaku File

• Notes on Using Shunsaku Files

5-1

Chapter 5: Shunsaku File

Overview of Shunsaku File Shunsaku File is a function to increase the independence of each job when Shunsaku is used to create and operate multiple jobs. The Shunsaku File function makes it possible to arrange the director data of each director into groups that can be managed as separate Files. If the name of a Shunsaku File is specified in an application, that application can operate on that particular File. This makes it possible to limit the range of directors that an application can access, and to determine the individual application logic and hardware resources that are needed to create a job.

Files that are used in multiple jobs can also be associated together in single Domains. By specifying the Domain name in an application, it is possible to create jobs that can search across data used in different jobs.

Even if Domains are used, individual jobs perform data update processes on Files. Only data search jobs can be performed on Domains.

These File and Domain structures are referred to collectively as “Shunsaku Files”.

Figure 5-1 Overview of Shunsaku File

The decision to use Shunsaku Files should be made on the basis of how much variation there is between jobs and the scale of the system. If the Shunsaku File function is not used, there is no need to specify Shunsaku Files in applications, and all director data will be accessible.

5-2

Shunsaku File Installation Guidelines

Shunsaku File Installation Guidelines This section provides guidelines for installing Shunsaku Files.

A feature of Shunsaku is its ability to simplify application and system design and operation, and to keep installation and operating costs down because there is no need to regulate (i.e., define) the format in which XML documents are stored. This means that normally there is no need to install Shunsaku Files.

The decision whether to install Shunsaku Files can be made on the basis of the following two perspectives:

• Application Development

• Hardware Resource Allocation

Application Development From the viewpoint of application development, the key factor determining whether to install Shunsaku Files is data relationships.

If Shunsaku is used to operate multiple jobs, and it is working with a mixture of data, some of which is strongly related to one or more jobs and some that is only weakly related, it can be difficult to find data that is strongly related to a particular job using only the search expression. In such cases, some of the data returned by an application may be of no use to the job in question.

By installing the Shunsaku File function and creating separate Files for data that is strongly related to each job, it is possible to return to applications only the data needed by each job.

Hardware Resource Allocation From the viewpoint of hardware resource allocation, the key factor determining whether to install Shunsaku Files is the performance requirements of each job.

The performance of Shunsaku is influenced by the amount of hardware resources allocated to it. However, when different jobs with different performance requirements are mixed together, it is necessary to provide hardware resources according to the job that has the highest performance requirements. In such cases, there is a need to provide redundant hardware resources.

By installing the Shunsaku File function and creating a File for each job, it is possible to allocate to each Shunsaku File the optimal amount of hardware resources needed to guarantee the performance of each job.

5-3


Creating and Deleting Shunsaku Files This section explains how to create and delete Shunsaku Files.

Shunsaku Files can be created and deleted by editing the conductor environment file.

Creating a Shunsaku File and Adding a Director to the File The following diagram shows the procedure used to create a Shunsaku File and add a director to it.

Figure 5-2 Procedure for Creating a Shunsaku File and Adding a Director

1. Determine the directors to be managed as a Shunsaku File.

2. Shunsaku is operated by assigning a name to each Shunsaku File. This name is referred to as the Shunsaku File name. Specify a string of up to 32 characters as the Shunsaku File name. Only single-byte alphanumeric characters can be used.

3. Edit the conductor environment file. To create a new Shunsaku File, add the File parameter. Edit the File parameter to add to an existing Shunsaku File a director that is not managed by a Shunsaku File. Add or edit the following execution parameter.

Execution parameter name Meaning

File File information

Note Refer to 'Conductor Environment Files' in Appendix A for more information on the execution parameters in the conductor environment file.

5-4

Creating and Deleting Shunsaku Files

4. To reflect Shunsaku File information on the conductor, execute the shunsyscfgedit command on the director server where the conductor is located. shunsyscfgedit -n Shunsaku system name

Note

• When creating Shunsaku Files, each director must be managed by a Shunsaku File. A director that is not managed by a Shunsaku File cannot be accessed by applications.

• A single director cannot be managed by more than one Shunsaku File.

Deleting Shunsaku Files The following diagram shows the procedure used to delete a Shunsaku File.

Figure 5-3 Procedure for Deleting a Shunsaku File

1. Edit the conductor environment file. Delete the File parameter to delete a Shunsaku File. Delete the following execution parameter.





5-5


Creating, Deleting and Changing Domains The following diagram shows the procedure used to create a Domain, add a Shunsaku File to an existing Domain, or delete a Domain.

Figure 5-4 Procedure for Adding a Shunsaku File to an existing Domain

1. When creating a Domain or adding a Shunsaku File to a Domain, determine the Shunsaku File to be associated with the Domain.

2. When creating a Domain, assign a name to the Domain. That name is referred to as the Domain name. Specify a string of up to 32 characters as the Domain name. Only single-byte alphanumeric characters can be used.

3. Edit the conductor environment file. To associate a Shunsaku File with a new Domain, add the Domain parameter. To delete a Domain, delete the Domain parameter. To change the information of an existing Domain, edit the Domain parameter. Add or edit the following execution parameter.



Note By specifying the same Shunsaku File in different Domain parameters, it is possible to associate a single Shunsaku File with more than one Domain. Refer to 'Conductor Environment Files' in Appendix A for more information on the execution parameters in the conductor environment file.


5-6

Methods Used to Access Shunsaku File

Methods Used to Access Shunsaku File This section explains the methods that are used to access Shunsaku Files. To access a Shunsaku File, the Shunsaku File must be specified in an application.

The following three methods can be used to specify a Shunsaku File in an application:

• Per-connection Specification

• Specification in the API Environment File

• Setting the Access Range of an Application that does not Specify a Shunsaku File

Each of these specification methods is explained below.

Per-connection Specification In this method, the Shunsaku File is specified as an API parameter when connecting to Shunsaku.

• Specifying with the Java API

• Specifying with the C API

Specifying with the Java API In this method, the Shunsaku File name is specified when creating the ShunConnection object.

This section explains how to specify the Shunsaku File name in the constructor directly.

Syntax ShunConnection object name = new ShunConnection(host name, port number, Shunsaku File name);

Arguments

host name Specifies the target host name or IP address.

port number Specifies the target port number.

Shunsaku File name Specifies the target Shunsaku File name.

Entry Example When the host name is “DirSvr1”, the port number is “33101” and the Shunsaku File name is “fileA” ShunConnection con = new ShunConnection("DirSvr1",33101, "fileA");

Note

This information can also be specified in the Java properties. Refer to the Java API Reference for more information.

5-7


Specifying with the C API In this method, the Shunsaku File name is specified in a parameter of the ShunConnect function.

Syntax ShunConnect(connection handle, host name, port number, Shunsaku File name);

Arguments

connection handle Specifies the connection handle to be used when establishing the connection.

host name Specifies the host name or IP address of the connection destination.

port number Specifies the port number of the connection destination.

Shunsaku File name Specifies the Shunsaku File name of the connection destination.

Entry Example When the host name is “DirSvr1”, the port number is “33101” and the Shunsaku File name is “fileA” SHUNHANDLE ConH; /* Allocate connection handle */ ShunAllocHandle(NULL, &ConH) /* Specify host name, port number and Shunsaku File name and */ /* establish the connection */ ShunConnect(ConH, "DirSvr1",33101, "fileA");

Specification in the API Environment File In this method, the Shunsaku File name is specified in the ShunsakuFile parameter in the API environment file.

The specification method used with the API environment file is common to both Java APIs and C APIs.

Syntax ShunsakuFile Shunsaku File name

Parameters

ShunsakuFile Specifies the Shunsaku File name of the connection destination.

5-8

Methods Used to Access Shunsaku File

Entry Example When the Shunsaku File name is “fileA” # Specifying File name ShunsakuFile fileA

Note

• Once the Shunsaku File name has been specified in the API environment file, there is no need to specify it in API parameters. The Shunsaku File to be accessed can be changed without having to modify the application.

• Restart the application process in order to reflect the results of the changes to the API environment file.

• Refer to 'API Environment Files' in Appendix A for more information on the API environment file.

Setting the Access Range of an Application that does not Specify a Shunsaku File

To set the access range of an application in which no Shunsaku File is specified, specify the Shunsaku File name in the DefaultFile parameter in the conductor environment file.

The following diagram shows the procedure used to set a particular Shunsaku File as the access range of an application that does not specify a Shunsaku File.

Figure 5-5 Procedure for Setting a Shunsaku File

5-9


1. Determine the Shunsaku File to be set as the access range of an application that does not specify a Shunsaku File.

2. Edit the conductor environment file. To set the access range of an application that does not specify a Shunsaku File, add the DefaultFile parameter. To change the access range, edit the DefaultFile parameter. Add or edit the following execution parameter.


DefaultFile Access range of an application in which no Shunsaku File is specified



5-10

Notes on Using Shunsaku Files

Notes on Using Shunsaku Files Always use one of the following three specification methods when creating a Shunsaku File:

Notes on creating Shunsaku Files 1. Per-connection specification

2. Specification in the API environment file

3. Specification in the DefaultFile parameter in the conductor environment file

Note that the order of priority of these methods is 1 to 3 as listed above.

Note on updating data from APIs Perform data updates from APIs on Files and not Domains. Only search jobs can be performed on Domains.

5-11


5-12

Chapter 6 HA Functions

This chapter explains Shunsaku's HA (High Availability) functions.

• Overview of Shunsaku's HA Functions

• Failover Operations for Director Servers

• Failover Operations for Search Servers

6-1

Chapter 6: HA Functions

Overview of Shunsaku's HA Functions For systems that cannot allow jobs to stop, system availability must be improved by preparing for system failures that may arise while jobs are running.

Failures may relate to:

• Hardware faults

• Software errors and

• Other unexpected events

Shunsaku provides the following HA functions in preparation for unexpected circumstances, allowing systems with high availability to be constructed.

• Failover Function for Director Servers

• Failover Function for Search Servers

Failover Function for Director Servers A failover function can be used with Shunsaku in cluster systems, in preparation for unexpected circumstances such as hardware faults or software errors with director servers.

The failover function for director servers works by using cluster systems where, in addition to the server that is operating (hereafter called the “active server”), another server (hereafter called the “standby server”) is prepared in case something goes wrong with the active server.

6-2

Overview of Shunsaku's HA Functions

Figure 6-1 Example of the Failover Function in Cluster Systems

Using the failover function for director servers allows users to perform searches and updates to Shunsaku without being aware of which is the active server and which is the standby server.

6-3


Figure 6-2 Failover Function for Director Servers

If something unexpected happens to the active server (such as a hardware fault or software error), the cluster system detects the error and performs a failover operation.

The failover operation minimizes system downtime by starting Shunsaku on the standby server and handing over control of jobs to the standby server. After jobs have been taken over, users can continue to perform searches and updates using Shunsaku in exactly the same way as before the server error occurred.

Additionally, because the server operating normally takes over the jobs from the failed server, the failure can be rectified without interrupting these jobs.

6-4


Cluster System The failover function for director servers runs on the following cluster systems.

• Microsoft Cluster Service (MSCS)

• PRIMECLUSTER V4.1A30 or later

Using the HA functions requires knowledge of these cluster systems. Refer to the manuals for each cluster system for more information.

Failover Operation Configuration The failover function for director servers can be used with the following operation configurations of cluster systems.

• 1:1 active/standby configuration

• N:1 active/standby configuration

1:1 Active/Standby Configuration In this operation configuration, there is one standby server for each active director server.

If an abnormality occurs with the active server, the cluster system fails over to the standby server, so jobs do not need to be interrupted.

Figure 6-3 1:1 Active/Standby Configuration

6-5


N:1 Active/Standby Configuration In this operation configuration, there are multiple active servers for each standby server.

This operation configuration reduces the cost of standby servers because, unlike the 1:1 active/standby configuration, there is no need to prepare a standby server for each active server.

Figure 6-4 N:1 Active/Standby Configuration

6-6


Failover Function for Search Servers If errors occur on search servers, operations can be recommenced in minimal time using degradation.

However, using degradation, the amount of data allocated to each searcher operating normally increases, and so the same performance as before degradation cannot be expected. Search data must also be reallocated once the failed search server is restored.

The following diagram shows the flow of degradation processing for search servers.

Figure 6-5 Flow of Degradation Processing for Search Servers

These problems can be resolved using the failover function for search servers.

By preparing alternative search servers to take over the processes from failed search servers, and allocating alternative searchers and starting them in advance, not only can operations recommence in minimal time (as with degradation), but the same level of performance as before the error occurred can also be maintained. Furthermore, search data need not be reallocated when the errors are resolved.

The following diagram shows the flow of processing performed when errors occur on search servers for which the failover function is being used.

6-7


Figure 6-6 Flow of Processing when Errors Occur on Search Servers

6-8

Failover Operations for Director Servers

Failover Operations for Director Servers This section explains the following topics:

• Designing director server failovers

• Procedure for setting up director server failovers using MSCS

• Procedure for setting up director server failovers using PRIMECLUSTER

• Recovery procedure for shared disk faults

Designing Director Server Failovers Make the following decisions when creating director server failovers:

• Shunsaku cluster system configuration

• Shunsaku File allocation

Shunsaku Cluster System Configuration For director server failovers, failovers occur in cluster system environments using the following units.

• For MSCS: cluster groups

• For PRIMECLUSTER: cluster applications

The following diagrams show the Shunsaku resource allocation for cluster systems and the relationships between resources.

6-9


Figure 6-7 Shunsaku Resource Allocation for Cluster Systems (MSCS)

6-10


Figure 6-8 Shunsaku Resource Allocation for Cluster Systems (PRIMECLUSTER)

6-11


Shunsaku File Allocation Decide where the files that comprise Shunsaku will be located.

Figure 6-9 Locations for the Files comprising Shunsaku

With cluster system environments, allocate Shunsaku files to the disks shown in the following table.

Where the location is listed as “Local disk”, files with exactly the same content must be placed on the local disks of both the active server and the standby server.

6-12


Table 6-1 Shunsaku File Locations for Cluster System Environments

Process name File type Location

- System environment file Local disk

Application environment file Local disk

Application log file Local disk

conductor

Core file Local disk



Director data file Shared disk

Operation log file Shared disk


director

Backup data for director data files

Shared disk or external media



sorter




searcher


Procedure for Setting up Director Server Failovers Using MSCS

This section explains how to set up director server failovers where the Shunsaku environment is created on a Microsoft Cluster Service (MSCS) cluster system.

Refer to the MSCS installation guide and manuals for more information on installing and setting up MSCS.

The following diagram shows the procedure for creating Shunsaku on MSCS.

6-13


Figure 6-10 Creating Shunsaku on MSCS

Making Preliminary MSCS Settings Before creating the Shunsaku environment, install MSCS and set up the MSCS environment.

Refer to the MSCS manuals for more information on MSCS.

Shared Disk Settings Director data files and operation log files must be placed on the shared disk in order to use the director server failover function with Shunsaku.

Finish setting up the shared disk drive for the cluster service before setting up the Shunsaku environment.

Inherited IP Address Settings Shunsaku uses inherited IP addresses for the connections between the conductor, directors, sorters, and searchers. Accordingly, finish setting inherited IP addresses using the cluster service configuration wizard before setting up the Shunsaku environment.

Installing Shunsaku Install Shunsaku on all servers.

The installation procedure is the same as that for customizing the server configuration.

Refer to the Installation Guide for more information on the installation procedure.

6-14


Setting Up Shunsaku This section explains how to set up Shunsaku.

Setting Up the Director Server Set up the director server on the active server.

Setting up the director server is the same as the normal setup procedure, except for the following points:

• Editing the system environment file

• Editing the director environment file

• Copying environment files

Note

• If the operation mode for the failover is the N:1 active/standby configuration, the setup procedure is the same as for director servers. Refer to the Installation Guide for more information.

• Refer to 'Installing and Setting Up Additional Director Servers' in the Installation Guide for more information on the setup procedure.

Editing the System Environment File Specify the inherited IP address of the director server in the following execution parameters in the system environment file.

• Conductor

• Director

• Sorter

Editing the Director Environment File Director data files and operation log files must be stored on shared disks.

Specify a directory on a shared disk for the following execution parameters in the director environment file.

• DataFileFolder

• OperationLogFolder

Note

• If director data files and operation log files are damaged at the same time, the latest data cannot be recovered.

• Specify different shared disks for the locations of the operation log files and the director data files.

Copying Environment Files For each environment file, the same settings must be used for the active server and the standby server. After the setup for the active server is complete, copy the following environment files over to the standby server.

• System environment file

• Conductor environment file

• Director environment file

• Sorter environment file

6-15


Setting Up Search Servers The procedure for setting up search servers is the same as for normal (non-cluster) systems.

Refer to 'Installing and Setting Up Additional Search Servers' in the Installation Guide for more information on the setup procedure.

Creating an Area for Storing Director Data Files Create on a shared disk the folder for storing director data files that is specified in the DataFileFolder parameter in the director environment file. Perform this operation on the active server.

Create the folder using either Windows Explorer or the Command Prompt.

Creating an Area for Storing Operation Log Files Create on a shared disk the folder for storing operation log files that is specified in the OperationLogFolder parameter in the director environment file. Perform this operation on the active server.

Create the folder using either Windows Explorer or the Command Prompt.

Registering Services Register services for the conductor, and all sorters and directors.

Refer to the Installation Guide for more information on registering services.

Creating Resources Create Shunsaku resources on MSCS.

Create Shunsaku as a generic service resource.

Create resources for the conductor, sorters, and directors using the following procedure:

Note that the conductor resources should be created after the sorter resources and director resources have been created.

1. Start the cluster service. To start the cluster service, select Administrative Tools, Services, and then Cluster Service from the Control Panel.

2. Start the Cluster Administrator. To start the Cluster Administrator, select Administrative Tools and then Cluster Administrator from the Control Panel.

3. Create new resources. Select New and then Resource from the File menu.

The following table shows the items that are displayed in each dialog box for creating resources, the settings for each item, and the operations that should be performed after the settings are made.

6-16


Table 6-2 Dialogs Displayed during Resource Creation

Dialog Item Setting

Name Specify one of the following: - Conductor-identifier - Sorter-identifier - Director-identifier

Resource Type

Specify Generic Service.

New Resource

Group Select the same group for all resources created. Possible Owner

Available Node

Set the nodes for both the active server and the standby server.

Dependencies Resource Dependencies

Add IP addresses, network names, and shared disks.

Generic Service Parameters

Service Name

Specify one of the following: - For the conductor:

“shuncsvc_shunsaku system name_conductor- identifier” - For sorters:

“shunosvc_shunsaku system name_sorter-identifier” - For directors:

“shundsvc_shunsaku system name_director-identifier”

Registry Replication

Registry Key Not required.

Cluster Administrator

- -

Prevent failback / Allow failback

Select Allow failback

Affect the group

Select this checkbox.

Threshold Specify “0”.

The Advanced tab of the Cluster Administrator Resource Properties dialog box

Period This value is the time that the cluster application will wait for Shunsaku to finish starting up after the nodes switch over. If Shunsaku does not finish starting up within the specified number of seconds, a fault will be regarded as having occurred. Measure the startup time in the user environment by performing startup tests for the cluster application, and set an appropriate value.

Note

− When creating the conductor resources, set all director and sorter resources as resource dependencies. This ensures that the conductor is not started until after the directors and sorters have started.

6-17


Varying Resources Online Vary the Shunsaku resources online in order to activate the director server failover function.

Use the following procedure to vary the Shunsaku resources online:

1. Check the resource status using the cluster administrator.

2. Vary the group where the resources have been registered online.

To check whether Shunsaku has started correctly, refer to the Event Viewer on the active server and the operation log files for the Shunsaku conductor, sorters, and directors.

Testing Failover Operation The director server failover function can be used when the cluster application finishes starting up. It is recommended that users check whether the environment has been set up correctly by performing a failover before actual operations commence.

To check whether Shunsaku has stopped correctly, refer to the Event Viewer on the active server and the operation log files for the Shunsaku conductor, sorters, and directors.

Varying Resources Offline Vary Shunsaku resources offline in order to stop the failover operation.

Use the following procedure to vary Shunsaku resources offline:

1. Check the resource status using the cluster administrator.

2. Vary the group where the resources have been registered offline.

To check whether Shunsaku has stopped correctly, refer to the Event Viewer on the active server and the operation log files for the Shunsaku conductor, sorters, and directors.

Procedure for Setting up Director Server Failovers Using PRIMECLUSTER

This section explains how to set up director server failovers where the Shunsaku environment is created on PRIMECLUSTER.

Refer to the PRIMECLUSTER installation guide and manuals for more information on installing and setting up PRIMECLUSTER.

The following diagram shows the procedure for creating the Shunsaku environment on PRIMECLUSTER.

6-18


Figure 6-11 Creating the Shunsaku Environment on PRIMECLUSTER

Making Preliminary PRIMECLUSTER Settings Before creating the Shunsaku environment, install PRIMECLUSTER and set up the PRIMECLUSTER environment.

Refer to the PRIMECLUSTER manuals for more information on PRIMECLUSTER.

Shared Disk Settings Director data files and operation log files must be placed on the shared disk in order to use the director server failover function with Shunsaku.

Before setting up the Shunsaku environment, register the shared disks as GDS resources so that the shared disks can be used as file systems from Shunsaku.

Also, specify “switchable disk” as the usage for the shared disks and use them as file systems.

The following table explains the items that must be set up in order to register shared disks with the cluster application as GDS resources.

6-19


Table 6-3 Items to be set up when Registering Shared Disks as GDS Resources

Location Item Setting

Disk Class attribute settings

MONITORONLY Select ‘No’. This item specifies whether the status is notified to the cluster application when the disk fails. If ‘No’ is selected, the GDS resource will vary offline and the cluster application will fail over.

Exclusive use Select ‘Yes’. Disk Class usage settings HotStandby operation Select ‘No’.

Note

• Refer to creating GDS resources in the PRIMECLUSTER Installation and Operation Guide for more information on creating GDS resources.

Inherited IP Address Settings Shunsaku uses inherited IP addresses for the connections between the conductor, directors, sorters, and searchers. Accordingly, finish setting the inherited IP addresses so that Shunsaku can be used before setting up the Shunsaku environment.

Refer to the section on creating inherited network resources in the PRIMECLUSTER Installation and Operation Guide for more information on inherited IP addresses.

Installing Shunsaku Install Shunsaku on all servers.

The installation procedure is the same as that for customizing the server configuration.

Refer to the Installation Guide for more information on the installation procedure.

Setting Up Shunsaku This section explains how to set up Shunsaku.

Setting up the Director Server Set up the director server on the active server.

Setting up the director server is the same as the normal setup procedure, except for the following points:

• Editing the system environment file

• Editing the director environment file

• Copying environment files

Refer to the Installation Guide for more information on the setup procedure.

6-20


Editing the System Environment File Specify the inherited IP address of the director server in the following execution parameters in the system environment file:

• Conductor

• Director

• Sorter

Editing the Director Environment File Director data files and operation log files must be stored on shared disks.

Specify a directory on a shared disk for the following execution parameters in the director environment file:

• DataFileFolder

• OperationLogFolder

Note

• If director data files and operation log files are damaged at the same time, the latest data cannot be recovered.

Accordingly, specify different shared disks for the locations of the operation log files and the director data files.

Copying Environment Files For each environment file, the same settings must be used for the active server and the standby server. After the setup for the active server is complete, copy the following environment files over to the standby server:





Setting up Search Servers The procedure for setting up search servers is the same as for normal (non-cluster) systems.

Refer to the Installation Guide for more information on the setup procedure.

Activating GDS Resources Make the shared disk recognizable from the active server so that director data files and operation log files can be stored. Start the GDS resource for the shared disk.

The procedure for starting GDS resources is as follows:

1. Start the Global Disk Services window for the cluster system.

2. Select the node from the ‘GDS Configuration’ tree and display the Volume Information frame.

3. Select the GDS resource to be started from the Volume Information menu, and then select Start Volume from the Operation menu.

Note

• Refer to the PRIMECLUSTER Installation and Operation Guide for more information on activating GDS resources.

6-21


Creating an Area for Storing Director Data Files Create on a shared disk the folder for storing director data files that is specified in the DataFileFolder parameter in the director environment file.

Perform this operation on the active server.

If the server does not recognize the shared disk, start the GDS volume and mount the file system on the shared disk. cd /share1/shunsk mkdir shunsaku

Creating an Area for Storing Operation Log Files Create on a shared disk the folder for storing operation log files that is specified in the OperationLogFolder parameter in the director environment file. Perform this operation on the active server.

If the server does not recognize the shared disk, start the GDS volume and mount the file system on the shared disk. cd /share2/shunsk mkdir opelog

Deactivating GDS Resources Deactivate any GDS resources that have been activated.

The procedure for stopping GDS resources is as follows:

1. Start the Global Disk Services window for the cluster system.

2. Select the node from the ‘GDS Configuration’ tree and display the Volume Information frame.

3. Select the GDS resource to be stopped from the Volume Information menu, and from the Operation menu select Stop Volume.

Note

• Refer to the PRIMECLUSTER Installation and Operation Guide for more information on deactivating GDS resources.

Creating Cluster Applications This section explains how to create cluster applications on PRIMECLUSTER.

Create Shunsaku as a cluster application for standby operations. If there are multiple director servers, perform this procedure for each director server.

Note

• Refer to the section on creating cluster applications in the PRIMECLUSTER Installation and Operation Guide for more information on creating cluster applications.

6-22


Creating and Setting Up a Cmdline Resource Link Shunsaku to the cluster application so that starting and stopping the cluster application starts and stops Shunsaku. Start, Stop, and Check scripts are provided for this reason.

Create a Cmdline resource and set Start, Stop, and Check scripts with this Cmdline resource.

The following table shows where each script is stored.

Table 6-4 Location of Shunsaku Scripts

Script Storage location

Start script /opt/FJSVshnsk/etc/HA/shunstart

Stop script /opt/FJSVshnsk/etc/HA/shunstop

Check script /opt/FJSVshnsk/etc/HA/shuncheck

Set the following items for the Cmdline resource settings.

Table 6-5 Cmdline Resource Settings


Start script /opt/FJSVshnsk/etc/HA/shunstart Shunsaku system name

Stop script /opt/FJSVshnsk/etc/HA/shunstop Shunsaku system name

Cmdline settings

Check script /opt/FJSVshnsk/etc/HA/shuncheck Shunsaku system name

Creating Cluster Applications Create a cluster application from the Create userApplication menu.

Set the following items in the settings for creating the cluster application.

Table 6-6 Settings for Creating the Cluster Application


userApplication name and operation configuration settings

Operation mode Select ‘Standby’

SysNode settings Selected SysNode Set the director server as the SysNode name that will execute the cluster application. Select the active server first and then the standby server.

AutoStartUp Select ‘Yes’.

AutoSwitchOver Set “HostFailure|ResouceFailure|ShutDown”.

StandbyTransitions Set “ClearFaultRequest|Startup|SwitchRequest”

Attribute settings

HaltFlag Select ‘Yes’.

Resource selection Selected Resource Select and add the resources used by the cluster application.

6-23


Starting Cluster Applications Start the cluster application so that Shunsaku failover operations can commence.

To start the cluster application, start the Cluster Admin window for the cluster system, then start RMS by clicking the rms tab and then from the Cluster Admin toolbar, select Start RMS..

Note

• To check if Shunsaku has started correctly, refer to the system log (syslog) for the active server and the operation log files for the Shunsaku conductor, sorters, and directors.

• If Shunsaku has been set up to start when the cluster application is varied online, stop Shunsaku by varying the cluster application offline.

Refer to the PRIMECLUSTER Installation and Operation Guide for more information on starting cluster applications.

Checking Cluster Application Operation The failover function can be used when the cluster application finishes starting up.

It is recommended that users check whether the environment has been set up correctly by performing a failover before actual operations commence.

Use the following procedure to cause the cluster application to fail over.

1. Start the Cluster Admin window for the cluster system, and then click the rms tab.

2. Right-click on the cluster application object in the rms tree view, and from the pop-up menu select Failover to induce failover.

Stopping the Cluster Applications In order to stop the failover function, stop the cluster application.

To stop the cluster application, open the ‘Cluster Admin’ window for the cluster system, then stop RMS by clicking the rms tab from the Cluster Admin toolbar, and selecting Stop RMS.

Note

• To check if Shunsaku has started correctly, refer to the system log (syslog) for the active server and the operation log files for the Shunsaku conductor, sorters, and directors.

• Refer to the PRIMECLUSTER Installation and Operation Guide for more information on stopping cluster applications.

Recovery Procedure for Shared Disk Faults If a shared disk used by both the active server and the standby server fails, the cluster system will detect the fault and cause the cluster group (or cluster application) to fail over.

In such cases, jobs can be recommenced by resolving the problem with the faulty resource (such as replacing the hardware) and then restarting the cluster group (or cluster application).

If a shared disk where director data files or operation log files are located fails, the faulty resource must be restored before data can be restored.

This section explains the following restoration procedures.

• For MSCS:

− Recovery procedure for director data files

− Recovery procedure for operation log files

6-24


• For PRIMECLUSTER:

− Recovery procedure for director data files

− Recovery procedure for operation log files

Recovery Procedure for Director Data Files (MSCS) If the shared disk where director data files are located fails, replace the disk, and then recover the director data files on the active server.

The following diagram shows the procedure for recovering director data files when MSCS is used.

Note

• Before recovering director data files, stop all applications to prevent data searches and updates from occurring.

Figure 6-12 Recovery Procedure for Director Data Files (MSCS)

1. Check the status of resources. Check the resource status using the cluster administrator. Vary any online Shunsaku resources offline.

2. Replace the shared disk. Replace the shared disk where the fault has occurred and restore its settings.

3. Activate the shared disk. Vary the shared disk resource online using the cluster administrator. Make sure that the status of the resource is online.

4. Create an area for storing director data files. Create a folder for storing director data files on the file system on the shared disk. Refer to Creating an Area for Storing Director Data Files for more information.

6-25


5. Ensure that all applications are stopped, then activate Shunsaku resources. Vary Shunsaku resources online using the cluster administrator, and then start Shunsaku. Once Shunsaku has started, make sure that the status of all resources is online.

6. Recover the director data files. Recover the director data files by executing the shundrecover command from the Command Prompt on the active director server.

a) Make a recovery start declaration by executing the shundrecover command with the -b option specified. shundrecover -s director-identifier –b

b) Restore the backup data for the director data files to the director data file folder using the copy function provided by the operating system. Copy /b backup-data director-data-file-folder

c) Make a recovery end declaration by executing the shundrecover command with the -e option specified. shundrecover -s director-identifier –e

This will cause Shunsaku to restore the director data files to their latest state by using the operation log files.

An example is shown below. − director-identifier: “shund1”

− backup-data: “Z:\backup\shund1\”

− director-data-file-folder: “Y:\shunsk\shund1” shundrecover -s shund1 –b copy /b Z:\backup\shund1\* Y:\shunsk\shund1 shundrecover -s shund1 –e

These commands will complete the restoration of the director data files, allowing jobs to recommence.

6-26


Recovery Procedure for Operation Log Files (MSCS) If the shared disk where operation log files are located fails, replace the disk, and then recover the operation log files.

The following diagram shows the recovery procedure:

Figure 6-13 Recovery Procedure for Operation Log Files (MSCS)

1. Check the status of resources. Check the resource status using the cluster administrator. Vary any online Shunsaku resources offline.

2. Replace the shared disk. Replace the shared disk where the fault has occurred and restore its settings.

3. Activate the shared disk. Vary the shared disk resource online using the cluster administrator. Make sure that the status of the resource is online.

4. Create an area for storing operation log files. Create a directory for storing operation log files on the file system on the shared disk if one does not exist. Refer to Creating an Area for Storing Operation Log Files for more information.

5. Activate Shunsaku resources. Vary Shunsaku resources online using the cluster administrator, and then start Shunsaku. Once Shunsaku has started, make sure that the status of all resources is online.

6-27


6. Recreate the operation log files. Recover the operation log files by executing the shundrecover command from the active director server.

a) Recover operation log files by using the shundrecover command. shundrecover -s director-identifier –l

b) After the recovery of the operation log files is complete, use the shundbackup command and the operating system copy command to back up the director data files. shundbackup –s director-identifier –b copy /b director-data-file-folder backup-data shundbackup –s director-identifier –e

Note

After the recovery of the operation log files is complete, be sure to take a backup of the director data files. If a backup is not taken, an error will occur if recovery processing is performed subsequently.

An example is shown below.

− director-identifier: “shund1”

− backup-data: “Z:\backup\shund1\”

− director-data-file-folder: “Y:\shunsk\shund1” shundrecover -s shund1 –l shundbackup –s shund1 –b copy /b Y:\shunsk\shund1\* Z:\backup\shund1\ shundbackup –s shund1 –e

These commands will complete the recovery of the operation log files, allowing jobs to recommence.

6-28


Recovery Procedure for Director Data Files (PRIMECLUSTER) If the shared disk where director data files are located fails, replace the disk, and then recover the director data files on the active server.

The following diagram shows the procedure for recovering director data files when PRIMECLUSTER is used:

Note


Figure 6-14 Recovery Procedure for Director Data Files (PRIMECLUSTER)

1. Check the status of the cluster application. Check the status of the cluster application from PRIMECLUSTER's Cluster Admin window. If the Shunsaku cluster application is online, stop it by right-clicking on the cluster application object in the rms tree view of the Cluster Admin window and selecting offline from the pop-up menu.

2. Replace the shared disk. Replace the shared disk where the fault has occurred. Then restore the configuration status for the volume.

3. Activate the GDS resource. Activate the GDS resource to restore the director data files. This will enable the file system on the shared disk to be recognized form the active server. Refer to 'Activating GDS Resources' in Setting Up Shunsaku for more information on the procedure for activating the GDS resource.

6-29


4. Create an area for storing director data files. Create a directory for storing director data files on the file system on the shared disk if one does not exist. Refer to Creating an Area for Storing Director Data Files for more information.

5. Deactivate the GDS resource. Deactivate the GDS resource that was activated in Step 3. Refer to Deactivating GDS Resources for more information on the procedure for deactivating the GDS resource.

6. Make sure that all applications are stopped, then activate the Shunsaku cluster application. Open the RMS main window by clicking on the rms tab of the main window in PRIMECLUSTER's Cluster Admin window. Then start the cluster application by right-clicking on the cluster application object in the RMS tree and selecting online from the pop-up menu.

7. Recover the director data files. Recover the director data files by executing the shundrecover command from the active director server.

a) Make a recovery start declaration by executing the shundrecover command with the -b option specified. shundrecover -s director-identifier –b

b) Copy the backup data for the director data files to the director data file folder using the copy function provided by the operating system. cp backup-data director-data-file-folder

c) Make a recovery end declaration by executing the shundrecover command with the -e option specified. shundrecover -s director-identifier –e

This will cause Shunsaku to restore the director data files to their latest state by using the operation log files.

An example is shown below. − director-identifier: “shund1”

− backup-data: “/backup/shund1/”

− director-data-file-folder: “/share1/shunsk/shund1” shundrecover -s shund1 –b cp /backup/shund1/* /share1/shunsk/shund1/ shundrecover -s shund1 –e

These commands will complete the restoration of the director data files, allowing jobs to recommence.

6-30


Operation Log File Recovery Procedure (PRIMECLUSTER) If the shared disk where operation log files are located fails, replace the disk, and then recover the operation log files. The following diagram shows the recovery procedure. Perform the restoration operations on the active server.

Figure 6-15 Recovery Procedure for Operation Log Files (PRIMECLUSTER)

1. Check the status of the cluster application. Check the status of the cluster application from PRIMECLUSTER's Cluster Admin window. If the Shunsaku cluster application is online, stop it by right-clicking on the cluster application object in the rms tree view of the Cluster Admin window and selecting offline from the pop-up menu.

2. Replace the shared disk. Replace the shared disk where the fault has occurred. Then restore the configuration status for the volume.

3. Activate the GDS resource. Activate the GDS resource to recover Shunsaku operation log files. This will enable the file system on the shared disk to be recognized form the active server. Refer to 'Activating GDS Resources' in Setting Up Shunsaku for more information on the procedure for activating the GDS resource.

4. Create an area for storing operation log files. Create a directory for storing operation log files on the file system on the shared disk if one does not exist. Refer to Creating an Area for Storing Operation Log Files for more information.

6-31


5. Deactivate the GDS resource. Deactivate the GDS resource that was activated in Step 3. Refer to Deactivating GDS Resources for more information on the procedure for deactivating the GDS resource.

6. Start the cluster application. Open the RMS main window by clicking on the rms tab of the main window in PRIMECLUSTER's Cluster Admin window. Then start the cluster application by right-clicking on the cluster application object in the RMS tree and selecting online from the pop-up menu.

7. Recover the operation log files. Recover the operation log files by executing the shundrecover command from the active director server. a) Recover operation log files using the shundrecover command. shundrecover -s director-identifier –l

b) After the recovery of the operation log files is complete, use the shundbackup command and the operating system copy command to back up the director data files. shundbackup –s director-identifier –b cp backup-data director-data-file-folder shundbackup –s director-identifier –e

Note After the recovery of the operation log files is complete, be sure to take a backup of the director data files. If a backup is not taken, an error will occur if recovery processing is performed subsequently.

An example is shown below.

− director-identifier: “shund1”

− director-data-file-folder: “/share1/shunsk/shund1”

− backup-data: “/backup/shund1/” shundrecover –s shund1 -l shundbackup –s shund1 –b cp /share1/shunsk/shund1/* /backup/shund1/ shundbackup –s shund1 –e

These commands will complete the recovery of the operation log files, allowing jobs to recommence.

6-32

Failover Operations for Search Servers

Failover Operations for Search Servers This section explains the following points about the failover function for search servers.

• Designing Failovers for Search Servers

• Setting Up Failovers for Search Servers

• Failover Operations

Designing Failovers for Search Servers As part of the design of failovers for search servers, decide how many search servers to prepare for alternative searchers, depending on the degree of availability demanded. Place as many alternative searchers on these search servers as there are regular searchers on the search servers used normally.

By setting information for these search servers and alternative searchers in the system environment file, the alternative searchers can be used from each director.

6-33


Figure 6-16 Failovers for Search Servers

Alternative searchers can be allocated so that they can be used from any director in the Shunsaku system, or allocated so that they can only be used from particular directors.

Additionally, settings can be made so that if a hardware fault occurs on a search server with multiple searchers then a single searcher error is regarded as an error with the entire search server.

6-34


Setting Up Failovers for Search Servers There are two ways of setting up failovers for search servers.

• Installing and Setting Up Shunsaku Systems Including Alternative Searchers

• Adding Alternative Searchers

Installing and Setting Up Shunsaku Systems Including Alternative Searchers The following diagram explains the procedure for installing and setting up Shunsaku systems including alternative searchers.

Figure 6-17 Installing Shunsaku Systems with Alternative Searchers

1. Perform the following operations on the director server.

− Install the conductor, directors, and sorters

− Set up the conductor

− Set up directors

− Set up sorters The procedure for installing and setting up Shunsaku on the director server is the same as for normal (non-cluster) systems. Refer to the Installation Guide for more information on installing and setting up Shunsaku on the director server.

2. Install and set up searchers on the search server. The procedure for installing and setting up Shunsaku on the search server is the same as for normal (non-cluster) systems. Refer to the Installation Guide for more information on installing and setting up Shunsaku on search servers.

3. Install the alternative searchers on search servers other than the ones that are being used normally. The procedure for installing and setting up alternative searchers is the same as for regular searchers.

6-35


4. Edit and distribute the system environment file. Edit the system environment file on the director server where the conductor is located. Edit the system environment file by adding information for the alternative searchers. In the alternative searcher information, add AlternativeSearcher parameters, and specify searcher identifiers for the alternative searchers, as well as their host name or IP address and the port number for receiving requests from directors. Example #--- Server information for alternative searchers---# AlternativeSearcher shuns1w SchWSvr01 33501 shund1 AlternativeSearcher shuns2w SchWSvr02 33501 shund2 AlternativeSearcher shuns3w SchWSvr03 33501 #---Error recognition range in the event of a searcher error---# SearcherFailureEscalation 1

After editing the system environment file, distribute it to all servers.

Note

− Refer to 'System Environment Files' in Appendix A, for more information on the execution parameters in the system environment file.

− If the SearcherFailureEscalation parameter is specified, degradation processing is also performed in search server units.

Place the system environment file in the following folder of all servers: Shunsaku installation folder\etc\system

Place the system environment file in the following folder of all servers: /etc/opt/FJSVshnsk/etc/system

5. Start Shunsaku.

Alternative searchers will start running in the same way as searchers operating normally when the shunsysstart command is executed. shunsysstart -n Shunsaku system name

Refer to 'Starting Shunsaku' in Chapter 3, Operation, for more information on starting Shunsaku.

Note

Services must be registered before starting alternative searchers.

6-36


Adding Alternative Searchers Alternative searchers can be added while Shunsaku is operating. The procedure for adding alternative searchers is as follows:

Figure 6-18 Adding Alternative Searchers

1. Install and set up alternative searchers. Install the alternative searchers on search servers other than the ones that are being used normally. The procedure for installing and setting up alternative searchers is the same as for regular searchers. Refer to 'Installing and Setting Up Additional Search Servers' in the Installation Guide for more information on installing and setting up searchers.

2. Edit and distribute the system environment file. Add AlternativeSearcher parameters to the system environment file, and specify the searcher identifiers for the alternative searchers, as well as their host names or IP addresses and port numbers. Distribute the edited system environment file to all servers. To limit the directors that each alternative searcher can use, add director identifiers to the end of the parameter specification. Example #--- Server information for alternative searchers---# AlternativeSearcher shuns4w SchWSvr04 33501 shund1 AlternativeSearcher shuns5w SchWSvr05 33501 shund2 AlternativeSearcher shuns6w SchWSvr06 33501

After editing the system environment file, distribute it to all servers. Refer to 'System Environment Files' in Appendix A, for more information on the execution parameters in the system environment file.

6-37


Note

Distribute the system environment file to the "\etc\system" folder in the Shunsaku installation folder on all servers.

Distribute the system environment file to the /etc/opt/FJSVshnsk/etc/system directory on all servers.

3. If host names have been specified in the system environment file without using IP addresses, specify the IP addresses and host names of the additional search servers in the following file on the director server where the conductor is located:


/etc/hosts

4. Dynamically add the alternative searchers. Execute the shunsyscfgedit command on all director servers to dynamically add the information for the alternative searchers specified in the AlternativeSearcher parameter in the system environment file. shunsyscfgedit -n Shunsaku system name

5. Start the alternative searchers.

Before starting the alternative searchers, register the service by executing the shunsservice command on the search servers where the alternative searchers are located. shunsservice -n Shunsaku system name -s alternative searcher identifier

Start the alternative searchers. Execute the shunsysstart command on the search servers where the alternative searchers are located. shunsysstart -n Shunsaku system name

6-38


Failover Operations When failovers occur, the following messages are output to the event log or the system log (syslog) on the director server. shn01903i: Searcher connection was broken. (192.168.10.1,33501) (director) [shund1] Shunsaku System Name=shunsaku shn01904i: Switching to alternative searcher. (director) [shund1] Shunsaku System Name=shunsaku shn01110i: Connected to searcher. (192.168.10.20,33501) (director) [shund1] Shunsaku System Name=shunsaku

To return both the failed searcher and the alternative searcher operating as a result of the failover back to their original configuration, after the failed searcher has been restored perform the following procedure.

1. Start the restored searcher by executing the shunsysstart command on the server where the searcher is located. shunsysstart -n Shunsaku system name

When the searcher is started, the search data on the alternative searcher is automatically sent back to the restored searcher. Once the search data has been resent, the alternative searcher that has been operating enters an alternative standby state to prepare for other searcher failures.

The following operations cannot be executed while search data is being automatically resent from the alternative searcher: − Update operations from applications

− shundclear command

− shundresend command

− shundcds command

− shundimport command

− shundexport command

− shundbackup command

− shundrecover command

− shunsysstop command

− shunsyscfgedit command

This restores the system to its original configuration and restarts failover operations.

6-39


6-40

Chapter 7 Backup and Recovery

This chapter explains how to perform backup and recovery operations.

• Overview of Backup and Recovery

• Backing Up and Restoring the Shunsaku Environment Files

• How to Recover to a Point Immediately Prior to a Malfunction

• How to Recover to the Latest Backup

• How to Recover to a Specified Point

• Recovering the Operation Log Files

7-1

Chapter 7: Backup and Recovery

Overview of Backup and Recovery This section provides an overview of Shunsaku’s backup and recovery functions.

Backup “Backup” refers to the process of saving the data needed for Shunsaku operation to guard against data loss caused by disk corruption and so on.

To restore data, the following files must be backed up:

• Shunsaku environment files

• Director data files

Shunsaku Environment Files Back up the Shunsaku environment files if any of the execution parameters in the files have been changed.

The files to be backed up are stored in the following locations:

Environment file Location

System environment file Shunsaku installation folder\Shunsaku\etc\system\Shunsaku system name.cfg

Conductor environment file Shunsaku installation folder\Shunsaku\etc\conductor\conductor-identifier.cfg

Director environment file Shunsaku installation folder\Shunsaku\etc\director\director-identifier.cfg

Sorter environment file Shunsaku installation folder\Shunsaku\etc\sorter\sorter-identifier.cfg

Searcher environment file Shunsaku installation folder\Shunsaku\etc\searcher\searcher-identifier.cfg

Environment file Location

System environment file /etc/opt/FJSVshnsk/etc/system/Shunsaku system name.cfg

Conductor environment file /etc/opt/FJSVshnsk/etc/conductor/conductor-identifier.cfg

Director environment file /etc/opt/FJSVshnsk/etc/director/director-identifier.cfg

Sorter environment file /etc/opt/FJSVshnsk/etc/sorter/sorter-identifier.cfg

Searcher environment file /etc/opt/FJSVshnsk/etc/searcher/searcher-identifier.cfg

Note

Refer to Backing Up and Restoring the Shunsaku Environment Files for more information.

7-2

Overview of Backup and Recovery

Director Data Files Perform backups according to a backup cycle that matches the operations performed by a user.

Back up all files in the folder that has the name of the director identifier and is located in the directory specified by the DataFileFolder parameter in the director environment file.

Note

Refer to Backup Cycle for more information on the backup cycle.

Recovery “Recovery” refers to the process of using backup data to restore Shunsaku to a state in which it can start operating again.

In Shunsaku, recovery can be used to restore the following files:

• Shunsaku environment files

• Director data files:

− Recovery to a point immediately preceding a malfunction

− Recovery to a backup point

− Recovery to a specified point

• Operation log files

Select the operation that matches the business objectives.

Relationship between Operation Log Files and Backup Data The operation log file accumulates Shunsaku update operations as update log data.

When Shunsaku data is updated, update log data will be written to the operation log file. This makes it possible to restore the system to any point after a backup has been made, and to use the API to perform update processes while a backup process is in progress.

The update log data accumulated in the operation log file is discarded after the backup process for the director data file is complete and a backup end declaration is issued. The operation log file therefore contains the log data that has accumulated between the latest backup and the current time.

In the following example, the director data file is backed up after the day’s operations have finished.

7-3


Figure 7-1 Example of Backing up Director Data Files

During recovery, the most recently backed up data together with the update log data in the operation log file is applied. This combination makes it possible to recover to any point between the time of the last backup and a time immediately preceding the malfunction.

It is important to ensure that the data to be recovered (director data) and the data needed at recovery time (backup data and operation log) are saved to different disks to prevent both types of data from being damaged at the same time.

If both the director data file and the operation log file are damaged at the same time, the update log data accumulated since the backup was taken will be lost, and it will only be possible to restore the system to the state that existed when the backup was taken.

Furthermore, if the director data file and the backup data are both lost at the same time, the information needed for recovery will also be lost, and it will not be possible to restore the director data file.

7-4

Overview of Backup and Recovery

Backup Cycle The interval between regular backups is referred to as the ‘backup cycle’.

If a backup is taken every morning, the backup cycle is one day.

The backup cycle should be determined at design time, taking the following factors into account:

• Recovery time

The information that accumulates in the operation log file is used for restoring the system. If a large number of operations accumulate in the operation log file between backups, the large size of the operation log may cause any recovery process to take a long time, which will delay the time before normal operations can recommence.

For this reason, the time needed to perform a recovery operation should be taken into account when determining the backup cycle.

• Capacity of the disk where the operation log file is located

The backup cycle should be used to determine the size of the disk where the operation log file will be stored. The operation log file contains update log data that accumulates between the end of one backup for the director data file and the beginning of the next, so the operation log file must be large enough to store the update log data with room to spare. For example, if a backup is taken every day, the capacity of the operation log file needs to be larger than that is needed to store a day’s update log data.

Note

− If the directory where the operation log file is located is too small, it will not be possible to update data. If this is the case, back up the director data file, discard the operation log file and then shorten the director data file backup cycle. Refer to 'The Size of the Operation Log Files' in Appendix C, Estimating Resources, for more information on operation log file capacity.

Backup Start Declaration and End Declaration An operating system command or disk device copy function is used to back up director data files.

Backup start and end declarations are used to synchronize the data update process and the director data file backup.

This operation prevents data that is in the process of being updated from being generated during a backup, which in turn ensures the consistency of the backup data.

Note

• It is recommended that the series of backup operations that must take place between the backup start declaration and the backup end declaration be performed during a period when there is as little update activity going on as possible. This will minimize the time required to complete the backup.

Confirming Backup Status The date and time of the latest backup can be verified with the shundstate command.

Refer to 'Monitoring Director Status' in Chapter 8, Maintenance, for more information on the content displayed by the shundstate command.

7-5


Backing Up and Restoring the Shunsaku Environment Files

This section explains how to back up and restore the Shunsaku environment files.

The following environment files are used by Shunsaku:





• Searcher environment file

Note

• If there is a malfunction on the disk containing the directories where the Shunsaku environment files are stored, Shunsaku will no longer be able to operate. For this reason, it is necessary to have backups of the latest Shunsaku environment files at all times.

Backup It is necessary to back up a Shunsaku environment file as soon as it has been changed.

The following procedure is used to back up Shunsaku environment files.

Figure 7-2 Backing up Shunsaku Environment Files

1. Use the following operating system commands to back up Shunsaku environment files.

copy command, etc.

cp command, etc.

7-6

Backing Up and Restoring the Shunsaku Environment Files

Recovery Shunsaku environment files can be recovered once they have been backed up.

Recover Shunsaku environment files if a disk error occurs in the following directory where the Shunsaku environment files are stored.

Shunsaku installation folder\Shunsaku\etc

/etc/opt/FJSVshnsk/etc

The following procedure is used to recover Shunsaku environment files.

Figure 7-3 Recovering Shunsaku Environment Files

1. Replace the faulty disk.

2. Use an operating system command to restore the backup of the Shunsaku environment files.

copy command, etc.

cp command, etc.

7-7


How to Recover to a Point Immediately Prior to a Malfunction

The following two types of information are used to restore director data files to a point in time immediately prior to a malfunction:

• Director data file backup data

• Operation log file

If Shunsaku data has been updated by an application that uses the Shunsaku APIs, it is possible to recover the director data files to a point immediately preceding any malfunction.

Figure 7-4 Performing Recovery to a Point Immediately Preceding a Malfunction

7-8


Preparation To recover director data files to a point immediately preceding a malfunction, it is necessary to set the operation log files in the OperationLogFolder parameter in the director environment file.

Refer to 'Director Environment Files' in Appendix A for more information on the execution parameters in the director environment file.

To prevent both the director data files and the operation log files from being damaged at the same time, which would make it impossible to perform an up-to-date recovery, always configure settings so that the operation log files and the director data files are stored on different disks.

Backup To recover director data files to a point immediately preceding a malfunction, it is necessary to have backed up the director data files beforehand.

The following procedure is used to take a backup that can be used to recover from a malfunction.

Figure 7-5 Backing up Director Data Files

1. Execute the shundbackup command with the –b option to declare the start of the backup. shundbackup -s director identifier –b

2. Back up the director data files. The directory specified by the DataFileFolder parameter in the director environment file contains a directory with the same name as the director identifier. Back up all the files from that directory. Use the following operating system commands or a disk device copy function to perform the backup:

copy command, etc.

cp command, etc.

7-9


3. Execute the shundbackup command with the –e option to declare the end of the backup. shundbackup -s director identifier –e

Notes

• Back up the director data files to a disk other than the disk where the director data files are located.

• Back up the director data files if any of the following commands have been executed:

− if the shundimport command has been executed with the –n option to import or delete a record without collecting the operation log file

− if the shundrecover command has been executed with the –p option to recover to a backup point or a specified point

• If, after the backup start declaration has been executed, an error occurs during a backup that uses an operating system command or a disk device copy function, do not execute the backup end declaration. Instead, execute the shundbackup command with the –c option to cancel the backup start declaration and then, after eliminating the cause of the error, execute the backup start declaration again.

The date and time of the latest backup can be verified with the shundstate command. Refer to 'Monitoring Director Status' in Chapter 8, Maintenance for more information on the content displayed by the shundstate command.

7-10


Recovery If a malfunction occurs in the disk containing the director data files while the system is in operation, Shunsaku will output a “shn30306u” message to the event log or system log (syslog) of the director server.

The following diagram shows an example of a recovery operation performed when the disk containing the director data files malfunctions.

Figure 7-6 Performing Recovery when the Disk Containing Director Data Files Malfunctions

The following procedure is used to restore the director data files to the state that existed immediately prior to the malfunction.

Note


7-11


Figure 7-7 Procedure for Restoring Director Data Files

1. Check the event log or system log (syslog) to identify the location of the fault. If a disk fault occurs while the director data file directory is being accessed, Shunsaku will output the following message to the event log or system log (syslog): shn30306u: An I/O error has occurred with a director data file. file=file name System call=function name errno=error number (director) [director identifier name] Shunsaku System Name=Shunsaku system name

Use the message to confirm the name of the device where the disk fault occurred or the full pathname of the file.

2. Execute the shundrecover command with the –b option to declare the start of the recovery process.

Note

If the operating system needs to be restarted during disk replacement, restart the operating system, then perform Step 3 before this step. Ensure that all applications are stopped, start Shunsaku, and then perform this step. shundrecover -s director identifier –b

3. Replace the faulty disk and recreate the directory that will contain the director data files.

7-12


4. Use an operating system command or a disk device copy function to restore the backup data of the director data files.

copy command, etc.

cp command, etc.

5. Execute the shundrecover command with the –e option to declare the end of the recovery process. The recovery end declaration will use the operation log file to restore the director data file to the state that existed immediately before the problem occurred.

shundrecover -s director identifier –e

Note The shundrecover command recovers only the backup data of the director data file that was backed up using the shundbackup command.

7-13


How to Recover to the Latest Backup If a malfunction occurs in a system that does not perform data updates and is only used for viewing data, it may be sufficient to restore the system to the state that existed when the director data files were last backed up.

Even if the system does perform data updates, in some cases the system can be fully restored by recovering to the latest backup point and then re-executing the updates that took place after the backup.

In such systems, it is possible to recover from problems by performing regular backups without using operation log files.

This section explains how to perform backup and recovery without using operation log files.

Figure 7-8 Recovery to the Latest Backup

Preparation To recover director data files to a backup point, there is no need to set up operation log files.

Confirm that the OperationLogFolder parameter is not specified in the director environment file.

Note


7-14

How to Recover to the Latest Backup

Backup To recover director data files to a backup point, it is necessary to have backed up the director data files beforehand.

The following example shows how to back up the director data files after the day’s operations have finished.

Figure 7-9 Backing Up Director Data Files

The following procedure is used to take a backup that can be used to recover from a malfunction.

7-15


Figure 7-10 Back Up Procedure

1. Execute the shundbackup command with the –b option to declare the start of the backup. shundbackup -s director identifier –b


copy command, etc.

cp command, etc.


Note

• Back up the director data files to a different disk from that where the directory containing the director data files is located.

• If after the backup start declaration has been executed, an error occurs during a backup that uses an operating system command or a disk device copy function, do not execute the backup end declaration. Instead, execute the shundbackup command with the –c option to cancel the backup start declaration and then, after eliminating the cause of the error, execute the backup start declaration again.

• The date and time of the latest backup can be verified with the shundstate command. Refer to 'Monitoring Director Status' in Chapter 8, Maintenance for more information on the content displayed by the shundstate command.

7-16


Recovery If a malfunction occurs in the disk containing the director data files while the system is in operation, Shunsaku will output a “shn30306u” message to the event log or system log (syslog) of the director server.

The following diagram shows an example of a recovery operation performed when the disk containing the director data files malfunctions.

Figure 7-11 Performing Recovery when the Disk Containing Director Data Files Malfunctions

The following procedure is used to recover the director data files to the state that existed at the previous backup.

Note


7-17


Figure 7-12 Performing Recovery to the Latest Backup

1. Check the event log or system log (syslog) to identify the location of the fault. If a disk fault occurs while the director data file directory is being accessed, Shunsaku will output the following message to the event log or system log (syslog): shn30306u: An I/O error has occurred with a director data file. file=file name System call=function name errno=error number (director) [director identifier name] Shunsaku System Name=Shunsaku system name


2. Execute the shundrecover command with the –b option to declare the start of the recovery process.

Note

If the operating system needs to be restarted during disk replacement, restart the operating system, and then perform Step 3 before this step. Ensure that all applications are stopped, start Shunsaku, and then perform this step. shundrecover -s director identifier –b

3. Replace the faulty disk and recreate the directory that will contain the director data files.

7-18



copy command, etc.

cp command, etc.

5. Execute the shundrecover command with the –e option and the –p option to declare the end of the recovery process. The recovery end declaration will restore the director data file to the state that existed at the previous backup. shundrecover -s director identifier -e –p

Note

• The shundrecover command only targets the backup data of the director data file that was obtained using the shundbackup command.

7-19


How to Recover to a Specified Point If data is lost due to human error, it is possible to restore the director data files to the state that existed at a specified point in time.

The following two types of information are used to restore the director data files to a specified point:

• Director data file backup data

• Operation log files

The recovery process can restore the data to any point following the latest backup.

Note

The date and time of the latest backup can be verified with the shundstate command. Refer to 'Monitoring Director Status' in Chapter 8, Maintenance for more information on the content displayed by the shundstate command.

Preparation To recover director data files to a specified point, it is necessary to set the operation log files in the OperationLogFolder parameter in the director environment file.


Backup To recover director data files to a specified point, it is necessary to have backed up the director data files beforehand.

The backup procedure used here is the same as the backup procedure used for recovering to a point immediately preceding a malfunction. Refer to Backup for more information.

7-20

How to Recover to a Specified Point

Recovery If data is lost during operation, the lost data can be recovered by returning the director data to the state that existed prior to the loss.

The following example shows the recovery procedure used when data has been lost by running an application accidentally.

Figure 7-13 Performing Recovery when Data Loss Occurs

The following procedure is used to recover the director data files to a specified point in time.

Note


7-21


Figure 7-14 Procedure for Recovery to a Specified Point in Time

1. Execute the shundrecover command with the –b option to declare the start of the recovery process. shundrecover -s director identifier –b


copy command, etc.

cp command, etc.

3. Execute the shundrecover command with the –e option and the –p option followed by a date and time immediately prior to the time that the data loss occurred to declare the end of the recovery process. The recovery end declaration will use the operation log files to restore the director data files to the specified date and time. shundrecover -s director identifier -e -p 2005-01-01:12:00:00

Note

• The shundrecover command recovers only the backup data of the director data file that was backed up using the shundbackup command.

• The shundrecover command will generate an error if it is executed with the –p option specifying a date and time prior to the latest backup. Always specify a date and time after the latest backup.

7-22

Recovering the Operation Log Files

Recovering the Operation Log Files If a malfunction occurs in the disk containing the operation log files, Shunsaku will output a “shn30307u” message to the event log or the system log (syslog).

Note

If an I/O error occurs in the operation log files, it will not be possible to perform a recovery to a state immediately before the malfunction or to a specified point in time. In such cases, the user should immediately perform the following procedure to restore the operation log files and then take a backup.

The following procedure is used to recover the operation log files.

Figure 7-15 Procedure for Recovering Operation Log Files

7-23


1. Check the event log or system log (syslog) to identify the location of the fault. If a disk fault occurs while the operation log file is being accessed, Shunsaku will output the following message to the event log or system log (syslog): shn30307u:An I/O error has occurred with an operation log file. file=file name System call=function name errno=error number (director) [director identifier] Shunsaku System Name=Shunsaku system name


2. Replace the faulty disk and recreate the directory that will contain the operation log files.

3. Execute the shundrecover command with the –l option to recover the operation log files. shundrecover -s director identifier –l

4. Execute the shundbackup command with the –b option to declare the start of the backup process. shundbackup -s director identifier –b


copy command, etc.

cp command, etc.


Note

• Always back up the director data files after recovering the operation log files.

• This operation is necessary only if the OperationLogFolder parameter has been specified in the director environment file. In such cases, executing the shundrecover command with the –l option will result in an error.


7-24

Chapter 8 Maintenance

This chapter explains Shunsaku maintenance.

• Monitoring

• Recreating Lookup Data

• Responding to Errors

8-1

Chapter 8: Maintenance

Monitoring This section explains Shunsaku monitoring.

The status of the Shunsaku system and each process can be monitored.

The following information can be monitored:

• Operation information

• Setup information (operating environment)

• Connection information.

This information can also be output in CSV format.

The following diagram shows the configuration for Shunsaku monitoring information.

Figure 8-1 Configuration of Monitoring Information

8-2

Monitoring

Table 8-1 Monitoring Targets and Information

Monitoring target Monitoring information Explanation

System operation information Displays the operating status of each Shunsaku system process on the server.

System status

System setup information Displays information about the parameters specified in the system environment file.

Connection status Connection information Displays the status of the connection between applications and the Shunsaku system.

Conductor operation information Displays the operating status of the conductor, and information about directors and sorters under the conductor.

Conductor status

Conductor setup information Displays the operating environment that has been set up for the conductor.

Sorter operation information Displays the operating status of sorters.

Sorter status

Sorter setup information Displays the operating environment that has been set up for sorters.

Director operation information Displays the operating status of directors and information about the searchers under them.

Director status

Director setup information Displays the operating environment that has been set up for directors.

Searcher operation information Displays the operating status of searchers.

Searcher status

Searcher setup information Displays the operating environment that has been set up for searchers.

Monitoring System Status This section explains the details and procedures involved in monitoring the status of the system.

Monitoring System Operation Information System operation information can be used for the following purposes:

• Checking the configuration of processes on the server

• Checking the current operating status of the processes on the server

System operation information can be obtained by executing the shunsysstate command.

8-3


Example 1

The following example shows how information for the processes on the server is displayed. shunsysstate -n shunsaku –p Shunsaku shunsysstate 2005/01/01 19:01:00 (1) (2) (3) (4) (5) (6) Type ID Hostname State StateChanged Director Conductor shunc DirSvr1 ACTIVE 2005/01/01 10:50:11 Sorter shuno1 DirSvr1 ACTIVE 2005/01/01 10:50:11 Director shund1 DirSvr1 ACTIVE 2005/01/01 10:50:11 Searcher shuns01 DirSvr1 ACTIVE 2005/01/01 10:50:11 shund1 Searcher shuns02 DirSvr1 ACTIVE 2005/01/01 10:50:11 shund1

(1) Process type One of the following is displayed for process type: − Conductor

− Sorter

− Director

− Searcher

− AlternativeSearcher

(2) Process identifier

(3) Host name or IP address

(4) Process status One of the following is displayed for the process status: For conductors and sorters:

− ACTIVE: Running

− INACTIVE: Stopped

For directors:

− ACTIVE: Running

− ACTIVATING: Starting up

− DEGRADED: Degraded

− DEGRADING: Degrading

− RESENDING: Reallocating data

− SWITCHING: Switching to an alternative searcher

− CONDENSING: Optimizing data

− BACKUP: Backing up data

− RECOVER: Recovering data

8-4

Monitoring


− WAITING: Waiting for searchers to connect

For searchers and alternative searchers:

− ACTIVE: Running



− IDLE: On standby

(5) The date and time when the process entered its current state (YYYY/MM/DD hh:mm:ss)

(6) Identifier for the connection destination director This is displayed in the following situations. − When the process type is “Searcher”

or − When the process type is "AlternativeSearcher" and its operating status is "ACTIVATING"

or "ACTIVE"

Example 2

In the following example, information for the processes on the server is output in CSV format.

shunsysstate -n shunsaku -p -C > C:\shunwork\monitor\sysstate.csv

shunsysstate –n shunsaku -p -C > /home/shunsaku/monitor/sysstate.csv

Content of sysstate.csv "Type","ID","Hostname","State","StateChanged","Director" "Conductor","shunc","DirSvr1","ACTIVE","2005/01/01 10:50:11", "Sorter","shuno1","DirSvr1","ACTIVE","2005/01/01 10:50:11", "Director","shund1","DirSvr1","ACTIVE","2005/01/01 10:50:11", "Searcher","shuns01","DirSvr1","ACTIVE","2005/01/01 10:50:11","shund1" "Searcher","shuns02","DirSvr1","ACTIVE","2005/01/01 10:50:11","shund1"

• Column headings are output on the first line, and information for the processes on the server is output from the second line.

• Each record ends with a linefeed code.

Note

CSV files are output using the following format:

• String items are enclosed in double quotes.

• Numeric items are not enclosed in double quotes.

8-5


Monitoring System Setup Information The execution parameters specified in the system environment file can be checked.

System setup information can be obtained by executing the shunsysstate command.

Example 1

The following example shows how system setup information is displayed. shunsysstate -n shunsaku –i Shunsaku shunsysstate 2005/01/01 19:01:00 Parameter Value Conductor shunc DirSvr1 33101 Sorter shuno1 DirSvr1 33201 Director shund1 DirSvr1 33301 33302 33102 Searcher shuns01 DirSvr1 33501 shund1 Searcher shuns02 DirSvr1 33502 shund1 CharacterCode UTF-8 ANKmix 0 KNJmix 0 SkipChar "\s,\S,\n,\t" ExtendedQuery 0 SeparateChar Default SearcherFailureEscalation 0 MonitoringStyle 0

Note

Refer to 'System Environment Files' in Appendix A for more information on the execution parameters in the system environment file.

Example 2

In the following example, system setup information is output in CSV format.

shunsysstate -n shunsaku -i –C > C:\shunwork\monitor\syssetting.csv

shunsysstate -n shunsaku -i –C > /home/shunsaku/monitor/syssetting.csv

Content of syssetting.csv "Parameter","Value" "Conductor","shunc DirSvr1 33101" "Sorter","shuno1 DirSvr1 33201" "Director","shund1 DirSvr1 33301 33302 33102" "Searcher","shuns01 DirSvr1 33501 shund1" "Searcher","shuns02 DirSvr1 33502 shund1" "CharacterCode","UTF-8" "ANKmix",0 "KNJmix",0

8-6

Monitoring

"SkipChar","\s,\S,\n,\t" "ExtendedQuery",0 "SeparateChar","Default" "SearcherFailureEscalation",0 "MonitoringStyle",0

• Column headings are output in the first line, and system setup information is output from the second line.

The column headings are as follows:

− Parameter: The name of an execution parameter in the system environment file

− Value: The value specified for the execution parameter


Note




Monitoring Connection Status This section explains the details and procedures involved in monitoring the status of connections.

Monitoring Connection Information Connection information can be used for the following purposes:

• Checking the status of connections between applications and Shunsaku

• Collecting information about connections where the time taken to respond to an application exceeds a certain interval.

• Analyzing which applications' requests are taking a long time to process

Connection status information can be obtained by executing the shuncstate command with the -c option or the -t option on the server where the conductor is located.

Example 1

The following example shows how connection information is displayed. shuncstate -s shunc –c Shunsaku shuncstate 2005/01/01 19:01:00 (1) (2) Client Application (3) (4) RequestType ShunsakuFile (5) (6) (7) QueryStarted QueryElapse(sec) Results (8) (9) Stage StageElapse(sec) (10) (11) (12) DirectorStarted DirectorEnded BusiestDirector (13) (14) (15)

8-7


SorterStarted SorterEnded Sorter (16) Expression 10.124.82.12 Application2 SEARCH file1 19:02:53.340 6.660 DIRECTOR 4.060 19:02:53.950 SEARCH: /root/company/employee/name == 'smith' RETURN: /root/company/name/text() 10.124.82.11 Application1 DELETE file1 19:02:54.140 3.870 DIRECTOR 3.020 19:02:58.740 10.124.82.12 Application.PKG01 SORT file1 19:02:49.780 9.120 25 SORTER 1.130 19:02:50.330 19:02:56.740 shund1 19:02:57.080 shuno1 SEARCH: /root/company/id = 2000 SORT: rlen(/root/company/name/text(),16) RETURN: /root/company/name/text()

(1) The IP address of the host where the application is running

(2) The name of the application (a package name is displayed in the case of the Java APIs, and a program name is displayed in the case of C APIs.)

(3) Type of request from the connected application One of the following is displayed for the type of request from the connected application: − CONNECT: Connection request

− SEARCH: Search request

− SORT: Sort request

− INSERT: Insertion request

− UPDATE: Update request

− DELETE: Deletion request

− COMMIT: Commit request

− ROLLBACK: Rollback request

(4) Shunsaku File name of the connection destination (displayed only when Shunsaku Files are used.)

(5) The time when the query was issued (HH:MM:SS.mmm)

(6) The time elapsed since the query was issued (seconds)

(7) Number of search results (displayed only if the RequestType is SEARCH or SORT.)

8-8

Monitoring

(8) Processing stage for the request (if the t option is specified, ’IDLE’ or ’ERROR’ will always be output.)

− CONDUCTOR: Conductor processing in progress

− DIRECTOR: Request being passed to director

− SORTER: Request being passed to sorter

− IDLE: Waiting for a request

− ERROR: Error

(9) The time (in seconds) that has elapsed since moving to the current processing stage (not displayed if the stage is “ERROR”.)

(10) Time when the director started processing (HH:MM:SS.mmm)

(11) Time when the director finished processing (HH:MM:SS.mmm)

(12) Identifier for the director with the longest processing time

(13) Time when the sorter started processing (HH:MM:SS.mmm)

(14) Time when the sorter finished processing (HH:MM:SS.mmm)

(15) Sorter identifier used by a search request with a sort specification

(16) Search request expression (displays the search expression, return expression, or sort expression of a search request. If NULL is specified for the return expression (get record ID), ’RECORD ID’ is displayed.)

Also, using the -t option makes it possible to collect information about queries that were executed within a certain time period.

Example 2

In the following example, information for connections relating to queries where requests were received within a certain period and where the response time to the application exceeds three seconds is output to a text file.

shuncstate -s shunc -t start -q 3 -o C:\shunwork\monitor\conection.txt shuncstate -s shunc -t end

shuncstate -s shunc -t start -q 3 -o /home/shunsaku/monitor/connection.txt shuncstate -s shunc -t end

• Specifying "-t start" starts the collection of connection information, while specifying "-t end" ends information collection.

8-9


• Content of connection.txt

Client Application RequestType ShunsakuFile QueryStarted QueryElapse(sec) Results Stage StageElapse(sec) DirectorStarted DirectorEnded BusiestDirector SorterStarted SorterEnded Sorter Expression 192.168.0.1 application2 SORT file1 19:19:46.053 3.160 201 IDLE 0.000 19:19:46.091 19:19:47.133 shund1 19:19:47.134 19:19:49.211 shuno1 SEARCH: /document/base/prefecture == 'Melbourne' SORT: val(/document/base/price/text()), /document/base/in/text() RETURN: /document/base/name, /document/base/price, /document/base/in 192.168.0.2 application1 SORT file1 19:19:46.250 3.162 283 IDLE 0.000 19:19:46.252 19:19:47.333 shund1 19:19:47.334 19:19:49.411 shuno1 SEARCH: /document/base/prefecture == 'Sydney' SORT: val(/document/base/price/text()), /document/base/in/text() RETURN: /document/base/name, /document/base/price, /document/base/in

Example 3

In the following example, information for connections relating to queries where requests were received within a certain time period is output in CSV format.

shuncstate -s shunc -t start -C -o C:\shunwork\monitor\conection.csv shuncstate -s shunc -t end

shuncstate -s shunc -t start -C -o /home/shunsaku/monitor/connection.csv shuncstate -s shunc -t end

8-10

Monitoring

• Specifying "-t start" starts the collection of connection information, while specifying "-t end" ends information collection.

Content of connection.csv "Client","Application","RequestType","ShunsakuFile","QueryStarted","QueryElapse(sec)","Results","Stage","StageElapse(sec)","DirectorStarted","DirectorEnded","BusiestDirector","SorterStarted","SorterEnded","Sorter","Expression" "192.168.0.1","application2","SORT","file1","19:19:46.053",3.160,201,"IDLE",0.000,"19:19:46.091","19:19:47.133","shund1","19:19:47.134","19:19:49.211","shuno1","SEARCH: /document/base/prefecture == 'Melbourne' SORT: val(/document/base/price/text()), /document/base/in/text() RETURN: /document/base/name, /document/base/price, /document/base/in" "192.168.0.2","application1","SORT","file1","19:19:46.250",3.162,283,"IDLE",0.000,"19:19:46.252","19:19:47.333","shund1","19:19:47.334","19:19:49.411","shuno1","SEARCH: /document/base/prefecture == 'Sydney' SORT: val(/document/base/price/text()), /document/base/in/text() RETURN: /document/base/name, /document/base/price, /document/base/in"

• Column headings are output in the first line, and connection information is output from the second line.


Note


• String items are enclosed in double quotes. If there are double quotes within a string, the double quotes will be output twice in a row.


Monitoring Conductor Status This section explains the details and procedures involved in monitoring the status of the conductor.

Monitoring Conductor Operation Information Conductor operation information can be used for the following purposes:

• Checking the operating status of the conductor

• Checking statistical information about requests from applications

• Checking which director or sorter has caused a communication problem if it occurs when an application is executed and attempts to communicate with a director or sorter

Conductor operation information can be obtained by executing the shuncstate command on the server where the conductor is located.

8-11


Example 1

The following example shows how conductor operation information is displayed. shuncstate -s shunc -w Shunsaku shuncstate 2005/01/01 19:01:00 (1) (2) (3) (4) (5) (6) (7) Time State Connections Searches Sorts Updates Errors 19:01:00 ACTIVE 310 100 20 150 0

(1) Output time (hh:mm:ss)

(2) The operating status of the conductor The operating status of the conductor is displayed as follows: − ACTIVE: Running

(3) Number of connections that have been established with the conductor

(4) Cumulative number of search requests

(5) Cumulative number of search requests with sort specifications

(6) Cumulative number of update (insert, update or delete) requests

(7) Cumulative number of error responses

Note

The statistical information in items (4) to (7) show cumulative values counted from when the conductor started or when statistical information was reset. Statistical information can be reset by executing this command with the -r option.

Conductor operation information can also be collected at fixed time intervals. In this case, items (4) to (7) show the increases that have occurred during the time interval.

Example 2

In the following example, operation information for the conductor is collected three times at 60-second intervals and then displayed. shuncstate -s shunc -w 60 3 Shunsaku shuncstate 2005/01/01 19:01:00 Time State Connections Searches Sorts Updates Errors 19:02:00 ACTIVE 310 20 10 7 0 19:03:00 ACTIVE 350 5 0 1 0 19:04:00 ACTIVE 203 7 0 0 0

Example 3

In the following example, operation information for the conductor is collected three times at 60-second intervals and then output in CSV format.

shuncstate -s shunc -w -C 60 3 > C:\shunwork\monitor\constate.csv

8-12

Monitoring

shuncstate -s shunc -w –C 60 3 > /home/shunsaku/monitor/constate.csv

Content of constate.csv "Time","State","Connections","Searches","Sorts","Updates","Errors" "2005/01/01 19:02:00","ACTIVE",310,10,20,7,0 "2005/01/01 19:03:00","ACTIVE",350,0,5,1,0 "2005/01/01 19:04:00","ACTIVE",203,0,7,0,0

• Column headings are output in the first line, and operation information is output from the second line.

• The "Time" column uses date/time format (YYYY/MM/DD hh:mm:ss).


Information for the sorters and directors connected to the conductor can be obtained by executing the shuncstate command with the -p option.

Example 4

The following example shows how information for the sorters and directors connected to the conductor is displayed. shuncstate -s shunc –p Shunsaku shuncstate 2005/01/01 19:01:00 (1) (2) (3) (4) (5) Type ID Hostname State LatestTime(sec) Sorter shuno1 DirSvr1 ACTIVE 2.000 Director shund1 DirSvr1 ACTIVE 10.000 Director shund2 DirSvr1 ACTIVE 12.000 Director shund3 DirSvr1 ACTIVE 15.000

(1) Process type One of the following is displayed for the process type: − Sorter

− Director

(2) Process identifier

(3) Host name or IP address

8-13


(4) Communication status The following is displayed for the process status. − ACTIVE: Running

− INACTIVE : Error

This parameter is ‘ACTIVE’ when the conductor is running. If a communication problem is detected while an application process is communicating with a director or sorter, this parameter will change to ‘INACTIVE’. Once the problem has been resolved and communications is restored, its status returns to ‘ACTIVE’.

(5) The most recent director/sorter response time (seconds)

Example 5

In the following example, information for the sorters and directors connected to the conductor is output in CSV format.

shuncstate -s shunc -p -C > C:\shunwork\monitor\constate.csv

shuncstate -s shunc -p -C > /home/shunsaku/monitor/constate.csv

Content of constate.csv "Type","ID","Hostname","State","LatestTime(sec)" "Sorter","shuno1","DirSvr1","ACTIVE",2.000 "Director","shund1","DirSvr1","ACTIVE",10.000 "Director","shund2","DirSvr1","ACTIVE",12.000 "Director","shund3","DirSvr1","ACTIVE",15.000



Note




8-14

Monitoring

Monitoring Conductor Setup Information The operating environment that has been set up for the conductor can be checked.

Conductor setup information can be obtained by executing the shuncstate command on the server where the conductor is located.

Example 1

The following example shows how conductor setup information is displayed.

shuncstate -s shunc -i Shunsaku shuncstate 2005/01/01 19:01:00 Parameter Value LogFileFolder "C:\Program Files\Interstage Shunsaku\Shunsaku\log\conductor\" LogFileSize 100MB CoreFileFolder "C:\Program Files\Interstage Shunsaku\Shunsaku\core\conductor\" AnsMax 100 MaxConnection 1000 File file1 shund1 shund2 File file2 shund3 shund4 Domain dom1 file1 file2 InsertPoint shund1 file1 InsertPoint shund3 file2 DefaultFile file1 SystemName shunsaku

shuncstate -s shunc -i Shunsaku shuncstate 2005/01/01 19:01:00 Parameter Value LogFileFolder /var/opt/FJSVshnsk/log/conductor/ LogFileSize 100MB CoreFileFolder /var/opt/FJSVshnsk/core/conductor/ AnsMax 100 MaxConnection 1000 File file1 shund1 shund2 File file2 shund3 shund4 Domain dom1 file1 file2 InsertPoint shund1 file1 InsertPoint shund3 file2 DefaultFile file1 SystemName shunsaku

Note

Refer to 'Conductor Environment Files' in Appendix A for more information on the execution parameters in the conductor environment file.

8-15


Example 2

In the following example, conductor setup information is output in CSV format.

shuncstate -s shunc -i -C > C:\shunwork\monitor\constate.csv

shuncstate -s shunc -i -C > /home/shunsaku/monitor/constate.csv

Content of constate.csv

"Parameter","Value" "LogFileFolder","/var/opt/FJSVshnsk/log/conductor/" "LogFileSize",100 "CoreFileFolder","/var/opt/FJSVshnsk/core/conductor/" "AnsMax",100 "MaxConnection",1000 "File","file1 shund1 shund2" "File","file2 shund3 shund4" "Domain","dom1 file1 file2" "InsertPoint","file1 shund1" "InsertPoint","file2 shund3" "DefaultFile","file1" "SystemName","shunsaku"

"Parameter","Value" "LogFileFolder","/opt/FJSVshnsk/log/conductor/" "LogFileSize",100 "CoreFileFolder","/opt/FJSVshnsk/core/conductor/" "AnsMax",100 "MaxConnection",1000 "File","file1 shund1 shund2" "File","file2 shund3 shund4" "Domain","dom1 file1 file2" "InsertPoint","file1 shund1" "InsertPoint","file2 shund3" "DefaultFile","file1" "SystemName","shunsaku"

8-16

Monitoring

• Column headings are output in the first line, and conductor setup information is output from the second line.


− Parameter: The name of an execution parameter in the conductor environment file



Note




Monitoring Sorter Status This section explains the details and procedures involved in monitoring the status of sorters.

Monitoring Sorter Operation Information Sorter operation information can be used for the following purposes:

• Checking the operating status of sorters

• Checking statistical information for sort requests

Sorter operation information can be obtained by executing the shunostate command on the server where the sorter is located.

Example 1

The following example shows how sorter operation information is displayed.

shunostate -s shuno1 -w Shunsaku shunostate 2005/01/01 19:01:00 (1) (2) (3) (4) Time State Requests Errors (5) (6) (7) AvgTime(sec) MaxTime(sec) MaxTimeDetected (8) (9) (10) (11) AvgUsedMemory MaxUsedMemory MaxUsedMemoryDetected Concurrency 19:01:00 ACTIVE 3515 0 2.000 30.000 2005/01/01 10:00:00 10.0MB 20.0MB 2005/01/01 10:00:00 5


(2) The operating status of the sorter The operating status of the sorter is displayed as follows: − ACTIVE: Running

(3) Cumulative number of sort requests

8-17


(4) Cumulative number of error responses

(5) Average response time (seconds)

(6) Maximum response time (seconds)

(7) Date/time when the maximum response time was detected (YYYY/MM/DD hh:mm:ss)

(8) Average amount of memory used per sort request

(9) Maximum amount of memory used for a single sort request

(10) Date/time when the maximum amount of memory used for a single sort request was detected (YYYY/MM/DD hh:mm:ss)

(11) Number of sort requests currently being processed

Note

• The statistical information in items (3) to (10) can be reset by executing this command with the -r option. Information is then calculated by counting from the time that the information was reset.

• Sorter operation information can also be collected at fixed time intervals. In this case, the following items show the increases that have occurred during the time interval.

− Cumulative number of sort requests

− Cumulative number of error responses

− Average response time (seconds)

− Average amount of memory used

Example 2

In the following example, operation information for the sorter is collected three times at 60-second intervals and then displayed. shunostate -s shuno1 -w 60 3 Shunsaku shunostate 2005/01/01 19:01:00 Time State Requests Errors AvgTime(sec) MaxTime(sec) MaxTimeDetected AvgUsedMemory MaxUsedMemory MaxUsedMemoryDetected Concurrency 19:02:00 ACTIVE 30 0 2.000 13.000 2005/01/01 19:01:05 20.0MB 28.0MB 2005/01/01 19:01:10 2 19:03:00 ACTIVE 7 0 2.300 15.000 2005/01/01 19:02:20 22.0MB 32.0MB 2005/01/01 19:02:30 5 19:04:00 ACTIVE 20 0 1.980 16.540 2005/01/01 19:03:20 23.0MB 33.0MB 2005/01/01 19:03:30 3

8-18

Monitoring

Example 3

In the following example, operation information for the sorter is collected three times at 60-second intervals and then output in CSV format.

shunostate -s shuno1 -w -C 60 3 > C:\shunwork\monitor\srtstate.csv

shunostate -s shuno1 -w -C 60 3 > /home/shunsaku/monitor/srtstate.csv

Output content of srtstate.csv "Time","State","Requests","Errors","AvgTime(sec)","MaxTime(sec)", "MaxTimeDetected","AvgUsedMemory","MaxUsedMemory","MaxUsedMemoryDetected", "Concurrency" "2004/07/13 19:02:00","ACTIVE",30,0,2.000,13.000,"2004/07/01 10:00:00",20971520,29360128,"2004/07/01 10:00:00",2 "2004/07/13 19:03:00","ACTIVE",7,0,2.300,15.000,"2004/07/01 10:00:00",23068672,33554432,"2004/07/01 10:00:00",5 "2004/07/13 19:04:00","ACTIVE",20,0,1.980,11.540,"2004/07/01 10:00:00",19922944,20971520,"2004/07/01 10:00:00",3 "Time","State","Requests","Errors","AvgTime(sec)","MaxTime(sec)","MaxTimeDetected","AvgUsedMemory","MaxUsedMemory","MaxUsedMemoryDetected","Concurrency" "2005/01/01 19:02:00","ACTIVE",30,0,2.000,13.000,"2005/01/01 19:01:05",20971520,29360128,"2005/01/01 19:01:10",2 "2005/01/01 19:03:00","ACTIVE",7,0,2.300,15.000,"2005/01/01 19:02:20",23068672,33554432,"2005/01/01 19:02:30",5 "2005/01/01 19:04:00","ACTIVE",20,0,1.980,16.540,"2005/01/01 19:03:20",24117248,34603008,"2005/01/01 19:03:30",3


• The “Time” column uses date/time format (YYYY/MM/DD hh:mm:ss).

• The values in the “AvgUsedMemory” and “MaxUsedMemory” columns are displayed in bytes.


Note




8-19


Monitoring Sorter Setup Information The operating environment that has been set up for sorters can be checked.

Sorter setup information can be obtained by executing the shunostate command on the server where the sorter is located.

Example 1

The following example shows how sorter setup information is displayed.

shunostate -s shuno1 -i Shunsaku shunostate 2005/01/01 19:01:00 Parameter Value MultiMax 20 LogFileFolder "C:\Program Files\Interstage Shunsaku\Shunsaku\log\sorter\" LogFileSize 100MB CoreFileFolder "C:\Program Files\Interstage Shunsaku\Shunsaku\core\sorter\" SystemName shunsaku

shunostate -s shuno1 -i Shunsaku shunostate 2005/01/01 19:01:00 Parameter Value MultiMax 20 LogFileFolder /var/opt/FJSVshnsk/log/sorter/ LogFileSize 100MB CoreFileFolder /var/opt/FJSVshnsk/log/sorter/ SystemName shunsaku

Note

Refer to 'Sorter Environment Files' in Appendix A for more information on the execution parameters in the sorter environment file.

Example 2

In the following example, sorter setup information is output in CSV format.

shunostate -s shuno1 -i -C > C:\shunwork\monitor\srtstate.csv

shunostate -s shuno1 -i -C > /home/shunsaku/monitor/srtstate.csv

8-20

Monitoring

Content output to srtstate.csv

"Parameter","Value" "MultiMax",20 "LogFileFolder","C:\Program Files\Interstage Shunsaku\Shunsaku\log\sorter\" "LogFileSize",100 "CoreFileFolder","C:\Program Files\Interstage Shunsaku\Shunsaku\core\sorter\" "SystemName","shunsaku"

"Parameter","Value" "MultiMax",20 "LogFileFolder","/var/opt/FJSVshnsk/log/sorter/" "LogFileSize",100 "CoreFileFolder","/var/opt/FJSVshnsk/core/sorter/" "SystemName","shunsaku"

• Column headings are output in the first line, and sorter setup information is output from the second line.


− Parameter: The name of an execution parameter in the sorter environment file



Note




Monitoring Director Status This section explains the details and procedures involved in monitoring the status of directors.

Monitoring Director Operation Information Director operation information can be used for the following purposes:

• Checking the operating status of directors

• Checking director data information (the amount of data, fragmentation, backup status)

• Checking whether there are any errors with any searchers connected to the director

• Checking whether there are any imbalances in the amounts of data allocated to searchers connected to the director.

8-21


Director operation information can be obtained by executing the shundstate command on the server where the director is located.

Example 1

The following example shows how director operation information is displayed. shundstate -s shund1 -w Shunsaku shundstate 2005/01/01 19:01:00 (1) (2) (3) (4) (5) Time State DataFileStatus DataSize Fragments(%) (6) (7) (8) LastBackedUp OperationLogStatus OperationLogSize (9) (10) (11) (12) Records ReadRecords ReadTime(sec) ReadSize 19:01:00 ACTIVE NORMAL 40.0GB 2 2005/01/01 12:00:00 NORMAL 10.0KB 100000 12000 5.020 2.00MB


(2) The operating status of the director The operating status of the director is displayed as follows: − ACTIVE: Running


− DEGRADED: Degraded

− DEGRADING: Degrading

− RESENDING: Reallocating data

− SWITCHING: Switching to an alternative searcher

− CONDENSING: Optimizing data

− BACKUP: Backing up data

− RECOVER: Recovering data

− WAITING: Waiting for searchers to connect

(3) Status of the storage directory for director data files The status of the storage directory for director data files is displayed as follows: − NORMAL: Normal status

− IOERROR: I/O fault with director data files

− FULL: Insufficient space for director data files

(4) Total amount of director data

(5) Fragmentation rate (%)

(6) Time when the most recent backup start declaration was made (YYYY/MM/DD hh:mm:ss)

8-22

Monitoring

(7) Status of the storage directory for the operation log files (output only when the OperationLogFolder parameter is specified in the director environment file) The status of the storage directory for operation log files is displayed as follows: − NORMAL: Normal status

− IOERROR: I/O fault with operation log files

− FULL: Insufficient space for operation log files

(8) Amount of space used by operation log files (output only when the OperationLogFolder parameter is specified in the director environment file)

(9) Total number of records that can be searched

(10) Cumulative number of XML data records that have been read (displays the cumulative number of XML data records that have been read from the director data file)

(11) Cumulative time (in seconds) spent reading XML data (displays the cumulative amount of time spent reading XML data from the director data file)

(12) Cumulative size of the XML data that has been read (displays the cumulative size of the XML data that has been read from the director data file).

Note

The statistical information in items (10) to (12) can be reset by executing this command with the -r option. Information is then calculated by counting from the time that the information was reset.

Director operation information can also be collected at fixed time intervals. In this case, the following items show the increases that have occurred during the time interval.

• Cumulative time (in seconds) spent reading XML data

• Cumulative number of XML data records that have been read

• Cumulative size of the XML data that has been read

Example 2

In the following example, operation information for a director is collected three times at 60-second intervals and then displayed. shundstate -s shund1 -w 60 3 Shunsaku shundstate 2005/01/01 19:01:00 Time State DataFileStatus DataSize Fragments(%) LastBackedUp OperationLogStatus OperationLogSize Records ReadRecords ReadTime(sec) ReadSize 19:02:00 ACTIVE NORMAL 40.0GB 2 2005/01/01 12:00:00 NORMAL 10.0KB 100000 12000 2.080 1.42MB 19:03:00 ACTIVE NORMAL 40.0GB 2 2005/01/01 12:00:00 NORMAL 10.0KB 100000 20000 7.102 3.00MB

8-23


19:04:00 ACTIVE NORMAL 40.0GB 2 2005/01/01 12:00:00 NORMAL 10.0KB 100000 3130 1.050 318KB

Example 3

In the following example, operation information for a director is collected three times at 60-second intervals and then output in CSV format.

shundstate -s shund1 -w -C 60 3 > C:\shunwork\monitor\dirstate.csv

shundstate -s shund1 -w -C 60 3 > /home/shunsaku/monitor/dirstate.csv

Content of dirstate.csv "Time","State","DataFileStatus","DataSize","Fragments(%)","LastBackedUp","OperationLogStatus","OperationLogSize","Records","ReadRecords","ReadTime(sec) ","ReadSize" "2005/01/01 19:02:00","ACTIVE","NORMAL",42949672960,2,"2005/01/01 12:00:00","NORMAL",10240,100000,12000,2.080,1488977 "2005/01/01 19:03:00","ACTIVE","NORMAL",42949672960,2,"2005/01/01 12:00:00","NORMAL",10240,100000,20000,7.102,3145728 "2005/01/01 19:04:00","ACTIVE","NORMAL",42949672960,2,"2005/01/01 12:00:00","NORMAL",10240,100000,3130,1.050,325632

• Column headings are output in the first line, and director operation information is output from the second line.


• The values in the “DataSize”, “OperationLogSize” and “ReadSize” columns are displayed in bytes.


Note




Information about searchers connected to the director can be obtained by executing the shundstate command with the -p option.

8-24

Monitoring

Example 4

The following example shows how information for the searchers connected to the director is displayed. shundstate -s shund1 -p Shunsaku shundstate 2005/01/01 19:01:00 (1) (2) (3) (4) (5) (6) (7) Type ID Hostname State Records DataSize LatestTime(sec) Searcher shuns01 SchSvr01 ACTIVE 50000 12.0MB 0.900 Searcher shuns02 SchSvr02 ACTIVE 50000 12.0MB 0.890 Searcher shuns03 SchSvr03 ACTIVE 50000 12.0MB 0.910 Searcher shuns04 SchSvr04 INACTIVE 0 0.00KB 0.000 AlternativeSearcher shuns05 SchWSvr01 ACTIVE 50000 12.0MB 0.900

(1) Process type One of the following is displayed for the process type: − Searcher

− AlternativeSearcher

(2) Searcher identifier

(3) Host name or IP address of the searcher

(4) Operating status of the searcher The operating status of the searcher is displayed as follows: − ACTIVE: Running



(5) Number of records that can be searched

(6) Amount of data that can be searched

(7) Searcher processing time (in seconds) during the most recent search process

Example 5

In the following example, information for the searchers connected to the director is output in CSV format.

shundstate -s shund1 -p -C > C:\shunwork\monitor\dirstate.csv

8-25


shundstate -s shund1 -p -C > /home/shunsaku/monitor/dirstate.csv

Content of dirstate.csv "Type","ID","Hostname","State","Records","DataSize","LatestTime(sec)" "Searcher","shuns01"," SchSvr01","ACTIVE",50000,12582912,0.900 "Searcher","shuns02"," SchSvr02","ACTIVE",50000,12582912,0.890 "Searcher","shuns03"," SchSvr03","ACTIVE",50000,12582912,0.910 "Searcher","shuns04"," SchSvr04","INACTIVE",0,0,0.000 "AlternativeSearcher","shuns05","SchWSvr01","ACTIVE",50000,12582912,0.900

• Column headings are output in the first line, and information for the searchers connected to the director is output from the second line.

• The values in the “DataSize” column are displayed in bytes.


Note




Monitoring Director Setup Information The operating environment that has been set up for directors can be checked.

Director setup information can be obtained by executing the shundstate command on the server where the director is located.

Example 1

The following example shows how director setup information is output.

shundstate –s shund1 -i Shunsaku shundstate 2005/01/01 19:00:00 Parameter Value DataFileFolder "C:\Shunsaku\" LogFileFolder "C:\Program Files\Interstage Shunsaku\Shunsaku\log\director\" LogFileSize 100MB CoreFileFolder "C:\Program Files\Interstage Shunsaku\Shunsaku\core\director\" OperationLogFolder "D:\Shunbackup\ope\" AnsMax 100 MultiMax 100 SearcherWTimer(sec) 180 ANKmix 0 KNJmix 0

8-26

Monitoring

SkipChar "\s,\S,\n,\t" ExtendedQuery 0 SeparateChar Default SystemName shunsaku

shundstate -s shund1 -i Shunsaku shundstate 2005/01/01 19:01:00 Parameter Value DataFileFolder /Shunsaku/ LogFileFolder /var/opt/FJSVshnsk/log/director/ LogFileSize 100MB CoreFileFolder /var/opt/FJSVshnsk/core/director/ OperationLogFolder /Shunbackup/ope/ AnsMax 100 MultiMax 100 SearcherWTimer(sec) 180 ANKmix 0 KNJmix 0 SkipChar "\s,\S,\n,\t" ExtendedQuery 0 SeparateChar Default SystemName shunsaku

Note


Example 2

In the following example, director setup information is output in CSV format.

shundstate -s shund1 -i -C > C:\shunwork\monitor\dirsetting.csv

shundstate -s shund1 -i -C > /home/shunsaku/monitor/dirsetting.csv

Content of dirsetting.csv

"Parameter","Value" "DataFileFolder","C:\Shunsaku\" "LogFileFolder","C:\Program Files\Interstage Shunsaku\Shunsaku\log\director\" "LogFileSize",100

8-27


"CoreFileFolder","C:\Program Files\Interstage Shunsaku\Shunsaku\core\director\" "OperationLogFolder","D:\Shunbackup\ope\" "AnsMax",100 "MultiMax",100 "SearcherWTimer(sec)",180 "ANKmix",0 "KNJmix",0 "SkipChar","\s,\S,\n,\t" "ExtendedQuery",0 "SeparateChar","Default" "SystemName","shunsaku"

"Parameter","Value" "DataFileFolder","/Shunsaku/" "LogFileFolder","/var/opt/FJSVshnsk/log/director/" "LogFileSize",100 "CoreFileFolder","/var/opt/FJSVshnsk/core/director/" "OperationLogFolder","/Shunbackup/ope/ "AnsMax",100 "MultiMax",100 "SearcherWTimer(sec)",180 "ANKmix",0 "KNJmix",0 "SkipChar","\s,\S,\n,\t" "ExtendedQuery",0 "SeparateChar","Default" "SystemName","shunsaku"

• Column headings are output in the first line, and director setup information is output from the second line.


− Parameter: The name of an execution parameter in the director environment file



Note




8-28

Monitoring

Monitoring Searcher Status This section explains the details and procedures involved in monitoring the status of searchers.

Monitoring Searcher Operation Information Searcher operation information can be used for the following purposes:

• Checking the operating status of directors

• Checking search processing performance

Searcher operation information can be obtained by executing the shunsstate command on the server where the searcher is located.

Example 1

The following example shows how searcher operation information is displayed. shunsstate -s shuns01 –w Shunsaku shunsstate 2005/01/01 19:01:00 (1) (2) (3) (4) (5) (6) (7) Time State Records DataSize AvgTime(sec) MaxTime(sec) MaxTimeDetected 19:01:00 ACTIVE 5000 100KB 1.000 1.300 2005/01/01 13:00:00


(2) The operating status of the searcher The operating status of the searcher is displayed as follows: − ACTIVE: Running


− IDLE: On standby

(3) Current number of records

(4) Current amount of data (displays the volume of internal data that has been converted in order to search the target data)

(5) Average search time (in seconds)

(6) Maximum search time (in seconds)

(7) Date/time when the maximum search time was detected (YYYY/MM/DD hh:mm:ss)

Note

The statistical information in items (5) to (7) can be reset by executing this command with the -r option. Information is then calculated by counting from the time that the information was reset.

Searcher operation information can also be collected at fixed time intervals. In this case, the following items show the increases that have occurred during the time interval.

• Average search time (in seconds)

8-29


Example 2

In the following example, operation information for a searcher is collected three times at 60-second intervals and then displayed. shunsstate -s shuns01 -w 60 3 Shunsaku shunsstate 2005/01/01 19:01:00 Time State Records DataSize AvgTime(sec) MaxTime(sec) MaxTimeDetected 19:02:00 ACTIVE 5000 100KB 1.010 1.300 2005/01/01 13:00:00 19:03:00 ACTIVE 5020 105KB 0.930 1.300 2005/01/01 13:00:00 19:04:00 ACTIVE 5025 106KB 0.805 1.300 2005/01/01 13:00:00

Example 3

In the following example, operation information for a searcher is collected three times at 60-second intervals and then output in CSV format.

shunsstate -s shuns01 -w -C 60 3 > C:\shunwork\monitor\srchstate.csv

shunsstate -s shuns01 -w -C 60 3 > /home/shunsaku/monitor/srchstate.csv

Content of srchstate.csv "Time","State","Records","DataSize","AvgTime(sec)","MaxTime(sec)","MaxTimeDetected" "2005/01/01 19:02:00","ACTIVE",5000,102400,1.010,1.300,"2005/01/01 13:00:00" "2005/01/01 19:03:00","ACTIVE",5020,107520,0.930,1.300,"2005/01/01 13:00:00" "2005/01/01 19:04:00","ACTIVE",5025,108544,0.805,1.300,"2005/01/01 13:00:00"



• The values in the “DataSize” column are displayed in bytes.


8-30

Monitoring

Note




Monitoring Searcher Setup Information The operating environment that has been set up for searchers can be checked.

Searcher setup information can be obtained by executing the shunsstate command on the server where the searcher is located.

Example 1

The following example shows how searcher setup information is displayed.

shunsstate -s shuns01 -i Shunsaku shunsstate 2005/01/01 19:01:00 Parameter Value LogFileFolder "C:\Program Files\Interstage Shunsaku\Shunsaku\log\searcher\" LogFileSize 100MB CoreFileFolder "C:\Program Files\Interstage Shunsaku\Shunsaku\core\searcher\" SystemName shunsaku

shunsstate -s shuns01 -i Shunsaku shunsstate 2005/01/01 19:01:00 Parameter Value LogFileFolder /var/opt/FJSVshnsk/log/searcher/ LogFileSize 100MB CoreFileFolder /var/opt/FJSVshnsk/core/searcher/ SystemName shunsaku

Note

Refer to 'Searcher Environment Files' in Appendix A for more information on the execution parameters in the searcher environment file.

Example 2

In the following example, searcher setup information is output in CSV format.

shunsstate -s shuns01 -i -C > C:\shunwork\monitor\srchstate.csv

8-31


shunsstate -s shuns01 -i -C > /home/shunsaku/monitor/srchstate.csv

Content output to srchstate.csv

"Parameter","Value" "LogFileFolder","/var/opt/FJSVshnsk/log/searcher/" "LogFileSize",100 "CoreFileFolder","/var/opt/FJSVshnsk/core/searcher/" "SystemName","shunsaku"

"Parameter","Value" "LogFileFolder","/opt/FJSVshnsk/log/searcher/" "LogFileSize",100 "CoreFileFolder","/opt/FJSVshnsk/core/searcher/" "SystemName","shunsaku"

• Column headings are output in the first line, and searcher setup information is output from the second line.


• Parameter: The name of an execution parameter in the searcher environment file

• Value: The value specified for the execution parameter

Each record ends with a linefeed code.

Note




8-32

Recreating Lookup Data

Recreating Lookup Data This section explains how to recreate lookup data.

When Lookup Data Needs to be Recreated Lookup data must be recreated if changes have been made to its source data, such as the case when data from another system has been imported as lookup data.

The following two methods can be used to change data:

• Using an application to update data

• Using a command to update data

This section explains how to update data with a command. Refer to the Application Development Guide for information on using applications to update data.

The following diagram shows the procedure for recreating lookup data.

Figure 8-2 Recreating Lookup Data

Deleting Lookup Data To recreate lookup data, first execute the shundclear command to delete the lookup data that is currently being used.

Use the following procedure to delete the data.

1. Execute the shundclear command to delete director data and search data. shundclear -s director identifier

2. Execute the shundstate command to check that the total number of records and the number of records in the information for connected searchers are both zero. shundstate -s director identifier –p

8-33


Recreating Lookup Data Recreate lookup data by preparing a text file containing the new data and executing the shundimport command.

Use the following procedure to recreate the data.

1. Execute the shundimport command to import the text file. shundimport -s director identifier -f file name

2. Execute the shundstate command to check that the total number of records has changed. shundstate -s director identifier

8-34

Responding to Errors

Responding to Errors This section explains the recovery procedures performed when errors occur with Shunsaku during operation and when the application server fails.

Handling Server Errors The procedure for recovering Shunsaku when errors occur is as follows:

Figure 8-3 Recovering Shunsaku after an Error Occurs

Detecting Errors and Assessing the Impact on Operations If an error occurs with any of the Shunsaku processes (conductor, directors, searchers, or sorters) during operations, a message notifying that a Shunsaku error has occurred will be output to the event log or system log (syslog) on the server where the faulty process is located.

Operations may not be able to continue, depending on the error that has occurred. The following table shows how different errors impact on operations.

Table 8-2 The Impact of Errors in Shunsaku

Message ID Cause Degree of impact on operations

Impact on active applications

shn06000u shn06002u

The conductor has not terminated correctly.

Applications cannot connect to Shunsaku until Shunsaku is restored.

Connections will be closed.

shn02003u shn02012u shn30302u

A director has not terminated correctly.

Searches and updates cannot be performed on Shunsaku until the director is restored.

Transactions will be canceled.

8-35


Message ID Cause Degree of impact on operations

Impact on active applications

Shn02000e An error has occurred with communication processing for searchers.

Degradation or failover will start automatically, and operation can recommence on completion. Note, however, that if degradation has occurred, it will not be possible to achieve the same level of performance as before the error.

Transactions will be canceled.

shn07015u A sorter has not terminated correctly.

Data cannot be sorted or aggregated until Shunsaku is restored.

Connections and transactions will not be affected.

Recovering from Errors If an error occurs with Shunsaku and prevents operations from continuing, operations can recommence once the administrator has eliminated the cause of the error and restarted Shunsaku.

The following diagram shows the flow from an error occurrence in Shunsaku through to recovery.

Figure 8-4 Flow from an Error to Recovery

1. Eliminate the cause of the error Eliminate the cause of the error by following the user response of the Shunsaku message output to either the event log or the system log (syslog) on the server where the error occurred. Refer to the Messages for information on how the user should respond to the message.

8-36


2. Restart Shunsaku. Once the cause of the error has been eliminated, restart Shunsaku by executing the shunsysstart command on the target server. shunsysstart -n Shunsaku system name

Normal operations will automatically recommence once Shunsaku has been restarted.

3. Reallocate search data. If an error has also occurred with a searcher, operations automatically switch from degradation operations to normal operations when Shunsaku is restarted. However, the search data that was allocated to other searchers as part of the degradation process is not restored to its pre-degradation state. To even out the distribution of search data among searchers, reallocate search data by executing the shundresend command. shundresend -s director identifier

Note While search data is being reallocated, lookup data cannot be updated using the shundimport command or APIs.

Handling Application Server Errors If the application server fails for any reason during operation such as power failure, all connections that the application has established will remain in place, using Shunsaku resources exclusively. If job operations continue in this state, errors may occur due to the maximum number of simultaneous connections being exceeded, or update requests being made to data that is currently being updated, or as a result of excessive memory utilization.

Therefore, before continuing jobs, the user should forcibly close these connections to free up the Shunsaku resources that were being exclusively used by the connections. Normal operations can then recommence.

8-37


Recovering from Errors The following diagram shows the flow from application server failure through to recovery.

Figure 8-5 Flow from Application Server Failure to Recovery

1. Check connection status. Check the status of the connections for the failed application server by executing the shuncstate command on the server where the conductor is located. Refer to Monitoring Connection Status for more information on the shuncstate command.

2. Forcibly close connections. If any connections from applications on the failed application server still remain, forcibly close them by executing the shuncterm command on the server where the conductor is located.

shuncterm –s conductor identifier –h host name or IP address

Specify in the -h option the host name or IP address of the server where the application was running.

3. Restart the application. Recommence jobs by restarting the application.

8-38


Handling errors Produced during Command Execution If a server fails while a director command is executing, the execution results for the command will either terminate normally, or with an error

Restart the director if it has stopped, and then use the following procedures to decide whether to execute the command again.

shundimport Command

Execute the shundstate command to check the total number of records that can be searched (the "Records" item).

• If the number of records is different from the number of records before the shundimport command was executed, the shundimport command has terminated normally.

• If the number of records is the same as the number of records before the shundimport command was executed, the shundimport command has not terminated normally. Execute the shundimport command again.

shundclear Command

Execute the shundstate command to check the total number of records that can be searched (the "Records" item).

• If the number of records is 0, the shundclear command has terminated normally.

• If the number of records is not 0, the shundclear command has not terminated normally. Execute the shundclear command again.

shundcds Command

Execute the shundstate command to check the fragmentation rate (the "Fragments" item).

• If the fragmentation rate is smaller than the one before the shudcds command was executed, the shundcds command has terminated normally.

• If the fragmentation rate is no smaller than the one before the shudcds command was executed, the shundcds command has not terminated normally. Execute the shundcds command again.

shundresend Command

Execute the shundstate command to check the information for searchers connected to the director.

• If the amount of data that can be searched (the DataSize item) has been evenly allocated to each searcher, the shundresend command has terminated normally.

• If the amount of data that can be searched (the DataSize item) has not been evenly allocated to each searcher, the shundresend command has not terminated normally. Execute the shundresend command again.

8-39


shundbackup Command

The procedure is different depending on which options were specified with the command.

• If a backup start declaration was executing (that is, if the -b option was specified), back up the director data files by executing the shundbackup command with the -b option again.

• If a backup end declaration was executing (that is, if the -e option was specified), the shundbackup command has terminated normally.

• If a backup start declaration was being canceled (that is, if the -c option was specified), the shundbackup command has terminated normally.

shundrecover Command

The procedure is different depending on which options were specified with the command.

• If a recovery start declaration was executing (that is, if the -b option was specified), recover the director data files by executing the shundrecover command with the -b option again.

• If a recovery end declaration was executing (that is, if the -e option was specified), recover the director data files by executing the shundrecover command with the -b option again.

• If a recovery start declaration was being canceled (that is, if the -c option was specified), the shundrecover command has terminated normally.

• If operation log files were being recovered (that is, if the -l option was specified), recover the operation log files by executing the shundrecover command with the -l option again.

shundexport Command

Delete the output file and execute the shundexport command again.

8-40

Appendix A Execution Parameters in Environment Files

This appendix explains the execution parameters defined in the Shunsaku environment files.

There are five types of environment file:

• System Environment Files

• Conductor Environment Files

• Director Environment Files

• Sorter Environment Files

• Searcher Environment Files

• API Environment Files

• Correspondence between V6.0 and V7.0 Execution Parameters

System environment files specify system configuration information for Shunsaku. These files can also set the default values for the entire system. If a particular parameter is specified in both a system environment file and another environment file, the settings in the other environment file take priority.

Execution Parameter Syntax The syntax for the execution parameters defined in environment files is as follows: <parameter name> <parameter value>

Note the following points on specifying execution parameters:

• The definition for a single execution parameter cannot extend across more than one line.

• The maximum length of a single line is 4,095 bytes (not including the linefeed code.)

• Separate parameter names and parameter values (and adjacent parameter values) using a space or tab.

• Lines that begin with ‘#’ are treated as comments. Similarly, any text following ‘#’ midway through a line is treated as a comment.

• A syntax error will occur if an execution parameter that should only be specified once appears more than once.

A-1

Appendix A: Execution Parameters in Environment Files

Example

In the example below, ‘#’ is specified midway through the line, and so ‘updated January 01, 2005’ is treated as a comment. CharacterCode UTF-8 #updated January 01, 2005

A-2

System Environment Files

System Environment Files Application environments for Shunsaku systems are defined in system environment files.

Execution Parameters and Their Meanings The following table shows the execution parameters specified in system environment files and their meanings. Each parameter is detailed in the section below.

Parameter Name Meaning Mandatory or Optional

Entry Example Remarks

Conductor Server information for the conductor

Mandatory

shunc DirSvr1 33101

-

Sorter Server information for the sorter

Mandatory

shuno1SortSvr1 19000

-

Director Server information for the director

Mandatory

shund1 DirSvr1 19002 19003 19004

-

Searcher Server information for the searcher

Mandatory

search01 SchSvr01 33501 shund1

-

AlternativeSearcher Server information for alternative searchers

Optional searchw1 SchWSvr01 33501 shund1

The director identifier for the director that the alternative searcher connects to (can be omitted or specified more than once)

CharacterCode Character encoding for lookup data

Optional UTF-8 Default value: UTF-8 For UTF-8: UTF-8 For Shift-JIS: SHIFT-JIS For EUC-JP: EUC For GB2312: GB2312 For Big5: big5 For KSC5601: KSC5601

ANKmix Case sensitivity for alphanumeric characters

Optional 0 Default value: 0 0: Match case 1: Do not match case

SkipChar Skip character (characters that will not be searched)

Optional “\s,\S,\n,\t” If this parameter is omitted, all strings will be searched.

ExtendedQuery Query format Optional 0 Default value: 0 0: Do not use compatible format queries 1: Use compatible format queries

A-3




SeparateChar Delimiter Optional “\t,\n,\s,\”,!,$,.” If this parameter is omitted, the character specified for “Default Delimiters“ is used as the delimiter.

SearcherFailureEscalation Scope that is regarded as abnormal when searcher abnormalities occur

Optional 0 Default value: 0 0: Target searcher only 1: Entire search server

MonitoringStyle Specifies the output format for monitoring

Optional 0 Default value: 0 0: New output format 1: Output format for V6.0

Execution Parameter Explanation

Conductor This parameter specifies server information for the conductor. Specify values for this parameter in the following order:

• Conductor identifier

• The host name or IP address allocated to the conductor

• The port number for receiving requests

Specify the port number for receiving requests from Shunsaku APIs.

Specify this port number with Shunsaku APIs. Refer to the Installation Guide for more information.

This parameter is mandatory.

Note

• Specify the conductor identifier using up to eight alphanumeric characters.

• For the port number for receiving requests, specify a port number that is not used by any other services.

• Each conductor on a single server must have a unique conductor identifier.

• If the Management Console is to be used, do not specify the loopback address 127.0.0.1 or a host name representing 127.0.0.1 (usually "localhost") for the host name or IP address allocated to the conductor. Specify a host name or IP address other than the loopback address.

Sorter This parameter specifies server information for the sorter. Specify values for this parameter in the following order:

• Sorter identifier

• The host name or IP address allocated to the sorter

• The port number for receiving sort requests from the conductor

A-4


This parameter must be specified over more than one line if multiple sorters are to be allocated.


Note

• Specify the sorter identifier using up to eight alphanumeric characters.

• Each sorter on a single server must have a unique sorter identifier.

• If the Management Console is to be used, do not specify the loopback address 127.0.0.1 or a host name representing 127.0.0.1 (usually "localhost") for the host name or IP address allocated to the sorter. Specify a host name or IP address other than the loopback address.

Director This parameter specifies server information for the director. Specify values for this parameter in the following order:

• Director identifier

• The host name or IP address allocated to the director

• The port number for issuing search requests to searchers

• The port number for issuing update requests to searchers

• The port number for receiving requests

Specify the port number for receiving requests.

Refer to the Installation Guide for more information on port numbers.

• Director number

The number corresponding to the specified director identifier

A unique number between 1 and 255 inclusive

The director number can be used if the StartPoint parameter in the conductor environment file is used, but can be omitted otherwise.

This parameter must be specified over more than one line if multiple directors are to be allocated. Up to 255 directors can be defined.


Note

• Specify the director identifier using up to eight alphanumeric characters.

• The director number can be omitted.

• Each director within a single Shunsaku system must have a unique director identifier.

• Each director on a single server must have a unique director identifier.

• If the Management Console is to be used, do not specify the loopback address 127.0.0.1 or a host name representing 127.0.0.1 (usually "localhost") for the host name or IP address allocated to the director. Specify a host name or IP address other than the loopback address.

A-5


Searcher This parameter specifies server information for the searcher. Specify information in the following order:

• Searcher identifier

• The host name or IP address allocated to the searcher

• The port number for receiving requests from directors. Refer to the Installation Guide for more information about defining port numbers.

• The director identifier for the director that the searcher is connected to.

This parameter must be specified over more than one line if multiple searchers are to be allocated. Up to 20 searchers can be defined for each director.


Note

• Specify the searcher identifier using up to eight alphanumeric characters.

• For the searcher port number, specify a port number that is not used by any other services.

• Within a single Shunsaku system, searchers with the same searcher identifier can be allocated, as long as they are allocated to different search servers.

• Each searcher on a single server must have a unique searcher identifier.

• If the Management Console is to be used, do not specify the loopback address 127.0.0.1 or a host name representing 127.0.0.1 (usually "localhost") for the host name or IP address allocated to the searcher. Specify a host name or IP address other than the loopback address.

AlternativeSearcher This parameter specifies server information for alternative searchers. Specify values for this parameter in the following order:

• Searcher identifier

• The host name or IP address allocated to the alternative searcher

• The port number for receiving requests from directors. Refer to the Installation Guide for more information on port numbers.

• Director identifiers for the directors that the alternative searcher can connect to.

Specify these identifiers to restrict the directors that an alternative connection can be made to.

Multiple director identifiers can be specified. If no director identifiers are specified, the alternative searcher can receive connections from all of the directors specified in the system environment file.

This parameter must be specified over more than one line if multiple alternative searchers are to be allocated. The number of alternative searchers that can be defined for each director is between 1 and 20 inclusive.

If this parameter is omitted, degradation operations will take place when searcher abnormalities occur.

A-6


Note

• Specify the searcher identifier using up to eight alphanumeric characters.

• For the port number for alternative searchers, specify a port number that is not used by any other services.

• Searchers or alternative searchers with the same searcher identifier can be allocated within a single Shunsaku system, as long as they are allocated to different search servers.

• Searchers or alternative searchers with the same searcher identifier cannot be created on a single system.

• If the Management Console is to be used, do not specify the loopback address 127.0.0.1 or a host name representing 127.0.0.1 (usually "localhost") for the host name or IP address allocated to the alternative searcher. Specify a host name or IP address other than the loopback address.

CharacterCode This parameter specifies the character encoding for the data to be searched.

If this parameter is omitted, it defaults to UTF-8.

Table A-1 Character Code Settings

Setting Meaning

UTF-8 For UTF-8

SHIFT-JIS For Shift-JIS (Japanese)

EUC For EUC-JP (Japanese)

GB2312 GB2312 (Chinese)

big5 Big5 (Chinese)

KSC5601 KSC5601 (Hangul alphabet (Korean))

Note

Create lookup data using the character encoding specified by the CharacterCode parameter.

ANKmix This parameter specifies whether to distinguish between upper and lower case for alphanumeric characters in the strings that will be searched.

0: Match case

1: Do not match case

If this parameter is omitted, it defaults to 0.

Table A-2 ANKmix Settings

Search Keyword String Being Searched 0: Match Case 1: Do Not Match Case

Ab

AB x

AB x

Ab

Ab x

A-7


Search Keyword String Being Searched 0: Match Case 1: Do Not Match Case

Ab x

AB

aB x

AB

Ab x

: Hit

x: Miss

Note

This parameter is valid with patterns and for string complete matches.

SkipChar This parameter specifies which characters are to be treated as skip characters (not part of the search target). Enclose the characters being specified in double quotes (“ “).

This parameter can be omitted.

More than one character can be specified in this parameter, provided that each string is separated by a comma (,).

Specify characters (other than control characters, the linefeed character, or horizontal tab) for the SkipChar parameter.

If this parameter is omitted, all characters will become the search target.

Note

• This parameter is valid with pattern string searches and for string complete matches.

• When specifying any character not shown below for this parameter, express the value using the character encoding specified in the CharacterCode parameter. Note that skip characters are case sensitive.

single-byte space \s

multi-byte space \S

Linefeed \n

Horizontal tab \t

Example

An example of specifying the following characters in this parameter is shown below:

Single-byte space, multi-byte space, horizontal tab, ＠(single-byte character), ＠ (multi-byte character), and linefeed character

Korean Chinese Character Encoding Specified in CharacterCode

UTF-8

KSC-5601 GB2312 big5

Single-byte space \s \s \s \s

Multi-byte space \S \S \S \S

A-8


Korean Chinese Character Encoding Specified in CharacterCode

UTF-8

KSC-5601 GB2312 big5

Horizontal tab \t \t \t \t

＠ (multi-byte) \40 \40 \40 \40

＠ (single-byte) \EF \BC \A0

\A3 \C0 \A3 \C0 \A2 \49

Linefeed \n \n \n \n

Entry examples are shown below:

Character Encoding Specified in CharacterCode

Entry Example

UTF-8 \s,\S,\t,\EF \BC \A0,\40,\n

Korean KSC-5601 \s,\S,\t,\40,\A3 \C0, \40,\n

GB2312 \s,\S,\t,\40,\A3 \C0, \40,\n Chinese

big5 \s,\S,\t,\40,\A2 \49, \40,\n

Reference

The difference between specifying a space (\s) and a multi-byte space (\S) as the skip character

Search Keyword String Being Searched

When \s and \S are not specified in SkipChar

When \s and \S are specified in SkipChar

John Smith

John Smith x

John Smith

JohnSmith x

John Smith x

John Smith

John Smith

JohnSmith x

John Smith x

John Smith x

John Smith

JohnSmith

: Hit

x: Miss

A-9


ExtendedQuery This parameter specifies whether to use compatible format queries.

0: Do not use compatible format queries

1: Use compatible format queries


SeparateChar This parameter specifies the entire delimiter (enclosed in double quotes (“ “) ) when word search searches are specified in search expressions.

This parameter can be omitted.

Multiple characters can be specified for the SeparateChar parameter. In this case, separate each string with a comma (,).

Specify an ASCII character (other than control characters, the linefeed character, or horizontal tab) for the SeparateChar parameter.

If the following characters are specified for the SeparateChar parameter, they must be preceded by the escape character. The escape character is “\”.

Delimiter Specification Method

Space \s

Linefeed \n

Horizontal tab \t

Comma \,

Double quote \”

Forward slash \\

Default Delimiters If this parameter is omitted, the following “delimiters” are regarded as having been specified.

\t \n \s ! \” #

$ % & ' ( )

* + \, - . /

: ; < = > ?

@ [ \\ ] ^ _

` { | } ~

A-10


SearcherFailureEscalation This parameter specifies the scope that the director will regard as abnormal if a searcher abnormality is detected on a search server where multiple searchers have been allocated.

0: Regard only the target searcher as abnormal

1: Regard the entire search server where the target searcher is allocated as abnormal

If value “1” is specified, all of the searchers allocated to the search server will be regarded as abnormal when a searcher abnormality is detected. If alternative searchers have been specified, the searchers regarded as abnormal will failover. Otherwise, degradation will be performed.

If this parameter is omitted, it defaults to “0”.

Note

Any searchers connected to another director but allocated to the same search server will not be included in the scope that is regarded as abnormal, even if “1” is specified for this parameter.

MonitoringStyle This parameter specifies the output format for monitoring.

0: The w option will be regarded as having been specified if none of the options w, p, i, or c are specified with the shuncstate, shunostate, shundstate, or shunsstate command.

1: Information will be displayed using the output format for V6.0 if none of the options w, p, i, or c are specified with the shuncstate, shunostate, shundstate, or shunsstate commands.


Note

Refer to Monitoring in Chapter 8, Maintenance for more information on the new output format for monitoring.

A-11


Entry Example An entry example for the execution parameters in the system environment file is shown below.

#---Server information for the conductor---# Conductor shunc DirSvr1 33101 #---Server information for the sorter---# Sorter shuno1 SortSvr1 33201 #---Server information for the director---# Director shund1 DirSvr1 33303 33304 33101 Director shund2 DirSvr2 33305 33306 33102 Director shund3 DirSvr3 33309 33310 33103 Director shund4 DirSvr4 33311 33312 33104 #---Server information for the searcher---# Searcher shuns01 SchSvr01 33501 shund1 Searcher shuns02 SchSvr02 33502 shund1 Searcher shuns03 SchSvr03 33503 shund2 Searcher shuns04 SchSvr04 33504 shund2 Searcher shuns05 SchSvr05 33505 shund3 Searcher shuns06 SchSvr06 33506 shund3 Searcher shuns07 SchSvr07 33507 shund4 Searcher shuns08 SchSvr08 33508 shund4 #---Server information for alternative searchers---# AlternativeSearcher shunsw01 SchWSvr01 33601 shund1 shund2 AlternativeSearcher shunsw02 SchWSvr02 33602 shund3 shund4 #---Character encoding for lookup data---# CharacterCode UTF-8 #---Case sensitivity for alphanumeric characters---# ANKmix 0 #---Skip character---# SkipChar “\s,\S,\n,\t” #---Query format---# ExtendedQuery 0 #---Delimiter---# SeparateChar "\t,\n,\s,!,\",#,$,%,&,',(,),*,+,\,,-,.,/,:,;,<,=,>,?,@,[,\\,],^,_,`,{,|,},~" #---Detection scope for search abnormalities---# SearcherFailureEscalation 0 #---Output format for monitoring---# MonitoringStyle 0

A-12

Conductor Environment Files

Conductor Environment Files Application environments for conductors are defined in conductor environment files.

Execution Parameters and Their Meanings The following table shows the execution parameters specified in conductor environment files and their meanings. Each parameter is detailed in the section below.

Parameter Name Meaning Mandatory or Optional Entry Example Remarks

LogFileFolder The directory where the conductor log file is stored

Optional “C:\Program Files\Interstage Shunsaku\Shunsaku\log\conductor\”

/var/opt/FJSVshnsk/log/conductor/

If this parameter is omitted, the conductor log file will not be created. Specify the absolute path of the directory.

LogFileSize The changeover size for the conductor log file (MB)

Optional 100 Default value: 100 MB Range: 1 to 500 MB

CoreFileFolder The directory where the conductor core file is stored

Mandatory “C:\Program Files\Interstage Shunsaku\Shunsaku\ core\conductor\”

/var/opt/FJSVshnsk/core/conductor/

Specify the absolute path of the directory.

AnsMax Maximum number of responses to the application

Optional 100 Default value: 100 Range: 1 to 100000

StartPoint The director number of the search-initiating director when search results are returned

Optional If File is not used: 1 If File is used: 1 file1

If this parameter is omitted, search results will be returned to directors in ascending order of director numbers. If director numbers have not been assigned to each director, Shunsaku will determine the order in which it returns search results to directors.

A-13



InsertPoint Information for the director where data is inserted

Optional If File is not used: shund1 If File is used: shund1 file1

Can be omitted if there is only one director. Cannot be omitted if a Shunsaku File is being used and there are two or more directors.

MaxConnection Maximum number of connections to the conductor



Optional file1 shund1 shund2 The director identifiers for the directors registered with the File (more than one director identifier can be specified)


Optional dom1 file1 file2 File names registered with the Domain (more than one file can be specified)

DefaultFile Default File information

Optional file1 Name of the File accessed by applications where no Shunsaku File has been specified

SystemName Shunsaku system name

Optional shunsaku


LogFileFolder This parameter specifies the absolute path of the directory for storing:

• Error event messages output by the conductor, and

• Files to which notification information is output.

The following files are created in this directory:

• conductor-identifier.log

This is the current conductor log file.

• conductor-identifier_old.log

This is the old conductor log file that was renamed when it reached the size specified by the LogFileSize parameter.

When the conductor-identifier.log file reaches the size specified by the LogFileSize parameter it is renamed as conductor-identifier_old.log and a new conductor-identifier.log file is created. Any existing conductor-identifier_old.log file will be overwritten.

If this parameter is omitted, the conductor log file will not be created.

A-14


Note

• Multi-byte characters cannot be specified in directory names.

• Specify the absolute paths for directory names using up to 220 characters.

• Specify a folder on the local disk in this parameter.

• If the folder name contains blank spaces or the hash character "#", enclose the entire folder name in double quotes.

• To increase the level of security for the system to be constructed, set up the access privileges to the folder specified by this parameter so that only users belonging to the Administrators group have full control.

The procedure for setting up the access privileges to this folder is shown below.

− For Windows 2000: 1. Select the Security tab of the Properties dialog box for the folder whose access privileges are to be set up. 2. Clear the Allow inheritable permissions from parent to propagate to this object checkbox. 3. Delete the Everyone group from the permissions list. 4. Add access permissions to the Administrators group, giving this group full control.

− For Windows Server 2003: 1. Select the Security tab of the Properties dialog box for the folder whose access privileges are to be set up. 2. Select the Set Details button. 3. Clear the Allow inheritable permissions from parent to propagate to this object and all its child objects and include them into the objects explicitly defined here checkbox in the Set Details dialog box. 4. Delete the Everyone group from the permissions list. Add access permissions to the Administrators group, giving this group full control.

• Directory names cannot include spaces or the hash character "#".

LogFileSize This parameter specifies the size at which the conductor log file will switch over. Specify a value between 1 and 500 inclusive. The units are megabytes.


When the size of the conductor log file exceeds the value specified in this parameter, the file will switch over automatically.

CoreFileFolder This parameter specifies the absolute path of the directory where the core file will be output if an abnormality occurs with the conductor.

The size of the core file that is output is the same as the amount of memory that the conductor uses.


A-15


Note


• Specify the absolute path for directory names using up to 220 characters.


• If the folder name contains blank spaces, enclose the entire folder name in double quotes (“ “).


Refer to the notes above for the LogFileFolder parameter in the conductor environment file for more information on how to set up the access privileges to this folder.

• Directory names cannot include spaces.

AnsMax This parameter specifies the maximum number of matching XML documents that can be returned from a search request from an application.. Specify a value between 1 and 100000 inclusive.


StartPoint This parameter is used to specify the order in which search results are returned to directors.

If Shunsaku File is not used:

Specify the first director in the sequence for returning search results to the application. This parameter can be specified only if a director number has been allocated to all directors. Specify values for this parameter in the following order:

• The number of the director where the search starts

The number corresponding to the specified director identifier. A unique number between 1 and 255 inclusive

If Shunsaku File is used:

Specify the first director in the specified File in the sequence for returning search results to the application. This parameter can be set only if a director number has been allocated to all directors in the specified File. Specify values for this parameter in the following order:

• The number of the director where the search starts

The number corresponding to the specified director identifier. A unique number between 1 and 255 inclusive

• File name

If this parameter is specified, search results will be returned to those directors whose director number is greater than or equal to the specified number, in ascending order of director numbers. Search results will then be returned to those directors whose director number is less than the specified number, in ascending order of director numbers.

A-16


InsertPoint If Shunsaku File is not used:

This parameter specifies the director where data will be inserted from the application. Specify information in the following order:

• Director identifier for the insertion destination director

“_LAST_” can be set for this parameter if a director number has been allocated to all directors. In this case, the last director in the search result return sequence becomes the data insertion destination.

If Shunsaku File is used:

This parameter specifies the director in a particular File where data will be inserted from the application. If there are multiple Files, separate settings must be made for each File. Specify information in the following order.

• Director identifier for the insertion destination director

“_LAST_” can be set for this parameter if a director number has been allocated to all directors in a particular file. In this case, the last director in the search result return sequence becomes the data insertion destination.

• File name

Note

• Specify the director identifier using up to eight alphanumeric characters.

• This parameter can be omitted if the data insertion destination is decided uniquely. In other words, this parameter can be omitted if no Shunsaku File is used in the environment, and there is only one director in the Shunsaku system. Similarly, this parameter can be omitted if Shunsaku File is used in the environment, and a File includes only one director. This parameter cannot be omitted if there are two or more directors.

MaxConnection

This parameter specifies the maximum number of simultaneous connections to the conductor. Specify a value between 1 and 30000 inclusive.


If the number of simultaneous connections to the conductor exceeds the number specified by this parameter, an error message will be output to the log or log file shown below and an error will be returned to the application.

• Application log (event log)

• The conductor log file

• System log (syslog)

• The conductor log file

A-17


File This parameter specifies file information. Specify information in the following order:

• File name

• The director identifiers for the directors registered with the File (more than one director identifier can be specified)

This parameter must be specified over more than one line if multiple Files are to be allocated.

The Shunsaku File function cannot be used if this parameter is omitted.

Note

Specify a string of up to 32 characters. Only alphanumeric characters can be used.

Domain This parameter specifies the scope that can be accessed by grouping multiple Files. Specify information in the following order.

• Domain name

• Name of the File registered with the Domain (more than one file can be specified)

This parameter must be specified over more than one line if multiple Domains are to be allocated.

The Domain function cannot be used if this parameter is omitted.

Note

Specify a string of up to 32 characters. Only alphanumeric characters can be used.

DefaultFile This parameter specifies the Shunsaku File that sets the scope of access permitted when applications are executed without a Shunsaku File specified. Specify the following information:

• File name or Domain name

If this parameter is omitted, the Shunsaku File to be accessed must be specified explicitly for APIs.

SystemName This parameter specifies the Shunsaku system name.

This parameter must be specified if the startup commands for V6.0 and earlier (shuncstart, shundstart, shunsstart, and shunostart) are used.

If this parameter is omitted, the startup commands for V6.0 and earlier (shuncstart, shundstart, shunsstart, and shunostart) cannot be used.

This parameter does not need to be specified if only the V7.0 startup command (shunsysstart) is used.

Note

Specify the Shunsaku system name using up to eight alphanumeric characters.

A-18


Entry Example An entry example for the execution parameters in the conductor environment file is shown below.

#---The directory where the conductor log file is stored---# LogFileFolder “C:\Program Files\Interstage Shunsaku\Shunsaku\log\conductor\” #---The changeover size for the conductor log file (MB)---# LogFileSize 100 #---The directory where the conductor core file is stored---# CoreFileFolder “C:\Program Files\Interstage Shunsaku\Shunsaku\core\conductor\” #---Maximum number of responses to the application---# AnsMax 100 #--- The director identifier for the data insertion destination---# InsertPoint shund1 file1 InsertPoint shund3 file2 #---Maximum number of connections to the conductor---# MaxConnection 1000 #---File information---# File file1 shund1 shund2 File file2 shund3 shund4 #---Domain information---# Domain dom1 file1 file2 #---Default File information---# DefaultFile file1

A-19


#---The directory where the conductor log file is stored---# LogFileFolder /var/opt/FJSVshnsk/log/conductor/ #---The changeover size for the conductor log file (MB)---# LogFileSize 100 #---The directory where the conductor core file is stored---# CoreFileFolder /var/opt/FJSVshnsk/core/conductor/ #---Maximum number of responses to the application ---# AnsMax 100 #---The director identifier for the data insertion destination---# InsertPoint shund1 file1 InsertPoint shund3 file2 #---Maximum number of connections to the conductor---# MaxConnection 1000 #---File information---# File file1 shund1 shund2 File file2 shund3 shund4 #---Domain information---# Domain dom1 file1 file2 #---Default File information---# DefaultFile file1

A-20

Director Environment Files

Director Environment Files Application environments for directors are defined in director environment files.

Execution Parameters and Their Meanings The following table shows the execution parameters specified in director environment files and their meanings. Each parameter is detailed in the section below.



DataFileFolder The directory where director data files are stored

Mandatory C:\Shunsaku\

/Shunsaku/


LogFileFolder The directory where the director log file is stored

Optional “C:\Program Files\Interstage Shunsaku\Shunsaku\log\director\”

/var/opt/FJSVshnsk/log/director/

If this parameter is omitted, the director log file will not be created. Specify the absolute path of the directory.

LogFileSize The changeover size for the director log file (MB)


CoreFileFolder The directory where the director core file is stored

Mandatory “C:\Program Files\Interstage Shunsaku\Shunsaku\core\director\”

/var/opt/FJSVshnsk/core/director/


OperationLogFolder The directory where operation log files are stored

Optional ”D:\Shunbackup\ ope\”

/Shunbackup/ope/

If this parameter is omitted, operation log files will not be created. Specify the absolute path of the directory.

A-21




AnsMax Maximum number of responses to the application


MultiMax The maximum number of search requests that can be received at once


SearcherWTimer The maximum time (in seconds) to wait for a search response from the searcher

Optional 180 Default value: 180 Range: 1 to 3600 seconds

ANKmix Case sensitivity for alphanumeric characters

Optional 0 Default value: Uses the value in the system environment file 0: Match case 1: Do not match case

SkipChar Skip character (characters that will not be searched)

Optional “\s,\S,\n,\t” Default value: Uses the value in the system environment file

ExtendedQuery Query format Optional 0 Default value: Uses the value in the system environment file 0: Do not use compatible format queries 1: Use compatible format queries

SeparateChar Delimiter Optional “\t,\n,\s,\”,!,$,.” Default value: Uses the value in the system environment file


Optional shunsaku

A-22



DataFileFolder This parameter specifies the absolute path of the directory for storing the director data files.

A subdirectory with the same name as the director identifier will be automatically created under the directory specified by this parameter, and the director data files will be created in this subdirectory.


Refer to 'The Size of Director Data Files' in Appendix C for information on the amount of disk space that these files require.

Note

• To back up director data files, use an operating system command to back up all of the files in the directory with the director identifier that was created under the directory specified by this parameter.






Refer to the notes for the LogFileFolder parameter in the conductor environment file for more information on how to set up the access privileges to this folder.


LogFileFolder This parameter specifies the directory for storing:

• Error event messages output by the director, and



• director identifier.log

This is the current director log file.

Note


Specify the absolute paths for directory names using up to 220 characters.

A-23







LogFileSize This parameter specifies the size at which the director log file will switch over. Specify a value between 1 and 500 inclusive. The units are megabytes.


When the size of the director log file exceeds the value specified in this parameter, the file will switch over automatically.

CoreFileFolder This parameter specifies the absolute path of the directory where the core file will be output if an abnormality occurs with the director.

The size of the core file that is output is the same as the amount of memory that the director uses.


Note








A-24


OperationLogFolder This parameter specifies the absolute path to the directory where operation log files are stored.

Files for storing information about operations on director data files will be created in the directory specified by this parameter.

This parameter cannot be omitted if recovery is performed using operation logs with the shundrecover command.

Note

Refer to 'The Size of Operation Log Files' in Appendix C for information on the amount of disk space that these files require.

• Place the storage directory for operation log files on a different disk to the storage directory for director data files.








AnsMax This parameter specifies the maximum number of matching XML documents that can be returned for search requests from applications that are directly connected to the director. Specify a value between 1 and 100000 inclusive.


MultiMax This parameter specifies the maximum number of search requests that can be received concurrently. Specify a value between 1 and 255 inclusive.


Note

Too large a value may require a large amount of memory.

A-25


SearcherWTimer Directors wait for responses from searchers. This parameter specifies longest time that a director should wait for a response. Specify a value between 1 and 3600 inclusive. The units are seconds.

If this parameter is omitted, it defaults to 180. If there is no response from a searcher within this time (starting from the time when the request is sent from the director to the searcher), the searcher is regarded as having an abnormality. If a searcher is found to be abnormal, the searcher will failover to an alternative searcher if one has been defined. If no alternative searchers have been defined, degradation will be performed.

ANKmix This parameter specifies whether to distinguish upper and lower case for alphanumeric characters in the strings that will be searched in a director.

0: Match case

1: Do not match case

If both this parameter and the “ANKmix” parameter in the system environment file have been specified, the value of this parameter takes effect.

If this parameter is omitted, the value of the “ANKmix” parameter in the system environment file is used.

Note

This parameter is valid with patterns and for string complete matches.

SkipChar This parameter specifies which characters are to be treated as skip characters (not part of the search target). Enclose the characters being specified in double quotes (“ “).

If both this parameter and the SkipChar parameter in the system environment file have been specified, the value of this parameter takes effect.

If this parameter is omitted, the value of the SkipChar parameter in the system environment file is used.

Refer to the “Note” and “Reference” in SkipChar in System Environment Files for more information on how to specify this parameter.

ExtendedQuery This parameter specifies whether to use compatible format queries

0: Do not use compatible format queries

1: Use compatible format queries

If both this parameter and the ExtendedQuery parameter in the system environment file have been specified, the value of this parameter takes effect.

If this parameter is omitted, the value of the ExtendedQuery parameter in the system environment file is used.

A-26


SeparateChar This parameter specifies the entire delimiter (enclosed in double quotes (“ “) ) when word searches are specified in search expressions.

If both this parameter and the SeparateChar parameter in the system environment file have been specified, the value of this parameter takes effect.

If this parameter is omitted, the value of the SeparateChar parameter in the system environment file is used.

Refer to SeparateChar in System Environment Files for more information on how to specify this parameter.





Note

• Specify the Shunsaku system name using up to eight alphanumeric characters.

Entry Example An entry example for the execution parameters in the director environment file is shown below.

#---The directory where director data files are stored---# DataFileFolder C:\Shunsaku\ #---The directory where the director log file is stored---# LogFileFolder “C:\Program Files\Interstage Shunsaku\Shunsaku\log\director\” #---The changeover size for the director log file (MB)---# LogFileSize 100 #---The directory where the director core file is stored---# CoreFileFolder “C:\Program Files\Interstage Shunsaku\Shunsaku\core\director\” #---The directory where operation log files are stored---# OperationLogFolder “D:\Shunbackup\ope\” #---Maximum number of responses to the application---# AnsMax 100 #---The maximum number of search requests that can be received at once---# MultiMax 100 #---The maximum time (in seconds) to wait for a search response from the searcher---# SearcherWTimer 180

A-27


#---Case sensitivity for alphanumeric characters---# ANKmix 0 #---Skip character---# SkipChar “\s,\S,\n,\t” #---Query format---# ExtendedQuery 0 #---Delimiter---# SeparateChar "\t,\n,\s,!,\",#,$,%,&,',(,),*,+,\,,-,.,/,:,;,<,=,>,?,@,[,\\,],^,_,`,{,|,},~"

#---The directory where director data files are stored---# DataFileFolder /Shunsaku/ #---The directory where the director log file is stored---# LogFileFolder /var/opt/FJSVshnsk/log/director/ #---The changeover size for the director log file (MB)---# LogFileSize 100 #---The directory where the director core file is stored---# CoreFileFolder /var/opt/FJSVshnsk/core/director/ #---The directory where operation log files are stored---# OperationLogFolder /Shunbackup/ope/ #---Maximum number of responses to the application---# AnsMax 100 #---The maximum number of search requests that can be received at once---# MultiMax 100 #---The maximum time (in seconds) to wait for a search response from the searcher---# SearcherWTimer 180 #---Case sensitivity for alphanumeric characters---# ANKmix 0 #---Skip character---# SkipChar “\s,\S,\n,\t” #---Query format---# ExtendedQuery 0 #---Delimiter---# SeparateChar "\t,\n,\s,!,\",#,$,%,&,',(,),*,+,\,,-,.,/,:,;,<,=,>,?,@,[,\\,],^,_,`,{,|,},~"

A-28

Sorter Environment Files

Sorter Environment Files Application environments for sorters are defined in sorter environment files.

Execution Parameters and Their Meanings The following table shows the execution parameters specified in sorter environment files and their meanings. Each parameter is detailed in the section below.

Parameter Name

Meaning Mandatory or Optional


MultiMax The maximum number of sort or aggregation requests that can be received at once


LogFileFolder The directory where the sorter log file is stored

Optional “C:\Program Files\Interstage Shunsaku\Shunsaku\log\sorter\”

/var/opt/FJSVshnsk/log/sorter/

If this parameter is omitted, the sorter log file will not be created. Specify the absolute path of the directory.

LogFileSize The changeover size for the sorter log file (MB)


CoreFileFolder The directory where the sorter core file is stored

Mandatory “C:\Program Files\Interstage Shunsaku\Shunsaku \core\sorter\”

/var/opt/FJSVshnsk/core/sorter/



Optional shunsaku


MultiMax Specify the maximum number of sort or aggregation requests that can be received at the same time. Specify a value between 1 and 50 inclusive.


An application error will be returned if the number of sort or aggregation requests received at the same time exceeds the value specified by this parameter.

A-29


LogFileFolder This parameter specifies the directory for storing:

• Error event messages output by the sorter, and



• sorter identifier.log

This is the current sorter log file.

• sorter identifier_old.log

This is the old sorter log file that was renamed when it reached the size specified by the LogFileSize parameter.

When the sorter-identifier.log file reaches the size specified by the LogFileSize parameter it is renamed as sorter-identifier_old.log and a new sorter-identifier.log file is created. Any existing sorter-identifier_old.log file will be overwritten.

If this parameter is omitted, the sorter log file will not be created.

Note








LogFileSize This parameter specifies the size at which the sorter log file will switch over. Specify a value between 1 and 500 inclusive. The units are megabytes.


When the size of the sorter log file exceeds the value specified in this parameter, the file will switch over automatically.

A-30

Sorter Environment Files

CoreFileFolder This parameter specifies the absolute path of the directory where the core file will be output if an abnormality occurs with the sorter.

The size of the core file that is output is the same as the amount of memory that the sorter uses.


Note












Note


A-31


Entry Example An entry example for the execution parameters in the sorter environment file is shown below.

#---The maximum number of sort or aggregation requests that can be received at once---# MultiMax 20 #---The directory where the sorter log file is stored---# LogFileFolder “C:\Program Files\Interstage Shunsaku\Shunsaku\log\sorter\” #---The changeover size for the sorter log file (MB)---# LogFileSize 100 #---The directory where the sorter core file is stored---# CoreFileFolder “C:\Program Files\Interstage Shunsaku\Shunsaku\core\sorter\”

#---The maximum number of sort or aggregation requests that can be received at once---# MultiMax 20 #---The directory where the sorter log file is stored---# LogFileFolder /var/opt/FJSVshnsk/log/sorter/ #---The changeover size for the sorter log file (MB)---# LogFileSize 100 #---The directory where the sorter core file is stored---# CoreFileFolder /var/opt/FJSVshnsk/core/sorter/

A-32

Searcher Environment Files

Searcher Environment Files Application environments for searchers are defined in searcher environment files.

Execution Parameters and Their Meanings The following table shows the execution parameters specified in searcher environment files and their meanings. Each parameter is detailed in the section below.



LogFileFolder The directory where the searcher log file is stored

Optional “C:\Program Files\Interstage Shunsaku\Shunsaku\log\searcher\”

/var/opt/FJSVshnsk/log/searcher/

If this parameter is omitted, the searcher log file will not be created. Specify the absolute path of the directory.

LogFileSize The changeover size for the searcher log file (MB)


CoreFileFolder The directory where the searcher core file is stored

Mandatory “C:\Program Files\Interstage Shunsaku\Shunsaku\core\searcher\”

/var/opt/FJSVshnsk/core/searcher/



Optional shunsaku


LogFileFolder This parameter specifies the absolute path of the directory for storing:

• Error event messages output by the searcher, and


A-33



• searcher identifier.log

This is the current searcher log file.

• searcher identifier_old.log

This is the old searcher log file that was renamed when it reached the size specified by the LogFileSize parameter.

When the searcher-identifier.log file reaches the size specified by the LogFileSize parameter it is renamed as searcher-identifier_old.log and a new searcher-identifier.log file is created. Any existing searcher-identifier_old.log file will be overwritten.

If this parameter is omitted, the searcher log file will not be created.

Note








LogFileSize This parameter specifies the size at which the searcher log file will switch over. Specify a value between 1 and 500 inclusive. The units are megabytes.


When the size of the searcher log file exceeds the value specified by this parameter, the file will switch over automatically.

CoreFileFolder This parameter specifies the absolute path of the directory where the core file will be output if an abnormality occurs with the searcher.

The size of the core file that is output is the same as the amount of memory that the searcher uses.


Note

• Multi-byte characters cannot be specified in directory names


A-34

Searcher Environment Files







This parameter must be specified if the startup commands for V6.0 and earlier shuncstart, shundstart, shunsstart, and shunostart) are used.



Note


Entry Example An entry example for the execution parameters in the searcher environment file is shown below.

#---The directory where the searcher log file is stored---# LogFileFolder “C:\Program Files\Interstage Shunsaku\Shunsaku\log\searcher\” #---The changeover size for the searcher log file (MB)---# LogFileSize 100 #---The directory where the searcher core file is stored---# CoreFileFolder “C:\Program Files\Interstage Shunsaku\Shunsaku\core\searcher\”

A-35


#---The directory where the searcher log file is stored---# LogFileFolder /var/opt/FJSVshnsk/log/searcher/ #---The changeover size for the searcher log file (MB)---# LogFileSize 100 #---The directory where the searcher core file is stored---# CoreFileFolder /var/opt/FJSVshnsk/core/searcher/

A-36

API Environment Files

API Environment Files Application environments for API are defined in API environment files.

Execution Parameters and Their Meanings The following table lists the execution parameters specified in API environment files and their meanings. Each parameter is detailed in the section below.


Host

Host name or IP address of the connection destination

Optional DirSvr1 Default value: Localhost

Port Port number for the connection destination

Optional 33101 Default value: 33101

ShunsakuFile

Shunsaku File name for the connection destination

Optional File1 Omitted if Shunsaku Files are not used

WaitTimer

Maximum time to wait for a response from Shunsaku (seconds)


SnapLevel API snapshot output level Optional 1 Default value: 0

LogFile Log file where API snapshots are output

Optional

C:\api\api.log

/api/api.log

This parameter must be specified if “1” or “2” has been specified for the SnapLevel parameter.

LogFileSize

Changeover size (MB) for the log file where API snapshots are output


LogFileBackup

Number of backups for the log file where snapshots are output


A-37



Host This parameter specifies the host name or IP address of the connection destination.

If this parameter is omitted, it defaults to “localhost”.

Port This parameter specifies the port number for the connection destination.


ShunsakuFile This parameter specifies a File name or Domain name for the connection destination Shunsaku File.

Note

Specifications for the Shunsaku File have the following priority order.

1. The specification made by the ShunConnection class constructor or the ShunConnect function

2. The specification made by this parameter

3. The specification made by the DefaultFile parameter in the conductor environment file

WaitTimer Specify the maximum time (in seconds) to wait for a response from Shunsaku. Specify a value between 0 and 3600 (seconds) inclusive. . If no response is received from Shunsaku within the specified time, a timeout will occur, an error will be sent to the application and the connection will be closed automatically.

If “0” is specified, the application will continue waiting for a response.


SnapLevel This parameter specifies the output level for the API snapshot.

SnapLevel Content Output

0 Does not output the API snapshot.

1 Outputs the API call sequence.

2 Outputs parameter information and data manipulation information when methods or functions are executed, in addition to level 1 information.


Note

If level 0 is specified execution parameters (LogFile, LogFileSize and LogFileBackup) do not need to be specified.

A-38

API Environment Files

LogFile This parameter specifies the absolute path to the log file where the API snapshot is output. It is used cyclically when it reaches the size specified by the LogFileSize parameter.

Be sure to specify this parameter if "1" or "2" has been specified for the SnapLevel parameter. If this parameter is omitted, API snaps will not be output.

A separate API snap is created for each application process.

The process ID is added to the file name as shown in the following examples.

Example

The absolute path specified for the LogFile parameter C:\api\api.log

The log file that is actually created C:\api\api_2435.log

Example

The absolute path specified for the LogFile parameter /api/api.log

The log file that is actually created /api/api_2435.log

LogFileSize This parameter specifies the file size at which the log file where the API snapshot is output switches over. Specify a value between 1 and 500 (megabytes) inclusive.

If this parameter is omitted, it defaults to 100

For example, if 1 is specified, a backup file will be created automatically when the file size exceeds 1 MB.

LogFileBackup This parameter specifies the number of backups for the log files where API snaps are output. Specify a value between 0 and 2147483647 inclusive. If the number of backups exceeds the number specified by this parameter, the content of the oldest backup file will be lost.

For example, suppose "3" is specified for this parameter and three backup files called "api_2435.log.1", "api_2435.log.2" and "api_2435.log.3" are created. The next time a backup is required, the content of "api_2435.log.1" will be lost.


A-39


Entry Example

#---Host name of the connection destination---# Host DirSvr1 #---Port number for the connection destination---# Port 33101 #---Maximum time to wait for a response---# WaitTimer 300 #---Snapshot output level---# SnapLevel 1 #---Storage directory for the log file---# LogFile “C:\api\api.log” #---Log file size---# LogFileSize 100 #--- Number of backups of the log file---# LogFileBackup 3

#---Host name of the connection destination---# Host DirSvr1 #---Port number for the connection destination---# Port 33101 #---Maximum time to wait for a response---# WaitTimer 300 #---Snapshot output level---# SnapLevel 1 #---Storage directory for the log file---# LogFile /api/api.log #---Log file size---# LogFileSize 100 #---Number of backups of the log file---# LogFileBackup 3

A-40

Correspondence between V6.0 and V7.0 Execution Parameters


The following tables show which execution parameters have been moved from one environment file to another inV7.0, which parameters have been deleted, and which parameters are compatible.

Parameters that have been Moved from a V6.0 Environment file to a V7.0 Environment File The following table lists execution parameters that define the Shunsaku operating environment, which have been moved from V6.0 environment file to a V7.0 environment file.

Original File V6.0 Parameter New File Resulting V7.0 Parameter

DirectorInfo - Director number

(first parameter) - Director identifier

(second parameter)

Director - Director number

(sixth parameter) - Director identifier

(first parameter)

MySearchPort Conductor - The port number for receiving requests

(third parameter)

SorterInfo - IP address or host

name (first parameter)

- Port number (second parameter)

Sorter - IP address or host name

(second parameter) - Port number

(third parameter)

CharacterCode

System environment files

CharacterCode

Conductor environment files

WaitTimer API environment files

WaitTimer (*1)

SearcherSearchPort Director - Search port number for

searchers (third parameter)

SearcherEntryPort Director - Update port number for

searchers (fourth parameter)

MySearchPort Director - The port number for receiving

requests (fifth parameter)

Director environment files

CharacterCode CharacterCode

Sorter environment files

MySortPort

System environment files

Sorter - Port number

(third parameter)

A-41


Original File V6.0 Parameter New File Resulting V7.0 Parameter

DirSvrName Director - IP address or host name

(second parameter)

DirectorSearchPort Director - Search port number for

searchers (third parameter)

DirectorEntryPort Director - Update port number for

searchers (fourth parameter)

Searcher environment files

MyPort

Searcher - Port number for searchers

(third parameter)

*1) The settings for the WaitTimer parameter in the conductor environment file are valid for applications executed on machines where API functions from V6.0 have been installed. To use the same function with applications that use APIs from V7.0 or later, specify the WaitTimer parameter in the API environment file.

Note

• If any parameters from the Resulting V7.0 Parameter column are specified in a file shown in the ’Original File’ column, an error will occur when Shunsaku starts.

Parameters that have been Deleted in V7.0 The following table lists execution parameters from V6.0 that have been removed for the Shunsaku V7.0 operating environment as they are not required.

Source file Deleted Parameter

Conductor environment files Memkey

Director environment files MemKey

Director environment files MyUpdatePort

Director environment files SearcherCnt

Director environment files WorkFolder

Sorter environment files MemKey

Searcher environment files MemKey

Note

• If any parameters from the table above are specified, an error will occur when Shunsaku starts.

A-42


Compatible Parameters The following table lists execution parameters that can exist in environment files that define the Shunsaku operating environment for both V6.0 and V7.0 .

Compatible Parameter

Conductor environment files WaitTimer (*1)

Director environment files DataRevSize

Director environment files SearcherDataEquality

*1) The settings for the WaitTimer parameter in the conductor environment file are valid for applications that are executed on machines where API functions from V6.0 have been installed.

To use the same function with applications that use APIs from V7.0 or later, specify the WaitTimer parameter in the API environment file.

A-43


A-44

Appendix B Allowable Values

The following table shows the values allowed for Shunsaku.

Item name Allowable values

Number of directors that can be registered with a single conductor

Up to 255

Number of searchers that can be registered with a single director

Up to 20

Number of alternative searchers that can be registered with a single director

Up to 20

Shunsaku system name Up to 8 bytes (alphanumeric characters)

Conductor identifier Up to 8 bytes (alphanumeric characters)

Director identifier Up to 8 bytes (alphanumeric characters)

Sorter identifier Up to 8 bytes (alphanumeric characters)

Searcher identifier Up to 8 bytes (alphanumeric characters)

Host name of director server Up to 127 bytes

Host name of search server Up to 127 bytes

Host name of sort server Up to 127 bytes

Length of each line in environment file Up to 4,095 bytes (excluding linefeed characters)

DataFileFolder Up to 220 bytes

LogFileFolder Up to 220 bytes

CoreFileFolder Up to 220 bytes

OperationLogFolder Up to 220 bytes

File name Up to 32 bytes (alphanumeric characters)

Domain name Up to 32 bytes (alphanumeric characters)

B-1

Appendix B: Allowable Values

Item name Allowable values

Number of directors that can be registered with a single file

Up to 255

Number of files that can be registered with a single domain

Up to 255, as long as they can be specified in a single line of an environment file

B-2

Appendix C Estimating Resources

This appendix explains how to estimate Shunsaku resources.

• Estimating Resources for Each Server

• Estimating Resources for the Conductor

• Estimating Resources for Sorter

• Estimating Resources for the Director

• Estimating Resources for Searcher

C-1

Appendix C: Estimating Resources

Estimating Resources for Each Server This section explains how to estimate resources for each Shunsaku server.

Resources can be estimated by using estimation formulas to calculate the resources for the processes on each server, and then totaling the results.

The estimation procedure is as follows:

1. Confirm which processes have been allocated to each server. Refer to 'System Configuration' in Chapter 2, Design and the Installation Guide for more information.

2. Estimate process resources. Use the estimation formulas to estimate the resources for each process.

3. Total the process resources for each server. Determine the resources required for each server by totaling the resources required for the processes on each server.

C-2

Estimating Resources for the Conductor

Estimating Resources for the Conductor Conductors require the following resources:

• The Size of Conductor Log Files

• Memory Requirements for Conductors

The Size of Conductor Log Files The formula for estimating the size of conductor log files is shown below.

Definition Unit Variable Elements Size (byte)

Conductor log files Changeover size for the conductor log file (*1)

Changeover size for the conductor log file × 2

*1 This is specified in the LogFileSize parameter in the conductor environment file.

Example Calculation An example calculation for the size of a conductor log file is shown below.

If the variable elements are as follows:

• Changeover size for the conductor log file: 100 (MB)

Conductor log file 100 MB x 2 = 200 MB

Memory Requirements for Conductors This section explains the formulas for estimating how much memory conductors will use.

Shared Memory The shared memory requirement is shown below.

Definition Unit Variable Elements Size (bytes)

The area for storing command information

Fixed 1,000,000

C-3


Local Memory The formulas for estimating local memory are shown below.


The area for storing search requests

The maximum number of requests that the conductor can receive simultaneously (*1)

192,000 x the maximum number of requests that the conductor can receive simultaneously

The area for storing connection information

- The maximum number of requests that the conductor can receive simultaneously (*1)

1,536 x the maximum number of requests that the conductor can receive simultaneously

The area for storing hit results

- Average record size

- Maximum number of responses to the application (*2)

- The maximum number of requests that the conductor can receive simultaneously (*1)

(32 + average record size) x the maximum number of responses to the application x the maximum number of requests that the conductor can receive simultaneously

The area for storing director information

The number of directors registered with the conductor (*3)

800 x 1,024 x the number of directors registered with the conductor

The area for storing sorter information

The number of sorters registered with the conductor (*4)

400 x 1,024 x the number of sorters registered with the conductor

*1 This is specified in the MaxConnection parameter in the conductor environment file.

*2 This is specified in the AnsMax parameter in the conductor environment file.

*3 This is specified in the Director parameter in the system environment file.

*4 This is specified in the Sorter parameter in the system environment file.

Example Calculation An example calculation for local memory is shown below.


• The maximum number of requests that the conductor can receive simultaneously: 1,000

• Average record size: 4,000 (bytes)

• Maximum number of responses to the application: 100

• The number of directors registered with the conductor: 1

• The number of sorters registered with the conductor: 1

C-4

Estimating Resources for the Conductor

The area for storing search requests

192,000 x 1,000 = 192,000,000 bytes (approx. 200 MB)

The area for storing connection information

1,536 x 1,000 = 1,536,000 bytes (approx. 1.5 MB)


(32 + 4,000) x 100 x 1,000 = 403,200,000 bytes (approx. 400 MB)

The area for storing director information

800 x 1,024 x 1 = 819,200 bytes (approx. 800 KB)

The area for storing sorter information

400 x 1,024 x 1 = 409,600 bytes (approx. 400 KB)

C-5


Estimating Resources for Sorter Sorters require the following resources:

• The Size of Sorter Log Files

• Memory Requirements for Sorters

The Size of Sorter Log Files The formula for estimating the size of sorter log files is shown below.


Sorter log file Changeover size for the sorter log file (*1)

Changeover size for the sorter log file x 2

*1 This is specified in the LogFileSize parameter in the sorter environment file.

Example Calculation An example calculation for the size of a sorter log file is shown below.


• Changeover size for the sorter log file: 100 (MB)

Sorter log file 100,000,000 x 2 = 200,000,000 bytes (approx. 200 MB)

Memory Requirements for Sorters This section explains the formulas for estimating how much memory sorters will use.


Definition Unit Variable Elements Size (byte)


Fixed 1,000,000

C-6

Estimating Resources for Sorter


Definition unit Variable elements Size (byte)

Fixed None 5,000,000

Threads per request The maximum number of sort or aggregation requests that can be received at the same time (*1)

1,000,000 x the maximum number of sort or aggregation requests that can be received at the same time

Memory for sort calculation

The maximum number of sort or aggregation requests that can be received at the same time (*1)

30,000,000 x the maximum number of sort or aggregation requests that can be received at the same time

Memory for storing reply data

Average reply data size

The maximum number of sort or aggregation requests that can be received at the same time (*1)

Average reply data size x 1,000 x the maximum number of sort or aggregation requests that can be received at the same time

*1 This is specified in the MultiMax parameter in the sorter environment file.



• The maximum number of sort or aggregation requests that can be received at the same time: 20

• Average reply data size: 4,000 (bytes)

Threads per request

1,000,000 x 20 = 20,000,000 bytes (approx. 20 MB)

C-7


Memory for sort calculation

30,000,000 x 20 = 600,000,000 bytes (approx. 600 MB)

Memory for storing reply data

4,000 x 1,000 x 20 = 80,000,000 bytes (approx. 80 MB)

C-8

Estimating Resources for the Director

Estimating Resources for the Director Directors require the following resources:

• The Size of Director Data Files

• The Size of Operation Log Files

• The Size of Director Log Files

• Memory Requirements for Directors

The Size of Director Data Files The formula for estimating the size of director data files is shown below.

Definition Unit Variable lements Size (bytes)

Director data file - Standard record size (bytes)

- The total number of records

- Temporary area for optimization

- Safety factor (at least 1.3)

(70 + standard record size) x the total number of records x 2 x safety factor

Example Calculation An example calculation for the size of a director data file is shown below.


• Standard record size: 4,000 (bytes)

• The total number of records: 100,000

• Safety factor: 1.3

(70 + 4,000) x 100,000 x 2 x 1.3 = 1,058,200,000 bytes (approx. 1,010 MB)

C-9


The Size of Operation Log Files The formula for estimating the size of operation log files is shown below.


Operation log file - Standard record size (bytes)

- Number of records updated during the backup cycle

- Safety factor (at least 1.3)

(100 + standard record size) x number of records updated during the backup cycle x safety factor

The number of records updated during the backup cycle is the number of records that are added, deleted, or updated from one backup of the director data files to the next.

Example Calculation An example calculation for the size of an operation log file is shown below.


• Standard record size: 4,000 (bytes)

• Number of records updated during the backup cycle: 10,000

• Safety factor: 1.3

(100 + 4,000) x 10,000 x 1.3 = 53,300,000 bytes (approx. 53 MB)

The Size of Director Log Files The formula for estimating the size of director log files is shown below.


Director log file Changeover size for the director log file (*1)

Changeover size for the director log file x 2

*1 This is specified in the LogFileSize parameter in the director environment file.

C-10


Example Calculation An example calculation for the size of a director log file is shown below.


• Changeover size for the director log file: 100 (MB)

Director log file 100,000,000 x 2 = 200,000,000 bytes (approx. 200 MB)

Memory Requirements for Directors This section explains the formulas for estimating how much memory directors will use.




Fixed 1,000,000



Amount of memory used during startup

- The maximum number of search requests that can be received at the same time (*1)


2,420,000 + 140,000 x the maximum number of search requests that can be received at the same time + 256 x the total number of records + the maximum number of search requests that can be received at the same time x the total number of records

C-11



Variable amount of memory used for search requests


- Maximum number of responses to the application (*2)

- Average search expression size

- Average return expression size

- Average sort expression size


- Standard size (in bytes) of records (*3)

(110 + average search expression size + average return expression size + average sort expression size + standard size of records x maximum number of responses to the application) x the maximum number of search requests that can be received at the same time

Variable amount of memory used for update requests

- The average number of updated records within a transaction (*4)

- The maximum number of transactions (*5)

- Standard size (in bytes) of records (*3)

(6,640 + 128 + standard size of records) x average number of updated records within a transaction) x maximum number of transactions

*1 This is specified in the MultiMax parameter in the director environment file.

*2 This is specified in the AnsMax parameter in the director environment file.

*3 Specify the standard size of the records to be searched or added/updated.

*4 Specify the average number of records to be updated per transaction.

*5 Specify the maximum number of transactions to be executed simultaneously.

C-12




• The maximum number of search requests that can be received at the same time: 100

• Maximum number of responses: 100

• The total number of records: 100,000

• Average search expression size: 1,024 (bytes)

• Average return expression size: 1,024 (bytes)

• Average sort expression size: 1,024 (bytes)

• Maximum number of transactions: 100

• Average number of update records within a transaction: 10

• Standard size of records: 4,000 (bytes)

Amount of memory used during startup

2,420,000 + 40,000 x 100 + 256 x 100,000 + 100 x 100,000 = 52,020,000 bytes (approx. 52 MB)

Variable amount of memory used for search requests

(110 + 1,024 + 1,024 + 1,024 + 4,000 x 100) x 100 = 40,318,200 bytes (approx. 40 MB)

Variable amount of memory used for update requests

(6,640 + (128 + 4,000) x 10) x 100 = 4,792,000 bytes (approx. 4.8 MB)

C-13


Estimating Resources for Searcher Searchers require the following resources:

• The Size of Searcher Log Files

• Memory Requirements for Searchers

The Size of Searcher Log Files The formula for estimating the size of searcher log files is shown below.


Searcher log file Changeover size for the searcher log file (*1)

Changeover size for the searcher log file x 2

*1 This is specified in the LogFileSize parameter in the searcher environment file.

Example Calculation An example calculation for the size of a searcher log file is shown below.


• Changeover size for the searcher log file: 100 (MB)

Searcher log file 100,000,000 x 2 = 200,000,000 bytes (approx. 200 MB)

Memory Requirements for Searchers This section explains the formulas for estimating how much memory searchers will use.




Fixed 1,000,000

C-14

Estimating Resources for Searcher



Fixed memory - None 14,000,000

Memory allocated for real data


- Number of records allocated to the searcher (*1)

(Average record size + 10) x number of records allocated to the searcher

The area for storing search requests (*2)

- Average number of characters in keywords (*3)


(Average number of characters in search expressions + 1,024) x the maximum number of search requests that can be received at the same time

The area for storing update requests


- Average number of update records within a transaction (*5)

- Maximum number of transactions (*6)

((Average record size+ 50) x average number of update records within a transaction + 1600) x maximum number of transactions


- Number of records allocated to the searcher (*1)


(Number of records allocated to the searcher + 10) x the maximum number of search requests that can be received at the same time

*1 Specify the number of records of search data allocated to the searcher.

*2 Note the following three points:

− The minimum value that can be used is 10 MB.

− When searches are executed using partial character specifications, character range specifications or numeric range specifications, the formula changes as follows: (number of partial characters or character range or numeric range x average number of characters for keywords x 1,024) x the maximum number of search requests that can be received at the same time

C-15


− When searches are executed using a combination of partial character specifications, character range specifications and numeric range specifications, the formula changes as follows: (number of partial characters x character range x numeric range x average number of characters for keywords x 1,024) x the maximum number of search requests that can be received at the same time

*3 Specify the average number of characters (bytes) used for keywords in search expressions.

*4 This is specified in the MultiMax parameter in the director environment file.

*5 Specify the average number of records to be updated per transaction.

*6 Specify the maximum number of transactions to be executed simultaneously.



• Average record size: 4,000 (bytes)

• Number of records allocated to the searcher: 10,000

• Average size of search expressions: 200 (bytes)

• Average number of characters in keywords: 80(bytes)

• The maximum number of search requests that can be received at the same time: 100

• Average number of update records within a transaction: 10

• Maximum number of transactions: 100

Memory allocated for real data (4,000 + 10) x 10,000 = 40,100,000 bytes (approx. 40MB)

The area for storing search requests (80 x 1024) x 100 = 8,192,000 bytes (approx. 8.1 MB) ->10MB

The area for storing update requests ((4,000 + 50) x 10 + 1,600) x 100 = 4,210,000 bytes (approx. 4.2 MB)

The area for storing hit results (10,000 + 10) x 100 = 1,001,000 bytes (approx. 1 MB)

C-16

Appendix D Conflict Relationships

This appendix explains both the conflict relationships between applications and commands, and also the conflict relationships between commands.

• Conflict Relationships between Applications and Commands

• Conflict Relationships between Commands

D-1

Appendix D: Conflict Relationships

Conflict Relationships between Applications and Commands

This section explains the following conflict relationships that apply when applications and commands are executed simultaneously.

• Leading Command/Following Application

This is the case where an application is executed while a command is executing.

• Leading Application/Following Command

This is the case where a command is executed while an application is executing.

The following symbols are used in the tables below:

−: Commands and APIs are never executed simultaneously

: No conflict

∆: Conflict under some conditions

x: Conflict

Behavior when conflict occurs: − If the explanation starts with ∆, the conditions for conflict to occur will be listed

(e.g. ∆: If there is a transaction) − If the explanation starts with x, the behavior when conflict occurs will be listed.

Leading Command/Following Application The following table shows the API classifications for the ‘leading command/following application’ scenario.

APIs not listed below do not conflict, and so do not need to be considered.

Table D-1 API Classifications for the Leading Command/Following Application

Java-API Classification C-API

Class Method

Establishing connections

ShunConnect ShunConnection ShunConnection()

ShunInsert ShunPreparedStatement executeInsert()

ShunDeleteRecId ShunPreparedRecordID deleteByRecordID()

Starting updates/ transactions

ShunUpdateRecId ShunPreparedRecordID updateByRecordID()

ShunCommit ShunConnection commit() Finishing transactions

ShunRollback ShunConnection rollback()

D-2


Java-API Classification C-API

Class Method

ShunGetHitCount ShunPreparedStatement executeSearch()

ShunSearch ShunPreparedStatement executeSearch()

Looking up data

ShunSearchRecId ShunPreparedRecordID searchByRecordID()

Conflict behavior for the ‘leading command/following Application’ scenario in the following table.

Table D-2 Conflict Behavior for the Leading Command/Following Application

Following application Leading command



Finishing transactions

Looking up data

Conflict conditions and behavior when conflict occurs (explanations start with either a ∆ or a x) (e.g. ∆: if there is a sort expression) (e.g. x: the API will produce an error)

shunsysstart x - - - x: The API will produce an error

shunsysstop x - - - x: The API will produce an error

shunsyscfgedit (except for dynamic changes to the Shunsaku File)

shunsyscfgedit (dynamically changing the Shunsaku File)

∆ ∆ ∆ ∆ ∆: When a director is deleted from a File or a File is deleted from a Domain, all existing connections to the director/File will be closed and new connections will be rejected.

shunsysstate

shuncstate

shuncterm

shuncservice

- - - -

D-3






Looking up data


shundstate

shundclear x - x: The API will produce an error.

shundimport ∆ ∆: The API will produce an error if the records being deleted by the shundimport command conflict with the records being updated or deleted by the API.

shundexport x - x: The API will produce an error.

shundresend x - x: The API will produce an error.

shundcds x - x: The API will produce an error.

shundservice

- - - - x: The API will produce an error.

shundbackup (Start declaration)

∆ - ∆: The API will produce an error if there is no specification for the OperationLogFolder parameter.

shundbackup (End declaration)

∆ - ∆: The API will produce an error if there is no specification for the OperationLogFolder parameter.

shundrecover (Start declaration)

x - x x: The API will produce an error.

D-4






Looking up data


shundrecover (End declaration)

x - x x: The API will produce an error.

shunostate

shunoservice - - - -

∆: The API will produce an error if there is a sort expression

shunsstate

shunsservice

- - - - x: The API will produce an error.

shunprtmsg

Leading Application/Following Command Conflict behavior for the ‘leading application/following command’ scenario is as follows:

Table D-3 Conflict Behavior for the Leading Application/Following Command

State of the executing application

Already connected

Executing a transaction

Looking up data

Following command

Conflict conditions and behavior when conflict occurs (Explanations start with either a ∆ or a ×) (e.g. ∆: If there is a sort expression) (e.g. ×: The API will produce an error)

- - - shunsysstart

x x x shunsysstop

(stop normally)

x: The connection will be closed if the application is already connected.

x: The command will terminate abnormally if the application is executing a transaction

x: Looking up data will produce an error.

D-5



Already connected


Looking up data

Following command


x x x shunsysstop

(stop forcibly)

x: The connection will be closed.

x: Transactions will be rolled back.

x: Looking up data will produce an error.

shunsyscfgedit

(Except for dynamic changes to the Shunsaku File)

∆ ∆ ∆ shunsyscfgedit

(Dynamically changing the Shunsaku File)

∆: The connection will be closed if the Shunsaku File being connected is deleted or its configuration is changed.

∆: Transactions will be rolled back if the Shunsaku File being updated is deleted or its configuration is changed.

∆: An error will occur if the Shunsaku File that the reference API is trying to access is deleted or its configuration is changed.

shunsysstate

shuncstate

D-6



Already connected


Looking up data

Following command


∆ ∆ ∆ shuncterm ∆: The connection will be closed if the application is already connected and the machine making the connection request is the one targeted by the shuncterm command (the application server).

∆: The transaction will be rolled back if the application is executing a transaction and the machine initiating the transaction is the one targeted by the shuncterm command (the application server).

∆: The data lookup will produce an error if the application is looking up data and the machine making the lookup request is the one targeted by the shuncterm command (the application server).

- - - shuncservice

shundstate

x shundclear x: The command will produce an error

∆ shundimport ∆: The shundimport command will produce an error if the records to be deleted by the shundimport command conflict with the records being updated or deleted by the API.

x shundexport x: The command will produce an error.

x shundresend x: The command will produce an error.

D-7



Already connected


Looking up data

Following command


x shundcds x: The command will produce an error.

- - - shundservice

∆ shundbackup

(Start declaration)

∆: The command will wait until the transaction finishes.

∆ shundbackup

(End declaration)


∆ shundrecover

(Start declaration)


- - shundrecover

(End declaration)

shunostate

- - - shunoservice

shunsstate

- - - shunsservice

shunprtmsg

D-8

Conflict Relationships between Commands

Conflict Relationships between Commands This section explains the conflict relationships between commands.

Command conflicts occur separately for conductors, directors, sorters and searchers. The following conflict relationships are explained:

• Conflict Relationships for Conductor Commands

• Conflict Relationships for Director Commands

• Conflict Relationships for Sorter Commands

• Conflict Relationships for Searcher Commands

Note

• For server commands, conflicts occur for all conductors, directors, sorters and searchers on the server where the command is submitted. Accordingly, server commands will be included in each table.

• The following commands do not enter into any conflict relationships, as they must be executed while the conductor, director, sorter, or searcher is stopped.

− shuncservice

− shundservice

− shunoservice

− shunsservice

Conflict Relationships for Conductor Commands The conflict relationships for conductor commands are as follows:

Table D-4 Conflict Relationships for Conductor Commands

Following command

shuncstate shunsysstate shunprtmsg

shuncterm shunsyscfgedit shunsysstart shunsysstop

shuncstate shunsysstate shunprtmsg

Can be executed simultaneously

Can be executed simultaneously Leading command

shuncterm shunsyscfgedit shunsysstart shunsysstop


Cannot be executed simultaneously (*1)

*1 The following command will return an error.

D-9


Conflict Relationships for Director Commands The conflict relationships for director commands are as follows:

Table D-5 Conflict Relationships for Director Commands

Following command

shundstate shunsysstate shunprtmsg

shundbackup shundcds shundcfgedit shundclear shundexport shundimport shundrecover shundresend shunsysstart shunsysstop

shundstate shunsysstate shunprtmsg



shundbackup shundcds shundcfgedit shundclear shundexport shundimport shundrecover shundresend shunsysstart shunsysstop



During the backup period (*2)

Can be executed Cannot be executed simultaneously (*4)

Leading command

During the recovery period (*3)

Can be executed Cannot be executed simultaneously (*5)


*2 This is the period from when the backup start declaration is made using the shundbackup command until the backup end declaration is made.

*3 This is the period from when the recovery start declaration is made using the shundrecover command until the recovery end declaration is made.

*4 The following command will return an error unless the backup end declaration or the backup start declaration is canceled using the shundbackup command.

*5 The following command will return an error unless the recovery end declaration or the recovery start declaration is canceled using the shundrecover command.

D-10

Conflict Relationships between Commands

Conflict Relationships for Sorter Commands The conflict relationships for sorter commands are as follows:

Table D-6 Conflict Relationships for Sorter Commands Following command

shunostate

shunsysstate

shunprtmsg

shunsysstart

shunsysstop

shunostate

shunsysstate

shunprtmsg



shunsysstart

shunsysstop




Conflict Relationships for Searcher Commands The conflict relationships for searcher commands are as follows:

Table D-7 Conflict Relationships for Searcher Commands Following command

shunsstate

shunsysstate

shunprtmsg

Shunsysstart

shunsysstop

shunsstate

shunsysstate

shunprtmsg



shunsysstart

shunsysstop




D-11


D-12

Appendix E Sample Programs

This appendix presents sample programs that use the transaction function and the Shunsaku File function.

• Sample Program Using the Java APIs

• Sample Program Using the C APIs

Usage examples are provided for Java and C APIs, using the university course registration example explained in 'Overview of Transactions' in Chapter 4, Transactions.

E-1

Appendix E: Sample Programs

Sample Program Using the Java APIs The following is a sample program using the Java APIs. import com.fujitsu.shun.ShunConnection; import com.fujitsu.shun.ShunPreparedRecordID; import com.fujitsu.shun.ShunPreparedStatement; import com.fujitsu.shun.ShunResultSet; import com.fujitsu.shun.common.ShunException; public class TranAppSample { public static void main(String[] args) { ShunConnection con = null; ShunPreparedStatement pstmt = null; ShunPreparedRecordID rid = null; ShunResultSet rs = null; try { // Set up a search expression and a return expression for searching for the course where the number of participants is to be increased // Search expression String sCourseQuery = "/course/name == 'Business negotiation'"; // Return expression String sCourseReturn = "/"; //Used to get the ID of the record to be updated String updateCourseRecordID = null; //Update data String updateCourseData = "<course>" + " <name>Business negotiation</name>" + " <instructor>" + " <first-name>Max</first-name>" + " <last-name>Cameron</last-name>" + " </instructor>" + " <capacity>40</capacity>" + " <current-auditors>31</current-auditors>" + "</course>"; //Set up a search expression and a return expression for searching for the students to be added to the attendance information // Search expression String sStudentQuery = "/student/e-mail == 'mary\\.tompson@def\\.com'"; // Return expression String sStudentReturn = "/"; //Used to get the ID of the record to be updated String updateStudentRecordID = null; //Update data String updateStudentData = "<student>" + " <first-name>Mary</first-name>" + " <last-name>Tompson</last-name>" + " <e-mail>[email protected]</e-mail>" + " <course>Chinese language</course>" + " <course>Business negotiation</course>"

E-2

Sample Program Using the Java APIs

+ "</student>"; // Create the ShunConnection object // Host name for the connection destination: DirSvr1 // Port number: 33101 // If the Shunsaku File name is shunsakuFile1 con = new ShunConnection("DirSvr1", 33101, "shunsakuFile1"); //Search for the course information and student information to be updated // Specify a search expression and create the ShunPreparedStatement object pstmt = con.prepareSearch(sCourseQuery, sCourseReturn); // Execute the search and create the ShunResultSet object rs = pstmt.executeSearch(); // Get the record ID for the course information obtained, and the XML document for that record while (rs.next()) { updateCourseRecordID = rs.getRecordID(); } rs.close(); pstmt.close(); // Specify a search expression and create the ShunPreparedStatement object pstmt = con.prepareSearch(sStudentQuery, sStudentReturn); // Execute the search and create the ShunResultSet object rs = pstmt.executeSearch(); // Get the record ID for the student information obtained, and the XML document for that record while (rs.next()) { updateStudentRecordID = rs.getRecordID(); } rs.close(); pstmt.close(); // Turn off auto-commit mode con.setAutoCommit(false); // Update course information rid = con.prepareUpdateRecordID(); // Add update data rid.add(updateCourseRecordID, updateCourseData); // Update data if the data was set up successfully if (0 < rid.getCount()) { rid.updateByRecordID(); System.out.println("Finished updating course information"); } rid.close(); // Update student information rid = con.prepareUpdateRecordID(); // Add update data rid.add(updateStudentRecordID, updateStudentData); // Update data if the data was set up successfully if (0 < rid.getCount()) { rid.updateByRecordID();

E-3


System.out.println("Finished updating student information"); } rid.close(); // Commit the updates made so far con.commit(); con.close(); } // Processing performed when an error occurs during execution of the application catch (ShunException ex) { //Roll the transaction back if (con != null) { try { con.rollback(); } catch (ShunException e) { int errorLevel = ex.getErrLevel(); switch( errorLevel ) { case ShunException.SHUN_ERROR : System.out.println("Error level: SHUN_ERROR"); break; case ShunException.SHUN_ERROR_TRANSACTION_ROLLEDBACK : System.out.println("Error level: SHUN_ERROR_TRANSACTION_ROLLEDBACK"); break; case ShunException.SHUN_ERROR_CONNECTION_TERMINATED : System.out.println("Error level: SHUN_ERROR_CONNECTION_TERMINATED"); break; } System.out.println("Error message: " + ex.getMessage()); ex.printStackTrace(); } } int errorLevel = ex.getErrLevel(); switch( errorLevel ) { case ShunException.SHUN_ERROR : System.out.println("Error level: SHUN_ERROR"); break; case ShunException.SHUN_ERROR_TRANSACTION_ROLLEDBACK : System.out.println("Error level: SHUN_ERROR_TRANSACTION_ROLLEDBACK"); break; case ShunException.SHUN_ERROR_CONNECTION_TERMINATED : System.out.println("Error level: SHUN_ERROR_CONNECTION_TERMINATED"); break; } System.out.println("Error message: " + ex.getMessage()); ex.printStackTrace(); } catch (Exception ex) {

E-4

Sample Program Using the Java APIs

System.out.println("Error message: " + ex.getMessage()); ex.printStackTrace(); } finally { try { if (rs != null) rs.close(); } catch (ShunException ex) { int errorLevel = ex.getErrLevel(); switch( errorLevel ) { case ShunException.SHUN_ERROR : System.out.println("Error level: SHUN_ERROR"); break; case ShunException.SHUN_ERROR_TRANSACTION_ROLLEDBACK : System.out.println("Error level: SHUN_ERROR_TRANSACTION_ROLLEDBACK"); break; case ShunException.SHUN_ERROR_CONNECTION_TERMINATED : System.out.println("Error level: SHUN_ERROR_CONNECTION_TERMINATED"); break; } System.out.println("Error message: " + ex.getMessage()); ex.printStackTrace(); } try { if (pstmt != null) pstmt.close(); } catch (ShunException ex) { int errorLevel = ex.getErrLevel(); switch( errorLevel ) { case ShunException.SHUN_ERROR : System.out.println("Error level: SHUN_ERROR"); break; case ShunException.SHUN_ERROR_TRANSACTION_ROLLEDBACK : System.out.println("Error level: SHUN_ERROR_TRANSACTION_ROLLEDBACK"); break; case ShunException.SHUN_ERROR_CONNECTION_TERMINATED : System.out.println("Error level: SHUN_ERROR_CONNECTION_TERMINATED"); break; } System.out.println("Error message: " + ex.getMessage()); ex.printStackTrace(); } try { if (rid != null) rid.close(); } catch (ShunException ex) { int errorLevel = ex.getErrLevel(); switch( errorLevel ) {

E-5


case ShunException.SHUN_ERROR : System.out.println("Error level: SHUN_ERROR"); break; case ShunException.SHUN_ERROR_TRANSACTION_ROLLEDBACK : System.out.println("Error level: SHUN_ERROR_TRANSACTION_ROLLEDBACK"); break; case ShunException.SHUN_ERROR_CONNECTION_TERMINATED : System.out.println("Error level: SHUN_ERROR_CONNECTION_TERMINATED"); break; } System.out.println("Error message: " + ex.getMessage()); ex.printStackTrace(); } try { if (con != null) con.close(); } catch (ShunException ex) { int errorLevel = ex.getErrLevel(); switch( errorLevel ) { case ShunException.SHUN_ERROR : System.out.println("Error level: SHUN_ERROR"); break; case ShunException.SHUN_ERROR_TRANSACTION_ROLLEDBACK : System.out.println("Error level: SHUN_ERROR_TRANSACTION_ROLLEDBACK"); break; case ShunException.SHUN_ERROR_CONNECTION_TERMINATED : System.out.println("Error level: SHUN_ERROR_CONNECTION_TERMINATED"); break; } System.out.println("Error message: " + ex.getMessage()); ex.printStackTrace(); } } } }

Execution Results

Finished updating course information Finished updating student information

E-6

Sample Program Using the C APIs

Sample Program Using the C APIs The following is a sample program using the C APIs. #include <stdio.h> #include <string.h> #include "libshun.h" /* Transaction processing sample */ int main() { /* Handle variables */ SHUNHCON connectionHandle; SHUNHSTMT statementHandle_search; SHUNHSTMT statementHandle_update; /* Working variables */ int status; /* Input parameters */ char *hostName; int portNo; char *ShunFileName; int startNo; int returnRequestCount; char *queryForm; char *returnForm; char *sortForm; /* Output Parameters */ int hitCount; int returnCount; int returnableCount; SHUNRECID *recID; SHUNDATA *dataInfo; SHUNPOS *firstPosition, *lastPosition; /* Update data */ SHUNRECID courseRecID[1]; SHUNDATA courseData[1]; SHUNRECID studentRecID[1]; SHUNDATA studentData[1]; /* Error variables */ SHUNHANDLE errorHandle; int errorLevel; char *errorMessage; /* Initialize variables */ connectionHandle = NULL; statementHandle_search = NULL; statementHandle_update = NULL; /* Allocate connection handle */ status = ShunAllocHandle( NULL, &connectionHandle ); if ( status != SHUN_SUCCESS ) {

E-7


errorHandle = (SHUNHANDLE)connectionHandle; goto ErrorEnd; } /* Establish a connection by specifying the host name, port number and Shunsaku File name */ hostName = "DirSvr1"; portNo = 33101; ShunFileName = "shunsakuFile1"; status = ShunConnect( connectionHandle, hostName, portNo, ShunFileName ); if ( status != SHUN_SUCCESS ) { errorHandle = (SHUNHANDLE)connectionHandle; goto ErrorEnd; } /* Allocate data manipulation handle */ status = ShunAllocHandle( connectionHandle, &statementHandle_search ); if ( status != SHUN_SUCCESS ) { errorHandle = (SHUNHANDLE)connectionHandle; goto ErrorEnd; } /* Specify the number of search records and the search expression, etc. */ startNo = 1; returnRequestCount = 10; queryForm = "/course/name == 'Business negotiation'"; returnForm = "/"; sortForm = NULL; /* Call API, search data, and search for course information */ status = ShunSearch( statementHandle_search, startNo, NULL, 0, returnRequestCount, queryForm, returnForm, sortForm, &hitCount, &returnCount, &returnableCount, &recID, &dataInfo, &firstPosition, &lastPosition ); if ( status != SHUN_SUCCESS ) { errorHandle = (SHUNHANDLE)statementHandle_search; goto ErrorEnd; } /* Save update data */ courseRecID[0] = recID[0]; /* Specify the number of search records and the search expression, etc. */ startNo = 1; returnRequestCount = 10; queryForm = "/student/e-mail == 'mary\\.tompson@def\\.com'"; returnForm = "/";

E-8


sortForm = NULL; /* Call API, search data, and search for student information */ status = ShunSearch( statementHandle_search, startNo, NULL, 0, returnRequestCount, queryForm, returnForm, sortForm, &hitCount, &returnCount, &returnableCount, &recID, &dataInfo, &firstPosition, &lastPosition ); if ( status != SHUN_SUCCESS ) { errorHandle = (SHUNHANDLE)statementHandle_search; goto ErrorEnd; } /* Save update data */ studentRecID[0] = recID[0]; /* Turn off auto-commit mode */ status = ShunSetConnectAttr( connectionHandle, SHUN_ATTR_AUTOCOMMIT, SHUN_FALSE ); if ( status != SHUN_SUCCESS ) { errorHandle = (SHUNHANDLE)connectionHandle; goto ErrorEnd; } /* Allocate data manipulation handle */ status = ShunAllocHandle( connectionHandle, &statementHandle_update ); if ( status != SHUN_SUCCESS ) { errorHandle = (SHUNHANDLE)connectionHandle; goto ErrorEnd; } /* Create course information data */ courseData[0].Data = "<course>" " <name>Business negotiation</name>" " <instructor>" " <first-name>Max</first-name>" " <last-name>Cameron</last-name>" " </instructor>" " <capacity>40</capacity>" " <current-auditors>31</current-auditors>" "</course>"; courseData[0].Data_Len = strlen( courseData[0].Data ); /* Update information (update data by adding the student) */ status = ShunUpdateRecId( statementHandle_update, 1, courseRecID, courseData ); if ( status != SHUN_SUCCESS ) {

E-9


errorHandle = (SHUNHANDLE)statementHandle_update; goto ErrorEnd; } printf( "Finished updating course information\n" ); /* Create student data */ studentData[0].Data = "<student>" " <first-name>Mary</first-name>" " <last-name>Tompson</last-name>" " <e-mail>[email protected]</e-mail>" " <course>Chinese language</course>" " <course>Business negotiation</course>" "</student>"; studentData[0].Data_Len = strlen( studentData[0].Data ); /* Add student */ status = ShunUpdateRecId( statementHandle_update, 1, studentRecID, studentData ); if ( status != SHUN_SUCCESS ) { errorHandle = (SHUNHANDLE)statementHandle_update; goto ErrorEnd; } printf( "Finished adding student information\n" ); /* Commit the updates made so far */ status = ShunCommit( connectionHandle ); if ( status != SHUN_SUCCESS ) { errorHandle = (SHUNHANDLE)connectionHandle; goto ErrorEnd; } /* Release the data manipulation handle */ status = ShunFreeHandle( statementHandle_search ); if ( status != SHUN_SUCCESS ) { errorHandle = (SHUNHANDLE)statementHandle_search; goto ErrorEnd; } /* Release the data manipulation handle */ status = ShunFreeHandle( statementHandle_update ); if ( status != SHUN_SUCCESS ) { errorHandle = (SHUNHANDLE)statementHandle_update; goto ErrorEnd; } /* Close connection */ status = ShunDisconnect( connectionHandle ); if ( status != SHUN_SUCCESS ) { errorHandle = (SHUNHANDLE)connectionHandle; goto ErrorEnd; } /* Release the connection handle */ status = ShunFreeHandle( connectionHandle ); if ( status != SHUN_SUCCESS ) { errorHandle = (SHUNHANDLE)connectionHandle;

E-10


goto ErrorEnd; } return 0; ErrorEnd: /* Error processing if an error occurs */ /* Get error information */ status = ShunGetErrorMessage(errorHandle, &errorLevel, &errorMessage); if ( status == SHUN_SUCCESS ) { switch ( errorLevel ) { case SHUN_ERROR_CONNECTION: printf("Error level = SHUN_ERROR_CONNECTION\n"); break; case SHUN_ERROR_TRANSACTION: printf("Error level = SHUN_ERROR_TRANSACTION\n"); break; case SHUN_ERROR_DATA: printf("Error level = SHUN_ERROR_DATA\n"); break; } printf("Error message = %s\n", errorMessage); } else { printf("Error obtaining error message: %d\n", status); } if ( connectionHandle != NULL ) { status = ShunRollback( connectionHandle ); if ( status != SHUN_SUCCESS ) { printf( "Failed to roll back transaction\n" ); } } /* Release the data manipulation handle */ if ( statementHandle_update != NULL ) { status = ShunFreeHandle( statementHandle_update ); if ( status != SHUN_SUCCESS ) { printf("Error releasing data manipulation handle : %d\n", status); } } /* Release the data manipulation handle */ if ( statementHandle_search != NULL ) { status = ShunFreeHandle( statementHandle_search ); if ( status != SHUN_SUCCESS ) { printf("Error releasing data manipulation handle: %d\n", status); } } /* Release the connection handle */ if ( connectionHandle != NULL) { int connection_dead; status = ShunGetConnectAttr( connectionHandle, SHUN_ATTR_CONNECTION_DEAD, &connection_dead ); if ( status != SHUN_SUCCESS ) { printf("Error obtaining connection handle attribute: %d\n", status); }

E-11


if ( connection_dead == SHUN_FALSE ) { status = ShunDisconnect( connectionHandle ); if ( status != SHUN_SUCCESS ) { printf("Connection close error: %d\n", status); } } status = ShunFreeHandle( connectionHandle ); if ( status != SHUN_SUCCESS ) { printf("Connection handle release error: %d\n", status); } } return 1; }

Execution Results

Finished updating course information Finished adding student information

E-12

Appendix F Java APIs Used in V6.0L20 and Earlier

This appendix details the JAVA APIs used in V6.0L20 and earlier.

• ShunSearch Class

• ShunBatch Class

• Character Encoding

• Error Codes Output

The following table lists the Java APIs for data searches and updates that were provided in V6.0L20 and earlier. These functions are compatible with V6.0L20 and earlier.

Table F-1 JAVA APIs for Data Searches and Updates provided in V6.0L20

Package Class Inheritance Constructor or method

Function

create Creates an instance of this API and sets configuration information for the search server.

executeSearch1 Sends a search request to Shunsaku, and receives only the number of hits as the results.

executeSearch2 Sends a search request and a return expression to Shunsaku, and receives the number of hits, conductor control information, the record identifier, and the data for the specified item as the results.

com.fujitsu.shun.api

ShunSearch Class

None

executeSearch3 Sends the conductor control information and record identifier for the search target to Shunsaku, and receives all of the corresponding record information as the results.

F-1

Appendix F: Java APIs Used in V6.0L20 and Earlier

Package Class Inheritance Constructor or method

Function

getHitCount Returns the number of hit records as the search results.

getRetNum Returns the number of returned records as the search results.

getRecNo Returns conductor control information and record identifiers as the search results.

getRecData Returns the content of the response data as the search results. The format used to return results will vary, depending on the search mode (executeSearch2 or executeSearch3) and the return expression.

ShunBatch Creates an instance of ShunBatch.

executeAdd Adds the XML data stored in the specified text file to a Shunsaku record.

ShunBatch Class

None

executeDelete Sends a search condition for finding records to be deleted to Shunsaku and deletes the records that match the condition.

F-2

ShunSearch Class

ShunSearch Class The ShunSearch class is used to perform data searches.

create

Function This method creates an instance and stores parameters in instance variables used for socket connection.

Syntax ShunSearch create ( String host_Name, /* (1) Host name */ int port_No, /* (2) Port number */ String Reserve2, /* (3) Reserved (null) */ String strCode, /* (4) Character encoding of data */ String getCode /* (5) Character encoding of returned result data*/ );

Return Value The instance is returned if the function completes normally.

Parameters (1) Host name

Specifies the host name of the server where the conductor (or director) is running.

If the conductor function is to be used, specify the host name of the server where the conductor will be running.

If the conductor is not to be used, specify the host name of the server where the director will be running.

(2) Port number

Specifies the port number of the conductor (or director).

If the conductor function is to be used, specify the MySearchPort parameter in the conductor environment file.

If the conductor is not to be used, specify the MySearchPort parameter in the director environment file.

(3) Reserved

Specify null.

F-3


(4) Character encoding of data

Specifies the character encoding of the data:

Setting Meaning

UTF-8 UTF-8

GB2312 GB2312 (for Chinese language)

big5 big5 (for Chinese language)

KSC5601 KSC5601 (for the Hangul alphabet (Korean))

The XML data specified when the ShunSearch class method runs is converted to the character encoding specified in this parameter, and then passed to Shunsaku.

Refer to Character Encoding for details on character code conversion.

(5) Character encoding of returned result data

Specifies the character encoding of the returned search result data:

Setting Meaning

UTF-8 UTF-8




Search result data is converted to the character encoding specified by this parameter and returned to the application. Refer to Character Encoding for details on character code conversion.

Exceptions ShunException

This is output when the parameter value is invalid.

Refer to Error Codes Output for details on error codes.

F-4

ShunSearch Class

executeSearch1

Function This method performs a search according to the specified parameter and obtains the number of hits.

Syntax void executeSearch1( String Query /* (1) Search expression */ );

Return Value None.

Parameters (1) Search expression

Specify the search expression using Unicode.

Ensure that the size of the search expression is between 1 and 65,535 bytes inclusive.

Refer to the Application Development Guide for details on search expressions.


This is output when an error occurs during the search process.


F-5


executeSearch2

Function This method performs a search according to the specified parameters and obtains the search result.

Syntax void executeSearch2( String Query, /* (1) Search expression */ int Reply_No, /* (2) Reply start number */ int Request_Cnt, /* (3) Number of reply requests */ String Return, /* (4) Return expression */ String Reserve1 /* (5) Reserved (null) */ );

Return Value None.





(2) Reply start number

Specifies the reply start number.

Specifies from which item of found data to commence retrieval. Specify a number greater than zero.

(3) Number of reply requests

Specifies the number of reply requests.

Specify the maximum number (at least one) of items to be obtained as a search result.

If the specified value exceeds the maximum number of responses to client requests, the number of items to be returned will be restricted to this maximum limit.

To find the maximum number of responses to client requests, refer to the AnsMax parameter in the conductor environment file (if a conductor is being used), or the AnsMax parameter in the director environment file (if a conductor is not being used).

(4) Return expression

Specify the return expression using Unicode.

Ensure that the size of the return expression is between 0 and 65,535 bytes inclusive.

Refer to the Application Development Guide for details on return expressions.

(5) Reserved

Specify null.

F-6

ShunSearch Class




F-7


executeSearch3

Function After executeSearch2 is executed, if the number of reply records is one or greater, this method obtains data corresponding to the conductor control information and record identifier specified in the parameters.

Syntax void executeSearch3( String cond_CntlInfo[], /* (1) Conductor control information */ String rec_ID[] /* (2) Record identifier */ );

Return Value None.

Parameters (1) Conductor control information

Specify the conductor control information items obtained by the getRecNo method, in the order in which the corresponding record information is to be received.

Use Unicode as the characters to be set.

(2) Record identifier

Specify the record identifiers obtained by the getRecNo method, in the order in which the corresponding record information is to be received.

Use Unicode as the characters to be set.




F-8

ShunSearch Class

getHitCount

Function This method returns the number of records found.

Syntax int getHitCount ();

Return Value The number of records found is returned.

Parameters None.


This is output when the preceding search method did not return the number of found records.


F-9


getRetNum

Function This method returns the number of reply records.

Syntax int getRetNum ();

Return Value The number of reply records is returned.

Parameters None.


This is output when the preceding search method did not return the number of reply records.


F-10

ShunSearch Class

getRecNo

Function This method returns the conductor control information and the record identifier.

Syntax String[] getRecNo ( int getNo /* (1) Reply data sequence number */ );

Return Value If the number of reply records is zero or greater after the executeSearch2 or the executeSearch3 method is executed, the value is returned as an array containing two elements. These are the conductor control information and the record identifier obtained from the conductor (or director).

Parameters (1) Reply data sequence number

Specifies the reply data sequence number of the conductor control information and record identifier to be obtained.


This is output when one of the following errors occurs:

• When the preceding search method did not return the conductor control information or the record identifier

• When a value greater than the number of reply records was specified as the reply data sequence number

• When a value of –1 or less was specified as the reply data sequence number


F-11


Reply Layout This method creates a two-element array each time it is invoked and returns it to the application.

The first element contains conductor control information and the second element contains the record identifier.

Execution Obtained array Content of array

Obtained array [0] Conductor control information for first record getRecNo(0)

Obtained array [1] Record identifier for first record

Obtained array [0] Conductor control information for second record getRecNo(1)

Obtained array [1] Record identifier for second record

Obtained array [0] Conductor control information for third record getRecNo(2)

Obtained array [1] Record identifier for third record

F-12

ShunSearch Class

getRecData

Function This method returns the content of the response data using the character code specified in the ‘character encoding of returned result data’ parameter of the create() method.

Syntax String[] getRecData ();

Return Value The content of the response data from the director is returned as an array having as many elements as the number of replies.

Parameters None.


This is output when one of the following errors occurs:

• When the preceding search method did not return response data content

• When there is no response data content (i.e., when the number of record replies is zero)


Reply Layout This method creates an array having as many elements as the number of replies and returns it to the application.

The content of the response data is stored in the first element.

Execution Obtained array Content of array

Obtained array [0] Content of the first response data

Obtained array [1] Content of the second response data

getRecData()

Obtained array [2] Content of the third response data

... … …

F-13


ShunBatch Class The ShunBatch class is used to perform data updates.

ShunBatch

Function This method creates an instance of ShunBatch.

Syntax public ShunBatch( String hostName, /* (1) Host name */ int port, /* (2) Port number */ String reserve2, /* (3) Reserved (null) */ String strCode, /* (4) Character encoding of data */ String getCode, /* (5) Character encoding of returned result data */ String directorName /* (6) Director identifier */ ) throws ShunException;

Parameters (1) Host Name

Specifies the host name of the server where the director that will perform the data update is running.

If a null character or space character (" ") is specified as the host name, the host name will default to "localhost".

(2) Port Number

Specifies the number of the port for receiving search requests on the director that will perform the data update.

Specify the MySearchPort parameter in the director environment file of the director that will perform the data update.

(3) Reserved

Specify null.

F-14

ShunBatch Class

(4) Character encoding of data

Specifies the character encoding of the data.

Setting Meaning

UTF-8 UTF-8




Specify the character encoding used on the director server.

(5) Character encoding of returned results data

Specifies the character encoding of the returned search results data.

Setting Meaning

UTF-8 UTF-8




Specify the character encoding used on the director server.

(6) Director identifier

Specifies the director identifier of the director that will perform the data update.


ShunException is output when the system properties are not set or the director identifier is specified as a null character or a space character (" ").


F-15


executeAdd

Function This method adds XML data stored in the specified text file to a Shunsaku record.

Syntax public void executeAdd( String xmlDatafileName /* (1) Path to the text file */ ) throws ShunException;

Return Value None

Parameters (1) Path to the text file

Specifies the full pathname of the text file containing XML data that is to be added.

The character encoding of the XML data to be added must be the same as that of the data stored in Shunsaku. Note, however, that the character encoding specified in the XML root tag is disabled.

Note

To add multiple items of XML data, it is recommended that the data be joined together into a single text file and specified as an argument.


ShunException is output when an error occurs while XML data is being added.


Notes • Because this method is invoked synchronously, control does not return to the invoking source

until all data has been added.

• The data added by the executeAdd method cannot be searched until the method completes normally.

• Do not delete the text file specified as an argument until this method completes.

F-16

ShunBatch Class

executeDelete

Function This method sends search conditions for finding records to be deleted to Shunsaku, and deletes records that match the conditions.

Syntax public void executeDelete( String query, /* (1) Search expression */ int replyNo, /* (2) Deletion start number */ int requestCnt, /* (3) Number of delete requests */ String reserve1, /* (4) Reserved (null) */ String recordIDfileName /* (5) Pathname of work file */ ) throws ShunException;

Return Value None.





(2) Deletion start number

Specifies the deletion start number.

Specify the item of found data from which to commence deletion. Specify a number greater than zero.

(3) Number of delete requests

Specifies the number of delete requests.

Specify either the maximum number of items to be deleted (a number greater than zero), or "-1".

Note that if the maximum number of items to be deleted is greater than the maximum number of responses to a client request (specified in AnsMax in the director environment file), or if "-1" is specified as the maximum number of items to be deleted, all records that match the search conditions will be deleted.

(4) Reserved

Specify null.

(5) Pathname of work file

Specifies the full pathname of the work file that is used by the internal processing of the API used to delete data.

At least 24 bytes of work area space is required for each record.

Note

The user must create in advance the directory that will contain the work file.

F-17



ShunException is output when an error occurs while XML data is being deleted.


Note Since this method is invoked synchronously, control does not return to the invoking source until all specified data has been deleted.

F-18

Character Encoding

Character Encoding Use Unicode as the encoding of the characters passed as Java API parameters.

The following diagram illustrates the overall process of converting character codes.

Figure F-1 Process for Converting Character Codes

F-19


Error Codes Output This section explains the error codes that are output when Java API functions for data searches and updates are used.

• Error codes can be obtained by using the getErrCode() method of the ShunException class.

• The getMessage() method of the ShunException class can be used to obtain error messages in the format shown in the following table (errCode = error code).

Table F-2 Error Codes

Error code Meaning User response

-1 An error occurred when an attempt was made to connect to the host.

Check the host name and port number of the target director server and rerun the command.

-10 The service is not ready to receive a request.

Wait a few moments and then rerun the command.

-11 The maximum number of simultaneous search requests has been exceeded.


-12 A timeout occurred while the system was waiting for a response from the director.


-20 A parameter error occurred.

Check all parameters and then rerun the command.

-21 A search expression syntax error occurred.

Check the search expression and then rerun the command.

-22 A return expression syntax error occurred.

Check the return expression and then rerun the command.

-23 A search expression size error occurred.

Check the length of the input parameter search expression and then rerun the command.

-24 A return expression size error occurred.

Check the length of the input parameter return expression and then rerun the command.

-30 The response data storage area was insufficient.

Increase the response data storage area and then rerun the command.

-100 An API internal error occurred.

Change the search condition so that the number of reply records is not zero, then call the data acquisition method.


Call a data acquisition method that conforms to the search method.

F-20

Error Codes Output



Change the search condition so that the number of reply records is not zero, then call the record identifier acquisition method.


Call a record identifier acquisition method that conforms to the search method.


Call the search method and then call the hit count acquisition method.


Create an instance normally (by executing the create method), then call the search method.


Call a reply record count acquisition method that conforms to the search method.


Change the return expression to a value other than NULL, then call the data acquisition method.

-201 The pathname of the Shunsaku command directory has not been specified.

Specify the pathname of the Shunsaku command directory in the system properties, and rerun the command.

-202 The director identifier has not been specified.

Set the director identifier, and rerun the command.

-203 The pathname of the XML data to be added has not been specified.

Set the pathname of the XML data to be added, and rerun the command.

-204 Addition of XML data with the shundimport command failed.

- Confirm that the content of the XML data to be added with the shundimport command is correct, and then rerun the command.

- Check the state of the director server, and then rerun the command.

-206 The work file has not been specified.

Specify the pathname of the work file, and rerun the command.

-207 Deletion of XML data with the shundimport command failed.

- Specify the full pathname for the recordIDfileName argument (the path to work files).

- Check the state of the director server and then rerun the command.

-208 The shundimport command failed to start.

- Confirm that the pathname of the shundimport command specified in the system properties is correct, and then rerun the command.

- Execute the command with privileged user authority.

-209 The shundimport command was suspended.

Follow the procedure described in ‘Handling Errors Produced During Command Execution’, in Chapter 8, and then rerun the command.

F-21



-210 The work file could not be opened.

Confirm that the pathname of the work file is correct, and then rerun the command.

-211 The work file could not be written to.

Check the status of the disk, and then rerun the command.

Note

• Java exceptions also have to be caught by an application. The following table shows the exception that may occur.

Exception Applicable method Meaning Action

OutOfMemoryError - executeSearch2

- executeSearch3

- executeDelete

Memory error

- Rerun the command after reducing the number of reply requests.

- Rerun the command after reducing the number of return items.

F-22

Appendix G C APIs Used in V6.0L30 and Earlier

This appendix details the C APIs used in V6.0L30 and earlier.

• C APIs

• Error Codes Output

The following table shows the APIs that Shunsaku provides for use by applications created with the C programming language.

Table G-1 C APIs

Function name Description

shunadd Adds the specified XML document to Shunsaku.

shundeletebyrecid Deletes an XML document by sending its conductor control information and record identifier to Shunsaku.

shunsearch1 Sends a search request to Shunsaku, and receives only the number of hits as the result.

shunsearch2 Sends a search request and a return expression to Shunsaku, and receives the number of hits, conductor control information, the record identifier and the XML document for the specified item as the results.

shunsearch3 Sends the conductor control information and record identifier for the search target to Shunsaku, and receives the applicable XML document as the result.

shunsort Sends a search request, a return expression and a sort expression to Shunsaku, and receives the number of hits, conductor control information, the record identifier and an XML document containing the sorted data for the specified item as the results.

Also sends a search request, and a return expression and a sort expression specifying an aggregation function to Shunsaku, and receives aggregated search results.

All of these functions run correctly in multithreaded environments.

G-1

Appendix G: C APIs Used in V6.0L30 and Earlier

C APIs

shunadd

Function Adds the XML document specified in the input parameter to Shunsaku.

Syntax #include “libshun.h” int shunadd( char *host_Name, /* in (1) Host name */ int Port_No, /* in (2) Port number */ int DataArray_Cnt, /* in (3) Number of arrays for */ /* additional data */ char **DataArray /* in (4) Area for arrays for */ /* additional data */ );

Return Value The error code reported by this function is returned.

Value ‘0’ is returned if the function completes normally.

The validity of the returned result cannot be guaranteed if a value other than ‘0’ is returned.


Parameters

(1) Host name

Specify the host name or IP address of the server where the conductor is running.

The host name can be up to 24 bytes long.

(2) Port number

Specify the MySearchPort parameter in the conductor environment file.

(3) Number of arrays for additional data

Specify the number of data arrays for storing the additional XML documents.

Specify a value of 1 or greater.

G-2

C APIs

(4) Area for arrays for additional data

Pointers to the areas where the additional XML documents are stored are set in each array. Specify pointers to these arrays.

Figure G-1 Area for Arrays for Additional Data

Notes

• Store at least one XML document in each area for additional data. Refer to the Application Development Guide for more information on the format of the XML documents that are stored in these data areas.

• A conductor is required to use this function.

G-3


shundeletebyrecid

Function Deletes the XML document corresponding to the record identifier in the input parameter.

Syntax #include “libshun.h” int shundeletebyrecid( char *host_Name, /* in (1) Host name */ int Port_No, /* in (2) Port number */ int Request_Cnt, /* in (3) Number of deletion requests */ char **cond_ctl, /* in (4) Conductor control information */ /* area */ char **rec_ID /* in (5) Deleted record identifier area*/ );





Parameters

(1) Host name



(2) Port number


(3) Number of deletion requests

Specify the number of data items to delete.

(4) Conductor control information area

Use the following procedure to set up the conductor control information area.

1. Allocate a conductor control information area equal to the length specified in COND_CTL_LEN in libshun.h. Set the conductor control information obtained with shunsearch2.

2. Allocate a pointer array with as many elements as the number of deletion requests.

3. Assign the conductor control information pointers allocated in Step 1 above to each element of the pointer array.

G-4

C APIs

4. Assign the pointer array pointers allocated in Step 2 above to the conductor control information area.

Figure G-2 Conductor Control Information Area

(5) Deleted record identifier area

Use the following procedure to set the deleted record identifier area.

1. Allocate a deleted record identifier area equal to the length specified in ROW_ID_LEN in libshun.h. Set the reply record identifiers obtained with shunsearch2.

2. Allocate a pointer array with as many elements as the number of deletion requests.

3. Assign the deleted record identifier pointers allocated in Step 1 above to each element of the pointer array.

4. Assign the pointer array pointers allocated in Step 2 above to the deleted record identifier area.

Figure G-3 Deleted Record Identifier Area

Note

A conductor is required to use this function.

G-5


shunsearch1

Function Performs searches based on input parameters and reports the number of hits only to the source invoking this function.

Syntax #include “libshun.h” int shunsearch1( char *host_Name, /* in (1) Host name */ int Port_No, /* in (2) Port number */ char *Query, /* in (3) Search expression area */ char *Reserve, /* in (4) Reserved (NULL) */ int *Hit_Cnt /* out (5) Number of hits */ );





Parameters (1) Host name

Specify the host name or IP address of the director server.

When the conductor function is to be used, specify the host name or IP address of the server where the conductor is running.

When the conductor function is not to be used, specify the host name or IP address of the server where the director is running.


(2) Port number

Specifies the number of the port that receives requests from the director server or the number of the port that receives search requests.

When the conductor function is to be used, specify the MySearchPort parameter in the conductor environment file.

When the conductor function is not to be used, specify the MySearchPort parameter in the director environment file.

G-6

C APIs

(3) Search expression area

Specifies a pointer to the search expression area. For the search expression, specify a string that has been converted to the character encoding specified in CharacterCode in the director environment file. The size of the search expression area must be between 1 and 65, 535 bytes inclusive.


(4) Reserved

Specify NULL.

(5) Number of hits

Returns the number of items that match the search condition.

Zero out the area storing the number of hits before specifying this parameter.

G-7


shunsearch2

Function Performs searches based on input parameters and reports search results to the source invoking this function.

Syntax #include “libshun.h” int shunsearch2( char *host_Name, /* in (1) Host name */ int Port_No, /* in (2) Port number */ int Reply_No, /* in (3) Reply start number */ int Request_Cnt, /* in (4) Number of items to return */ /* per request */ char *Query, /* in (5) Search expression area */ char *Return, /* in (6) Return expression area */ char *Reserve1, /* in (7) Reserved (NULL) */ char *Reserve2, /* in (8) Reserved (NULL) */ int Secure_Size, /* in (9) Size of returned data */ /* storage area */ int *Hit_Cnt, /* out (10) Number of hits */ int *Return_Cnt, /* out (11) Number of response replies*/ int *Stored_Size, /* out (12) Size of reply data */ /* storage area */ Sdsma *Dsma, /* out (13) Reply data storage */ /* management array */ char *Data /* out (14) Reply data storage area */ /* (Len=Secure_Size) */ );





Parameters

(1) Host name





G-8

C APIs

(2) Port number





Specify from which item of found data to commence reporting. Specify a number greater than one.

(4) Number of items to return per request

Specify the maximum number of data items to be reported as the output result of this function.

Specify a number greater than one.

If a number greater than the maximum number of responses to client requests is specified, it will be superceded by this maximum limit. To find the maximum number of responses to client requests, refer to the AnsMax parameter in either the conductor environment file (if a conductor is being used) or the director environment file (if a conductor is not being used).


Specify a pointer to the search expression area. For the search expression, specify a string that has been converted to the character encoding specified in CharacterCode in the director environment file. The size of the search expression area must be between 1 and 65, 535 bytes inclusive.


(6) Return expression area

Specify a pointer to the return expression area. The return expression area must be allocated by the source invoking this function. The size of the return expression area must be between 0 and 65, 535 bytes inclusive.


(7) Reserved

Specify NULL.

(8) Reserved

Specify NULL.

(9) Size of returned data storage area

Specifies the size of the area used to store reply data.

(10) Number of hits



(11) Number of response replies

Returns the number of response replies corresponding to the search results. As many reply data items as the number of response reply items will be returned to the reply data storage area.

Zero out the area storing the number of response replies before specifying this parameter.

G-9


(12) Size of reply data storage area

Returns the size of the area used to return the reply data corresponding to search results.

Zero out the area storing the size of the reply data storage area before specifying this parameter.

If the size of the returned data storage area is insufficient, the size of the area needed to store the reply data will be returned.

(13) Reply data storage management array

Sets a pointer to a reply data storage management array where the search results are returned. The reply data storage management array must be allocated by the source invoking this function.

Refer to

Reply Data Storage Management Structure below for details on the reply data storage management array.

(14) Reply data storage area

Sets a pointer to a reply data storage area where the result of executing this function is returned.

Zero out the reply data storage area before specifying this parameter.

A reply data storage area equal to the size of the reply data storage area specified in (9) above must be allocated by the source invoking this function.

To obtain information about each item of reply data stored in the reply data storage area, check the pointer to and size of each item of reply data reported to the reply data storage management array.

Reply Data Storage Management Structure This is a reply data storage management array structure where the results of executing this function are returned. This area must be allocated by the source invoking this function beforehand, and the size of the area must be as large as the array for the number of items to return per request. typedef struct /* Data storage management area */ { char *Rec_Ctl; /* (1) Pointer to conductor control */ /* information area */ char *Rec_ID; /* (2) Pointer to reply record identifier */ /* area */ char *Rec_Ptr; /* (3) Pointer to reply data */ int Rtn_Len; /* (4) Size of reply data */ } Sdsma;

(1) Pointer to conductor control information area

Set up a pointer to the conductor control information area for the reply data in advance. Allocate a notification area in advance, and zero out this area before specifying this parameter.

Allocate a notification area equal to the length specified in COND_CTL_LEN in libshun.h.

Note

The conductor control information is in binary format. Do not use string functions such as the str function.

(2) Pointer to reply record identifier area

Set up a pointer to the record identifier area for the reply data in advance. Allocate a notification area in advance, and zero out this area before specifying this parameter.

Allocate a notification area equal to the length specified in ROW_ID_LEN in libshun.h.

G-10

C APIs

Note

The reply record identifier is in binary format. Do not use string functions such as the str function.

(3) Pointer to reply data

The pointer to the reply data in the reply data storage area is returned after this function is called.

Zero out the area storing the pointer to the reply data before specifying this parameter.

(4) Size of reply data

The size of the reply data is returned after this function is called.

Zero out the area storing the size of the reply data before specifying this parameter.

The following figure shows the parameters of the shunsearch2 function.

Figure G-4 Shunsearch2 Parameters

G-11


Notes

• If the size of the reply data storage area exceeds the value specified in the ‘size of returned data storage area’ parameter, the following information will be set:

− The size of the area needed to store data will be returned as the size of the reply data storage area.

− The error code will be –30.

− The data that matched the search condition (the number of hits) will be returned as the number of search hits.

− All record identifiers that matched the search condition will be returned as reply record identifiers.

• If NULL has been set in the return expression, it will be assumed that no reply data is required, and only record identifiers will be returned.

• If the null character string ‘\0’ or ‘/’ is specified in the return expression, all records that matched the search condition will be returned.

G-12

C APIs

shunsearch3

Function Reports the data corresponding to the record identifier specified in the input parameter to the source invoking this function.

Syntax #include “libshun.h” int shunsearch3( char *host_Name, /* in (1) Host name */ int Port_No, /* in (2) Port number */ int Request_Cnt, /* in (3) Number of items to return */ /* per request */ char **cond_ctl, /* in (4) Conductor control */ /* information area */ char **rec_ID, /* in (5) Reply record identifier */ /* area */ char *Reserve, /* in (6) Reserved (NULL) */ int Secure_Size, /* in (7) Size of returned data */ /* storage area */ int *Hit_Cnt, /* out (8) Number of hits */ int *Return_Cnt, /* out (9) Number of response replies */ int *Stored_Size, /* out (10) Size of reply data storage */ /* area */ Sdsma *Dsma, /* out (11) Reply data storage */ /* management array */ char *Data /* out (12) Reply data storage area */ /* (Len=Secure_Size) */ );





Parameters

(1) Host name





G-13


(2) Port number





Specify the maximum number of data items to be returned as the output result of this function.

Specify a value greater than one.

(4) Conductor control information area

Use the following procedure to set up the conductor control information area.

1. Allocate a conductor control information area equal to the length specified in COND_CTL_LEN in libshun.h. Set the conductor control information obtained with shunsearch2.

2. Allocate a pointer array with as many elements as the number of search requests.

3. Assign the conductor control information pointers allocated in Step 1 above to each element of the pointer array.

4. Assign the pointer array pointers allocated in Step 2 above to the conductor control information area.

Figure G-5 Conductor Control Information Area

(5) Reply record identifier area

Use the following procedure to set up the reply record identifier area.

1. Allocate a reply record identifier area equal to the length specified in ROW_ID_LEN in libshun.h. Set the reply record identifiers obtained with shunsearch2.

2. Allocate a pointer array with as many elements as the number of search requests.

3. Assign the reply record identifier pointers allocated in Step 1 above to each element of the pointer array.

4. Assign the pointer array pointers allocated in Step 2 above to the reply record identifier area.

G-14

C APIs

Figure G-6 Reply Record Identifier Area

(6) Reserved

Specify NULL.


Specify the size of the area used to store reply data.

(8) Number of hits











Sets a pointer to a reply data storage area where the search results are returned.

The reply data storage management array must be allocated by the source invoking this function.

Refer to Reply Data Storage Management Structure below for details on the reply data storage management array.




A reply data storage area equal to the size specified in the ‘size of returned data storage area’ parameter in (7) above must be allocated by the source invoking this function.


G-15


Reply Data Storage Management Structure This is a reply data storage management array structure where the results of executing this function are returned. This area must be allocated by the source invoking this function beforehand, and the size of the area must be as large as the array for the number of items to return per request. typedef struct /* Data storage management area */ { char *Rec_Ctl; /* (1) Pointer to conductor control information */ /* area */ char *Rec_ID; /* (2) Pointer to reply record identifier area */ char *Rec_Ptr; /* (3) Pointer to reply data */ int Rtn_Len; /* (4) Size of reply data */ } Sdsma;




Note





Note







Zero out the area for the size of the reply data before specifying this parameter.

G-16

C APIs

The following figure shows the parameters of the shunsearch3 function.

Figure G-7 Shunsearch3 Parameters

Notes





− All record identifiers that matched the search condition will be returned as reply record identifiers.

• The number of items to return per request is returned as the execution result of the ‘number of response replies’ parameter. If the number of hits is less than the number of response replies, it indicates that non-existing record identifiers have been specified. In such cases, NULL will be returned to the pointer to the reply data in the reply data storage management array corresponding to each of these non-existing record identifiers, and the size of the reply data will be returned as zero.

G-17


shunsort

Function This method performs searches based on input parameters and sorts or aggregates the search results before reporting them to the source invoking this function.

Syntax #include “libshun.h” int shunsort( char *host_Name, /* in (1) Host name */ int Port_No, /* in (2) Port number */ int Reply_No, /* in (3) Reply start number */ int Request_Cnt, /* in (4) Number of items to */ /* return per request */ char *Query, /* in (5) Search expression area */ char *Return, /* in (6) Return expression area */ char *Sort, /* in (7) Sort expression area */ int Secure_Size, /* in (8) Size of returned data */ /* storage area */ int *Hit_Cnt, /* out (9) Number of hits */ int *Return_Cnt, /* out (10) Number of response */ /* replies */ int *Available_Cnt, /* out (11) Number of possible */ /* replies */ int *Stored_Size, /* out (12) Size of reply data */ /* storage area */ Sdsma *Dsma, /* out (13) Reply data storage */ /* management array */ char *Data /* out (14) Reply data storage area*/ /* (Len=Secure_Size) */ );





Parameters

(1) Host name



(2) Port number



Specify from which item of found data to commence reporting. Specify a number greater than one.

G-18

C APIs


Specify the maximum number of data items to be reported as the output result of this function.

If a number greater than the maximum number of responses to client requests (specified in AnsMax in the conductor environment file) is specified, it will be superceded by this maximum limit.


Specify a pointer to the search expression area. For the search expression, specify a string that has been converted to the character encoding specified in CharacterCode in the conductor environment file. The size of the search expression area must be between 1 and 65, 535 bytes inclusive.


(6) Return expression area

Specify a pointer to the return expression area. The return expression area must be allocated by the source invoking this function. The size of the return expression must be between 1 and 65,535 bytes inclusive.

If the return expression includes an aggregation function, results will be aggregated before they are returned.


(7) Sort expression area

Specify a pointer to the sort expression area. The area for the sort expression must be allocated by the source invoking this function. The size of the sort expression area must be between 1 and 65, 535 bytes inclusive.

Refer to the Application Development Guide for details on sort expressions.


Specify the size of the area used to store reply data.

(9) Number of hits






(11) Number of possible replies

If a sort request is made, the number of replies that can be returned will be returned as the sort result. Up to 1,000 items can be returned. As the total length of the sort keys increases, the number of replies that can be returned decreases. Refer to the Application Development Guide for the relationship between the total length of the sort keys and the total number of replies that can be returned.

If an aggregation request is made, the number of groups that can be returned will be returned as the aggregation result. Up to 1,000 groups can be returned. As the total length of the group keys increases, the number of groups that can be returned decreases. Refer to the Application Development Guide for the relationship between the total length of the group keys and the total number of groups that can be returned.

Zero out the area storing the number of replies that can be returned before specifying this parameter.

G-19







Sets a pointer to a reply data storage management array where the search results are returned. The reply data storage management array must be allocated by the source invoking this function.

Refer to Reply Data Storage Management Structure below for details on the reply data storage management array.




A reply data storage area equal to the size of the returned data storage area specified in (8) above must be allocated by the source invoking this function.


Reply Data Storage Management Structure This is a reply data storage management array structure where the results of executing this function are returned. This area must be allocated by the source invoking this function beforehand. typedef struct /* Data storage management area */ { char *Rec_Ctl; /* (1) Pointer to conductor control */ /* information area */ char *Rec_ID; /* (2) Pointer to reply record identifier */ /* area */ char *Rec_Ptr; /* (3) Pointer to reply data */ int Rtn_Len; /* (4) Size of reply data */ } Sdsma;




Note


Do not specify the record identifiers and conductor control information returned by aggregation requests when executing the shunsearch3 function. Doing so will return no records.




G-20

C APIs

Note


Do not specify the record identifiers and conductor control information returned by aggregation requests when executing the shunsearch3 function. Doing so will return no records.






Zero out the area for the size of the reply data before specifying this parameter.

The following figure shows the parameters of the shunsort function.

Figure G-8 Shunsort Parameters

G-21


Notes





− With sort requests, all record identifiers that matched the search condition will be returned to the conductor control information area and the reply record identifier area.

− With aggregation requests, the conductor control information area and the reply record identifier area remain in the same state as when the function was invoked.

• If NULL has been set in the return expression for the sort request, it will be assumed that no reply data is required and only conductor control information and record identifiers will be returned.

• If the null character string ‘\0’ or ‘/’ is specified in the return expression for the sort request, all records that matched the search condition will be returned.

• Aggregation requests require both a return expression area and a sort expression area.

• A conductor is required to use this function.

G-22

Error Codes Output

Error Codes Output The following table lists the error codes that may be returned when using the C APIs provided by Shunsaku.

Table G-2 Error Codes


-1 Cannot connect to host. Check the host name or IP address and the port number of the connection destination. If they are all correct, wait a few moments and then rerun the command.

-10 The service is not ready to receive a request.


When updating, sorting or aggregating data, make sure that the connection destination is a conductor.

If an error occurs when the command is rerun, check the cause of the error using the event log, eliminate the cause, and then rerun the command.

If an error occurs when the command is rerun, check the cause of the error using the system log (syslog), eliminate the cause, and then rerun the command.

-11 The maximum number of simultaneous requests has been exceeded.


-12 A timeout occurred while the system was waiting for a response.


-13 The maximum number of items that can be updated, sorted or aggregated has been exceeded.


-20 An error was found in the specification of a passed parameter.

Set the correct parameter and rerun the command.

-21 A search expression syntax error occurred.

Check the input parameter search expression and rerun the command.

-22 A return expression syntax error occurred.

Check the input parameter return expression and rerun the command.

G-23



-23 A search expression size error occurred.

Check the length of the input parameter search expression and then rerun the command. (The size of the search expression must be from 1 to 65, 535 bytes.)

-24 A return expression size error occurred.

Check the length of the input parameter return expression and then rerun the command. (The size of the return expression must be from 0 to 65, 535 bytes.)

-25 A sort expression syntax error occurred.

Check the sort expression input parameter and rerun the command.

-26 A sort expression size error occurred.

Check the length of the input parameter sort expression and rerun the command. (The size of the sort expression must be from 1 to 65, 535 bytes.)

-27 There is an inconsistency between the content specified in the sort expression and the return expression.

Check the input parameter sort expression and return expression and then rerun the command.

-30 The reply data storage area was insufficient. (*1)

Increase the size of the returned data storage area and perform the search again.

-201 There is a problem with the Shunsaku environment.

Reinstall Shunsaku.

-204 The addition processing has failed. Check the content of the XML document and rerun the command. Check the status of the director server and rerun the command.

-207 The deletion processing has failed. Check the content of the conductor control information and the record identifier(s) and rerun the command.

Check the status of the director server and rerun the command.

-208 The process that executes either the addition or deletion processing failed to start.


-209 The addition process or the deletion process was interrupted.


-210 Failed to open a work file. Check the value of the WorkFolder parameter in the director environment file and rerun the command.

-211 Failed to write to a work file. Check the status of the disk specified in the WorkFolder parameter in the director environment file and rerun the command.

*1 If error -30 occurs, reply data is not saved.

G-24