Administrator Guide for Domino Server maintenance

 White paper

 

AbstractThe purpose of this document is to describe when to use the Lotus Domino utilities (Updall, Compact, Fixup) and to explain how to use them for normal and abnormal maintenance.   

ContentTable of ContentsI. The utilities: Updall, Compact, Fixup

A. UpdallB. CompactC. Fixup

II. Best utilities for weekly maintenanceIII. Best utilities for recovery

*******************************************************I. The utilities: Updall, Compact, Fixup

A. UpdallUpdall is similar to Update but it does not run continually or work from a queue. Instead, you run Updall as needed. You can specify options when you run Updall but without them, Updall updates any view indexes or full-text search indexes on the server that need updating. To save disk space, Updall also purges deletion stubs from databases and discards view indexes for views that are unused for 45 days, unless there are different criteria specified for discarding view indexes. Use the notes.ini file setting, Default_Index_Lifetime_Days, to change when Updall discards unused view indexes.

Like Update, Updall rebuilds all corrupt view indexes and full-text-search indexes that it encounters.

By default, Updall is included in the notes.ini setting, ServerTasksAt2, so it runs daily at 2 AM. Running Updall daily helps save disk space by purging deletion stubs and discarding unused view indexes. It also ensures that all full-text-search indexes set for daily updates are updated.

Updall: Basic options

Option in Task - Start tool

Command line option

Description

Index all databases

Index only this database or folder

databasepath "Only this database" updates only the specified database. To update a database in the Domino Data folder, enter the file name, xxxx.nsf. To update databases in a folder within the Data folder, specify the database path relative to the data folder, for example, doc\readme.nsf. "Index all databases" (or no database path) updates all databases on the server.

Update this view only database -T Updates a specific view in a

viewtitle database. Use with -R to solve corruption problems.

Updall - Update options

Option in Task - Start tool

Command line option

Description

Update: All built views -V Updates built views and does not update full-text indexes.

Update: Full text indexes

-F Updates full-text indexes and does not update views.

Update: Full text indexes; Only those with frequency set to Immediate

-H Updates full-text indexes assigned "Immediate" as an update frequency.

Update: Full text indexes; Only those with frequency set to Immediate or Hourly

-M Updates full-text indexes assigned "Immediate" or "Hourly" as an update frequency.

Update: Full text indexes; Only those with frequency set to Immediate, Hourly or Daily

-L Updates full-text indexes assigned "Immediate," "Hourly," or "Daily" as an update frequency.

Updall - Rebuild options

Option in Task - Start tool

Command line option

Description

Rebuild: Full text indexes only

-X Rebuilds full-text indexes and does not rebuild views. Use to rebuild full-text indexes that are corrupted.

Rebuild: All used views -R Rebuilds all used views. Using this option is resource-intensive, so use it as a last resort to solve corruption problems with a specific database.

Rebuild: Full-text indexes and additionally: All unused views

database -C Rebuilds unused views and a full-text index in a database. Requires you to specify a database.

Updall - Search Site options Option in Task - Start

toolCommand line

optionDescription

Update database configurations: Incremental

-A Rebuilds full-text indexes and does not rebuild views. Use to rebuild full-text indexes that are corrupted.

Update database configurations: Incremental

-B Does a full update of search-site database configurations for search site databases.

B. CompactWhen documents and attachments are deleted from a database, Domino tries to reuse the unused space instead of immediately reducing the file size. Sometimes Domino cannot reuse the space because of fragmentation until you compact the database.

There are three styles of compacting:- In-place compacting with space recovery- In-place compacting with space recovery and reduction in file size- Copy-style compacting

In-place compacting with space recovery onlyThis style of compacting recovers unused space in a database but does not reduce the size of the database on disk. Databases retain the same database instance IDs (DBIIDs), so the relationship between the compacted databases and the transaction log remains intact. Users and servers can continue to access and edit databases during compacting. This style of compacting is useful for databases that you expect to stay the same size or to grow in size.

When you run Compact without specifying options, Domino uses this style of compacting on all databases enabled for transaction logging. Domino also uses this style of compacting when you use the -b option (case sensitive) when compacting any database.

Tip: Use this compacting method most frequently since it is the fastest and causes the least system impact.

In-place compacting with space recovery and reduction in file sizeThis style of compacting reduces the file size of databases as well as recovers unused space in databases. This style of compacting is somewhat slower than in-place compacting with space recovery only and assigns new DBIIDs to databases. If you use it on logged databases with a certified backup utility, perform full backups of the databases shortly after compacting is complete. This style of compacting allows users and servers to continue to access and edit databases during compacting.

When you run Compact without specifying options, Domino uses this style of compacting on databases that are not enabled for transaction logging. Domino also uses this style of compacting when you use the -B option. To optimize disk space, it is recommended that you run Compact, using the -B option on all databases once a week or once a month.

Copy-style compactingCopy-style compacting creates copies of databases and then deletes the original databases after compacting completes, so extra disk space is required to make the database copies. This style of compacting essentially creates a new database with a new database ID. If you use copy-style compacting on logged databases (using the -c option), compacting assigns new DBIIDs, so when using a certified backup utility, perform full backups of databases shortly after compacting completes. When you use copy-style compacting, users and servers cannot edit databases during compacting and they can only read databases if the -L option is used.

Domino uses copy-style compacting by default when you use an option with Compact to enable a database property that requires a structural change to a database, or when you run Compact on a database that has a structural change pending that was initiated from the Database Properties box. Enabling or disabling the database properties, "Document table bitmap optimization" and "Don't support specialized response hierarchy," require structural database changes.

When to compact databasesIt is recommended that you compact databases weekly or monthly, using the -B option to recover disk space. If you use a certified backup utility, remember to run it after compacting is complete.

Compact - Basics

Option Command line equivalent

Description

Compact only this database or folder. To specify databases to compact using the Files tab, select the databases in the files pane.

database pathSpecify any additional options after the database path.

To compact a database in the Domino Data folder, enter the file name, such as sales.nsf. To compact databases in a folder within the Data folder, specify the database path relative to the data folder, i.e., to compact all databases in the folder, Data\Sales, specify Sales. If you choose "Compact all databases" or don't specify a database path), Compact compacts all databases in the Data folder and folders within the Data folder.

Compact - Options

Option Command line equivalent

Description

Compact database only if unused space is greater than x percent

-S percent Compacts all databases with a specified percent of unused space. For example, if you specify 10, databases with 10% or more recorded unused space are compacted. Note that the unused space calculation is not always a reliable measure of unused space.

Discard any built view indexes

-D Discards built view indexes. Use this option to compact databases just before you store them on tape. Does copy-style compacting.

Keep or revert database to previous format

-R Compacts databases without converting to the current release file format of the server that stores the databases or reverts databases to the previous release file format. For example, on Domino 6 servers, this option compacts Domino 5 databases without converting them to the Domino 6 file format and converts Domino 6 databases to the Domino 5 file format. This option uses copy-style compacting.

Compact - Style

Option Command line equivalent

Description

In-place (recommended) -b Uses in-place compacting and recovers unused space without reducing file size unless there's a pending structural change to a database, in which case copy-style compacting occurs. Recommended method of compacting.

In-place with file size -B Uses in-place compacting,

reduction recovers unused space and reduces file size unless there's a pending structural change in which case copy-style compacting occurs. If you use transaction logging, do full database backups after compacting completes.

Copy-style -c Uses copy-style compacting. Use this option to solve database corruption problems.

Copy-style: Allow access while compacting

-L Enables users to continue to access databases during compacting. If a user edits a database during compacting, compacting is canceled. Useful only when copy-style compacting is done.

Copy-style: Ignore errors and proceed

-i Enables compacting to continue even if it encounters errors such as document corruption. Only used for copy-style compacting.

Compact - AdvancedNote that advanced Compact options are not available through the Compact tool on the Files tab of Domino Administrator.

Option* Commandline equivalent

Description

Document table bitmap optimization: Off

-f Disables "Document table bitmap optimization" database property. Does copy-style compacting.

Document table bitmap optimization: On

-F Enables "Document table bitmap optimization" database property. Does copy-style compacting.

Don't support specialized response hierarchy: Off

-h Disables "Don't support specialized response hierarchy" database property; in other words, support specialized response hierarchy. Does copy-style compacting.

Don't support specialized response hierarchy: On

-H Enables "Don't support specialized response hierarchy" database property; in other words, do not support specialized response hierarchy. Does copy-style compacting.

Enable transaction logging: Off

-t Disables transaction logging.

Enable transaction logging: On

-T Enables transaction logging.

Don't maintain unread marks: Off

-u Disables "Don't maintain unread marks" database property; in other words, maintain unread marks.

Don't maintain unread marks: On

-U Enables "Don't maintain unread marks" database property; in other words, do not maintain unread marks.

NOTE: Select "Set advanced properties" before you enable or disable any of the above properties.

C. FixupWhen you restart a server, the server quickly searches for any unlogged databases that were modified but improperly closed because of a server failure, power failure, hardware failure, and so on. A few minutes after server startup is complete, the Fixup task then runs on these databases to attempt to fix any inconsistencies that resulted from partially written operations caused by a failure. When users attempt to access one of these databases and Fixup has not yet run on the database, the users see the message "This database cannot be opened because a consistency check of it is in progress." A similar Fixup process occurs when you restart a Lotus Notes client.

Multiple Fixup tasks run simultaneously at server startup to reduce the time required to fix databases. The number of Fixup tasks that Domino runs by default at startup is equal to twice the number of processors available on the server. Although this default behavior should be adequate in most circumstances, you can edit the NOTES.INI file to include the Fixup_Tasks setting. The actual number of tasks run is the smaller of the configured number of tasks that can run and the number of databases that require fixing. For example, if you set Fixup_Tasks to 4, but only one database requires fixing, then only one Fixup task runs.

Keep in mind that, after you set up transaction logging, Fixup is not needed or used to bring databases back to a consistent state.

Fixup- Options

Fixup options in Fixup tool and

Task - Start tool

Command line equivalent

Description

Fixup all databasesFixup only this database or folder

databasepath "Fixup only this database or folder" runs Fixup only on a specifid database or all databases in a specified folder. To run Fixup on a database in the Domino data folder, enter the file name, for example SALES.NSF. To run Fixup on a database or databases in folders within the data folder, enter the path relative to the data folder. For example, to run Fixup on all databases in the DATA\SALES folder, specify SALES. "Fixup all databases" or no command line database path runs Fixup on all databases on the server. Note To specify databases or folders to run on using the Fixup tool, select the database(s) or folder(s).

Report all processed databases to log file

-L Reports to the log file every database that Fixup opens and checks for corruption. Without this argument, Fixup logs only actual problems encountered.

Scan only since last fixup

-I When you run Fixup on a specific database, Fixup checks only documents modified since Fixup last ran. Without this option, Fixup checks all documents.

Scan all documents -F When you run Fixup on all databases, Fixup checks all

documents in the databases. Without this option, Fixup checks only documents modified since it last ran. Note To specify this option using the Fixup tool, deselect "Scan only since last fixup."

Perform quick fixup -Q Checks documents more quickly but less thoroughly. Without this option, Fixup checks documents thoroughly.

Exclude views (faster)

-V Prevents Fixup from running on views. This option reduces the time it takes Fixup to run. Use if view corruption isn't a problem.

Don't purge corrupted documents

-N Prevents Fixup from purging corrupted documents so that the next time Fixup runs or the next time a user opens the database, Fixup must check the database again. Use this option to salvage data in documents if the corruption is minor or if there are no replicas of the database.

Optimize user unread lists

-U Reverts ID tables in a database to the previous release format. Don't select this option unless Customer Support recommends doing so.

Fixup transaction-logged databases

-J Runs on databases that are enabled for transaction logging. Without this option, Fixup generally doesn't run on logged databases. If you are using a certified backup utility, it's important that you schedule a full backup of the database as soon after Fixup finishes as possible.

Fixup open databases

-O If you run Fixup on open databases, Fixup takes the databases offline to perform the fixup. This is the default if you run Fixup and specify a database name. Without this option, when you do not specify database names, Fixup does not run on open databases.

Don't fixup open databases

-Z Applies only to running Fixup on a single database. When a database isn't taken offline and is in use, then Fixup is not run. This is the default when Fixup is run on multiple databases.

Verify only -C Verifies the integrity of the database and reports errors. Does not modify the database (for example, does not purge corrupted documents).

Fixup subdirectories -Y Runs Fixup on databases in subfolders (subdirectories).

Don't fixup subdirectories

-y Does not run Fixup on databases in subfolders (subdirectories).

II. Best utilities for weekly maintenanceIf you are not running transactional logs, run the following for weekly maintenance, preferably on the weekend when your server is not under a great work load:

Compact –B –s 10

If you are running transactional logs, run the following for weekly maintenance:

Compact –b –s 10

The above command compacts any database with more than 10% of white space. The "-b" and "-B" means an in-place compaction. The "–b" is used with transaction logs so that no new dbIIDs are assigned to the databases.

There is no need to run Fixup or Updall as part of a weekly maintenance schedule. Fixup should be run only if corruption is suspected of a database and Updall is run every night by default. Run Updall with switches if view corruption is suspected.

NOTE: It is good practice to periodically rebuild views using Updall -r and it is very important to rebuild full text indexes periodically using Updall -x.

III. Best utilities for recoveryWhen databases should be repaired, used the steps below for recovery. This section also contains instructions for running Domino maintenance utilities from the command line on the various operating system platforms.

When should you use unscheduled maintenance?Unscheduled maintenance is not scheduled and includes database repair and database upgrade. The exact maintenance depends on the task and database type. If corruption occurs in the Domino Directory, the Domino server must be in a stopped state to run maintenance.

Database corruptionDatabase corruption is caused by something happening outside the ordinary that makes the database unusable such as user complaints or error messages are reported. For example, when the server generates messages indicating database corruption (with or without a crash), the following output may occur:

-- "database.nsf is damaged....-- DbMarkCorrupt(DbFixup: invalid slot found, could not be repaired)...-- database.nsf is CORRUPT - Now Read-Only!"

If the database uses transactional loggingWhen you use transactional logging to log changes to Domino databases, a server automatically uses the transaction log to restore and recover any transaction-logged databases after a system failure. Databases that are not set to transaction logging are checked via a normal consistency check. If a disk failure occurs and archived-style transaction logging is used, you can use the transaction log along with a certified backup utility to restore and recover databases.

If these steps fail, use the following:1. fixup database.nsf -J -F 2. compact database.nsf -b3. updall database.nsf -R -X

These utilities will change the dbIID associated with transaction logging so that if you run archived-style transaction logging, immediately perform a full database backup.

If no transactional logging is in place for the databaseWhen the database is not transaction logged, run the utilities as follows:

1. fixup database.nsf -F2. compact database.nsf -c -i3. updall database.nsf -R -X

Another optionIf the steps above are unsuccessful, try creating a new replica of the database and replacing the original with the new replica. Creating a new replica forces a complete rebuild of the database and repairs corruption or removes items that are irreparable.

Managing corruption during the business day

1. Run Fixup as follows (without quotes) to check the integrity of the corrupt database and report problems without making any changes: "fixup database.nsf -L –F -O -C"

2. It is not recommended to use the Fixup utility to perform repairs during business hours. If attempts to repair the database absolutely cannot wait until non-business hours, then use the following options with the compact task:

compact database.nsf -c -L -i

NOTE: The best recommendation is to enable transaction logging which provides database integrity. Fixup is not needed when transaction logging is enabled.

3. After running either of these, run the following to rebuild the views and indexes (if possible, wait until off-hours).

updall database.nsf –R -XFor further information, refer to Document #1193738, "Managing corruption recovery during the business day."

Domino Server upgradeWhen upgrading from one Domino version to the next, the way the database is physically laid out on disk (disk structure) is also upgraded. It is important before upgrading to ensure that the database structure is valid and consistent. This prevents errors during the database upgrade process. Utilities in a new version of Domino can provide assistance if database errors are encountered after the upgrade. Upgrading the full text search index for each database is optional but easy enough to do and recommended, so that you are using the latest full text search engine in the new version of Domino.

Before upgrading

1. Before upgrading, it is recommended that you complete database clean-up on the existing server by running Updall on the views, as follows:

updall –R –X

There are several options that can be run before upgrading. Running Updall -R -X is the most favorable option for avoiding corruption because it completely rebuilds the view indexes and full text index. Other choices are:

Updall -v (updates the view indexes)Updall -R (completely rebuilds the view indexes)Updall -X (completely rebuilds the full text index)

2. You should also run "fixup –F" to run consistency checks.NOTE: If you are running transaction logging, you don't have to run Fixup but if you want to, be sure to add the -J switch.

3. Run Compact as follows: "compact –c –i"

NOTE: If you are running transaction logging, remove the –c and –i switches and be sure to add the -b switch. This is different from upper case -B, as -B will change the dbIID of any transactionally logged databases.

If mail files are corrupt after server upgrade

1. Run "fixup –F".

2. Run "compact -c -D -i –K" (copy-style, delete view indexes, ignore errors, set large UNK table bit).

3. Run "updall –R –X".

For further information, refer to Document #1201020, "Error: 'Database Is Corrupt; Cannot Allocate Space' when Opening Database After Server Upgrade."

After upgradingAfter an upgrade, it is recommended to upgrade to the latest search engine. Even though Domino is backward compatible to all previous full text engines, it is a good idea to upgrade to the latest, using "updall –X". This will rebuild all existing full text indexes using the new engine and enable its new features.

When should you NOT use unscheduled maintenance?Domino may automatically detect and repair database problems or there may be no database problems. So even though your first instinct is to perform maintenance on an affected database immediately, consider the following before you act:

The first time a crash occursAlthough a crash can leave databases in an inconsistent state, Domino attempts to correct this inconsistency upon restart by performing a consistency check (if transaction logging is not enabled). During this check, Domino attempts to repair any damaged databases/documents. If no failure is reported, then Fixup is generally not necessary and is not recommended as routine maintenance. If a server experiences repeated crashes and the NSD indicates issues with a database, running Fixup may be needed. In this case, running Fixup with Domino in a stopped state is recommended.

If a crash occurs that does not indicate it was due to a database issueIf the crash stack does not indicate anything related to database access, then database corruption is unlikely (except for possible inconsistent states mentioned above), so running maintenance is probably not warranted.

How do you run the Domino maintenance utilities from the command line?The methods and syntax for running maintenance from the command line vary by platform. The following example shows how to run Fixup on the names.nsf for Windows, iSeries, and Unix/Linux/zSeries. Use these examples as a guide for also running Compact and Updall from the command line.

NOTE: If corruption occurs in the Domino Directory, the Domino server must be in a stopped state to run Fixup.

For Windows, use "nfixup names.nsf –F". Note that if you are running transaction logging, be sure to add the -J switch.

For the iSeries, use the following:RUNDOMCMD SERVER(serverName) CMD(CALL PGM(QNOTES/FIXUP)

PARM(‘names.nsf’ ‘-F')) BATCH(*NO)

If you are running transaction logging, be sure to add the -J switch after -F. Also, it MUST run from QSECOFR or the equivalent.

For Unix/Linux/zSeries, use "<path>/fixup names.nsf -F". Note that if you are running transaction logging, be sure to add the -J switch. Also, you MUST be logged in as a Notes user with the Notes Data directory as the current directory.  

Related informationManaging corruption recovery during the business dayDomino R5 Mail Files Show Corruption After Server ...Error: 'Database Is Corrupt; Cannot Allocate Space' ...How to view UPDALL and FIXUP and COMPACT Options from..How to run Compact against multiple databases simultaneRunning Fixup -C on a Database with Attachments Results