Summary

This article explains how to back up the Orbix services data and log files and perform a full recovery in case of a database corruption.

Article Number

34775

Environment

Orbix 3.6.x

All supported platforms

Question/Problem Description

How to perform Orbix database backup and recovery for Orbix Services.

Clarifying Information

Each time you start an Orbix service that uses Berkeley DB, the service performs a normal recovery. If the service was stopped in the middle of an update, the transaction is rolled back, and the service persistent data is restored to a consistent state.

In some cases, however, the data files or the log files are missing or corrupted and normal recovery is not sufficient. Then you must perform a catastrophic recovery. This article explains how to back up your data and log files and perform a full recovery.

It is important that you archive a stable snapshot of your services database, which can be used in case a recovery is needed. This is referred to as a "full backup" and can be performed by making a backup of the entire "dbs" directory.

The purpose of this backup is that if a PERSIST_STORE error occurs for any Orbix services, you can replace the corrupted directory with the backup. The services should then start without a problem. The backup can be made at any time. The only requirement is that the service be in a stable state (can run and function without errors). You can take the backup directly after configuring your domain, or after the system has been running for a while. The backup that you make will determine the snapshot that your system will return to in the case of a recovery. For example, if you have numerous entries into the IMR (registered POAs, ORBs, and so on), you may wish to add these entries before backing up the locator database. This prevents you from having to do the extra re-configuration if you ever need to recover.

Error Message

E - DB_ENV->open() failed for DB env it_locator_priv; errno is -30975 (DB_RUNRECOVERY: Fatal error, run database recovery)

Cause

Corruption of the Orbix service's underlying Berkeley database.

Resolution

To do a full backup, perform the following steps:

• • Firstly disable the default periodic deletion and/or archival of old log files during the period while you are backing up the database. To disable this, please run the following command:

itadmin pss_db pre_backup env.ior

• • The "env.ior" above represents a handle to the database. Each service should have its corresponding env.ior file within the "[ORBIX HOME]/var/[DOMAIN NAME]/dbs/[SERVICE NAME]" directory.

Note: If you can bring the services down before doing the backup, you can skip this first step. If you have a live system, and are unable to bring down the services, you can do a backup while the services are running.

• • Make a backup of the following directories:

[ORBIX HOME]/var/[DOMAIN NAME]/dbs/[SERVICE NAME]/data/
[ORBIX HOME]/var/[DOMAIN NAME]/dbs/[SERVICE NAME]/logs/

• • Store these backups in a safe location. After a successful full backup, you can discard older full backups (if any).
  • Re-enable the default periodic deletion and/or archival of old log files:

itadmin pss_db post_backup env.ior

To do a full backup recovery, perform the following steps:

• Determine which service is failing on start-up.
• Ensure that all Orbix services are stopped.
• Make a temporary backup of the "[ORBIX HOME]/var/[DOMAIN NAME]/dbs/[SERVICE NAME]/" directory for the service you wish to recover.
• Delete the "[ORBIX HOME]/var/[DOMAIN NAME]/dbs/[SERVICE NAME]/" directory for the service you wish to recover.
• Replace the deleted "[ORBIX HOME]/var/[DOMAIN NAME]/dbs/[SERVICE NAME]/" directory in the environment with the latest full backup of this directory.
• Restart the services.

Support Incidents

• 2808524