Previous Page

Chapter 7 - EBR and System Maintenance

Using EBR from the Server Console Menus

Important: If you have configured UNIX access on your server and feel comfortable working with the UNIX file system, you may follow the alternate Disaster Recovery procedure outlined in "Using EBR from a UNIX Shell."

EBR Service Layout

The service' s home directory, /disk1/banyan/ebr, which includes all of the service executables.
The service' s data directory, which holds the configuration data, logs, and servers running both the EBR agent and services software, indexes. This defaults to /disk1/EBR, but this default may be changed when the service is created. The directory will always be created as /diskn/EBR, where /diskn is the one specified in the Optional Parameters list when the service is created.

Using EBR to Recover from Server Disk Failures

The server's boot disk, which contains the operating system and the EBR executables is damaged or destroyed.

In this case, you will probably need to replace and reformat the disk, or both, then reinstall VINES, the EBR agent, and possibly the EBR service.
A secondary disk (/disk2), which contains other services, but not the EBR service data directory, is damaged or destroyed.

You may have to replace and reformat the disk, or both. Once the disk volume is usable again, which includes initialization using VINES, you can usually use EBR to recover the service data from previous backups.
The disk that contains the EBR service's data directory, which contains the on-line indexes, is damaged or destroyed.

In this case, you may have to replace and/or reformat the disk, and then you will have to recover the EBR indexes before using EBR to recover any services on the disk. If the disk is also the boot disk, the problem is further complicated by the need to reinstall VINES and EBR first.

Have a VINES "System Configuration" backup, created using the original VINES Backup and Restore utility. This backup only has to be done once and it will only be used in the event of a failure of the boot disk.
Re-install EBR using the same data directory location for the on-line indexes as in the original installation.

Important: Once you understand the procedures for disaster recovery, make sure you have devised a disaster recovery plan for your site. If possible, you should test the ability to recover from a disaster at your site.

Preparing for a Disaster

1. After EBR has been installed on the server, create a VINES System Configuration backup tape, using the standard VINES backup/restore utility. This only needs to be done once, immediately after EBR is installed and configured.

2. Select Backup/Restore from the Operator Menu.

3. Select Backup from the Backup/Restore Menu.

4. Select System Configuration Backup from the backup menu, and when prompted, select Confirm Readability of the backup.

5. After the backup completes, store it in a safe, but convenient location. It will be needed if the boot disk is damaged or destroyed.

May 09 20:59:03 1995 EBR bootstrap information Page 1

date time       level ssid file record
5/09/95 20:29:08 full 8380 39   0
5/09/95 20:52:25 full 8382 41   0

Recovering Services on the Boot Disk

Important: It is impossible to provide step-by-step instructions on how to recover your system from a disaster, because every site is unique. The examples in this chapter are designed to give you general principles on how to recover files on /disk1 or /disk2 and to help you understand the procedures. They are meant to be examples only, not specific instructions.

1. Re-install VINES from the distribution media.

As part of the VINES installation procedure, you will be asked "Do you want to reload a complete system backup (y/n):" Type "y" and press [Enter].

2. Select Continue with System Rebuild from the System Rebuild menu.

3. When prompted, insert the VINES System Configuration backup media and choose Continue to start the restore operation.

4. When the restore is complete, you are prompted to "Load Next Incremental in Series" or "Continue with Installation." Choose "Continue with Installation."

VINES initializes the Mail service, if there is one, and loads the device drivers. Shortly thereafter, the Service Monitor initializes. All services, including EBR, show a status of inoperative, until they have been re-installed.

5. Re-install the EBR agent and, if necessary, the EBR service from the distribution media.

You will not have to create the EBR service because the service was created when the VINES System configuration backup was restored. However, you must re-configure EBR through the EBR Administrator utility if you made modifications to the original configuration - for example, changed the tape device from 8mm to 4mm.

6. After installing EBR, you need to reboot the server to start the portmapper before you can start the EBR service or agent.

7. Re-install any additional "Third-Party" services by following the appropriate procedures.

Note: If the disk you are recovering also contains the EBR data directory, recover the EBR service's on-line indexes before you can recover the service data. Before proceeding further, follow the procedure in "Recovering an EBR Service's On-line Indexes" found later in this chapter.

To Restore VINES Services

1. Select Backup/Restore from the Operator Menu.

2. Select Enterprise Backup/Restore (EBR) from the Backup/Restore menu.

3. When prompted, enter the VINES administrator' s ID and password.

4. Verify that the server you are doing the recover on is looking for its files on the correct EBR server.

5. Select Change Recover Source from the menu and set the parameter that says "Use the EBR service on server:___" to the name of the EBR server that backs up this particular server.

6. Select Recover Server Data, from the Enterprise Backup/Restore menu.

7. When prompted, select Continue to continue the current operation.

A list of services will be displayed followed by the EBR recover prompt ("\\>" ).

This list can be displayed by typing the "dir" command. The entire list of commands can be displayed by typing "?"

8. Using the mark command, select all services you want recovered. In this case, it would be all services on the Boot/Primary disk:

\\> mark servicename

You may use this command as many times as necessary. You could also select all services:

\\> mark *

and then un-select those services you do not want restored, with the unmark command:

\\> unmark servicename

Important: If the servicename contains spaces, you must place quotes (" " ) around the name or EBR will not accept it.

9. To display which tape volumes will be needed for the restore operation, enter:

> volumes

10. To force an overwrite of existing files, enter:

> force

Note: This is necessary because the data restored from the VINES System configuration backup could be out of date.

11. Insert the first volume listed in the output of the volumes command into the tape drive.

12. Log in to a client workstation configured for EBR administration, using an ID which has EBR administrator rights, then click on the EBR Administrator Icon and mount the volume you just inserted into the tape drive.

13. Go back to the server console and enter:

> recover

to start the recovery procedure.

14. Messages similar to these will be displayed:

Recovering 13794 files into their original locations.
Total estimated disk space needed for recover is 246MB
Not all Volumes are on-line (some off-line)
SNI1.003 at /dev/rct4
SNI1.004

15. At the prompt "Proceed with Recover command at this time? [Y]" , press [Enter] for yes. To know when to mount the next volume needed for the restore, monitor the Messages and Pending windows using the EBR Administrator.

Recovering Services on a Secondary Disk

Important: It is impossible to provide step-by-step instructions on how to recover your system from a disaster, since every site is unique. The examples in this chapter are designed to give you general principles on how to recover files on primary or secondary disk and to help you understand the procedures. They are meant to be examples only, not specific instructions.

Recovering an EBR Service's On-Line Indexes

To Recover an EBR Service's On-Line Indexes

1. Select Backup/Restore from the Operator Menu.

2. Select Enterprise Backup/Restore (EBR) from the Backup/Restore menu.

3. When prompted, enter the VINES administrator' s ID and password.

4. Select Local Tape Operations from the Enterprise Backup/Restore menu.

5. Select Rebuild Media and On-Line Indexes from the Local Tape Operations menu.

6. At the prompt, select yes to continue. This operation could take anywhere from 45 minutes to three or four hours, depending on size.

7. When index recovery has completed, press [Enter] to continue.

8. Press the Esc key to return to the Local Tape Operations menu.

9. Follow steps 6 through 9 for each EBR media volume used after the last full backup.

10. After recovering all necessary indexes, press the Esc key until you are returned to the Enterprise Backup/Restore menu.

11. You have completed restoring the on-line Indexes and you are now ready to recover the service data itself.

Using EBR from a UNIX Shell

EBR Service Layout

A service home directory that includes all of the service executables.
A service data directory that holds configuration data, logs, and (for EBR services) indexes.

Using the UNIX Recover Browser

Using EBR to Recover from Server Disk Failures

A primary disk, /disk1, which contains the operating system and EBR binaries, is damaged or destroyed. In this case, you will probably need to replace and/or reformat the disk, then reinstall VINES and the EBR agent, and possibly the EBR service.
A secondary disk which contains other file systems is damaged or destroyed. You may have to replace and reformat the disk, or both, but once the disk volume itself is usable again, you can usually just recover the disk's contents from previous backups.
The disk that contains an EBR service's on-line indexes is damaged or destroyed. You must recover the indexes before using EBR to recover any file systems. If the disk is also /disk1, the problem is further complicated by the need to reinstall VINES and EBR executables first.

Name the replacement server with the same hostname as in the original installation
Re-install EBR using the same directory locations for the on-line indexes as in the original installation

Important: Once you understand the procedure for disaster recovery, make sure you have devised a disaster recovery plan for your site. If possible, you should test the ability to recover from a disaster at your site.

Preparing for a Disaster

Keep a file containing printed copies of the bootstrap records. Place these daily sheets of paper in a three-ring binder or a file folder.
Make a hardcopy record of the disks, partition sizes, and mount points for the server and any clients that have a local hard disk. This information could make recovery smoother for you in the future.

File the Bootstrap Information

May 09 20:59:03 1995 EBR bootstrap information Page 1

date time        level ssid file record
5/09/95 20:29:08 full  8380 39   0
5/09/95 20:52:25 full  8382 41   0

Banyan> cd /disk1/banyan/ebr
Banyan>
saveindex -c server_name

File the Disk Information

Banyan> df

/ (adsys ): 452 blocks 5128 i-nodes
/disk1 (/dev/adusr ): 252108 blocks 126154 i-nodes

Recovering a Secondary Disk

Important: It is impossible to provide step-by-step instructions on how to recover your system from a disaster, since every site is unique. The examples in this chapter are designed to give you general principles on how to recover a primary or secondary disk, and to help you understand the procedure. They are meant to be examples only, not instructions.

Example

1. Install the replacement disk.

2. Make sure the operating system and kernel recognize the new disk.

3. Label and partition the new disk so you can recover the file systems. Use the hard copy of the disk information to see the size of each partition. (See "File the Disk Information" in this chapter.)

Tip: If you do not already have this information, look at the directory /etc/fstab. Use the cat command to find out how the disk was partitioned into file systems. You will have to guess how much space to give each partition. However, you still have /disk1, so the partition information is available.

4. Make file systems for each raw partition that you are going to recover, and mount the block partition, consulting the hard copy of the df (or cat /etc/fstab) output. (EBR will not initialize file systems; it recovers data into existing ones.)

EBR will recover all files in one file system but will stop at a mount point.
You may run out of swap space if you recover too many file systems at once to your list of data to recover.

1. Run the recover command:

Banyan> cd /disk1/banyan/ebr
Banyan>
recover -s server_name -Zusername -zpassword

Here servername is the name of the EBR server that backs up this server. If you have already logged in to VINES as part of obtaining root access, you will not need to supply your username and password on the recover command line. You must be a member of the server' s AdminList to recover data to any of the server' s disks.

2. In the recover browser, mark the disk you want to recover. For example, if you want to recover /disk3, enter:

> mark /disk3

3. Recover the file system:

> recover

/disk3 is being recovered into its original location
Requesting 5023 files, this may take a while...

4. Exit the recover program.

> quit

Recovering /disk1

1. Run the recover command:

Banyan> cd /disk1/banyan/ebr
Banyan>
recover -s server_name -Zusername -zpassword

Here servername is the name of the EBR server that backs up this server. If you have already logged in to VINES as part of obtaining root access, you will not need to supply your username and password on the recover command line. You must be a member of the server' s AdminList to recover data to any of the server's disks.

2. In the recover browser, mark the root directory and /disk1/banyan.

> mark /disk1/banyan
>
mark /

3. Enter the following commands to forcibly overwrite files on disk with data from the tape:

> force
>
recover

4. Now that server configuration data has been recovered, shut down, then reboot the server.

5. After the reboot, access UNIX again and run the recover command as described in step 1.

6. Mark /disk1, then unmark /disk1/banyan.

> mark /disk1
>
unmark /disk1/banyan

7. Enter the following commands to forcibly overwrite files on disk with data from the tape:

> force
>
recover

This will recover any system software updates (patches) that may have been applied after VINES was installed from release media.

8. You may want to reboot the server again if there are patches that need to be in effect before you recover additional disks.

9. Recover data to any additional disks on your server, one disk at a time, using the following procedure:

> mark /disk1<n>
>
force
>
recover

Marking one disk at a time conserves server resources (memory and swap space).

10. Exit the recover program.

> quit

Important: Always reboot a system after recovering a primary disk.

Recovering an EBR Service's On-Line Indexes

August 20 03:30 1995 EBR bootstrap information Page 1

date time       level ssid       file record volume
8/08/95 7:44:38 full  1148869706 55   0 Full .008
8/09/95 6:12:09 9     1148869754 48   0 Nonfull .001
8/10/95 6:14:23 9     1148869808 63   0 Nonfull .001
8/11/95 6:29:58 9     1148869870 88   0 Nonfull .001

Note: You should temporarily disable any groups scheduled to run during the period that you are recovering the index.

1. Find the bootstrap save set id information. You will need it for the next two steps.

2. Retrieve the backup tape that contains the most recent backup named bootstrap, and load it into the server' s tape device.

3. Use the recoverindex command to extract the contents of the bootstrap backup:

Banyan> cd /disk1/banyan/ebr
Banyan>
recoverindex -s servername

recoverindex: Using servername as server

NOTICE: recoverindex is used to recover the EBR
server' s on-line file and media indexes from media
(backup tapes or disks) when either of the server' s
on-line file or media index has been lost or damaged.
Note that this command will OVERWRITE the server' s
existing on-line file and media indexes. recoverindex
is not used to recover EBR clients' on-line
indexes; normal recover procedures may be used for
this purpose.

What is the name of the tape drive you plan on using [dev/rct4]?
Enter the latest bootstrap save set id []: 1148869870
Enter starting file number (if known) [0]: 88
Enter starting record number (if known) [0]: 0
Please insert the volume on which save set id 1148869870 started into dev/rct4.
When you have done this, press <RETURN>:

Scanning dev/rct4 for save set 1148869870; this may take a while...
scanner: scanning 8mm 5GB tape space.006 on dev/rct4
uasm -r ebr/res/ebr.res
uasm -r ebr/res/ebrjb.res
uasm -r ebr/res/
ebrmmdbasm -r ebr/mm/mmvolume
ebr/mm/mmvolume: file exists, overwriting
uasm -r ebr/index/space/
ebrindexasm -r ebr/index/space/db
scanner: ssid 449955156: scan complete
scanner: ssid 449955156: 31 KB, 10 files
ebr/index/space/db: file exists, overwriting
uasm -r ebr/index/
uasm -r ebr/mm/
uasm -r ebr/
uasm -r
space: 31 records recovered, 0 discarded.
ebrindexasm: Building indexes for servername...
ebrindexasm: Caching save times for servername...
8mm 5GB tape space.006 mounted on dev/rct4, write protected

The bootstrap entry in the on-line index for mars has been recovered. The complete index is now being reconstructed from the various partial indexes which were saved during the normal saves for this server.

# ebrindexasm: Pursuing index pieces of ebr/index/space/db from servername.

Recovering 2 files into their original locations
Total estimated disk space needed for recover is 11 MB
Requesting 2 files, this may take a while...
ebrindexasm -r .db
.db: file exists, overwriting
: 25711 records recovered, 0 discarded.
ebrindexasm -r .db
.db: file exists, overwriting
ebrindexasm: waiting for lock on ../db.SCAVENGE
ebrindexasm: lock on ../db.SCAVENGE acquired
Received 2 files from ebr server ` mars'
: 733 records recovered, 0 discarded.
ebrindexasm: Building indexes for mars...
ebrindexasm: Caching save times for mars...
ebrindexasm: Suppressing duplicate entries in mars - 50 duplicates discarded.

The on-line index for space is now fully recovered.

Replacing EBR Configuration Data

nsrindexasm: The on-line index is now fully recovered.

1. Stop the EBR service.

2. Save the original configuration database directory and move the recovered version into the correct location. For example, if the EBR service' s data directory is located in /disk2/EBR, then the configuration databases are located in /disk2/EBR/res. After recovering the data from backup media, there will be two copies of the res directory. Use the one named res.R. To complete the recovery, enter the following commands:

Banyan> cd /disk2/EBR
Banyan>
mv res res.orig
Banyan>
mv res.R res

This preserves to old res directory for potential re-use if the new res directory causes problems. Once you have verified that the EBR configuration is correct, you can remove res.orig directory.

Banyan> rm -r /disk2/EBR/res.orig

3. Restart the EBR service. When it restarts, it will use the recovered configuration data.

Finding the Bootstrap Save Set ID

1. Place the most recent media used for backups in the server device.

2. Read the contents of the backup media with the scanner command:

Banyan> scanner /dev/rct4

scanner: scanning mars.2 on /dev/nrst8
client symbolic name save time size files ssid S
venus / 04/10/93 03:12:40 4326980 599 17475596 E
mars / 04/10/93 03:06:11 4478460 621 17475595 E
venus /usr 04/10/93 03:13:58 6918900 965 17475597 E
venus /home/venus 04/10/93 03:14:14 9661320 825 17475598 E
mars bootstrap 04/10/93 03:33:51 26607144 2 17850937 E

Summary

1. If the operating system is lost, reload and boot the system using the same hostname and disk partitioning.

2. If necessary, replace the damaged disk, format it, partition it, make new file systems, and then mount them in the same locations.

3. Re-install any EBR binaries that were lost.

4. If the EBR service's on-line indexes are destroyed, use recoverindex to recover them.

5. Recover the lost file systems, one at a time.

Previous PageTop Of Page