A Universal Backup Tool for Oracle Recovery Manager (RMAN)

Steve Bourgeois, Polaris Database Systems

Last Updated 07/20/2005
Created 04/14/2005

  1. Introduction
  2. Goals
  3. Requirements
  4. Installing and Configuring the PDS Kit
  5. Examples
  6. Conclusions
  7. References

1. Introduction

Why Bother Doing This?

Not so long ago, I was looking through the collection of RMAN backup scripts that I have accumulated over the past 6+ years. As I looked through the files, I found scripts for online database backups, offline database backups, archived redo log backups, backups to Legato NetWorker, backups to disk, backups for UNIX platforms, backups for Windows platforms, et al.

At that point, I decided that it was finally time to buckle down and integrate all this disparate code into a single platform-independent tool. However, once I started the initial design and coding process, it was obvious that the effort required to meet the requirements was a fairly serious undertaking.

This document outlines the development of this tool from initial goals and functionality criteria to examples and sample output from the finished PDS rman_backup.pl tool. It also introduces a lite version of the tool that the reader can download and experiment with.

It is assumed that the reader has a working knowledge of Oracle RDBMS and Oracle Recovery Manager (RMAN) technology, as well as a solid understanding of Oracle database backup and recovery concepts and procedures.

All of the sample scripts and files referenced in this document can be downloaded here.

2. Goals

What Should This Tool Do?
  • One tool to replace all existing RMAN backup tools.
  • Run on all supported database platforms (currently Solaris, Linux, AIX, DG/UX, and Windows) with a single code base.
  • Compatible with Oracle RDBMS server version 8.0 and greater.
  • Simple to use.
  • Scalable and easily extensible.
  • Support functionality introduced in newer versions of RMAN while maintaining backward compatibility with earlier versions of RMAN.
  • Support online database, offline database, and archived redo log backups to disk or tape.
  • Support RMAN incremental backup functionality.
  • Support RMAN channel parallelization functionality.
  • Support RMAN restartable backup functionality.
  • Support RMAN physical and logical block corruption detection functionality.
  • Support multiple media management software vendors (Legato NetWorker, Veritas NetBackup, etc.).
  • Work seamlessly with or without an RMAN recovery catalog database.
  • Intelligent archived redo log monitoring/backup/purge.
  • Dry run functionality to generate and validate RMAN scripts.

3. Requirements

This section documents the minimum requirements that your system must meet to install, configure, and run the PDS rman_backup.pl backup tool.

  1. Oracle RDBMS Server 8.0 or Greater

  2. A Perl 5 Interpreter

    3. To meet the requirement of a single code base for multiple platforms, this tool is written in Perl, a cross-platform programming language. A binary distribution of the Perl interpreter is available free of charge for most platforms (including Windows and UNIX variants), which makes it an attractive choice for this project. You can easily verify if Perl is installed on your server with the following command:

    # perl -v

    This is perl, v5.6.1 built for sun4-solaris

    Copyright 1987-2001, Larry Wall

    Perl may be copied only under the terms of either the Artistic License or the GNU General Public License, which may be found in the Perl 5 source kit.

    Complete documentation for Perl, including FAQ lists, should be found on this system using `man perl' or `perldoc perl'. If you have access to the Internet, point your browser at http://www.perl.com/, the Perl Home Page.

    If you do not have a version of Perl 5 installed, links to many binary distributions can be found at the CPAN archive. If you are using Oracle on a Windows platform, ActiveState's ActivePerl distribution is an excellent choice.

  3. Operating System Authentication for Target Database Connectivity

    4. This utility requires password-free administrative connectivity to the target database. If you see an error like the one below when connecting to the target database with the connect string "connect / as sysdba", then this utility will not work in your environment:

    # sqlplus

    SQL*Plus: Release 9.2.0.5.0 - Production on Fri Apr 15 15:56:24 2005

    Copyright (c) 1982, 2002, Oracle Corporation. All rights reserved.

    Enter user-name: / as sysdba
    ERROR:
    ORA-01031: insufficient privileges


    If you are running on a UNIX platform and see this error, verify that the UNIX logon being used is a member of the UNIX group configured for the 'SYSDBA' privilege (dba by default).

    If you are running on a Windows platform and see this error, verify that the Windows logon being used is a member of the Windows groups 'ORA_DBA' or 'ORA_%ORACLE_SID%_DBA'. If the Windows logon is a member of one of these groups and this problem persists, adding the following line to the sqlnet.ora file on the database server may resolve the issue:

    SQLNET.AUTHENTICATION_SERVICES = (NTS)

  4. Win32::AdminMisc on Microsoft Windows Servers

    5. By default, Microsoft does not provide a utility to programmatically determine the space utilization of a file system (like df on UNIX). Without this functionality, we lose the ability to monitor the utilization of the file system where the archived redo logs reside.

    We work around this limitation by using the handy Win32::AdminMisc Perl module developed by Roth Consulting. If you wish to use the Win32::AdminMisc module in your environment, you can download and install the appropriate distribution from their ftp site.

    One problem that I found with a Windows 2000 server was that error "Unable to locate DLL MSVCR70.DLL" was raised when Win32::AdminMisc was called. Apparently, Win32::AdminMisc was compiled with newer version of Microsoft Visual C++ that my Windows 2000 server was not prepared to handle. I was able to resolve this issue by copying this Visual C++ runtime library from another computer to the \WINNT\SYSTEM32 directory on the Windows 2000 server.

4. Installing and Configuring the PDS Kit

The PDS LITE distribution can be downloaded free of charge from the Polaris Database Systems web site: ZIP TAR.

Please note that the PDS LITE kit is freeware and only supports RMAN script generation and backups to disk channels. The PDS LITE kit is intended to be used for educational purposes and as a companion to this paper. This software is downloaded at your own risk. No warranty of any kind is provided, and all use is entirely at your own risk.

(NOTE: The full featured PDS kit is only available to customers of our Oracle DBA consulting services)

  1. Choose the Top-Level Installation Directory ($PDS_HOME)

    2. Choose the $PDS_HOME directory where you would like the PDS kit to reside. On an Oracle OFA-compliant system, we recommend the $ORACLE_BASE/tools/PDS directory. Typical values would be /u01/app/oracle/tools/PDS on UNIX environments and C:\ORACLE\TOOLS\PDS on Windows environments.

  2. Unpack the Archive

    3. An example of unpacking the TAR archive on a UNIX platform:

    # cd /u01/app/oracle/tools
    # gzip -dc PDS.tar.gz|tar xvf -

    On a Windows platform, the TAR archive can be unpacked with the WinZip utility.

    The ZIP archive can also be unpacked on a Windows platform with the WinZip utility or with native Windows compressed folder support introduced in Windows XP/2003. On a UNIX platform, the ZIP archive can be unpacked with the UnZip utility.

    The PDS kit has the directory structure show below:

                PDS
  ___________|_________
  |    |    |    |    |
 bin  etc  log rman  tmp
            |          
         examples

  3. Associate the rman_backup.pl Program with the Perl Interpreter

    4. rman_backup.pl is a Perl script, and on UNIX platforms the first line of the script should reflect the location of the Perl interpreter on your server (#!/usr/local/bin/perl by default). If the Perl interpreter is installed in a different location on your server, you should update the first line of rman_backup.pl to reflect your environment. Please take care not to remove the -w option from this line since it enables the output of useful Perl warning messages.

    You can determine the location of the Perl interpreter on a UNIX server with the following command:

    # which perl

    If you are using a Windows platform, the ActivePerl distribution associates files with the .pl extension by default. Alternatively, you could use the pl2bat tool included with most Windows Perl distributions.

  4. Verify that the Target Database Has a Valid oratab Entry Configured

    5. The oratab file (typically found in the /etc or /var/opt/oracle directories) was originally used to configure Oracle SQL*Net V1 on UNIX platforms. Although SQL*Net V1 was put out to pasture with Oracle RDBMS 7.2, the oratab file is still in use today by the dbstart, dbshut, and dbhome utilities included with the Oracle RDBMS software. We also make use of this handy file to validate the $ORACLE_HOME and $ORACLE_SID environment variables for the target database. Oratab entries are of the form:

    # $ORACLE_SID:$ORACLE_HOME:(N|Y):
    O920:/d01/app/oracle/product/9.2.0:N
    O817:c:\oracle\ora817:N

    One caveat is that the oratab file was never supported on the Microsoft Windows platform. We work around this by supporting a local copy of this file, called pdstab (located in $PDS_HOME/etc).

    If a valid oratab or pdstab entry is not configured, an error like the following will be returned when rman_backup.pl is executed:

    04/21/2005:11:02:57 : rman_backup.pl: Invalid or missing entry for testdb in /etc/oratab


  5. Set Environment Variables (Optional)

    6. For ease of use, it is recommended that the $PDS_HOME environment variable should be set to the directory where the PDS kit was installed (see Choose the Top-Level Installation Directory ($PDS_HOME)). It is also recommended to include $PDS_HOME/bin in the $PATH environment variable .

    Bourne shell example values for a UNIX platform (which can be set automatically upon login by modifying the appropriate shell configuration file):

    PDS_HOME=/u01/app/oracle/tools/PDS
    PATH=$PATH:$PDS_HOME/bin
    export PDS_HOME PATH

    Example values for a Windows platform (which can be set automatically with the Environment Variables setting from Start->Control Panel->System):

    set PDS_HOME=c:\oracle\tools\PDS
    set PATH=%PATH%;%PDS_HOME%\bin

5. Examples

An overview the rman_backup.pl functionality can be provided by executing the following command:

# rman_backup.pl -h

Usage :

rman_backup.pl -Cdn [-c conn_str] [-D bkup_dir] [-i inc_level] [-k keep_time]
[-p #channels] [-P thresh_pct%] [-t mmtype] [-s mmserver] [-r restart_hours]
[-g grp/pol] [-e env_vars] $ORACLE_SID ONLINE

rman_backup.pl -Cdn [-c conn_str] [-D bkup_dir] [-i inc_level] [-k keep_time]
[-p #channels] [-P thresh_pct%] [-t mmtype] [-s mmserver] [-r restart_hours]
[-g grp/pol] [-e env_vars] $ORACLE_SID OFFLINE

rman_backup.pl -dmn [-c conn_str] [-D bkup_dir] [-k keep_time]
[-p #channels] [-P thresh_pct%] [-t mmtype] [-s mmserver]
[-g grp/pol] [-e env_vars] $ORACLE_SID REDO

rman_backup.pl -dn [-c conn_str] $ORACLE_SID RESYNC

rman_backup.pl -dn $ORACLE_SID CHECK

For help, type: rman_backup.pl -h

The following options are supported:

  -c    Specify the SQL*Net connect string to connect to the RMAN catalog
        database.  If none is specified, RMAN is run in NOCATALOG mode.
  -C    Perform an RMAN cumulative backup.  This option is ignored for
        archived redo log backups.
  -d    Enable RMAN debug mode
  -D    Specify a comma-delimited list of backup directories for an RMAN
        backup to disk.
  -e    Specify a colon or semicolon-delimited list of environment variables
        to pass to the media management server via the RMAN 'parms' clause.
  -g    Specify the group/policy which the media management server will use
        to select the appropriate media pool.  This value is ignored when
        an RMAN backup to disk is specified.
  -h    Print out help text
  -i    Specify the RMAN incremental backup level.  This option is ignored
        for archived redo log backups.
  -k    Keep archived redo logs on disk for the specified minimum number of
        hours.  Archived redo logs containing changes made within the keep
        period will not be purged from disk even if the ARCHIVELOG file
        system exceeds the threshold specified with the -P parameter.
        Please note that the newest archived redo log will always remain on
        disk even if it contains changes made outside of the keep period.
        The default keep period is 1 hour.
  -m    Enable archived redo log maintenance mode.  An archived redo log
        backup/purge is only performed if the file system utilization of
        the ARCHIVELOG file system exceeds the threshold specified with
        the -P parameter.  Otherwise, no backup is performed.
        This option is ignored for online and offline database backups.
  -n    No execution mode.  Performs a simulated RMAN backup without
        actually executing the RMAN commands.  This option is useful for
        generating RMAN scripts that can be modified and executed manually.
  -p    Parallelize the RMAN backup by specifying the number of disk or
        tape backup channels.
  -P    Specify ARCHIVELOG file system utilization threshold percentage.
        Archived redo logs are purged after backup when the utilization of
        the file system they reside on exceeds this value.  This value is
        disabled by default and is ignored when a database is not running
        in ARCHIVELOG mode.
  -r    Specify the number of hours for an RMAN restartable backup.
        When this option is enabled, RMAN will only back up database
        datafiles and archived redo log files not backed up within the
        last X hours (where X is numeric argument for the -r parameter).
        This option is not supported prior to Oracle9i.
  -s    Specify which machine to use as a media management server.  This
        value is ignored when an RMAN backup to disk is specified.
  -t    Specify the media management server type.  Set this parameter to
        "NSR" for a Legato NetWorker server or to "NBU" for a Veritas
        NetBackup server.  "NSR" is selected by default.

Now we're ready to try a few examples:

  1. UNIX: Online Oracle 9i R2 Database Backup to Tape

    2. The command line :

    # rman_backup.pl -i 1 -P 50 -s nsr01-g db01 O920 online

    main log | RMAN backup log | RMAN backup script

    Summary :

    • ONLINE database backup to Legato NetWorker (default media management server type).
    • Backup to Legato NetWorker server nsr01 (-s nsr01).
    • Backup to Legato NetWorker group db01 (-g db01).
    • Level 1 incremental backup (-i 1).
    • Purge archived redo logs when file system utilization is greater than 50% (-P 50).
    • Run without RMAN recovery catalog (default).

  2. Windows: Offline Oracle 9i R2 Database Backup to Disk

    3. The command line :

    # rman_backup.pl -c rman/rman@rman920 -p 2 -D c:\backups\O920 O920 offline

    main log | RMAN catalog lookup log | RMAN catalog lookup script
    RMAN backup log | RMAN backup script

    Summary :

    • OFFLINE database backup to directory c:\backups\O920 (-D c:\backups\O920).
    • Level 0 incremental backup (default).
    • Run with RMAN recovery catalog (-c rman/rman@rman920).
    • Parallelize backup to 2 disk channels (-p 2) .

  3. UNIX: Oracle 8i R3 Archived Redo Log Backup to Tape

    4. The command line :

    # rman_backup.pl -m -c rman/rman@O920 -P 65 -s nsr01 -g db01 O817 redo

    main log | RMAN catalog lookup log | RMAN catalog lookup script
    RMAN backup log | RMAN backup script

    Summary :

    • Archived redo log backup to Legato NetWorker (default media management server type).
    • Backup to Legato NetWorker server nsr01 (-s nsr01).
    • Backup to Legato NetWorker group db01 (-g db01).
    • Purge archived redo logs when file system utilization is greater than 65% (-P 65).
    • Run in maintenance mode. Only backup and purge archived redo logs if the file system utilization of the ARCHIVELOG file system exceeds 65% (-m).
    • Run with RMAN recovery catalog (-c rman/rman@rman920).

  4. Windows: Oracle 8i R3 Database Corruption Check

    5. The command line :

    # rman_backup.pl O817 check

    main log | RMAN backup log | RMAN backup script

    Summary :

    • Check database O817 for physical and logical block corruption.
    • Run without RMAN recovery catalog (default).

  5. UNIX: Oracle 9i R2 Database Resynchronization with RMAN Recovery Catalog

    6. The command line :

    # rman_backup.pl -c rman/rman@rman920 O920 resync

    main log | RMAN catalog lookup log | RMAN catalog lookup script
    RMAN resynchronization log | RMAN resynchronization script

    Summary :

    • Resynchronize database O920 with the RMAN Recovery Catalog.
    • Run with RMAN recovery catalog (-c rman/rman@rman920).

6. Conclusions

This document gives an overview of the PDS platform-independent RMAN backup tool rman_backup.pl that I developed to support the diverse set of Oracle database server versions and platforms that my clients use. I hope that the functionality requirements in conjunction with the freeware PDS kit and real world examples will help address Oracle RMAN backup challenges in your environment(s).

There is still quite a bit of work that could be done in this space, including Oracle 10g enhancements, duplex backup support, support for additional media management server vendors, etc. And of course, building a companion utility for rman_backup.pl that provides intelligent RMAN database restore and recoverability functionality would also be a logical addition. However, that is another project in itself.

If you have any questions or comments about this document or want to report any errata or omissions, please feel free to email me.

If you are reporting a problem, please include the following information:

  • The operating system name and full version.
  • The full Oracle RDBMS version (i.e. 9.2.0.5.0).
  • The Perl version information (from the following command):

    # perl -v

  • The name of the operating system that the Perl binary was compiled for (from the following command):

    # perl -e 'print "$^O\n"'

  • A brief description of the problem.

 

7. References

  1. Oracle8i Recovery Manager User’s Guide and Reference (8.1.6) Oracle Part No. A76990-01, December 1999
  2. Oracle9i Recovery Manager Reference Release 2 (9.2) Oracle Part No. A96565-01, March 2002
  3. Oracle9i Recovery Manager User’s Guide Release 2 (9.2) Oracle Part No. A96566-01, March 2002