A
Universal Backup Tool for Oracle Recovery Manager (RMAN)
Steve Bourgeois,
Polaris Database Systems
Last Updated 07/20/2005
Created 04/14/2005
- Introduction
- Goals
- Requirements
- Installing and Configuring the PDS Kit
- Examples
- Conclusions
- References
1.
Introduction
Why Bother Doing This?
Not so long
ago, I was looking through the collection of RMAN backup scripts
that I have accumulated over the past 6+ years. As I looked through
the files, I found scripts for online database backups, offline
database backups, archived redo log backups, backups to Legato NetWorker,
backups to disk, backups for UNIX platforms, backups for Windows
platforms, et al.
At that point,
I decided that it was finally time to buckle down and integrate
all this disparate code into a single platform-independent tool.
However, once I started the initial design and coding process, it
was obvious that the effort required to meet the requirements was
a fairly serious undertaking.
This document
outlines the development of this tool from initial goals and functionality
criteria to examples and sample output from the finished PDS rman_backup.pl
tool. It also introduces a lite version of the tool that the reader
can download and experiment with.
It is assumed
that the reader has a working knowledge of Oracle RDBMS and Oracle
Recovery Manager (RMAN) technology, as well as a solid understanding
of Oracle database backup and recovery concepts and procedures.
All of the sample
scripts and files referenced in this document can be downloaded
here.
2.
Goals
What Should This Tool Do?
- One tool to replace all existing
RMAN backup tools.
- Run on all supported database
platforms (currently Solaris, Linux, AIX, DG/UX, and Windows)
with a single code base.
- Compatible with Oracle RDBMS
server version 8.0 and greater.
- Simple to use.
- Scalable and easily extensible.
- Support functionality introduced
in newer versions of RMAN while maintaining backward compatibility
with earlier versions of RMAN.
- Support online database, offline
database, and archived redo log backups to disk or tape.
- Support RMAN incremental backup
functionality.
- Support RMAN channel parallelization
functionality.
- Support RMAN restartable backup
functionality.
- Support RMAN physical and logical
block corruption detection functionality.
- Support multiple media management
software vendors (Legato NetWorker, Veritas NetBackup, etc.).
- Work seamlessly with or without
an RMAN recovery catalog database.
- Intelligent archived redo log
monitoring/backup/purge.
- Dry run functionality to generate
and validate RMAN scripts.
3.
Requirements
This section
documents the minimum requirements that your system must meet to
install, configure, and run the PDS rman_backup.pl backup
tool.
- Oracle RDBMS Server 8.0 or Greater
- A Perl 5 Interpreter
To meet the
requirement of a single code base for multiple platforms, this
tool is written in Perl, a cross-platform programming language.
A binary distribution of the Perl interpreter is available free
of charge for most platforms (including Windows and UNIX variants),
which makes it an attractive choice for this project. You can
easily verify if Perl is installed on your server with the following
command:
# perl -v
This is perl, v5.6.1 built for sun4-solaris
Copyright 1987-2001, Larry Wall
Perl may be copied only under the terms of either the Artistic
License or the GNU General Public License, which may be found
in the Perl 5 source kit.
Complete documentation for Perl, including FAQ lists, should
be found on this system using `man perl' or `perldoc perl'.
If you have access to the Internet, point your browser at
http://www.perl.com/, the Perl Home Page.
|
If you do
not have a version of Perl 5 installed, links to many binary distributions
can be found at the CPAN
archive. If you are using Oracle on a Windows platform, ActiveState's
ActivePerl
distribution is an excellent choice.
- Operating System Authentication for
Target Database Connectivity
This utility
requires password-free administrative connectivity to the target
database. If you see an error like the one below when connecting
to the target database with the connect string "connect / as
sysdba", then this utility will not work in your environment:
# sqlplus
SQL*Plus: Release 9.2.0.5.0 - Production on Fri Apr 15 15:56:24
2005
Copyright (c) 1982, 2002, Oracle Corporation. All rights reserved.
Enter user-name: / as sysdba
ERROR:
ORA-01031: insufficient privileges
|
If you are
running on a UNIX platform and see this error, verify that the
UNIX logon being used is a member of the UNIX group configured
for the 'SYSDBA' privilege (dba by default).
If you are
running on a Windows platform and see this error, verify that
the Windows logon being used is a member of the Windows groups
'ORA_DBA' or 'ORA_%ORACLE_SID%_DBA'. If the Windows logon is a
member of one of these groups and this problem persists, adding
the following line to the sqlnet.ora file on the database
server may resolve the issue:
SQLNET.AUTHENTICATION_SERVICES =
(NTS)
|
- Win32::AdminMisc on Microsoft
Windows Servers
By default,
Microsoft does not provide a utility to programmatically determine
the space utilization of a file system (like df on UNIX).
Without this functionality, we lose the ability to monitor the
utilization of the file system where the archived redo logs reside.
We work around
this limitation by using the handy Win32::AdminMisc
Perl module developed by Roth
Consulting. If you wish
to use the Win32::AdminMisc module in your environment,
you can download and install the appropriate distribution from
their ftp
site.
One problem
that I found with a Windows 2000 server was that error "Unable
to locate DLL MSVCR70.DLL" was raised when Win32::AdminMisc
was called. Apparently, Win32::AdminMisc was compiled with
newer version of Microsoft Visual C++ that my Windows 2000 server
was not prepared to handle. I was able to resolve this issue by
copying this Visual C++ runtime library from another computer
to the \WINNT\SYSTEM32 directory on the Windows 2000 server.
4.
Installing and Configuring the PDS Kit
The PDS LITE
distribution can be downloaded free of charge from the Polaris Database
Systems web site: ZIP
TAR.
Please note
that the PDS LITE kit is freeware and only supports RMAN script
generation and backups to disk channels. The PDS LITE kit is intended
to be used for educational purposes and as a companion to this paper.
This software is downloaded at your own risk. No warranty of any
kind is provided, and all use is entirely at your own risk.
(NOTE: The
full featured PDS kit is only available to customers of our Oracle
DBA consulting services)
- Choose the Top-Level Installation Directory
($PDS_HOME)
Choose the
$PDS_HOME directory where you would like the PDS kit to
reside. On an Oracle OFA-compliant system, we recommend the $ORACLE_BASE/tools/PDS
directory. Typical values would be /u01/app/oracle/tools/PDS
on UNIX environments and C:\ORACLE\TOOLS\PDS on Windows
environments.
- Unpack the Archive
An example
of unpacking the TAR archive on a UNIX platform:
# cd /u01/app/oracle/tools
# gzip -dc PDS.tar.gz|tar xvf -
|
On a Windows
platform, the TAR archive can be unpacked with the WinZip
utility.
The ZIP archive
can also be unpacked on a Windows platform with the WinZip
utility or with native Windows compressed folder support introduced
in Windows XP/2003. On a UNIX platform, the ZIP archive can be
unpacked with the UnZip
utility.
The PDS kit
has the directory structure show below:
PDS
___________|_________
| | | | |
bin etc log rman tmp
|
examples
|
- Associate the rman_backup.pl Program
with the Perl Interpreter
rman_backup.pl
is a Perl script, and on UNIX platforms the first line of the
script should reflect the location of the Perl interpreter on
your server (#!/usr/local/bin/perl by default). If the
Perl interpreter is installed in a different location on your
server, you should update the first line of rman_backup.pl
to reflect your environment. Please take care not to remove the
-w option from this line since it enables the output of
useful Perl warning messages.
You can determine
the location of the Perl interpreter on a UNIX server with the
following command:
If you are
using a Windows platform, the ActivePerl
distribution associates files with the .pl extension by
default. Alternatively, you could use the pl2bat tool included
with most Windows Perl distributions.
- Verify that the Target Database Has
a Valid oratab Entry Configured
The oratab
file (typically found in the /etc or /var/opt/oracle directories)
was originally used to configure Oracle SQL*Net V1 on UNIX platforms.
Although SQL*Net V1 was put out to pasture with Oracle RDBMS 7.2,
the oratab file is still in use today by the dbstart, dbshut,
and dbhome utilities included with the Oracle RDBMS software.
We also make use of this handy file to validate the $ORACLE_HOME
and $ORACLE_SID environment variables for the target database.
Oratab entries are of the form:
# $ORACLE_SID:$ORACLE_HOME:(N|Y):
O920:/d01/app/oracle/product/9.2.0:N
O817:c:\oracle\ora817:N
|
One caveat
is that the oratab file was never supported on the Microsoft Windows
platform. We work around this by supporting a local copy of this
file, called pdstab (located in $PDS_HOME/etc).
If a valid
oratab or pdstab entry is not configured, an error like the following
will be returned when rman_backup.pl is executed:
04/21/2005:11:02:57 : rman_backup.pl:
Invalid or missing entry for testdb in /etc/oratab
|
- Set Environment Variables (Optional)
For ease of
use, it is recommended that the $PDS_HOME environment variable
should be set to the directory where the PDS kit was installed
(see Choose the Top-Level
Installation Directory ($PDS_HOME)).
It is also recommended to include $PDS_HOME/bin in the
$PATH environment variable .
Bourne shell
example values for a UNIX platform (which can be set automatically
upon login by modifying the appropriate shell configuration file):
PDS_HOME=/u01/app/oracle/tools/PDS
PATH=$PATH:$PDS_HOME/bin
export PDS_HOME PATH
|
Example values
for a Windows platform (which can be set automatically with the
Environment Variables setting from Start->Control
Panel->System):
set PDS_HOME=c:\oracle\tools\PDS
set PATH=%PATH%;%PDS_HOME%\bin
|
5.
Examples
An overview
the rman_backup.pl functionality can be provided by executing
the following command:
# rman_backup.pl -h
Usage :
rman_backup.pl -Cdn [-c conn_str] [-D bkup_dir] [-i inc_level] [-k keep_time]
[-p #channels] [-P thresh_pct%] [-t mmtype] [-s mmserver] [-r restart_hours]
[-g grp/pol] [-e env_vars] $ORACLE_SID ONLINE
rman_backup.pl -Cdn [-c conn_str] [-D bkup_dir] [-i inc_level] [-k keep_time]
[-p #channels] [-P thresh_pct%] [-t mmtype] [-s mmserver] [-r restart_hours]
[-g grp/pol] [-e env_vars] $ORACLE_SID OFFLINE
rman_backup.pl -dmn [-c conn_str] [-D bkup_dir] [-k keep_time]
[-p #channels] [-P thresh_pct%] [-t mmtype] [-s mmserver]
[-g grp/pol] [-e env_vars] $ORACLE_SID REDO
rman_backup.pl -dn [-c conn_str] $ORACLE_SID RESYNC
rman_backup.pl -dn $ORACLE_SID CHECK
For help, type: rman_backup.pl -h
The following options are supported:
-c Specify the SQL*Net connect string to connect to the RMAN catalog
database. If none is specified, RMAN is run in NOCATALOG mode.
-C Perform an RMAN cumulative backup. This option is ignored for
archived redo log backups.
-d Enable RMAN debug mode
-D Specify a comma-delimited list of backup directories for an RMAN
backup to disk.
-e Specify a colon or semicolon-delimited list of environment variables
to pass to the media management server via the RMAN 'parms' clause.
-g Specify the group/policy which the media management server will use
to select the appropriate media pool. This value is ignored when
an RMAN backup to disk is specified.
-h Print out help text
-i Specify the RMAN incremental backup level. This option is ignored
for archived redo log backups.
-k Keep archived redo logs on disk for the specified minimum number of
hours. Archived redo logs containing changes made within the keep
period will not be purged from disk even if the ARCHIVELOG file
system exceeds the threshold specified with the -P parameter.
Please note that the newest archived redo log will always remain on
disk even if it contains changes made outside of the keep period.
The default keep period is 1 hour.
-m Enable archived redo log maintenance mode. An archived redo log
backup/purge is only performed if the file system utilization of
the ARCHIVELOG file system exceeds the threshold specified with
the -P parameter. Otherwise, no backup is performed.
This option is ignored for online and offline database backups.
-n No execution mode. Performs a simulated RMAN backup without
actually executing the RMAN commands. This option is useful for
generating RMAN scripts that can be modified and executed manually.
-p Parallelize the RMAN backup by specifying the number of disk or
tape backup channels.
-P Specify ARCHIVELOG file system utilization threshold percentage.
Archived redo logs are purged after backup when the utilization of
the file system they reside on exceeds this value. This value is
disabled by default and is ignored when a database is not running
in ARCHIVELOG mode.
-r Specify the number of hours for an RMAN restartable backup.
When this option is enabled, RMAN will only back up database
datafiles and archived redo log files not backed up within the
last X hours (where X is numeric argument for the -r parameter).
This option is not supported prior to Oracle9i.
-s Specify which machine to use as a media management server. This
value is ignored when an RMAN backup to disk is specified.
-t Specify the media management server type. Set this parameter to
"NSR" for a Legato NetWorker server or to "NBU" for a Veritas
NetBackup server. "NSR" is selected by default.
|
Now we're ready
to try a few examples:
- UNIX: Online Oracle 9i R2 Database
Backup to Tape
The command
line :
# rman_backup.pl -i 1 -P 50 -s nsr01-g
db01 O920 online
|
main
log | RMAN
backup log | RMAN
backup script
Summary :
- ONLINE database backup to Legato
NetWorker (default media management server type).
- Backup to Legato NetWorker
server nsr01 (-s nsr01).
- Backup to Legato NetWorker
group db01 (-g db01).
- Level 1 incremental backup
(-i 1).
- Purge archived redo logs when
file system utilization is greater than 50% (-P 50).
- Run without RMAN recovery catalog
(default).
- Windows: Offline Oracle 9i R2 Database
Backup to Disk
The command
line :
# rman_backup.pl -c rman/rman@rman920
-p 2 -D c:\backups\O920 O920 offline
|
main
log | RMAN
catalog lookup log | RMAN
catalog lookup script
RMAN
backup log | RMAN
backup script
Summary :
- OFFLINE database backup to
directory c:\backups\O920 (-D c:\backups\O920).
- Level 0 incremental backup
(default).
- Run with RMAN recovery catalog
(-c rman/rman@rman920).
- Parallelize backup to 2 disk
channels (-p 2) .
- UNIX: Oracle 8i R3 Archived Redo Log
Backup to Tape
The command
line :
# rman_backup.pl -m -c rman/rman@O920
-P 65 -s nsr01 -g db01 O817 redo
|
main
log | RMAN
catalog lookup log | RMAN
catalog lookup script
RMAN
backup log | RMAN
backup script
Summary :
- Archived redo log backup to
Legato NetWorker (default media management server type).
- Backup to Legato NetWorker
server nsr01 (-s nsr01).
- Backup to Legato NetWorker
group db01 (-g db01).
- Purge archived redo logs when
file system utilization is greater than 65% (-P 65).
- Run in maintenance mode. Only
backup and purge archived redo logs if the file system utilization
of the ARCHIVELOG file system exceeds 65% (-m).
- Run with RMAN recovery catalog
(-c rman/rman@rman920).
- Windows: Oracle 8i R3 Database Corruption
Check
The command
line :
# rman_backup.pl O817 check
|
main log | RMAN
backup log | RMAN
backup script
Summary :
- Check database O817 for physical
and logical block corruption.
- Run without RMAN recovery catalog
(default).
- UNIX: Oracle 9i R2 Database Resynchronization
with RMAN Recovery Catalog
The command
line :
# rman_backup.pl -c rman/rman@rman920
O920 resync
|
main log | RMAN
catalog lookup log | RMAN
catalog lookup script
RMAN
resynchronization log | RMAN
resynchronization script
Summary :
- Resynchronize database O920
with the RMAN Recovery Catalog.
- Run with RMAN recovery catalog
(-c rman/rman@rman920).
6.
Conclusions
This document
gives an overview of the PDS platform-independent RMAN backup tool
rman_backup.pl that I developed to support the diverse set
of Oracle database server versions and platforms that my clients
use. I hope that the functionality requirements in conjunction with
the freeware PDS kit and real world examples will help address Oracle
RMAN backup challenges in your environment(s).
There is still
quite a bit of work that could be done in this space, including
Oracle 10g enhancements, duplex backup support, support for additional
media management server vendors, etc. And of course, building a
companion utility for rman_backup.pl that provides intelligent
RMAN database restore and recoverability functionality would also
be a logical addition. However, that is another project in itself.
If you have
any questions or comments about this document or want to report
any errata or omissions, please feel free to email
me.
If you are reporting
a problem, please include the following information:
- The operating system name and
full version.
- The full Oracle RDBMS version
(i.e. 9.2.0.5.0).
- The Perl version information
(from the following command):
- The name of the operating system
that the Perl binary was compiled for (from the following command):
# perl -e 'print "$^O\n"'
|
- A brief description of the problem.
7.
References
- Oracle8i Recovery Manager
User’s Guide and Reference (8.1.6) Oracle Part No. A76990-01,
December 1999
- Oracle9i Recovery Manager
Reference Release 2 (9.2) Oracle Part No. A96565-01, March
2002
- Oracle9i Recovery Manager
User’s Guide Release 2 (9.2) Oracle Part No. A96566-01,
March 2002
|