Thursday, March 31, 2011

Temp Usage.

Introduction

When a sort operation is too large to fit in memory, Oracle allocates space in a temporary tablespace in order to write data off to disk. Temporary space is a resource shared by multiple sessions on the database, and quotas cannot be set to limit how much temporary space can be used by any individual user or session. If a user submits a query with an incomplete WHERE clause, an enormous Cartesian product may result. That is all it takes to fill the temporary tablespace and possibly impact many other users on the system. When the temporary tablespace fills, any statement that requires additional temporary space will fail with an “ORA-1652: unable to extend temp segment” error.

In this paper, we will first briefly review how Oracle manages sorting operations. Next we’ll discuss how a database administrator can determine if any statements on the database have failed because the temporary tablespace ran out of space. We’ll also present two techniques a DBA can use to understand how space in the temporary tablespace is being used and how users are being impacted by a full temporary tablespace. The first technique we’ll look at includes how to direct Oracle to log every statement that fails for lack of temporary space. The second technique provides a set of queries a DBA can run at any time to capture in real time how temporary space is currently being used on a per-session or per-statement basis. These techniques can help DBAs address chronic or intermittent shortages of temporary space.

Oracle Sorting Basics

Many different circumstances can cause Oracle to sort data. For example, Oracle sorts data when creating an index and when processing most queries that include an ORDER BY or GROUP BY clause. Oracle sessions begin sorting data in memory. If the amount of data being sorted is small enough, the entire sort will be completed in memory with no intermediate data written to disk. When Oracle needs to store data in a global temporary table or build a hash table for a hash join, Oracle also starts the operation in memory and completes the task without writing to disk if the amount of data involved is small enough. While populating a global temporary table or building a hash is not a sorting operation, we will lump all of these activities together in this paper because they are handled in a similar way by Oracle.

If an operation uses up a threshold amount of memory, then Oracle breaks the operation into smaller ones that can each be performed in memory. Partial results are written to disk in a temporary tablespace. The threshold for how much memory may be used by any one session is controlled by instance parameters. If the workarea_size_policy parameter is set to AUTO, then the pga_aggregate_target parameter indicates how much memory can be used collectively by all sessions for activities such as sorting and hashing. Oracle will automatically assess and decide how much of this memory any individual session should be allowed to use. If the workarea_size_policy parameter is set to MANUAL, then instance parameters such as sort_area_size, hash_area_size, and bitmap_merge_area_size dictate how much memory each session can use for these operations.

Each database user has a temporary tablespace (or temporary tablespace group in Oracle 10g) designated in their user definition. Whenever a sort operation grows too large to be performed entirely in memory, Oracle will allocate space in the temporary tablespace designated for the user performing the operation. You can see a user’s temporary tablespace designation by querying the dba_users view.

Temporary segments in temporary tablespaces—which we will call “sort segments”—are owned by the SYS user, not the database user performing a sort operation. There typically is just one sort segment per temporary tablespace, because multiple sessions can share space in one sort segment. Users do not need to have quota on the temporary tablespace in order to perform sorts on disk. In fact, quotas on temporary tablespaces are ignored by Oracle.

Temporary tablespaces can only hold sort segments. Oracle’s internal behavior is optimized for this fact. For example, writes to a sort segment do not generate redo or undo. Also, allocations of sort segment blocks to a specific session do not need to be recorded in the data dictionary or a file allocation bitmap. Why? Because data in a temporary tablespace does not need to persist beyond the life of the database session that created it.

One SQL statement can cause multiple sort operations, and one database session can have multiple SQL statements active at the same time—each potentially with multiple sorts to disk. When the results of a sort to disk are no longer needed, its blocks in the sort segment are marked as no longer in use and can be allocated to another sort operation.

A sort operation will fail if a sort to disk needs more disk space and there are 1.) no unused blocks in the sort segment, and 2.) no space available in the temporary tablespace for the sort segment to allocate an additional extent. This will most likely cause the statement that prompted the sort to fail with the Oracle error, “ORA-1652: unable to extend temp segment.” This error message also gets logged in the alert log for the instance.

It is important to note that not all ORA-1652 errors indicate temporary tablespace issues. For example, moving a table to a different tablespace with the ALTER TABLE…MOVE statement will cause an ORA-1652 error if the target tablespace does not have enough space for the table.

Identifying SQL Statements that Fail Due to Lack of Temporary Space

It is helpful that Oracle logs ORA-1652 errors to the instance alert log as it informs a database administrator that there is a space issue. The error message includes the name of the tablespace in which the lack of space occurred, and a DBA can use this information to determine if the problem is related to sort segments in a temporary tablespace or if there is a different kind of space allocation problem.

Unfortunately, Oracle does not identify the text of the SQL statement that failed. Thus we are informed that a problem has occurred but we are not given tools with which to identify the cause of the problem nor measure the user impact of the statement failure.

However, Oracle does have a diagnostic event mechanism that can be used to give us more information whenever an ORA-1652 error occurs by causing Oracle server processes to write to a trace file. This trace file will contain a wealth of information, including the exact text of the SQL statement that was being processed at the time that the ORA-1652 error occurred. This diagnostic event imposes very little overhead on the system, because Oracle only writes information to the trace file when an ORA-1652 error occurs.

You can set a diagnostic event for the ORA-1652 error in your individual database session with the following statement:

ALTER SESSION SET EVENTS '1652 trace name errorstack';
You can set the diagnostic event instance-wide with the following statement:

ALTER SYSTEM SET EVENTS '1652 trace name errorstack';
The above statement will affect the current instance only and will not edit the server parameter file. That is to say, if you stop and restart the instance, the diagnostic event setting will no longer be active. I don’t recommend setting this diagnostic event on a permanent basis, but if you want to edit your server parameter file, you could use a statement like the following:

ALTER SYSTEM SET EVENT = '1652 trace name errorstack' SCOPE = SPFILE;
You can also set diagnostic events in another session (without affecting all sessions instance-wide) by using the “oradebug event” command in SQL*Plus.

You can deactivate the ORA-1652 diagnostic event or remove all diagnostic event settings from the server parameter file with statements such as the following:

ALTER SESSION SET EVENTS '1652 trace name context off';
ALTER SYSTEM SET EVENTS '1652 trace name context off';
ALTER SYSTEM RESET EVENT SCOPE = SPFILE SID = '*';
If a SQL statement fails due to lack of space in the temporary tablespace and the ORA-1652 diagnostic event has been activated, then the Oracle server process that encountered the error will write a trace file to the directory specified by the user_dump_dest instance parameter. The entry in the instance alert log that indicates an ORA-1652 error occurred will also indicate that a trace file was written. An entry in the instance alert log will look like this:

Tue Jan 2 17:21:14 2007
Errors in file /u01/app/oracle/admin/rpkprod/udump/rpkprod_ora_10847.trc:
ORA-01652: unable to extend temp segment by 128 in tablespace TEMP
The top portion of a sample trace file is as follows:

Oracle Database 10g Release 10.2.0.2.0 - 64bit Production
ORACLE_HOME = /u01/app/oracle/product/10.2.0/db_2
System name: SunOS
Node name: rpk
Release: 5.8
Version: Generic_108528-27
Machine: sun4u
Instance name: rpkprod
Redo thread mounted by this instance: 1
Oracle process number: 18
Unix process pid: 10847, image: oracle@rpk (TNS V1-V3)

*** ACTION NAME:() 2007-01-02 17:21:14.871
*** MODULE NAME:(SQL*Plus) 2007-01-02 17:21:14.871
*** SERVICE NAME:(SYS$USERS) 2007-01-02 17:21:14.871
*** SESSION ID:(130.13512) 2007-01-02 17:21:14.871
*** 2007-01-02 17:21:14.871
ksedmp: internal or fatal error
ORA-01652: unable to extend temp segment by 128 in tablespace TEMP
Current SQL statement for this session:
SELECT "A1"."INVOICE_ID", "A1"."INVOICE_NUMBER", "A1"."INVOICE_DAT
E", "A1"."CUSTOMER_ID", "A1"."CUSTOMER_NAME", "A1"."INVOICE_AMOUNT",
"A1"."PAYMENT_TERMS", "A1"."OPEN_STATUS", "A1"."GL_DATE", "A1"."ITE
M_COUNT", "A1"."PAYMENTS_TOTAL"
FROM "INVOICE_SUMMARY_VIEW" "A1"
ORDER BY "A1"."CUSTOMER_NAME", "A1"."INVOICE_NUMBER"
----- Call Stack Trace -----
From the trace file you can clearly see the full text of the SQL statement that failed. You can also see when it failed along with attributes of the database session such as module, action, and service name. It is important to note that the statements captured in trace files with this method may not themselves be the cause of space issues in the temporary tablespace. For example, one query could run successfully and consume 99.9% of the temporary tablespace due to a Cartesian product, while a second query fails when trying to allocate just a small amount of sort space. The second query is the one that will get captured in a trace file, while the first query is more likely to be the root cause of the problem.

The trace file will contain additional information, including a call stack trace and a binary stack dump. This information is not likely to be useful, unless perhaps you want to learn more about Oracle internals.

The diagnostic event facility has been built into the Oracle database product for a very long time, but it is not widely documented. Oracle Support’s position appears to be that you should not use this facility unless directed to do so by Oracle Support. There are certain widely-known diagnostic events such as 10046 for extended SQL trace and 10053 for tracing the cost-based optimizer, and there are certain events that can alter Oracle’s behavior significantly. In general, you absolutely should not try setting diagnostic events in a production database unless you have a very good idea of what they do.

Although I am not aware of an Oracle Support document that officially blesses setting diagnostic event 1652 for identifying SQL statements that fail due to lack of sort space, there are bulletins on Metalink that do show how to set events to dump an error stack for basic Oracle errors. Metalink document 217274.1, for example, shows how to set a diagnostic event for the ORA-942 (“table or view does not exist”) error. We are doing the exact same thing here for the ORA-1652 error, and therefore it seems like a relatively safe thing to do.

Like most debugging or diagnostic facilities, you should only use the ORA-1652 diagnostic event to the extent you really need to. If you regularly get ORA-1652 errors in one batch job and you can add an ALTER SESSION statement to the beginning of the batch job, then doing so would be preferable to setting the diagnostic event at the instance-level. Typically there shouldn’t be a need to set this diagnostic event at the instance level on a permanent basis or in the server parameter file.

Monitoring Temporary Space Usage

Instead of waiting for a temporary tablespace to fill and for statements to fail, you can monitor temporary space usage in the database in real time. At any given time, Oracle can tell you about all of the database’s temporary tablespaces, sort space usage on a session basis, and sort space usage on a statement basis. All of this information is available from v$ views, and the queries shown in this section can be run by any database user with DBA privileges.

Temporary Segments

The following query displays information about all sort segments in the database. (As a reminder, we use the term “sort segment” to refer to a temporary segment in a temporary tablespace.) Typically, Oracle will create a new sort segment the very first time a sort to disk occurs in a new temporary tablespace. The sort segment will grow as needed, but it will not shrink and will not go away after all sorts to disk are completed. A database with one temporary tablespace will typically have just one sort segment.

SELECT A.tablespace_name tablespace, D.mb_total,
SUM (A.used_blocks * D.block_size) / 1024 / 1024 mb_used,
D.mb_total - SUM (A.used_blocks * D.block_size) / 1024 / 1024 mb_free
FROM v$sort_segment A,
(
SELECT B.name, C.block_size, SUM (C.bytes) / 1024 / 1024 mb_total
FROM v$tablespace B, v$tempfile C
WHERE B.ts#= C.ts#
GROUP BY B.name, C.block_size
) D
WHERE A.tablespace_name = D.name
GROUP by A.tablespace_name, D.mb_total;

The query displays for each sort segment in the database the tablespace the segment resides in, the size of the tablespace, the amount of space within the sort segment that is currently in use, and the amount of space available. Sample output from this query is as follows:

TABLESPACE MB_TOTAL MB_USED MB_FREE
------------------------------- ---------- ---------- ----------
TEMP 10000 9 9991

This example shows that there is one sort segment in a 10,000 Mb tablespace called TEMP. Right now, 9 Mb of the sort segment is in use, leaving a total of 9,991 Mb available for additional sort operations. (Note that the available space may consist of unused blocks within the sort segment, unallocated extents in the TEMP tablespace, or a combination of the two.)

Sort Space Usage by Session

The following query displays information about each database session that is using space in a sort segment. Although one session may have many sort operations active at once, this query summarizes the information by session. This query will need slight modification to run on Oracle 8i databases, since the dba_tablespaces view did not have a block_size column in Oracle 8i.

SELECT S.sid || ',' || S.serial# sid_serial, S.username, S.osuser, P.spid, S.module,
S.program, SUM (T.blocks) * TBS.block_size / 1024 / 1024 mb_used, T.tablespace,
COUNT(*) sort_ops
FROM v$sort_usage T, v$session S, dba_tablespaces TBS, v$process P
WHERE T.session_addr = S.saddr
AND S.paddr = P.addr
AND T.tablespace = TBS.tablespace_name
GROUP BY S.sid, S.serial#, S.username, S.osuser, P.spid, S.module,
S.program, TBS.block_size, T.tablespace
ORDER BY sid_serial;

The query displays information about each database session that is using space in a sort segment, along with the amount of sort space and the temporary tablespace being used, and the number of sort operations in that session that are using sort space. Sample output from this query is as follows:

SID_SERIAL USERNAME OSUSER SPID MODULE PROGRAM MB_USED TABLESPACE SORT_OPS
---------- -------- ------ ---- ------ --------- ------- ---------- --------
33,16998 RPK_APP rpk 3061 inv httpd@db1 9 TEMP 2


This example shows that there is one database session using sort segment space. Session 33 with serial number 16998 is connected to the database as the RPK_APP user. The connection was initiated by the httpd@db1 process running under the rpk operating system user, and the Oracle server process has operating system process ID 3061. The application has identified itself to the database as module “inv.” The session has two active sort operations that are using a total of 9 Mb of sort segment space in the TEMP tablespace.

Sort Space Usage by Statement

The following query displays information about each statement that is using space in a sort segment. This query will need slight modification to run on Oracle 8i databases, since the dba_tablespaces view did not have a block_size column in Oracle 8i.

SELECT S.sid || ',' || S.serial# sid_serial, S.username,
T.blocks * TBS.block_size / 1024 / 1024 mb_used, T.tablespace,
T.sqladdr address, Q.hash_value, Q.sql_text
FROM v$sort_usage T, v$session S, v$sqlarea Q, dba_tablespaces TBS
WHERE T.session_addr = S.saddr
AND T.sqladdr = Q.address (+)
AND T.tablespace = TBS.tablespace_name
ORDER BY S.sid;

The query displays information about each statement using space in a sort segment, including information about the database session that issued the statement and the temporary tablespace and amount of sort space being used. Sample output from this query is as follows:

SID_SERIAL USERNAME MB_USED TABLESPACE ADDRESS HASH_VALUE
---------- -------- ------- ---------- ---------------- ----------
SQL_TEXT
--------------------------------------------------------------------------------
33,16998 RPK_APP 8 TEMP 000000038865B058 3641290170
SELECT * FROM NOTIFY_MESSAGES NM WHERE NM.AWAITING_SENDING = 'y' AND NOT EXISTS
( SELECT 1 FROM NOTIFY_MESSAGE_GROUPS NMG WHERE NMG.MESSAGE_GROUP_ID = NM.MESSAG
E_GROUP_ID AND NMG.INCOMPLETE = 'y' ) ORDER BY NM.NOTIFY_MESSAGE_ID

33,16998 RPK_APP 1 TEMP 00000003839FFE20 1874671316
select * from rpk_stat where sample_group_id = :b1 order by stat#, seq#
This example shows that session 33 with serial number 16998, connected to the database as the RPK_APP user, has two statements currently using sort segment space in the TEMP tablespace. One statement is currently using 8 Mb of sort segment space, while the other is using 1 Mb. The text of each statement, along with its hash value and address in the shared SQL area are also displayed.



SELECT S.sid || ',' || S.serial# sid_serial, S.username, S.osuser, P.spid, S.module,sq.sql_Text,
S.program, SUM (T.blocks) * TBS.block_size / 1024 / 1024 mb_used, T.tablespace,
COUNT(*) sort_ops
FROM v$sort_usage T, v$session S, dba_tablespaces TBS, v$process P, V$sql sq
WHERE T.session_addr = S.saddr
AND S.paddr = P.addr
AND T.tablespace = TBS.tablespace_name
and S.sql_hash_Value = sq.hash_Value and s.sql_address = sq.address
GROUP BY S.sid, S.serial#, S.username, S.osuser, P.spid, S.module,sq.sql_text,
S.program, TBS.block_size, T.tablespace
ORDER BY sid_serial
/





Conclusion

When an operation such as a sort, hash, or global temporary table instantiation is too large to fit in memory, Oracle allocates space in a temporary tablespace for intermediate data to be written to disk. Temporary tablespaces are a shared resource in the database, and you can’t set quotas to limit temporary space used by one session or database user. If a sort operation runs out of space, the statement initiating the sort will fail. It may only take one query missing part of its WHERE clause to fill an entire temporary tablespace and cause many users to encounter failure because the temporary tablespace is full.

It is easy to detect when failures have occurred in the database due to a lack of temporary space. With the setting of a simple diagnostic event, it is also easy to see the exact text of each statement that fails for this reason. There are also v$ views that DBAs can query at any time to monitor temporary tablespace usage in real time. These views make it possible to identify usage at the database, session, and even statement level.

Oracle DBAs can use the techniques outlined in this paper to diagnose temporary tablespace problems and monitor sorting activity in a proactive way. These tactics can be helpful for addressing both chronic and intermittent shortages of temporary space.


Reference :-

http://www.dbspecialists.com/files/presentations/temp_space.html

Thursday, March 24, 2011

Cloning Oracle Home to New Server.

About Cloning oracle home

Cloning is the process of copying an existing Oracle installation to a different location and updating the copied bits to work in the new environment. The changes made by one-off patches applied on the source Oracle home, would also be present after the clone operation. The source and the destination path (host to be cloned) need not be the same. During cloning, OUI replays the actions that were run to install the home. Cloning is similar to installation except that OUI runs the actions in a special mode that is referred to as clone mode. Some situations in which cloning is useful are:

•Creating an installation that is a copy of a production, test, or development installation. Cloning enables you to create a new installation with all patches applied to it in a single step. This is in contrast with going through the installation process by performing separate steps to install, configure, and patch the installation.

•Rapidly deploying an instance and the applications that it hosts.

•Preparing an Oracle home and deploying it to many hosts.

The cloned installation behaves the same as the source installation. For example, the cloned Oracle home can be removed using OUI or patched using OPatch. You can also use the cloned Oracle home as the source for another cloning operation. You can create a cloned copy of a test, development, or production installation by using the command-line cloning scripts. The default cloning procedure is adequate for most usage cases. However, you can also customize various aspects of cloning, for example, to specify custom port assignments, or to preserve custom settings.

The cloning process works by copying all of the files from the source Oracle home to the destination Oracle home. Thus, any files used by the source instance that are located outside the source Oracle home's directory structure are not copied to the destination location.

The size of the binaries at the source and the destination may differ because these are relinked as part of the clone operation and the operating system patch levels may also differ between these two locations. Additionally, the number of files in the cloned home would increase because several files copied from the source, specifically those being instantiated, are backed up as part of the clone operation.

OUI Cloning is more beneficial than using the tarball approach because cloning configures the Central Inventory and the Oracle home inventory in the cloned home. Cloning also makes the home manageable and allows the paths in the cloned home and the target home to be different.

Example :-

We have a oracle 10g Software been installed in the souce server now we have a new server been setup into which we are going to copy the oracle home from the source.

Source Server : 192.168.1.3 Hostname : rhel
Target Server : 192.168.1.11 Hostname : oracln

Setting up the environment in the new server.

a) Create Oralce User

[root@oracln ~]# groupadd oinstall
[root@oracln ~]# groupadd dba
[root@oracln ~]# useradd -g oinstall -G dba oracle
[root@oracln ~]# passwd oracle
Changing password for user oracle.
New UNIX password:
BAD PASSWORD: it is based on a dictionary word
Retype new UNIX password:
passwd: all authentication tokens updated successfully.

b) Create Oracle home directory structure

[root@oracln ~]# mkdir -p /u01/app/oracle/product/10.2.0/db_1
[root@oracln ~]# chown -R oracle:dba /u01
[root@oracln ~]# ls -l

c) Setup the Kernel Parameters.

[root@oracln ~]# vi /etc/sysctl.conf

kernel.shmmax = 2147483648
kernel.shmmni = 4096
kernel.shmall = 2097152
kernel.sem = 250 32000 100 128
fs.file-max = 65536
net.ipv4.ip_local_port_range = 1024 65000
net.core.rmem_default = 262144
net.core.wmem_default = 262144
net.core.rmem_max = 262144
net.core.wmem_max = 262144

d) Add the Oracle user lime in the profile file.

[root@oracln ~]# vi /etc/profile

if [ $USER = "oracle" ]; then
if [ $SHELL = "/bin/ksh" ]; then
ulimit -p 16384
ulimit -n 65536
else
ulimit -u 16384 -n 65536
fi
umask 022
fi

e) Set up the Security limits for oracle User.

[root@oracln ~]# vi /etc/security/limits.conf

oracle soft nofile 65536
oracle hard nofile 65536
oracle soft nproc 16384
oracle hard nproc 16384

f) Set up the environment variables for the oracle user.

[root@oracln ~]# su - oracle

[oracle@oracln ~]$ vi .bash_profile

export ORACLE_BASE=/u01/app/oracle
export ORACLE_HOME=$ORACLE_BASE/product/10.2.0/db_1
export ORACLE_TERM=Xterm
export PATH=$ORACLE_HOME/bin:$PATH
export ORACLE_OWNER=oracle
export LD_LIBRARY_PATH=$ORACLE_HOME/lib
CLASSPATH=$ORACLE_HOME/JRE:$ORACLE_HOME/jlib:$ORACLE_HOME/rdbms/jlib
CLASSPATH=$CLASSPATH:$ORACLE_HOME/network/jlib; export CLASSPATH
export TMP=/tmp
export TMPDIR=$TMP
export TNS_ADMIN=$ORACLE_HOME/network/admin

Configure the below steps to setup ssh , if ssh is already been setup then
skip the below and go to "Steps after ssh been confgured"

g) Add the Source and Target hosts details in the /etc/hosts file.

[root@oracln ~]# vi /etc/hosts

127.0.0.1 localhost.localdomain localhost
192.168.1.11 oracln.manzoor.com oracln
192.168.1.3 rhel.manzoor.com rhel

h) Generate the rsa and dsa keys for ssh.

oracle@oracln ~]$ pwd
/home/oracle
[oracle@oracln ~]$ mkdir .ssh
[oracle@oracln ~]$ chmod 700 .ssh
[oracle@oracln ~]$ cd .ssh
[oracle@oracln .ssh]$ ssh-keygen -t rsa
Generating public/private rsa key pair.
Enter file in which to save the key (/home/oracle/.ssh/id_rsa):
Enter passphrase (empty for no passphrase):
Enter same passphrase again:
Your identification has been saved in /home/oracle/.ssh/id_rsa.
Your public key has been saved in /home/oracle/.ssh/id_rsa.pub.
The key fingerprint is:
57:af:54:e5:14:b4:14:c8:3b:93:7f:e8:6f:a4:e9:8d oracle@oracln.manzoor.com
[oracle@oracln .ssh]$ ssh-keygen -t dsa
Generating public/private dsa key pair.
Enter file in which to save the key (/home/oracle/.ssh/id_dsa):
Enter passphrase (empty for no passphrase):
Enter same passphrase again:
Your identification has been saved in /home/oracle/.ssh/id_dsa.
Your public key has been saved in /home/oracle/.ssh/id_dsa.pub.
The key fingerprint is:
43:54:c2:28:16:c6:0f:42:78:52:b5:9b:a1:d9:48:6d oracle@oracln.manzoor.com




i) Now log on to the source server and copy the contents of the rsa and dsa keys of target server and append it to the authorized keys file and then copy the same to the new Server.

[root@rhel] # vi /etc/hosts

127.0.0.1 localhost.localdomain localhost
192.168.1.11 oracln.manzoor.com oracln
192.168.1.3 rhel.manzoor.com rhel

[root@rhel] # su - oracle
[oracle@rhel ~]$ pwd
/home/oracle
[oracle@rhel ~]$ mkdir .ssh
[oracle@rhel ~]$ cd .ssh
[oracle@rhel .ssh]$ cd ..
[oracle@rhel ~]$ chmod 700 .ssh
[oracle@rhel ~]$ cd .ssh
[oracle@rhel .ssh]$ ls -l
total 0
[oracle@rhel .ssh]$ ssh-keygen -t rsa
[oracle@rhel .ssh]$ ssh-keygen -t dsa
[oracle@rhel .ssh]$ cat id_rsa.pub >> authorized_keys
[oracle@rhel .ssh]$ cat id_dsa.pub >> authorized_keys
[oracle@rhel .ssh]$ ssh oracln cat /home/oracle/.ssh/id_rsa.pub >> authorized_keys
oracle@oracln's password:
[oracle@rhel .ssh]$ ssh oracln cat /home/oracle/.ssh/id_dsa.pub >> authorized_keys


j) Now copy the authorized keys file to the New server.

[oracle@rhel .ssh]$ scp authorized_keys oracln:/home/oracle/.ssh/
oracle@oracln's password:
authorized_keys 100% 2544 2.5KB/s 00:00



Steps after ssh been confgured.

1) On Source Server.

Check the Oracle home to clone.

[oracle@rhel Opatch]$ opatch lsinv
Invoking OPatch 10.2.0.1.0

Oracle interim Patch Installer version 10.2.0.1.0
Copyright (c) 2005, Oracle Corporation. All rights reserved..


Oracle Home : /u01/app/oracle/product/10.2.0/db_1
Central Inventory : /u01/app/oracle/oraInventory
from : /u01/app/oracle/product/10.2.0/db_1/oraInst.loc
OPatch version : 10.2.0.1.0
OUI version : 10.2.0.1.0
OUI location : /u01/app/oracle/product/10.2.0/db_1/oui
Log file location : /u01/app/oracle/product/10.2.0/db_1/cfgtoollogs/opatch/opatch-2011_Mar_25_08-28-27-IST_Fri.log

Lsinventory Output file location : /u01/app/oracle/product/10.2.0/db_1/cfgtoollogs/opatch/lsinv/lsinventory-2011_Mar_25_08-28-27-IST_Fri.txt

--------------------------------------------------------------------------------
Installed Top-level Products (1):

Oracle Database 10g 10.2.0.1.0
There are 1 products installed in this Oracle Home.


There are no Interim patches installed in this Oracle Home.

--------------------------------------------------------------------------------

OPatch succeeded.

2) Check the Size of the Oracle Home to be cloned.

[oracle@rhel ~]$ cd /u01/app/oracle/product/10.2.0/db_1
[oracle@rhel db_1]$
[oracle@rhel db_1]$ du -h . | tail -1
1.4G

3) Execute the tar command to tape archive the oracle home and trasfer the contents over the network to the new server.

[oracle@rhel db_1]$ tar cvf - . | ( ssh oracln "cd /u01/app/oracle/product/10.2.0/db_1 ; tar xvf -" )

(or)

[oracle@rhel ~]$ cd /u01/app/oracle/product/10.2.0/
[oracle@rhel ~]$ tar -cvf /tmp/10.2.0_bin.tar db_1

Secure copy to the remote server.

$ scp /tmp/10.2.0_bin.tar oracle@cloned-server/u01/app/oracle/product/10.2.0/db_1/.

Login to the cloned server using ssh or telnet …
# ssh –l oracle cloned-server
# cd /u01/app/oracle/product/10.2.0/db_1/

Untar the file.
# tar –xvf 10.2.0_bin.tar



The above tar command will archive the oracle home and tranfer it over the network to the new server and un compress it in your new server. This process will run for 30-45 minutes depending upon the size of the oracle home and the network speed.

4) Once the copy is completed check the size of oracle home in the target server, the size should match with the size of source oracle home.

[oracle@oracln ~]$ cd /u01/app/oracle/product/10.2.0/db_1
[oracle@oracln db_1]$
[oracle@oracln db_1]$ du -h . | tail -1
1.4G

5) Complete the Cloning Operation using the OUI in the target server.

[oracle@oracln ~]$ cd $ORACLE_HOME/oui/bin
[oracle@oracln bin]$ ./runInstaller -clone -silent -ignorePreReq ORACLE_HOME="/u01/app/oracle/product/10.2.0/db_1" ORACLE_HOME_NAME="Ora10gHome" ORACLE_BASE="/u01/app/oracle" OSDBA_GROUP=dba OSOPER_GROUP=dba
Starting Oracle Universal Installer...

No pre-requisite checks found in oraparam.ini, no system pre-requisite checks will be executed.
Preparing to launch Oracle Universal Installer from /tmp/OraInstall2011-03-25_08-10-36AM. Please wait ...[oracle@oracln bin]$ Oracle Universal Installer, Version 10.2.0.1.0 Production
Copyright (C) 1999, 2005, Oracle. All rights reserved.

You can find a log of this install session at:
/u01/app/oracle/oraInventory/logs/cloneActions2011-03-25_08-10-36AM.log
.................................................................................................... 100% Done.

Installation in progress (Fri Mar 25 08:11:26 IST 2011)
......................................................................... 73% Done.
Install successful

Linking in progress (Fri Mar 25 08:11:44 IST 2011)
. 74% Done.
Link successful

Setup in progress (Fri Mar 25 08:14:01 IST 2011)
.................... 100% Done.
Setup successful

End of install phases.(Fri Mar 25 08:14:03 IST 2011)
WARNING:A new inventory has been created in this session. However, it has not yet been registered as the central inventory of this system.
To register the new inventory please run the script '/u01/app/oracle/oraInventory/orainstRoot.sh' with root privileges.
If you do not register the inventory, you may not be able to update or patch the products you installed.

The following configuration scripts
/u01/app/oracle/product/10.2.0/db_1/root.sh
need to be executed as root for configuring the system.

The cloning of Ora10gHome was successful.
Please check '/u01/app/oracle/oraInventory/logs/cloneActions2011-03-25_08-10-36AM.log' for more details.

(or)

$ cd /u01/app/oracle/product/10.2.0/db_1/clone/bin
$ perl clone.pl ORACLE_HOME="/u01/app/oracle/product/10.2.0/db_1" ORACLE_HOME_NAME="Ora10gHome" ORACLE_BASE="/u01/app/oracle" OSDBA_GROUP=dba OSOPER_GROUP=dba
===========
./runInstaller -silent -clone -waitForCompletion "ORACLE_HOME=/u01/app/oracle/product/10.2.0/db_1" "ORACLE_HOME_NAME=Ora10ghome" -noConfig -nowait
Starting Oracle Universal Installer...

No pre-requisite checks found in oraparam.ini, no system pre-requisite checks will be executed.
Preparing to launch Oracle Universal Installer from /tmp/OraInstall2012-06-25_11-03-17PM. Please wait ...Oracle Universal Installer, Version 10.2.0.4.0 Production
Copyright (C) 1999, 2008, Oracle. All rights reserved.

You can find a log of this install session at:
/optware/oracle/oraInventory/logs/cloneActions2012-06-25_11-03-17PM.log
.................................................................................................... 100% Done.



Installation in progress (Monday, June 25, 2012 11:03:58 PM PDT)
........................................................................ 72% Done.
Install successful

Linking in progress (Monday, June 25, 2012 11:04:15 PM PDT)
Link successful

Setup in progress (Monday, June 25, 2012 11:04:22 PM PDT)
Setup successful

End of install phases.(Monday, June 25, 2012 11:04:25 PM PDT)
WARNING:
The following configuration scripts need to be executed as the "root" user.
#!/bin/sh
#Root script to run
/u01/app/oracle/product/10.2.0/db_1/root.sh
To execute the configuration scripts:
1. Open a terminal window
2. Log in as "root"
3. Run the scripts






6) Once done execute the configuration scripts as the root user.

[root@oracln ~]# sh /u01/app/oracle/oraInventory/orainstRoot.sh
Changing permissions of /u01/app/oracle/oraInventory to 770.
Changing groupname of /u01/app/oracle/oraInventory to oinstall.
The execution of the script is complete


[root@oracln ~]# sh /u01/app/oracle/product/10.2.0/db_1/root.sh
Running Oracle10 root.sh script...

The following environment variables are set as:
ORACLE_OWNER= oracle
ORACLE_HOME= /u01/app/oracle/product/10.2.0/db_1

Enter the full pathname of the local bin directory: [/usr/local/bin]:
Copying dbhome to /usr/local/bin ...
Copying oraenv to /usr/local/bin ...
Copying coraenv to /usr/local/bin ...


Creating /etc/oratab file...
Entries will be added to the /etc/oratab file as needed by
Database Configuration Assistant when a database is created
Finished running generic part of root.sh script.
Now product-specific root actions will be performed.

7) Now Check the patch details using the opatch tool, it should match with the souce server.

[oracle@oracln OPatch]$ ./opatch lsinventory
Invoking OPatch 10.2.0.1.0

Oracle interim Patch Installer version 10.2.0.1.0
Copyright (c) 2005, Oracle Corporation. All rights reserved..


Oracle Home : /u01/app/oracle/product/10.2.0/db_1
Central Inventory : /u01/app/oracle/oraInventory
from : /u01/app/oracle/product/10.2.0/db_1/oraInst.loc
OPatch version : 10.2.0.1.0
OUI version : 10.2.0.1.0
OUI location : /u01/app/oracle/product/10.2.0/db_1/oui
Log file location : /u01/app/oracle/product/10.2.0/db_1/cfgtoollogs/opatch/opatch-2011_Mar_25_08-28-27-IST_Fri.log

Lsinventory Output file location : /u01/app/oracle/product/10.2.0/db_1/cfgtoollogs/opatch/lsinv/lsinventory-2011_Mar_25_08-28-27-IST_Fri.txt

--------------------------------------------------------------------------------
Installed Top-level Products (1):

Oracle Database 10g 10.2.0.1.0
There are 1 products installed in this Oracle Home.


There are no Interim patches installed in this Oracle Home.

--------------------------------------------------------------------------------

OPatch succeeded.


9) Oracle home has been successfully cloned to the new server.


Refer for more details:-

Oracle® Universal Installer and OPatch User's Guide
10g Release 2 (10.2) for Windows and UNIX
Part Number B16227-12

Thursday, March 3, 2011

DBA Related Shell Scripts

---------------------------------------------------------------------
Script to remove the logs
a) Remove archivelogs which are older than 3 days.
b) Remove logs from db admin directory which are older than 14 days
---------------------------------------------------------------------

#!/bin/ksh

# Cleanup PROD archive logs more than 3 days old
/usr/bin/find /u01/app/oracle/admin/PROD/arch/arch_prod*.arc -ctime +3 -exec rm {} \;

# Cleanup trace and dump files over 14 days old
for ORACLE_SID in `cat /etc/oratab|egrep ':N|:Y'|grep -v \*|cut -f1 -d':'`
do
ORACLE_HOME=`cat /etc/oratab|grep ^$ORACLE_SID:|cut -d":" -f2`
DBA=`echo $ORACLE_HOME | sed -e 's:/product/.*::g'`/admin
/usr/bin/find $DBA/$ORACLE_SID/bdump -name \*.trc -mtime +14 -exec rm {} \;
/usr/bin/find $DBA/$ORACLE_SID/udump -name \*.trc -mtime +14 -exec rm {} \;
/usr/bin/find $ORACLE_HOME/rdbms/audit -name \*.aud -mtime +14 -exec rm {} \;
done

-----------------------------------------------------------------------------------

Scripts by Daniel T. Liu
----------------------------------------------------------------------------------

Introduction
This article focuses on the DBA's daily responsibilities for monitoring Oracle databases and provides tips and techniques on how DBAs can turn their manual, reactive monitoring activities into a set of proactive shell scripts. The article first reviews some commonly used Unix commands by DBAs. It explains the Unix Cron jobs that are used as part of the scheduling mechanism to execute DBA scripts. The article covers eight important scripts for monitoring Oracle database:

Check instance availability

Check listener availability

Check alert log files for error messages

Clean up old log files before log destination gets filled

Analyze tables and indexes for better performance

Check tablespace usage

Find out invalid objects

Monitor users and transactions

UNIX Basics for the DBA
Basic UNIX Command

The following is a list of commonly used Unix command:

ps - Show process

grep - Search files for text patterns

mailx - Read or send mail

cat - Join files or display them

cut - Select columns for display

awk - Pattern-matching language

df - Show free disk space

Here are some examples of how the DBA uses these commands:


List available instances on a server:

$ ps -ef | grep smon
oracle 21832 1 0 Feb 24 ? 19:05 ora_smon_oradb1
oracle 898 1 0 Feb 15 ? 0:00 ora_smon_oradb2
dliu 25199 19038 0 10:48:57 pts/6 0:00 grep smon
oracle 27798 1 0 05:43:54 ? 0:00 ora_smon_oradb3
oracle 28781 1 0 Mar 03 ? 0:01 ora_smon_oradb4


List available listeners on a server:

$ ps -ef | grep listener | grep -v grep
oracle 23879 1 0 Feb 24 ? 33:36 /8.1.7/bin/tnslsnr listener_db1 -inherit
oracle 27939 1 0 05:44:02 ? 0:00 /8.1.7/bin/tnslsnr listener_db2 -inherit
oracle 23536 1 0 Feb 12 ? 4:19 /8.1.7/bin/tnslsnr listener_db3 -inherit
oracle 28891 1 0 Mar 03 ? 0:01 /8.1.7/bin/tnslsnr listener_db4 -inherit


Find out file system usage for Oracle archive destination:

$ df -k | grep oraarch
/dev/vx/dsk/proddg/oraarch 71123968 4754872 65850768 7% /u09/oraarch


List number of lines in the alert.log file:

$ cat alert.log | wc -l
2984


List all Oracle error messages from the alert.log file:

$ grep ORA- alert.log
ORA-00600: internal error code, arguments: [kcrrrfswda.1], [], [], [], [], []
ORA-00600: internal error code, arguments: [1881], [25860496], [25857716], []

CRONTAB Basics
A crontab file is comprised of six fields:

Minute 0-59
Hour 0-23
Day of month 1-31
Month 1 - 12
Day of Week 0 - 6, with 0 = Sunday
Unix Command or Shell Scripts


To edit a crontab file, type:

Crontab -e


To view a crontab file, type:

Crontab -l
0 4 * * 5 /dba/admin/analyze_table.ksh
30 3 * * 3,6 /dba/admin/hotbackup.ksh /dev/null 2>&1


In the example above, the first entry shows that a script to analyze a table runs every Friday at 4:00 a.m. The second entry shows that a script to perform a hot backup runs every Wednesday and Saturday at 3:00 a.m.

Top DBA Shell Scripts for Monitoring the Database
The eight shell scripts provided below cover 90 percent of a DBA's daily monitoring activities. You will need to modify the UNIX environment variables as appropriate.

Check Oracle Instance Availability

The oratab file lists all the databases on a server:

$ cat /var/opt/oracle/oratab
###################################################################
## /var/opt/oracle/oratab ##
###################################################################
oradb1:/u01/app/oracle/product/8.1.7:Y
oradb2:/u01/app/oracle/product/8.1.7:Y
oradb3:/u01/app/oracle/product/8.1.7:N
oradb4:/u01/app/oracle/product/8.1.7:Y


The following script checks all the databases listed in the oratab file, and finds out the status (up or down) of databases:

###################################################################
## ckinstance.ksh ##
###################################################################
ORATAB=/var/opt/oracle/oratab
echo "`date` "
echo "Oracle Database(s) Status `hostname` :\n"


db=`egrep -i ":Y|:N" $ORATAB | cut -d":" -f1 | grep -v "\#" | grep -v "\*"`
pslist="`ps -ef | grep pmon`"
for i in $db ; do
echo "$pslist" | grep "ora_pmon_$i" > /dev/null 2>$1
if (( $? )); then
echo "Oracle Instance - $i: Down"
else
echo "Oracle Instance - $i: Up"
fi
done


Use the following to make sure the script is executable:

$ chmod 744 ckinstance.ksh
$ ls -l ckinstance.ksh
-rwxr--r-- 1 oracle dba 657 Mar 5 22:59 ckinstance.ksh*

Here is an instance availability report:

$ ckinstance.ksh
Mon Mar 4 10:44:12 PST 2002

Oracle Database(s) Status for DBHOST server:
Oracle Instance - oradb1: Up
Oracle Instance - oradb2: Up
Oracle Instance - oradb3: Down
Oracle Instance - oradb4: Up

Check Oracle Listener's Availability
A similar script checks for the Oracle listener. If the listener is down, the script will restart the listener:

#######################################################################
## cklsnr.sh ##
#######################################################################
#!/bin/ksh
DBALIST="primary.dba@company.com,another.dba@company.com";export DBALIST
cd /var/opt/oracle
rm -f lsnr.exist
ps -ef | grep mylsnr | grep -v grep > lsnr.exist
if [ -s lsnr.exist ]
then
echo
else
echo "Alert" | mailx -s "Listener 'mylsnr' on `hostname` is down" $DBALIST
TNS_ADMIN=/var/opt/oracle; export TNS_ADMIN
ORACLE_SID=db1; export ORACLE_SID
ORAENV_ASK=NO; export ORAENV_ASK
PATH=$PATH:/bin:/usr/local/bin; export PATH
. oraenv
LD_LIBRARY_PATH=${ORACLE_HOME}/lib;export LD_LIBRARY_PATH
lsnrctl start mylsnr
fi

Check Alert Logs (ORA-XXXXX)
Some of the environment variables used by each script can be put into one profile:

#######################################################################
## oracle.profile ##
#######################################################################
EDITOR=vi;export EDITOR ORACLE_BASE=/u01/app/oracle; export
ORACLE_BASE ORACLE_HOME=$ORACLE_BASE/product/8.1.7; export
ORACLE_HOME LD_LIBRARY_PATH=$ORACLE_HOME/lib; export
LD_LIBRARY_PATH TNS_ADMIN=/var/opt/oracle;export
TNS_ADMIN NLS_LANG=american; export
NLS_LANG NLS_DATE_FORMAT='Mon DD YYYY HH24:MI:SS'; export
NLS_DATE_FORMAT ORATAB=/var/opt/oracle/oratab;export
ORATAB PATH=$PATH:$ORACLE_HOME:$ORACLE_HOME/bin:/usr/ccs/bin:/bin:/usr/bin:/usr/sbin:/
sbin:/usr/openwin/bin:/opt/bin:.; export
PATH DBALIST="primary.dba@company.com,another.dba@company.com";export
DBALIST

The following script first calls oracle.profile to set up all the environment variables. The script also sends the DBA a warning e-mail if it finds any Oracle errors:

####################################################################
## ckalertlog.sh ##
####################################################################
#!/bin/ksh
. /etc/oracle.profile
for SID in `cat $ORACLE_HOME/sidlist`
do
cd $ORACLE_BASE/admin/$SID/bdump
if [ -f alert_${SID}.log ]
then
mv alert_${SID}.log alert_work.log
touch alert_${SID}.log
cat alert_work.log >> alert_${SID}.hist
grep ORA- alert_work.log > alert.err
fi
if [ `cat alert.err|wc -l` -gt 0 ]
then
mailx -s "${SID} ORACLE ALERT ERRORS" $DBALIST < alert.err
fi
rm -f alert.err
rm -f alert_work.log
done

Clean Up Old Archived Logs
The following script cleans up old archive logs if the log file system reaches 90-percent capacity:

$ df -k | grep arch
Filesystem kbytes used avail capacity Mounted on
/dev/vx/dsk/proddg/archive 71123968 30210248 40594232 43% /u08/archive


#######################################################################
## clean_arch.ksh ##
#######################################################################
#!/bin/ksh
df -k | grep arch > dfk.result
archive_filesystem=`awk -F" " '{ print $6 }' dfk.result`
archive_capacity=`awk -F" " '{ print $5 }' dfk.result`



if [[ $archive_capacity > 90% ] ]
then
echo "Filesystem ${archive_filesystem} is ${archive_capacity} filled"
# try one of the following option depend on your need
find $archive_filesystem -type f -mtime +2 -exec rm -r {} \;
tar
rman
fi

Analyze Tables and Indexes (for Better Performance)
Below, I have shown an example on how to pass parameters to a script:

####################################################################
## analyze_table.sh ##
####################################################################
#!/bin/ksh #
input parameter: 1: password # 2: SID if (($#<1 br="br" echo="echo" enter="enter" lease="lease" then="then">'oracle'
user password as the first parameter !" exit 0 fi if (($#<2 br="br" echo="echo" then="then">"Please enter
instance name as the second parameter!" exit 0 fi

To execute the script with parameters, type:

$ analyze_table.sh manager oradb1


The first part of script generates a file analyze.sql, which contains the syntax for analyzing table. The second part of script analyzes all the tables:

#####################################################################
## analyze_table.sh ##
#####################################################################
sqlplus -s <<!
oracle/$1@$2
set heading off
set feed off
set pagesize 200
set linesize 100
spool analyze_table.sql
select 'ANALYZE TABLE ' || owner || '.' || segment_name ||
' ESTIMATE STATISTICS SAMPLE 10 PERCENT;'
from dba_segments
where segment_type = 'TABLE'
and owner not in ('SYS', 'SYSTEM');
spool off
exit
!
sqlplus -s <<!
oracle/$1@$2
@./analyze_table.sql
exit
!


Here is an example of analyze.sql:

$ cat analyze.sql
ANALYZE TABLE HIRWIN.JANUSAGE_SUMMARY ESTIMATE STATISTICS SAMPLE 10 PERCENT;
ANALYZE TABLE HIRWIN.JANUSER_PROFILE ESTIMATE STATISTICS SAMPLE 10 PERCENT;
ANALYZE TABLE APPSSYS.HIST_SYSTEM_ACTIVITY ESTIMATE STATISTICS SAMPLE 10 PERCENT;
ANALYZE TABLE HTOMEH.QUEST_IM_VERSION ESTIMATE STATISTICS SAMPLE 10 PERCENT;
ANALYZE TABLE JSTENZEL.HIST_SYS_ACT_0615 ESTIMATE STATISTICS SAMPLE 10 PERCENT;
ANALYZE TABLE JSTENZEL.HISTORY_SYSTEM_0614 ESTIMATE STATISTICS SAMPLE 10 PERCENT;
ANALYZE TABLE JSTENZEL.CALC_SUMMARY3 ESTIMATE STATISTICS SAMPLE 10 PERCENT;
ANALYZE TABLE IMON.QUEST_IM_LOCK_TREE ESTIMATE STATISTICS SAMPLE 10 PERCENT;
ANALYZE TABLE APPSSYS.HIST_USAGE_SUMMARY ESTIMATE STATISTICS SAMPLE 10 PERCENT;
ANALYZE TABLE PATROL.P$LOCKCONFLICTTX ESTIMATE STATISTICS SAMPLE 10 PERCENT;

Check Tablespace Usage
This scripts checks for tablespace usage. If tablespace is 10 percent free, it will send an alert e-mail.

#####################################################################
## ck_tbsp.sh ##
#####################################################################
#!/bin/ksh
sqlplus -s <<!
oracle/$1@$2
set feed off
set linesize 100
set pagesize 200
spool tablespace.alert
SELECT F.TABLESPACE_NAME,
TO_CHAR ((T.TOTAL_SPACE - F.FREE_SPACE),'999,999') "USED (MB)",
TO_CHAR (F.FREE_SPACE, '999,999') "FREE (MB)",
TO_CHAR (T.TOTAL_SPACE, '999,999') "TOTAL (MB)",
TO_CHAR ((ROUND ((F.FREE_SPACE/T.TOTAL_SPACE)*100)),'999')||' %' PER_FREE
FROM (
SELECT TABLESPACE_NAME,
ROUND (SUM (BLOCKS*(SELECT VALUE/1024
FROM V\$PARAMETER
WHERE NAME = 'db_block_size')/1024)
) FREE_SPACE
FROM DBA_FREE_SPACE
GROUP BY TABLESPACE_NAME
) F,
(
SELECT TABLESPACE_NAME,
ROUND (SUM (BYTES/1048576)) TOTAL_SPACE
FROM DBA_DATA_FILES
GROUP BY TABLESPACE_NAME
) T
WHERE F.TABLESPACE_NAME = T.TABLESPACE_NAME
AND (ROUND ((F.FREE_SPACE/T.TOTAL_SPACE)*100)) < 10;
spool off
exit
!
if [ `cat tablespace.alert|wc -l` -gt 0 ]
then
cat tablespace.alert -l tablespace.alert > tablespace.tmp
mailx -s "TABLESPACE ALERT for ${2}" $DBALIST < tablespace.tmp
fi


An example of the alert mail output is as follows:

TABLESPACE_NAME USED (MB) FREE (MB) TOTAL (MB) PER_FREE
------------------- --------- ----------- ------------------- ------------------
SYSTEM 2,047 203 2,250 9 %
STBS01 302 25 327 8 %
STBS02 241 11 252 4 %
STBS03 233 19 252 8 %

Find Out Invalid Database Objects
The following finds out invalid database objects:

#####################################################################
## invalid_object_alert.sh ##
#####################################################################
#!/bin/ksh
. /etc/oracle.profile
sqlplus -s <<!
oracle/$1@$2
set feed off
set heading off
column object_name format a30
spool invalid_object.alert
SELECT OWNER, OBJECT_NAME, OBJECT_TYPE, STATUS
FROM DBA_OBJECTS
WHERE STATUS = 'INVALID'
ORDER BY OWNER, OBJECT_TYPE, OBJECT_NAME;
spool off
exit
!
if [ `cat invalid_object.alert|wc -l` -gt 0 ]
then
mailx -s "INVALID OBJECTS for ${2}" $DBALIST < invalid_object.alert
fi
$ cat invalid_object.alert


OWNER OBJECT_NAME OBJECT_TYPE STATUS
----------------------------------------------------------------------
HTOMEH DBMS_SHARED_POOL PACKAGE BODY INVALID
HTOMEH X_$KCBFWAIT VIEW INVALID
IMON IW_MON PACKAGE INVALID
IMON IW_MON PACKAGE BODY INVALID
IMON IW_ARCHIVED_LOG VIEW INVALID
IMON IW_FILESTAT VIEW INVALID
IMON IW_SQL_FULL_TEXT VIEW INVALID
IMON IW_SYSTEM_EVENT1 VIEW INVALID
IMON IW_SYSTEM_EVENT_CAT VIEW INVALID
LBAILEY CHECK_TABLESPACE_USAGE PROCEDURE INVALID
PATROL P$AUTO_EXTEND_TBSP VIEW INVALID
SYS DBMS_CRYPTO_TOOLKIT PACKAGE INVALID
SYS DBMS_CRYPTO_TOOLKIT PACKAGE BODY INVALID
SYS UPGRADE_SYSTEM_TYPES_TO_816 PROCEDURE INVALID
SYS AQ$_DEQUEUE_HISTORY_T TYPE INVALID
SYS HS_CLASS_CAPS VIEW INVALID
SYS HS_CLASS_DD VIEW INVALID

Monitor Users and Transactions (Dead Locks, et al)
This script sends out an alert e-mail if dead lock occurs:

###################################################################
## deadlock_alert.sh ##
###################################################################
#!/bin/ksh
. /etc/oracle.profile
sqlplus -s <<!
oracle/$1@$2
set feed off
set heading off
spool deadlock.alert
SELECT SID, DECODE(BLOCK, 0, 'NO', 'YES' ) BLOCKER,
DECODE(REQUEST, 0, 'NO','YES' ) WAITER
FROM V$LOCK
WHERE REQUEST > 0 OR BLOCK > 0
ORDER BY block DESC;
spool off
exit
!
if [ `cat deadlock.alert|wc -l` -gt 0 ]
then
mailx -s "DEADLOCK ALERT for ${2}" $DBALIST < deadlock.alert
fi

Conclusion

0,20,40 7-17 * * 1-5 /dba/scripts/ckinstance.sh > /dev/null 2>&1
0,20,40 7-17 * * 1-5 /dba/scripts/cklsnr.sh > /dev/null 2>&1
0,20,40 7-17 * * 1-5 /dba/scripts/ckalertlog.sh > /dev/null 2>&1
30 * * * 0-6 /dba/scripts/clean_arch.sh > /dev/null 2>&1
* 5 * * 1,3 /dba/scripts/analyze_table.sh > /dev/null 2>&1
* 5 * * 0-6 /dba/scripts/ck_tbsp.sh > /dev/null 2>&1
* 5 * * 0-6 /dba/scripts/invalid_object_alert.sh > /dev/null 2>&1
0,20,40 7-17 * * 1-5 /dba/scripts/deadlock_alert.sh > /dev/null 2>&1

Now my DBA friends, you can have more uninterrupted sleep at night. You may also have time for more important things such as performance tuning.

References
Unix in a Nutshell, O'Reilly & Associates, Inc.;
“Using Oracle9i Application Server to Build Your Web-Based Database Monitoring Tool,” Daniel T. Liu; Select Magazine - November 2001 Volume 8, No. 1;
“Net8: A Step-by-Step Setup of Oracle Names Server,” Daniel T. Liu; Oracle Open World 2000, Paper#271.
I would also like to acknowledge the assistance of Johnny Wedekind of ADP, Ann Collins, Larry Bailey, Husam Tomeh and Archana Sharma of FARES.

--

Daniel Liu is a senior Oracle Database Administrator at First American Real Estate Solutions in Anaheim, CA, and co-author of Oracle Database 10g New Features. His expertise includes Oracle database administration, performance tuning, Oracle networking, and Oracle Application Server. As an Oracle Certified Professional, he taught Oracle certified DBA classes and IOUG University Seminar. Daniel has published articles with DBAzine, Oracle Internals, and SELECT Journal. Daniel holds a Master of Science degree in computer science from Northern Illinois University.

-----------------------------------------------------------------------------
-----------------------------------------------------------------------------

cpu Usage:-

#!/bin/sh

ALERTNAME="/opt/MonitoringScript/cpumem.txt"
Date=`date +'%Y%m%d-%H:%M'`
host=`hostname`
Usage=`prstat -Z 1 3|grep -i $host |tail -1|awk '{print "MEMORY:",$5,"CPU:",$7}'`
echo " $Usage $Date ">> alertlog

##cat alertlog|tail -1|mailx -s "CPU and MEMORY USAGE `hostname`" -c "user@company.com" "user@company.com"
cat alertlog|tail -1|mailx -s "CPU and MEMORY USAGE `hostname`" "user@company.com"



Disk usage:-

#!/bin/ksh

LOGNAME="/opt/MonitoringScript/alert.txt"
host_name=`hostname`
for mou in /u01 /u05 /u02
do
disk_space=`df -k $mou | grep $mou | tr -s " " | cut -d" " -f5 | tr -d "%"`

if [ $disk_space -ge 75 ]
then
echo " $mou on $host_name - ( $disk_space"%" ) - `date '+%d%m%Y - %H:%M'`\n" >> $LOGNAME
fi

done

if [ -f $LOGNAME ]
then
cat $LOGNAME | mailx -s "TADDS DISK ALERT" "username@company.com"
rm $LOGNAME
fi


Long running queries :-

/dboracle/orabase/product/10.2.0.3/bin/sqlplus -s user/pass@db << !END > $path 2>&1
set termout off;
set serveroutput on;
select sql_text, round((elapsed_time/executions)/1000000,2) "Time_Taken in secs",sql_id, child_number from
v\$sql where executions > 0;
exit;
sql_end

cat $path | mailx -s 'Alert: TEST' "user@company.com"
EOF

Tablesapce usage script:-
#!/bin/sh
###This script is used to find the Free tablespace less than 40% and e-mail to ASG group e-mail id
################################################

path="/var/TADDSTABLESPACE.log"

echo "================================================================================== " > $path

ORACLE_BASE=/dboracle/orabase/product
export ORACLE_BASE

ORACLE_HOME=/dboracle/orabase/product/10.2.0.3
export ORACLE_HOME

ORACLE_SID=tadds
export ORACLE_SID

TNS_ADMIN=/dborafiles/orabase/admin/netadmin
export TNS_ADMIN


/dboracle/orabase/product/10.2.0.3/bin/sqlplus -s user/pas@db << !END >> $path 2>&1

show user
select tablespace_name, PCT_FREE
from (SELECT sysdate, Total.tablespace_name Tablespace_Name,
Dt.EXTENT_MANAGEMENT,
total_space total_Size_MB,
round(nvl(total_space-free_space,0),2) used_MB,
round(nvl(free_space,0),2) free_MB,
round(nvl(total_space-free_space,0)/total_space*100,2) pct_used,
round(nvl(free_space,0)/total_space*100,2) pct_free,
round(Maxsize,2) Maxsize_MB
FROM
(select tablespace_name,
sum(bytes/1024/1024) Free_Space
from sys.dba_free_space
group by tablespace_name
) Free,
(select TABLESPACE_NAME, sum(MAXBYTES)/1024/1024 Maxsize,
sum(bytes)/1024/1024 Total_space
From dba_data_files
group by tablespace_name ) Total,
(select TABLESPACE_NAME,
EXTENT_MANAGEMENT
from dba_tablespaces
order by TABLESPACE_NAME) dt
WHERE Free.Tablespace_name(+) = Total.tablespace_name
and total.Tablespace_name = dt. TABLESPACE_NAME
ORDER BY Total.tablespace_name,dt.tablespace_name) WHERE PCT_FREE < 40
ORDER BY PCT_FREE;
exit
!END

cd

counter=`cat $path | sed '/^$/d'| wc -l`
echo "count="$counter
if [ $counter -gt 0 ]
then
cat $path | mailx -s 'Alert: tablespace free PCT' "user@compnay.com"
rm $path
fi

=========================================================

Script for generating AWR report.


#!/bin/ksh
# For generating Awr report
#
#
########## Getting the input ###############
export ORACLE_SID=layatest
export ORACLE_HOME=$ORACLE_HOME
echo "Pls enter the no. of reports to generate : "
read n
k=1
i=1
while [ $i -le $n ]
do
               echo "Enter the Start Time for $i report :"
               read sttime
               echo "Enter the End   Time for $i report :"
               read endtime
               a[$k]=$sttime:$endtime
               i=`expr $i + 1`
               k=`expr $k + 1`
done
i=1
conndb()
{
sqlplus -s "/ as sysdba" < awr_pre_report_$i.sql
set trims off;
set feedback off;
set serveroutput on;
declare
        dbid  number;
        bgnsp number;
        endsp number;
        begin
                select dbid into dbid from dba_hist_snapshot where rownum < 2;
                select snap_id into bgnsp from dba_hist_snapshot where to_char(begin_interval_time,'DD-MON-YY HH24') = '$starttime' and rownum < 2;
                select snap_id into endsp from dba_hist_snapshot where to_char(begin_interval_time,'DD-MON-YY HH24') = '$endtime' and rownum < 2;
dbms_output.put_line('spool awrreport_'||bgnsp||endsp||'.html');
dbms_output.put_line(
'select output from table (dbms_workload_repository.awr_report_html('||
dbid||
',1,'||
bgnsp||
','||
endsp
||'));');
end;
        /
EOF
}

genawr()
{
sqlplus -s "/ as sysdba" <@awr_pre_report_$i.sql
EOF
}


while [ $i -le $n ]
do
        starttime=$(echo ${a[$i]} | cut -d: -f1)
        endtime=$(echo ${a[$i]} | cut -d: -f2)
        conndb
        genawr
        i=`expr $i + 1`
done
exit 0
#
#

================================================================