Recovering the primary database using the backup taken from the Standby Database

Recovering the primary database using the backup taken from the Standby Database
===============================================================================

1) Take the backup from the standby database.

[oracle@standalone2 bkp]$ rman target /

Recovery Manager: Release 11.2.0.3.0 - Production on Thu Jan 29 03:49:35 2015

Copyright (c) 1982, 2011, Oracle and/or its affiliates.  All rights reserved.

connected to target database: SOURCE (DBID=2931643800)

RMAN> backup as compressed backupset full database format '/u01/app/oracle/bkp/full_bkp_%T_%U.bkp' ;

Starting backup at 29-JAN-15
using target database control file instead of recovery catalog
allocated channel: ORA_DISK_1
channel ORA_DISK_1: SID=34 device type=DISK
channel ORA_DISK_1: starting compressed full datafile backup set
channel ORA_DISK_1: specifying datafile(s) in backup set
input datafile file number=00003 name=/u01/app/oracle/oradata/source/undotbs01.dbf
input datafile file number=00001 name=/u01/app/oracle/oradata/source/system01.dbf
input datafile file number=00002 name=/u01/app/oracle/oradata/source/sysaux01.dbf
input datafile file number=00005 name=/u01/app/oracle/oradata/source/test_tbls_01.dbf
input datafile file number=00004 name=/u01/app/oracle/oradata/source/users01.dbf
channel ORA_DISK_1: starting piece 1 at 29-JAN-15
channel ORA_DISK_1: finished piece 1 at 29-JAN-15
piece handle=/u01/app/oracle/bkp/full_bkp_20150129_16pttekb_1_1.bkp tag=TAG20150129T035243 comment=NONE
channel ORA_DISK_1: backup set complete, elapsed time: 00:03:06
channel ORA_DISK_1: starting compressed full datafile backup set
channel ORA_DISK_1: specifying datafile(s) in backup set
including current control file in backup set
including current SPFILE in backup set
channel ORA_DISK_1: starting piece 1 at 29-JAN-15
channel ORA_DISK_1: finished piece 1 at 29-JAN-15
piece handle=/u01/app/oracle/bkp/full_bkp_20150129_17ptteq5_1_1.bkp tag=TAG20150129T035243 comment=NONE
channel ORA_DISK_1: backup set complete, elapsed time: 00:00:02
Finished backup at 29-JAN-15

RMAN> exit


Recovery Manager complete.


-- Now copy the backup pieces to the primary site.

[oracle@standalone2 bkp]$ ls -lrt
total 284192
-rw-r----- 1 oracle oinstall 289587200 Jan 29 03:55 full_bkp_20150129_16pttekb_1_1.bkp
-rw-r----- 1 oracle oinstall   1130496 Jan 29 03:55 full_bkp_20150129_17ptteq5_1_1.bkp

[oracle@standalone2 bkp]$ scp * oracle@rhel11gr2rac1:/oracle/bkp/
oracle@rhel11gr2rac1's password:
full_bkp_20150129_16pttekb_1_1.bkp                                                                                                    100%  276MB   6.4MB/s   00:43
full_bkp_20150129_17ptteq5_1_1.bkp                                                                                                    100% 1104KB   1.1MB/s   00:00


-- Now in primary database remove one datafile (Test case)


SQL> select name from V$datafile;

NAME
--------------------------------------------------------------------------------
/u01/app/oracle/oradata/source/system01.dbf
/u01/app/oracle/oradata/source/sysaux01.dbf
/u01/app/oracle/oradata/source/undotbs01.dbf
/u01/app/oracle/oradata/source/users01.dbf
/u01/app/oracle/oradata/source/test_tbls_01.dbf


SQL> ! rm -f /u01/app/oracle/oradata/source/test_tbls_01.dbf

SQL> alter system checkpoint;
alter system checkpoint
*
ERROR at line 1:
ORA-03113: end-of-file on communication channel
Process ID: 12508
Session ID: 37 Serial number: 57


SQL> startup
ORACLE instance started.

Total System Global Area  313159680 bytes
Fixed Size                  2227944 bytes
Variable Size             289407256 bytes
Database Buffers           16777216 bytes
Redo Buffers                4747264 bytes
Database mounted.
ORA-01157: cannot identify/lock data file 5 - see DBWR trace file
ORA-01110: data file 5: '/u01/app/oracle/oradata/source/test_tbls_01.dbf'


-- Now lets recover the datafile.


[oracle@rhel11gr2rac1 bkp]$ rman target /

Recovery Manager: Release 11.2.0.3.0 - Production on Thu Jan 29 12:29:16 2015

Copyright (c) 1982, 2011, Oracle and/or its affiliates.  All rights reserved.

connected to target database: SOURCE (DBID=2931643800, not open)

RMAN> catalog start with '/oracle/bkp/';

using target database control file instead of recovery catalog
searching for all files that match the pattern /oracle/bkp/

List of Files Unknown to the Database
=====================================
File Name: /oracle/bkp/full_bkp_20150129_16pttekb_1_1.bkp
File Name: /oracle/bkp/full_bkp_20150129_17ptteq5_1_1.bkp

Do you really want to catalog the above files (enter YES or NO)? YES
cataloging files...
cataloging done

List of Cataloged Files
=======================
File Name: /oracle/bkp/full_bkp_20150129_16pttekb_1_1.bkp
File Name: /oracle/bkp/full_bkp_20150129_17ptteq5_1_1.bkp


RMAN> restore datafile 5;

Starting restore at 29-JAN-15
allocated channel: ORA_DISK_1
channel ORA_DISK_1: SID=21 device type=DISK

channel ORA_DISK_1: starting datafile backup set restore
channel ORA_DISK_1: specifying datafile(s) to restore from backup set
channel ORA_DISK_1: restoring datafile 00005 to /u01/app/oracle/oradata/source/test_tbls_01.dbf
channel ORA_DISK_1: reading from backup piece /oracle/bkp/full_bkp_20150129_16pttekb_1_1.bkp
channel ORA_DISK_1: piece handle=/oracle/bkp/full_bkp_20150129_16pttekb_1_1.bkp tag=TAG20150129T035243
channel ORA_DISK_1: restored backup piece 1
channel ORA_DISK_1: restore complete, elapsed time: 00:00:07
Finished restore at 29-JAN-15

RMAN> recover datafile 5;

Starting recover at 29-JAN-15
using channel ORA_DISK_1

starting media recovery
media recovery complete, elapsed time: 00:00:00

Finished recover at 29-JAN-15

RMAN> exit;

=============================================


SQL> select status from V$instance;

STATUS
------------
MOUNTED



SQL> alter database open;

Database altered.


SQL> alter database datafile 5 online;

Database altered.

SQL> select tablespace_name, status from dba_tablespaces;

TABLESPACE_NAME                STATUS
------------------------------ ---------
SYSTEM                         ONLINE
SYSAUX                         ONLINE
UNDOTBS1                       ONLINE
TEMP                           ONLINE
USERS                          ONLINE
TEST_TBLS                      ONLINE

6 rows selected.

==============================

Flashback usage on Data guard for Data Recover and Re-instate Standby database after failover

Flashback usage in Standby for Data recovery
==================================

1) Enable flashback on primary and standby database.



SQL> select DB_UNIQUE_NAME, FLASHBACK_ON from V$database;

DB_UNIQUE_NAME                 FLASHBACK_ON
------------------------------ ------------------
source                         YES


SQL> select DB_UNIQUE_NAME, FLASHBACK_ON from V$database;

DB_UNIQUE_NAME                 FLASHBACK_ON
------------------------------ ------------------
sourstby                       YES



Primary
=======

SQL> select count(*) from test_user.emp;

  COUNT(*)
----------
      5000

SQL> delete from test_user.emp where rownum < 2000;

1999 rows deleted.

SQL> commit;

Commit complete.


SQL> alter system switch logfile;

System altered.

SQL> select count(*) from test_user.emp;

  COUNT(*)
----------
      3001



In Standby
==========

SQL> select count(*) from test_user.emp;

  COUNT(*)
----------
      3001


-- In primary it has been deleted erroronously.

Lets restore the database back in the standby site.


SQL> alter database recover managed standby database cancel;

Database altered.


SQL> shut immediate;
Database closed.
Database dismounted.
ORACLE instance shut down.

SQL> exit

SQL> startup mount;
ORACLE instance started.

Total System Global Area  313159680 bytes
Fixed Size                  2227944 bytes
Variable Size             230687000 bytes
Database Buffers           75497472 bytes
Redo Buffers                4747264 bytes
Database mounted.

SQL> flashback database to timestamp to_timestamp('29-JAN-15 00:30:00','DD-MON-YY HH24:MI:SS') ;

Flashback complete.

SQL> alter database open read only;

Database altered.

SQL> select count(*) from test_user.emp;

  COUNT(*)
----------
      5000

-- Now export the table and import in the primary.



[oracle@standalone2 ~]$ exp file=emp_tbl.dmp tables=test_user.emp log=emp_tbl.log

Export: Release 11.2.0.3.0 - Production on Thu Jan 29 02:27:44 2015

Copyright (c) 1982, 2011, Oracle and/or its affiliates.  All rights reserved.


Username: / as sysdba


About to export specified tables via Conventional Path ...
Current user changed to TEST_USER
. . exporting table                            EMP       5000 rows exported
Export terminated successfully without warnings.


[oracle@standalone2 ~]$ exp file=emp_tbl.dmp tables=test_user.emp log=emp_tbl.log

Export: Release 11.2.0.3.0 - Production on Thu Jan 29 02:27:44 2015

Copyright (c) 1982, 2011, Oracle and/or its affiliates.  All rights reserved.


Username: / as sysdba

Connected to: Oracle Database 11g Enterprise Edition Release 11.2.0.3.0 - 64bit Production
With the Partitioning, OLAP, Data Mining and Real Application Testing options
Export done in US7ASCII character set and AL16UTF16 NCHAR character set
server uses AL32UTF8 character set (possible charset conversion)

About to export specified tables via Conventional Path ...
Current user changed to TEST_USER
. . exporting table                            EMP       5000 rows exported
Export terminated successfully without warnings.


scp the dmp file to the prod.


Truncate the table in prod


SQL> truncate table test_user.emp;

Table truncated.


-- Import the dmp file.

[oracle@rhel11gr2rac1 oracle]$ imp file=emp_tbl.dmp tables=emp fromuser=test_user touser=test_user ignore=y

Import: Release 11.2.0.3.0 - Production on Thu Jan 29 10:39:25 2015

Copyright (c) 1982, 2011, Oracle and/or its affiliates.  All rights reserved.

Username: / as sysdba

Connected to: Oracle Database 11g Enterprise Edition Release 11.2.0.3.0 - 64bit Production
With the Partitioning, OLAP, Data Mining and Real Application Testing options

Export file created by EXPORT:V11.02.00 via conventional path
import done in US7ASCII character set and AL16UTF16 NCHAR character set
import server uses AL32UTF8 character set (possible charset conversion)
. importing TEST_USER's objects into TEST_USER
. . importing table                          "EMP"       5000 rows imported
Import terminated successfully without warnings.


Now start the MRP process in standby

SQL> alter database recover managed standby database disconnect from session;

Database altered.

SQL> select count(*) from test_user.emp;

  COUNT(*)
----------
      3001
SQL> /

  COUNT(*)
----------
      3001

SQL> /

  COUNT(*)
----------
      3001

SQL> /
/select count(*) from test_user.emp
*
ERROR at line 1:
ORA-08103: object no longer exists

SQL> /

  COUNT(*)
----------
      5000

-- Now both the db are in sync





====*********************************************================================================================



Flashback usage to re-instate the standby database after the failover
=====================================================================


Create standby redologs on both primary and standby logs.

SQL> alter database add standby logfile group 4 ('/u01/app/oracle/oradata/source/stdb_redo0.log') size 52428800;

Database altered.

SQL> alter database add standby logfile group 5 ('/u01/app/oracle/oradata/source/stdb_redo05.log') size 52428800;

Database altered.

SQL> alter database add standby logfile group 6 ('/u01/app/oracle/oradata/source/stdb_redo06.log') size 52428800;

Database altered.


SQL> select * from V$standby_log;

    GROUP# DBID                                        THREAD#  SEQUENCE#      BYTES  BLOCKSIZE       USED ARC STATUS     FIRST_CHANGE# FIRST_TIM NEXT_CHANGE# NEXT_TIME LAST_CHANGE# LAST_TIME
---------- ---------------------------------------- ---------- ---------- ---------- ---------- ---------- --- ---------- ------------- --------- ------------ --------- ------------ ---------
         4 UNASSIGNED                                        0          0   52428800        512          0 YES UNASSIGNED
         5 UNASSIGNED                                        0          0   52428800        512          0 YES UNASSIGNED
         6 UNASSIGNED                                        0          0   52428800        512          0 YES UNASSIGNED


-- Now

Standby

SQL> alter database recover managed standby database cancel;

Database altered.

SQL> alter database add standby logfile group 4 ('/u01/app/oracle/oradata/source/stdb_redo0.log') size 52428800;

Database altered.

SQL> alter database add standby logfile group 5 ('/u01/app/oracle/oradata/source/stdb_redo05.log') size 52428800;

Database altered.

SQL> alter database add standby logfile group 6 ('/u01/app/oracle/oradata/source/stdb_redo06.log') size 52428800;

Database altered.

SQL> alter database recover managed standby database using current logfile disconnect from session;

Database altered.


SQL> SELECT UNIQUE THREAD# AS THREAD, MAX(SEQUENCE#) OVER (PARTITION BY thread#) AS LAST from V$ARCHIVED_LOG;

    THREAD       LAST
---------- ----------
         1        123


SQL> SELECT THREAD#, LOW_SEQUENCE#, HIGH_SEQUENCE# FROM V$ARCHIVE_GAP;

no rows selected

SQL> ALTER DATABASE RECOVER MANAGED STANDBY DATABASE CANCEL;

Database altered.

SQL> ALTER DATABASE RECOVER MANAGED STANDBY DATABASE FINISH;

Database altered.

SQL> ALTER DATABASE COMMIT TO SWITCHOVER TO PRIMARY WITH SESSION SHUTDOWN;

Database altered.

SQL> alter database open;

Database altered.


SQL> SELECT TO_CHAR(STANDBY_BECAME_PRIMARY_SCN) FROM V$DATABASE;

TO_CHAR(STANDBY_BECAME_PRIMARY_SCN)
----------------------------------------
2937170


On the primary flashback the database to the above scn.



SQL> shut immediate;
Database closed.
Database dismounted.
ORACLE instance shut down.
SQL>
SQL>
SQL>
SQL>
SQL> startup mount;
ORACLE instance started.

Total System Global Area  313159680 bytes
Fixed Size                  2227944 bytes
Variable Size             301990168 bytes
Database Buffers            4194304 bytes
Redo Buffers                4747264 bytes
Database mounted.


SQL> flashback database to scn 2937170;

Flashback complete.


SQL> alter database flashback off;

Database altered.

-- Flashback off will delete all the flashback logs, since we will be creating a standby controlfile and mounting it, it will
overwrite all the flashback details will become obsolete, so we can turn off the flashback.


SQL> alter database flashback off;

Database altered.

=== For conver this database to physical standby either we can create a standby controlfile on the database and over write
the existing controlfiles or can execute the command to conver to physcial stanby

===============================================================================
SQL> alter database create standby controlfile as '/home/oracle/ctl_file.ctl';

Database altered.

SQL> shut immediate;
ORA-01109: database not open


Database dismounted.
ORACLE instance shut down.

SQL> exit

SQL> startup mount;
ORACLE instance started.

Total System Global Area  313159680 bytes
Fixed Size                  2227944 bytes
Variable Size             301990168 bytes
Database Buffers            4194304 bytes
Redo Buffers                4747264 bytes
Database mounted.

SQL> select database_role from V$database;

DATABASE_ROLE
----------------
PHYSICAL STANDBY

-- Overwrite the existing controlfile with the created standby controlfile.

[oracle@rhel11gr2rac1 ~]$ cp ctl_file.ctl /u01/app/oracle/oradata/source/control01.ctl
[oracle@rhel11gr2rac1 ~]$ cp ctl_file.ctl /u01/app/oracle/fast_recovery_area/source/control02.ctl

==============================(or)=============================================

SQL> alter database convert to physical standby ;

Database altered.

SQL> shut immediate;
ORA-01507: database not mounted


ORACLE instance shut down.
SQL> startup mount;
ORACLE instance started.

Total System Global Area  313159680 bytes
Fixed Size                  2227944 bytes
Variable Size             230687000 bytes
Database Buffers           75497472 bytes
Redo Buffers                4747264 bytes
Database mounted.

SQL> select database_role from V$database;

DATABASE_ROLE
----------------
PHYSICAL STANDBY


=====================================================================================



-- Now enable the archive dest 2 in the current primay database.

SQL> show parameter dest_state_2;

NAME                                 TYPE        VALUE
------------------------------------ ----------- ------------------------------
log_archive_dest_state_2             string      ENABLE


-- Now start the MRP in the current standby db.

SQL> alter database recover managed standby database using current logfile disconnect from session;

Database altered.



SQL> select database_role from V$database;

DATABASE_ROLE
----------------
PHYSICAL STANDBY



SQL> alter database recover managed standby database cancel;

Database altered.


SQL> alter database flashback on;

Database altered.


SQL> alter database open read only;

Database altered.

SQL> alter database recover managed standby database using current logfile disconnect from session;

Database altered.


In prod

======

SQL> select count(*) from test_user.emp;

  COUNT(*)
----------
      3001


SQL> delete from test_user.emp where rownum < 1000;

999 rows deleted.

SQL> commit;

Commit complete.


In Current Standby:=
====================

Now check the propogation in standby.

SQL> select count(*) from test_user.emp;

  COUNT(*)
----------
      3001

SQL> /

  COUNT(*)
----------
      2002


=================================================================================================================

Recovering Physical Standby using Incremental backup of Primary

Recovering Physical Standby using Incremental backup of Primary
===============================================================

Scenario :- Physical standby went out of sync from the primary as some of the archive logs are missing,
even those archivelog is deleted in the primary as well and no backup is available. To overcome this
we can take the incremental backup from primary and apply to the standby to keep it is in sync.


Primary DB

SQL> select current_scn from V$database;

CURRENT_SCN
-----------
    2923116


SQL> select max(sequence#) from V$archived_log where applied = 'YES' and dest_id = 2;

MAX(SEQUENCE#)
--------------
            91


Standby DB
----------

SQL> select current_scn from V$database;

CURRENT_SCN
-----------
    2921713


SQL> select process, STATUS, sequence# from V$managed_standby where process like '%MRP%';

PROCESS   STATUS        SEQUENCE#
--------- ------------ ----------
MRP0      WAIT_FOR_GAP         92


SQL> SELECT * FROM v$ARCHIVE_GAP;

   THREAD# LOW_SEQUENCE# HIGH_SEQUENCE#
---------- ------------- --------------
         1            92            113


--


As per the above we could see that there is a gap between 93 and 113 sequence which is missing on the standby site,
the current scn in standby site is 2921713, around 20 archive logs are missing also it is not availble in backup
in the primary site. Lets take the incremental backup from the primary site.


RMAN> run
{
allocate channel a1 device type disk format '/oracle/scn_incre_bkp_%U_%T.bkp';
backup incremental from scn 2921713 database;
}

using target database control file instead of recovery catalog
allocated channel: a1
channel a1: SID=34 device type=DISK

Starting backup at 28-JAN-15

backup will be obsolete on date 04-FEB-15
archived logs will not be kept or backed up
channel a1: starting full datafile backup set
channel a1: specifying datafile(s) in backup set
input datafile file number=00003 name=/u01/app/oracle/oradata/source/undotbs01.dbf
input datafile file number=00001 name=/u01/app/oracle/oradata/source/system01.dbf
input datafile file number=00002 name=/u01/app/oracle/oradata/source/sysaux01.dbf
input datafile file number=00005 name=/u01/app/oracle/oradata/source/test_tbls_01.dbf
input datafile file number=00004 name=/u01/app/oracle/oradata/source/users01.dbf
channel a1: starting piece 1 at 28-JAN-15
channel a1: finished piece 1 at 28-JAN-15
piece handle=/oracle/scn_incre_bkp_13ptrh7n_1_1_20150128.bkp tag=TAG20150128T102454 comment=NONE
channel a1: backup set complete, elapsed time: 00:02:05

backup will be obsolete on date 04-FEB-15
archived logs will not be kept or backed up
channel a1: starting full datafile backup set
channel a1: specifying datafile(s) in backup set
including current control file in backup set
channel a1: starting piece 1 at 28-JAN-15
channel a1: finished piece 1 at 28-JAN-15
piece handle=/oracle/scn_incre_bkp_14ptrhbl_1_1_20150128.bkp tag=TAG20150128T102454 comment=NONE
channel a1: backup set complete, elapsed time: 00:00:01
Finished backup at 28-JAN-15
released channel: a1


-- Take the current controlfile trace for standby.


SQL> alter database create standby controlfile as '/oracle/for_standby_ctl.ctl';

Database altered.


--


scp the controlfile and the backup piece to the standby location.

[oracle@rhel11gr2rac1 oracle]$ scp *.bkp oracle@standalone2.manzoor.com:/u01/app/oracle/bkp/
oracle@standalone2.manzoor.com's password:
scn_incre_bkp_13ptrh7n_1_1_20150128.bkp                                                                                               100% 2552KB   2.5MB/s   00:00
scn_incre_bkp_14ptrhbl_1_1_20150128.bkp                                                                                               100% 9792KB   9.6MB/s   00:01

[oracle@rhel11gr2rac1 oracle]$ scp for_standby_ctl.ctl oracle@standalone2.manzoor.com:/u01/app/oracle/bkp/
oracle@standalone2.manzoor.com's password:
for_standby_ctl.ctl                                                                                                                   100% 9744KB   9.5MB/s   00:01
[oracle@rhel11gr2rac1 oracle]$


On standby, stop the MRP process , shut down db and start with the controlfile which been copied from the primary site.


SQL> alter database recover managed standby database cancel;

Database altered.

SQL> show parameter control_files;

NAME                                 TYPE        VALUE
------------------------------------ ----------- -----------------------------------------------------------------------------------------------------
control_files                        string      /u01/app/oracle/oradata/source/control01.ctl, /u01/app/oracle/fast_recovery_area/source/control02.ctl

SQL> shut immediate;
ORA-01109: database not open


Database dismounted.
ORACLE instance shut down.


-- Overwrite the old controlfiles with the new one.



[oracle@standalone2 bkp]$ cp for_standby_ctl.ctl /u01/app/oracle/oradata/source/control01.ctl
[oracle@standalone2 bkp]$ cp for_standby_ctl.ctl /u01/app/oracle/fast_recovery_area/source/control02.ctl


[oracle@standalone2 bkp]$ rman target /

Recovery Manager: Release 11.2.0.3.0 - Production on Wed Jan 28 03:04:45 2015

Copyright (c) 1982, 2011, Oracle and/or its affiliates.  All rights reserved.

connected to target database (not started)

RMAN> startup mount;

Oracle instance started
database mounted

Total System Global Area     313159680 bytes

Fixed Size                     2227944 bytes
Variable Size                226492696 bytes
Database Buffers              79691776 bytes
Redo Buffers                   4747264 bytes


-- catalog the backup piece


RMAN> catalog start with '/u01/app/oracle/bkp/scn';

Starting implicit crosscheck backup at 28-JAN-15
using target database control file instead of recovery catalog
allocated channel: ORA_DISK_1
channel ORA_DISK_1: SID=28 device type=DISK
Crosschecked 17 objects
Finished implicit crosscheck backup at 28-JAN-15

Starting implicit crosscheck copy at 28-JAN-15
using channel ORA_DISK_1
Crosschecked 4 objects
Finished implicit crosscheck copy at 28-JAN-15

searching for all files in the recovery area
cataloging files... files cataloged

searching for all files that match the pattern /u01/app/oracle/bkp/scn

List of Files Unknown to the Database
=====================================
File Name: /u01/app/oracle/bkp/scn_incre_bkp_14ptrhbl_1_1_20150128.bkp
File Name: /u01/app/oracle/bkp/scn_incre_bkp_13ptrh7n_1_1_20150128.bkp

Do you really want to catalog the above files (enter YES or NO)? YES
cataloging files...
cataloging done

List of Cataloged Files
=======================
File Name: /u01/app/oracle/bkp/scn_incre_bkp_14ptrhbl_1_1_20150128.bkp
File Name: /u01/app/oracle/bkp/scn_incre_bkp_13ptrh7n_1_1_20150128.bkp

-- Start the recover.

RMAN> recover database;

Starting recover at 28-JAN-15
using channel ORA_DISK_1
channel ORA_DISK_1: starting incremental datafile backup set restore
channel ORA_DISK_1: specifying datafile(s) to restore from backup set
destination for restore of datafile 00001: /u01/app/oracle/oradata/source/system01.dbf
destination for restore of datafile 00002: /u01/app/oracle/oradata/source/sysaux01.dbf
destination for restore of datafile 00003: /u01/app/oracle/oradata/source/undotbs01.dbf
destination for restore of datafile 00004: /u01/app/oracle/oradata/source/users01.dbf
destination for restore of datafile 00005: /u01/app/oracle/oradata/source/test_tbls_01.dbf
channel ORA_DISK_1: reading from backup piece /u01/app/oracle/bkp/scn_incre_bkp_13ptrh7n_1_1_20150128.bkp
channel ORA_DISK_1: piece handle=/u01/app/oracle/bkp/scn_incre_bkp_13ptrh7n_1_1_20150128.bkp tag=TAG20150128T102454
channel ORA_DISK_1: restored backup piece 1
channel ORA_DISK_1: restore complete, elapsed time: 00:00:04

starting media recovery

archived log for thread 1 with sequence 116 is already on disk as file /u01/app/oracle/archive/1_116_868359284.dbf
archived log file name=/u01/app/oracle/archive/1_116_868359284.dbf thread=1 sequence=116
unable to find archived log
archived log thread=1 sequence=117
RMAN-00571: ===========================================================
RMAN-00569: =============== ERROR MESSAGE STACK FOLLOWS ===============
RMAN-00571: ===========================================================
RMAN-03002: failure of recover command at 01/28/2015 03:07:01
RMAN-06054: media recovery requesting unknown archived log for thread 1 with sequence 117 and starting SCN of 2928165


-- Now the restore is completed.

-- Lets check the scn now.

SQL> select current_scn from V$database;

CURRENT_SCN
-----------
    2928164

-- Now start the MRP process, both primary and standby are in sync now.