How spfile is discovered in 11gr2 Grid
--------------------------------------
As we all aware in 11gr2 grid infrastructure the asm spfile will be placed
under the asm diskgroup.
[oracle@oelrac1 ~]$ gpnptool get
Warning: some command line parameters were defaulted. Resulting command line:
/u01/grid/11.2.0/bin/gpnptool.bin get -o-
Cannot get GPnP profile. Error CLSGPNP_NO_DAEMON (GPNPD daemon is not running).
GPnP service is not running on localhost. Found locally cached profile...
micVNAZ3LnolkupldzDRgpM/sEw= HgZp5wx5t55NW7mWiEHSQsevlOd4jpzkDDoyNXYaVUf3fOYK+FLueGTLRr9WZLYmRedUuGVORGECF7v/apacc6dC+nLIa6XxutkHAu6rYuSbjcKoqpB8aCdSlG1gV1v7mzuSRQqGizs3ZW74Np/HZHXpwKsUGvbr6TLzLOwBT2Y=
Success.
--In the above output we could see that the asm_diskstring is mentioned as '/dev/oracleasm/disk*' which is
used to identify the spfile and the voting disks.
[oracle@oelrac1 ~]$ crsctl query css votedisk
## STATE File Universal Id File Name Disk group
-- ----- ----------------- --------- ---------
1. ONLINE 8f20abfc981c4f0dbf4707016284a667 (/dev/oracleasm/disks/OCR_VOTING_1) [OCR_VOTING]
2. ONLINE 3434c752d9654f2dbf5225b202432ce3 (/dev/oracleasm/disks/OCR_VOTING_2) [OCR_VOTING]
3. ONLINE 20a19fe75c3f4f94bf4f3e95866128ba (/dev/oracleasm/disks/OCR_VOTING_3) [OCR_VOTING]
Located 3 voting disk(s).
SQL> show parameter asm_disk;
NAME TYPE VALUE
------------------------------------ ----------- ------------------------------
asm_diskgroups string DATADG
asm_diskstring string /dev/oracleasm/disks/*
SQL> ! ls -lrt /dev/oracleasm/disks/*
brw-rw---- 1 oracle dba 8, 113 Mar 15 09:27 /dev/oracleasm/disks/DATA_DISK_4
brw-rw---- 1 oracle dba 8, 81 Mar 15 09:28 /dev/oracleasm/disks/DATA_DISK_2
brw-rw---- 1 oracle dba 8, 49 Mar 15 09:28 /dev/oracleasm/disks/OCR_VOTING_3
brw-rw---- 1 oracle dba 8, 33 Mar 15 09:28 /dev/oracleasm/disks/OCR_VOTING_2
brw-rw---- 1 oracle dba 8, 17 Mar 15 09:28 /dev/oracleasm/disks/OCR_VOTING_1
brw-rw---- 1 oracle dba 8, 97 Mar 15 09:28 /dev/oracleasm/disks/DATA_DISK_3
brw-rw---- 1 oracle dba 8, 65 Mar 15 09:28 /dev/oracleasm/disks/DATA_DISK_1
SQL> alter system set asm_diskstring = '/dev/oracleasm/disks/DATA*' scope=spfile;
System altered.
[root@oelrac1 ~]# cd /u01/grid/11.2.0/bin/
[root@oelrac1 bin]# ./crsctl stop crs
CRS-2793: Shutdown of Oracle High Availability Services-managed resources on 'oelrac1' has completed
CRS-4133: Oracle High Availability Services has been stopped.
[root@oelrac1 bin]# ./crsctl start crs
CRS-4123: Oracle High Availability Services has been started.
[root@oelrac1 bin]# ./crsctl stat res -t -init
--------------------------------------------------------------------------------
NAME TARGET STATE SERVER STATE_DETAILS
--------------------------------------------------------------------------------
Cluster Resources
--------------------------------------------------------------------------------
ora.asm
1 ONLINE OFFLINE Instance Shutdown
ora.cluster_interconnect.haip
1 ONLINE OFFLINE
ora.crf
1 ONLINE ONLINE oelrac1
ora.crsd
1 ONLINE OFFLINE
ora.cssd
1 ONLINE OFFLINE STARTING
ora.cssdmonitor
1 ONLINE ONLINE oelrac1
ora.ctssd
1 ONLINE OFFLINE
ora.diskmon
1 OFFLINE OFFLINE
ora.evmd
1 ONLINE OFFLINE
ora.gipcd
1 ONLINE ONLINE oelrac1
ora.gpnpd
1 ONLINE ONLINE oelrac1
ora.mdnsd
1 ONLINE ONLINE oelrac1
-- Now the CSSD process is not coming up.. this is expected.
Some snaps from occsd.log
2014-03-14 17:38:23.393: [ GPNP][3455813376]clsgpnp_profileCallUrlInt: [at clsgpnp.c:2234] Result: (0) CLSGPNP_OK. Successful get-profile CALL to remote "ipc://GPNPD
_oelrac1" disco ""
2014-03-14 17:38:23.393: [ CSSD][3455813376]clssscGetParameterProfile: buffer passed for parameter ASM discovery (3) is too short, required 27, passed 20
2014-03-14 17:38:23.393: [ CSSD][3455813376]clssnmReadDiscoveryProfile: voting file discovery string(/dev/oracleasm/disks/DATA*)
2014-03-14 17:38:23.393: [ CSSD][3455813376]clssnmvDDiscThread: using discovery string /dev/oracleasm/disks/DATA* for initial discovery
2014-03-14 17:38:23.393: [ SKGFD][3455813376]Discovery with str:/dev/oracleasm/disks/DATA*:
2014-03-14 17:38:23.393: [ SKGFD][3455813376]UFS discovery with :/dev/oracleasm/disks/DATA*:
2014-03-14 17:38:23.398: [ SKGFD][3455813376]Fetching UFS disk :/dev/oracleasm/disks/DATA_DISK_1:
2014-03-14 17:38:23.398: [ SKGFD][3455813376]Fetching UFS disk :/dev/oracleasm/disks/DATA_DISK_2:
2014-03-14 17:38:23.398: [ SKGFD][3455813376]Fetching UFS disk :/dev/oracleasm/disks/DATA_DISK_3:
2014-03-14 17:38:23.398: [ SKGFD][3455813376]Fetching UFS disk :/dev/oracleasm/disks/DATA_DISK_4:
2014-03-14 17:38:23.398: [ SKGFD][3455813376]OSS discovery with :/dev/oracleasm/disks/DATA*:
2014-03-14 17:38:23.398: [ SKGFD][3455813376]Handle 0x7fd5b80930f0 from lib :UFS:: for disk :/dev/oracleasm/disks/DATA_DISK_1:
2014-03-14 17:38:23.398: [ SKGFD][3455813376]Handle 0x7fd5b8137d80 from lib :UFS:: for disk :/dev/oracleasm/disks/DATA_DISK_2:
2014-03-14 17:38:23.398: [ SKGFD][3455813376]Handle 0x7fd5b8125730 from lib :UFS:: for disk :/dev/oracleasm/disks/DATA_DISK_3:
2014-03-14 17:38:23.399: [ SKGFD][3455813376]Handle 0x7fd5b8136cf0 from lib :UFS:: for disk :/dev/oracleasm/disks/DATA_DISK_4:
2014-03-14 17:38:23.399: [ SKGFD][3455813376]Lib :UFS:: closing handle 0x7fd5b80930f0 for disk :/dev/oracleasm/disks/DATA_DISK_1:
2014-03-14 17:38:23.399: [ SKGFD][3455813376]Lib :UFS:: closing handle 0x7fd5b8137d80 for disk :/dev/oracleasm/disks/DATA_DISK_2:
2014-03-14 17:38:23.399: [ SKGFD][3455813376]Lib :UFS:: closing handle 0x7fd5b8125730 for disk :/dev/oracleasm/disks/DATA_DISK_3:
2014-03-14 17:38:23.399: [ SKGFD][3455813376]Lib :UFS:: closing handle 0x7fd5b8136cf0 for disk :/dev/oracleasm/disks/DATA_DISK_4:
2014-03-14 17:38:23.399: [ CSSD][3455813376]clssnmvDiskVerify: Successful discovery of 0 disks
2014-03-14 17:38:23.399: [ CSSD][3455813376]clssnmCompleteInitVFDiscovery: Completing initial voting file discovery
2014-03-14 17:38:23.399: [ CSSD][3455813376]clssnmvFindInitialConfigs: No voting files found
--- Its is not able to find the voting disks.
[oracle@oelrac1 ~]$ cd /u01/grid/11.2.0/gpnp/oelrac1/profiles/peer/
[oracle@oelrac1 peer]$ cat profile.xml
xmlns:gpnp="http://www.grid-pnp.org/2005/11/gpnp-profile" xmlns:orcl="http://www.oracle.com/gpnp/2005/11/gpnp-profile"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://www.grid-pnp.org/2005/11/gpnp-profile
gpnp-profile.xsd" ProfileSequence="5" ClusterUId="5785d8b1a7aa5ff5ff072fe8b2bf44d1" ClusterName="oelrac"
PALocation="">Adapter="eth0" Use="public"/> DiscoveryString="+asm" LeaseDuration="400"/> 7jxgRZIBJookyQCAC5wBSF4s31I= iwoqZDP0rKW2D+uUI/NJZ5ckxgsHW49XQqzabeYi/9ebvXPb/bN2p0z9nuIrvRqj9a3E1DwEfP7JAKfD1ApHCRrK+v1vQ6VfypVXR2dToMrSwQk0tG6sEKl3wL3Pcm6DYEY3eVjRc+qA6bB0Xr/ENcZ0Hs9apAmYY6/19i0xi80= [oracle@oelrac1 peer]$
-- From the bove we could see DiscoveryString="/dev/oracleasm/disks/DATA*"
[oracle@oelrac1 peer]$ crsctl check crs
CRS-4638: Oracle High Availability Services is online
CRS-4535: Cannot communicate with Cluster Ready Services
CRS-4530: Communications failure contacting Cluster Synchronization Services daemon
CRS-4534: Cannot communicate with Event Manager
[oracle@oelrac1 peer]$ cp profile.xml profile.xml.bkp
[oracle@oelrac1 peer]$ gpnptool unsign -p=profile.xml.bkp
Warning: some command line parameters were defaulted. Resulting command line:
/u01/grid/11.2.0/bin/gpnptool.bin unsign -p=profile.xml.bkp -o-
Success.
[oracle@oelrac1 peer]$ gpnptool edit -asm_dis='/dev/oracleasm/disks/*' -p=profile.xml.bkp -o=profile.xml.bkp -ovr
Resulting profile written to "profile.xml.bkp".
Success.
[oracle@oelrac1 peer]$ gpnptool sign -p=profile.xml.bkp -w=/u01/grid/11.2.0/gpnp/oelrac1/wallets/peer/ -o=profile.new
Resulting profile written to "profile.new".
Success.
Restart the cluster.
[root@oelrac1 bin]# ./crsctl stop crs -f
CRS-4133: Oracle High Availability Services has been stopped.
[root@oelrac1 bin]# ./crsctl start crs
CRS-4123: Oracle High Availability Services has been started.
[root@oelrac1 bin]# ./crsctl stat res -t -init
--------------------------------------------------------------------------------
NAME TARGET STATE SERVER STATE_DETAILS
--------------------------------------------------------------------------------
Cluster Resources
--------------------------------------------------------------------------------
ora.asm
1 ONLINE ONLINE oelrac1 Started
ora.cluster_interconnect.haip
1 ONLINE ONLINE oelrac1
ora.crf
1 ONLINE ONLINE oelrac1
ora.crsd
1 ONLINE INTERMEDIATE oelrac1
ora.cssd
1 ONLINE ONLINE oelrac1
ora.cssdmonitor
1 ONLINE ONLINE oelrac1
ora.ctssd
1 ONLINE ONLINE oelrac1 ACTIVE:0
ora.diskmon
1 OFFLINE OFFLINE
ora.evmd
1 ONLINE INTERMEDIATE oelrac1
ora.gipcd
1 ONLINE ONLINE oelrac1
ora.gpnpd
1 ONLINE ONLINE oelrac1
ora.mdnsd
1 ONLINE ONLINE oelrac1
[oracle@oelrac1 ~]$ gpnptool get
Warning: some command line parameters were defaulted. Resulting command line:
/u01/grid/11.2.0/bin/gpnptool.bin get -o-
bOe6eT67erYiu4fT4ZwLIyKW9EA= KfwbH0soTVN3yVuwMA7ln57LUlvRgIPWGdxdRQlkghZZisg31NaZYXg4AHeXDo9xEs5LsYf5wR+Ay4ez84JWSCZ31G0k7o4FS43hB9CvrBW6q1+54JsC8iQGPZT61uGjqAVkTw8K9E0Kp5TIwTg0Tm54KwnxSu+UGo/O+w22ppY=
Success.
Now the - DiscoveryString="/dev/oracleasm/disks/*"
and also the crs started without issues.
--------------------------------------
As we all aware in 11gr2 grid infrastructure the asm spfile will be placed
under the asm diskgroup.
[oracle@oelrac1 ~]$ gpnptool get
Warning: some command line parameters were defaulted. Resulting command line:
/u01/grid/11.2.0/bin/gpnptool.bin get -o-
Cannot get GPnP profile. Error CLSGPNP_NO_DAEMON (GPNPD daemon is not running).
GPnP service is not running on localhost. Found locally cached profile...
Success.
--In the above output we could see that the asm_diskstring is mentioned as '/dev/oracleasm/disk*' which is
used to identify the spfile and the voting disks.
[oracle@oelrac1 ~]$ crsctl query css votedisk
## STATE File Universal Id File Name Disk group
-- ----- ----------------- --------- ---------
1. ONLINE 8f20abfc981c4f0dbf4707016284a667 (/dev/oracleasm/disks/OCR_VOTING_1) [OCR_VOTING]
2. ONLINE 3434c752d9654f2dbf5225b202432ce3 (/dev/oracleasm/disks/OCR_VOTING_2) [OCR_VOTING]
3. ONLINE 20a19fe75c3f4f94bf4f3e95866128ba (/dev/oracleasm/disks/OCR_VOTING_3) [OCR_VOTING]
Located 3 voting disk(s).
SQL> show parameter asm_disk;
NAME TYPE VALUE
------------------------------------ ----------- ------------------------------
asm_diskgroups string DATADG
asm_diskstring string /dev/oracleasm/disks/*
SQL> ! ls -lrt /dev/oracleasm/disks/*
brw-rw---- 1 oracle dba 8, 113 Mar 15 09:27 /dev/oracleasm/disks/DATA_DISK_4
brw-rw---- 1 oracle dba 8, 81 Mar 15 09:28 /dev/oracleasm/disks/DATA_DISK_2
brw-rw---- 1 oracle dba 8, 49 Mar 15 09:28 /dev/oracleasm/disks/OCR_VOTING_3
brw-rw---- 1 oracle dba 8, 33 Mar 15 09:28 /dev/oracleasm/disks/OCR_VOTING_2
brw-rw---- 1 oracle dba 8, 17 Mar 15 09:28 /dev/oracleasm/disks/OCR_VOTING_1
brw-rw---- 1 oracle dba 8, 97 Mar 15 09:28 /dev/oracleasm/disks/DATA_DISK_3
brw-rw---- 1 oracle dba 8, 65 Mar 15 09:28 /dev/oracleasm/disks/DATA_DISK_1
SQL> alter system set asm_diskstring = '/dev/oracleasm/disks/DATA*' scope=spfile;
System altered.
[root@oelrac1 ~]# cd /u01/grid/11.2.0/bin/
[root@oelrac1 bin]# ./crsctl stop crs
CRS-2793: Shutdown of Oracle High Availability Services-managed resources on 'oelrac1' has completed
CRS-4133: Oracle High Availability Services has been stopped.
[root@oelrac1 bin]# ./crsctl start crs
CRS-4123: Oracle High Availability Services has been started.
[root@oelrac1 bin]# ./crsctl stat res -t -init
--------------------------------------------------------------------------------
NAME TARGET STATE SERVER STATE_DETAILS
--------------------------------------------------------------------------------
Cluster Resources
--------------------------------------------------------------------------------
ora.asm
1 ONLINE OFFLINE Instance Shutdown
ora.cluster_interconnect.haip
1 ONLINE OFFLINE
ora.crf
1 ONLINE ONLINE oelrac1
ora.crsd
1 ONLINE OFFLINE
ora.cssd
1 ONLINE OFFLINE STARTING
ora.cssdmonitor
1 ONLINE ONLINE oelrac1
ora.ctssd
1 ONLINE OFFLINE
ora.diskmon
1 OFFLINE OFFLINE
ora.evmd
1 ONLINE OFFLINE
ora.gipcd
1 ONLINE ONLINE oelrac1
ora.gpnpd
1 ONLINE ONLINE oelrac1
ora.mdnsd
1 ONLINE ONLINE oelrac1
-- Now the CSSD process is not coming up.. this is expected.
Some snaps from occsd.log
2014-03-14 17:38:23.393: [ GPNP][3455813376]clsgpnp_profileCallUrlInt: [at clsgpnp.c:2234] Result: (0) CLSGPNP_OK. Successful get-profile CALL to remote "ipc://GPNPD
_oelrac1" disco ""
2014-03-14 17:38:23.393: [ CSSD][3455813376]clssscGetParameterProfile: buffer passed for parameter ASM discovery (3) is too short, required 27, passed 20
2014-03-14 17:38:23.393: [ CSSD][3455813376]clssnmReadDiscoveryProfile: voting file discovery string(/dev/oracleasm/disks/DATA*)
2014-03-14 17:38:23.393: [ CSSD][3455813376]clssnmvDDiscThread: using discovery string /dev/oracleasm/disks/DATA* for initial discovery
2014-03-14 17:38:23.393: [ SKGFD][3455813376]Discovery with str:/dev/oracleasm/disks/DATA*:
2014-03-14 17:38:23.393: [ SKGFD][3455813376]UFS discovery with :/dev/oracleasm/disks/DATA*:
2014-03-14 17:38:23.398: [ SKGFD][3455813376]Fetching UFS disk :/dev/oracleasm/disks/DATA_DISK_1:
2014-03-14 17:38:23.398: [ SKGFD][3455813376]Fetching UFS disk :/dev/oracleasm/disks/DATA_DISK_2:
2014-03-14 17:38:23.398: [ SKGFD][3455813376]Fetching UFS disk :/dev/oracleasm/disks/DATA_DISK_3:
2014-03-14 17:38:23.398: [ SKGFD][3455813376]Fetching UFS disk :/dev/oracleasm/disks/DATA_DISK_4:
2014-03-14 17:38:23.398: [ SKGFD][3455813376]OSS discovery with :/dev/oracleasm/disks/DATA*:
2014-03-14 17:38:23.398: [ SKGFD][3455813376]Handle 0x7fd5b80930f0 from lib :UFS:: for disk :/dev/oracleasm/disks/DATA_DISK_1:
2014-03-14 17:38:23.398: [ SKGFD][3455813376]Handle 0x7fd5b8137d80 from lib :UFS:: for disk :/dev/oracleasm/disks/DATA_DISK_2:
2014-03-14 17:38:23.398: [ SKGFD][3455813376]Handle 0x7fd5b8125730 from lib :UFS:: for disk :/dev/oracleasm/disks/DATA_DISK_3:
2014-03-14 17:38:23.399: [ SKGFD][3455813376]Handle 0x7fd5b8136cf0 from lib :UFS:: for disk :/dev/oracleasm/disks/DATA_DISK_4:
2014-03-14 17:38:23.399: [ SKGFD][3455813376]Lib :UFS:: closing handle 0x7fd5b80930f0 for disk :/dev/oracleasm/disks/DATA_DISK_1:
2014-03-14 17:38:23.399: [ SKGFD][3455813376]Lib :UFS:: closing handle 0x7fd5b8137d80 for disk :/dev/oracleasm/disks/DATA_DISK_2:
2014-03-14 17:38:23.399: [ SKGFD][3455813376]Lib :UFS:: closing handle 0x7fd5b8125730 for disk :/dev/oracleasm/disks/DATA_DISK_3:
2014-03-14 17:38:23.399: [ SKGFD][3455813376]Lib :UFS:: closing handle 0x7fd5b8136cf0 for disk :/dev/oracleasm/disks/DATA_DISK_4:
2014-03-14 17:38:23.399: [ CSSD][3455813376]clssnmvDiskVerify: Successful discovery of 0 disks
2014-03-14 17:38:23.399: [ CSSD][3455813376]clssnmCompleteInitVFDiscovery: Completing initial voting file discovery
2014-03-14 17:38:23.399: [ CSSD][3455813376]clssnmvFindInitialConfigs: No voting files found
--- Its is not able to find the voting disks.
[oracle@oelrac1 ~]$ cd /u01/grid/11.2.0/gpnp/oelrac1/profiles/peer/
[oracle@oelrac1 peer]$ cat profile.xml
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://www.grid-pnp.org/2005/11/gpnp-profile
gpnp-profile.xsd" ProfileSequence="5" ClusterUId="5785d8b1a7aa5ff5ff072fe8b2bf44d1" ClusterName="oelrac"
PALocation="">
-- From the bove we could see DiscoveryString="/dev/oracleasm/disks/DATA*"
[oracle@oelrac1 peer]$ crsctl check crs
CRS-4638: Oracle High Availability Services is online
CRS-4535: Cannot communicate with Cluster Ready Services
CRS-4530: Communications failure contacting Cluster Synchronization Services daemon
CRS-4534: Cannot communicate with Event Manager
[oracle@oelrac1 peer]$ cp profile.xml profile.xml.bkp
[oracle@oelrac1 peer]$ gpnptool unsign -p=profile.xml.bkp
Warning: some command line parameters were defaulted. Resulting command line:
/u01/grid/11.2.0/bin/gpnptool.bin unsign -p=profile.xml.bkp -o-
Success.
[oracle@oelrac1 peer]$ gpnptool edit -asm_dis='/dev/oracleasm/disks/*' -p=profile.xml.bkp -o=profile.xml.bkp -ovr
Resulting profile written to "profile.xml.bkp".
Success.
[oracle@oelrac1 peer]$ gpnptool sign -p=profile.xml.bkp -w=/u01/grid/11.2.0/gpnp/oelrac1/wallets/peer/ -o=profile.new
Resulting profile written to "profile.new".
Success.
Restart the cluster.
[root@oelrac1 bin]# ./crsctl stop crs -f
CRS-4133: Oracle High Availability Services has been stopped.
[root@oelrac1 bin]# ./crsctl start crs
CRS-4123: Oracle High Availability Services has been started.
[root@oelrac1 bin]# ./crsctl stat res -t -init
--------------------------------------------------------------------------------
NAME TARGET STATE SERVER STATE_DETAILS
--------------------------------------------------------------------------------
Cluster Resources
--------------------------------------------------------------------------------
ora.asm
1 ONLINE ONLINE oelrac1 Started
ora.cluster_interconnect.haip
1 ONLINE ONLINE oelrac1
ora.crf
1 ONLINE ONLINE oelrac1
ora.crsd
1 ONLINE INTERMEDIATE oelrac1
ora.cssd
1 ONLINE ONLINE oelrac1
ora.cssdmonitor
1 ONLINE ONLINE oelrac1
ora.ctssd
1 ONLINE ONLINE oelrac1 ACTIVE:0
ora.diskmon
1 OFFLINE OFFLINE
ora.evmd
1 ONLINE INTERMEDIATE oelrac1
ora.gipcd
1 ONLINE ONLINE oelrac1
ora.gpnpd
1 ONLINE ONLINE oelrac1
ora.mdnsd
1 ONLINE ONLINE oelrac1
[oracle@oelrac1 ~]$ gpnptool get
Warning: some command line parameters were defaulted. Resulting command line:
/u01/grid/11.2.0/bin/gpnptool.bin get -o-
Success.
Now the - DiscoveryString="/dev/oracleasm/disks/*"
and also the crs started without issues.
No comments:
Post a Comment