|
1. 检查votedisk和 ocr备份
[[email?protected] bin]./ocrconfig -showbackup
rh6rac1???? 2019/03/19 16:57:40???? /oracle/grid/cdata/rh6rac-cluster/backup00.ocr
rh6rac1???? 2019/03/19 12:57:40???? /oracle/grid/cdata/rh6rac-cluster/backup01.ocr
rh6rac1???? 2019/03/19 08:57:40???? /oracle/grid/cdata/rh6rac-cluster/backup02.ocr
rh6rac1???? 2019/03/18 00:57:38???? /oracle/grid/cdata/rh6rac-cluster/day.ocr
rh6rac2???? 2019/03/13 04:13:03???? /oracle/grid/cdata/rh6rac-cluster/week.ocr
[[email?protected] bin]# ./crsctl query css votedisk
##? STATE??? File Universal Id??????????????? File Name Disk group
--? -----??? -----------------??????????????? --------- ---------
?1. ONLINE?? 4621da78f6cb4f45bfc427515ba7d5fc (/dev/asm-diskb) [OCRVOTE]
Located 1 voting disk(s).
2. 彻底关闭所有节点上的clusterware,OHASD
[[email?protected] bin]#./crsctl stop has -f
使用dd 命令 破坏ocr和votedisk所在diskgroup
dd if=/dev/zero of=/dev/asm-diskb bs=1024k count=1
[[email?protected] bin]# ./crsctl start has
CRS-4123: Oracle High Availability Services has been started.
[email?protected] bin]# ./crsctl check crs
CRS-4638: Oracle High Availability Services is online
CRS-4535: Cannot communicate with Cluster Ready Services
CRS-4530: Communications failure contacting Cluster Synchronization Services daemon
CRS-4534: Cannot communicate with Event Manager
alertrh6rac1.log
[/oracle/grid/bin/orarootagent.bin(3725)]CRS-5822:Agent ‘/oracle/grid/bin/orarootagent_root‘ disconnected from server. Details at (:CRSAGF00117
:) {0:5:903} in /oracle/grid/log/rh6rac1/agent/crsd/orarootagent_root//orarootagent_root.log.2019-03-19 17:10:20.811:
[ctssd(3174)]CRS-2405:The Cluster Time Synchronization Service on host rh6rac1 is shutdown by user
2019-03-19 17:10:20.830:
[mdnsd(2820)]CRS-5602:mDNS service stopping by request.
[client(15431)]CRS-10001:19-Mar-19 17:10 ACFS-9290: Waiting for ASM to shutdown.
2019-03-19 17:10:31.655:
[cssd(2915)]CRS-1603:CSSD on node rh6rac1 shutdown by user.
2019-03-19 17:10:31.762:
[ohasd(2230)]CRS-2767:Resource state recovery not attempted for ‘ora.cssdmonitor‘ as its target state is OFFLINE
2019-03-19 17:10:31.857:
[cssd(2915)]CRS-1660:The CSS daemon shutdown has completed
2019-03-19 17:10:35.402:
[gpnpd(2841)]CRS-2329:GPNPD on node rh6rac1 shutdown.
2019-03-19 17:12:07.003:
[ohasd(16139)]CRS-2112:The OLR service started on node rh6rac1.
2019-03-19 17:12:07.013:
[ohasd(16139)]CRS-1301:Oracle High Availability Service started on node rh6rac1.
2019-03-19 17:12:07.014:
[ohasd(16139)]CRS-8017:location: /etc/oracle/lastgasp has 2 reboot advisory log files,0 were announced and 0 errors occurred
2019-03-19 17:12:10.417:
[/oracle/grid/bin/orarootagent.bin(16241)]CRS-2302:Cannot get GPnP profile. Error CLSGPNP_NO_DAEMON (GPNPD daemon is not running).
2019-03-19 17:12:14.840:
[gpnpd(16359)]CRS-2328:GPNPD started on node rh6rac1.
2019-03-19 17:12:17.341:
[cssd(16429)]CRS-1713:CSSD daemon is started in clustered mode
2019-03-19 17:12:19.093:
[ohasd(16139)]CRS-2767:Resource state recovery not attempted for ‘ora.diskmon‘ as its target state is OFFLINE
2019-03-19 17:12:25.937:
[cssd(16429)]CRS-1714:Unable to discover any voting files,retrying discovery in 15 seconds; Details at (:CSSNM00070:) in /oracle/grid/log/rh6r
ac1/cssd/ocssd.log2019-03-19 17:12:40.950:
[cssd(16429)]CRS-1714:Unable to discover any voting files,retrying discovery in 15 seconds; Details at (:CSSNM00070:) in /oracle/grid/log/rh6r
ac1/cssd/ocssd.log2019-03-19 17:12:55.959:
[cssd(16429)]CRS-1714:Unable to discover any voting files,retrying discovery in 15 seconds; Details at (:CSSNM00070:) in /oracle/grid/log/rh6r
ac1/cssd/ocssd.log2019-03-19 17:13:10.968:
[cssd(16429)]CRS-1714:Unable to discover any voting files,retrying discovery in 15 seconds; Details at (:CSSNM00070:) in /oracle/grid/log/rh6r
ac1/cssd/ocssd.log2019-03-19 17:13:25.976:
[cssd(16429)]CRS-1714:Unable to discover any voting files,retrying discovery in 15 seconds; Details at (:CSSNM00070:) in /oracle/grid/log/rh6r
ac1/cssd/ocssd.log
ocssd.log：
2019-03-19 17:14:56.025: [??? CSSD][1181738752]clssnmReadDiscoveryProfile: voting file discovery string(/dev/asm*)
2019-03-19 17:14:56.025: [??? CSSD][1181738752]clssnmvDDiscThread: using discovery string /dev/asm* for initial discovery
2019-03-19 17:14:56.025: [?? SKGFD][1181738752]Discovery with str:/dev/asm*:
2019-03-19 17:14:56.025: [?? SKGFD][1181738752]UFS discovery with :/dev/asm*:
2019-03-19 17:14:56.025: [?? SKGFD][1181738752]Execute glob on the string /dev/asm*
2019-03-19 17:14:56.025: [?? SKGFD][1181738752]running stat on disk:/dev/asm-diske
2019-03-19 17:14:56.026: [?? SKGFD][1181738752]running stat on disk:/dev/asm-diskd
2019-03-19 17:14:56.026: [?? SKGFD][1181738752]running stat on disk:/dev/asm-diskc
2019-03-19 17:14:56.027: [?? SKGFD][1181738752]running stat on disk:/dev/asm-diskb
2019-03-19 17:14:56.027: [?? SKGFD][1181738752]Fetching UFS disk :/dev/asm-diskb:
2019-03-19 17:14:56.027: [?? SKGFD][1181738752]Fetching UFS disk :/dev/asm-diskc:
2019-03-19 17:14:56.027: [?? SKGFD][1181738752]Fetching UFS disk :/dev/asm-diskd:
2019-03-19 17:14:56.027: [?? SKGFD][1181738752]Fetching UFS disk :/dev/asm-diske:
2019-03-19 17:14:56.027: [?? SKGFD][1181738752]OSS discovery with :/dev/asm*:
2019-03-19 17:14:56.027: [?? SKGFD][1181738752]Handle 0x7fe334136b60 from lib :UFS:: for disk :/dev/asm-diskb:
2019-03-19 17:14:56.027: [?? SKGFD][1181738752]Handle 0x7fe334130f60 from lib :UFS:: for disk :/dev/asm-diskc:
2019-03-19 17:14:56.028: [?? SKGFD][1181738752]Handle 0x7fe334131790 from lib :UFS:: for disk :/dev/asm-diskd:
2019-03-19 17:14:56.028: [?? SKGFD][1181738752]Handle 0x7fe33413f4a0 from lib :UFS:: for disk :/dev/asm-diske:
2019-03-19 17:14:56.028: [?? SKGFD][1181738752]Lib :UFS:: closing handle 0x7fe334136b60 for disk :/dev/asm-diskb:
2019-03-19 17:14:56.028: [?? SKGFD][1181738752]Lib :UFS:: closing handle 0x7fe334130f60 for disk :/dev/asm-diskc:
2019-03-19 17:14:56.028: [?? SKGFD][1181738752]Lib :UFS:: closing handle 0x7fe334131790 for disk :/dev/asm-diskd:
2019-03-19 17:14:56.028: [?? SKGFD][1181738752]Lib :UFS:: closing handle 0x7fe33413f4a0 for disk :/dev/asm-diske:
2019-03-19 17:14:56.028: [??? CSSD][1181738752]clssnmvDiskVerify: Successful discovery of 0 disks
2019-03-19 17:14:56.028: [??? CSSD][1181738752]clssnmCompleteInitVFDiscovery: Completing initial voting file discovery
2019-03-19 17:14:56.028: [??? CSSD][1181738752]clssnmvFindInitialConfigs: No voting files found
2019-03-19 17:14:56.028: [??? CSSD][1181738752](:CSSNM00070:)clssnmCompleteInitVFDiscovery: Voting file not found. Retrying discovery in 15 sec
onds
正式的恢复ocr和votedisk所在diskgroup的步骤如下：
清除grid所有进程。
1. 以-excl -nocrs 方式启动cluster，这将可以启动ASM实例 但不启动CRS
[[email?protected] bin]# ./crsctl start crs -excl -nocrs
CRS-4123: Oracle High Availability Services has been started.
CRS-2672: Attempting to start ‘ora.mdnsd‘ on ‘rh6rac1‘
CRS-2676: Start of ‘ora.mdnsd‘ on ‘rh6rac1‘ succeeded
CRS-2672: Attempting to start ‘ora.gpnpd‘ on ‘rh6rac1‘
CRS-2676: Start of ‘ora.gpnpd‘ on ‘rh6rac1‘ succeeded
CRS-2672: Attempting to start ‘ora.cssdmonitor‘ on ‘rh6rac1‘
CRS-2672: Attempting to start ‘ora.gipcd‘ on ‘rh6rac1‘
CRS-2676: Start of ‘ora.cssdmonitor‘ on ‘rh6rac1‘ succeeded
CRS-2676: Start of ‘ora.gipcd‘ on ‘rh6rac1‘ succeeded
CRS-2672: Attempting to start ‘ora.cssd‘ on ‘rh6rac1‘
CRS-2672: Attempting to start ‘ora.diskmon‘ on ‘rh6rac1‘
CRS-2676: Start of ‘ora.diskmon‘ on ‘rh6rac1‘ succeeded
CRS-2676: Start of ‘ora.cssd‘ on ‘rh6rac1‘ succeeded
CRS-2672: Attempting to start ‘ora.drivers.acfs‘ on ‘rh6rac1‘
CRS-2679: Attempting to clean ‘ora.cluster_interconnect.haip‘ on ‘rh6rac1‘
CRS-2672: Attempting to start ‘ora.ctssd‘ on ‘rh6rac1‘
CRS-2681: Clean of ‘ora.cluster_interconnect.haip‘ on ‘rh6rac1‘ succeeded
CRS-2672: Attempting to start ‘ora.cluster_interconnect.haip‘ on ‘rh6rac1‘
CRS-2676: Start of ‘ora.drivers.acfs‘ on ‘rh6rac1‘ succeeded
CRS-2676: Start of ‘ora.ctssd‘ on ‘rh6rac1‘ succeeded
CRS-2676: Start of ‘ora.cluster_interconnect.haip‘ on ‘rh6rac1‘ succeeded
CRS-2672: Attempting to start ‘ora.asm‘ on ‘rh6rac1‘
CRS-2676: Start of ‘ora.asm‘ on ‘rh6rac1‘ succeeded
2.重建原ocr和votedisk所在diskgroup,注意compatible.asm必须是11.2
[[email?protected] bin]# su - grid
[[email?protected] ~]$ sqlplus / as sysasm
SQL*Plus: Release 11.2.0.4.0 Production on Tue Mar 19 17:29:39 2019
Copyright (c) 1982,2013,Oracle.? All rights reserved.
Connected to:
Oracle Database 11g Enterprise Edition Release 11.2.0.4.0 - 64bit Production
With the Real Application Clusters and Automatic Storage Management options
SQL> create diskgroup OCRVOTE external redundancy disk ‘/dev/asm-diskb‘ ATTRIBUTE ‘compatible.rdbms‘ = ‘11.2‘,‘compatible.asm‘ = ‘11.2‘;
Diskgroup created.
3.从ocr backup中恢复ocr并做ocrcheck检验:
[[email?protected] bin]# ./ocrconfig -restore /oracle/grid/cdata/rh6rac-cluster/backup00.ocr
[[email?protected] bin]#
[[email?protected] bin]# ./ocrcheck
Status of Oracle Cluster Registry is as follows :
?? ? Version????????????????? :????????? 3
?? ? Total space (kbytes)???? :???? 262120
?? ? Used space (kbytes)????? :?????? 3084
?? ? Available space (kbytes) :???? 259036
?? ? ID?????????????????????? :? 276343585
?? ? Device/File Name???????? :?? +OCRVOTE
??????????????????????????????????? Device/File integrity check succeeded
??????????????????????????????????? Device/File not configured
??????????????????????????????????? Device/File not configured
??????????????????????????????????? Device/File not configured
??????????????????????????????????? Device/File not configured
?? ? Cluster registry integrity check succeeded
?? ? Logical corruption check succeeded
?? ?
4. 准备恢复votedisk ，可能会遇到下面的错误:
[[email?protected] bin]# ./crsctl replace votedisk? +OCRVOTE
CRS-4602: Failed 27 to add voting file 942480699ad84f50bfbd253181a05ad1.
Failed to replace voting disk group with +OCRVOTE.
CRS-4000: Command Replace failed,or completed with errors.
需要重新配置一下ASM的参数，并重启ASM:
SQL> alter system set asm_diskstring=‘/dev/asm*‘;
System altered.
SQL> create pfile from memory;
File created.
SQL> startup force mount
ORA-32004: obsolete or deprecated parameter(s) specified for ASM instance
ASM instance started
Total System Global Area 1135747072 bytes
Fixed Size?? ??? ???? 2260728 bytes
Variable Size?? ??? ? 1108320520 bytes
ASM Cache?? ??? ??? 25165824 bytes
ASM diskgroups mounted
ASM diskgroups volume enabled
[[email?protected] bin]# ./crsctl replace votedisk? +OCRVOTE
Successful addition of voting disk 59835fcfdc874f55bfccc60a6be79ca4.
Successfully replaced voting disk group with +OCRVOTE.
CRS-4266: Voting file(s) successfully replaced
5. 重启has服务，检验cluster是否正常：
[[email?protected] bin]# ./crsctl stop has -f
[[email?protected] bin]# ./crsctl start has
CRS-4123: Oracle High Availability Services has been started.
[[email?protected] bin]# ./crsctl check crs
CRS-4638: Oracle High Availability Services is online
CRS-4535: Cannot communicate with Cluster Ready Services
CRS-4529: Cluster Synchronization Services is online
CRS-4533: Event Manager is online
[[email?protected] bin]# ./crsctl query css votedisk
##? STATE??? File Universal Id??????????????? File Name Disk group
--? -----??? -----------------??????????????? --------- ---------
?1. ONLINE?? 59835fcfdc874f55bfccc60a6be79ca4 (/dev/asm-diskb) [OCRVOTE]
Located 1 voting disk(s).
查看crs状态
./crsctl status res -t
（编辑：李大同）
【声明】本站内容均来自网络，其相关言论仅代表作者个人观点，不代表本站立场。若无意侵犯到您的权利，请及时与联系站长删除相关内容!
|
