Need urgent help on ASM issue – disk header status problem

If you cannot recover the data by yourself, ask Parnassusdata, the professional ORACLE database recovery team for help.

Parnassusdata Software Database Recovery Team

Service Hotline:  +86 13764045638 E-mail: service@parnassusdata.com

 

 

 
ODA system
+In order to workaround a known issue (startup hang when using Hitachi disks) FE/customer was in the process of replacing Hitachi drives on the system.
+ They pulled 2 disks out simultaneously and new disks put in
+ Diskgroups (DATA and RECO) dismounted  – as diskgroups built in  NORMAL redundancy .
+ Clusterware went down and realizing the problem- customer reinstated the original disks.

Current issue:
Disk groups are not mounting.
ASM disks from slot 0 are not being seen by ASM
ASM disks from slot 1 are being seen, but reported as new disks ( Header status=CANDIDATE)

Mounting the diskgroup with FORCE option has also not helped ( because there is 1 disk from slot 0 missing and 1 disk from slot 1 being reported as candidate)
** Customer has no backup and he needs to find out if it is fixable, or the system needs to be rebuilt from scratch.

 

--------------------------------------------------------------------------------
Disk          Size Header    Path                                     Disk Group   User     Group   
================================================================================
1:     491520 Mb CANDIDATE /dev/mapper/HDD_E0_S01_717882548p1       #            grid     asmadmin
2:      75080 Mb CANDIDATE /dev/mapper/HDD_E0_S01_717882548p2       #            grid     asmadmin
3:     491520 Mb MEMBER    /dev/mapper/HDD_E0_S04_717894368p1       DATA         grid     asmadmin
4:      75080 Mb MEMBER    /dev/mapper/HDD_E0_S04_717894368p2       RECO         grid     asmadmin
5:     491520 Mb MEMBER    /dev/mapper/HDD_E0_S05_717844560p1       DATA         grid     asmadmin
6:      75080 Mb MEMBER    /dev/mapper/HDD_E0_S05_717844560p2       RECO         grid     asmadmin
7:     491520 Mb MEMBER    /dev/mapper/HDD_E0_S08_717882264p1       DATA         grid     asmadmin
8:      75080 Mb MEMBER    /dev/mapper/HDD_E0_S08_717882264p2       RECO         grid     asmadmin
9:     491520 Mb MEMBER    /dev/mapper/HDD_E0_S09_717844480p1       DATA         grid     asmadmin
10:      75080 Mb MEMBER    /dev/mapper/HDD_E0_S09_717844480p2       RECO         grid     asmadmin
11:     491520 Mb MEMBER    /dev/mapper/HDD_E0_S12_717844976p1       DATA         grid     asmadmin
12:      75080 Mb MEMBER    /dev/mapper/HDD_E0_S12_717844976p2       RECO         grid     asmadmin
13:     491520 Mb MEMBER    /dev/mapper/HDD_E0_S13_717845048p1       DATA         grid     asmadmin
14:      75080 Mb MEMBER    /dev/mapper/HDD_E0_S13_717845048p2       RECO         grid     asmadmin
15:     491520 Mb MEMBER    /dev/mapper/HDD_E0_S16_717895116p1       DATA         grid     asmadmin
16:      75080 Mb MEMBER    /dev/mapper/HDD_E0_S16_717895116p2       RECO         grid     asmadmin
17:     491520 Mb MEMBER    /dev/mapper/HDD_E0_S17_717888848p1       DATA         grid     asmadmin
18:      75080 Mb MEMBER    /dev/mapper/HDD_E0_S17_717888848p2       RECO         grid     asmadmin
19:     491520 Mb MEMBER    /dev/mapper/HDD_E1_S02_717825396p1       DATA         grid     asmadmin
20:      75080 Mb MEMBER    /dev/mapper/HDD_E1_S02_717825396p2       RECO         grid     asmadmin
21:     491520 Mb MEMBER    /dev/mapper/HDD_E1_S03_717894252p1       DATA         grid     asmadmin
22:      75080 Mb MEMBER    /dev/mapper/HDD_E1_S03_717894252p2       RECO         grid     asmadmin
23:     491520 Mb MEMBER    /dev/mapper/HDD_E1_S06_717886840p1       DATA         grid     asmadmin
24:      75080 Mb MEMBER    /dev/mapper/HDD_E1_S06_717886840p2       RECO         grid     asmadmin
25:     491520 Mb MEMBER    /dev/mapper/HDD_E1_S07_717888592p1       DATA         grid     asmadmin
26:      75080 Mb MEMBER    /dev/mapper/HDD_E1_S07_717888592p2       RECO         grid     asmadmin
27:     491520 Mb MEMBER    /dev/mapper/HDD_E1_S10_717843708p1       DATA         grid     asmadmin
28:      75080 Mb MEMBER    /dev/mapper/HDD_E1_S10_717843708p2       RECO         grid     asmadmin
29:     491520 Mb MEMBER    /dev/mapper/HDD_E1_S11_717852256p1       DATA         grid     asmadmin
30:      75080 Mb MEMBER    /dev/mapper/HDD_E1_S11_717852256p2       RECO         grid     asmadmin
31:     491520 Mb MEMBER    /dev/mapper/HDD_E1_S14_717895376p1       DATA         grid     asmadmin
32:      75080 Mb MEMBER    /dev/mapper/HDD_E1_S14_717895376p2       RECO         grid     asmadmin
33:     491520 Mb MEMBER    /dev/mapper/HDD_E1_S15_717843800p1       DATA         grid     asmadmin
34:      75080 Mb MEMBER    /dev/mapper/HDD_E1_S15_717843800p2       RECO         grid     asmadmin
35:     491520 Mb MEMBER    /dev/mapper/HDD_E1_S18_717882696p1       DATA         grid     asmadmin
36:      75080 Mb MEMBER    /dev/mapper/HDD_E1_S18_717882696p2       RECO         grid     asmadmin
37:     491520 Mb MEMBER    /dev/mapper/HDD_E1_S19_717849420p1       DATA         grid     asmadmin
38:      75080 Mb MEMBER    /dev/mapper/HDD_E1_S19_717849420p2       RECO         grid     asmadmin
39:      70005 Mb MEMBER    /dev/mapper/SSD_E0_S20_805725574p1       REDO         grid     asmadmin
40:      70005 Mb MEMBER    /dev/mapper/SSD_E0_S21_805708282p1       REDO         grid     asmadmin
41:      70005 Mb MEMBER    /dev/mapper/SSD_E1_S22_805706766p1       REDO         grid     asmadmin
42:      70005 Mb MEMBER    /dev/mapper/SSD_E1_S23_805706623p1       REDO         grid     asmadmin
--------------------------------------------------------------------------------
ORACLE_SID ORACLE_HOME                                                          
================================================================================
+ASM1 /u01/app/11.2.0.3/grid                                               
+ASM2 /u01/app/11.2.0.3/grid                                               

 
What is the backup block status ,

kfed read <device_name> aunum=1 blknum=254

Does it shows proper header ,if so the run kfed repair command.

if other blocks are fine except header this will work … else on next mount while doing COD recovery ,it will crash .

 

ASM log file info
==============

NOTE: cache closing disk 0 of grp 1: (not open) _DROPPED_0000_DATA
ERROR: Disk 1 cannot be offlined, since all the disks [1, 0] with mirrored data would be offline.
ERROR: too many offline disks in PST (grp 1)

Here the disks 0,1 have been put back in respective slots but still same issue.

 

Seems similar to the one described in ORA-15042: ASM disk is missing after add disk took place (Doc ID 1529397.1)

 

Comment

*

沪ICP备14014813号

沪公网安备 31010802001379号