存在这种可能性，即ORACLE ASM在add disk扩盘时add disk操作正常完成，disk group的rebalance其实还没有开始，但是由于新加入的disk存在硬件故障，导致add disk后写入到disk header的所有metadata元数据全部丢失，且由于diskgroup是外部冗余即EXTERNAL REdundancy所以该diskgroup由于已经加入了一个DISK，而该DISK上的metadata全部丢失的缘故，所以该diskgroup 将无法正常MOUNT。

且由于新加入的disk上的所有metadata都丢失了，而不仅仅是丢失了disk header的KFBTYP_DISKHEAD，所以还不能是仅仅将KFBTYP_DISKHEAD的信息通过kfed merge其他的disk信息并做修改来还原，其需要通过特殊的手工处理才能绕过该问题。

如下面的例子:

 

SUCCESS: diskgroup TESTDG03 was created
NOTE: cache deleting context for group TESTDG03 1/0x86485c30
NOTE: cache registered group TESTDG03 number=1 incarn=0xab385c36
NOTE: cache began mount (first) of group TESTDG03 number=1 incarn=0xab385c36
NOTE: Assigning number (1,3) to disk (/oracleasm/asm-disk04)
NOTE: Assigning number (1,2) to disk (/oracleasm/asm-disk03)
NOTE: Assigning number (1,1) to disk (/oracleasm/asm-disk02)
NOTE: Assigning number (1,0) to disk (/oracleasm/asm-disk01)
Thu Jan 29 08:21:07 2015
NOTE: GMON heartbeating for grp 1
GMON querying group 1 at 92 for pid 20, osid 20176
Thu Jan 29 08:21:07 2015
NOTE: cache opening disk 0 of grp 1: TESTDG03_0000 path:/oracleasm/asm-disk01
NOTE: F1X0 found on disk 0 au 2 fcn 0.0
NOTE: cache opening disk 1 of grp 1: TESTDG03_0001 path:/oracleasm/asm-disk02
NOTE: cache opening disk 2 of grp 1: TESTDG03_0002 path:/oracleasm/asm-disk03
NOTE: cache opening disk 3 of grp 1: TESTDG03_0003 path:/oracleasm/asm-disk04
NOTE: cache mounting (first) external redundancy group 1/0xAB385C36 (TESTDG03)
NOTE: cache recovered group 1 to fcn 0.0
NOTE: redo buffer size is 256 blocks (1053184 bytes)
Thu Jan 29 08:21:07 2015
NOTE: LGWR attempting to mount thread 1 for diskgroup 1 (TESTDG03)
NOTE: LGWR found thread 1 closed at ABA 0.10750
NOTE: LGWR mounted thread 1 for diskgroup 1 (TESTDG03)
NOTE: LGWR opening thread 1 at fcn 0.0 ABA 2.0
NOTE: setting 11.2 start ABA for group TESTDG03 thread 1 to 2.0
NOTE: cache mounting group 1/0xAB385C36 (TESTDG03) succeeded
NOTE: cache ending mount (success) of group TESTDG03 number=1 incarn=0xab385c36
GMON querying group 1 at 93 for pid 13, osid 4612
Thu Jan 29 08:21:07 2015
NOTE: Instance updated compatible.asm to 11.2.0.0.0 for grp 1
SUCCESS: diskgroup TESTDG03 was mounted
SUCCESS: CREATE DISKGROUP TESTDG03 EXTERNAL REDUNDANCY  DISK '/oracleasm/asm-disk01' SIZE 129500M ,
'/oracleasm/asm-disk02' SIZE 128800M ,
'/oracleasm/asm-disk03' SIZE 129200M ,
'/oracleasm/asm-disk04' SIZE 128800M  ATTRIBUTE 'compatible.asm'='11.2.0.0.0','au_size'='1M' /* ASMCA */
Thu Jan 29 08:21:07 2015
NOTE: diskgroup resource ora.TESTDG03.dg is online
NOTE: diskgroup resource ora.TESTDG03.dg is updated
Thu Jan 29 08:21:23 2015
SQL> alter diskgroup testdg03  add disk '/oracleasm/asm-disk06' 
ORA-15032: not all alterations performed
ORA-15260: permission denied on ASM disk group
ERROR: alter diskgroup testdg03  add disk '/oracleasm/asm-disk06'
Thu Jan 29 08:21:31 2015
SQL> alter diskgroup testdg03  add disk '/oracleasm/asm-disk06' 
NOTE: Assigning number (1,4) to disk (/oracleasm/asm-disk06)
NOTE: requesting all-instance membership refresh for group=1
NOTE: initializing header on grp 1 disk TESTDG03_0004
NOTE: requesting all-instance disk validation for group=1
Thu Jan 29 08:21:32 2015
NOTE: skipping rediscovery for group 1/0xab385c36 (TESTDG03) on local instance.
NOTE: requesting all-instance disk validation for group=1
NOTE: skipping rediscovery for group 1/0xab385c36 (TESTDG03) on local instance.
NOTE: initiating PST update: grp = 1
Thu Jan 29 08:21:32 2015
GMON updating group 1 at 94 for pid 21, osid 22706
NOTE: PST update grp = 1 completed successfully 
NOTE: membership refresh pending for group 1/0xab385c36 (TESTDG03)
GMON querying group 1 at 95 for pid 13, osid 4612
NOTE: cache opening disk 4 of grp 1: TESTDG03_0004 path:/oracleasm/asm-disk06
GMON querying group 1 at 96 for pid 13, osid 4612
SUCCESS: refreshed membership for 1/0xab385c36 (TESTDG03)
SUCCESS: alter diskgroup testdg03  add disk '/oracleasm/asm-disk06'
NOTE: Attempting voting file refresh on diskgroup TESTDG03
Thu Jan 29 08:22:09 2015
SQL> alter diskgroup testdg03 dismount 
NOTE: cache dismounting (clean) group 1/0xAB385C36 (TESTDG03) 
NOTE: messaging CKPT to quiesce pins Unix process pid: 22730, image: oracle@mlab2.oracle.com (TNS V1-V3)
Thu Jan 29 08:22:10 2015
NOTE: LGWR doing clean dismount of group 1 (TESTDG03)
NOTE: LGWR closing thread 1 of diskgroup 1 (TESTDG03) at ABA 2.15
NOTE: cache dismounted group 1/0xAB385C36 (TESTDG03) 
Thu Jan 29 08:22:10 2015
GMON dismounting group 1 at 97 for pid 21, osid 22730
NOTE: Disk  in mode 0x8 marked for de-assignment
NOTE: Disk  in mode 0x8 marked for de-assignment
NOTE: Disk  in mode 0x8 marked for de-assignment
NOTE: Disk  in mode 0x8 marked for de-assignment
NOTE: Disk  in mode 0x8 marked for de-assignment
SUCCESS: diskgroup TESTDG03 was dismounted
NOTE: cache deleting context for group TESTDG03 1/0xab385c36
Thu Jan 29 08:22:10 2015
NOTE: diskgroup resource ora.TESTDG03.dg is offline
SUCCESS: alter diskgroup testdg03 dismount
NOTE: diskgroup resource ora.TESTDG03.dg is updated
SQL> alter diskgroup testdg03 mount 
NOTE: cache registered group TESTDG03 number=1 incarn=0x83f85c5f
NOTE: cache began mount (first) of group TESTDG03 number=1 incarn=0x83f85c5f
NOTE: Assigning number (1,3) to disk (/oracleasm/asm-disk04)
NOTE: Assigning number (1,2) to disk (/oracleasm/asm-disk03)
NOTE: Assigning number (1,1) to disk (/oracleasm/asm-disk02)
NOTE: Assigning number (1,0) to disk (/oracleasm/asm-disk01)
Thu Jan 29 08:22:22 2015
NOTE: GMON heartbeating for grp 1
GMON querying group 1 at 100 for pid 21, osid 22730
Thu Jan 29 08:22:22 2015
NOTE: Assigning number (1,4) to disk ()
GMON querying group 1 at 101 for pid 21, osid 22730
NOTE: cache dismounting (clean) group 1/0x83F85C5F (TESTDG03) 
NOTE: messaging CKPT to quiesce pins Unix process pid: 22730, image: oracle@mlab2.oracle.com (TNS V1-V3)
NOTE: dbwr not being msg'd to dismount
NOTE: lgwr not being msg'd to dismount
NOTE: cache dismounted group 1/0x83F85C5F (TESTDG03) 
NOTE: cache ending mount (fail) of group TESTDG03 number=1 incarn=0x83f85c5f
NOTE: cache deleting context for group TESTDG03 1/0x83f85c5f
GMON dismounting group 1 at 102 for pid 21, osid 22730
NOTE: Disk  in mode 0x8 marked for de-assignment
NOTE: Disk  in mode 0x8 marked for de-assignment
NOTE: Disk  in mode 0x8 marked for de-assignment
NOTE: Disk  in mode 0x8 marked for de-assignment
NOTE: Disk  in mode 0x8 marked for de-assignment
ERROR: diskgroup TESTDG03 was not mounted
ORA-15032: not all alterations performed
ORA-15040: diskgroup is incomplete
ORA-15042: ASM disk "4" is missing from group number "1" 
ERROR: alter diskgroup testdg03 mount
Thu Jan 29 08:27:37 2015
SQL> alter diskgroup testdg03 mount 
NOTE: cache registered group TESTDG03 number=1 incarn=0x56985c64
NOTE: cache began mount (first) of group TESTDG03 number=1 incarn=0x56985c64
NOTE: Assigning number (1,3) to disk (/oracleasm/asm-disk04)
NOTE: Assigning number (1,2) to disk (/oracleasm/asm-disk03)
NOTE: Assigning number (1,1) to disk (/oracleasm/asm-disk02)
NOTE: Assigning number (1,0) to disk (/oracleasm/asm-disk01)
Thu Jan 29 08:27:43 2015
NOTE: GMON heartbeating for grp 1
GMON querying group 1 at 105 for pid 21, osid 23017
NOTE: cache opening disk 0 of grp 1: TESTDG03_0000 path:/oracleasm/asm-disk01
NOTE: F1X0 found on disk 0 au 2 fcn 0.0
NOTE: cache opening disk 1 of grp 1: TESTDG03_0001 path:/oracleasm/asm-disk02
NOTE: cache opening disk 2 of grp 1: TESTDG03_0002 path:/oracleasm/asm-disk03
NOTE: cache opening disk 3 of grp 1: TESTDG03_0003 path:/oracleasm/asm-disk04
NOTE: cache mounting (first) external redundancy group 1/0x56985C64 (TESTDG03)
NOTE: cache recovered group 1 to fcn 0.609
NOTE: redo buffer size is 256 blocks (1053184 bytes)
Thu Jan 29 08:27:43 2015
NOTE: LGWR attempting to mount thread 1 for diskgroup 1 (TESTDG03)
NOTE: LGWR found thread 1 closed at ABA 2.15
NOTE: LGWR mounted thread 1 for diskgroup 1 (TESTDG03)
NOTE: LGWR opening thread 1 at fcn 0.609 ABA 3.16
NOTE: cache mounting group 1/0x56985C64 (TESTDG03) succeeded
NOTE: cache ending mount (success) of group TESTDG03 number=1 incarn=0x56985c64
GMON querying group 1 at 106 for pid 13, osid 4612
Thu Jan 29 08:27:43 2015
NOTE: Instance updated compatible.asm to 11.2.0.0.0 for grp 1
SUCCESS: diskgroup TESTDG03 was mounted
SUCCESS: alter diskgroup testdg03 mount
Thu Jan 29 08:27:43 2015
NOTE: diskgroup resource ora.TESTDG03.dg is online
NOTE: diskgroup resource ora.TESTDG03.dg is updated
Thu Jan 29 08:33:52 2015
SQL> alter diskgroup testdg03 check all norepair 
NOTE: starting check of diskgroup TESTDG03
Thu Jan 29 08:33:52 2015
GMON checking disk 0 for group 1 at 107 for pid 21, osid 23017
GMON checking disk 1 for group 1 at 108 for pid 21, osid 23017
GMON checking disk 2 for group 1 at 109 for pid 21, osid 23017
GMON checking disk 3 for group 1 at 110 for pid 21, osid 23017
ERROR: no kfdsk for  (4)
ERROR: check of diskgroup TESTDG03 found 1 total errors
ORA-15049: diskgroup "TESTDG03" contains 1 error(s)
ORA-15032: not all alterations performed
ORA-15049: diskgroup "TESTDG03" contains 1 error(s)
ERROR: alter diskgroup testdg03 check all norepair
Thu Jan 29 08:34:07 2015
SQL> alter diskgroup testdg03 check all 
NOTE: starting check of diskgroup TESTDG03
Thu Jan 29 08:34:07 2015
GMON checking disk 0 for group 1 at 111 for pid 21, osid 23017
GMON checking disk 1 for group 1 at 112 for pid 21, osid 23017
GMON checking disk 2 for group 1 at 113 for pid 21, osid 23017
GMON checking disk 3 for group 1 at 114 for pid 21, osid 23017
ERROR: no kfdsk for  (4)
ERROR: check of diskgroup TESTDG03 found 1 total errors
ORA-15049: diskgroup "TESTDG03" contains 1 error(s)
ORA-15032: not all alterations performed
ORA-15049: diskgroup "TESTDG03" contains 1 error(s)
ERROR: alter diskgroup testdg03 check all
SQL> alter diskgroup testdg03 check all repair 
NOTE: starting check of diskgroup TESTDG03
GMON checking disk 0 for group 1 at 115 for pid 21, osid 23017
GMON checking disk 1 for group 1 at 116 for pid 21, osid 23017
GMON checking disk 2 for group 1 at 117 for pid 21, osid 23017
GMON checking disk 3 for group 1 at 118 for pid 21, osid 23017
ERROR: no kfdsk for  (4)
ERROR: check of diskgroup TESTDG03 found 1 total errors
ORA-15049: diskgroup "TESTDG03" contains 1 error(s)
ORA-15032: not all alterations performed
ORA-15049: diskgroup "TESTDG03" contains 1 error(s)
ERROR: alter diskgroup testdg03 check all repair

[oracle@mlab2 oracleasm]$ oerr ora 15042
15042, 00000, “ASM disk \”%s\” is missing from group number \”%s\” ”
// *Cause:  The specified disk, which is a necessary part of a diskgroup,
//          could not be found on the system.
// *Action: Check the hardware configuration.
//

ORA-15042错误正是因为add disk的磁盘上的metadata全部丢失了，但搞笑的时候新加入的盘上可能因为还没有开始rebalance而没有一点真正有意义的数据，但因为ASM认为该disk已经add进来了，所以必须要该disk可用才能mount diskgroup。 而且用户甚至无法强制DROP这个DISK，原因是需要DISKGROUP在MOUNT状态下才可以drop disk， 这就变成了鸡生蛋 蛋生鸡的死循环， 要DROP这个disk必须MOUNT DISKGROUP，但要MOUNT DISKGROUP要先DROP该DISK。

 

对于此问题一般需要诗檀软件工程师手动修改ASM metadata来绕过问题，或者如果有之前的ASM metadata也可以采用。

 

如果自己搞不定可以找诗檀软件专业ORACLE数据库修复团队成员帮您恢复!

诗檀软件专业数据库修复团队

服务热线 ： 13764045638   QQ号:47079569    邮箱：service@parnassusdata.com

如果你在使用ASM时遇到ASM Diskgroup无法MOUNT且ASM实例的alert.log中出现以下错误的话：

ERROR: no read quorum in group: required 1, found 0 disks

ORA-15032: not all alterations performed
ORA-15017: diskgroup “TESTDG04” cannot be mounted
ORA-15063: ASM discovered an insufficient number of disks for diskgroup “TESTDG04”
ERROR: /* ASMCMD */ALTER DISKGROUP testdg04 MOUNT

那么可能说明该ASM Diskgroup上的部分Metadata元数据已经存在逻辑损坏/不一致了，对于这种场景可以尝试先用kfed repair来修复  （10.2.0.5版本以后)

但如果kfed repair无法搞定的话，那么一般需要对ASM底层结构十分熟悉的专业技术人员来手工修复逻辑损坏了

 

 

ASMCMD> mount testdg04
ORA-15032: not all alterations performed
ORA-15017: diskgroup "TESTDG04" cannot be mounted
ORA-15063: ASM discovered an insufficient number of disks for diskgroup "TESTDG04" (DBD ERROR: OCIStmtExecute)

SQL> /* ASMCMD */ALTER DISKGROUP testdg04 MOUNT
NOTE: cache registered group TESTDG04 number=1 incarn=0x22485c89
NOTE: cache began mount (first) of group TESTDG04 number=1 incarn=0x22485c89
NOTE: Assigning number (1,0) to disk (/oracleasm/asm-disk01)
Fri Jan 30 02:33:05 2015
ERROR: no read quorum in group: required 1, found 0 disks
NOTE: cache dismounting (clean) group 1/0x22485C89 (TESTDG04)
NOTE: messaging CKPT to quiesce pins Unix process pid: 8724, image: oracle@mlab2.oracle.com (TNS V1-V3)
NOTE: dbwr not being msg'd to dismount
NOTE: lgwr not being msg'd to dismount
NOTE: cache dismounted group 1/0x22485C89 (TESTDG04)
NOTE: cache ending mount (fail) of group TESTDG04 number=1 incarn=0x22485c89
NOTE: cache deleting context for group TESTDG04 1/0x22485c89
GMON dismounting group 1 at 139 for pid 20, osid 8724
NOTE: Disk  in mode 0x8 marked for de-assignment
ERROR: diskgroup TESTDG04 was not mounted
ORA-15032: not all alterations performed
ORA-15017: diskgroup "TESTDG04" cannot be mounted
ORA-15063: ASM discovered an insufficient number of disks for diskgroup "TESTDG04"
ERROR: /* ASMCMD */ALTER DISKGROUP testdg04 MOUNT
Fri Jan 30 02:33:05 2015
NOTE: No asm libraries found in the system



SQL> create diskgroup testdg04 external redundancy disk '/oracleasm/asm-disk01';

Diskgroup created.


[oracle@mlab2 ~]$ kfed read /oracleasm/asm-disk01 aun=1 blkn=0|less
kfbh.endian:                          1 ; 0x000: 0x01
kfbh.hard:                          130 ; 0x001: 0x82
kfbh.type:                           17 ; 0x002: KFBTYP_PST_META
kfbh.datfmt:                          1 ; 0x003: 0x01
kfbh.block.blk:                     256 ; 0x004: blk=256
kfbh.block.obj:              2147483648 ; 0x008: disk=0
kfbh.check:                  4150372545 ; 0x00c: 0xf761a8c1
kfbh.fcn.base:                        0 ; 0x010: 0x00000000
kfbh.fcn.wrap:                        0 ; 0x014: 0x00000000
kfbh.spare1:                          0 ; 0x018: 0x00000000
kfbh.spare2:                          0 ; 0x01c: 0x00000000
kfdpHdrBv0.time.hi:            33015746 ; 0x000: HOUR=0x2 DAYS=0x1e MNTH=0x1 YEAR=0x7df
kfdpHdrBv0.time.lo:          2005330944 ; 0x004: USEC=0x0 MSEC=0x1bb SECS=0x38 MINS=0x1d
kfdpHdrBv0.last:                      2 ; 0x008: 0x00000002
kfdpHdrBv0.next:                      2 ; 0x00c: 0x00000002
kfdpHdrBv0.copyCnt:                   1 ; 0x010: 0x01
kfdpHdrBv0.version:                   0 ; 0x011: 0x00
kfdpHdrBv0.ub2spare:                  0 ; 0x012: 0x0000
kfdpHdrBv0.incarn:                    1 ; 0x014: 0x00000001
kfdpHdrBv0.copy[0]:                   0 ; 0x018: 0x0000
kfdpHdrBv0.copy[1]:                   0 ; 0x01a: 0x0000
kfdpHdrBv0.copy[2]:                   0 ; 0x01c: 0x0000
kfdpHdrBv0.copy[3]:                   0 ; 0x01e: 0x0000
kfdpHdrBv0.copy[4]:                   0 ; 0x020: 0x0000
kfdpHdrBv0.dtaSz:                     1 ; 0x022: 0x0001
ub1[0]:                               2 ; 0x024: 0x02
ub1[1]:                               0 ; 0x025: 0x00
ub1[2]:                               0 ; 0x026: 0x00
ub1[3]:                               0 ; 0x027: 0x00
ub1[4]:                               0 ; 0x028: 0x00
ub1[5]:                               0 ; 0x029: 0x00
ub1[6]:                               0 ; 0x02a: 0x00
ub1[7]:                               0 ; 0x02b: 0x00
ub1[8]:                               0 ; 0x02c: 0x00
ub1[9]:                               0 ; 0x02d: 0x00

如果自己搞不定可以找诗檀软件专业ORACLE数据库修复团队成员帮您恢复!

诗檀软件专业数据库修复团队

服务热线 ： 13764045638   QQ号:47079569    邮箱：service@parnassusdata.com

 

ORACLE PRM是诗檀软件独立研发的ORACLE数据库灾难恢复软件，其具有全程图形化界面、简单高效等特点。

欢迎下载使用ORACLE PRM。 下载地址：http://parnassusdata.com/sites/default/files/ParnassusData_PRMForOracle_3206.zip

PRM 用户使用手册。http://www.parnassusdata.com/sites/default/files /ParnassusData%20Recovery%20Manager%20For%20Oracle%20Database%E7%94%A8%E6%88%B7%E6%89%8B%E5%86%8C%20v0.3.pdf

[oracle@mlab2 trace]$ oerr ora 15063
15063, 00000, “ASM discovered an insufficient number of disks for diskgroup \”%s\””
// *Cause:  ASM was unable to find a sufficient number of disks belonging to the
//          diskgroup to continue the operation.
// *Action: Check that the disks in the diskgroup are present and functioning,
//          that the owner of the ORACLE binary has read/write permission to
//          the disks, and that the ASM_DISKSTRING initialization parameter
//          has been set correctly.  Verify that ASM discovers the appropriate
//          disks by querying V$ASM_DISK from the ASM instance.
//
[oracle@mlab2 trace]$ oerr ora 15017
15017, 00000, “diskgroup \”%s\” cannot be mounted”
// *Cause:  The disk group could not be mounted by the ASM instance either
//          of an IO error or due to other reasons explained in alert log or
//          trace files.
// *Action: Check for additional errors reported. Correct the error and
//          retry the operation.
//
[oracle@mlab2 trace]$ oerr ora 15032
15032, 00000, “not all alterations performed”
// *Cause:  At least one ALTER DISKGROUP action failed.
// *Action: Check the other messages issued along with this summary error.
//