【Oracle RAC调优】RAC多节点使用不同的gcs_server_processes参数可能导致gc cr multi block request等待事件; 例如一个RAC实例 的 gcs_server_processes=5 而另一个实例为gcs_server_processes=4 就可能引起额外的gc cr multi block request等待事件， 该问题在RAC 11.2.0.3 中仍存在。 建议是使用相同的gcs_server_processes。
遇到该gc cr multi block request等待事件，在参数方面建议检查：
show parameter gcs_server_process
show parameter cpu_count
show parameter db_file_multiblock_read_count
OS参数 ：udp_recvspace 、 udp_sendspace
NUM_CPUS NUM_CPU_CORE
Yes, it is supported/allowed to set different GCS_SERVER_PROCESSES for RAC instances.
This is confirmed in the Online Doc of both 10.2 and 11.2:
10.2:
http://docs.oracle.com/cd/B19306_01/server.102/b14237/initparams075.htm#REFRN10259
Real Application Clusters: Multiple instances can have different values.
11.2:
http://docs.oracle.com/cd/E11882_01/server.112/e25513/initparams094.htm#REFRN10259
Oracle RAC Multiple instances can have different values.
There is a Bug 11693109 which can lead to ORA-481 during DRM when there is a rather unbalanced LMS set up.
This Bug has been fixed by 11.2.0.3.
2. For RAC database, we recommend to leave oracle to determine the initial value of GCS_SERVER_PROCESSES when all your CPUs are in place. However if you have encountered serious GC waits which is caused by lack of LMSs, you may need to increase the number of LMS process. The number of database instances running on a given cluster node should be limited such that the total number of real-time LMS processes on a node does not exceed the number of #CPUs(core) –
GCS_SERVER_PROCESSES
LMS Real Time Priority in RAC 10g Release 2 – Things to Consider Before Changing (Doc ID 433105.1)
Version: 10.2.0.1 to 11.1.0.6
The rule of thumb is that you should keep the number of real time LMS processes to be at least 1 less than the number of CPUs (cores) in the system.
RAC and Oracle Clusterware Best Practices and Starter Kit (Platform Independent) (Doc ID 810394.1)
Version 10.2.0.1 to 11.2.0.3
The number of database instances running on a given cluster node should be limited such that the total number of real-time LMS processes on a node
does not exceed the recommended number of #CPUs – 1.
10gR2 RAC Reference (INTERNAL ONLY) (Doc ID 341965.1)
the number of LMS processes can be set with the parameter GCS_SERVER_PROCESSES, and the value is dependent on the number of CPUs. At startup time, we start (CPU_COUNT / 4) LMS processes, but there are always at least two LMS processes.
Auto-Adjustment of LMS Process Priority in Oracle RAC with 11.2.0.3 and later (Doc ID 1392248.1)
For 11.2.0.3 Oracle RAC database instances, the number of LMS processes that will be started can be calculated using the following formula:
n = “number of CPUs reported by the OS and used for CPU_COUNT per default”
n < 4 => 1 LMS process will be started
4 <= n < 16 => 2 LMS processes will be started
n >=16 => 2 LMS + 1 LMS processes for every 32 CPU will be started.
Keywords:lms unbalanced
Bug 11693109 – ORA-481 during DRM results in LMS communication issues between nodes / hang (Doc ID 11693109.8)
ORA-481 during DRM, which results in LMS communication issues between nodes.
This problem can occur if there is a rather unbalanced LMS set up
eg: where there is 4 LMSn on Node1 and Node6 and 16 LMSn on Node2/Node3/Node4/Node5
<==Already fixed by 11.2.0.3
# no -a | grep ipqmaxlen
ipqmaxlen = 512
# no -a | grep rfc1323
rfc1323 = 1
# no -a | grep sb_max
sb_max = 41943040
# no -a | grep tcp_recvspace
tcp_recvspace = 65536
# no -a | grep tcp_sendspace
tcp_sendspace = 65536
# no -a | grep udp_recvspace
udp_recvspace = 20971520
# no -a | grep udp_sendspace
udp_sendspace = 2097152
IBM POWER7 AIX and Oracle Database performance considerations — 10g & 11g (Doc ID 1507249.1)
These values are generally suggested for Oracle, and can be considered as starting points:
· sb_max >= 1MB (1048576) and must be greater than maximum tpc or udp send or recvspace (if you are using RAC and very large udp_recvspace, you might need to increase sb_max)
· tcp_sendspace = 262144
· tcp_recvspace = 262144
· udp_sendspace = db_block_size * db_file_multiblock_read_count
· udp_recvspace= 10 * (udp_sendspace)
· rfc1323 = 1 (see Recent suggestions and open issues)
· Ephemerals (non-defaults suggested for a large number of connecting hosts or a high degree of parallel query; also to avoid install-time warnings)
· tcp_ephemeral_low=9000
· tcp_ephemeral_high=65500
· udp_ephemeral_low=9000
· udp_ephemeral_high=65500
——————————————————-
How many database in the node and how many cpus in the node?
Ct may increase gcs_server_processes if possible
refer to Doc ID 558185.1 for general guideline
LMS and Real Time Priority in Oracle RAC 10g and 11g (Doc ID 558185.1)
Oracle Database (RDBMS) on Unix AIX,HP-UX,Linux,Mac OS X,Solaris,Tru64 Unix Operating Systems Installation and Configuration Requirements
Quick Reference (8.0.5 to 11.2) (Doc ID 169706.1)
1.Please run the below output.
lsattr -E -l sys0 no -a
show parameter db_file
2.Refer the below doc Doc ID 169706.1
From Section 2.8.2 of the 11.2 Install Guide for AIX:
3.Please upload awr /ash report after you reset db_file_multiblock_read_count=8
4.http://docs.oracle.com/cd/E11882_01/install.112/e24332/pre_install.htm#BABHAEDH
$ netstat -s |grep overflow
0 socket buffer overflows <=======Not found
0 ipintrq overflows
1) gc cr multi block request is an expected latency when your db file multiblock read count parameter is set to too high and not 8 or 16….reduce it
2) gc cr multi block request is an expected latency when your UDP related kernel parameters like udp_sendspace and udp_receivespace is not set to as per recommended value of 65536
3) gc cr multi block request is an expected latency when your LMS processes are not running in real time high priority:-
LMS and Real Time Priority in Oracle RAC 10g and 11g (Doc ID 558185.1)
Auto-Adjustment of LMS Process Priority in Oracle RAC with 11.2.0.3 and later (Doc ID 1392248.1)
NOTE:840982.1 – How to check lms process is in Real Time mode or not at Linux
Comment