【11g新特性】Cardinality Feedback基数反馈

Cardinality Feedback基数反馈是版本11.2中引入的关于SQL 性能优化的新特性,该特性主要针对 统计信息陈旧、无直方图或虽然有直方图但仍基数计算不准确的情况, Cardinality基数的计算直接影响到后续的JOIN COST等重要的成本计算评估,造成CBO选择不当的执行计划。以上是Cardinality Feedback特性引入的初衷。

 

Cardinality Feedback2

Cardinality Feedback1

 

但是每一个Oracle新版本引入的新特性 都被一些老外DBA称之为buggy ,Cardinality Feedback基数反馈多少也造成了一些麻烦,典型的情况是测试语句性能时,第一次的性能最好,之后再运行其性能变差。

 

我们来看一下 Cardinality Feedback基数反馈是如何作用的:
注意使用普通用户来测试Cardinality Feedback,sys用户被默认禁用该特性

 

 

Oracle Database 11g Enterprise Edition Release 11.2.0.3.0 - 64bit Production
With the Partitioning, OLAP, Data Mining and Real Application Testing options

SQL> conn maclean/oracle
已连接。

SQL> show parameter dynamic

NAME                                 TYPE                   VALUE
------------------------------------ ---------------------- ------------------------------
optimizer_dynamic_sampling           integer                0

SQL> create table test as select * from dba_tables;

表已创建。

SQL> select /*+ gather_plan_statistics */ count(*) from test;

  COUNT(*)
----------
      2873

SQL> select * from table(dbms_xplan.display_cursor(null,null,'ALLSTATS LAST'));

PLAN_TABLE_OUTPUT
------------------------------------------------------------------------------------------------------
SQL_ID  0p4u1wqwg6t9z, child number 0
-------------------------------------
select /*+ gather_plan_statistics */ count(*) from test

Plan hash value: 1950795681

-------------------------------------------------------------------------------------
| Id  | Operation          | Name | Starts | E-Rows | A-Rows |   A-Time   | Buffers |
-------------------------------------------------------------------------------------
|   0 | SELECT STATEMENT   |      |      1 |        |      1 |00:00:00.01 |     104 |
|   1 |  SORT AGGREGATE    |      |      1 |      1 |      1 |00:00:00.01 |     104 |
|   2 |   TABLE ACCESS FULL| TEST |      1 |   8904 |   2873 |00:00:00.01 |     104 |
-------------------------------------------------------------------------------------

已选择14行。

SQL> select /*+ gather_plan_statistics */ count(*) from test;

  COUNT(*)
----------
      2873

SQL> select * from table(dbms_xplan.display_cursor(null,null,'ALLSTATS LAST'));

PLAN_TABLE_OUTPUT
---------------------------------------------------------------------------------------
SQL_ID  0p4u1wqwg6t9z, child number 1
-------------------------------------
select /*+ gather_plan_statistics */ count(*) from test

Plan hash value: 1950795681

-------------------------------------------------------------------------------------
| Id  | Operation          | Name | Starts | E-Rows | A-Rows |   A-Time   | Buffers |
-------------------------------------------------------------------------------------
|   0 | SELECT STATEMENT   |      |      1 |        |      1 |00:00:00.01 |     104 |
|   1 |  SORT AGGREGATE    |      |      1 |      1 |      1 |00:00:00.01 |     104 |
|   2 |   TABLE ACCESS FULL| TEST |      1 |   2873 |   2873 |00:00:00.01 |     104 |
-------------------------------------------------------------------------------------

Note
-----
   - cardinality feedback used for this statement

已选择18行。

 

 

上例中第一次运行时,由于未收集表上的统计信息且optimizer_dynamic_sampling=0 关闭了动态采样所以基数评估值(1)和实际值(2873)有着较大的差距。

 

 

cardinality feedback used for this statement这个信息说明第二次执行时使用了Cardinality Feedback基数反馈,且其基数评估也十分精确了,这是因为第二次执行时考虑到第一次执行时的基数反馈,我们来看看Oracle到底是如何做到的:

 

 

 

SQL> alter system flush shared_pool;

系统已更改。

SQL>
SQL> alter session set events '10053 trace name context forever, level 1';

会话已更改。

SQL>  select /*+ gather_plan_statistics */ count(*) from test;

  COUNT(*)
----------
      2873

SQL>  select /*+ gather_plan_statistics */ count(*) from test;

  COUNT(*)
----------
      2873

10053 trace:

第一次执行:

sql= select /*+ gather_plan_statistics */ count(*) from test
----- Explain Plan Dump -----
----- Plan Table -----

============
Plan Table
============
--------------------------------------+-----------------------------------+
| Id  | Operation           | Name    | Rows  | Bytes | Cost  | Time      |
--------------------------------------+-----------------------------------+
| 0   | SELECT STATEMENT    |         |       |       |    31 |           |
| 1   |  SORT AGGREGATE     |         |     1 |       |       |           |
| 2   |   TABLE ACCESS FULL | TEST    |  8904 |       |    31 |  00:00:01 |
--------------------------------------+-----------------------------------+

SELECT /*+ OPT_ESTIMATE (TABLE "TEST" ROWS=2873.000000 ) */ COUNT(*) "COUNT(*)" FROM "MACLEAN"."TEST" "TEST"

SINGLE TABLE ACCESS PATH 
  Single Table Cardinality Estimation for TEST[TEST] 
  Table: TEST  Alias: TEST
    Card: Original: 8904.000000    >> Single Tab Card adjusted from:8904.000000 to:2873.000000
  Rounded: 2873  Computed: 2873.00  Non Adjusted: 8904.00
  Access Path: TableScan
    Cost:  31.10  Resp: 31.10  Degree: 0
      Cost_io: 31.00  Cost_cpu: 1991217
      Resp_io: 31.00  Resp_cpu: 1991217
  Best:: AccessPath: TableScan
         Cost: 31.10  Degree: 1  Resp: 31.10  Card: 2873.00  Bytes: 0

sql= select /*+ gather_plan_statistics */ count(*) from test
----- Explain Plan Dump -----
----- Plan Table -----

============
Plan Table
============
--------------------------------------+-----------------------------------+
| Id  | Operation           | Name    | Rows  | Bytes | Cost  | Time      |
--------------------------------------+-----------------------------------+
| 0   | SELECT STATEMENT    |         |       |       |    31 |           |
| 1   |  SORT AGGREGATE     |         |     1 |       |       |           |
| 2   |   TABLE ACCESS FULL | TEST    |  2873 |       |    31 |  00:00:01 |
--------------------------------------+-----------------------------------+

 

 

 

 

可以看到第二次执行时SQL最终转换加入了 OPT_ESTIMATE (TABLE “TEST” ROWS=2873.000000 )的HINT ,OPT_ESTIMATE HINT一般由 kestsaFinalRound()内核函数生成。该HINT用以纠正各种类型的优化器评估,例如某表上的基数或某个列的最大、最小值。反应出优化的不足或者BUG。

 

可以通过V$SQL_SHARED_CURSOR和来找出现有系统shared pool中仍存在的 使用了Cardinality Feedback基数反馈的子游标:

 

 

SQL> select sql_ID,USE_FEEDBACK_STATS  FROM V$SQL_SHARED_CURSOR where USE_FEEDBACK_STATS ='Y';

SQL_ID                     US
-------------------------- --
159sjt1f6khp2              Y

 

 

 

还可以使用cardinality HINT来强制使用Cardinality Feedback 。

select /*+ cardinality(test,  1) */ count(*) from test;

 

 

如何禁用Cardinality Feedback基数反馈

 

对于这些”惹火”特性,为了stable,往往考虑关闭该特性。

可以通过多种方法禁用该特性

1. 使用 _optimizer_use_feedback 隐藏参数

session 级别

SQL> alter session set “_optimizer_use_feedback”=false;

会话已更改。

system级别

SQL> alter system set “_optimizer_use_feedback”=false;

系统已更改。

 

2. 使用opt_param(‘_optimizer_use_feedback’ ‘false’) HINT

例如:

select /*+ opt_param(‘_optimizer_use_feedback’ ‘false’) cardinality(test,1) */ count(*) from test;

 

 

 

Library cache: mutex X

Library cache: mutex X

对于该Library cache: mutex X等待事件而言,可能存在多种原因:

  1. “library cache: mutex X” 等待事件症状在11g之后较为常见,原因多样,所以一定要找root cause。
  2. 已知有不少bug引起该”library cache: mutex X”的。这些bug有些与mutex有关有些又无关。bug信息见附录。
  3. 在一些例子中问题常由于应用程序变更而引发。 同时也要注意登陆 logon 或 登出logoff暴风所引起的问题。
  4. 这个mutex在KGL的bucket中。

 

Library cache: mutex X的其他一些常见原因:

  • 频繁的硬解析hard parse
  • 如果硬解析真的十分激烈,那么争用常发生在pin上
  • 高的version count,SQL游标版本过多
  • 当某些SQL的version count非常高时,一大串的version要被检验那么可能导致Library cache: mutex X
  • SQL的Invalidations 和 reloads
  • BUG
  • 操作系统相关问题 例如 Bug:7441165 – CPU Pre-emption can cause problems on Solaris (this fix only applies to this OS)

如果自己搞不定可以找诗檀软件专业ORACLE数据库优化团队成员帮您调优!

诗檀软件专业数据库优化团队

服务热线 : 13764045638   QQ号:47079569    邮箱:service@parnassusdata.com

 

 

 

library cache: mutex X相关BUG:

 

NB Bug Fixed Description
15858022 12.1.0.0 Frequent invalidation of tuning objects with VPD
14401731 11.2.0.4, 12.1.0.0 Cursor leak using "SET TRANSACTION" within PLSQL
13810393 12.1.0.0 Deadlock waiting for 'library cache: mutex x' while producing an ORA-4031
diagnostic dump
13720753 11.2.0.4, 12.1.0.0 Diagnosability issue - mutex wait "idn" value truncated in session wait
output in tracefile
13588248
11.2.0.2.BP17, 11.2.0.3.3,
11.2.0.3.BP06, 12.1.0.0
"library cache: mutex X" contention for objects with very many library cache
locks
12976376
11.2.0.2.6, 11.2.0.2.BP16, 11.2.0.3.3,
11.2.0.3.BP05, 12.1.0.0
High VERSION_COUNT for SQL with binds, including recursive dictionary
SQL - superseded
12797420 11.2.0.3.3, 11.2.0.3.BP07, 12.1.0.0
"library cache: mutex X" waits on DB instance handle with
CURSOR_SHARING
11818335
11.1.0.7.9, 11.2.0.2.2, 11.2.0.2.BP06,
11.2.0.3, 12.1.0.0
Additional support for bug 10411618 to allow dynamic Mutex wait scheme
changes
11719151 11.2.0.3, 12.1.0.0 SQL Plan Management capture causes slowness
10632113 11.2.0.3, 12.1.0.0 OLS calls cause mutex contention even with low number of concurrent users
10417716
11.2.0.2.5, 11.2.0.2.BP05,
11.2.0.2.GIPSU05, 11.2.0.3, 12.1.0.0 Mutex X waits in 11G on an instance with heavy JAVA usage
10284845 11.2.0.3, 12.1.0.0 Need index on SID for X$KGLLK and X$KGLPN based views
(V$OPEN_CURSOR etc..)
10284838
11.2.0.2.5, 11.2.0.2.BP13,
11.2.0.2.GIPSU05, 11.2.0.3, 12.1.0.0
Cache line contention while waiting for mutexes
10204505 11.2.0.3, 12.1.0.0
SGA autotune can cause row cache misses, library cache reloads and
parsing
10086843 11.2.0.3, 12.1.0.0
Recursive SQL cursors not reused - PMON crashes instance with ORA-600
[kglLockOwnersListDelete]
9530750 11.2.0.2, 12.1.0.0 High waits for 'library cache: mutex X' for cursor Build lock
9239863 11.2.0.1.BP06, 11.2.0.2, 12.1.0.0 Excessive "library cache:mutex X" contention on hot objects
8981059
11.1.0.7.4, 11.2.0.1.2, 11.2.0.1.BP06,
11.2.0.2, 12.1.0.0
High Version Count (due to USER_BIND_PEEK_MISMATCH) with bind
peeking
8860198 11.2.0.2, 12.1.0.0 "library cache:mutex X" waits using XMLType
7352775 11.2.0.2, 12.1.0.0 Many child cursors when PARALLEL_INSTANCE_GROUP set wrong
12670165 11.2.0.2.4, 11.2.0.2.BP12, 11.2.0.3 Library cache mutex contention caused by lookup of triggers
12633340 11.2.0.2.6, 11.2.0.2.BP13, 11.2.0.3 Heavy "library cache lock" and "library cache: mutex X" contention for a
"$BUILD$.xx" lock
8793492 11.2.0.1.BP07, 11.2.0.2 Mutex Waits with Resource Manager
10145558 11.1.0.7.7, 11.2.0.1.BP12, 11.2.0.2 Selects on library cache V$/X$ views cause "library cache: mutex X" waits
9398685 11.2.0.2 High "library cache: mutex X" when using Application Context
9282521 11.2.0.2 Excessive "library cache:mutex X" contention on hot objects
9140262 11.2.0.2 ORA-600 [ksliwat5] followed by cpu spike/"library cache: mutex X" Waits
9003145 11.2.0.1.BP03, 11.2.0.2 Dump (kglIsOwnerVersionable) / "library cache: mutex X" waits
7502237 11.1.0.7.7, 11.2.0.1 Unnecessary "library cache: mutex X" waits using stored Java
7307972 11.1.0.7.2, 11.2.0.1 Excessive waits on 'library cache: mutex x'
8431767 High "library cache: mutex X" when using Application Context
9312879 11.1.0.7.7, 11.2.0.1 "library cache: mutex x" waits after killing sessions / PMON slow to clean up
7648406 10.2.0.5, 11.1.0.7.4, 11.2.0.1
Child cursors not shared for "table_..." cursors (that show as "SQL Text Not Available") when NLS_LENGTH_SEMANTICS = CHAR
7155860 11.2.0.1 Spin on kgllkde causes 'library cache: mutex X'
8499043 11.1.0.7.2 SET_CONTEXT incurs unnecessary DLM overhead in RAC
5928271 11.1.0.7 Excessive waits on "library cache: mutex X"
7317117 11.2.0.1 Unnecessary "library cache: mutex X" waits on LOB operations

 

 

【Maclean Liu技术分享】拨开Oracle优化器迷雾探究Histogram之秘

【Maclean Liu技术分享】拨开Oracle CBO 优化器迷雾, 探究Histogram直方图之秘,讲座文档正式版已上传 
http://t.askmac.cn/thread-2172-1-1.html  

 

 

预计时长: 1.5个小时

适合参与成员: 对于性能调优和CBO优化器有兴趣的同学,或急于提升SQL调优技能的同学。

讲座材料presentation 当前正式版本下载:

 

【Maclean Liu技术分享】拨开Oracle CBO优化器迷雾,探究Histogram直方图之秘_0321.pdf.pdf(1.1 MB, 下载次数: 749) 

CBO Cost Formulas基于成本优化器的成本计算公式大全

CBO Cost Formulas成本计算公式大全:

成本模型:连接方式Join method

注意 连接基数(Join Cardinality)不受到连接方式(join method) 的影响, oracle中主要的三种join method HASH JOIN、Nested Loops、Sort Merge:

  • Nested Loops嵌套循环成本公式:
    • Cost(outer)+Cost(inner))*cardinality(outer)
  • Sort merge 合并排序连接成本公式:
    • Cost(outer) + Cost(inner) + Sort(outer) + Sort(inner)
  • Hash Join 哈希连接公式:
    • Cost(outer) + Cost(inner) + Build(outer) + Probe(inner)

 

Index Unique Scan Cost成本计算
INDEX UNIQUE SCAN COST = (BLEVEL (1-(OIC/100)) + 1) * (OICA/100)

 

Index Range Scan Cost成本计算
INDEX RANGE SCAN COST = (BLEVEL + FF*LFBL)*(1-(OIC/100))+ FF*CLUF)* (OICA/100)

 

 

formula does not include the CPU cost

  • BLEVEL = number of branch levels in index
  • add +1 for leaf block
  • FF = filtering factor – selectivity
  • LFBL = number of leaf blocks
  • CLUF = index clustering factor
  • OIC = optimizer_index_caching(default 0)
  • OICA = optimizer_index_cost_adj parameter(default=100)

 

CPU costing启用的情况下:

mreadtime -Average time , in milliseconds, for a multi-block read (according to sys.aux_stats$)

sreadtime – Average time , in milliseconds, for a single-block read (according to sys.aux_stats$)

MBRC – Average number of blocks to be read in a multi-block read (according to sys.aux_stats$

#SRDs – number of single block reads

#MRDs – number of multi block reads

#CPUCycles – number of CPU Cycles

sreadtime = ioseektim + db_block_size/iotfrspeed

mreadtim = ioseektim + db_file_multiblock_read_count * db_block_size / iotrfspeed

#MRds = #Blks/MBRC

Cost 成本本身 =(#SRds * sreadtim +#MRds * mreadtim + #CPUCycles/cpuspeed)/sreadtim ,  

Cost成本的单位 为 single-block read time=sreadtim

 

 

OSS Description

Provide a description of the component including how it will be built and what it will do, with a reference to the functional requirements (from the Functional Specification) that are being addressed.

Optimizer system statistics contains hardware characteristics. With OSS optimizer combines IO and CPU resources needed to execute query into single unit – estimated execution time.

OSS Components:

 

Component Initialization Maintenance Description Abbreviated component’s name (get_system_stats and set_system_stats)
CPU speed At system startup Gathering system stats with gathering_mode =  ‘NOWORKLOAD’ or ‘START’, ‘STOP’ or ‘INTERVAL’ or setting manually # CPU cycles per second cpuspeed
IO seek time At system startup Gathering system stats with gathering_mode =  ‘NOWORKLOAD’ or setting manually Seek time + latency time + OS overhead time ioseektim
IO transfer speed At system startup Gathering system stats with gathering_mode =  ‘NOWORKLOAD’ or setting manually Rate at which oracle can read data in the single read request iotfrspeed
Max IO throughput None Gathering system stats with gathering_mode =  ‘NOWORKLOAD’ or ‘START’, ‘STOP’ or ‘INTERVAL’ or setting manually This number characterizes maximum throughput (MB / sec) IO subsystem can deliver maxthr
Average slave IO throughput None Gathering system stats with gathering_mode =  ‘START’, ‘STOP’ or ‘INTERVAL’ or setting manually Average parallel slave IO throughput slavethr
Average Single Block Read Time None Gathering system stats with gathering_mode =  ‘START’, ‘STOP’ or ‘INTERVAL’ or setting manually sreadtim
Average Multi Block Read Time None Gathering system stats with gathering_mode =  ‘START’, ‘STOP’ or ‘INTERVAL’ or setting manually mreadtim
Average multi block read count None Gathering system stats with gathering_mode =  ‘START’, ‘STOP’ or ‘INTERVAL’ or setting manually mbrc

 

  1. cpuspeed, ioseektim, iotfrspeed are always collected
  2. maxthr, slavethr, sreadtim, mreadtim and mbrc collected only when user gathers workload statistics.

These two sets contain equivalent information. The difference is that A) does not relate to workload and B) does. At any moment of time only one set of OSS can be used.

 

 

OSS data located in data dictionary in the aux_stats$ table and in the SGA variable kkossga. aux_stats$ keeps persistent copy of the OSS. kkossga keeps working copy. Data in kkossga and  aux_stats$ is always synchronized. User can modify, delete and gather OSS through interface provided in the DBMS_STATS package.

 

OSS used to represent cost as query estimated running time (it’s implemented as #CPU cycles and # multi block reads conversion to # single block reads) and to adjust FTS cost for parallel reads.

SRead_Cost(#Cycles) = (#Cycles * (1 / CPUSpeed)) / sreadtim

SRead_Cost(#MReads) = (#MReads * mreadtim) / sreadtim

When no workload stats available optimizer uses NOWORKLOAD stats to compute sreadtim and mreadtim:

sreadtim = ioseektim + block_size / iotfrspeed

mreadtim = ioseektim + mbrc * block_size / iotfrspeed

Optimizer converts multi block reads to single block reads (even if cost formula looks elegant the actual processing has to support old days behavior and it causes that internally everything converted to single block reads)

aux_stats$

table aux_stats$ (

sname varchar2(30) not null,       /* Prefix */

pname varchar2(30) not null,       /* Name of parameter */

pval1 number,                      /* NUMBER parameter value */

pval2 varchar2(255)                /* VARCHAR2 parameter value */

)

This table stores OSS. It also used to store the intermediate values when analyzing workload.

sname column used to store global prefixes of the stats SYSSTATS_MAIN, SYSSTATS_TEMP and SYSSTATS_INFO.

if sname = SYSSTATS_MAIN then pname and pval1 columns store name-value pairs for data representing current stats:

cpuspeed (# cpu cycles per second) in millions;

ioseektim (Seek time + latency time + OS overhead time) in milliseconds;

iotfrspeed (IO transfer speed) in bytes/ second;

maxthr (maximum I/O system throughput) in bytes/sec;

slavethr (average slave throughput);

sreadtim (wait time to read single block) in milliseconds;

mreadtim (wait time to read a multiblock) in milliseconds;

mbrc (multiblock read count) in blocks;

if sname = SYSSTATS_TEMP then pname and pval1 columns store name-value pairs for intermediate data, generated than user issues DBMS_STAT.GATHER_SYSTEM_STATS procedure and removed then gathering completes.

if sname = SYSSTATS_INFO then pname, pval2 columns store name-values for current and intermediate stats:

DSTART – then gathering was started, format “MM-DD-YYYY HH:MI”

DSTOP – then gathering was (will be, had to be) finished format “MM-DD-YYYY HH:MI”

STATUS – ‘COMPLETED’, ‘AUTOGATHERING’, ‘MANUALGATHERING’, ‘INVALID’

关于10053 trace中的UNCOMPBKTS和ENDPTVALS

关于10053 trace中的Histogram部分的UNCOMPBKTS和ENDPTVALS

当Histogram直方图类型为frequency histograms( Histogram: Freq)时UncompBkts 等于统计信息中采样的总行数-NULLS(Card: Original- NULLS,因为dbms_stats默认是auto_sample_size采样,所以这栏其实是采样到的原始Card-NULLS), 而EndPtVals 等于bucket总数,或者说NDV,因为frequency histograms中 NDV=number of buckets 

当直方图类型为height balanced histograms (Histogram: HtBal) UncompBkts  等于bucket的数目(其实也等于10053 trace中#Bkts的数目),而EndPtVals 等于已经被压缩的Histogram的大小,其实是等于: select count(*) from dba_tab_histograms where table_name=’MACLEANLIU’ and column_name=’MACLEANLIU’的实际总和。  通过这2个值对比,可以了解到popular值的多少以及数据的倾斜度, 是有多个大量重复的值(popular value)还是仅有一个巨大的重复值。

 

仍不明确少数概念的话,来看看下面这个图:

Histogram representation

Oracle CBO术语大集合

最近准备写点Histogram和density相关的文章,先把术语给大家理一理:

cardinality (CDN)
Legend
CBQT – cost-based query transformation
JPPD – join predicate push-down
OJPPD – old-style (non-cost-based) JPPD
FPD – filter push-down
PM – predicate move-around
CVM – complex view merging
SPJ – select-project-join
SJC – set join conversion
SU – subquery unnesting
OBYE – order by elimination
OST – old style star transformation
ST – new (cbqt) star transformation
CNT – count(col) to count(*) transformation
JE – Join Elimination
JF – join factorization
SLP – select list pruning
DP – distinct placement
qb – query block
LB – leaf blocks
DK – distinct keys
LB/K – average number of leaf blocks per key
DB/K – average number of data blocks per key
CLUF – clustering factor
NDV – number of distinct values
Resp – response cost
Card – cardinality
Resc – resource cost
NL – nested loops (join)
SM – sort merge (join)
HA – hash (join)
CPUSPEED – CPU Speed
IOTFRSPEED – I/O transfer speed
IOSEEKTIM – I/O seek time
SREADTIM – average single block read time
MREADTIM – average multiblock read time
MBRC – average multiblock read count
MAXTHR – maximum I/O system throughput
SLAVETHR – average slave I/O throughput
dmeth – distribution method
1: no partitioning required
2: value partitioned
4: right is random (round-robin)
128: left is random (round-robin)
8: broadcast right and partition left
16: broadcast left and partition right
32: partition left using partitioning of right
64: partition right using partitioning of left
256: run the join in serial
0: invalid distribution method
sel – selectivity
ptn – partition
adop Automatic degree of parallelism

TABLE: Table Name
ALIAS: Table Alias
QBS: Query Block Signature
#ROWS: Number of Rows
#BLKS: Number of Blocks
ARL: Average Row Length
COR: Cardinality Original
CRD: Cardinality Rounded
CCM: Cardinality Computed
CNA: Cardinality Non Adjusted

AVGLEN: Average Column Length
NDV: Number of Distinct Values
NULLS: Number of Nulls in Column
DEN: Column Density
MIN: Minimum Column Value
MAX: Maximum Column Value
TYPE: Histogram Type
#BKTS: Histogram Buckets
UNCOMPBKTS: Histogram Uncompressed Buckets   
ENDPTVALS: Histogram End Point Values 
OOR: Out-of-Range Predicate

TABLE: Table Name
ALIAS: Table Alias
INDEX: Index Name
QBS: Query Block Signature
LVLS: Index Levels
#LB: Number of Leaf Blocks
#DK: Number of Distinct Keys
LB/K: Average Number of Leaf Blocks Per Key
DB/K: Average Number of Data Blocks Per Key
CLUF: Clustering Factor
INDEX_COLS: Index Column Numbers

COST: Cost of the Join
CARD: Cardinality of the Join
BC: Best Cost
LINE#: Line Number in the 10053 Trace File Where Cost Value is Located
JOIN#: Join Number in the 10053 Trace File Associated With Key
STATUS: If Permutation was Computed for all Table Joins the Status = COMPL. If Not, status = ABORT
*: In ANY Column Indicates Value Not Found in File

Freq 频率直方图
HtBal 高度平衡直方图

 

 

关于 UNCOMPBKTS和ENDPTVALS

 

当直方图类型为frequency histograms( Histogram: Freq)时UncompBkts  等于统计信息中采样的总行数-NULLS(Card: Original- NULLS,因为dbms_stats默认是auto_sample_size采样,所以这栏其实是采样到的原始Card-NULLS), 而EndPtVals 等于bucket总数,或者说NDV,因为frequency histograms中 NDV=number of buckets 

当直方图类型为height balanced histograms (Histogram: HtBal) UncompBkts  等于bucket的数目(其实也等于10053 trace中#Bkts的数目),而EndPtVals 等于已经被压缩的Histogram的大小,其实是等于: select count(*) from dba_tab_histograms where table_name=’YOUR_TABLE_NAME’ and column_name=’YOUR_COLUMN_NAME’的实际总和。  通过这2个值对比,可以了解到popular值的多少以及数据的倾斜度, 是有多个大量重复的值(popular value)还是仅有一个巨大的重复值。

 

 

SQL Performance Analyzer SPA常用脚本汇总

SPA常用脚本汇总

附件为 一个SPA报告 spa_buffergets_summary

 

SQL 性能分析器 SQL Performance Analyzer SPA

Oracle Database 11g 引入了 SQL 性能分析器;使用该工具可以准确地评估更改对组成工作量的 SQL 语句的影响。SQL 性能分析器可帮助预测潜在的更改对 SQL 查询工作量的性能影响。这种功能可向 DBA 提供有关 SQL 语句性能的详细信息,例如,执行前后的统计信息,提高或降低性能的语句。这样一来,您就可以执行诸如以下操作的操作:在测试环境中进行更改,以确定数据库升级是否会改进工作量性能。

 

  1. 11g 的新增功能
  2. 目标用户:DBA、QA、应用程序开发人员
  3. 帮助预测系统更改对 SQL 工作量响应时间的影响
  4. 建立不同版本的 SQL 工作量性能(即 SQL 执行计划和执行统计信息)
  5. 以串行方式执行 SQL(不考虑并发性)
  6. 分析性能差异
  7. 提供对单个 SQL 的细粒度性能分析
  8. 与 SQL 优化指导集成在一起以优化回归

SQL 性能分析器:使用情形
SQL 性能分析器可用于预测和防止会影响 SQL 执行计划结构的任何数据库环境更改所带来的潜在性能问题。这些更改可以包括(但不限于)以下任何一种更改:

  1. 数据库升级
  2. 实施优化建议
  3. 更改方案
  4. 收集统计信息
  5. 更改数据库参数
  6. 更改操作系统和硬件

 

DBA 甚至可以使用 SQL 性能分析器为最复杂的环境预测先期更改导致的 SQL 性能更改。例如,随着应用程序在开发周期中的变化,数据库应用程序开发人员可以测试对方案、 数据库对象和重写应用程序的更改,以减轻任何潜在的性能影响。
使用 SQL 性能分析器还可以比较 SQL 性能统计信息。

SQL 性能分析器:概要

1.  收集 SQL:在这个阶段中,将收集用于表示生产系统中的 SQL 工作量的 SQL 语句集。可以使用 SQL 优化集或自动工作量资料档案库 (AWR) 来捕获要传送的信息。因为 AWR 本质上是捕获高负载的 SQL,所以应考虑修改默认的 AWR 快照设置和捕获的顶级 SQL,以确保 AWR 捕获最大数量的 SQL 语句。这可以确保捕获更加完整的 SQL 工作量。

2.  传送:在这个阶段中,应将得到的工作量结果传送到测试系统。从生产系统导出 STS,然后将 STS 导入到测试系统。

3.  计算“之前版本”性能:在进行任何更改之前,执行 SQL 语句,收集评估将来的更改对工作量性能的可能影响所需的基线信息。在此阶段收集的信息给出了系统工作量当前状态的一个快照。性能数据包括:

-执行计划(如由解释计划生成的计划)
-执行统计信息(如由占用时间、缓冲获取次数、磁盘读取次数和已处理的行数组成的信息)

4. 进行更改:获得了之前版本数据后,可以实施计划的更改,然后开始查看对性能的影响。

5.  计算“之后版本”性能:在数据库环境中进行了更改之后才执行此步骤。SQL 工作量的每个语句都在虚拟执行(仅收集统计信息)模式下运行,收集与步骤 3 所捕获的信息相同的信息。

6.  比较和分析 SQL 性能:在获得了两个版本的 SQL 工作量性能数据后,可以通过比较之后版本与之前版本的数据来进行性能分析。比较的根据是执行统计信息,如所用时间、CPU 时间和缓冲区获取次数等。

7.  优化回归的 SQL:在此阶段中,已经准确地确认了哪些 SQL 语句在进行数据库更改时可能导致性能问题。在此阶段中可以使用任何一种数据库工具来优化系统。例如,可以对确认的语句使用 SQL 优化指导或访问指导,然后实施相应的建议。也可以使用在步骤 3 中捕获的计划植入 SQL 计划管理 (SPM) 以确保计划保持不变。在实施了任何优化操作后,应重复该过程来创建新的之后版本,然后分析性能差异以确保新的性能是可接受的。
默认情况下SPA若涉及到DML语句则只有查询部分Query会被执行,但是貌似是从11.2开始可以执行完全的DML了,需要加入参数EXECUTE_FULLDML,但是该参数目前有一些BUG:

Bug 10428438 : WITH EXECUTE_FULLDML ROWS IS ALWAYS SET TO 0 11.2.0.1

Bug 14635522 : SPA SHOULD CAPTURE AND REPLAY TRANSACTIONS 11.2.0.3

 

By default, only the query portion of DMLs is executed. Using APIs, you can execute the full DML by using the EXECUTE_FULLDML task parameter.EXECUTE_FULLDML when set to TRUE executes DML statement fully, including acquiring row locks and modifying rows; When EXECUTE_FULLDML is set to FALSE (the default value is false) to execute only the query part of the DML without modifying data. When TRUE, SQL Performance Analyzer will issue a rollback following DML execution to prevent persistent changes from being made by the DML. So SPA does not make make any change to the data in the tables.

 

执行方法如下:

 

execute DBMS_SQLPA.SET_ANALYSIS_TASK_PARAMETER(task_name   => 'TASK_21137', -
                                               parameter   => 'EXECUTE_FULLDML', -
                                               value       => 'TRUE');

 

 

 

 

从cursor cache中收集tuning set, 持续12分钟,间隔5秒钟

 

 

begin
DBMS_SQLTUNE.CREATE_SQLSET (sqlset_name => 'MAC_SPA');
dbms_sqltune.capture_cursor_cache_sqlset(
sqlset_name => 'MAC_SPA' ,
time_limit => 12*60,
repeat_interval => 5);
end ;
/

basic_filter=> q'# module like 'DWH_TEST%' and sql_text not like '%applicat%' and parsing_schema_name in ('APPS') #'

basic_filter   => 'sql_text LIKE ''%my_objects%'' and parsing_schema_name = ''SPA_TEST_USER''',

==>过滤条件使用

 

从当前cursor cache中匹配条件 获得SQLset ROW

 

 

SELECT sql_id, sql_text 
FROM table(DBMS_SQLTUNE.SELECT_CURSOR_CACHE('buffer_gets > 500')) 
ORDER BY sql_id;

SELECT * 
FROM table(DBMS_SQLTUNE.SELECT_CURSOR_CACHE('sql_id = ''4rm4183czbs7j'''));

 DECLARE
  cur sys_refcursor;
BEGIN
  OPEN cur FOR
    SELECT value(P) 
    FROM table(DBMS_SQLTUNE.SELECT_CURSOR_CACHE) P;

  -- Process each statement (or pass cursor to load_sqlset).

  CLOSE cur;
END;
/

 -- create the tuning set
EXEC DBMS_SQLTUNE.CREATE_SQLSET('MAC_SPA');
-- populate the tuning set from the cursor cache
DECLARE
 cur DBMS_SQLTUNE.SQLSET_CURSOR;
BEGIN
 OPEN cur FOR
   SELECT VALUE(P)
     FROM table(
       DBMS_SQLTUNE.SELECT_CURSOR_CACHE(
         'parsing_schema_name <> ''SYS'' AND elapsed_time > 5000000',
          NULL, NULL, NULL, NULL, 1, NULL,
         'ALL')) P;

DBMS_SQLTUNE.LOAD_SQLSET(sqlset_name => 'MAC_SPA',
                        populate_cursor => cur);

END;
/

 

 

从AWR快照中加载SQLset ROW到SQL TUNING SET

 

 

DECLARE
  cur sys_refcursor;
BEGIN
  OPEN cur FOR
    SELECT VALUE (P) 
    FROM table(dbms_sqltune.select_workload_repository(4146,4161)) P;

  -- Process each statement (or pass cursor to load_sqlset)
  DBMS_SQLTUNE.LOAD_SQLSET(sqlset_name => 'MAC_SPA',
                        populate_cursor => cur);
  CLOSE cur;
END;
/

 

 

 

将SQL TUNING SET Pack到表中:

 

 

set echo on
select name,statement_count from dba_sqlset;

drop table maclean.pack_sqlset purge;

exec DBMS_SQLTUNE.CREATE_STGTAB_SQLSET('PACK_SQLSET','MACLEAN');

exec DBMS_SQLTUNE.PACK_STGTAB_SQLSET('MAC_SPA','SYS','PACK_SQLSET','MACLEAN');

SQL> desc maclean.pack_sqlset;
 Name                                      Null?    Type
 ----------------------------------------- -------- ----------------------------
 NAME                                               VARCHAR2(30)
 OWNER                                              VARCHAR2(30)
 DESCRIPTION                                        VARCHAR2(256)
 SQL_ID                                             VARCHAR2(13)
 FORCE_MATCHING_SIGNATURE                           NUMBER
 SQL_TEXT                                           CLOB
 PARSING_SCHEMA_NAME                                VARCHAR2(30)
 BIND_DATA                                          RAW(2000)
 BIND_LIST                                          SQL_BIND_SET
 MODULE                                             VARCHAR2(48)
 ACTION                                             VARCHAR2(32)
 ELAPSED_TIME                                       NUMBER
 CPU_TIME                                           NUMBER
 BUFFER_GETS                                        NUMBER
 DISK_READS                                         NUMBER
 DIRECT_WRITES                                      NUMBER
 ROWS_PROCESSED                                     NUMBER
 FETCHES                                            NUMBER
 EXECUTIONS                                         NUMBER
 END_OF_FETCH_COUNT                                 NUMBER
 OPTIMIZER_COST                                     NUMBER
 OPTIMIZER_ENV                                      RAW(1000)
 PRIORITY                                           NUMBER
 COMMAND_TYPE                                       NUMBER
 FIRST_LOAD_TIME                                    VARCHAR2(19)
 STAT_PERIOD                                        NUMBER
 ACTIVE_STAT_PERIOD                                 NUMBER
 OTHER                                              CLOB
 PLAN_HASH_VALUE                                    NUMBER
 PLAN                                               SQL_PLAN_TABLE_TYPE
 SPARE1                                             NUMBER
 SPARE2                                             NUMBER
 SPARE3                                             BLOB
 SPARE4                                             CLOB

 

 

 

将测试对应 schema的数据和 上述PACK TABLE 导出导入到 目标测试库中:

 

set echo on
exec DBMS_SQLTUNE.UNPACK_STGTAB_SQLSET('MAC_SPA','SYS',TRUE,'PACK_SQLSET','MACLEAN');
alter system flush buffer_cache;
alter system flush shared_pool;

 

 

创建SPA任务 并运行;

 

 

var sts_task varchar2(64);
exec :sts_task:= dbms_sqlpa.create_analysis_task(task_name => '10g_11g_spa',description => 'experiment for 10gR2 to 11gR2 upgrade',sqlset_name=> 'MAC_SPA');

PL/SQL procedure successfully completed.

var exe_task varchar2(64);
exec :exe_task:=dbms_sqlpa.execute_analysis_task(task_name=>'10g_11g_spa',execution_name=>'10g_trail',execution_type=>'CONVERT SQLSET',execution_desc=>'10g sql trail');

var exe_task varchar2(64);
exec :exe_task:=dbms_sqlpa.execute_analysis_task(task_name=>'10g_11g_spa',execution_name=>'11g_trail',execution_type=>'TEST EXECUTE',execution_desc=>'11g sql trail');

 

 

 

执行任务比较

 

 

 

比较CPU_TIME
EXEC dbms_sqlpa.execute_analysis_task( -
  task_name => '10g_11g_spa', -
  execution_name => 'compare_10g_112_cpu', -
  execution_type => 'COMPARE PERFORMANCE', -
  execution_params => dbms_advisor.arglist('COMPARISON_METRIC','CPU_TIME','EXECUTION_NAME1','10g_trail','EXECUTION_NAME2','11g_trail'), -
  execution_desc => 'Compare 10g SQL Trace Performance to 11g Test-Execute for CPU_TIME')
  /

比较BUFFER_GETS
EXEC dbms_sqlpa.execute_analysis_task( -
  task_name => '10g_11g_spa', -
  execution_name => 'compare_10g_112_buffergets', -
  execution_type => 'COMPARE PERFORMANCE', -
  execution_params => dbms_advisor.arglist('COMPARISON_METRIC','BUFFER_GETS','EXECUTION_NAME1','10g_trail','EXECUTION_NAME2','11g_trail'), -
  execution_desc => 'Compare 10g SQL Trace Performance to 11g Test-Execute for BUFFER_GETS')
  /

比较实际执行时长 

begin 
DBMS_SQLPA.EXECUTE_ANALYSIS_TASK( 
task_name => 'SPA_TEST', 
execution_type => 'COMPARE PERFORMANCE', 
execution_name => 'Compare_elapsed_time', 
execution_params => dbms_advisor.arglist('execution_name1', '10g_trail', 'execution_name2', '11g_trail', 'comparison_metric', 'elapsed_time') ); 
end; 
/

比较物理读

begin 
DBMS_SQLPA.EXECUTE_ANALYSIS_TASK( 
task_name => '10g_11g_spa', 
execution_type => 'COMPARE PERFORMANCE', 
execution_name => 'Compare_physical_reads0', 
execution_params => dbms_advisor.arglist('execution_name1', '10g_trail', 'execution_name2', '11g_trail', 'comparison_metric', 'disk_reads') ); 
end; 
/

Set the comparison_metric parameter to specify an expression of execution 
statistics to use in the performance impact analysis. Possible values include 
the following metrics or any combination of them: elapsed_time (default), 
cpu_time, buffer_gets, disk_reads, direct_writes, and optimizer_cost.

 

 

 

获得SPA报告:

 

 

 

set long 100000 longchunksize 100000 linesize 200 head off feedback off echo off 
spool spa_report_elapsed_time.html 
SELECT dbms_sqlpa.report_analysis_task('SPA_TEST', 'HTML', 'ALL','ALL', execution_name=>'Compare_elapsed_time') FROM dual; 
spool off

产生buffergets 比较report    

set heading off long 100000000 longchunksize 10000 echo off;
set linesize 1000 trimspool on;
spool buffergets_summary.html
select xmltype(dbms_sqlpa.report_analysis_task('10g_11g_spa',
                                                'html',
                                                'typical',
                                                'all',
                                                null,
                                                100,
                                                'compare_10g_112_buffergets')).getclobval(0,0)
from dual;
spool off

产生errors比较report 
spool errors_summary.html
select xmltype(dbms_sqlpa.report_analysis_task('10g_11g_spa',
                                                'html',
                                                'errors',
                                                'summary',
                                                null,
                                                100,
                                                '11g_trail')).getclobval(0,0)
from dual;
spool off

产生unsupport比较report 
spool unsuppor_all.html
select xmltype(dbms_sqlpa.report_analysis_task('10g_11g_spa',
                                                'html',
                                                'unsupported',
                                                'all',
                                                null,
                                                100,
                                                '11g_trail')).getclobval(0,0)
from dual;
spool off

 

 

 

 

 

execution_type
Type of the action to perform by the function. If NULL it will default to the value of the DEFAULT_EXECUTION_TYPE parameter. Possible values are:
[TEST] EXECUTE – test-execute every SQL statement and collect its execution plans and execution statistics. The resulting plans and statistics will be stored in the advisor framework. This is default.
EXPLAIN PLAN – generate explain plan for every statement in the SQL workload. This is similar to the EXPLAIN PLAN command. The resulting plans will be stored in the advisor framework in association with the task.
COMPARE [PERFORMANCE] – analyze and compare two versions of SQL performance data. The performance data is generated by test-executing or generating explain plan of the SQL statements. Use this option when two executions of type EXPLAIN_PLAN or TEST_EXECUTE already exist in the task
CONVERT SQLSET – used to read the statistics captured in a SQL Tuning Set and model them as a task execution. This can be used when you wish to avoid executing the SQL statements because valid data for the experiment already exists in the SQL Tuning Set.

 

 

For 9i Upgrade to 10g

 

 

exec dbms_stats.gather_system_stats(gathering_mode=>'NOWORKLOAD');

alter system set "_optim_peek_user_binds"=false;           ==> 禁用BIND PEEK特性,该特性在10g中有

exec DBMS_STATS.SET_PARAM( 'method_opt','FOR ALL COLUMNS SIZE 1' );
commit;

9i 

?/rdbms/admin/dbmssupp

exec dbms_support.start_trace(binds=>TRUE, waits=> FALSE);

exec dbms_support.stop_trace;

exec dbms_support.start_trace_in_session(sid=>sid,serial=>ser, binds=>TRUE, waits=>FALSE);

select sid,serial# from v$SESSION WHERE ... ;

exec dbms_support.stop_trace_in_session(sid=>SID,serial=>ser);

create table mapping_table tablespace USERS as
select object_id id, owner, substr(object_name, 1, 30) name
  from dba_objects
 where object_type not in ('CONSUMER GROUP',
                           'EVALUATION CONTEXT',
                           'FUNCTION',
                           'INDEXTYPE',
                           'JAVA CLASS',
                           'JAVA DATA',
                           'JAVA RESOURCE',
                           'LIBRARY',
                           'LOB',
                           'OPERATOR',
                           'PACKAGE',
                           'PACKAGE BODY',
                           'PROCEDURE',
                           'QUEUE',
                           'RESOURCE PLAN',
                           'SYNONYM',
                           'TRIGGER',
                           'TYPE',
                           'TYPE BODY')
union all
select user_id id, username owner, null name from dba_users;

declare
  mycur dbms_sqltune.sqlset_cursor;
begin
  dbms_sqltune.create_sqlset('9i_prod_wkld');
    open mycur for
      select value(p)
      from table(dbms_sqltune.select_sql_trace(
                   directory=>'SPADIR',
                   file_name=>'%trc',
                   mapping_table_name => 'MAPPING_TABLE',
                   select_mode => dbms_sqltune.single_execution)) p;
  dbms_sqltune.load_sqlset(
    sqlset_name => '9i_prod_wkld',
    populate_cursor => mycur,
    commit_rows => 1000);

  close mycur;
end;
/

create user spadba identified by oracle;
grant dba to spadba;
grant all on dbms_sqlpa to spadba;

create public database link to10g connect to spadba identified by oracle using 'STRINGS';

var sts_task varchar2(64);
exec :sts_task:= dbms_sqlpa.create_analysis_task(task_name => '9i_11g_spa1',description => 'experiment for 9i to 11gR2 upgrade',sqlset_name=> '9i_prod_wkld');

var exe_task varchar2(64);
exec :exe_task:=dbms_sqlpa.execute_analysis_task(task_name=>'9i_11g_spa1',execution_name=>'9i_trail1',execution_type=>'CONVERT SQLSET',execution_desc=>'9i sql trail generated from sts');

dbms_sqlpa.execute_analysis_task(task_name=>'9i_11g_spa1',execution_name=>'10g_trail1',execution_type=>'TEST EXECUTE',execution_desc=>'10g trail test',-
execution_params=>dbms_advisor.arglist('DATABASE_LINK','DBLINKNAME'));

select sofar,totalwork from V$ADVISOR_PROGRESS where task_id=<TID>;

为什么说log file sync(其实是写redo慢)会造成buffer busy wait?

《gc buffer busy/gcs log flush sync与log file sync》一文中我介绍了 redo flush慢造成RAC中gc buffer busy争用的原理, 而在《【技术分享】开Oracle调优鹰眼,深入理解AWR性能报告》 中我又介绍了 log file sync(其实本质是lgwr 写redo慢)也会造成单实例single instance环境中的buffer busy wait等待, 这是为什么呢?

 

我们来做一个演示说明该问题:

 

示例用表:

conn maclean/oracle

create table  maclog (t1 int);

 

打开一个Session A 并获得它的OS进程号

[oracle@vrh8 ~]$ sqlplus / as sysdba

SQL*Plus: Release 10.2.0.5.0 – Production on Sat Mar 9 09:32:25 2013

Copyright (c) 1982, 2010, Oracle. All Rights Reserved.
Connected to:
Oracle Database 10g Enterprise Edition Release 10.2.0.5.0 – 64bit Production
With the Partitioning, OLAP, Data Mining and Real Application Testing options

 

 

[oracle@vrh8 ~]$ ps -ef|grep LOCAL
oracle 18441 18438 0 09:41 ? 00:00:00 oracleG10R25 (DESCRIPTION=(LOCAL=YES)(ADDRESS=(PROTOCOL=beq)))
oracle 18445 18348 0 09:42 pts/3 00:00:00 grep LOCAL

 

使用 gdb debug 该18375的前台进程,并指定breakpoint

[oracle@vrh8 ~]$ gdb $ORACLE_HOME/bin/oracle 18441
GNU gdb (GDB) Red Hat Enterprise Linux (7.0.1-37.el5)

(gdb) b kcrf_commit_force
Breakpoint 1 at 0x286519c

 

之后在session A执行:

 

SQL> insert into maclog values(1);

1 row created.

若insert hang,则在GDB 中输入多次C  – continue 这样能够继续

 

输入 commit; ==》 此时session A会hang住

 

SQL> commit;   ====》HANG

 

在GDB 中输入bt 可以看到stack call

(gdb) bt
#0 0x000000000286519c in kcrf_commit_force ()
#1 0x00000000028620ef in kcrfw_redo_gen ()
#2 0x00000000010e7dba in kcbchg1_main ()
#3 0x00000000010e6d99 in kcbchg ()
#4 0x000000000143f65a in ktucmt ()
#5 0x00000000013c7a06 in ktcCommitTxn ()
#6 0x00000000042a559e in ktdcmt ()
#7 0x00000000024fe09c in k2lcom ()
#8 0x0000000002418993 in k2send ()
#9 0x0000000001418b47 in xctctl ()
#10 0x00000000014174dd in xctcom_with_options ()
#11 0x000000000211fc26 in kksExecuteCommand ()
#12 0x00000000030ef87a in opiexe ()
#13 0x0000000003232d47 in kpoal8 ()
#14 0x00000000013b7c10 in opiodr ()
#15 0x0000000003c3c9da in ttcpip ()
#16 0x00000000013b3144 in opitsk ()
#17 0x00000000013b60ec in opiino ()
#18 0x00000000013b7c10 in opiodr ()
#19 0x00000000013a92f8 in opidrv ()
#20 0x0000000001fa3936 in sou2o ()
#21 0x000000000072d40b in opimai_real ()
#22 0x000000000072d35c in main ()

其stack call为ktcCommitTxn=> ktucmt => kcbchg => kcbchg1_main => kcrfw_redo_gen => kcrf_commit_force

 

kcbchg==> block change ,为什么要发生block change呢? 因为commit需要对在Buffer Cache里的block做immediate block cleanout

此时开一个session B  查询maclog表

SQL> select * from maclog; ==》阻塞在buffer busy wait上

 

这样就通过 无法完成的immediate block cleanout 去pin住buffer ,来形成了一个buffer busy wait

 

做一个system state dump :
SQL> oradebug setmypid
Statement processed.
SQL>
SQL> oradebug dump systemstate 266;
Statement processed.
SQL> oradebug tracefile_name
/s01/admin/G10R25/udump/g10r25_ora_18551.trc

 

分析该systemstate dump

 

Session B一直在等 buffer busy wait

SO: 0xaa42fff8, type: 4, owner: 0xaa3048f8, flag: INIT/-/-/0x00
(session) sid: 164 trans: (nil), creator: 0xaa3048f8, flag: (100051) USR/- BSY/-/-/-/-/-
DID: 0001-0008-00000002, short-term DID: 0000-0000-00000000
txn branch: (nil)
oct: 0, prv: 0, sql: (nil), psql: (nil), user: 0/SYS
service name: SYS$BACKGROUND
waiting for ‘buffer busy waits’ wait_time=0, seconds since wait started=245
file#=2, block#=89, class#=21
blocking sess=0x(nil) seq=12413
Dumping Session Wait History
for ‘buffer busy waits’ count=1 wait_time=0.215431 sec
file#=2, block#=89, class#=21
for ‘buffer busy waits’ count=1 wait_time=0.977438 sec
file#=2, block#=89, class#=21
for ‘buffer busy waits’ count=1 wait_time=0.977538 sec
file#=2, block#=89, class#=21
for ‘buffer busy waits’ count=1 wait_time=0.977512 sec
file#=2, block#=89, class#=21
for ‘buffer busy waits’ count=1 wait_time=0.977480 sec
file#=2, block#=89, class#=21
for ‘buffer busy waits’ count=1 wait_time=0.977488 sec
file#=2, block#=89, class#=21
for ‘buffer busy waits’ count=1 wait_time=0.977639 sec
file#=2, block#=89, class#=21

其stack call为:

Short stack dump:
ksdxfstk()+32<-ksdxcb()+1573<-sspuser()+111<-__restore_rt()+0<-__GI_semtimedop()+10<-sskgpwwait()+265<-skgpwwait()+162<-kslwaitns_timed()+1102<-kskthbwt()+246<-kslwait(
)+228<-kcbzwb()+1496<-kcbgtcr()+23190<-ktugct()+588<-ktbgcl1()+4711<-ktrgcm()+1979<-ktrget()+486<-kdst_fetch()+524<-kdstf0000001kmP()+3137<-kdsttgr()+2427<-qertbFetch()
+650<-qergsFetch()+444<-opifch2()+2944<-opiall0()+2206<-opikpr()+642<-opiodr()+1184<-rpidrus()+196<-skgmstack()+158<-rpidru()+116<-rpiswu2()+409<-kprball()+1270<-kkescF
etch()+83<-kkedsamp()+6304<-kkedsSel()+1495<-kkecdn()+3055<-kkotap()+859<-kkoiqb()+9830<-kkooqb()+904<-kkoqbc()+2093<-apakkoqb()+167<-apaqbdDescendents()+414<-apaqbdLis
tReverse()+68<-apadrv()+573<-opitca()+1545<-kksLoadChild()+9714<-kxsGetRuntimeLock()+1454<-kksfbc()+14910<-kkspsc0()+979<-kksParseCursor()+142<-opiosq0()+1641<-kpooprx(
)+318<-kpoal8()+964<-opiodr()+1184<-ttcpip()+1226<-opitsk()+1310<-opiino()+1024<-opiodr()+1184<-opidrv()+548<-sou2o()+114<-opimai_real()+163<-main()+116<-__libc_start_m
ain()+244<-_start()+41

 

 

file=2 block=0x89 即 137 md: EXCL  被  owner: 0xaa30b888  PID=22持有

SO: 0xaadd91d8, type: 24, owner: 0xaa44f240, flag: INIT/-/-/0xc0
(buffer) (CR) PR: 0xaa30c878 FLG: 0x0
class bit: (nil)
kcbbfbp: [BH: 0x69fe2948, LINK: 0xaadd9218] (WAITING)
where: ktuwh05: ktugct, why: 0
BH (0x69fe2948) file#: 2 rdba: 0x00800089 (2/137) class: 33 ba: 0x69cc6000
set: 3 blksize: 8192 bsi: 0 set-flg: 0 pwbcnt: 0
dbwrid: 0 obj: -1 objn: 0 tsn: 1 afn: 2
hash: [aacc0958,aacc0958] lru: [75fc3228,7eff15a8]
obj-flags: object_ckpt_list
ckptq: [6ef77368,aadf28f8] fileq: [6ef77378,aadf2938] objq: [a7bdd678,6afdf518]
use: [aaddb748,aaddb748] wait: [aaddef10,aadd9218]
st: XCURRENT md: EXCL tch: 465
flags: mod_started gotten_in_current_mode block_written_once
redo_since_read
change state: ACTIVE
change count: 1
LRBA: [0x2d5.10d5a.0] HSCN: [0x0.242e76a] HSUB: [1]
Using State Objects
—————————————-
SO: 0xaaddb708, type: 24, owner: 0xaa454568, flag: INIT/-/-/0x00
(buffer) PR: 0xaa30b888 FLG: 0x1000
class bit: (nil)
kcbbfbp: [BH: 0x69fe2948, LINK: 0xaaddb748]
where: ktuwh02: ktugus, why: 0
Waiting State Objects
—————————————-
SO: 0xaaddeed0, type: 24, owner: 0xaa450598, flag: INIT/-/-/0xc0
(buffer) PR: 0xaa3048f8 FLG: 0x0
class bit: (nil)
kcbbfbp: [BH: 0x69fe2948, LINK: 0xaaddef10] (WAITING)
where: ktuwh02: ktugus, why: 0

 

简单来说这个示例说明了几点:

  1. OLTP类型的小DML操作一般都会是immediate block cleanout的,这要求在commit之前对block做change kcbchg
  2. 在commit kcrf_commit_force完成前都不会释放对该block buffer的buffer pin
  3. 由上述2点造成的buffer pin最终会影响select和其他insert/update/delete 形成buffer busy wait
  4. 由于慢的lgwr写redo log会造成 kcrf_commit_force commit的缓慢,表现在等待事件上就是log file sync
  5. 由于block cleanout时pin block buffer且commit 慢,则会导致更长时间的buffer busy wait
  6. 若log file sync是由lgwr 写redo log慢(log file parallel write)引起的,则它的另一个效应就是buffer busy wait增多
  7. 若看到AWR中log file sync+buffer busy wait是主要等待事件,则优先解决log file sync ,因为buffer busy wait实际可能是受害者

 

AWR中与commit cleanout相关的 Instance activity 有好几个

 

commit cleanout failures: block lost
commit cleanout failures: buffer being written
commit cleanout failures: callback failure
commit cleanout failures: cannot pin
commit cleanouts
commit cleanouts successfully completed

 

【技术分享】开Oracle调优鹰眼,深入理解AWR性能报告

Oracle调优鹰眼系列只有2讲,对AWR感兴趣的同学更多指标可以参考 【性能调优】Oracle AWR报告指标全解析 https://www.askmac.cn/archives/performance-tuning-oracle-awr.html

 

 

【技术分享】开Oracle调优鹰眼,深入理解AWR性能报告 第一讲https://zcdn.askmac.cn/%E3%80%90Maclean%20Liu%E6%8A%80%E6%9C%AF%E5%88%86%E4%BA%AB%E3%80%91%E5%BC%80Oracle%E8%B0%83%E4%BC%98%E9%B9%B0%E7%9C%BC%EF%BC%8C%E6%B7%B1%E5%85%A5%E7%90%86%E8%A7%A3AWR%E6%80%A7%E8%83%BD%E6%8A%A5%E5%91%8A.mp4

适合参与成员: 对性能优化有兴趣或急于提升自己oracle技术水平的学员

 

 

 

 

 

讲座材料presentation 当前版本下载:

【Maclean Liu技术分享】开Oracle调优鹰眼,深入理解AWR性能报告_20130303版.pdf.pdf (1.79 MB, 下载次数: 32641)

 

【技术分享】开Oracle调优鹰眼,深入理解AWR性能报告 第二讲

涉及性能优化教学知识:Host CPU、Instance CPU、Wait Class、SQL Statistics、AWR FOR RAC集群特定调优

适合的学员: 对性能优化有兴趣,或给予提升自己Oracle调优技能的同学

预计时长: 2个小时左右

本次公开教学的视频地址:

 

 

 

正式版文档材料已上传:

 

【Maclean Liu技术分享】开Oracle调优鹰眼,深入理解AWR性能报告 第二讲 正式版 20130.pdf (2.27 MB, 下载次数: 30699)

_library_cache_advice和latch:shared pool、latch:shared pool simulator

版本10.2.0.4和11.1.0.6中”_library_cache_advice”=TRUE的情况下可能出现高latch:shared pool、latch: shared pool simulator等latch争用等待事件,默认情况下_library_cache_advice受到参数”statistics_level”的影响为TRUE,当_library_cache_advice=TRUE时他启用library cache simulator特性。

 

该library cache simulator特性负责估算shared pool LRU的表现,simulator模拟器收集heap内存堆大小以及load载入、pin、unpin的次数信息;通过这些数据来估算出若我们有更大的shared pool,我们可以由更大的共享池来缓存更多的SQL、PLSQL在共享池中,以此来节约加载时间。若我们设置更小的shared pool size,则又会对加载时间有何等的影响?

 

题外话:另一个对ASMM 下shared pool有作用的参数:

_memory_broker_shrink_heaps:
  • If 0, will not try to shrink shared pool or Java pool
  • If greater than zero, will wait this many seconds after failed shrink request to ask again

 

禁用library cache simulator设置”_library_cache_advice”=false”可能”(具体仍需要诊断)解决高latch:shared pool、latch: shared pool simulator、Library Cache – Mutex X具体等的问题,禁用library cache simulator会导致AWR中”shared pool advisory”和 “java pool advisory”2个环节不可用,但是这些特性实际可有可无。

但是”_library_cache_advice”=false”时且启用了ASMM(sga_target>0)的情况,注意为shared_pool_size设置一个合理的最小值!

 

分别在10.2.0.4和11.1.0.6上进行了针对解析的压力测试:

10204 no change baseline Executes/second = 3,610, DB Time = 12,349s, DB CPU = 8,938s, latch:library cache wait = 598s, avg.wait = 34ms
10204 – _library_cache_advice=off Executes/second = 3,843, DB Time = 16,208s, DB CPU = 9,402s, latch:library cache wait = 616s, avg. wait = 50ms
11106- no change -baseline Executes/second = 3,529, DB Time = 14,148s, DB CPU = 9,286s, library cache: mutex X wait = 2,725s, avg. wait = 1ms
11106 -session_cache=500, instantiation=150 Executes/second = 3,436, DB Time = 13,396s, DB CPU = 9,040s, library cache: mutex X wait = 2,383s avg. wait = 1ms
11106 – _library_cache_advice=off Executes/second = 6,059, DB Time = 75,134s, DB CPU = 17,321s, library cache: mutex X wait = 38,892s,avg. wait = 1ms

 

 

针对高latch:shared pool、latch: shared pool simulator、Library Cache – Mutex X解析类等待事件,解决的思路包括:

  1. 升级到最新的Patch set + PSU
  2. 考虑cursor_sharing=FORCE
  3. 注意即使_optim_peek_user_binds=false,若你的SQL本身还是有硬绑定的自由变量,则dc_histogram仍可能是硬解析争用的焦点
  4. 设置较大的 session_cachced_cursor和instantiation
  5. 设置library_cache_advice=false
  6. 关闭11g中的ACS自适应游标特性
  7. 关闭11g中的cardinality feedback特性
  8. 使用MSSM,或者 ASMM下 _memory_broker_shrink_heaps=0 + _enable_shared_pool_durations=false

沪ICP备14014813号-2

沪公网安备 31010802001379号