Upload
kyle-hailey
View
124
Download
6
Embed Size (px)
DESCRIPTION
Citation preview
Buffer Cache Waits
In This Section
1. latch: cache buffers chains
2. latch: cache buffers lru chain 3. Free Buffer Wait
4. Buffer Busy Wait
5. Write Complete Wait
6. Buffer Exterminate
REDO Log FilesREDO Log Files Data FilesData Files
DBWRDBWRLGWRLGWR
User3User3 User2User2User1User1
Log BufferLog Buffer
Buffer CacheBuffer CacheLog Log BufferBuffer
Buffer Buffer CacheCache
SGASGALibrary Library CacheCache
Oracle Memory Structures
RAM :
Machine
Memory
Buffer Cache Access
Buffer Cache ManagementLocating Free blocksFinding data blocksManaging LRU listsCleaning Dirty Blocks
Buffer Cache management can cause contentionDifferent from IO ( reading blocks of disk )
Is Block in cache?
Now you have a file# and block#
How do you know if a block is cached?
ShadowShadow
ProcessProcess ??Do you search all the blocks?
Could be 1000s of blocks to search.
Buffer caches are in the multi Gig
Buffer Cache
Find a block by:
1) Hash of
Data file #
Block#
2) Result = Bucket #
3) Search linked list for that bucket #
What is a hash value
What are Buckets
What is the linked list?
Double Linked Lists
03C39000 03C39478
03C38F60 03C39000
03C3900003C38F60 03C39478Address
Next
Previous
Hashing Function Simple hash could be a Mod function
1 mod 4 = 1 2 mod 4 = 2 3 mod 4 = 3 4 mod 4 = 0 5 mod 4 = 1 6 mod 4 = 2 7 mod 4 = 3 8 mod 4 = 0
Using “mod 4” as a hash function creates 4 “buckets” to store things
Hash Bucket Fill
Data Block Header
11
22
00
33
Hash Block’s file# block #’s
Result in a bucket#Put Block in bucket
?
?
?
?
Hash Block’s 1 file# 437 block
#’s
(1+437) mod 4 = 2After a while the buckets become populated with blocksAfter a while the buckets become populated with blocks
Latches Protect Bucket Contents
BufferHeaders
Data BlocksHashbucket
latches
Buffer Headers contents described by X$BH
X$bh Describes Contents of Buffer Headers
SQL> desc x$bh Name Type ------------ -------- ADDR RAW(4) DBARFIL NUMBER DBABLK NUMBER OBJ NUMBER HLADDR RAW(4) NXT_HASH RAW(4) PRV_HASH RAW(4)… much more
ADDR DBARFIL DBABLK OBJ HLADDR NXT_HASH PRV_HASH…
A each buffer header contains Information about the data block It points to and the previous and next Buffer header in a linked list
X$BH describes Headers
BufferHeaders
Data BlocksHashbucket
latches
HLADDR
NXT_HASH
PRV_HASH
ADDR ADDR
DBARFIL
DBABLKOBJ
x$bh ADDR DBARFIL DBABLK OBJ HLADDR NXT_HASH PRV_HASH
To Find a Block
1. Hash the block address
2. Get Bucket latch
3. Look for header
4. Found, read block in cache
5. Not Found Read block off disk
ShadowShadowProcessProcess
BufferHeaders
Data BlocksHashbucket
22 33 5544
latches
1122334455
Cache Buffers ChainsHash Buckets
s5s5
s4s4
s3s3
s2s2
s1s1
SessionsSessions
Contention if too many accesses on a bucketContention if too many accesses on a bucket
latches BlockHeaders Data
Blocks
Cache Buffer Chain
Examples
S1 S2 S3 S4
1. Look up Table
2. Nested Loops
Select t1.val, t2.val
from t1, t2
where t1.c1 = {value}
and t2.id = t1.id;
t1Index_t2
t2
CBC Solutions
Find SQL ( Why is application hitting the block so hard? ) Nested loops, possibly
Hash Partition Uses Hash Join Hash clusters
Look up tables (“select language from lang_table where ...”) Change application
Use plsql function Spread data out to reduce contention (pctfree=99)
Select from dual Possibly use x$dual
How do you find the SQL?
Which SQL
select count(*), sql_idfrom v$active_session_history ashwhere event like 'latch: cache buffers chains'group by sql_idorder by count(*)/
CNT SQL_ID
---- -------------
84 a09r4dwjpv01q
CBC: OEM
CBC: ADDM
Problem
SQL Statement
Solution?
Finding the Hot Block:
X$BH.TCH
Updated when block read Updated by no more than 1 every 3 seconds Can be used to find “hot” blocks
Note: set back to zero when block cycles through the buffer cache
CBC – Further Investigation
select * from v$event_name
where name = 'latch: cache buffers chains'
EVENT# NAME
---------- ----------------------------
58 latch: cache buffers chains
PARAMETER1 PARAMETER2 PARAMETER3
---------- ---------- ----------
address number tries
NOTE: _db_block_hash_buckets = # of hash buckets
_db_block_hash_latches= # of hash latches
CBC – Real Timeselect
count(*),
lpad(replace(to_char(p1,'XXXXXXXXX'),' ','0'),16,0) laddr
from v$active_session_history
where event='latch: cache buffers chains'
group by p1;
select o.name, bh.dbarfil, bh.dbablk, bh.tch
from x$bh bh, obj$ o
where tch > 100
and hladdr='00000004D8108330'
and o.obj#=bh.obj
order by tch
COUNT(*) LADDR
---------- ----------------
4933 00000004D8108330
NAME DBARFIL DBABLK TCH
----------- ------- ------ ----
EMP_CLUSTER 4 394 120
Putting into one Query
select name, file#, dbablk, obj, tch, hladdr from x$bh bh , obj$ o where o.obj#(+)=bh.obj and hladdr in ( select ltrim(to_char(p1,'XXXXXXXXXX') ) from v$active_session_history where event like 'latch: cache%' group by p1 having count(*) > 5) and tch > 5order by tch
NAME FILE# DBABLK OBJ TCH HLADDR
------------- ----- ------ ------ --- --------
BBW_INDEX 1 110997 66051 17 6BD91180
IDL_UB1$ 1 54837 73 18 6BDB8A80
VIEW$ 1 6885 63 20 6BD91180
VIEW$ 1 6886 63 24 6BDB8A80
DUAL 1 2082 258 32 6BDB8A80
DUAL 1 2081 258 32 6BD91180
MGMT_EMD_PING 3 26479 50312 272 6BDB8A80
This can be misleading, as TCH gets set to 0 ever rap around the LRU and it only gets updated once every 3 seconds, so in this case DUAL was my problem table not MGMT_EMD_PING
Hot Block via Tanel Poder
Sessions
Loop 100,000 times
Latch Holders
Event names
Consistent Read Blocks
Current
Block
(XCUR)
s1s1 s2s2
Update Select
Consistent
Read
(CR)& Undo Both have same file#
and block# and hash to same bucket
latches
CBC: Consistent Read Blocks
Cache Buffer Chain
Contention: Too Many Buffers in BucketContention: Too Many Buffers in Bucket
s5s5
s4s4
s3s3
s2s2
s1s1
Hash Buckets
BlockHeaders
Max length :_db_block_max_cr_dba = 6(in 10g)
Consistent Read Copies
select count(*) , name , file# , dbablk , hladdr from x$bh bh , obj$ owhere o.obj#(+)=bh.obj and hladdr in ( select ltrim(to_char(p1,'XXXXXXXXXX') ) from v$active_session_history where event like 'latch: cache%' group by p1 )group by name,file#, dbablk, hladdrhaving count(*) > 1order by count(*);
CNT NAME FILE# DBABLK HLADDR
--- ---------- ------ ------- --------
14 MYDUAL 1 93170 2C9F4B20
CBC : Solution
Find the SQL causing the problem Change Application Logic
Eliminate hot spots Look up tables
Uses pl/sql functions Minimize data per block (pctfree=99) Possibly using x$dual instead of dual
Oracle added fast dual in 10g , as long as the field “dummy” isn’t accessed uses fast dual
Index Nested loops Hash join Hash partition index Hash Cluster
Updates, inserts , select for update on blocks while reading those blocks
Cause multiple copies
select ash.sql_id, count(*), sql_textfrom v$active_session_history ash, v$sqlstats sqlwhere event='latch: cache buffers chains'and sql.sql_id(+)=ash.sql_idgroup by ash.sql_id, sql_text;
Block Pinning
BufferHeaders
Data BlocksHashbucket
latches
Free Buffer Wait
Finding a Free Block
If the data block isn’t in cache Get a free block and header in the buffer cacheRead it off diskUpdate the free headerRead the block into the buffer cache
Need Free Block to Read in New Data Block
Tune by Increase data blocksTry to tune DBWR
Finding a Free Block
ShadowShadow
ProcessProcess
When a session reads a block When a session reads a block
Into the bufffer cache how does Into the bufffer cache how does
it find a it find a FREEFREE spot? spot?
Finding a Free Block
BufferHeaders
Data BlocksHashbucket
latches
1. Arrange the Buffer Headers into an LRU List
2. Scan LRU for a free block
Cache Buffers LRU
= entry in x$bh
X$bh Describes Buffer Headers
SQL> desc x$bh Name Type ---------------------- --------- ADDR RAW(4) DBARFIL NUMBER DBABLK NUMBER OBJ NUMBER HLADDR RAW(4) NXT_HASH RAW(4) PRV_HASH RAW(4) NXT_REPL RAW(4) PRV_REPL RAW(4)
NXT_REPL RAW(4) PRV_REPL RAW(4)
HLADDR RAW(4) NXT_HASH RAW(4) PRV_HASH RAW(4)
Cache buffer chains
LRU
LRU Chain
03C39000 03C38F60
03C38F60 03C39000
03C3900003C38F60 03C39478ADDR
NXT_HASH
PRV_HASH
03C39478
03C38514
03C38638
03C38620
03C385F4
03C38554
NXT_REPL
PRV_REPL
Cache Buffers LRU list
Cache Buffers LRU list
LRU Chain of Buffer Headers
Buffer Cache
Cache Buffers LRU Latch
MRUMRU LRULRUBuffer Headers
“Cold”
LRU = Least Recently UsedMRU = Most Recently Used
One LRU Latch protects the linked list during changes to the listOne LRU Latch protects the linked list during changes to the list
“Hot”
LRU latch
Session Searching for Free Blocks
MRUMRU LRULRUBuffer Headers
Session
Shadow
1. Go to the LRU end of data blocks
2. Look for first non-dirty block
3. If search too many post DBWR to make free
4. Free Buffer wait
Free Buffer Wait Solutions Tune by
Increase data blocksTry to tune DBWR
ASYNC If no ASYNC use I/O Slaves (dbwr_io_slaves)
Multiple DBWR (db_writer_processes) Direct I/O
Tune Inefficient SQL requesting large # of blocks
Session Finding a Free Block
MRUMRU LRULRU
Hot End
Mid-Point Mid-Point InsertionInsertion
Get LRU LatchGet LRU Latch
Find Free Find Free BlockBlockInsert HeaderInsert Header
Release LRU LatchRelease LRU Latch
session
LRU Latch
Note: FTS read in at the cold end CR copies as well
DBWR taking Dirty Blocks off
MRUMRU LRULRUBuffer Headers LRU
DBWR
Dirty List of Buffer Headers LRUW
latch
LRU latch also covers DBWR list of dirty blocs
Cache Buffers LRU Latch
MRUMRU LRULRU
Mid-Point Mid-Point InsertionInsertion
Oracle Tracks the touch count of blocks. As the Oracle Tracks the touch count of blocks. As the block is pushed to the LRU end, if it’s touch count block is pushed to the LRU end, if it’s touch count is 3 or more, it’s promoted to the MRU endis 3 or more, it’s promoted to the MRU end
Solution: Multiple SetsMultiple Sets
_db_block_lru_latches = 8_db_block_lru_latches = 810gR2 with cpu_count = 210gR2 with cpu_count = 2 X$KCBWDS – set descriptorX$KCBWDS – set descriptor
Set 1Set 1
Set 2Set 2
LRU Latch 1
LRU Latch 2
Test Case
8 Sessions reading separate tables Tables were too big to hold in cache cache option set on each table
Result : lots of buffer cache churn Expected to get “latch: cache buffer chains LRU”
simulator lru latch
CBC – Further Investigation
select p2, count(*)
from v$active_session_history
where event='latch free'
group by p2
select * from v$latchname where latch#=127
P2 COUNT(*)---------- ---------- 127 3556
LATCH# NAME
---------- --------------
127 simulator lru latch
select * from v$event_name
where name = 'latch free'PARAMETER1 PARAMETER2 PARAMETER3
---------- ---------- ----------
address number tries
db_cache_advice
Alter system set db_cache_advice=off;
Group “other” is very small compared to I/O wait time – not a problem
Cache Buffers LRU Latch : Solution Other
Increase Size of Buffer Cache Using multiple cache buffers
Keep, recycle
Possibly increase _db_block_lru_latchesNot supported
write complete waits
Usually happens in tandem with free buffer Tune by
Increase data block cache
Happens because shadow wants to access blocks that are currently being written to disk by DBWR
also seen it happen when there is a lot of write to sort the waits are on block 2 of the temp tablespace file
Write Complete Waits
LRU
DBWR
Dirty List of Buffer Headers LRUW
Session
Buffer Busy Waits
User 1 tries to change a buffer header User 2 has buffer header “locked” (pinned)
11
22
00
33
User1User1User2User2
10g Buffer Busy Waits
Mainly, on 10g (There are a few other cases )
BLOCK CLASS=data block (and some segment header)
Object Type = INDEX Object Type = TABLE
Indicates DML contention To verify use P3. P3 = Block Class (on 8.1.6-9iR2 was BBW type)
Other Class Types : Segment header
When also seeing “data block” on the same object and the object is of OTYPE= “TABLE” then confirms that the TABLE needs to use free lists or ASSM.
File Header Block Most likely extent allocation problems, look at extent size on tablespace and increase
the extent size
Free lists Add free list groups
Undo header Not enough UNDO segments, if using old RBS then switch to AUM “alter system set undo_management=auto scope=spfile;”
1st level bmb Contention on ASSM bitmap blocks, might do better using freelists.
Undo block Hot spot in UNDO, application issue
How to get Class Nameselect rownum n,ws.classfrom v$waitstat;
NAME P1 P2 P3
----------------- ----- ------ -----
buffer busy waits file# block# class#
NAME P1 P2 P3
----------------- ----- ------ -----
buffer busy waits file# block# class#
select * from v$event_namewhere name = 'buffer busy waits'
N CLASS
--- ------------------
1 data block 2 sort block 3 save undo block 4 segment header 5 save undo header 6 free list 7 extent map
8 1st level bmb 9 2nd level bmb 10 3rd level bmb 11 bitmap block 12 bitmap index block 13 file header block 14 unused 15 system undo header 16 system undo block 17 undo header 18 undo block
Note: Before 10g, P3 was BBW type
If P3 in 100,110,120,130 then read
Now “read by other session”
Else Write, P3 in 200,210,220,230, 231
P3 = class#, how do we get class name?
Joining ASH with v$waitstatselect o.object_name obj, o.object_type otype, ash.SQL_ID, w.classfrom v$active_session_history ash, ( select rownum class#, class from v$waitstat ) w, all_objects owhere event='buffer busy waits' and w.class#(+)=ash.p3 and o.object_id (+)= ash.CURRENT_OBJ#Order by sample_time;
OBJ OTYPE SQL_ID CLASS
------ ------ ------------- ------------------
TOTO1 TABLE 8gz51m9hg5yuf data blockTOTO1 TABLE 8gz51m9hg5yuf data blockTOTO1 TABLE 8gz51m9hg5yuf segment headerTOTO1 TABLE 8gz51m9hg5yuf data block
Example: Lack of Free List
S1 S2 S3 S4
4 Sessions running
Insert into toto1 values (null, ‘a’);Commit;
OBJN OTYPE FILEN BLOCKN SQL_ID BLOCK_TYPE
----------- ------ ------ ------ ------------- ------------
54962 TOTO1 TABLE 16 45012 8gz51m9hg5yuf data block 54962 TOTO1 TABLE 16 161 8gz51m9hg5yuf segment header
Example: BBW with Insert
Concurrent inserts will insert into the same blockEach session has to wait for the previous session to
finish it’s writeUsually pretty fastContention builds on highly concurrent applications
Lack of Free Lists Not Using ASSM (Automatic Segment Space
Management)
Solution1: Free Lists
S1 S2 S3 S4
4 Sessions running
Insert into toto values (null, ‘a’);Commit;
Solution2: ASSM
S1 S2 S3 S4
HeaderandLevel 3 BMB
Level 1BMB
Level 1BMB
Level 1BMB
Level 2BMB
Level 1BMB
Tablespace Types : ASSM
select
tablespace_name, extent_management LOCAL, allocation_type EXTENTS, segment_space_management ASSM, initial_extentfrom dba_tablespaces
TABLESPACE_NAME LOCAL EXTENTS ASSM
--------------- ---------- --------- ------
SYSTEM LOCAL SYSTEM MANUALUNDOTBS1 LOCAL SYSTEM MANUALSYSAUX LOCAL SYSTEM AUTOTEMP LOCAL UNIFORM MANUALUSERS LOCAL SYSTEM AUTOEXAMPLE LOCAL SYSTEM AUTODATA LOCAL SYSTEM MANUAL
create tablespace data2 datafile '/d3/kyle/data2_01.dbf' size 200Msegment space management auto;
BBW: ASSM
Consider using Freelists instead of ASSM Normally waits on ASSM blocks should be too
small to warrant using Freelists ASSM is easier, automatically managed
BBW on Index
Index
Session 1
Session 2
Session 3
Increasing index key creates a hot spot on the leading index leaf
OBJN OTYPE FILEN BLOCKN SQL_ID BLOCK_TYPE
--------- ----- ----- ------- ------------- ------------
BBW_INDEX_INDEX 1 113599 97dgthz60u28d data block 1
Use Reverse Key indexesBreaks Index scans
Hash Partition IndexMore IOs per index access
BBW on IndexSolutions
1. Hash Partitions
2. Reverse Keys
BBW on Index : ADDM Recs
Also consider “reversing” the key
BBW: File Header Querying ASH, make sure
P1=current_file#P2=current_block#
If not, use p1, p2 and not current_object#Time P1 P2 OBJN OTYPE FN BLOCKN BLOCK_TYPE
----- --- --- ---- ----- -- ------ -----------------
11:44 202 2 -1 0 0 file header block11:44 202 2 TOTO TABLE 1 60218 file header block11:44 202 2 TOTO TABLE 1 60218 file header block11:44 202 2 TOTO TABLE 1 60218 file header block11:44 202 2 TOTO TABLE 1 60218 file header block
SELECT A.OBJECT_ID FROM ALL_OBJECTS A,
( SELECT * FROM ALL_OBJECTS WHERE ROWNUM < 1000) B ORDER BY A.OBJECT_NAME
Copyright 2006 Kyle Hailey
Temporary File #’s
SQL> select file# from v$datafile; FILE#
----------
1 2 3 4 5 6 7 8SQL> select file# from v$tempfile; FILE#
----------
2 1
SQL> show parameters db_files
NAME VALUE
---------- -------------
db_files 200
Wait Temporary File#’s =
Db_files + file#
File# 202 = v$tempfile 2
BBW : File HeaderTime P1 P2 OBJN OTYPE FN BLOCKN BLOCK_TYPE
----- --- --- ---- ----- -- ------ -----------------
11:44 202 2 TOTO TABLE 1 60218 file header block
Solution is make initial and next extent larger in Temp Table Space
ADDM doesn’t say much
Buffer Exterminate
Buffer cache dynamically resized
V$SGA_DYNAMIC_COMPONENTS displays information about the dynamic SGA components. This view summarizes information based on all completed SGA resize operations since instance startup.
V$SGA_CURRENT_RESIZE_OPS displays information about SGA resize operations which are currently in progress. An operation can be a grow or a shrink of a dynamic SGA component.
V$SGA_DYNAMIC_FREE_MEMORY displays information about the amount of SGA memory available for future dynamic SGA resize operations.
Alter system set db_cache_size=50M;
Summary Buffer Cache Waits
1. latch: cache buffers chains - find SQL Eliminate hot spots
2. latch: cache buffers lru chain – increase sets
3. Free Buffer Wait - increase cache size
4. Buffer Busy Wait Index : alleviate hot spots, partition Table: add free lists or use ASSM File Segment Header : looked at high extent
allocations
5. Write Complete Waits - increase cache size