Java memory problem cases solutions

Java Memory Problem Cases Solution

bluedavy 2012-03

Basic Concepts

Java Runtime Data Area

PC

Local Vars

Operator

Stack Frame

Method Stack Heap

Native method Stack

Method Area

-Xss

-XX:PermSize –XX:MaxPermSize

-Xms -Xmx

Basic Concepts

Oracle JDK Java Heap

Eden S0 S1 Old Generation

New Generation

-XX:SurvivorRatio

-Xmn

Basic Concepts

Oracle JDK startup options for memory manage -Xms –Xmx –Xmn –XX:PermSize –XX:MaxPermSize -XX:SurvivorRatio -XX:+UseParallelGC || -XX:+UseParallelOldGC▪ -XX:MaxTenuringThreshold -XX:ParallelGCThreads▪ -XX:-UseAdaptiveSizePolicy

-XX:+UseConcMarkSweepGC▪ -XX:CMSInitiatingOccupancyFraction -XX:

+UseCMSInitiatingOccupancyOnly -XX:+DisableExplicitGC –XX:

+ExplicitGCInvokesConcurrent

Basic Concepts

Startup options or tools for memory problem solution -XX:+PrintGCDateStamps –XX:+PrintGCDetails -Xloggc -XX:+HeapDumpOnOutOfMemoryError -XX:+HeapDumpBeforeFullGC -XX:+PrintFlagsFinal（ JDK 6u21+） for CMS GC▪ -XX:+PrintPromotionFailure -XX:+CMSDumpAtPromotionFailure

jinfo jstat jmap MAT btrace google perf-tools

Memory problems maybe occured

OOM java.lang.OutOfMemoryError: {$reason}

Full GC Frequently CMS GC

promotion failed concurrent mode failure

OOM Case I

OOM Case II

OOM Case III

-Xms4g –Xmx4g –Xmn2560m

OOM Case IV

OOM Case V

OOM Case V

Then to find who use Deflater.init

OOM

java.lang.OutOfMemoryError: {$reason} GC overhead limit exceeded Java Heap Space Unable to create new native thread PermGen Space Direct buffer memory request {} bytes for {}. Out of swap

space?

OOM will cause

GC overhead limit exceeded Java Heap Space

if code didn’t catch OutOfMemoryError,thread will exit;

slow response or no response at all▪ because gc did frequently

Out of Swap Java process crash

OOM Solution

GC overhead limit exceededJava Heap Space monitor app log & gc log heap dump or jmap –histo|-dump when

oom mat analyze heap dump app developer or btrace to find where

cause

OOM Solution

Out of swap crash log google-perftools then btrace to find where cause

ps: maybe u can try delete -XX:+DisableExplicitGC if exist,reason can be found in this link。

write friendly code to OOM

use ThreadLocal carefully; limit Collection/StringBuilder etc.

size； limit batch ops size; limit the data size return by

database; avoid infinite loop;

Memory problems maybe occured

OOM java.lang.OutOfMemoryError: {$reason}

Full GC Frequently CMS GC

promotion failed concurrent mode failure

Full GC Frequently Case

Most Cases because memory consumes too much

a special case a special frequently GC Case

Full GC frequently will cause

slow response or no response at all

Full GC frequently solution

According gc log to see if the reason is consuming too much memory if yes then add -XX:

+HeapDumpBeforeFullGC or write a shell file to dump memory when full gc;

if no then use pstack & source code to see why full gc executes;

Memory problems maybe occured

OOM java.lang.OutOfMemoryError: {$reason}

Full GC Frequently CMS GC

promotion failed concurrent mode failure

CMS GC

promotion failed Case I 363439.937: [Full GC 363439.938:

[ParNew (promotion failed): 523840K->523840K(523840K), 0.3404490 secs] 1690606K->1712399K(1834560K), 0.3412880 secs]

Taobao JDK help to find the reason▪ ==WARNNING==  allocating large array:

thread_id[0x00002aaac2d85800], thread_name[ajp-0.0.0.0-8009-48], array_size[782611408 bytes], array_length[195652847 elememts]

CMS GC

promotion failed Case II 1768.432: [GC 1768.432: [ParNew (promotion

failed): 1572827K->1572827K(1572864K), 0.2845480 secs]

1972.208: [CMS: 10545160K->5853155K(14680064K), 8.5355320 secs] 12105682K->5853155K(16252928K), [CMS Perm : 20907K->20873K(34952K)], 212.3113910 secs] [Times: user=53.38 sys=22.45, real=212.28 secs]

because cms gc fragmention

CMS GC

four cases promotion failed case III▪ because CMSInitiatingOccupancyFraction too

high... concurrent mode failure case I▪ because CMSInitiatingOccupancyFraction too

high... concurrent mode failure case II▪ because old gen is too small

concurrent mode failure case III▪ because permgen

CMS GC problem will cause

CMS GC problem will cause serial full gc,so the app response time will be very slow...

CMS GC problem solution According gc log to see if the reason is

because CMSInitiatingOccupancyFraction is too high,if yes then adjust the option;

if no,then promotion failed▪ if because memory ran out,then dump memory;▪ if because cms gc fragmention,currently we only

can do is execute jmap –histo:live at regular time; concurrent mode failure▪ the same as promotion failed...

Memory problems maybe occured

OOM java.lang.OutOfMemoryError: {$reason}

Full GC频繁 CMS GC

promotion failed concurrent mode failure

Maybe u’ll occur other problems

Young GC slow CMS GC Concurrent-Mark causes app

stop... ...