Memory footprint of the JVM (Heap & Non-Heap Memory)

JVM consumes the available space on host OS memory. However, inside the JVM, there are separate memory spaces to store different types of data and code.

The JVM divides its memory into two main categories: heap memory and non-heap memory.

1) Heap Memory

The Java Heap is the area where all java class instances and arrays are allocated at runtime.

It is managed by the JVM's garbage collector, which automatically handles memory allocation and deallocation for objects that are no longer in use.

The size of the Java Heap can be configured using JVM command-line options, such as -Xmx (maximum heap size) and -Xms (initial heap size).

As the application runs, the JVM may dynamically adjust the size of the heap based on the application's memory requirements and the garbage collector's behavior.

The Java Heap is further divided into regions, such as the Young Generation, Old Generation, and possibly other special regions like Eden space and Survivor spaces.

These regions are used by the garbage collector to manage object lifetimes, promote long-lived objects, and perform garbage collection to free memory occupied by unreachable objects.

1.1) Young Generation

This is where new objects are allocated. It is further divided into three parts — Eden Memory and two Survivor Memory spaces (S0, S1).

A) Eden Space

This is the initial allocation space where new objects are created.

When the Eden Space fills up, a minor garbage collection (young generation garbage collection) is triggered to reclaim the memory occupied by unreachable objects and move the surviving objects to one of the survivor spaces.

Minor GC also checks the survivor objects and move them to the other survivor space. So at a time, one of the survivor space is always empty.

B) Survivor Spaces (S0 and S1)

The Survivor Spaces hold objects that have survived one or more garbage collection cycles.

Objects that are survived after many cycles of GC, are moved to the Old generation memory space. Usually it's done by setting a threshold for the age of the young generation objects before they become eligible to promote to Old generation.

1.2) Old Generation (Tenured Generation)

This generation contains objects that have survived multiple garbage collection cycles in the Young Generation.

When Old Gen space is full, Major GC (Old Collection) is performed.

Major garbage collections are performed less frequently in the Old Generation compared to the Young Generation because they are more expensive in terms of time and processing.

2) Non-Heap Memory/Native memory

The Non-Heap memory is an area in the JVM that stores other data structures and runtime information that are not part of the Java Heap. The JVM's non-heap memory is divided into several different areas:

2.1) Metaspace (replaces Permanent Generation - PermGen)

Since Java 8, Permanent Generation is replaced by Metaspace. It can auto increase its size (up to what the underlying OS provides) even though Perm Gen always has a fixed maximum size (e.g., -XX:PermSize and -XX:MaxPermSize options).

This change was made due to variety of reasons, including but not limited to:

A) The required size of permgen was hard to predict. It resulted in either under-provisioning triggering java.lang.OutOfMemoryError: Permgen size errors or over-provisioning resulting in wasted resources.

B) GC performance improvements, enabling concurrent class data de-allocation without GC pauses and specific iterators on metadata.

C) Support for further optimizations such as G1 concurrent class unloading.

As a class is loaded by the JVM, its metadata (i.e. its runtime representation in the JVM) is allocated into the Metaspace.

In Java 8, the memory layout of the HotSpot JVM underwent significant changes compared to earlier versions. These changes led to a clear separation of memory regions, including the introduction of Metaspace and the reorganization of other memory regions.

The Method Area, which contains artifacts like the run-time constant pool, field and method data, and the bytecode for methods and constructors, has been integrated into the Metaspace.

The Metaspace occupancy grows as more and more classes are loaded. And, when a classloader and all its loaded classes become unreachable in the Java heap, the associated class metadata in the Metaspace becomes eligible for deallocation.

Cleaning/Resizing of metaspace requires full GCs. However, they are not collected on most full GCs, but only collected when full.

The JVM option -XX:+PrintFlagsFinal can be used to see default values set by the JVM.

There are two JVM options, -XX:MetaspaceSize and -XX:MaxMetaspaceSize that can be used to control the size of the Metaspace.

-XX:MetaspaceSize sets the initial capacity and threshold for the Metaspace at which a GC is invoked to clean up the space, or to expand the capacity if enough space can not be reclaimed.

-XX:MaxMetaspaceSize sets the maximum size of the metaspace. Default is unlimited, which means that the system memory is the limit.

If the size of the metaspace is not configured large enough, it can cause java.lang.OutOfMemoryError: Metaspace.

2.2) Code Cache

The code cache in Java is a reserved memory area where the Just-In-Time (JIT) compiler stores compiled native code.

When Java bytecode is executed, the JIT compiler dynamically compiles frequently executed parts of the bytecode into native machine code, which is then stored in the code cache for faster execution in subsequent invocations.

The code cache has a fixed size, which means there is a limit to the amount of native code that can be stored in it. Once the code cache is full, the JVM won't be able to store additional compiled code.

When this happens, the JIT compiler is effectively turned off, and the JVM will fall back to interpreting the bytecode rather than using the optimized native code. Furthermore, at times we find something interesting in the logs - CodeCache is full. The compiler has been disabled. Try increasing the code cache size using -XX:ReservedCodeCacheSize=

As a result of the JIT compiler deactivation, the application's performance may degrade significantly.

The "InitialCodeCacheSize", "ReservedCodeCacheSize", and "CodeCacheExpansionSize" options can be adjusted to allocate more memory for the code cache and allow it to handle a larger number of compiled methods.

InitialCodeCacheSize - This option sets the initial size of the code cache when the JVM starts (default is 250mb).

ReservedCodeCacheSize - This option sets the maximum size of the code cache (default is 250mb).

CodeCacheExpansionSize - This option sets the expansion size for the code cache when it needs to grow (default is 64kb).

Increasing the ReservedCodeCacheSize can provide a temporary solution, but it's important to monitor the application's behavior to ensure the code cache doesn't keep growing indefinitely and consume excessive memory.

The JVM also provides the UseCodeCacheFlushing option, which can be set to true to enable flushing of the code cache area under certain conditions. This option helps in freeing up memory in the code cache when it becomes full or when certain precompiled code is not considered hot enough (i.e., it's not frequently used).

2.3) Direct Memory

Direct memory is allocated outside of the Java heap (using the java.nio.ByteBuffer class), and allows storing data that is needed by native libraries or I/O operations.

By default, the JVM does not set a limit on how much memory is used for Direct Byte Buffers. A soft limit of 64 MB is set, which the JVM automatically expands in 32 MB chunks, as required.

The java.nio.DirectByteBuffer class is a special implementation of the java.nio.ByteBuffer class in Java. It is designed to work with native (direct) memory rather than memory managed by the Java heap.

Unlike traditional ByteBuffer instances that are backed by a byte[] array in the Java heap, DirectByteBuffer instances use memory allocated outside of the Java heap.

We can limit the direct buffer memory size using the –XX:MaxDirectMemorySize parameter.

If this memory space outside of heap is exhausted, the "java.lang.OutOfMemoryError: Direct buffer memory" Error will be throw.

2.4) Thread Stacks

Each thread in a Java application has its own stack, used for storing local variables and method call information. The size of thread stacks can be configured using the -Xss or -XX:ThreadStackSize option.

When the method finishes execution, its corresponding stack frame is flushed, the flow goes back to the calling method, and space becomes available for the next method.

If this memory is full, Java throws java.lang.StackOverFlowError.

Native Memory Tracking (NMT)

To monitor native memory usage in a Java application, you can enable Native Memory Tracking (NMT) using the -XX:NativeMemoryTracking JVM tuning flag. The -XX:NativeMemoryTracking flag can take one of the following three values: off|sumary|detail

java -XX:NativeMemoryTracking=detail -jar app.jar

Once you enable Native Memory Tracking (NMT), you can retrieve the native memory information at any time using the jcmd command.

jcmd 12114 VM.native_memory

12114:

Native Memory Tracking:

(Omitting categories weighting less than 1KB)

Total: reserved=3637060KB, committed=275948KB
-                 Java Heap (reserved=2097152KB, committed=87040KB)
                            (mmap: reserved=2097152KB, committed=87040KB) 
 
-                     Class (reserved=1049547KB, committed=7371KB)
                            (classes #10997)
                            (  instance classes #10253, array classes #744)
                            (malloc=971KB #23072) 
                            (mmap: reserved=1048576KB, committed=6400KB) 
                            (  Metadata:   )
                            (    reserved=40960KB, committed=39296KB)
                            (    used=38989KB)
                            (    waste=307KB =0.78%)
                            (  Class space:)
                            (    reserved=1048576KB, committed=6400KB)
                            (    used=6188KB)
                            (    waste=212KB =3.31%)
 
-                    Thread (reserved=33873KB, committed=33873KB)
                            (thread #34)
                            (stack: reserved=33792KB, committed=33792KB)
                            (malloc=44KB #202) 
                            (arena=37KB #64)
 
-                      Code (reserved=248548KB, committed=16152KB)
                            (malloc=860KB #5657) 
                            (mmap: reserved=247688KB, committed=15292KB) 
 
-                        GC (reserved=128566KB, committed=53978KB)
                            (malloc=17442KB #4956) 
                            (mmap: reserved=111124KB, committed=36536KB) 
 
-                  Compiler (reserved=229KB, committed=229KB)
                            (malloc=64KB #472) 
                            (arena=165KB #5)
 
-                  Internal (reserved=341KB, committed=341KB)
                            (malloc=309KB #7312) 
                            (mmap: reserved=32KB, committed=32KB) 
 
-                     Other (reserved=8256KB, committed=8256KB)
                            (malloc=8256KB #14) 
 
-                    Symbol (reserved=10777KB, committed=10777KB)
                            (malloc=10226KB #255605) 
                            (arena=552KB #1)
 
-    Native Memory Tracking (reserved=5199KB, committed=5199KB)
                            (malloc=257KB #3660) 
                            (tracking overhead=4942KB)
 
-        Shared class space (reserved=12288KB, committed=12112KB)
                            (mmap: reserved=12288KB, committed=12112KB) 
 
-               Arena Chunk (reserved=175KB, committed=175KB)
                            (malloc=175KB) 
 
-                   Logging (reserved=5KB, committed=5KB)
                            (malloc=5KB #217) 
 
-                 Arguments (reserved=2KB, committed=2KB)
                            (malloc=2KB #53) 
 
-                    Module (reserved=109KB, committed=109KB)
                            (malloc=109KB #1229) 
 
-                 Safepoint (reserved=8KB, committed=8KB)
                            (mmap: reserved=8KB, committed=8KB) 
 
-           Synchronization (reserved=835KB, committed=835KB)
                            (malloc=835KB #13612) 
 
-            Serviceability (reserved=1KB, committed=1KB)
                            (malloc=1KB #14) 
 
-                 Metaspace (reserved=41148KB, committed=39484KB)
                            (malloc=188KB #107) 
                            (mmap: reserved=40960KB, committed=39296KB) 
 
-      String Deduplication (reserved=1KB, committed=1KB)
                            (malloc=1KB #8) 

NMT can provide very detailed information about a map of the entire memory space.