How the JVM Runs Your Code: Memory, GC and JIT

Most of the time you can write Java without thinking about the machine underneath. The exceptions are expensive: an OutOfMemoryError at three in the morning, a program that gets slower for hours, a StackOverflowError from a recursive method, or an interviewer who asks "where do objects live, and who frees them?"

The answers come from a good mental model of the Java Virtual Machine (JVM). This article builds one in stages, and it is honest about where a model is a simplification. Where something is a detail of one JVM implementation (HotSpot, the one in OpenJDK and Oracle JDK) rather than something the specification guarantees, the text says so.

• The parts of the JVM and what each does.
• How classes are loaded, linked and initialised.
• What lives on the stack and what lives on the heap.
• How garbage collection works, and why Java programs still leak memory.
• What the JIT compiler does, and why programs warm up.
• Common myths, corrected.

You already know the journey from article 1: source code is compiled to bytecode, and the JVM runs it. Inside the JVM, three groups of machinery do the work:

              .class files                   |        +----------v-----------+        |    Class loading     |   find, load, link, initialise classes        +----------+-----------+                   |   +---------------v----------------------------------+   | Runtime data areas                                |   |   per thread: stack frames, program counter       |   |   shared:     heap (objects), class metadata      |   +---------------+----------------------------------+                   |        +----------v-----------+        |   Execution engine   |   interpreter, JIT compiler, garbage collector        +----------------------+

The Java Virtual Machine Specification defines the runtime data areas (what memory areas must exist) without dictating how a particular JVM implements them. The specification describes a per-thread program counter and stack, a heap shared by all threads, a method area for class structures, and native method stacks. We will focus on the three that matter most for everyday work: classes, the stack and the heap.

A JVM does not load every class at startup. It loads classes on demand. The specification describes three processes: loading (finding the binary form and creating the class), linking (verification of the bytecode, preparing static fields with default values, and resolving symbolic references) and initialisation (running the class's static initialisers).

Initialisation happens at first use. The specification says a class may be initialised only as a result of specific events, such as creating an instance with new, reading or writing a static field, or calling a static method.

// File: Main.javapublic class Main {    static class Config {        static final int MAX = 5;                       // a compile-time constant        static int counter = init("Config.counter");        static {            System.out.println("Config static block runs");        }        static int init(String name) {            System.out.println("initialising " + name);            return 1;        }        static void hello() {            System.out.println("Config.hello()");        }    }    static class Lazy {        static {            System.out.println("Lazy is initialised");        }        Lazy() {            System.out.println("Lazy constructor");        }    }    public static void main(String[] args) {        System.out.println("main starts");        System.out.println("MAX = " + Config.MAX);        // constant: does not trigger initialisation        System.out.println("about to call hello()");        Config.hello();                                  // first active use: initialisation runs now        Config.hello();                                  // already initialised        new Lazy();        new Lazy();        System.out.println("app loader: " + Main.class.getClassLoader().getName());        System.out.println("String loader: " + String.class.getClassLoader());        System.out.println("java.sql loader: " + java.sql.Driver.class.getClassLoader().getName());    }}

main startsMAX = 5about to call hello()initialising Config.counterConfig static block runsConfig.hello()Config.hello()Lazy is initialisedLazy constructorLazy constructorapp loader: appString loader: nulljava.sql loader: platform

Read the order carefully. Config was not initialised when main read Config.MAX, because a static final constant of a primitive type is a compile-time constant that the compiler copies into the caller. Initialisation happened at the first call to hello(): field initialisers and static blocks ran in the order they appear, once. Lazy's static block ran once even though two objects were created.

The last three lines show that classes come from class loaders, arranged in a hierarchy. The bootstrap loader loads the core classes such as String (shown as null, because it is built into the JVM and has no Java object), the platform loader loads other JDK modules (here java.sql), and the application ("app") loader loads your classes. A loader normally asks its parent first, which is why you cannot replace java.lang.String with your own version. Frameworks use custom class loaders to load plugins and reload code, and Spring Boot's launcher is one example. You will rarely write one, but errors such as ClassNotFoundException and NoClassDefFoundError are class loading problems, and now you know where to look: is the class on the classpath?

Two areas of memory matter most for the code you write.

Every thread has its own stack, made of frames. Each time a method is called, the JVM pushes a frame holding that call's local variables (including parameters) and its working data. When the method returns, the frame is popped. This is the call stack that appears in a stack trace.

• Frames are private to the thread: another thread cannot see your local variables.
• Local variables of primitive type hold the value directly in the frame.
• Local variables of reference type hold a reference in the frame, which points to an object elsewhere.
• Stack memory is reclaimed automatically when the method returns, with no garbage collection.
• The stack has a limited size. If methods call each other too deeply, usually through runaway recursion, the JVM throws StackOverflowError. The specification names that error for a stack that grows beyond its permitted size.

All objects and arrays are allocated on the heap. The specification says the heap is shared among all threads and is the area from which memory for all class instances and arrays is allocated, and that heap storage is reclaimed by an automatic storage management system, the garbage collector, because objects are never explicitly deallocated. Fields of an object live inside that object on the heap.

 Stack of the main thread               Heap (shared) +-----------------------+ | main()                | |   int count = 3       | |   Person p  ---------------------->  [ Person | name -------> [ String "Asha" ] ] |   int[] scores -------------------->  [ int[3]: 80, 90, 70 ] +-----------------------+

This matches what you learned about references: p is a local variable (a reference) in the stack frame, and the Person object, its String, and the array are on the heap. Two variables that hold the same reference point to one object.

You may have heard the slogan "primitives live on the stack and objects on the heap". It is a useful first approximation, but it is not the whole truth:

• A primitive field of an object lives inside the object, on the heap.
• Local variables live in stack frames, whether primitive or reference.
• A JIT compiler can do escape analysis: if it proves that an object never leaves a method, it may avoid allocating it on the heap at all, so "every object is on the heap" is a statement about the language model and not a promise about the machine code.
• A third area, the method area, holds class data (the runtime constant pool, method bytecode). In HotSpot it is implemented in native memory called metaspace. That is an implementation detail.

// File: Main.javapublic class Main {    static int depth = 0;    static void recurse() {        depth++;        recurse();                  // never returns: every call adds a frame    }    static int factorial(int n) {        return n <= 1 ? 1 : n * factorial(n - 1);   // deep but finite recursion    }    public static void main(String[] args) {        System.out.println(factorial(10));        try {            recurse();        } catch (StackOverflowError e) {            System.out.println("StackOverflowError after thousands of calls: " + (depth > 1000));        }        System.out.println("the JVM is still fine: " + factorial(5));    }}

3628800StackOverflowError after thousands of calls: truethe JVM is still fine: 120

recurse has no base case, so frames pile up until the stack's limit. The exact depth depends on the stack size and the method, so the program only reports that it was large. The error is catchable here only to demonstrate it. In real code, the fix for StackOverflowError is a base case or an iterative algorithm, not a catch. The stack size can be changed with a JVM option, but a bigger stack mostly postpones the problem.

In languages such as C, the programmer frees memory by hand and mistakes cause crashes and security holes. Java's designers made memory management automatic: the garbage collector (GC) finds objects that are no longer needed and reclaims their memory. Oracle's tuning guide describes the job: the GC automatically manages the application's dynamic memory allocation, handing out memory as requested, determining which parts are still in use, and reclaiming what is not.

An object is garbage when the program can no longer reach it. The collector starts from the GC roots, which include the local variables in all live thread stacks and the static fields of loaded classes, and follows references to find all reachable objects. Anything not found is unreachable and can be reclaimed. Java does not use simple reference counting, which means that two objects that point at each other but that nothing else references (a cycle) are still garbage.

// File: Main.javaimport java.lang.ref.WeakReference;public class Main {    static class Node {        Node other;        final byte[] payload = new byte[1024 * 1024];     // 1 MB so the object is substantial    }    public static void main(String[] args) throws Exception {        Node a = new Node();        Node b = new Node();        a.other = b;        b.other = a;                                       // a cycle        WeakReference<Node> watchA = new WeakReference<>(a);        WeakReference<Node> watchB = new WeakReference<>(b);        System.out.println("before: " + (watchA.get() != null) + " " + (watchB.get() != null));        a = null;        b = null;                                          // no variable reaches the cycle any more        System.gc();                                       // a request, not a command        Thread.sleep(100);        System.out.println("after: " + (watchA.get() != null) + " " + (watchB.get() != null));    }}

before: true trueafter: false false

A weak reference does not keep its target alive: it lets you observe whether the object has been collected. After the two variables were set to null, nothing reached the cycle, and the collector reclaimed both objects despite their references to each other. A word of caution: System.gc() is only a hint. The specification gives no guarantee that a collection happens, so this output is what typical HotSpot runs produce and not a promise, and you should never write real code that depends on it.

Collecting the whole heap every time would be slow. Most collectors exploit an observation about real programs, often called the weak generational hypothesis: most objects die young, such as temporary strings and loop variables, and the few that survive tend to live a long time. Oracle's tuning guide says that HotSpot collectors use generational scavenging together with aging to concentrate effort on the areas of the heap most likely to contain reclaimable memory. In the usual picture:

• New objects are allocated in the young generation. A minor collection there is quick, and most objects are found to be dead.
• Objects that survive several minor collections are promoted to the old generation, which is collected less often, with a more expensive major or full collection.
• Collections can pause the application threads (a "stop-the-world" pause), though modern collectors do much of their work concurrently.

This is a simplified view. Collectors differ in how they lay out the heap (G1 divides it into many regions, for example), and the newest collectors are not strictly generational in the same way. Treat it as a model for reasoning.

The JDK ships several collectors, chosen for different goals. Oracle's JDK 21 documentation says the G1 collector is selected by default on most hardware and operating system configurations.

You choose with JVM options, and for most applications the default is fine. The important point for now is that collection is a trade-off between throughput (how much work the application gets done), latency (how long it pauses) and memory footprint.

Garbage collection frees unreachable objects. A memory leak in Java therefore means objects that are no longer needed but still reachable: something is still holding a reference. The heap grows until the JVM throws OutOfMemoryError.

// File: Main.java// JVM: -Xmx32mimport java.util.ArrayList;import java.util.List;public class Main {    private static final List<byte[]> cache = new ArrayList<>();      // lives as long as the class    public static void main(String[] args) {        try {            while (true) {                cache.add(new byte[1024 * 1024]);                    // "caching" without any eviction            }        } catch (OutOfMemoryError e) {            cache.clear();                                            // release memory so we can print            System.out.println("caught: " + e.getMessage());        }    }}

caught: Java heap space

The program was run with a deliberately small heap so that it fails quickly: the comment on its second line is a note that you start it with java -Xmx32m Main (the -Xmx option sets the maximum heap size). The static list is a GC root, so every array stays reachable and cannot be collected. Real leaks look like this but less obviously.

Common causes of leaks:

• Static collections or caches with no limit or eviction (use a bounded cache, or weak references where appropriate).
• Listeners and callbacks that are registered and never removed.
• ThreadLocal values in long-lived pooled threads that are never removed.
• Inner classes and lambdas that capture a reference to a large enclosing object.
• Unclosed resources (files, connections), which is not strictly heap memory but is a related leak. The garbage collector manages memory, not other resources, so use try-with-resources (article 10).

Things that do not fix memory problems: setting a local variable to null in the middle of a method (almost always pointless), and calling System.gc().

A normal reference is a strong reference, and as long as one exists, the object is kept. Java also provides soft references (cleared when memory is tight, used for memory-sensitive caches), weak references (cleared as soon as the object is otherwise unreachable, used by WeakHashMap) and phantom references (used for clean-up after collection). Beginners rarely need them, and caches in production code usually come from a library.

1. Confirm the symptom: is memory steadily growing, or just high? A JVM often takes as much heap as it is allowed before collecting.
2. Capture evidence: run with -XX:+HeapDumpOnOutOfMemoryError to get a heap dump when it fails, and use a profiler or the JDK's tools (jcmd, jstat, Java Flight Recorder) to see what is growing.
3. Look at what holds the references, not only what is large.
4. Change one thing at a time, and measure.

Bytecode is not machine code. The JVM begins by interpreting bytecode, which starts quickly but runs slowly. While running, it profiles the program to find the hot methods and loops that run often. A just-in-time (JIT) compiler then translates those to optimised machine code while the program is running. HotSpot uses tiered compilation: quick, lightly optimised compilation first, and heavily optimised compilation for the hottest code, using information gathered at runtime.

Because the compiler sees how the program actually behaves, it can perform optimisations that a compiler working ahead of time cannot, such as:

• Inlining: replacing a small method call with the method's body.
• Escape analysis: avoiding heap allocation of objects that never escape a method.
• Removing dead code and unneeded checks once it has observed that they are never needed, with a fallback (deoptimisation) if the assumption is later violated.

Consequences you will notice:

• Warm-up. A Java program is slowest at the start and speeds up as the JIT works. Short-lived programs, such as command-line tools, spend most of their time warming up. Long-running servers benefit.
• Microbenchmarks mislead. A quick System.nanoTime() around a loop measures the interpreter and compiler warming up and not the steady-state code. Use a proper benchmark tool (the OpenJDK project provides JMH) before concluding anything about speed.
• "Interpreted" does not mean "slow". Hot code in a warmed-up JVM can be very fast. Saying that Java is slow because it is interpreted is an outdated claim.

You can watch the compiler work with the -XX:+PrintCompilation option. Its output is voluminous and version-specific, which is why we do not print it here.

• "Primitives are on the stack, objects on the heap." Only a first approximation. See above.
• "Java passes objects by reference." Java passes everything by value. For objects the value is the reference (article 3).
• "System.gc() frees memory." It only requests a collection and should not appear in application code.
• "Java cannot leak memory." It cannot leak unreachable memory, and it leaks reachable-but-unneeded memory often.
• "Setting an object to null frees it." It only removes one reference. The object is collected when none remain.
• Relying on finalize(). It is deprecated for removal and unreliable. Use try-with-resources and, for rare cases, java.lang.ref.Cleaner.
• Catching StackOverflowError or OutOfMemoryError and carrying on. The program may be in an inconsistent state. Fix the cause.
• Benchmarking without warm-up.

• Next: Concurrency Fundamentals: Threads, Races and Synchronisation builds on the stack-and-heap model: each thread has its own stack, and all of them share the heap.
• For finding memory and performance problems: Debugging: Errors, Stack Traces, the Debugger and Logging.
• For the interview version of this topic: Interview Concepts, Part 1.