Accelerating V8 Performance: In-Place Mutable Heap Numbers for JavaScript

Overview

JavaScript engines like V8 are constantly evolving to squeeze every ounce of performance from the code developers write. One often overlooked area is how numeric values are stored and manipulated, especially when they don't fit into the fast path of small integers (SMIs). This tutorial dives deep into a specific optimization that V8 implemented: in-place mutable heap numbers. The motivation came from a benchmark, async-fs in JetStream2, which showed a surprising bottleneck in a custom Math.random implementation. By allowing certain heap numbers to be mutated in place rather than allocating a new immutable object each time, V8 achieved a 2.5x speedup in that benchmark and an overall boost in the suite score. We'll explore the problem, the solution, and how you can apply similar thinking in your own JavaScript projects.

Prerequisites

• Basic understanding of V8 internals: concepts like tag values, SMIs, heap objects, and pointer compression.
• Familiarity with JavaScript number handling: the distinction between integers and floating-point, and the Math.random API.
• Knowledge of performance profiling: ability to read flame graphs or allocation traces.
• Node.js or Chrome DevTools for testing code snippets.

Step-by-Step Guide

1. Understanding the Original Bottleneck

The async-fs benchmark uses a deterministic pseudo-random number generator for reproducible results. The core function looks like this:

let seed;
Math.random = (function() {
  return function () {
    seed = ((seed + 0x7ed55d16) + (seed << 12))  & 0xffffffff;
    seed = ((seed ^ 0xc761c23c) ^ (seed >>> 19)) & 0xffffffff;
    seed = ((seed + 0x165667b1) + (seed << 5))   & 0xffffffff;
    seed = ((seed + 0xd3a2646c) ^ (seed << 9))   & 0xffffffff;
    seed = ((seed + 0xfd7046c5) + (seed << 3))   & 0xffffffff;
    seed = ((seed ^ 0xb55a4f09) ^ (seed >>> 16)) & 0xffffffff;
    return (seed & 0xfffffff) / 0x10000000;
  };
})();

The variable seed lives in a ScriptContext – an array of tagged values. On 64-bit V8, each slot is 32 bits and uses a tag bit: 0 means a 31-bit SMI, and 1 means a compressed heap pointer. Since the operations involve bitwise and arithmetic up to 32 bits, seed is stored as a heap number (a 64-bit float) pointing to an immutable HeapNumber object. Each call to Math.random must allocate a new HeapNumber on the heap because the original is immutable.

This allocation is costly: it requires memory allocation, garbage collection pressure, and pointer dereferencing. Profiling showed that nearly all time in Math.random was spent on these heap number allocations.

2. The Solution: Mutable Heap Numbers

V8 engineers realized that certain heap numbers – specifically those stored in script contexts or other well-known locations – could be safely mutated in place without breaking object immutability contracts. The key insight is that the location (e.g., a specific slot in a script context) is the only reference to that number, and the value is updated entirely in a single-threaded context. Therefore, instead of creating a new HeapNumber object for each assignment, V8 can directly modify the 64-bit double value in the existing object.

This optimization is enabled by a concept called contextual heap numbers. When V8's optimizing compiler (TurboFan) encounters a store to a property that it knows is a heap number stored in a context, it can generate code that overwrites the double value in place rather than allocating a new object. The result is a dramatic reduction in allocation rate and memory traffic.

3. Implementation Details

The optimization required changes in multiple V8 components:

• Type system: Introduce a new representation for mutable heap numbers in the compiler's intermediate representation.
• Code generation: Emit instructions that directly modify the double value in the heap number object.
• Garbage collector: Treat mutable heap numbers as mutable objects; they are still traced but their fields can change.

The micro‑architecture change is subtle but powerful. For the Math.random example, after the optimization, the assembly code for each assignment becomes something like:

// Assume %rdi points to the HeapNumber object
movsd [%rdi + offset_of_value], %xmm0   // new double value from calculation

This is a single store instruction, whereas before it would have been a call to allocate a new HeapNumber and then store a pointer.

4. Performance Impact

The async-fs benchmark saw a 2.5x improvement in its score. Overall JetStream2 score increased noticeably. In real-world code, patterns that update numbers frequently in closure variables or local scripts can benefit similarly. For example, counters, accumulators, and state machines often update a single numeric variable in a tight loop.

To measure this in your own code, use V8's internal trace flags (e.g., --trace-gc, --trace-opt) or Chrome DevTools memory profiling to observe heap number allocations before and after applying the optimization concept (although the actual optimization is engine‑internal).

Common Mistakes and Pitfalls

1. Assuming all heap numbers are now mutable. This optimization only applies to heap numbers that are stored in specific contexts and are known to have a single, unique reference. Most heap numbers remain immutable.
2. Forgetting that SMIs are still faster. Where possible, keep numbers within SMI range (31‑bit signed) to avoid heap allocation entirely. The mutable heap number optimization still involves a heap object, but the value is updated in place.
3. Not profiling first. Before assuming heap number allocation is a bottleneck, use profiling tools to confirm. The optimization is only beneficial when allocation pressure is high.
4. Misusing assignment. In JavaScript, assigning a new number to a variable always creates a new value. The engine decides how to store it. You cannot manually force mutable heap numbers – it's an internal V8 optimization.

Summary

In‑place mutable heap numbers are a clever optimization that V8 introduced to eliminate a performance cliff in benchmarks and real‑world code. By allowing certain heap numbers to be mutated directly without reallocation, V8 reduces memory traffic and speeds up code that frequently updates numeric values in closure variables or script contexts. The 2.5x gain in the async-fs benchmark demonstrates the impact. While this optimization is transparent to developers, understanding it helps you write number‑crunching code that V8 can optimize effectively. Always keep numbers in SMI range if possible, but when you can't, trust that modern engines are improving the fast paths for mutable numerics.