<?xml version="1.0" encoding="UTF-8"?><rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0"><channel><title><![CDATA[Slow but constant memory increase rendering xlsx templates]]></title><description><![CDATA[<p>Re: <a href="/topic/3529/slow-but-constant-memory-leak-with-chrome-pdf-recipe-on-docker-jsreport-4-12-0">Slow but constant memory leak with chrome-pdf recipe on Docker (jsreport 4.12.0)</a></p>
<p>Hello, following the discussion about chrome-pdf, I investigated similar memory behavior with large XLSX reports after a customer experienced OOM errors during repeated report generation. Memory appeared to remain elevated even after rendering had completed.</p>
<p>Below is the analysis, carried out with the help of GPT 6 Astra, along with the patch it helped me develop. I'd like to share it for your review and possible inclusion in a future release.</p>
<h1>Cached async Handlebars request-data retention in XLSX reports</h1>
<h2>Summary</h2>
<p>We identified and reproduced JavaScript object retention caused by the async<br />
Handlebars adapter mutating a cached compiled template specification.</p>
<p>The adapter attaches a decorator that closes over a per-render Handlebars<br />
instance. That instance holds registered helper closures, which can retain<br />
request context and report data. Handlebars also attaches that decorator to<br />
the compiled program function. The cached specification therefore keeps<br />
request-specific objects reachable after rendering has finished.</p>
<p>The proposed fix creates a private specification and private program-function<br />
wrappers for each invocation. It preserves compilation caching without letting<br />
the cached objects point back to the request.</p>
<p>This correction was applied downstream and exercised with real XLSX workloads<br />
in a production-like Kubernetes clone. The minimal reproduction below does<br />
not require XLSX, customer data, or the custom extension.</p>
<p>This finding concerns async Handlebars, as used by our XLSX transformation<br />
path. It does not establish the same root cause for every Chrome PDF memory<br />
issue.</p>
<h2>Environment and scope</h2>
<table class="table table-bordered table-striped">
<thead>
<tr>
<th>Component</th>
<th>Investigated configuration</th>
</tr>
</thead>
<tbody>
<tr>
<td>jsreport</td>
<td>4.12.0</td>
</tr>
<tr>
<td>jsreport core</td>
<td>4.9.0</td>
</tr>
<tr>
<td><code>@jsreport/jsreport-handlebars</code></td>
<td>4.1.0</td>
</tr>
<tr>
<td>Handlebars</td>
<td>4.7.7</td>
</tr>
<tr>
<td>Node.js</td>
<td>22.23.1</td>
</tr>
<tr>
<td>Operating system / allocator</td>
<td>Debian Bookworm / glibc 2.36</td>
</tr>
<tr>
<td>XLSX extension</td>
<td>Custom extension declaring version 4.6.1</td>
</tr>
<tr>
<td>Initial isolated investigation</td>
<td>One persistent rendering worker</td>
</tr>
<tr>
<td>Subsequent clone deployment</td>
<td>Three replicas, four rendering workers per replica</td>
</tr>
</tbody>
</table>
<p>The custom XLSX extension is relevant to interpreting the release versions:<br />
its <code>lib/generation/processXlsx.js</code> is byte-for-byte identical to the file<br />
published in jsreport 4.13.0. This was checked both locally and in the installed<br />
extension in the clone. It does not imply that the entire custom extension is<br />
identical to upstream 4.13.0.</p>
<h2>Rendering flow</h2>
<ol>
<li>A rendering worker receives the request, loads the template and helpers,<br />
and prepares the report data.</li>
<li>jsreport compiles the template specification and caches it in that worker.</li>
<li>A per-render Handlebars instance is created and request-specific helper<br />
functions are registered on it.</li>
<li>The XLSX transformation sets <code>req.context.asyncHandlebars = true</code> and<br />
evaluates the original report content against <code>req.data</code>, including parsed<br />
XLSX structures and the report dataset.</li>
<li>The async adapter creates the runtime template from the cached specification.</li>
<li>After rendering, request-specific objects should become collectible while<br />
the compiled specification remains cached for reuse.</li>
</ol>
<p>The relevant implementation locations are:</p>
<ul>
<li><code>packages/jsreport-core/lib/worker/render/executeEngine.js</code></li>
<li><code>packages/jsreport-handlebars/lib/handlebarsEngine.js</code></li>
<li><code>packages/jsreport-handlebars/async-helpers/index.js</code></li>
<li><code>packages/jsreport-xlsx/lib/transformation/index.js</code></li>
<li>Handlebars' <code>lib/handlebars/runtime.js</code></li>
</ul>
<p>The Handlebars engine's compilation code explicitly intends to return a<br />
specification that is independent of the current Handlebars instance so it<br />
can be cached safely.</p>
<h2>Root cause</h2>
<h3>Mutation of the cached specification</h3>
<p>The original async adapter assigns <code>spec.main_d</code> inside<br />
<code>handlebars.template(spec)</code>. Its decorator references the per-render<br />
<code>handlebars</code> instance, including <code>handlebars.wrapHelperResult</code>.</p>
<p>The specification passed to this function is the same object retained by<br />
jsreport's template cache.</p>
<h3>Mutation of shared program functions</h3>
<p>Handlebars' runtime also performs:</p>
<pre><code class="language-js">templateSpec.main.decorator = templateSpec.main_d
</code></pre>
<p>For numbered program functions, it similarly attaches the corresponding<br />
decorator to the program function object.</p>
<p>Consequently, copying only the specification object would not be enough:<br />
its <code>main</code> and numbered program functions would still be shared with the<br />
cached specification.</p>
<h3>Effective retention path</h3>
<pre><code class="language-text">Worker template cache
  -&gt; compiled template specification / program function
  -&gt; async decorator
  -&gt; per-render Handlebars instance
  -&gt; registered helper closures
  -&gt; request context and report data
</code></pre>
<p>This path was established through source inspection and a standalone<br />
reachability reproduction. It is not a claim based on an exported heap<br />
snapshot.</p>
<p>As long as this path remains reachable, GC cannot collect the corresponding<br />
objects. This is different from an allocator delaying the return of already<br />
freed memory to the operating system.</p>
<h3>Retention does not necessarily accumulate once per render</h3>
<p>Repeated executions of the same cached specification replaced the reference<br />
to the previous runtime in our reproduction. They retained the latest<br />
payload, rather than retaining every historical payload.</p>
<p>However, different cached specifications and different workers can each<br />
retain a large payload. Cache warming, additional templates, or changing<br />
payload sizes can therefore make the application footprint grow.</p>
<h2>Observed evidence</h2>
<h3>Isolated real XLSX workload</h3>
<p>With one rendering worker and the original adapter, a completed heavy XLSX<br />
render left approximately <strong>707 MiB of worker heap after spontaneous major<br />
GC</strong>. Repeating the same report returned to approximately the same retained<br />
level.</p>
<p>After applying the adapter correction and restarting the diagnostic process,<br />
two executions of the same heavy report returned the worker heap to<br />
approximately <strong>39-40 MiB after spontaneous GC</strong>.</p>
<p>The patched process RSS after those executions was approximately<br />
<strong>269-292 MiB</strong>. These RSS numbers are not equivalent to JavaScript heap usage.</p>
<p>No application GC was forced for these real-report measurements.</p>
<h3>Standalone reachability checks</h3>
<p>A synthetic reproduction kept the compiled specification strongly reachable,<br />
registered a helper closing over a payload, and held only <code>WeakRef</code> references<br />
to the payload and per-render Handlebars instance.</p>
<ul>
<li>Ordinary Handlebars released the payload and runtime after GC.</li>
<li>The original async adapter retained them while the specification was cached.</li>
<li>A second execution released the first payload but retained the new one.</li>
<li>The original adapter also retained the payload after a helper threw.</li>
<li>The patched async adapter released the payload and runtime.</li>
</ul>
<p>Forced GC was used only in disposable synthetic test processes.</p>
<h3>Clone workload checks</h3>
<p>The correction was also applied to the three normal application replicas in<br />
the clone, retaining four rendering workers per replica.</p>
<p>In a later controlled workload, all twelve worker heaps returned to around<br />
40 MiB after twelve heavy XLSX renders, including two concurrent reports per<br />
replica. XLSX outputs were readable archives and retained the expected workbook,<br />
worksheet, row, and cell counts.</p>
<p>These are structural output checks, not a claim that every cell was manually<br />
reviewed or that the generated files were byte-identical.</p>
<p>Subsequent user-triggered report workloads also returned all twelve worker<br />
heaps to approximately 39-42 MiB after the final rendering activity and idle<br />
period. There were no observed OOM events or container restarts in these runs.</p>
<p>The observations support the correction for the reproduced retention path.<br />
They do not prove the absence of every other memory issue or arbitrary<br />
long-term stability.</p>
<h2>Exact adapter patch applied</h2>
<p>Target: <code>packages/jsreport-handlebars/async-helpers/index.js</code>.</p>
<p>The diff below uses normalized LF line endings. Our downstream installer<br />
normalizes CRLF before checking and patching the source.</p>
<pre><code class="language-diff">diff --git a/packages/jsreport-handlebars/async-helpers/index.js b/packages/jsreport-handlebars/async-helpers/index.js
--- a/packages/jsreport-handlebars/async-helpers/index.js
+++ b/packages/jsreport-handlebars/async-helpers/index.js
@@ -47,7 +47,19 @@
   const _template = handlebars.VM.template
   handlebars.wrapHelperResult = (p) =&gt; p
   handlebars.template = function (spec) {
-    spec.main_d = (prog, props, container, depth, data, blockParams, depths) =&gt; async (context) =&gt; {
+    const renderSpec = { ...spec }
+
+    // The runtime attaches decorators to program functions; keep cached functions untouched.
+    for (const key of Object.keys(renderSpec)) {
+      if ((key === 'main' || /^\d+$/.test(key)) &amp;&amp; typeof renderSpec[key] === 'function') {
+        const program = renderSpec[key]
+        renderSpec[key] = function (...args) {
+          return program.apply(this, args)
+        }
+      }
+    }
+
+    renderSpec.main_d = (prog, props, container, depth, data, blockParams, depths) =&gt; async (context) =&gt; {
       const originalFn = container.fn
       container.fn = (...args) =&gt; {
         const rf = originalFn(...args)
@@ -58,11 +70,11 @@
       }
 
       // here I've changed the last param from `depths` to `[context]`. This was needed to make the ../gotoparent working
-      const v = spec.main(container, context, container.helpers, container.partials, data, blockParams, [context])
+      const v = renderSpec.main(container, context, container.helpers, container.partials, data, blockParams, [context])
       // result can be actually SafeString
       return v.then((r) =&gt; r.toString())
     }
-    return _template(spec, handlebars)
+    return _template(renderSpec, handlebars)
   }
 
   handlebars.compile = function (template, options) {
</code></pre>
<h3>Why this fixes the reproduced path</h3>
<ul>
<li><code>renderSpec</code> is private to the runtime invocation.</li>
<li>Its main and numbered program functions are new wrapper objects, so runtime<br />
decorator assignments do not modify the cached functions.</li>
<li>The wrappers delegate to the original compiled code with the same arguments<br />
and <code>this</code>.</li>
<li>The per-render decorator is attached only to the private specification.</li>
<li>The cache continues to retain compiled code, not the request-specific runtime.</li>
</ul>
<p>This does not deep-copy the request data, disable compilation caching, force<br />
GC, or recycle workers.</p>
<p>The fix should be applied before workers start, or followed by a restart.<br />
Replacing the adapter file does not retroactively clean already-mutated<br />
specifications or reload modules in a running process.</p>
<h2>Minimal reproduction</h2>
<p>Save the following as <code>repro.cjs</code> in a project with<br />
<code>@jsreport/jsreport-handlebars</code> 4.1.0 and Handlebars 4.7.7 installed.<br />
It invokes the engine directly and does not require a running jsreport server.</p>
<pre><code class="language-js">const assert = require('node:assert/strict');
const path = require('node:path');
const { setImmediate: nextTurn } = require('node:timers/promises');

const engine = require('@jsreport/jsreport-handlebars/lib/handlebarsEngine')({
  handlebarsModulePath: path.dirname(require.resolve('handlebars/package.json')),
});
const cache = new Map();

function runtime() {
  const context = engine.createContext({
    context: { asyncHandlebars: true },
  });
  return {
    context,
    require(name) {
      assert.equal(name, 'handlebars');
      return context.handlebars;
    },
  };
}

async function execute(spec) {
  const current = runtime();
  const payload = { rows: new Array(10000).fill(123) };
  const references = {
    payload: new WeakRef(payload),
    handlebars: new WeakRef(current.context.handlebars),
  };
  const output = await engine.execute(
    spec,
    { value: () =&gt; payload.rows.length },
    {},
    { require: current.require },
  );
  assert.equal(output, '10000');
  return references;
}

async function main() {
  assert.equal(typeof global.gc, 'function', 'Run with --expose-gc');
  const compiler = runtime();
  const spec = engine.compile('{{value}}', { require: compiler.require });
  cache.set('template', spec);
  const references = await execute(spec);

  for (let i = 0; i &lt; 4; i++) {
    await nextTurn();
    global.gc();
  }

  assert.equal(cache.get('template'), spec);
  const retainedPayload = references.payload.deref() !== undefined;
  const retainedHandlebars = references.handlebars.deref() !== undefined;
  const expected = process.env.HANDLEBARS_EXPECT_RETENTION === 'true';
  assert.equal(retainedPayload, expected);
  assert.equal(retainedHandlebars, expected);
  console.log(JSON.stringify({
    retainedPayload,
    retainedHandlebars,
    cachedSpecifications: cache.size,
  }));
}

main().catch((error) =&gt; {
  console.error(error);
  process.exitCode = 1;
});
</code></pre>
<p>On an unpatched installation:</p>
<pre><code class="language-sh">HANDLEBARS_EXPECT_RETENTION=true node --expose-gc repro.cjs
</code></pre>
<p>Expected result:</p>
<pre><code class="language-json">{&quot;retainedPayload&quot;:true,&quot;retainedHandlebars&quot;:true,&quot;cachedSpecifications&quot;:1}
</code></pre>
<p>After applying the adapter correction:</p>
<pre><code class="language-sh">HANDLEBARS_EXPECT_RETENTION=false node --expose-gc repro.cjs
</code></pre>
<p>Expected result:</p>
<pre><code class="language-json">{&quot;retainedPayload&quot;:false,&quot;retainedHandlebars&quot;:false,&quot;cachedSpecifications&quot;:1}
</code></pre>
<h2>Additional regression coverage</h2>
<p>Our broader synthetic harness exercised:</p>
<ul>
<li>Ordinary Handlebars as a non-retaining control.</li>
<li>Repeated execution of the same cached async specification.</li>
<li>Cleanup after a helper throws.</li>
<li>Two concurrent executions sharing the same compiled specification.</li>
<li>Async helpers inside <code>each</code>, parent-context lookup, and block parameters.</li>
<li>Partials, escaped output, and unescaped output.</li>
<li>Execution with the specification and compiled program functions frozen.</li>
<li>Absence of <code>main_d</code> and <code>main.decorator</code> mutations on the cached objects.</li>
</ul>
<p>The frozen-specification case is particularly useful for protecting the<br />
cache-immutability invariant.</p>
<p>Upstream review should also consider supported custom decorators and any<br />
function metadata expectations not represented by these cases.</p>
<h2>Relation to the XLSX improvements in jsreport 4.13.0</h2>
<p>The 4.13.0 release includes two relevant generation improvements:</p>
<ol>
<li>Skip generation work when the input workbook has no Handlebars tags.</li>
<li>Separate dynamic data evaluation from final XML construction, excluding<br />
static XML and <code>sharedStrings.xml</code> from unnecessary evaluation.</li>
</ol>
<p>These changes improve the generation path. They do not replace the async<br />
Handlebars retention correction.</p>
<p>The async adapter source is identical between the 4.12.0 and 4.13.0 tags.<br />
In 4.13.0 it still assigns <code>spec.main_d</code> and passes the shared specification<br />
to <code>_template(spec, handlebars)</code>. The legacy XLSX transformation still enables<br />
async Handlebars.</p>
<p>The release updates <code>@jsreport/jsreport-handlebars</code> from 4.1.0 to 4.1.1 and<br />
Handlebars from 4.7.7 to 4.7.9, but Handlebars' runtime still attaches<br />
decorators to the compiled program functions.</p>
<p>This conclusion is based on tagged-source comparison. We did not perform a<br />
full application upgrade or an end-to-end benchmark of jsreport 4.13.0.</p>
<h2>Separate issue: native allocator RSS retention</h2>
<p>After fixing the JavaScript retention path, process RSS can still remain<br />
higher than its cold baseline.</p>
<p>In a separate controlled experiment on an idle clone replica, glibc reported<br />
approximately 583 MiB of free blocks in its arenas. A single explicitly<br />
authorized <code>malloc_trim(0)</code> reduced RSS from approximately <strong>912 MiB to<br />
461 MiB</strong>, while the measured worker heap stayed around <strong>40 MiB</strong>.</p>
<p>This demonstrates a substantial contribution from already-freed native<br />
pages. Allocator-free bytes are not necessarily all resident or reclaimable,<br />
and this experiment does not prove the absence of other native leaks.</p>
<p>We separately applied the following startup mitigation to the clone and to<br />
the downstream runtime Dockerfile:</p>
<pre><code class="language-dockerfile">ENV MALLOC_ARENA_MAX=&quot;2&quot;
</code></pre>
<p>It limits glibc arena proliferation and can reduce native-memory retention,<br />
but may increase allocation lock contention. It is platform-specific and<br />
workload-dependent, and should not be treated as the fix for the<br />
Handlebars object-retention bug.</p>
<p>No native addon or periodic <code>malloc_trim</code> was added to the application image.<br />
The native addon used for the one-off investigation was diagnostic only.</p>
<p>Neither correction guarantees that RSS returns to the cold baseline or that<br />
arbitrarily heavy concurrent reports fit the container's memory limit.<br />
The adapter patch does not change worker concurrency or V8 heap limits.</p>
<h2>Suggested upstream action</h2>
<p>Treat cached compiled async Handlebars specifications and their program<br />
functions as immutable with respect to per-render state.</p>
<p>Consider incorporating the private-specification/private-function approach<br />
above, together with reachability and frozen-specification regression tests.<br />
In particular, do not rely on a shallow specification copy alone.</p>
<h2>Public references</h2>
<ul>
<li><a href="https://forum.jsreport.net/topic/3529/slow-but-constant-memory-leak-with-chrome-pdf-recipe-on-docker-jsreport-4-12-0/2">Original memory discussion</a></li>
<li><a href="https://github.com/jsreport/jsreport/blob/4.12.0/packages/jsreport-handlebars/async-helpers/index.js" rel="nofollow">Async adapter in jsreport 4.12.0</a></li>
<li><a href="https://github.com/jsreport/jsreport/blob/4.13.0/packages/jsreport-handlebars/async-helpers/index.js" rel="nofollow">Async adapter in jsreport 4.13.0</a></li>
<li><a href="https://github.com/jsreport/jsreport/blob/4.13.0/packages/jsreport-handlebars/lib/handlebarsEngine.js" rel="nofollow">Handlebars engine in jsreport 4.13.0</a></li>
<li><a href="https://github.com/jsreport/jsreport/blob/4.13.0/packages/jsreport-xlsx/lib/transformation/index.js" rel="nofollow">XLSX transformation in jsreport 4.13.0</a></li>
<li><a href="https://github.com/handlebars-lang/handlebars.js/blob/dce542c9a660048d31f0981ac8a45c08b919bddb/lib/handlebars/runtime.js" rel="nofollow">Handlebars 4.7.9 runtime</a></li>
<li><a href="https://github.com/jsreport/jsreport/releases/tag/4.13.0" rel="nofollow">jsreport 4.13.0 release</a></li>
<li><a href="https://github.com/jsreport/jsreport/commit/9a8c7af4a6d82fd059b52941532fc438532d810a" rel="nofollow">Two-phase XLSX generation change</a></li>
<li><a href="https://github.com/jsreport/jsreport/commit/c6b171efc16890f34ac4d64eaef64bbd24c48f2c" rel="nofollow">Skip generation without dynamic tags</a></li>
<li><a href="https://nodejs.org/docs/latest-v22.x/api/process.html#processmemoryusage" rel="nofollow">Node.js memoryUsage documentation</a></li>
</ul>
]]></description><link>https://forum.jsreport.net/topic/3544/slow-but-constant-memory-increase-rendering-xlsx-templates</link><generator>RSS for Node</generator><lastBuildDate>Wed, 30 Sep 2026 23:34:36 GMT</lastBuildDate><atom:link href="https://forum.jsreport.net/topic/3544.rss" rel="self" type="application/rss+xml"/><pubDate>Wed, 30 Sep 2026 15:35:26 GMT</pubDate><ttl>60</ttl><item><title><![CDATA[Reply to Slow but constant memory increase rendering xlsx templates on Invalid Date]]></title><description><![CDATA[<p>Re: <a href="/topic/3529/slow-but-constant-memory-leak-with-chrome-pdf-recipe-on-docker-jsreport-4-12-0">Slow but constant memory leak with chrome-pdf recipe on Docker (jsreport 4.12.0)</a></p>
<p>Hello, following the discussion about chrome-pdf, I investigated similar memory behavior with large XLSX reports after a customer experienced OOM errors during repeated report generation. Memory appeared to remain elevated even after rendering had completed.</p>
<p>Below is the analysis, carried out with the help of GPT 6 Astra, along with the patch it helped me develop. I'd like to share it for your review and possible inclusion in a future release.</p>
<h1>Cached async Handlebars request-data retention in XLSX reports</h1>
<h2>Summary</h2>
<p>We identified and reproduced JavaScript object retention caused by the async<br />
Handlebars adapter mutating a cached compiled template specification.</p>
<p>The adapter attaches a decorator that closes over a per-render Handlebars<br />
instance. That instance holds registered helper closures, which can retain<br />
request context and report data. Handlebars also attaches that decorator to<br />
the compiled program function. The cached specification therefore keeps<br />
request-specific objects reachable after rendering has finished.</p>
<p>The proposed fix creates a private specification and private program-function<br />
wrappers for each invocation. It preserves compilation caching without letting<br />
the cached objects point back to the request.</p>
<p>This correction was applied downstream and exercised with real XLSX workloads<br />
in a production-like Kubernetes clone. The minimal reproduction below does<br />
not require XLSX, customer data, or the custom extension.</p>
<p>This finding concerns async Handlebars, as used by our XLSX transformation<br />
path. It does not establish the same root cause for every Chrome PDF memory<br />
issue.</p>
<h2>Environment and scope</h2>
<table class="table table-bordered table-striped">
<thead>
<tr>
<th>Component</th>
<th>Investigated configuration</th>
</tr>
</thead>
<tbody>
<tr>
<td>jsreport</td>
<td>4.12.0</td>
</tr>
<tr>
<td>jsreport core</td>
<td>4.9.0</td>
</tr>
<tr>
<td><code>@jsreport/jsreport-handlebars</code></td>
<td>4.1.0</td>
</tr>
<tr>
<td>Handlebars</td>
<td>4.7.7</td>
</tr>
<tr>
<td>Node.js</td>
<td>22.23.1</td>
</tr>
<tr>
<td>Operating system / allocator</td>
<td>Debian Bookworm / glibc 2.36</td>
</tr>
<tr>
<td>XLSX extension</td>
<td>Custom extension declaring version 4.6.1</td>
</tr>
<tr>
<td>Initial isolated investigation</td>
<td>One persistent rendering worker</td>
</tr>
<tr>
<td>Subsequent clone deployment</td>
<td>Three replicas, four rendering workers per replica</td>
</tr>
</tbody>
</table>
<p>The custom XLSX extension is relevant to interpreting the release versions:<br />
its <code>lib/generation/processXlsx.js</code> is byte-for-byte identical to the file<br />
published in jsreport 4.13.0. This was checked both locally and in the installed<br />
extension in the clone. It does not imply that the entire custom extension is<br />
identical to upstream 4.13.0.</p>
<h2>Rendering flow</h2>
<ol>
<li>A rendering worker receives the request, loads the template and helpers,<br />
and prepares the report data.</li>
<li>jsreport compiles the template specification and caches it in that worker.</li>
<li>A per-render Handlebars instance is created and request-specific helper<br />
functions are registered on it.</li>
<li>The XLSX transformation sets <code>req.context.asyncHandlebars = true</code> and<br />
evaluates the original report content against <code>req.data</code>, including parsed<br />
XLSX structures and the report dataset.</li>
<li>The async adapter creates the runtime template from the cached specification.</li>
<li>After rendering, request-specific objects should become collectible while<br />
the compiled specification remains cached for reuse.</li>
</ol>
<p>The relevant implementation locations are:</p>
<ul>
<li><code>packages/jsreport-core/lib/worker/render/executeEngine.js</code></li>
<li><code>packages/jsreport-handlebars/lib/handlebarsEngine.js</code></li>
<li><code>packages/jsreport-handlebars/async-helpers/index.js</code></li>
<li><code>packages/jsreport-xlsx/lib/transformation/index.js</code></li>
<li>Handlebars' <code>lib/handlebars/runtime.js</code></li>
</ul>
<p>The Handlebars engine's compilation code explicitly intends to return a<br />
specification that is independent of the current Handlebars instance so it<br />
can be cached safely.</p>
<h2>Root cause</h2>
<h3>Mutation of the cached specification</h3>
<p>The original async adapter assigns <code>spec.main_d</code> inside<br />
<code>handlebars.template(spec)</code>. Its decorator references the per-render<br />
<code>handlebars</code> instance, including <code>handlebars.wrapHelperResult</code>.</p>
<p>The specification passed to this function is the same object retained by<br />
jsreport's template cache.</p>
<h3>Mutation of shared program functions</h3>
<p>Handlebars' runtime also performs:</p>
<pre><code class="language-js">templateSpec.main.decorator = templateSpec.main_d
</code></pre>
<p>For numbered program functions, it similarly attaches the corresponding<br />
decorator to the program function object.</p>
<p>Consequently, copying only the specification object would not be enough:<br />
its <code>main</code> and numbered program functions would still be shared with the<br />
cached specification.</p>
<h3>Effective retention path</h3>
<pre><code class="language-text">Worker template cache
  -&gt; compiled template specification / program function
  -&gt; async decorator
  -&gt; per-render Handlebars instance
  -&gt; registered helper closures
  -&gt; request context and report data
</code></pre>
<p>This path was established through source inspection and a standalone<br />
reachability reproduction. It is not a claim based on an exported heap<br />
snapshot.</p>
<p>As long as this path remains reachable, GC cannot collect the corresponding<br />
objects. This is different from an allocator delaying the return of already<br />
freed memory to the operating system.</p>
<h3>Retention does not necessarily accumulate once per render</h3>
<p>Repeated executions of the same cached specification replaced the reference<br />
to the previous runtime in our reproduction. They retained the latest<br />
payload, rather than retaining every historical payload.</p>
<p>However, different cached specifications and different workers can each<br />
retain a large payload. Cache warming, additional templates, or changing<br />
payload sizes can therefore make the application footprint grow.</p>
<h2>Observed evidence</h2>
<h3>Isolated real XLSX workload</h3>
<p>With one rendering worker and the original adapter, a completed heavy XLSX<br />
render left approximately <strong>707 MiB of worker heap after spontaneous major<br />
GC</strong>. Repeating the same report returned to approximately the same retained<br />
level.</p>
<p>After applying the adapter correction and restarting the diagnostic process,<br />
two executions of the same heavy report returned the worker heap to<br />
approximately <strong>39-40 MiB after spontaneous GC</strong>.</p>
<p>The patched process RSS after those executions was approximately<br />
<strong>269-292 MiB</strong>. These RSS numbers are not equivalent to JavaScript heap usage.</p>
<p>No application GC was forced for these real-report measurements.</p>
<h3>Standalone reachability checks</h3>
<p>A synthetic reproduction kept the compiled specification strongly reachable,<br />
registered a helper closing over a payload, and held only <code>WeakRef</code> references<br />
to the payload and per-render Handlebars instance.</p>
<ul>
<li>Ordinary Handlebars released the payload and runtime after GC.</li>
<li>The original async adapter retained them while the specification was cached.</li>
<li>A second execution released the first payload but retained the new one.</li>
<li>The original adapter also retained the payload after a helper threw.</li>
<li>The patched async adapter released the payload and runtime.</li>
</ul>
<p>Forced GC was used only in disposable synthetic test processes.</p>
<h3>Clone workload checks</h3>
<p>The correction was also applied to the three normal application replicas in<br />
the clone, retaining four rendering workers per replica.</p>
<p>In a later controlled workload, all twelve worker heaps returned to around<br />
40 MiB after twelve heavy XLSX renders, including two concurrent reports per<br />
replica. XLSX outputs were readable archives and retained the expected workbook,<br />
worksheet, row, and cell counts.</p>
<p>These are structural output checks, not a claim that every cell was manually<br />
reviewed or that the generated files were byte-identical.</p>
<p>Subsequent user-triggered report workloads also returned all twelve worker<br />
heaps to approximately 39-42 MiB after the final rendering activity and idle<br />
period. There were no observed OOM events or container restarts in these runs.</p>
<p>The observations support the correction for the reproduced retention path.<br />
They do not prove the absence of every other memory issue or arbitrary<br />
long-term stability.</p>
<h2>Exact adapter patch applied</h2>
<p>Target: <code>packages/jsreport-handlebars/async-helpers/index.js</code>.</p>
<p>The diff below uses normalized LF line endings. Our downstream installer<br />
normalizes CRLF before checking and patching the source.</p>
<pre><code class="language-diff">diff --git a/packages/jsreport-handlebars/async-helpers/index.js b/packages/jsreport-handlebars/async-helpers/index.js
--- a/packages/jsreport-handlebars/async-helpers/index.js
+++ b/packages/jsreport-handlebars/async-helpers/index.js
@@ -47,7 +47,19 @@
   const _template = handlebars.VM.template
   handlebars.wrapHelperResult = (p) =&gt; p
   handlebars.template = function (spec) {
-    spec.main_d = (prog, props, container, depth, data, blockParams, depths) =&gt; async (context) =&gt; {
+    const renderSpec = { ...spec }
+
+    // The runtime attaches decorators to program functions; keep cached functions untouched.
+    for (const key of Object.keys(renderSpec)) {
+      if ((key === 'main' || /^\d+$/.test(key)) &amp;&amp; typeof renderSpec[key] === 'function') {
+        const program = renderSpec[key]
+        renderSpec[key] = function (...args) {
+          return program.apply(this, args)
+        }
+      }
+    }
+
+    renderSpec.main_d = (prog, props, container, depth, data, blockParams, depths) =&gt; async (context) =&gt; {
       const originalFn = container.fn
       container.fn = (...args) =&gt; {
         const rf = originalFn(...args)
@@ -58,11 +70,11 @@
       }
 
       // here I've changed the last param from `depths` to `[context]`. This was needed to make the ../gotoparent working
-      const v = spec.main(container, context, container.helpers, container.partials, data, blockParams, [context])
+      const v = renderSpec.main(container, context, container.helpers, container.partials, data, blockParams, [context])
       // result can be actually SafeString
       return v.then((r) =&gt; r.toString())
     }
-    return _template(spec, handlebars)
+    return _template(renderSpec, handlebars)
   }
 
   handlebars.compile = function (template, options) {
</code></pre>
<h3>Why this fixes the reproduced path</h3>
<ul>
<li><code>renderSpec</code> is private to the runtime invocation.</li>
<li>Its main and numbered program functions are new wrapper objects, so runtime<br />
decorator assignments do not modify the cached functions.</li>
<li>The wrappers delegate to the original compiled code with the same arguments<br />
and <code>this</code>.</li>
<li>The per-render decorator is attached only to the private specification.</li>
<li>The cache continues to retain compiled code, not the request-specific runtime.</li>
</ul>
<p>This does not deep-copy the request data, disable compilation caching, force<br />
GC, or recycle workers.</p>
<p>The fix should be applied before workers start, or followed by a restart.<br />
Replacing the adapter file does not retroactively clean already-mutated<br />
specifications or reload modules in a running process.</p>
<h2>Minimal reproduction</h2>
<p>Save the following as <code>repro.cjs</code> in a project with<br />
<code>@jsreport/jsreport-handlebars</code> 4.1.0 and Handlebars 4.7.7 installed.<br />
It invokes the engine directly and does not require a running jsreport server.</p>
<pre><code class="language-js">const assert = require('node:assert/strict');
const path = require('node:path');
const { setImmediate: nextTurn } = require('node:timers/promises');

const engine = require('@jsreport/jsreport-handlebars/lib/handlebarsEngine')({
  handlebarsModulePath: path.dirname(require.resolve('handlebars/package.json')),
});
const cache = new Map();

function runtime() {
  const context = engine.createContext({
    context: { asyncHandlebars: true },
  });
  return {
    context,
    require(name) {
      assert.equal(name, 'handlebars');
      return context.handlebars;
    },
  };
}

async function execute(spec) {
  const current = runtime();
  const payload = { rows: new Array(10000).fill(123) };
  const references = {
    payload: new WeakRef(payload),
    handlebars: new WeakRef(current.context.handlebars),
  };
  const output = await engine.execute(
    spec,
    { value: () =&gt; payload.rows.length },
    {},
    { require: current.require },
  );
  assert.equal(output, '10000');
  return references;
}

async function main() {
  assert.equal(typeof global.gc, 'function', 'Run with --expose-gc');
  const compiler = runtime();
  const spec = engine.compile('{{value}}', { require: compiler.require });
  cache.set('template', spec);
  const references = await execute(spec);

  for (let i = 0; i &lt; 4; i++) {
    await nextTurn();
    global.gc();
  }

  assert.equal(cache.get('template'), spec);
  const retainedPayload = references.payload.deref() !== undefined;
  const retainedHandlebars = references.handlebars.deref() !== undefined;
  const expected = process.env.HANDLEBARS_EXPECT_RETENTION === 'true';
  assert.equal(retainedPayload, expected);
  assert.equal(retainedHandlebars, expected);
  console.log(JSON.stringify({
    retainedPayload,
    retainedHandlebars,
    cachedSpecifications: cache.size,
  }));
}

main().catch((error) =&gt; {
  console.error(error);
  process.exitCode = 1;
});
</code></pre>
<p>On an unpatched installation:</p>
<pre><code class="language-sh">HANDLEBARS_EXPECT_RETENTION=true node --expose-gc repro.cjs
</code></pre>
<p>Expected result:</p>
<pre><code class="language-json">{&quot;retainedPayload&quot;:true,&quot;retainedHandlebars&quot;:true,&quot;cachedSpecifications&quot;:1}
</code></pre>
<p>After applying the adapter correction:</p>
<pre><code class="language-sh">HANDLEBARS_EXPECT_RETENTION=false node --expose-gc repro.cjs
</code></pre>
<p>Expected result:</p>
<pre><code class="language-json">{&quot;retainedPayload&quot;:false,&quot;retainedHandlebars&quot;:false,&quot;cachedSpecifications&quot;:1}
</code></pre>
<h2>Additional regression coverage</h2>
<p>Our broader synthetic harness exercised:</p>
<ul>
<li>Ordinary Handlebars as a non-retaining control.</li>
<li>Repeated execution of the same cached async specification.</li>
<li>Cleanup after a helper throws.</li>
<li>Two concurrent executions sharing the same compiled specification.</li>
<li>Async helpers inside <code>each</code>, parent-context lookup, and block parameters.</li>
<li>Partials, escaped output, and unescaped output.</li>
<li>Execution with the specification and compiled program functions frozen.</li>
<li>Absence of <code>main_d</code> and <code>main.decorator</code> mutations on the cached objects.</li>
</ul>
<p>The frozen-specification case is particularly useful for protecting the<br />
cache-immutability invariant.</p>
<p>Upstream review should also consider supported custom decorators and any<br />
function metadata expectations not represented by these cases.</p>
<h2>Relation to the XLSX improvements in jsreport 4.13.0</h2>
<p>The 4.13.0 release includes two relevant generation improvements:</p>
<ol>
<li>Skip generation work when the input workbook has no Handlebars tags.</li>
<li>Separate dynamic data evaluation from final XML construction, excluding<br />
static XML and <code>sharedStrings.xml</code> from unnecessary evaluation.</li>
</ol>
<p>These changes improve the generation path. They do not replace the async<br />
Handlebars retention correction.</p>
<p>The async adapter source is identical between the 4.12.0 and 4.13.0 tags.<br />
In 4.13.0 it still assigns <code>spec.main_d</code> and passes the shared specification<br />
to <code>_template(spec, handlebars)</code>. The legacy XLSX transformation still enables<br />
async Handlebars.</p>
<p>The release updates <code>@jsreport/jsreport-handlebars</code> from 4.1.0 to 4.1.1 and<br />
Handlebars from 4.7.7 to 4.7.9, but Handlebars' runtime still attaches<br />
decorators to the compiled program functions.</p>
<p>This conclusion is based on tagged-source comparison. We did not perform a<br />
full application upgrade or an end-to-end benchmark of jsreport 4.13.0.</p>
<h2>Separate issue: native allocator RSS retention</h2>
<p>After fixing the JavaScript retention path, process RSS can still remain<br />
higher than its cold baseline.</p>
<p>In a separate controlled experiment on an idle clone replica, glibc reported<br />
approximately 583 MiB of free blocks in its arenas. A single explicitly<br />
authorized <code>malloc_trim(0)</code> reduced RSS from approximately <strong>912 MiB to<br />
461 MiB</strong>, while the measured worker heap stayed around <strong>40 MiB</strong>.</p>
<p>This demonstrates a substantial contribution from already-freed native<br />
pages. Allocator-free bytes are not necessarily all resident or reclaimable,<br />
and this experiment does not prove the absence of other native leaks.</p>
<p>We separately applied the following startup mitigation to the clone and to<br />
the downstream runtime Dockerfile:</p>
<pre><code class="language-dockerfile">ENV MALLOC_ARENA_MAX=&quot;2&quot;
</code></pre>
<p>It limits glibc arena proliferation and can reduce native-memory retention,<br />
but may increase allocation lock contention. It is platform-specific and<br />
workload-dependent, and should not be treated as the fix for the<br />
Handlebars object-retention bug.</p>
<p>No native addon or periodic <code>malloc_trim</code> was added to the application image.<br />
The native addon used for the one-off investigation was diagnostic only.</p>
<p>Neither correction guarantees that RSS returns to the cold baseline or that<br />
arbitrarily heavy concurrent reports fit the container's memory limit.<br />
The adapter patch does not change worker concurrency or V8 heap limits.</p>
<h2>Suggested upstream action</h2>
<p>Treat cached compiled async Handlebars specifications and their program<br />
functions as immutable with respect to per-render state.</p>
<p>Consider incorporating the private-specification/private-function approach<br />
above, together with reachability and frozen-specification regression tests.<br />
In particular, do not rely on a shallow specification copy alone.</p>
<h2>Public references</h2>
<ul>
<li><a href="https://forum.jsreport.net/topic/3529/slow-but-constant-memory-leak-with-chrome-pdf-recipe-on-docker-jsreport-4-12-0/2">Original memory discussion</a></li>
<li><a href="https://github.com/jsreport/jsreport/blob/4.12.0/packages/jsreport-handlebars/async-helpers/index.js" rel="nofollow">Async adapter in jsreport 4.12.0</a></li>
<li><a href="https://github.com/jsreport/jsreport/blob/4.13.0/packages/jsreport-handlebars/async-helpers/index.js" rel="nofollow">Async adapter in jsreport 4.13.0</a></li>
<li><a href="https://github.com/jsreport/jsreport/blob/4.13.0/packages/jsreport-handlebars/lib/handlebarsEngine.js" rel="nofollow">Handlebars engine in jsreport 4.13.0</a></li>
<li><a href="https://github.com/jsreport/jsreport/blob/4.13.0/packages/jsreport-xlsx/lib/transformation/index.js" rel="nofollow">XLSX transformation in jsreport 4.13.0</a></li>
<li><a href="https://github.com/handlebars-lang/handlebars.js/blob/dce542c9a660048d31f0981ac8a45c08b919bddb/lib/handlebars/runtime.js" rel="nofollow">Handlebars 4.7.9 runtime</a></li>
<li><a href="https://github.com/jsreport/jsreport/releases/tag/4.13.0" rel="nofollow">jsreport 4.13.0 release</a></li>
<li><a href="https://github.com/jsreport/jsreport/commit/9a8c7af4a6d82fd059b52941532fc438532d810a" rel="nofollow">Two-phase XLSX generation change</a></li>
<li><a href="https://github.com/jsreport/jsreport/commit/c6b171efc16890f34ac4d64eaef64bbd24c48f2c" rel="nofollow">Skip generation without dynamic tags</a></li>
<li><a href="https://nodejs.org/docs/latest-v22.x/api/process.html#processmemoryusage" rel="nofollow">Node.js memoryUsage documentation</a></li>
</ul>
]]></description><link>https://forum.jsreport.net/post/14968</link><guid isPermaLink="true">https://forum.jsreport.net/post/14968</guid><dc:creator><![CDATA[l-dalleaste_lectra1]]></dc:creator><pubDate>Invalid Date</pubDate></item></channel></rss>