Inconsistent Errors for Multi PDF Append

snaptsoft

We have implemented JS report to allow users to create documents that consist of single or multiple templates based on what they select in our application. All of the templates function correctly when rendered individually. However, when using PDF Utils to append the documents together (e.g. take 3 document templates, render them, and return 1 PDF of all of the templates), we receive an error approximately 70% of the time. The remainder of the time the document generates exactly as it should. This is using the same method and test data with no changes.

We are generating multiple PDFs using the JSReport client SDK and the following options:

  let reportRequest = {
    template: {
      name: rootTemplate.name,
      recipe: "chrome-pdf",
      pdfOperations: additionaltemplates.map((template) => {
        return {
          template: {
            name: template.name,
            recipe: "chrome-pdf",
          },
          data: template.contextData,
          type: "append",
          mergeWholeDocument: true,
          renderForEveryPage: true,
        };
      }),
    },
    data: rootTemplate.contextData,
  };
  const render = await jsreport.renderAsync(reportRequest);

We receive the following error message (or slightly different ones depending on the templates included). I have confirmed that the data is certainly being sent (it generates properly 30% of the time on repeated requests with no changes) and have verified in the API call that the request payload contains all of the correct information.

0|server       | 2021-02-26T22:37:13.762Z - warn: Error when processing render request 828 Error while executing pdf-utils operations. Error while executing templating engine. Cannot convert undefined or null to object.
0|server       |   271 |
0|server       |   272 | function body_adj_hrs(lbradjust, refhrs) {
0|server       | > 273 |     const total = Object.values(lbradjust).reduce((acc, val) => {
0|server       |       |                          ^
0|server       |   274 |         return acc + (val || 0 )
0|server       |   275 |         }, 0);
0|server       |   276 |     return subtraction_hrs(total, refhrs);
0|server       |  Error: Error while executing pdf-utils operations. Error while executing templating engine. Cannot convert undefined or null to object.
0|server       |   271 |
0|server       |   272 | function body_adj_hrs(lbradjust, refhrs) {
0|server       | > 273 |     const total = Object.values(lbradjust).reduce((acc, val) => {
0|server       |       |                          ^
0|server       |   274 |         return acc + (val || 0 )
0|server       |   275 |         }, 0);
0|server       |   276 |     return subtraction_hrs(total, refhrs);
0|server       |     at module.exports (/home/imex/node_modules/jsreport-core/lib/util/createError.js:11:13)
0|server       |     at Reporter.createError (/home/imex/node_modules/jsreport-core/lib/reporter.js:332:12)
0|server       |     at AsyncFunction.reporter.afterRenderListeners.insert (/home/imex/node_modules/jsreport-pdf-utils/lib/main.js:382:22)
0|server       |     at <anonymous>
0|server       |     at process._tickCallback (internal/process/next_tick.js:188:7)
0|server       | caused by: Error: Error while executing templating engine. Cannot convert undefined or null to object.
0|server       |   271 |
0|server       |   272 | function body_adj_hrs(lbradjust, refhrs) {
0|server       | > 273 |     const total = Object.values(lbradjust).reduce((acc, val) => {
0|server       |       |                          ^
0|server       |   274 |         return acc + (val || 0 )
0|server       |   275 |         }, 0);
0|server       |   276 |     return subtraction_hrs(total, refhrs);
0|server       |     at Object.callbackRequests.(anonymous function) (/home/imex/node_modules/script-manager/lib/worker-servers.js:250:31)
0|server       |     at process.<anonymous> (/home/imex/node_modules/script-manager/lib/worker-servers.js:195:30)
0|server       |     at emitTwo (events.js:131:20)
0|server       |     at process.emit (events.js:214:7)
0|server       |     at emit (internal/child_process.js:772:12)
0|server       |     at _combinedTickCallback (internal/process/next_tick.js:141:11)
0|server       |     at process._tickDomainCallback (internal/process/next_tick.js:218:9)

I have so far been unable to determine why the issue is intermittent. It appears that sometimes JS Report attempts to render the other templates without passing the data, but I'm unsure of why. Does anyone have any insights as to why this may be erroring randomly when generating the same data?[0_1614379681630_jsreport.config.json](Uploading 100%)

jan_blaha

Could you please try to prepare a minimal workspace with minimal code replicating the error?

snaptsoft

I'm trying to create a minimal workspace to recreate the error, but unfortunately, I've had no luck so far. Will continue trying to reproduce.

Is it possible that this is a memory/server capacity issue? The jsreport server is currently only used by 1 developer and has minimal resources (1 cpu, 2GB ram). From my monitoring, it doesn't appear to be having any issues, but wanted to rule that out.

jan_blaha

Hmm, I don't think this can have something to do with it.

snaptsoft

Hi Jan,

I've continued to try and reproduce it minimally on the playground and code sandbox to no avail.

I have narrowed down the issue somewhat. I believe it has to do with the serialization of data. Some of our templates use child templates that receive complicated objects passed to them as follows.

{#child /components/job_totals @data.job-totals={{{childTemplateSerializeData jobs_by_pk.job_totals}}}}

The job_totals object can go 4-5 levels deep and have 20+ keys at certain levels. I have noticed that when the template that calls this is the first in the render order (the root object that is), the rendering works. However, if it is included in the pdf operations as a template to append, it throws the error.

I have tried to pass the data without serializing it to the child templates, however, no data comes through. Are there any limits to the serialization of the data? Any potential troubleshooting or debugging on our server that might be helpful that you can think of?

jan_blaha

The data should be passed to the child templates like this actually :

{#child myChild @data.foo$={{{childTemplateSerializeData foo}}} }

See the documentation here
https://jsreport.net/learn/child-templates#set-child-template-parameters

But I'm not sure if this is the problem.
Maybe you can try to set the parallelLimit config to 1 to eliminate the bugs in parallel execution
https://jsreport.net/learn/child-templates#configuration

Nothing else comes to my mind so far.
However, will be still glad if you will be able to create a minimal workspace replicating the bug so I can take a look.

snaptsoft

Will give that a try and report back.

A quick clarification on our application side implementation just to make sure that it is correct. We currently pass the data object to the root template, but also pass data objects to the templates to be appended (comments added indicating below).

    template: {
      name: rootTemplate.name,
      recipe: "chrome-pdf",
      pdfOperations: additionaltemplates.map((template) => {
        return {
          template: {
            name: template.name,
            recipe: "chrome-pdf",
          },
          data: template.contextData, //********Data specific to this nth template getting appended
          type: "append",
          mergeWholeDocument: true,
          renderForEveryPage: true,
        };
      }),
    },
    data: rootTemplate.contextData, //********Data specific to the first template
  };

Is passing the additional data to subsequent templates supported? I've taken a closer look at the documentation, and there is no data key. If it is not supported, this may be the root of the problem and explain why the ordering indicates the success of the render.

jan_blaha

It isn't supported to pass the data to the pdf operation.

snaptsoft

Thank you Jan. We've refactored our code to merge all required data at the top most level and we are reliably able to generate the documents.