Error Occurred when rendering with large amount of data

[Karthik22296]

I have a report which takes a Data array of length 6061 records. while rendering this error occurs. Can you say why this occurs and a solution for this.!

[meetvishalt]

Hi Jan,

I am facing similar problem while rendering large data of around 500 pages. Basically I have a single page template and have added {{foreach}} helper in a page and I am passing json array of 500 objects to get document of 500 pages.

I am doing this on DocX recipe with handlebars engine.
My hosting server has 64GB RAM and I have installed dockerized JsReport on it.

Below is the configuration of jsReport.
{
"store": {
"provider": "fs"
},
"httpPort": 5488,
"allowLocalFilesAccess": true,
"blobStorage": {
"provider": "fs"
},
"logger": {
"console": {
"transport": "console",
"level": "debug"
},
"file": {
"transport": "file",
"level": "info",
"filename": "logs/reporter.log"
},
"error": {
"transport": "file",
"level": "error",
"filename": "logs/error.log"
}
},
"chrome": {
"timeout": 1800000
},
"templatingEngines": {
"numberOfWorkers": 50,
"timeout": 1800000,
"strategy": "http-server"
},
"extensions": {
"docx": {
"timeout": 1800000
},
"chrome-pdf": {
"timeout": 1800000
}
},
"express": {
"renderTimeout": 1800000
},
"scripts": {
"allowedModules": "*",
"timeout": 60000000
},
"office": {
"preview": {
"enabled": false,
"showWarning": false
}
}
}

Template and data for 500 pages are placed at below playground URL.
https://playground.jsreport.net/w/meetvishalt/aQGUr9g8

I am starting JsReport by setting Node.Js memory size around 15 GB by running below command
NODE_OPTIONS=--max-old-space-size=16000

I am getting error in console logs as below:
Error
2019-11-28T15:38:04.984Z - warn: Error when processing render request socket hang up Error: socket hang up
at connResetException (internal/errors.js:561:14)
at Socket.socketOnEnd (_http_client.js:440:23)
at Socket.emit (events.js:214:15)
at endReadableNT (_stream_readable.js:1178:12)
at processTicksAndRejections (internal/process/task_queues.js:80:21)
2019-11-28T15:38:04.996Z - warn: Error when processing render request socket hang up Error: socket hang up
at connResetException (internal/errors.js:561:14)
at Socket.socketOnEnd (_http_client.js:440:23)
at Socket.emit (events.js:214:15)
at endReadableNT (_stream_readable.js:1178:12)
at processTicksAndRejections (internal/process/task_queues.js:80:21)

Kindly help in this.

Thanks,
Vishal

[jan_blaha]

Hi, there are some optimizations done in the not yet released docx recipe version which can be installed from the github master branch.
https://forum.jsreport.net/topic/1340/generate-large-docx-document-more-than-1000-pages/8

Here are some notes on how to extend jsreport official docker image
https://jsreport.net/learn/customize-docker-image

[meetvishalt]

Hi Jan,

Thanks for your pointers. I installed jsReport without docker, used docx recipe from git as you mentioned and set the memory size as 8GB while starting the JsReport.

Below is what I tried on local JsReport server:

1. Increase memory size to 8GB and could render 100 and 200 page doc successfully.
2. I tried 300, and it gave error as “Error: Request body larger than maxBodyLength limit”. This was coming from follow-redirects module and I changed the code to set maxBodyLength as 3GB.
3. Tried 300 page request and it worked successfully.
4. Fired 400 page request this time and got below logs in jsReport

2019-11-29T13:18:48.668Z - debug: Executing recipe html
2019-11-29T13:18:48.765Z - debug: Skipping storing report.
2019-11-29T13:18:48.777Z - info: Rendering request 2 finished in 18955 ms
2019-11-29T13:21:52.063Z - error: unable to load planned schedules Error: Timeou
t during waiting for file system, try it again later.
at Object.rejectItemsWithTimeout (c:\jsreportapp\node_modules\jsreport-fs-st
ore\lib\queue.js:53:23)
at Timeout._onTimeout (c:\jsreportapp\node_modules\jsreport-fs-store\lib\pro
vider.js:346:73)
at listOnTimeout (internal/timers.js:531:17)
at processTimers (internal/timers.js:475:7)
2019-11-29T13:21:53.706Z - error: Error while trying to flush studio logs: Timeo
ut during waiting for file system, try it again later. - Error: Timeout during w
aiting for file system, try it again later.
at Object.rejectItemsWithTimeout (c:\jsreportapp\node_modules\jsreport-fs-st
ore\lib\queue.js:53:23)
at Timeout._onTimeout (c:\jsreportapp\node_modules\jsreport-fs-store\lib\pro
vider.js:346:73)
at listOnTimeout (internal/timers.js:531:17)
at processTimers (internal/timers.js:475:7)
2019-11-29T13:21:56.056Z - debug: Successfully zipped now.
2019-11-29T13:21:56.566Z - error: Error when processing render request Request f
ailed with status code 413 Error: Request failed with status code 413
at createError (c:\jsreportapp\node_modules\axios\lib\core\createError.js:16
:15)
at settle (c:\jsreportapp\node_modules\axios\lib\core\settle.js:17:12)
at IncomingMessage.handleStreamEnd (c:\jsreportapp\node_modules\axios\lib\ad
apters\http.js:237:11)
at IncomingMessage.emit (events.js:215:7)
at endReadableNT (_stream_readable.js:1183:12)
at processTicksAndRejections (internal/process/task_queues.js:80:21)
2019-11-29T13:21:56.607Z - error: Error during processing request at http://localhost:5488/api/report/HCFA

Looking at jsreport logs, it seems that it has generated document internally from green line marked above, but while sending this doc content to playground it fails and give HTTP error 413 which is related to large payload in response.

Appreciate your help in this.

Thanks,
Vishal

[jan_blaha]

Our server hosting temporary office documents for preview has some input limits and you are reaching them.
Please use the "Download" button in the studio instead of run.
This won't try to upload the document to the public server and shouldn't fail.
Or check how to disable the preview here
https://jsreport.net/learn/office-preview