HTML-XLSX craches with too large JSON
-
My JSON data is about 40,000 lines in pretty-format. If I save the file to disk it is approx 1 MB in file size. The template is quite simple, but some columns are formatted as "datetime" or "number".
If I remove the upper or lower half of the json data, everything works as it should. Since I can get it to work with either half of the file I assume that te error has nothing to do with the actual data, but with the size of the data.
How should I go on and debug this?
Here is what I get from jsreport:
Page crashed! Error: Page crashed! at Page._onTargetCrashed (/app/node_modules/puppeteer/lib/Page.js:170:24) at CDPSession. (/app/node_modules/puppeteer/lib/Page.js:125:56) at CDPSession.emit (events.js:200:13) at CDPSession._onMessage (/app/node_modules/puppeteer/lib/Connection.js:200:12) at Connection._onMessage (/app/node_modules/puppeteer/lib/Connection.js:112:17) at WebSocket. (/app/node_modules/puppeteer/lib/WebSocketTransport.js:41:24) at WebSocket.onMessage (/app/node_modules/puppeteer/node_modules/ws/lib/event-target.js:120:16) at WebSocket.emit (events.js:200:13) at Receiver.receiverOnMessage (/app/node_modules/puppeteer/node_modules/ws/lib/websocket.js:789:20) at Receiver.emit (events.js:200:13) at Receiver.dataMessage (/app/node_modules/puppeteer/node_modules/ws/lib/receiver.js:422:14) at Receiver.getData (/app/node_modules/puppeteer/node_modules/ws/lib/receiver.js:352:17) at Receiver.startLoop (/app/node_modules/puppeteer/node_modules/ws/lib/receiver.js:138:22) at Receiver._write (/app/node_modules/puppeteer/node_modules/ws/lib/receiver.js:74:10) at doWrite (_stream_writable.js:417:12) at writeOrBuffer (_stream_writable.js:401:5) at Receiver.Writable.write (_stream_writable.js:301:11) at Socket.socketOnData (/app/node_modules/puppeteer/node_modules/ws/lib/websocket.js:864:35) at Socket.emit (events.js:200:13) at addChunk (_stream_readable.js:290:12) at readableAddChunk (_stream_readable.js:271:11) at Socket.Readable.push (_stream_readable.js:226:10) at TCP.onStreamRead (internal/stream_base_commons.js:166:17) +0 Starting rendering request 216 (user: null) +1 Rendering template { name: ControlpointList_raw_excel, recipe: html-to-xlsx, engine: jsrender, preview: true } +3 Inline data specified. +3 Resources not defined for this template. +4 Executing script dateTimeConstants +204 Base url not specified, skipping its injection. +206 Rendering engine jsrender +303 Taking compiled template from engine cache +788 Executing recipe html-to-xlsx
-
Hmm.
I kind of solved this, but I'm not happy with the solution. I bumped up the memory from 1GB to 2GB on my Fargate container running jsreport. There has to be something fishy here.. How can it take 2 GB to parse a 1 MB json file?
-
It is not about parsing json. It crashes when it is evaluating the html you produced using chrome. This can be a very long HTML table and chrome just gets out of resources and crashes.
Maybe it can help if you put this line to your dockerfile if you have the custom one.
ENV chrome_launchOptions_args --no-sandbox,--disable-dev-shm-usage
or use the full jsreport docker image which has this already. We will put this also to the default jsreport image in the next version as well.The xlsx recipe is ugly but it gives you better performance for extra long excel sheets with low sized resources.
Just saying...
-
Thanks, I'll try that.
What will those arguments do?I was stoked to get rid of the ugly xlsx and start using html-xlxs instead, but I suppose I have to go back if I want a stable environment. If the server can't even handle one render of such a report, what will happen when I'm in production and multiple renders can hit the server simultaneously?
-
Ok, this settles it. I have some small quirks to solve in the XLXS version, but it's way faster.
Two almost identical reports based on the same dataset:
HTML-XLXS: approx 15 seconds to render
XLXS: less than a second to render
-
Hi ,
for Recipe.HtmlToXlsx). file get corrupted for lage data(5000 rows)
