¿Is there some kind of limitation on the size of the data when generating a report? (I have a report that is about 170MB)

jan_blaha

The current default is 50mb.
You can increase it using config extensions.express.inputRequestLimit=200mb

Pablo Narvaez

When I generate a report with a data of 7mb or more, it always generates a 460-page pdf that must be extensions.express.inputRequestLimit=200mb, there is an example of how to handle such large pdfs and possible errors at the time of sending it as response?

jan_blaha

What kind of error do you get?

Pablo Narvaez

2020-10-14T13:52:43.999Z - debug: Report stored as JtlZhCcTSIjgAkSz.pdf
2020-10-14T13:52:44.000Z - info: Rendering request 1 finished in 4407 ms
2020-10-14T13:53:48.274Z - debug: OData query on templates
2020-10-14T13:53:48.280Z - debug: OData query on folders
2020-10-14T13:53:48.285Z - debug: OData query on tags
2020-10-14T13:53:48.287Z - debug: OData query on data
2020-10-14T13:53:48.291Z - debug: OData query on assets
2020-10-14T13:53:48.293Z - debug: OData query on scripts
2020-10-14T13:53:48.299Z - debug: OData query on xlsxTemplates
2020-10-14T13:53:48.302Z - debug: OData query on users
2020-10-14T13:53:48.306Z - debug: OData query on schedules
2020-10-14T13:53:48.314Z - debug: OData query on settings
2020-10-14T13:53:48.423Z - debug: OData query on templates
2020-10-14T13:53:48.437Z - debug: OData query on settings
2020-10-14T13:53:57.684Z - debug: OData query on reports
2020-10-14T13:54:06.329Z - debug: OData query on templates
2020-10-14T13:54:06.335Z - debug: OData query on settings
2020-10-14T14:35:27.396Z - debug: API logging in user admin
2020-10-14T14:35:27.403Z - info: Starting rendering request 4 (user: admin)
2020-10-14T14:35:27.406Z - info: Rendering anonymous template { recipe: phantom-pdf, engine: handlebars }
2020-10-14T14:35:27.407Z - debug: Inline data specified.
2020-10-14T14:35:27.407Z - debug: Resources not defined for this template.
2020-10-14T14:35:27.409Z - debug: Base url not specified, skipping its injection.
2020-10-14T14:35:27.417Z - debug: Rendering engine handlebars
2020-10-14T14:36:16.471Z - debug: OData query on reports
2020-10-14T14:36:38.590Z - debug: API logging in user admin
2020-10-14T14:36:38.595Z - info: Starting rendering request 5 (user: admin)
2020-10-14T14:36:38.598Z - info: Rendering anonymous template { recipe: phantom-pdf, engine: handlebars }
2020-10-14T14:36:38.600Z - debug: Inline data specified.
2020-10-14T14:36:38.601Z - debug: Resources not defined for this template.
2020-10-14T14:36:38.603Z - debug: Base url not specified, skipping its injection.
2020-10-14T14:36:38.609Z - debug: Rendering engine handlebars

<--- Last few GCs --->

[10508:0000026C8E1918C0] 3050691 ms: Mark-sweep 2050.5 (2067.9) -> 2050.3 (2069.6) MB, 100.8 / 0.0 ms (average mu = 0.146, current mu = 0.027) allocation failure scavenge might not succeed
[10508:0000026C8E1918C0] 3050800 ms: Mark-sweep 2052.2 (2069.6) -> 2051.9 (2071.6) MB, 105.0 / 0.0 ms (average mu = 0.087, current mu = 0.028) allocation failure scavenge might not succeed

<--- JS stacktrace --->

==== JS stack trace =========================================

Security context: 0x00a08e92acd9 <JSObject>
0: builtin exit frame: stringify(this=0x02a888348d69 <Object map = 000003442D245341>,0x0070498001b1 <null>,0x017d12438ec1 <JSFunction (sfi = 000001670C9503D9)>,0x017d12438ef9 <Object map = 000002BBC3A5DDF1>,0x02a888348d69 <Object map = 000003442D245341>)

1: serialize [000000A08E920171] [D:\ajsreport\node_modules\serializator\index.js:23] [bytecode=000001670C950489 offset=149](this=0x030...

FATAL ERROR: Ineffective mark-compacts near heap limit Allocation failed - JavaScript heap out of memory
1: 00007FF6DA41094F napi_wrap+124431
2: 00007FF6DA3B2696 v8::base::CPU::has_sse+34502
3: 00007FF6DA3B3356 v8::base::CPU::has_sse+37766
4: 00007FF6DABB6F4E v8::Isolate::ReportExternalAllocationLimitReached+94
5: 00007FF6DAB9EF91 v8::SharedArrayBuffer::Externalize+833
6: 00007FF6DAA6C85C v8::internal::Heap::EphemeronKeyWriteBarrierFromCode+1436
7: 00007FF6DAA77C00 v8::internal::Heap::ProtectUnprotectedMemoryChunks+1312
8: 00007FF6DAA74734 v8::internal::Heap::PageFlagsAreConsistent+3204
9: 00007FF6DAA69FC3 v8::internal::Heap::CollectGarbage+1283
10: 00007FF6DAA68794 v8::internal::Heap::AddRetainedMap+2356
11: 00007FF6DAA90787 v8::internal::Factory::NewRawTwoByteString+87
12: 00007FF6DA785B4B v8::internal::StringStream::ClearMentionedObjectCache+9579
13: 00007FF6DA9A64F7 v8::internal::TimedHistogram::Stop+2823
14: 00007FF6DA9A79A2 v8::internal::TimedHistogram::Stop+8114
15: 00007FF6DA9A9C61 v8::internal::TimedHistogram::Stop+17009
16: 00007FF6DA9AC890 v8::internal::TimedHistogram::Stop+28320
17: 00007FF6DAB30CE4 v8::internal::Builtins::builtin_handle+86836
18: 00007FF6DAFE2D6D v8::internal::SetupIsolateDelegate::SetupHeap+568205
19: 00007FF6DAF627FC v8::internal::SetupIsolateDelegate::SetupHeap+42524
20: 00007FF6DAF627FC v8::internal::SetupIsolateDelegate::SetupHeap+42524
21: 00007FF6DAF627FC v8::internal::SetupIsolateDelegate::SetupHeap+42524
22: 00007FF6DAF627FC v8::internal::SetupIsolateDelegate::SetupHeap+42524
23: 00007FF6DAF627FC v8::internal::SetupIsolateDelegate::SetupHeap+42524
24: 00007FF6DAF5C339 v8::internal::SetupIsolateDelegate::SetupHeap+16729
25: 00007FF6DAF627FC v8::internal::SetupIsolateDelegate::SetupHeap+42524
26: 00007FF6DAF5FDB1 v8::internal::SetupIsolateDelegate::SetupHeap+31697
27: 00007FF6DAF5F99C v8::internal::SetupIsolateDelegate::SetupHeap+30652
28: 00007FF6DAAC4F43 v8::internal::Execution::CallWasm+1395
29: 00007FF6DAAC48C6 v8::internal::Execution::Call+182
30: 00007FF6DABB7877 v8::Script::Run+711
31: 00007FF6DA3BB4D8 v8::base::CPU::has_sse+70920
32: 00007FF6DA3BDF0F v8::base::CPU::has_sse+81727
33: 00007FF6DAB6B150 v8::internal::Builtins::builtin_handle+325536
34: 00007FF6DAB6A697 v8::internal::Builtins::builtin_handle+322791
35: 00007FF6DAB6A9D8 v8::internal::Builtins::builtin_handle+323624
36: 00007FF6DAB6A7DE v8::internal::Builtins::builtin_handle+323118
37: 00007FF6DAFE2D6D v8::internal::SetupIsolateDelegate::SetupHeap+568205
38: 00007FF6DAF627FC v8::internal::SetupIsolateDelegate::SetupHeap+42524
39: 00007FF6DAF627FC v8::internal::SetupIsolateDelegate::SetupHeap+42524
40: 00007FF6DAF627FC v8::internal::SetupIsolateDelegate::SetupHeap+42524
41: 00007FF6DAF627FC v8::internal::SetupIsolateDelegate::SetupHeap+42524
42: 00007FF6DAF627FC v8::internal::SetupIsolateDelegate::SetupHeap+42524
43: 00007FF6DAF627FC v8::internal::SetupIsolateDelegate::SetupHeap+42524
44: 00007FF6DAF627FC v8::internal::SetupIsolateDelegate::SetupHeap+42524
45: 00007FF6DAF5BBAC v8::internal::SetupIsolateDelegate::SetupHeap+14796
46: 00007FF6DAF627FC v8::internal::SetupIsolateDelegate::SetupHeap+42524
47: 0000022313D1ABEF
2020-10-14T14:38:19.359Z - warn: Error when processing render request read ECONNRESET Error: read ECONNRESET
at TCP.onStreamRead (internal/stream_base_commons.js:201:27)
2020-10-14T14:38:19.365Z - warn: Error during processing request at http://localhost:5488/api/report
2020-10-14T14:38:19.372Z - warn: Error when processing render request read ECONNRESET Error: read ECONNRESET
at TCP.onStreamRead (internal/stream_base_commons.js:201:27)
2020-10-14T14:38:19.376Z - warn: Error during processing request at http://localhost:5488/api/report

Pablo Narvaez

This post is deleted!

Pablo Narvaez

files configuration
https://gist.github.com/pablonarvaez20152016/2a6fa4755a07a0fb5ceb318fe5969ea0

Pablo Narvaez

{
"extensions": {

"authentication": {
  "cookieSession": {
    "secret": "000000"
  },
"admin": {
  "username": "admin",
  "password": "0000"
},
"enabled": true
},
"scripts": {
  "timeout": 9400000,
  "strategy": "http-server"
},
"express":{
  "inputRequestLimit":"5000mb",
  "renderTimeout":9400000
},
"phantom-pdf":{
  "timeout":9400000
},
"xlsx":{
  "escapeAmp":true,
  "numberOfParsedAddIterations":50,
  "addBufferSize":50000000
},
"sample-template": {
  "createSamples": true
}

},

"httpPort": 5488,
"store": {
"provider": "memory"
},
"blobStorage": {
"provider": "memory"
},
"logger": {
"console": {
"transport": "console",
"level": "debug"
},
"file": {
"transport": "file",
"level": "info",
"filename": "logs/reporter.log"
},
"error": {
"transport": "file",
"level": "error",
"filename": "logs/error.log"
}
},
"allowLocalFilesAccess": true,
"templatingEngines": {
"timeout":9400000,
"strategy": "http-server"
},
"chrome": {
"timeout": 9400000
},
"Phantom": {
"timeout": 9400000
},
"reportTimeout":960000
}

jan_blaha

You are getting out of memory error in nodejs.
The default is just 2Gb, please try to increase it:

Set the NODE_OPTIONS environment variable

NODE_OPTIONS=--max-old-space-size=6096

and start jsreport

jsreport start

Pablo Narvaez

[nodemon] starting node --max-old-space-size=8192000 ./bin/www
that error arises when having that amount of memory allocated to the project

jan_blaha

If you watch the process in an os tool. At which memory consumption does it actually crash?
The value you use 8192000 represents actually 8192 GB.
Although not sure what you actually run in ./bin/www.