3. ¿Is there some kind of limitation on the size of the data when generating a report? (I have a report that is about 170MB)

¿Is there some kind of limitation on the size of the data when generating a report? (I have a report that is about 170MB)


  • "dependencies": {
    "jsreport": "2.6.1",
    "jsreport-phantom-pdf": "^2.4.0"
    }

    0

  • The current default is 50mb.
    You can increase it using config extensions.express.inputRequestLimit=200mb

    0

  • When I generate a report with a data of 7mb or more, it always generates a 460-page pdf that must be extensions.express.inputRequestLimit=200mb, there is an example of how to handle such large pdfs and possible errors at the time of sending it as response?

    0

  • What kind of error do you get?

    0

  • 2020-10-14T13:52:43.999Z - debug: Report stored as JtlZhCcTSIjgAkSz.pdf
    2020-10-14T13:52:44.000Z - info: Rendering request 1 finished in 4407 ms
    2020-10-14T13:53:48.274Z - debug: OData query on templates
    2020-10-14T13:53:48.280Z - debug: OData query on folders
    2020-10-14T13:53:48.285Z - debug: OData query on tags
    2020-10-14T13:53:48.287Z - debug: OData query on data
    2020-10-14T13:53:48.291Z - debug: OData query on assets
    2020-10-14T13:53:48.293Z - debug: OData query on scripts
    2020-10-14T13:53:48.299Z - debug: OData query on xlsxTemplates
    2020-10-14T13:53:48.302Z - debug: OData query on users
    2020-10-14T13:53:48.306Z - debug: OData query on schedules
    2020-10-14T13:53:48.314Z - debug: OData query on settings
    2020-10-14T13:53:48.423Z - debug: OData query on templates
    2020-10-14T13:53:48.437Z - debug: OData query on settings
    2020-10-14T13:53:57.684Z - debug: OData query on reports
    2020-10-14T13:54:06.329Z - debug: OData query on templates
    2020-10-14T13:54:06.335Z - debug: OData query on settings
    2020-10-14T14:35:27.396Z - debug: API logging in user admin
    2020-10-14T14:35:27.403Z - info: Starting rendering request 4 (user: admin)
    2020-10-14T14:35:27.406Z - info: Rendering anonymous template { recipe: phantom-pdf, engine: handlebars }
    2020-10-14T14:35:27.407Z - debug: Inline data specified.
    2020-10-14T14:35:27.407Z - debug: Resources not defined for this template.
    2020-10-14T14:35:27.409Z - debug: Base url not specified, skipping its injection.
    2020-10-14T14:35:27.417Z - debug: Rendering engine handlebars
    2020-10-14T14:36:16.471Z - debug: OData query on reports
    2020-10-14T14:36:38.590Z - debug: API logging in user admin
    2020-10-14T14:36:38.595Z - info: Starting rendering request 5 (user: admin)
    2020-10-14T14:36:38.598Z - info: Rendering anonymous template { recipe: phantom-pdf, engine: handlebars }
    2020-10-14T14:36:38.600Z - debug: Inline data specified.
    2020-10-14T14:36:38.601Z - debug: Resources not defined for this template.
    2020-10-14T14:36:38.603Z - debug: Base url not specified, skipping its injection.
    2020-10-14T14:36:38.609Z - debug: Rendering engine handlebars

    <--- Last few GCs --->

    [10508:0000026C8E1918C0] 3050691 ms: Mark-sweep 2050.5 (2067.9) -> 2050.3 (2069.6) MB, 100.8 / 0.0 ms (average mu = 0.146, current mu = 0.027) allocation failure scavenge might not succeed
    [10508:0000026C8E1918C0] 3050800 ms: Mark-sweep 2052.2 (2069.6) -> 2051.9 (2071.6) MB, 105.0 / 0.0 ms (average mu = 0.087, current mu = 0.028) allocation failure scavenge might not succeed

    <--- JS stacktrace --->

    ==== JS stack trace =========================================

    Security context: 0x00a08e92acd9 <JSObject>
    0: builtin exit frame: stringify(this=0x02a888348d69 <Object map = 000003442D245341>,0x0070498001b1 <null>,0x017d12438ec1 <JSFunction (sfi = 000001670C9503D9)>,0x017d12438ef9 <Object map = 000002BBC3A5DDF1>,0x02a888348d69 <Object map = 000003442D245341>)

    1: serialize [000000A08E920171] [D:\ajsreport\node_modules\serializator\index.js:23] [bytecode=000001670C950489 offset=149](this=0x030...

    FATAL ERROR: Ineffective mark-compacts near heap limit Allocation failed - JavaScript heap out of memory
    1: 00007FF6DA41094F napi_wrap+124431
    2: 00007FF6DA3B2696 v8::base::CPU::has_sse+34502
    3: 00007FF6DA3B3356 v8::base::CPU::has_sse+37766
    4: 00007FF6DABB6F4E v8::Isolate::ReportExternalAllocationLimitReached+94
    5: 00007FF6DAB9EF91 v8::SharedArrayBuffer::Externalize+833
    6: 00007FF6DAA6C85C v8::internal::Heap::EphemeronKeyWriteBarrierFromCode+1436
    7: 00007FF6DAA77C00 v8::internal::Heap::ProtectUnprotectedMemoryChunks+1312
    8: 00007FF6DAA74734 v8::internal::Heap::PageFlagsAreConsistent+3204
    9: 00007FF6DAA69FC3 v8::internal::Heap::CollectGarbage+1283
    10: 00007FF6DAA68794 v8::internal::Heap::AddRetainedMap+2356
    11: 00007FF6DAA90787 v8::internal::Factory::NewRawTwoByteString+87
    12: 00007FF6DA785B4B v8::internal::StringStream::ClearMentionedObjectCache+9579
    13: 00007FF6DA9A64F7 v8::internal::TimedHistogram::Stop+2823
    14: 00007FF6DA9A79A2 v8::internal::TimedHistogram::Stop+8114
    15: 00007FF6DA9A9C61 v8::internal::TimedHistogram::Stop+17009
    16: 00007FF6DA9AC890 v8::internal::TimedHistogram::Stop+28320
    17: 00007FF6DAB30CE4 v8::internal::Builtins::builtin_handle+86836
    18: 00007FF6DAFE2D6D v8::internal::SetupIsolateDelegate::SetupHeap+568205
    19: 00007FF6DAF627FC v8::internal::SetupIsolateDelegate::SetupHeap+42524
    20: 00007FF6DAF627FC v8::internal::SetupIsolateDelegate::SetupHeap+42524
    21: 00007FF6DAF627FC v8::internal::SetupIsolateDelegate::SetupHeap+42524
    22: 00007FF6DAF627FC v8::internal::SetupIsolateDelegate::SetupHeap+42524
    23: 00007FF6DAF627FC v8::internal::SetupIsolateDelegate::SetupHeap+42524
    24: 00007FF6DAF5C339 v8::internal::SetupIsolateDelegate::SetupHeap+16729
    25: 00007FF6DAF627FC v8::internal::SetupIsolateDelegate::SetupHeap+42524
    26: 00007FF6DAF5FDB1 v8::internal::SetupIsolateDelegate::SetupHeap+31697
    27: 00007FF6DAF5F99C v8::internal::SetupIsolateDelegate::SetupHeap+30652
    28: 00007FF6DAAC4F43 v8::internal::Execution::CallWasm+1395
    29: 00007FF6DAAC48C6 v8::internal::Execution::Call+182
    30: 00007FF6DABB7877 v8::Script::Run+711
    31: 00007FF6DA3BB4D8 v8::base::CPU::has_sse+70920
    32: 00007FF6DA3BDF0F v8::base::CPU::has_sse+81727
    33: 00007FF6DAB6B150 v8::internal::Builtins::builtin_handle+325536
    34: 00007FF6DAB6A697 v8::internal::Builtins::builtin_handle+322791
    35: 00007FF6DAB6A9D8 v8::internal::Builtins::builtin_handle+323624
    36: 00007FF6DAB6A7DE v8::internal::Builtins::builtin_handle+323118
    37: 00007FF6DAFE2D6D v8::internal::SetupIsolateDelegate::SetupHeap+568205
    38: 00007FF6DAF627FC v8::internal::SetupIsolateDelegate::SetupHeap+42524
    39: 00007FF6DAF627FC v8::internal::SetupIsolateDelegate::SetupHeap+42524
    40: 00007FF6DAF627FC v8::internal::SetupIsolateDelegate::SetupHeap+42524
    41: 00007FF6DAF627FC v8::internal::SetupIsolateDelegate::SetupHeap+42524
    42: 00007FF6DAF627FC v8::internal::SetupIsolateDelegate::SetupHeap+42524
    43: 00007FF6DAF627FC v8::internal::SetupIsolateDelegate::SetupHeap+42524
    44: 00007FF6DAF627FC v8::internal::SetupIsolateDelegate::SetupHeap+42524
    45: 00007FF6DAF5BBAC v8::internal::SetupIsolateDelegate::SetupHeap+14796
    46: 00007FF6DAF627FC v8::internal::SetupIsolateDelegate::SetupHeap+42524
    47: 0000022313D1ABEF
    2020-10-14T14:38:19.359Z - warn: Error when processing render request read ECONNRESET Error: read ECONNRESET
    at TCP.onStreamRead (internal/stream_base_commons.js:201:27)
    2020-10-14T14:38:19.365Z - warn: Error during processing request at http://localhost:5488/api/report
    2020-10-14T14:38:19.372Z - warn: Error when processing render request read ECONNRESET Error: read ECONNRESET
    at TCP.onStreamRead (internal/stream_base_commons.js:201:27)
    2020-10-14T14:38:19.376Z - warn: Error during processing request at http://localhost:5488/api/report

    0

  • This post is deleted!
    0

  • files configuration
    https://gist.github.com/pablonarvaez20152016/2a6fa4755a07a0fb5ceb318fe5969ea0

    0

  • {
    "extensions": {

    "authentication": {
  "cookieSession": {
    "secret": "000000"
  },
"admin": {
  "username": "admin",
  "password": "0000"
},
"enabled": true
},
"scripts": {
  "timeout": 9400000,
  "strategy": "http-server"
},
"express":{
  "inputRequestLimit":"5000mb",
  "renderTimeout":9400000
},
"phantom-pdf":{
  "timeout":9400000
},
"xlsx":{
  "escapeAmp":true,
  "numberOfParsedAddIterations":50,
  "addBufferSize":50000000
},
"sample-template": {
  "createSamples": true
}

    },

    "httpPort": 5488,
    "store": {
    "provider": "memory"
    },
    "blobStorage": {
    "provider": "memory"
    },
    "logger": {
    "console": {
    "transport": "console",
    "level": "debug"
    },
    "file": {
    "transport": "file",
    "level": "info",
    "filename": "logs/reporter.log"
    },
    "error": {
    "transport": "file",
    "level": "error",
    "filename": "logs/error.log"
    }
    },
    "allowLocalFilesAccess": true,
    "templatingEngines": {
    "timeout":9400000,
    "strategy": "http-server"
    },
    "chrome": {
    "timeout": 9400000
    },
    "Phantom": {
    "timeout": 9400000
    },
    "reportTimeout":960000
    }

    0

  • You are getting out of memory error in nodejs.
    The default is just 2Gb, please try to increase it:

    Set the NODE_OPTIONS environment variable

    NODE_OPTIONS=--max-old-space-size=6096

    and start jsreport

    jsreport start
    0

  • [nodemon] starting node --max-old-space-size=8192000 ./bin/www
    that error arises when having that amount of memory allocated to the project

    0

  • If you watch the process in an os tool. At which memory consumption does it actually crash?
    The value you use 8192000 represents actually 8192 GB.
    Although not sure what you actually run in ./bin/www.

    0
Log in to reply
 

Looks like your connection to jsreport forum was lost, please wait while we try to reconnect.