Possible memory issues?

dfiorentin

Good morning, we're monitoring a jsreport installation as we've revealed some anomalies. In particular, the customer informed us that periodically the jsreport service was not responding for a certain period of time.

In fact, we verified that what was happening was a memory saturation. The execution times close to 100% expand exponentially causing a slowdown of the whole system.

We solved the situation momentarily by imposing a restart of the service every night, but especially as a result of a large daily load (there are about 100 users, with a pool of a thousand daily reports executed) or very heavy print runs the phenomenon may reoccur: we noticed that the memory starts to rise inexorably, even on the night where there is no work, however, there is an almost constant increase in memory.

The service is on a 4 core 8 GB ram machine, which instantiates two jsreports balanced with Round Robin with the mongodb store provider to avoid blocking the print queue in case of a print with a high runtime. We installed Grafana to monitor the situation, and this is an example where in the middle of the day there was a forced restart. The two attached graphs include the first one the garbage collector situation and the other one is the busy memory graph, which never seems to empty completely after printing.

What could it be?

Some of the peaks are near reports executed with xlsx recipes, whose template is an excel of about 200 columns for a thousand rows. The doubt that has come to us is that the template remains in memory even after the execution is finished and therefore keeps a part of the memory locked. However it can't be only this, as also the other pdf reports (with chrome-pdf templates) cause a slight increase of memory that then accumulates inexorably. We removed the various console.logs but we don't know what to try anymore.

Thank you for help.

garbage collector

jsreport memory

jan_blaha

Thank you for the nice reports.
I would like to know what customizations in jsreport.config.json have you done except the mongo store?
What version of jsreport do you use?
Is it xlsx recipe or html-to-xlsx recipe you use?
Do you run jsreport inside docker or it runs directly?
Do you persist outputs using reports extension?
Can't your code in script or templating engine memory leak? In case you use http-server strategy.

In case you use http-server strategy for templating engines/scripts. Could you try to switch it back to the default dedicated-process temporary? Just that we try to isolate the problem.
Thank you

dfiorentin

I would like to know what customizations in jsreport.config.json have you done except the mongo store?

This is the prod configuration:

{
  "certificate": {
    "key": "certificates/jsreport.net.key",
    "cert": "certificates/jsreport.net.cert"
  },
  "httpPort": 3001,
  "httpsPort": null,
  "logger": {
    "console": {
      "transport": "console",
      "level": "debug"
    },
    "error": {
      "transport": "console",
      "level": "error"
    }
  },
  "blobStorage": {
    "provider": "gridFs"
  },
  "store": {
    "provider": "mongodb"
  },
  "phantom": {
    "defaultPhantomjsVersion": "2.1.1",
    "numberOfWorkers": 2,
    "strategy": "phantom-server",
    "timeout": 60000
  },
  "allowLocalFilesAccess": false,
  "templatingEngines": {
    "strategy": "http-server",
    "numberOfWorkers": 4,
    "timeout": 1800000,
    "allowedModules": "*"
  },
  "chrome": {
    "timeout": 40000,
    "strategy": "chrome-pool",
    "numberOfWorkers": 4,
    "launchOptions": {
        "args": "--no-sandbox"
      }
  },
  "extensions": {
    "mongodb-store": {
      "uri": "<mongouri>",
      "prefix": "jsreport_"
    },
    "authentication": {
      "cookieSession": {
        "secret": "<secret>"
      },
      "admin": {
        "username": "<user>",
        "password": "<psw>"
      }
    },
    "scripts": {
      "allowedModules": "*",
      "timeout": 1800000
    },
    "sample-template": {
      "createSamples": true
    },
    "chrome-pdf": { 
    "timeout": 40000,
      "strategy": "chrome-pool", 
      "numberOfWorkers": 4,
      "launchOptions": {
        "args": "--no-sandbox"
      }
    }
  }
}

What version of jsreport do you use?

Those are the version of the packages:

    "jsreport": "2.6.1",
    "jsreport-chrome-pdf": "1.6.1",
    "jsreport-client-app": "^0.1.1",
    "jsreport-fs-store": "2.6.0",
    "jsreport-import-export": "1.4.0",
    "jsreport-mongodb-store": "1.2.2",
    "jsreport-pdf-utils": "1.5.2",
    "jsreport-phantom-pdf": "2.2.0",
    "jsreport-studio": "2.6.6",

Is it xlsx recipe or html-to-xlsx recipe you use?

The recipe is xlsx, with a custom xslx template used for every excel reports. For pdf reports we always use chrome-pdf (on jsreport settings you'll see phantom just for legacy reports on other customers, but it's not used here)

Do you run jsreport inside docker or it runs directly?

jsreport starts in a docker, with the starting script that just set some env variable and calls jsreport.init()

Do you persist outputs using reports extension?

No, we'll use an API request to create the report using jsreport, then it send back the b64 and we save the pdf in a different folder using a fs.writeFileSync(tmpName, (new Buffer(report, 'base64')).toString('binary'), 'binary');

Can't your code in script or templating engine memory leak? In case you use http-server strategy.

That was our first thought, in fact, we have worked considerably to make a general review for memory leak, but despite this we can not understand why this trend in memory. The strategy is included on the conf that is published before:

"templatingEngines": {
    "strategy": "http-server",
    "numberOfWorkers": 4,
    "timeout": 1800000,
    "allowedModules": "*"
  },

We could try to change the strategy to the dedicated-process to see if something change.

Thank you for your support; we will stay in touch with you for any further reports or updates.