3. Unable to fetch data from remote server in beforeRender()

Unable to fetch data from remote server in beforeRender()

  • P

    I'm trying to fetch model data to be rendered from a remote server (Couchbase server) using scripts and beforeRender() function. I'm running jsreport from Docker image

    First, I have enabled modules in script config section:

    ...
  "scripts": {
    "allowedModules": "*",
    "timeout": 60000
  },
...

    I've installed Couchbase module during docker image build with the following Dockerfile:

    FROM jsreport/jsreport

WORKDIR /usr/src/app
RUN JOBS=MAX npm install --save couchbase --production  && npm cache clean --force && rm -rf /tmp/*

COPY ./jsreport-home /jsreport

EXPOSE 5488

    then I try to fetch data with this code in scripts:

    const couchbase = require('couchbase');
// Couchbase init

function beforeRender(req, res, done) {
    var myCluster = new couchbase.Cluster(`couchbase://xxx.yyy.zzz`);
    myCluster.authenticate('xxx', 'xxx');
    const N1qlQuery = couchbase.N1qlQuery;
    const query = N1qlQuery.fromString("select ...");
    var bucket = myCluster.openBucket("xxx");
    bucket.query(query, (err, rows) => {
      console.warn('query ret');
      if (!err) {
        console.warn("query result: " + JSON.stringify(rows));
        req.data = Object.assign({}, req.data, {rows: body});
        done();
      } else {
        console.error("query error: " + err);
      }
      bucket.disconnect();
      done();
    });
}

    It seems that the script never returns and I face render timeout. The same code executed with node.js inside the same Docker image returns data in less than 1 second.

    What's wrong with my script? Does the external module is installed in the right dir?

    Thanks.
    Paolo

    0

  • Hm. The module require would fail and script would immediately return if the module is not found.
    At which line the script hangs? Try to remove the code step by step to find it out.
    Hopefully it will give us some more insight.

    It is likely not the problem. However I see that you call done function twice.
    Try to fix that.

    0
  • P

    Hi Jan,
    thanks for you quick reply. The script hangs inside the callback of

    bucket.query(...)

    If i put some logging before and stop script execution with done just before bucket.query(...) everything goes, I see log output and report is rendered,
    This is log for the script without the aync call:

    jsreport_1_547a27b11a49 | 2018-11-21T16:53:28.536Z - info: Starting rendering request 4 (user: null) requestId=4
jsreport_1_547a27b11a49 | 2018-11-21T16:53:28.537Z - info: Rendering template {shortid:Bka06O2a7, recipe:xlsx, engine:handlebars, preview:true} requestId=4
jsreport_1_547a27b11a49 | 2018-11-21T16:53:28.545Z - debug: Executing script rkWalpW07 requestId=4
jsreport_1_547a27b11a49 | 2018-11-21T16:53:28.567Z - warn: beforeRender timestamp=Wed Nov 21 2018 16:53:28 GMT+0000 (UTC), requestId=4  <-- LOGS FROM SCRIPT
jsreport_1_547a27b11a49 | 2018-11-21T16:53:28.567Z - warn: query:{...} timestamp=Wed Nov 21 2018 16:53:28 GMT+0000 (UTC), requestId=4  <-- LOGS FROM SCRIPT
jsreport_1_547a27b11a49 | 2018-11-21T16:53:28.569Z - debug: Resources not defined for this template. requestId=4
jsreport_1_547a27b11a49 | 2018-11-21T16:53:28.570Z - debug: Base url not specified, skipping its injection. requestId=4
jsreport_1_547a27b11a49 | 2018-11-21T16:53:28.571Z - debug: Rendering engine handlebars requestId=4
jsreport_1_547a27b11a49 | 2018-11-21T16:53:28.611Z - debug: Taking compiled template from engine cache timestamp=Wed Nov 21 2018 16:53:28 GMT+0000 (UTC), requestId=4
jsreport_1_547a27b11a49 | 2018-11-21T16:53:28.612Z - debug: Replaced images [] requestId=4
jsreport_1_547a27b11a49 | 2018-11-21T16:53:28.617Z - debug: Executing recipe xlsx requestId=4
jsreport_1_547a27b11a49 | 2018-11-21T16:53:28.617Z - debug: Parsing xlsx content requestId=4
jsreport_1_547a27b11a49 | 2018-11-21T16:53:28.659Z - debug: Zipping prepared xml files into /tmp/jsreport/f85a4230-edad-11e8-bc2e-1f43372a20a5.xlsx requestId=4
jsreport_1_547a27b11a49 | escape "
jsreport_1_547a27b11a49 | escape "
jsreport_1_547a27b11a49 | escape "
jsreport_1_547a27b11a49 | escape "
jsreport_1_547a27b11a49 | escape "
jsreport_1_547a27b11a49 | escape "
jsreport_1_547a27b11a49 | escape "
jsreport_1_547a27b11a49 | escape "
jsreport_1_547a27b11a49 | 2018-11-21T16:53:28.686Z - debug: Successfully zipped now. requestId=4
jsreport_1_547a27b11a49 | 2018-11-21T16:53:28.687Z - debug: Skipping storing report. requestId=4
jsreport_1_547a27b11a49 | 2018-11-21T16:53:28.693Z - info: Rendering request 4 finished in 156 ms requestId=4
jsreport_1_547a27b11a49 | 2018-11-21T16:53:40.097Z - debug: OData update on scripts

    ...while removing the intermediate done the script hangs and I also can't see any console output of previous warn logs:

    jsreport_1_547a27b11a49 | 2018-11-21T16:53:41.178Z - info: Starting rendering request 5 (user: null) requestId=5
jsreport_1_547a27b11a49 | 2018-11-21T16:53:41.180Z - info: Rendering template {shortid:Bka06O2a7, recipe:xlsx, engine:handlebars, preview:true} requestId=5
jsreport_1_547a27b11a49 | 2018-11-21T16:53:41.182Z - debug: Adding sample data r19z0uha7 requestId=5
jsreport_1_547a27b11a49 | 2018-11-21T16:53:41.187Z - debug: Executing script rkWalpW07 requestId=5
jsreport_1_547a27b11a49 | 2018-11-21T16:54:41.169Z - warn: Error when processing render request Error during rendering report: Timeout Error: Timeout
jsreport_1_547a27b11a49 |     at Timeout._onTimeout (/usr/src/app/node_modules/script-manager/lib/manager-servers.js:137:23)
jsreport_1_547a27b11a49 |     at ontimeout (timers.js:475:11)
jsreport_1_547a27b11a49 |     at Timer.unrefdHandle (timers.js:588:5) requestId=5
jsreport_1_547a27b11a49 | 2018-11-21T16:54:41.170Z - warn: Error during processing request: http://127.0.0.1:8080/api/report/iPatrolTours details: Error during rendering report: Timeout Error: Timeout
jsreport_1_547a27b11a49 |     at Timeout._onTimeout (/usr/src/app/node_modules/script-manager/lib/manager-servers.js:137:23)
jsreport_1_547a27b11a49 |     at ontimeout (timers.js:475:11)
jsreport_1_547a27b11a49 |     at Timer.unrefdHandle (timers.js:588:5)

    I'm using JSreport 1.10.0

    0

  • It took me half day but I have found the problem for you.
    It is actually bug in the couchbase node driver.
    It doesn't work correctly in the nodejs vm which we use for sandboxing scripts.

    I prepared fix for the driver and sent PR
    https://github.com/couchbase/couchnode/pull/89

    Fortunately you should be able to workaround it.
    You need to stop using callbacks when working with driver and use events instead.
    Something like this

    bucket.query(...).on('rows', (...) => ...)
    0

  • Also we found out that this problem disappear in the jsreport v2 because it uses different sandboxing strategy

    0
  • P

    That's amazing, really thankful, I'll asap give a try to your solution and let you know. I'll also consider to move to jsreport v2 (already gave a try but struggled installing couchbase module).

    Thank you very much for your support.
    Paolo

    0
  • P

    I've been able to get it working with bucket.query(...).on('rows', (...) => ...). Thanks a lot!

    0
Log in to reply
 

Looks like your connection to jsreport forum was lost, please wait while we try to reconnect.