External API call

[Ntinos Kefis]

Hello there,

I am trying to build an automatic reporting mechanism that creates schedules and sends completed reports with dynamic data through email.

So far the jsreport tutorial has been a lot of help but I am having trouble on error handling.

Firstly I should explain that I am trying to distinguish the jsreport scheduler from the report rendering by putting external API calls between the 2 tasks.
So when a schedule is triggered, it is linked to an empty template but its beforeRender script is posting to an (external) API sending some customOptions. Then the external API gathers data and calls back at jsreport, providing a template shortid and the afforementioned data to fill it with.

So, when there is an error on the external API (say, it's down) and a schedule triggers, since it's linked on an empty template, rendering is done correctly and schedule is marked as succesful, which is not the case.

Is there a way to re-run a schedule through beforeRender function based on the responses I get (or lack thereof) from the external API?
When is a schedule considered faulty in order to re-run?

[bjrmatos]

hi! you can control in beforeRender if the report rendering should fail or if it should continue, just need to use the done callback function provided in beforeRender, for example here is an empty template that always fails in beforeRender.

for your case you can detect when you API is down or something based on the result of posting to the API (probably you are using the request module to communicate with your API, which has ways to detect when there is an error in the http request), when that happens you can call done with your error done(err) to mark that this report request has failed.

is your schedule mechanism something built on your own or are you using the scheduling extension? if it is the scheduling extensions, then the report will re-run again in the next configured interval. if it is something on your own i think it is better to just repeat the request to jsreport at some time later when you detect that there is an error on the report (you can make the report fail by using the steps mentioned above), so basically re-tries is part of the job of your scheduler.

[Oshri Cohen]

I use this too call our backend API for data retrieval and it works great.

[Ntinos Kefis]

Hi, thanks for the reply, really helpful.

"basically re-tries is part of the job of your scheduler"

I am indeed using jsreport's scheduling extension, can I do these re-tries internally?

[bjrmatos]

"basically re-tries is part of the job of your scheduler"

yes, this part only applies if you were using your own scheduler, with your own logic (not the scheduling extension).

I am indeed using jsreport's scheduling extension, can I do these re-tries internally?

right now the scheduling extension will just try to render the template again on the next interval. it will not treat the failing report in a special way.

maybe later we can add an option to allow to retry failing reports in a time more short than the next interval, that should give more control about retries.

[Ntinos Kefis]

Any guidelines on how I could start implementing such a custom feature?
Maybe a way to temporarily update the schedule's nextRun, or cron fields?
Any info would be greatly appreciated, thanks!

[jan_blaha]

We will consider to add a feature to re-plan the current rendering report, however it can take some time. The best for you is to take advantage of fact that the jsreport is granular and extensions MIT licensed. Check the sources of the scheduling extension. It won't take you long time to understand it and change it to your needs
https://github.com/jsreport/jsreport-scheduling

[Ntinos Kefis]

OK, thanks a lot for the quick replies and support.

[Ntinos Kefis]

I'm bringing this thread back to ask about some issues I'm having with memory leaks on js report when I'm using the scheduling extension.

While stretch testing the extension, triggering multiple schedules at the same time I get 'FATAL ERROR: CALL_AND_RETRY_LAST Allocation failed - JavaScript heap out of memory'
when I run more than about 50 schedules simultaneously. I'm suspecting the memory leak happening due to asynchronous requests, so schedules don't finish rendering reports while new requests are coming, consequently not releasing memory.

I've tried experimenting with the extension's maxParallelJobs and interval options with little results. My machine has 8gb of ram memory and I'm running jsreport with visual studio code debugger with development configuration

Is there an optimum configuration to avoid memory leaks as much as possible? Any advice in general would be appreciated, thanks.

[bjrmatos]

when I run more than about 50 schedules simultaneously. I'm suspecting the memory leak happening due to asynchronous requests, so schedules don't finish rendering reports while new requests are coming, consequently not releasing memory.

are you sure these are memory leaks? i think that maybe your reports are big and consume a lot of data, so running 50 schedules of them simultaneously reach easily the default node.js limit of 1.76 GB for heap memory. it does not matter if you have 8gb of ram, node.js by default will just use 1.76 GB and after that it will throw an error.

maybe with maxParallelJobs and increasing the node.js heap memory limit you can make it work normally. just start jsreport with this node flag node --max_old_space_size=4096 server.js (you can adjust the max_old_space_size according to your case and available memory)

[jan_blaha]

I've just updated long reports blog post to include the "out of memory" section.
Note that the solution provided by @bjrmatos will work only for in-process rendering strategy. See details in the blog post.

If this won't help, please try to provide us a minimal repo or sources replicating the issue so we can take a look.