Posts made by Evan Furbeyre

We are currently trying to generate a large CSV (via the Handlebars engine, as a text file) asynchronously and save it to S3, but are running into this error when polling the status of the report: Report generation failed. Error: Cannot create a string longer than 0x1fffffe8 characters

1. Does anyone know of an alternate solution to this?
2. Is it possible to stream the report to S3 rather than build the entire thing in memory?
3. Would an XLSX file work any better?

Thanks

Hello!

We are asking for more information on how to handle very large report generation.
We have need for generating tabular reports with millions of rows / GB in size as CSV / XLSX.

• Do you have any sort of framework to handle creating large reports in an asynchronous (not schedules though, ultimately UI clicks) manner, and if so, how can we use it?
• Are you aware of any wrappers that may include this functionality?
• What are your suggestions with respect to handling a large number of rows, and any performance tuning to have it work faster.

Thanks

That worked great. Thank you!

Currently getting the following error:

Could not install from "jsreport/jsreport-scheduling@2.2.0" as it does not contain a package.json file.

Ok I am trying that, but it seems that long overdue schedules are now being sent.

Is the following block supposed to prevent schedules that are more than 5 minutes overdue from sending? Or is there a different configuration now?

{
    "extensions": {
        "scheduling": {
            "misfireThreshold": 300000
        },
    }
}

Our deployment of JSReport has stopped working. Reverting to previously working commits has not fixed anything. Checking the browser console shows these errors.

client.995b9a360c73213831bd.js:1 TypeError: Cannot read property 'object' of undefined
at Object.<anonymous> (studio-extensions.client.ad226f628cd42cdbbd0a.js:17377)
at webpack_require (studio-extensions.client.ad226f628cd42cdbbd0a.js:16814)
at Object.<anonymous> (studio-extensions.client.ad226f628cd42cdbbd0a.js:17458)
at webpack_require (studio-extensions.client.ad226f628cd42cdbbd0a.js:16814)
at studio-extensions.client.ad226f628cd42cdbbd0a.js:16878
at Object.1575 (studio-extensions.client.ad226f628cd42cdbbd0a.js:16881)
at i (client.995b9a360c73213831bd.js:1)
at client.995b9a360c73213831bd.js:66
i.oe @ client.995b9a360c73213831bd.js:1
studio-extensions.client.ad226f628cd42cdbbd0a.js:17377 Uncaught (in promise) TypeError: Cannot read property 'object' of undefined
at Object.<anonymous> (studio-extensions.client.ad226f628cd42cdbbd0a.js:17377)
at webpack_require (studio-extensions.client.ad226f628cd42cdbbd0a.js:16814)
at Object.<anonymous> (studio-extensions.client.ad226f628cd42cdbbd0a.js:17458)
at webpack_require (studio-extensions.client.ad226f628cd42cdbbd0a.js:16814)
at studio-extensions.client.ad226f628cd42cdbbd0a.js:16878
at Object.1575 (studio-extensions.client.ad226f628cd42cdbbd0a.js:16881)
at i (client.995b9a360c73213831bd.js:1)
at client.995b9a360c73213831bd.js:66

This is our Dockerfile:

FROM jsreport/jsreport:2.6.1-full

RUN npm install \
        jsreport/jsreport-scheduling \
	dotenv \
	moment \
	nodemailer \
	numeral \
	pg 
WORKDIR /app
COPY jsreport.config.json ./

COPY data ./data/

So I followed your instructions in a fresh testing repo and it worked as expected (over-due schedules being detected as misfires).

After a while (way too long...) of being perplexed why our own JSReports repo was not detecting misfires, I wondered if Docker was somehow caching the modules. I did a docker-compose down command, and afterwards over-due schedules were detected as misfires.

Side note - the logs showed:
info: Using extension scheduling@github:jsreport/jsreport-scheduling#0a6b8c39577c829b525a82078691d4f100426ee5
when that extension seemed to not actually be used, which was confusing.

Thanks for your help,
Evan

P.S.
Info about docker setup, for reference/posterity.
docker-compose.yaml:

version: '3.3'

services:
  jsreports:
    build: ./
    ports: 
      - 5488:5488
    environment:
      - extensions_fsStore_syncModifications=false
      - extensions_fsStore_sync_usePolling=false
      - ...

docker-compose.override.yaml

version: '3.3'

services:
  jsreports:

#Override file will run by default if not excluded or -f is not included at local runtime.  This is standard docker-compose behavior
#for local branch, docker compose will look different, attaching the local volume:  Want to keep the file system files updated for version control.
    volumes:
      - ./data:/app/data

Adding the following lines to the jsreport.config.json did not work. Overdue schedules were still being sent out.

{
    "extensions": {
        "scheduling": {
            "misfireThreshold": 300000 
        {
    }
}

I suspect it might be because we are currently using the jsreports/jsreport:2.6.1-full image. It looks like it was last updated ~2 months ago, and the scheduling fix was added only ~1 month ago. I tried adding npm i jsreport/jsreport-scheduling to the Dockerfile hoping that it would use the updated scheduling package, but it still didn't work.

FROM jsreport/jsreport:2.6.1-full

RUN npm install \
        jsreport/jsreport-scheduling \
	dotenv \
	moment \
	nodemailer \
	numeral \
	pg 
WORKDIR /app
COPY jsreport.config.json ./

COPY data ./data/

Several questions:
A) Do you think my explanation for why it's not working is correct?
B) Will there be an updated docker image soon?
C) Would you recommend using a node:alpine image and installing each package/version explicitly?

We have recently moved from a Postgres store to a File-System store, so that we can store and deploy all of our reports from version control. We develop new templates in a local docker container, and re-deploy periodically. This has worked well, except for one issue. The problem is that many of the schedules are being sent right after de-deploy, since we commit the local state of schedules stored in the data/schedules file, and that overwrites the state on the previously running instance. So on deployment, it seems the CRON logic thinks that many schedules are over due from their last run time.

I am wondering if this issue has been solved before, and if there are any recommendations.

Thanks,
Evan