Topics created by Evan Furbeyre

Does anyone know of an alternate solution to this?

Unfortunately, the handlebars need to hold results in the memory so you can't generate CSV bigger than ~500mb this way. We have a solution we want to implement mentioned here:
https://github.com/jsreport/jsreport/issues/839

Is it possible to stream the report to S3 rather than build the entire thing in memory?

You could make the template "empty" and not use the handlebars to concatenate the csv. Instead, you implement the afterRender script. This script would "batch loop" your data, create csv text lines and stream the result to the s3 using aws s3 nodejs SDK.

Would an XLSX file work any better?

Yes, the xlsxAdd helper batches the XML into files which are later streamed and compressed to the xlsx. So the memory consumption there is better.

That worked great. Thank you!

So I followed your instructions in a fresh testing repo and it worked as expected (over-due schedules being detected as misfires).

After a while (way too long...) of being perplexed why our own JSReports repo was not detecting misfires, I wondered if Docker was somehow caching the modules. I did a docker-compose down command, and afterwards over-due schedules were detected as misfires.

Side note - the logs showed:
info: Using extension scheduling@github:jsreport/jsreport-scheduling#0a6b8c39577c829b525a82078691d4f100426ee5
when that extension seemed to not actually be used, which was confusing.

Thanks for your help,
Evan

P.S.
Info about docker setup, for reference/posterity.
docker-compose.yaml:

version: '3.3' services: jsreports: build: ./ ports: - 5488:5488 environment: - extensions_fsStore_syncModifications=false - extensions_fsStore_sync_usePolling=false - ...

docker-compose.override.yaml

version: '3.3' services: jsreports: #Override file will run by default if not excluded or -f is not included at local runtime. This is standard docker-compose behavior #for local branch, docker compose will look different, attaching the local volume: Want to keep the file system files updated for version control. volumes: - ./data:/app/data