I updated all the packages including jsreport by doing "npm install package@latest" for each one and things appear to be working correctly now in the production instance. Should have tried that first I guess. Thanks for your help!

FWIW, i did just install latest version of jsreport-cli globally as per on-prem instructions, and recreate an the app on the production server on a different port and in a different folder in the production environment, and xlsx works correctly, so I will stare and compare the configs - must be something didn't get added when we upgraded from version 2 to 3 maybe... If I figure it out i will post - thanks for the suggestion.

Here is the package.json:

{
"name": "jsreport-server",
"main": "server.js",
"scripts": {
"start": "node server",
"jsreport": "jsreport"
},
"jsreport": {
"entryPoint": "server.js"
},
"dependencies": {
"@jsreport/jsreport-unoconv": "^3.0.0",
"axios": "^0.24.0",
"cheerio-page-eval": "^1.0.0",
"dayjs": "^1.10.7",
"jsreport": "^3.2.0",
"mssql": "^7.3.0",
"pg": "^8.7.1"
}
}

I work with tincanium - here are the logs - our production server is Alma Linux 8 FWIW - I installed unoconv and restarted because i noticed the playground had that and ours did not but that didn't seem to fix it - i notice playground also has "office password" extension but can't think that is required? This is our first time trying the xlsx recipe - but pdf and html-to-xlsx are working fine in production so I'm not sure what the problem is.

2022-06-01T21:26:49.956Z - info: Initializing jsreport (version: 3.4.1, configuration file: jsreport.config.json, nodejs: 16.11.1)
2022-06-01T21:26:49.959Z - info: Searching for available extensions in /redacted/
2022-06-01T21:26:49.966Z - info: Extensions location cache contains up to date information, skipping crawling in /redacted/
2022-06-01T21:26:50.008Z - info: Found 36 extension(s)
2022-06-01T21:26:50.036Z - debug: Writing extension locations cache to /tmp/jsreport/core/locations.json
2022-06-01T21:26:50.037Z - debug: Discovered 36 extensions
2022-06-01T21:26:50.092Z - info: Using extension authentication@3.2.0
2022-06-01T21:26:50.230Z - debug: Extension authentication@3.2.0 was disabled
2022-06-01T21:26:50.231Z - info: Using extension base@3.0.0
2022-06-01T21:26:50.235Z - info: Using extension child-templates@3.0.1
2022-06-01T21:26:50.236Z - info: Using extension cli@3.1.0
2022-06-01T21:26:50.239Z - info: Using extension components@3.2.0
2022-06-01T21:26:50.240Z - info: Using extension data@3.0.1
2022-06-01T21:26:50.241Z - info: Using extension express@3.3.1
2022-06-01T21:26:50.462Z - info: Using extension freeze@3.0.1
2022-06-01T21:26:50.464Z - info: Using extension fs-store@3.1.0
2022-06-01T21:26:50.536Z - info: Using extension handlebars@3.0.0
2022-06-01T21:26:50.538Z - info: Using extension import-export@3.1.1
2022-06-01T21:26:50.547Z - info: Using extension jsrender@3.0.0
2022-06-01T21:26:50.548Z - info: Using extension licensing@3.0.2
2022-06-01T21:26:50.565Z - info: Using extension localization@3.2.0
2022-06-01T21:26:50.565Z - info: Using extension npm@3.1.1
2022-06-01T21:26:50.569Z - info: Using extension pdf-utils@3.3.0
2022-06-01T21:26:50.570Z - info: Using extension reports@3.0.3
2022-06-01T21:26:50.573Z - info: Using extension tags@3.1.0
2022-06-01T21:26:50.576Z - info: Using extension text@3.0.0
2022-06-01T21:26:50.577Z - debug: Extension version-control@3.1.0 is disabled, skipping
2022-06-01T21:26:50.577Z - info: Using extension assets@3.3.0
2022-06-01T21:26:50.605Z - info: Using extension authorization@3.2.0
2022-06-01T21:26:50.611Z - debug: Extension authorization@3.2.0 was disabled
2022-06-01T21:26:50.612Z - info: Using extension browser-client@3.1.0
2022-06-01T21:26:50.616Z - info: Using extension chrome-pdf@3.1.0
2022-06-01T21:26:50.620Z - debug: Chrome strategy is chrome-pool, numberOfWorkers: 1
2022-06-01T21:26:50.620Z - info: Using extension docx@3.2.0
2022-06-01T21:26:50.623Z - info: Using extension html-to-xlsx@3.2.0
2022-06-01T21:26:50.872Z - info: html-to-xlsx detected chrome as available html engine
2022-06-01T21:26:50.873Z - info: html-to-xlsx detected cheerio as available html engine
2022-06-01T21:26:50.873Z - info: Using extension pptx@3.1.0
2022-06-01T21:26:50.875Z - info: Using extension scheduling@3.0.1
2022-06-01T21:26:50.912Z - info: Using extension scripts@3.2.0
2022-06-01T21:26:50.913Z - info: Using extension static-pdf@3.0.0
2022-06-01T21:26:50.913Z - info: Using extension studio@3.4.0
2022-06-01T21:26:50.979Z - info: Using extension xlsx@3.1.0
2022-06-01T21:26:50.999Z - info: Using extension public-templates@3.0.0
2022-06-01T21:26:51.000Z - debug: Extension public-templates@3.0.0 was disabled
2022-06-01T21:26:51.000Z - info: Using extension sample-template@3.1.0
2022-06-01T21:26:51.002Z - info: Using extension studio-theme-dark@3.0.1
2022-06-01T21:26:51.003Z - info: Using extension unoconv@3.0.0
2022-06-01T21:26:51.005Z - info: Using general timeout for rendering (reportTimeout: 600000)
2022-06-01T21:26:51.005Z - info: Using fs provider for template store.
2022-06-01T21:26:51.013Z - info: fs store is persisting using fs for /redacted/data
2022-06-01T21:26:51.024Z - info: fs store is loading data
2022-06-01T21:26:52.937Z - info: fs store is initialized successfully
2022-06-01T21:26:52.945Z - debug: studio default theme is: light
2022-06-01T21:26:52.969Z - info: Creating default express app.
2022-06-01T21:26:52.986Z - debug: Reading ssl certificate from /redacted/fullchain.cer
2022-06-01T21:26:52.998Z - info: jsreport server successfully started on https port: 5489
2022-06-01T21:26:53.005Z - info: Verifying license key free
2022-06-01T21:26:53.006Z - info: License key being verified and license key stored in /redacted/jsreport.license.json doesn't match, proce>
(node:2387678) Warning: Setting the NODE_TLS_REJECT_UNAUTHORIZED environment variable to '0' makes TLS connections and HTTPS requests insecure by disabling certifi>
(Use node --trace-warnings ... to show where the warning was created)
2022-06-01T21:26:53.599Z - info: License key verified as yearly subscription
2022-06-01T21:26:53.600Z - info: Storing license verification information to /redacted/jsreport.license.json
2022-06-01T21:26:53.603Z - debug: Creating samples is disabled
2022-06-01T21:26:53.604Z - info: Initializing worker threads
2022-06-01T21:26:53.604Z - debug: Extensions in workers: base, child-templates, components, data, express, handlebars, jsrender, localization, npm, pdf-utils, repo>
2022-06-01T21:26:53.819Z - info: 2 worker threads initialized in 215ms
2022-06-01T21:26:53.819Z - info: Starting temp files cleanup with 600000ms threshold
2022-06-01T21:26:53.820Z - info: reporter initialized

Actually i jumped the gun here in blaming it on jsreport. I turned the old server on and compared the results of:

https://oldserver:5489/odata/templates('WREPnPpjt5FaVlAu')

vs

https://newserver:5489/odata/templates('WREPnPpjt5FaVlAu')

and the old one is also malformed. Both return something like:

{
  "0": {
    "shortid": "Ssh0qJ_M1t",
    "name": "Multiple Data Sources to XLSX",
    "recipe": "html-to-xlsx",
    "engine": "handlebars",
    "chrome": { "printBackground": true },
    "htmlToXlsx": { "htmlEngine": "chrome" },
    "creationDate": "2021-02-12T22:16:32.865Z",
    "modificationDate": "2022-03-21T21:02:06.922Z",
    "_id": "WREPnPpjt5FaVlAu",
    "data": { "shortid": "alzfT9oXW" },
    "scripts": [ { "shortid": "YV1W9PMikJ" } ],
    "content": "...",
    "helpers": "...",
    "folder": { "shortid": "sYpZKHs" }
  },
  "@odata.context": "https://192.168.7.198:5489/odata/$metadata#templates/$entity",
  "value": [
    {
      "shortid": "Ssh0qJ_M1t",
      "name": "Multiple Data Sources to XLSX",
      "recipe": "html-to-xlsx",
      "engine": "handlebars",
      "chrome": { "printBackground": true },
      "htmlToXlsx": { "htmlEngine": "chrome" },
      "creationDate": "2021-02-12T22:16:32.865Z",
      "modificationDate": "2022-03-21T21:02:06.922Z",
      "_id": "WREPnPpjt5FaVlAu",
      "data": { "shortid": "alzfT9oXW" },
      "scripts": [ { "shortid": "YV1W9PMikJ" } ],
      "content": "...",
      "helpers": "...",
      "folder": { "shortid": "sYpZKHs" }
    }
  ]
}

I believe the corrrect odata response for a "by key" URL would be just:

{
  "@odata.context": "https://192.168.7.198:5489/odata/$metadata#templates/$entity",
  "shortid": "Ssh0qJ_M1t",
  "name": "Multiple Data Sources to XLSX",
  "recipe": "html-to-xlsx",
  "engine": "handlebars",
  "chrome": { "printBackground": true },
  "htmlToXlsx": { "htmlEngine": "chrome" },
  "creationDate": "2021-02-12T22:16:32.865Z",
  "modificationDate": "2022-03-21T21:02:06.922Z",
  "_id": "WREPnPpjt5FaVlAu",
  "data": { "shortid": "alzfT9oXW" },
  "scripts": [ { "shortid": "YV1W9PMikJ" } ],
  "content": "...",
  "helpers": "...",
  "folder": { "shortid": "sYpZKHs" }
}

Turns out we also upgraded our server code from .net 3.1 to .net 6 and switched from newtonsoft the the microsoft json stuff, and something with that is causing the problem which i will track down. Sorry for the false alarm in my first post.

At some point it would be nice if jsreport returned valid odata responses for "by key" type queries but we can work around it for now.

We took v3 live in production yesterday and one thing our testing unfortunately did not reveal before going live was that odata api seems to behave differently in v3. We have some places in code code that make calls like:

https://server:5489/odata/templates('WREPnPpjt5FaVlAu')

This worked fine with v2 - and I'm pretty sure that value between the ticks used to be the shortid - but now in v3 it is _id and the shortid is something different. Worse than that, the above call returns an invalid odata payload (in addition to "@odata.context" and "value" which are set correctly, it has an array that duplicates what is under "value" at the start which messes things up).

Oddly, if i change the query to be:

https://server:5489/odata/templates?$filter=shortid eq 'newshortid'

I get the expected valid odata response. But if i change it to be:

https://server:5489/odata/templates?$filter=_id eq 'WREPnPpjt5FaVlAu'

It returns the same malformed odata response with the array at the front you get when using the "by key" syntax.

Advice? Seems like it is a bug that could probably easily be fixed?

1. Can you use folders?

The documentation says:

{ "options": { "reports": { "save": true, "blobName": "myfilename" } }

Are paths supported??? i.e.

{ "options": { "reports": { "save": true, "blobName": "folder1/folder2/myfilename" } }

where jsreport will create the folder hierarchy as needed - it seems otherwise everything is in the same folder, which might get to be a mess with tens or hundreds of thousands of files.

2. Relatedly, is this scalable - are there people actually storing/serving a lot of files using this method?

3. I have a bunch of legacy PDFs that were created by the system we used before jsreport, but i will still need to have somewhere and serve up for a period. Could i import them in using the odata api maybe or do i just copy the files over into the storage folder and jsreport will automatically pick them up - the documentation is kinda thin on this?

4. In addition or in lieu of the current config file cleanup mechanism, it might be nice to come at it from the other way via the api, i.e.:

{ "options": { "reports": { "save": true, "blobName": "myfilename", "keepFor": "2 months" } } or something like that

Do you have to do anything special to get emojis to work when using the html-to-xlsx recipe? It looks right in the JSON, but I get an error when trying to render the spreadsheet? I was able to duplicate the behavior by just going to the "learn" site and modifying the data.json for the basic html2xlsx example to have an emoji which will hopefully help you duplicate the problem:

Report "basic" render failed.

Invalid character "�", char code: 55357 in string "Economist😊" at index: 9
Error: Invalid character "�", char code: 55357 in string "Economist😊" at index: 9
at Object.assetLegalXMLChar (/app/node_modules/html-to-xlsx/lib/utils.js:180:11)
at addRow (/app/node_modules/html-to-xlsx/lib/tableToXlsx.js:80:11)
at /app/node_modules/html-to-xlsx/lib/tableToXlsx.js:45:7
at /app/packages/jsreport-html-to-xlsx/lib/htmlToXlsxProcess.js:103:17
at new Promise (<anonymous>)
at Object.getRows (/app/packages/jsreport-html-to-xlsx/lib/htmlToXlsxProcess.js:97:16)
at tableToXlsx (/app/node_modules/html-to-xlsx/lib/tableToXlsx.js:44:17)
at async convert (/app/node_modules/html-to-xlsx/lib/conversion.js:53:20)
at async scriptHtmlToXlsxProcessing (/app/packages/jsreport-html-to-xlsx/lib/htmlToXlsxProcess.js:55:20)
at async module.exports (/app/packages/jsreport-html-to-xlsx/lib/recipe.js:131:18)
at async invokeRender (/app/packages/jsreport-core/lib/worker/render/render.js:97:5)
at async WorkerReporter._render (/app/packages/jsreport-core/lib/worker/render/render.js:151:7)
at async /app/packages/jsreport-core/lib/worker/reporter.js:165:19
at async Domain.<anonymous> (/app/packages/advanced-workers/lib/workerHandler.js:134:19)

FWIW, a better workaround I think is just to switch the CA that acme.sh or whatever you ACME client may be uses - everybody is getting in on the free certificate game now:

https://github.com/acmesh-official/acme.sh/wiki/ZeroSSL.com-CA

That fixed it for me.

One of letsencrypt root certs expired today, and their backup trust relationship seems to be causing problems with api calls to or from jsreport running on CentOS8.

If I open jsreport studio or my web api up in a browser, it works fine, because the browser understands the new trust relationship. However, if I make api calls to jsreport from my c# code, i get certificate errors - and conversely, if my jsreport script makes api calls to my c# web api also running on the same machine, it gets certificate errors - so it seems that both node and dotnet core 3.1 for linux don't know how to deal with the newer root cert letsencrypt is now using.

I was able to work around the problem by changing my c# code and the jsreport startup script to ignore certificate errors but am hoping someone else runs into this problem who knows how to fix it correctly. I presume there is a way to add a new root cert to both node and dotnet on linux, but it looked to me like at least for node, they are maybe compiled in at build time rather than read from say the filesystem? I tried upgrading both node and dotnet to latest LTS, but that didn't fix it unfortunately.

You can read more about the change that broke things here:

https://letsencrypt.org/docs/dst-root-ca-x3-expiration-september-2021/