Error Starting JSREPORT on K8S

DaniloOliveira28

Earlier this morning, we encountered an issue with our filesystem managed by GKE (Kubernetes). After this event, jsreport stopped starting.

Before the incident, the startup process used to take a maximum of 60 seconds. Now, it takes about an hour, failing at the last step with the error message below:

[33mwarn[39m: Error when cleaning fs journal, no worry, we will run again Timeout during waiting for file system, try it again later. stack=Error: Timeout during waiting for file system, try it again later.
    at Object.rejectItemsWithTimeout (/app/node_modules/@jsreport/jsreport-fs-store/lib/queue.js:52:23)
    at Timeout._onTimeout (/app/node_modules/@jsreport/jsreport-fs-store/lib/transaction.js:31:11)
    at listOnTimeout (node:internal/timers:569:17)
    at process.processTimers (node:internal/timers:512:7)

[31merror[39m: Error occurred during reporter init: Error: Timeout occurred when waiting for the worker action "init"
    at Timeout.<anonymous> (/app/node_modules/@jsreport/advanced-workers/lib/threadWorker.js:43:23)
    at listOnTimeout (node:internal/timers:569:17)
    at process.processTimers (node:internal/timers:512:7)

2024-11-25 12:54:20.061 BRT
Error: Timeout occurred when waiting for the worker action "init" at Timeout.<anonymous> (/app/node_modules/@jsreport/advanced-workers/lib/threadWorker.js:43:23) at listOnTimeout (node:internal/timers:569:17) at process.processTimers (node:internal/timers:512:7) {
2024-11-25 12:54:20.061 BRT
code: 'WORKER_TIMEOUT'
2024-11-25 12:54:20.061 BRT
}

I am trying to understand what could be causing this issue. Do you have any tips or suggestions?

We are using jsreport 4.7.0 and this is our jsreport config file

{
  "extensions": {
    "authentication": {
      "cookieSession": {
        "secret": "{{JSREPORT_AUTH_COOKIE}}"
      },
      "admin": {
        "username": "{{JSREPORT_ADMIN_USERNAME}}",
        "password": "{{JSREPORT_ADMIN_PASSWORD}}"
      },
      "enabled": true
    },
    "scripts": {
      "strategy": "http-server",
      "timeout": 1800000
    },
    "fs-store": {
      "dataDirectory": "/var/jsreport-data"
    },
    "express": {
      "renderTimeout": 1800000
    },
    "reports": {
      "cleanInterval": "1d",
      "cleanThreshold": "3d",
      "cleanParallelLimit": 10
    }
  },
  "httpPort": 8080,
  "store": {
    "provider": "fs"
  },
  "logger": {
    "console": {
      "transport": "console",
      "level": "debug"
    },
    "file": { 
      "transport": "file", 
      "level": "debug", 
      "filename": "/var/jsreport-data/jsreport-log.txt" 
    },
    "error": {
      "transport": "file",
      "level": "error",
      "filename": "/var/jsreport-data/jsreport-error.txt"
    }
  },
  "trustUserCode": true,
  "templatingEngines": {
    "strategy": "http-server"
  },
  "enableRequestReportTimeout": true,
  "reportTimeout": 1800000,
  "office": {
    "preview": {  
      "enabled": false  
    },
    "timeout": 1800000
  },
  "licenseKey": "{{KEY}}",
  "license": {
    "development": {{LICENSE_DEVELOPMENT}}
  }  
}

admin

If you compress your data folder and email it to me, I will try to analyze what's happening.

What could help is deleting files fs.journal and fs.version before starting jsreport.

DaniloOliveira28

I have removed fs.journal, fs.lock and and fs.version before starting jsreport. it worked.

But it taking too much time to process reports. Can I remove the storage/profile and storage/reports folder?

admin

But it taking too much time to process reports. Can I remove the storage/profile and storage/reports folder?

These aren't being loaded during the startup, so it shouldn't slow things down.

I am guessing you have too many reports in the data/reports. Maybe you should lower the extensions.reports cleaning config. If you render many async reports, I can imagine that loading a million of them during boot can take some time.
If you want to get rid of the stored reports, you can manually delete data/reports and storage/reports.

DaniloOliveira28

Jan, thank you for the support!

I deleted the data/reports file and the storage/reports directory, then redeployed the deployment in our Kubernetes cluster, and it worked.

My guess is that GKE (Kubernetes Engine) applied an update that affected latency or something in the deployment, which might have corrupted some of the files I deleted.

Anyway, everything is working now. Thanks again!