3. Error Starting JSREPORT on K8S

Error Starting JSREPORT on K8S

  • D

    Earlier this morning, we encountered an issue with our filesystem managed by GKE (Kubernetes). After this event, jsreport stopped starting.

    Before the incident, the startup process used to take a maximum of 60 seconds. Now, it takes about an hour, failing at the last step with the error message below:

    [33mwarn: Error when cleaning fs journal, no worry, we will run again Timeout during waiting for file system, try it again later. stack=Error: Timeout during waiting for file system, try it again later.
    at Object.rejectItemsWithTimeout (/app/node_modules/@jsreport/jsreport-fs-store/lib/queue.js:52:23)
    at Timeout._onTimeout (/app/node_modules/@jsreport/jsreport-fs-store/lib/transaction.js:31:11)
    at listOnTimeout (node:internal/timers:569:17)
    at process.processTimers (node:internal/timers:512:7)

[31merror: Error occurred during reporter init: Error: Timeout occurred when waiting for the worker action "init"
    at Timeout.<anonymous> (/app/node_modules/@jsreport/advanced-workers/lib/threadWorker.js:43:23)
    at listOnTimeout (node:internal/timers:569:17)
    at process.processTimers (node:internal/timers:512:7)

2024-11-25 12:54:20.061 BRT
Error: Timeout occurred when waiting for the worker action "init" at Timeout.<anonymous> (/app/node_modules/@jsreport/advanced-workers/lib/threadWorker.js:43:23) at listOnTimeout (node:internal/timers:569:17) at process.processTimers (node:internal/timers:512:7) {
2024-11-25 12:54:20.061 BRT
code: 'WORKER_TIMEOUT'
2024-11-25 12:54:20.061 BRT
}

    I am trying to understand what could be causing this issue. Do you have any tips or suggestions?

    We are using jsreport 4.7.0 and this is our jsreport config file

    {
  "extensions": {
    "authentication": {
      "cookieSession": {
        "secret": "{{JSREPORT_AUTH_COOKIE}}"
      },
      "admin": {
        "username": "{{JSREPORT_ADMIN_USERNAME}}",
        "password": "{{JSREPORT_ADMIN_PASSWORD}}"
      },
      "enabled": true
    },
    "scripts": {
      "strategy": "http-server",
      "timeout": 1800000
    },
    "fs-store": {
      "dataDirectory": "/var/jsreport-data"
    },
    "express": {
      "renderTimeout": 1800000
    },
    "reports": {
      "cleanInterval": "1d",
      "cleanThreshold": "3d",
      "cleanParallelLimit": 10
    }
  },
  "httpPort": 8080,
  "store": {
    "provider": "fs"
  },
  "logger": {
    "console": {
      "transport": "console",
      "level": "debug"
    },
    "file": { 
      "transport": "file", 
      "level": "debug", 
      "filename": "/var/jsreport-data/jsreport-log.txt" 
    },
    "error": {
      "transport": "file",
      "level": "error",
      "filename": "/var/jsreport-data/jsreport-error.txt"
    }
  },
  "trustUserCode": true,
  "templatingEngines": {
    "strategy": "http-server"
  },
  "enableRequestReportTimeout": true,
  "reportTimeout": 1800000,
  "office": {
    "preview": {  
      "enabled": false  
    },
    "timeout": 1800000
  },
  "licenseKey": "{{KEY}}",
  "license": {
    "development": {{LICENSE_DEVELOPMENT}}
  }  
}
    0

  • If you compress your data folder and email it to me, I will try to analyze what's happening.

    What could help is deleting files fs.journal and fs.version before starting jsreport.

    0
  • D

    I have removed fs.journal, fs.lock and and fs.version before starting jsreport. it worked.

    But it taking too much time to process reports. Can I remove the storage/profile and storage/reports folder?

    0

  • But it taking too much time to process reports. Can I remove the storage/profile and storage/reports folder?

    These aren't being loaded during the startup, so it shouldn't slow things down.

    I am guessing you have too many reports in the data/reports. Maybe you should lower the extensions.reports cleaning config. If you render many async reports, I can imagine that loading a million of them during boot can take some time.
    If you want to get rid of the stored reports, you can manually delete data/reports and storage/reports.

    0
  • D

    Jan, thank you for the support!

    I deleted the data/reports file and the storage/reports directory, then redeployed the deployment in our Kubernetes cluster, and it worked.

    My guess is that GKE (Kubernetes Engine) applied an update that affected latency or something in the deployment, which might have corrupted some of the files I deleted.

    Anyway, everything is working now. Thanks again!

    0
  • N

    Hi, I am getting this error. I am trying to deploy it in a serverless environment: Google's CloudRun.

    It's a small deployment with only a few templates.

    About 30 seconds after the service starts up, I get:

    Error: Timeout occurred when waiting for the worker action "init"
    at Timeout.<anonymous> (/app/reporting/node_modules/@jsreport/advanced-workers/lib/threadWorker.js:43:23)
    at listOnTimeout (node:internal/timers:594:17)
    at process.processTimers (node:internal/timers:529:7) { code: 'WORKER_TIMEOUT' }

    I don't have this issue if the container runs locally.

    I do not have a license and am evaluating JSReport as a solution. I wanted to make sure I could get it running in a serverless environment.

    Any help would be appreciated.

    0

  • What kind of container do you deploy there? An official jsreport image? Or something custom? Can you try the default and full jsreport images?

    Note we have a dedicated tutorial for using serverless google cloud functions here
    https://jsreport.net/learn/gcloud-functions-serverless

    0
  • N

    It is a custom container. After reviewing your example and reading further, I wonder if the problem is with writing files. Like Cloud Functions, Cloud Run seems only to want files to be written in /tmp/, as you are doing in your GitHub link. I have tried to implement what you have done in my codebase, but I still have the same issue.

    Cloud Run is similar to Cloud Functions except that it keeps the server warm and running and can spin up more of them if demand is high.

    Thoughts on how to run JSReport server in a more friendly way on Cloud Run?

    0
  • N

    I've looped back around to this, and I am trying to use the official container and modifying it to work with Google Cloud Run, which has similar rules as Cloud Function, such as needing to make disk changes in the /tmp/ folder.

    Here is my current dockerfile:

    FROM jsreport/jsreport:4.8.0-full

# Create directory with proper permissions
RUN mkdir -p /tmp/data/storage && \
    chown -R jsreport:jsreport /tmp/data && \
    chmod -R 755 /tmp/data

# Copy only the template data and configuration files
COPY ./reporting/data /tmp/data
COPY ./reporting/cloudrun.config.json /app/jsreport.config.json

# For Google Cloud Run - use the PORT environment variable
ENV PORT=8080
ENV httpPort=8080
EXPOSE 8080

    Here is the current custom-config file:

    {
    "httpPort": 5488,
    "trustUserCode": true,
    "store": {
        "provider": "fs"
    },
    "blobStorage": {
        "provider": "fs",
        "dataDirectory": "/tmp/storage"
    },
    "reportTimeout": 60000,
    "logger": {
        "file": {
            "silent": true
        },
        "error": {
            "silent": true
        }
    },
    "extensions": {
        "licensing": {
            "useSavedLicenseInfo": false
        },
        "authentication": {
            "enabled": false
        },
        "authorization": {
            "enabled": false
        },
        "cli": {
            "enabled": false
        },
        "express": {
            "enabled": true
        },
        "freeze": {
            "enabled": false
        },
        "fs-store": {
            "dataDirectory": "/tmp/data"
        },
        "import-export": {
            "enabled": false
        },
        "public-templates": {
            "enabled": false
        },
        "sample-template": {
            "enabled": false
        },
        "scheduling": {
            "enabled": false
        },
        "studio": {
            "enabled": true
        },
        "studio-theme-dark": {
            "enabled": false
        },
        "tags": {
            "enabled": false
        },
        "version-control": {
            "enabled": false
        },

        "base": { "enabled": false },
        "child-templates": { "enabled": false },
        "components": { "enabled": true },
        "data": { "enabled": true },
        "handlebars": { "enabled": true },
        "jsrender": { "enabled": false },
        "localization": { "enabled": false },
        "npm": { "enabled": false },
        "pdf-utils": { "enabled": false },
        "reports": { "enabled": false },
        "text": { "enabled": false },
        "assets": { "enabled": true },
        "browser-client": { "enabled": false },
        "chrome-pdf": { "enabled": true },
        "docx": { "enabled": false },
        "html-to-xlsx": { "enabled": false },
        "pptx": { "enabled": false },
        "scripts": { "enabled": false },
        "static-pdf": { "enabled": false },
        "xlsx": { "enabled": false }
    },
    "chrome": {
        "launchOptions": {
            "args": ["--no-sandbox"]
        }
    }
}

    However, on start up, even locally, I get:

    2025-04-17T12:50:34.714Z - info: fs store is loading data
2025-04-17T12:50:34.716680854Z 2025-04-17T12:50:34.716Z - error: Error occurred during reporter init: Error: EACCES: permission denied, open '/tmp/data/storage/config.json'
2025-04-17T12:50:34.717906560Z [Error: EACCES: permission denied, open '/tmp/data/storage/config.json'] {
2025-04-17T12:50:34.717925837Z   errno: -13,
2025-04-17T12:50:34.717929404Z   code: 'EACCES',
2025-04-17T12:50:34.717932209Z   syscall: 'open',
2025-04-17T12:50:34.717934734Z   path: '/tmp/data/storage/config.json'
2025-04-17T12:50:34.717937309Z }

    Do you have any suggestions on how to overcome this issue? I don't suspect that config.json file is there at all, it isn't when I copy the data folder.

    0
  • N

    Scrath that, I got it working with this:

    FROM jsreport/jsreport:4.8.0-full

# Create directory with proper permissions
USER root
RUN mkdir -p /tmp/data /tmp/storage

# Copy only the template data and configuration files
COPY ./reporting/data /tmp/data
RUN rm /tmp/data/fs.*
RUN rm /tmp/data/settings
RUN rm /tmp/data/profiles
RUN rm -R /tmp/data/storage/
COPY ./reporting/cloudrun.config.json /app/jsreport.config.json

# Set proper permissions for jsreport user
RUN chown -R jsreport:jsreport /tmp/data /tmp/storage && \
    chmod -R 755 /tmp/data /tmp/storage

# For Google Cloud Run - use the PORT environment variable
ENV NODE_ENV=production
ENV PORT=8080
ENV httpPort=8080
EXPOSE 8080
    0
  • N

    So even with the changes above, I still get Timeout occurred when waiting for the worker action 'init in Google Cloud Run.

    Any suggestions? Is the problem just that something is taking too long to initialize? I think I tracked the code down in JSReport's source, and it appears to be hardcoded to 15 seconds - is that accurate? Any way to increase it?

    0

  • The default is 30s and can be configured using config workers.initTimeout.
    Have you tried running on a better HW?
    I will try the google cloud run in the next days to see what is the problem.

    0
Log in to reply
 

Looks like your connection to jsreport forum was lost, please wait while we try to reconnect.