3. Increase rendering time in same short template on Windows and got 0kb when receive the ETIMEDOUT error

Increase rendering time in same short template on Windows and got 0kb when receive the ETIMEDOUT error

  • M

    Hi, let me describe the background of it: the jsreport nodejs application (node: v8.14.0; jsreport: 2.4.0) is running in a standalone Windows server and in server.js we just simply call jsreport.init without coding the render logic by ourselves.

    Back in the day before 19/11/2024, the rendering worked fine, but after client applied Windows security patch on that day, it took 20mins to render the templates and some of them are 0kb, we had no choice but to bypass it by end the task once a day first. And there was a second security patch on 3/3/2025, which made the issue more serious: the end task/restart solution only works for ~20 mins and it still took a long time to render short template. We thought it was something related to the Windows security patches because none of us have modified the jsreport server and its application, so we suggested the client to roll back the patches.

    However, even though they have rollbacked the patches of 19/11/2024 and 3/3/2025, the problem still exists: we were downloading the same template with same data size one by one, it takes longer time to finish the render and after the restart for around 25mins/downloaded the fourth template (I am not sure whether the time or the amount of the render matters), we can see the error log of "2025-03-08T15:06:08.261Z - error: Error when processing render request connect ETIMEDOUT 68.219.241.39:80 Error: connect ETIMEDOUT 68.219.241.39:80
    at TCPConnectWrap.afterConnect [as oncomplete] (net.js:1191:14)
    2025-03-08T15:06:08.261Z - error: Error during processing request at https://10.40.164.13:8888/api/report/ats_asset_export" and then we will receive a 0kb template.

    The cpu is in normal usage and I am not sure what is going on here, have anyone experience this kind of issue?

    The following is the content of jsreport.config.json:

    {
  "extensions": {
  "express": {
    "inputRequestLimit":"500mb" } ,   "authentication": {
      "cookieSession": {
        "secret": "xxxxxx"
      },
      "admin": {
        "username": "xxxxxx",
        "password": "xxxxxx"
      },
      "enabled": false
    },
    "scripts": {
      "timeout": 4000000,
      "strategy": "http-server"
    }
  },
  "httpPort": 8888,
  "store": {
    "provider": "fs"
  },
  "blobStorage": {
    "provider": "fs"
  },
  "logger": {
    "console": {
      "transport": "console",
      "level": "debug"
    },
    "file": {
      "transport": "file",
      "level": "info",
      "filename": "logs/reporter.log"
    },
    "error": {
      "transport": "file",
      "level": "error",
      "filename": "logs/error.log"
    }
  },
  "allowLocalFilesAccess": true,
  "templatingEngines": {
    "timeout": 1000000,
    "inputRequestLimit":"500mb",  "strategy": "http-server"
  },
  "chrome": {
    "timeout": 4000000
  },
  "phantom":{"strategy":"dedicated-process"},"tasks":{"strategy":"dedicated-process"}
}

    and the logs which can see it takes longer time to render the next one of the same template with same data size:
    2025-03-08T14:27:20.790Z - info: Starting rendering request 1 (user: null)
    2025-03-08T14:27:26.796Z - info: Rendering template { name: ats_disposal_application_form, recipe: xlsx, engine: handlebars, preview: false }
    2025-03-08T14:28:14.184Z - info: Rendering request 1 finished in 53395 ms
    2025-03-08T14:29:57.315Z - info: Starting rendering request 2 (user: null)
    2025-03-08T14:30:17.825Z - info: Rendering template { name: ats_disposal_application_form, recipe: xlsx, engine: handlebars, preview: false }
    2025-03-08T14:33:54.047Z - info: Rendering request 2 finished in 236732 ms
    2025-03-08T14:34:10.772Z - info: Starting rendering request 3 (user: null)
    2025-03-08T14:34:49.672Z - info: Rendering template { name: ats_disposal_application_form, recipe: xlsx, engine: handlebars, preview: false }
    2025-03-08T14:37:08.046Z - info: fs store is loading data
    2025-03-08T14:38:25.679Z - info: Starting rendering request 4 (user: null)
    2025-03-08T14:39:41.065Z - info: Rendering template { name: ats_asset_export, recipe: xlsx, engine: handlebars, preview: true }
    2025-03-08T14:41:56.251Z - info: Rendering request 3 finished in 465479 ms

    I can raise some strange behaviors in my observation: 1) only production env has issues, non-production which has the same security patches are working fine. 2) I saw the logs of "2025-03-08T14:27:07.581Z - info: License info stored in jsreport.license.json is no longer valid, processing remote license verification
    2025-03-08T14:27:11.149Z - info: The licensing server was not reachable during instance startup. The instance now runs in the enterprise mode.
    2025-03-08T14:27:11.149Z - info: Verifying license key free
    2025-03-08T14:27:11.975Z - info: License key being verified and license key stored in jsreport.license.json doesn't match, processing remote license verification
    2025-03-08T14:27:15.561Z - info: The licensing server was not reachable during instance startup. The instance now runs in the enterprise mode.", but not sure whether it is the root cause of the issue. 3) it takes really long to go to 127.0.0.1:8888, and when I clicked "debug" button, we will receive ETIMEOUT 68.219.241.39:90 logs in both production and non-production environment

    0

  • Could you tell me what is the ip/port mentioned in the error? 68.219.241.39:80

    Could you try to change the configs extensions.scripts.strategy="dedicated-process" and templatingEngines.strategy="dedicated-process"

    0
  • M

    I think the ip/port(68.219.241.39:90) in the error is pointing to jsreport, since there is no server with this ip set up in the environment and I have searched it online "jsreport 68.219.241.39", it showed up "jsreport forum"

    and for your suggestion, I will try to change it with my client. Thank you.

    0

  • Ah ok, this is the IP address where jsreport uploads the office files for preview. However this is happening only when you run the preview request and display the output in the office live browser's app. jsreport isn't reaching this ip in other cases. Also it never goes to the port 90 where nothing is running. So that is very strange.

    I can imagine that the security patch breaks some kind of networking. However maybe there is a jsreport bug that you just didn't notice. jsreport 2.4.0 is 6years old. Perhaps you can update jsreport. At least to jsreport 2.11.0 which shouldn't have breaking changes.

    0
  • M

    Thank you for your advice.
    I am scheduling some solution includes upgrading the version of jsreport with my client, and I will update the result whether we can fix the problem and how did we fix it here.
    Hope everything is going smooth :)

    0
  • M

    Hi, I am here to update the status now.

    At first, we were planning to clone the jsreport server from UAT which has no problem with all Windows security patches into production, but due to expired license with Windows 2012, the client failed to do so. Fortunately, we figured out there was a snapshot before the second security patch on 3/3/2025, so they just applied the snapshot in production jsreport server and then I helped them to upgrade jsreport version into 2.11.0.

    After that, they have been observed within these three days, and it turns out they don't need to restart the jsreport application once a day to bypass the rendering issue left by the security patch on 17/11/2024, maybe it is fixed by the upgraded jsreport version, we are both happy to kind of solve the problem now.

    However, since the client still need to know whether they can apply the security patch in the future, they would like to understand how the patch affected jsreport application and why the problem only occurs in production environment.

    For our IT team, it is hard to tell what happen because we do not release any new changes for a long time, and we could not even simulate it in non-production environment.

    Do you guys have any insights on this problem? Could they apply the new security patch with the upgraded jsreport version of 2.11.0?

    0

  • Unfortunately, I have even no theory why windows security patch breaks things.

    0
  • M

    I understand and had anticipated that as well. Nevertheless, I appreciate your assistance. Thank you for your response.

    0
  • M

    Since clients need to know what the exact root cause in this case is, and they doubt that it does not fully relate to the broken networking caused by the security patches, otherwise jsreport should not receive any requests in the connection failure, so there may have some defeats in jsreport.

    We could provide the information of the patches for testing, please feedback with some insights that what may possibly affect jsreport. Thank you.

    CVE-2024-43572 - Microsoft Management Console Remote Code Execution Vulnerability
    CVE-2024-43573 - Windows MSHTML Platform Spoofing Vulnerability
    CVE-2024-43583 - Winlogon Elevation of Privilege Vulnerability
    CVE-2024-20659 - Windows Hyper-V Security Feature Bypass Vulnerability
    CVE-2024-43468 - Microsoft Configuration Manager Remote Code Execution Vulnerability
    Microsoft's October 2024 Patch Tuesday, which includes security updates for 118 flaws, including five publicly disclosed zero-days, two of which are actively exploited.

    This Patch Tuesday fixed three critical vulnerabilities, all remote code execution flaws.

    The number of bugs in each vulnerability category is listed below:
    • 28 Elevation of Privilege vulnerabilities
    • 7 Security Feature Bypass vulnerabilities
    • 43 Remote Code Execution vulnerabilities
    • 6 Information Disclosure vulnerabilities
    • 26 Denial of Service vulnerabilities
    • 7 Spoofing vulnerabilities

    CVE-2025-21333, CVE-2025-21334, CVE-2025-21335 - Windows Hyper-V NT Kernel Integration VSP Elevation of Privilege Vulnerability
    CVE-2025-21298 - Windows OLE Remote Code Execution Vulnerability
    CVE-2025-21354, CVE-2025-21362 - Microsoft Excel Remote Code Execution Vulnerability
    CVE-2025-21309, CVE-2025-21297 - Windows Remote Desktop Services Remote Code Execution Vulnerability
    CVE-2025-21296 - BranchCache Remote Code Execution Vulnerability
    CVE-2025-21365 – Microsoft Office Remote Code Execution Vulnerability

    Today is Microsoft's January 2025 Patch Tuesday, which includes security updates for 159 flaws, including eight zero-day vulnerabilities, with three actively exploited in attacks.

    This Patch Tuesday also fixes twelve "Critical" vulnerabilities, including information disclosure, privileges elevation, and remote code execution flaws.

    The number of bugs in each vulnerability category is listed below:

    • 40 Elevation of Privilege Vulnerabilities
    • 14 Security Feature Bypass Vulnerabilities
    • 58 Remote Code Execution Vulnerabilities
    • 24 Information Disclosure Vulnerabilities
    • 20 Denial of Service Vulnerabilities
    • 5 Spoofing Vulnerabilities

    0
Log in to reply
 

Looks like your connection to jsreport forum was lost, please wait while we try to reconnect.