jsreport became unresponsive for PDF requests
-
We faced an unusual issue. A report that generally takes around 30 seconds to generate did not generate. We did not see any 'Rendering request finished' log in the console. And all the subsequent requests started piling up in the queue.
Then, once hitting the reportTimeout, all the reports started throwing 'chrome pdf generation timed out'.We had to restart the Pod. That pod was working fine for 15 days and processed around 6000 requests, but yesterday we started facing this issue.
Any idea what could be the reason here?
-
That is very hard to tell. It could be a Chromium leaking bug causing it to become stuck.
Have you checked the memory consumption? Isn't Chromium leaking and growing the memory endlessly?
What if you try to loop and render those 6000 requests locally? Does it show the same behavior?
Check the /tmp and /dev/shm usage, isn't it getting full?Perhaps setting config
chrome.strategy=dedicated-processcould help you.
This will ensure you have a fresh new instance of Chromium every time you render the request.
-
We have two pods running in production, each with one worker. Requests are distributed across these two instances. Each instance had processed approximately 6,000 requests; however, one instance became stuck while processing a PDF report, and we had to restart that pod. The report that caused the issue was generated successfully later without any problems.
We did not notice any memory or CPU spike during the time it was stuck.Will
chrome.strategy=dedicated-processcause more CPU and slowness compared tochrome-pool?