AWS Lambda with S3 and SNS synchronization, race condition?

cachenbach

Hello,

this is our simplified config:

store.sync.name = aws-sns
extensions.fs-store.sync = aws-sns
extensions.fs-store.persistence = aws-s3

When we change a template in out local instance of jsReport, we can see a message in the SQS queue of the lambda waiting. When the lambda is started it will fetch the message, but it will use the old version of the template. Only the second call of the lambda takes the changes into account. I think jsReport in a Lambda renders first and processes the messages from SQS afterwards.
Can the behavior be changed?

Is it even a good idea to use SNS with SQS in a Lambda? I think every cold start of the Lambda will generate a new SQS queue and a new SNS subscription. How can we synchronize without SNS?

Thanks!
Christian

jan_blaha

Strange. The scenario you use should work as follow.

You edit a template on local which causes an s3 update and a sns mesage.
The lambda starts and loads the fresh s3 templates already with your change.
The sns message arrives but it is ignored because it is older than the load from s3.

I would probably prepare lambda package already with the templates in the package.
Or use a mongo to fetch them.
However s3 driver should still work.

cachenbach

At first we tried it without SNS (only S3) and we expected the behavior you described. But only if the Lambda made a cold start it uses the newest version of the template from S3. When the lambda is warm, then it continues to use the version from the first start. Is there any kind of caching we can disable?

Every start of the lambda leaves more files in "/tmp":
/tmp/core.headless-chromi.57
/tmp/core.headless-chromi.93
/tmp/headless-chromium
/tmp/jsreport/autocleanup/c67d9130-978d-411a-a740-42c2e40ea34b-chrome-pdf.html
/tmp/jsreport/autocleanup/ecbbb725-a1ab-4580-a7c7-837a712b31d4-chrome-pdf.html
/tmp/jsreport/core/locations.json

Perhaps if we delete the right one we can force him the fetch the newest version from S3?

Thanks for your help!

jan_blaha

You call this before every render. It would reload templates from s3. However not sure about the performane in the end.
You can check it out and decide if it is good enough for you

await jsreport.documentStore.provider._load()

cachenbach

Yes, thanks! Perhaps not the perfect solution, but it works.
We keep an eye on execution times of the lambda.

Thanks again!