Repository navigation
Allow to choose the PREFERRED_ENCODING order #220
Description
Activity
same here. We added a header in the client to force gzip as the compression mode
Accept-Encoding: gzipI'm not sure if we want to add that option, and the issue is that we rely on the
Accept-Encodingheader, which tells us which encodings are accepted. Since Brotli is listed in that header, we assume that Brotli can be used to compress the request.Another option would be to allow passing
falsein the configuration to indicate that we don't want Brotli to be available. cc: @UlisesGasconvar compression = require('compression') var express = require('express') var app = express() // compress all responses app.use(compression({ brotli: false }))
Reacted by Alex ANNOUNEI'm not sure if we want to add that option, and the issue is that we rely on the
Accept-Encodingheader, which tells us which encodings are accepted. Since Brotli is listed in that header, we assume that Brotli can be used to compress the request.Another option would be to allow passing
falsein the configuration to indicate that we don't want Brotli to be available. cc: @UlisesGasconvar compression = require('compression')
var express = require('express')var app = express()
// compress all responses
app.use(compression({ brotli: false }))That is also a solution : Disabling brotli would set the preferred compression to Gzip.
@victorsferreira it could be a solution if you have hand on the clients. In my case, unfortunately i do not ..
to fix that issue i had to make an extra middleware that filter the Accept-Encoding headers before passing the request to the Compression middleware@victorsferreira @aannoune is there any error when compressing the response?
In our case
compressionis a dependency of another package and we do not control it.We added the Header in the API gateway layer asking for a Gzip, because something in the middle wasn't dealing well with the BR response.
In a second application we added [email protected] as dependency and NPM used it (since it is downgraded) for the other package.
I still don't really understand what their problems are, whether it's an issue with the package or because their clients are sending the header with Brotli support when they actually don't support it for decompression. More information would be very helpful.
@bjohansebas we aren't 100% sure what is causing the issue on our application, but it's either the API gateway that is not understanding the response compressed with BR or the WAF that is filtering the packages compressed with BR.
either way it's causing a major problem in our application as the frontend is receiving a binary instead of the JSON response (still compressed).
I believe it's a major change in terms of architecture and infrastructure and I recommend reverting this priority order to what it previously was. Some people will need to fix their applications with urgency when this package automatically updates after a deploy.
Reacted by Alex ANNOUNE@bjohansebas : i detected the issue with postman: it does accept brotli encoding , but API gateway returns a binary stream.
User's Browser will send brotli in Accept-Encoding as default.. but api that are behind AWS API gateway with this library will return bad responses
I don't have much idea of how AWS infrastructure works, but after reading a bit, I understood it better.
Since changing the order of preference at this moment is a breaking change, we could add that option, but you would have to verify that it actually works. As some have mentioned, not everyone has full control over this package to manage the preference, so I'm not sure if it makes sense.
PRs are welcome
@bjohansebas ,I'll be glad to test the change anyway
After reflexion, an option to disable a compression type might appear as a more understandable option to most of the usersAt Circular Now, we run on GCP CloudRun with microservices.
After automatic package updates updated this library, we started seeing very strange timeout behaviour in our app on certain microservices. A request that previously averaged 5s would now sometimes take 30+ seconds (not consistently)
Investigating with Datadog traces, it would show with low precision that vaguely something was using CPU, and this seemed to happen after a network request promise was resolved. So we'd do an outbound network request, then a long period of time used CPU. I assume this was event loop delay.
The issue would only happen on microservices where the client had included br in the accept-encoding header. Certain other endpoints were requested without br in the accept header, were not impacted.
When we rolled back the compression library to a version without brotli, the issue went away.
This was using Node 22 on alpine linux docker
Reacted by Raivo Laanemets
Hello ,
Since v1.8.0, the preferred compression method is brotli.
I'm using the Compression module in an aws lambda behind Api Gateway but brotli compression is not handled by it.
As most of the clients are now passing
gzip, deflate, bras defaultAccept-Encodingheader this cause decompression errors.It would be nice to have an option that allows to change the order of the PREFERRED_ENCODING array to be able to have gzip as preferred compression method.