I have a file in s3 that is compressed using lz4 (or bz2) compression and served from an API endpoint as an application/octet-stream. The API fetches the compressed file from S3 and serves it to the frontend where I want to decompress it. The trouble I’m having is with converting the string to a Buffer because I believe the lz4 compressed file contains non-utf8 characters and so I’m unable to create a Uint8Array or a Buffer from it without converting non-utf8 characters to U+FFFD.
Some of the errors I’m seeing:
Error: Invalid magic number
Error: Invalid stream descriptor checksum
RangeError: invalid array length
Does anyone know how to get around this? Is there something simple I’m missing? Thanks in advance!