Inconsistent Cipher Output Between Node.js and PHP When Reading Large Files by Chunks

Problem Description

I’m facing an issue with ensuring consistent cipher output between Node.js and PHP when reading large files by chunks. The output differs even though the same values are being read and processed. Below is the code for both PHP and Node.js implementations.

PHP Code

<?php
require 'vendor/autoload.php';

define('PLAINTEXT_DATA_KEY', 'poSENHhkGVG/4fEHvhRO6j9W3goETWZAg+ZgTWxhw34=');
define('ENCRYPTED_DATA_KEY', 'AQIDAHg4CVTLXnBsl/PpZjf+n1uhc03fcdvJT7Ytc8qauQHURgE9yj3RsilBWvaeXRwGWajRAAAAfjB8BgkqhkiG9w0BBwagbzBtAgEAMGgGCSqGSIb3DQEHATAeBglghkgBZQMEAS4wEQQMik8zaXe7Vw+JvKPrAgEQgDsyaXx8zP7xpmOEfMUuqD0VWhp0eAZZHXFWd6KymDhpEp1Y6+Mo8wdYBycT0m96VWw/56pWtXJ+Pngw9g==');
define('IV', "X1bIRjgIoDn/BDFhHIbg7g==");
define('ALGORITHM', 'aes-256-cbc');
define('CHUNK_SIZE', 16 * 1024);

function logMessage($message) {
    echo $message . PHP_EOL;
}
logMessage(CHUNK_SIZE . "n");

class Cipher {
    private function pkcs7_pad(string $data, int $blockSize): string {
        $padLength = $blockSize - (strlen($data) % $blockSize);
        return $data . str_repeat(chr($padLength), $padLength);
    }

    private function pkcs7_unpad(string $data): string {
        $padLength = ord($data[strlen($data) - 1]);
        return substr($data, 0, -$padLength);
    }

    public function encrypt($source, $dest, $calcMd5 = true, $debugMode = true) {
        $flag = true;
        $totalSize = 0;
        $md5Context = $calcMd5 ? hash_init('md5') : null;
        $md5ContextEnc = null;
        if ($debugMode && $flag) {
            $md5ContextEnc = hash_init('md5');
        }

        $initialBuffer = base64_decode(IV) . base64_decode(ENCRYPTED_DATA_KEY);
        $inputFile = fopen($source, 'rb');
        $outputFile = fopen($dest, 'wb');
        $outfileBase64 = null;
        if ($debugMode) {
            $outfileBase64 = fopen('outfiles/encryptedbuffer.log', 'wb');
        }

        try {
            fwrite($outputFile, $initialBuffer);

            while (!feof($inputFile)) {
                logMessage("-----");
                $buffer = fread($inputFile, CHUNK_SIZE);
                logMessage('Read plaintext Len: ' . strlen($buffer));
                $totalSize += strlen($buffer);
                if ($calcMd5 && $md5Context) {
                    hash_update($md5Context, $buffer);
                }
                if (feof($inputFile)) {
                    $buffer = $this->pkcs7_pad($buffer, 16);
                }
                $cipherText = openssl_encrypt($buffer, ALGORITHM, PLAINTEXT_DATA_KEY, OPENSSL_NO_PADDING, base64_decode(IV));
                if ($cipherText === false) {
                    unlink($dest);
                    throw new Exception("Error while encrypting the data.");
                }
                logMessage('cipher text len: ' . strlen($cipherText));

                if ($debugMode && $flag) {
                    hash_update($md5ContextEnc, $cipherText);
                }

                if ($flag) {
                    $md5HashEnc = hash_final($md5ContextEnc);
                    logMessage(("md5 of encrypted file: " . $md5HashEnc));
                    $flag = false;
                }
                $written = fwrite($outputFile, $cipherText);
                if ($debugMode) {
                    fwrite($outfileBase64, base64_encode($cipherText));
                    fwrite($outfileBase64, "n");
                    logMessage('base64 encode cipherText len ' . strlen(base64_encode($cipherText)));
                }

                if ($written !== strlen($cipherText)) {
                    unlink($dest);
                    throw new Exception("Error while writing the encrypted data to file");
                }
            }

        } catch (Exception $e) {
            throw $e;
        } finally {
            fclose($inputFile);
            fclose($outputFile);
        }
        $md5Hash = $calcMd5 ? hash_final($md5Context) : null;
        logMessage('MD5 hash of downloaded data: ' . $md5Hash);
        if ($debugMode && $flag) {
            $md5HashEnc = hash_final($md5ContextEnc);
            logMessage(("md5 of encrypted file: " . $md5HashEnc));
        }
        logMessage("Total size: " . $totalSize);
    }
}

$cipher = new Cipher();
$cipher->encrypt("infiles/a.txt", "outfiles/a.txt.enc");
?>

import { createCipheriv, createHash } from "crypto";
import { BASE64_ENCODING, DATA_EVENT, HEX, MD5 } from "./constants";
import { base64ToBuffer, base64ToUint8Array, bufferToBase64, ensurePaths } from "./common";
import { createReadStream, createWriteStream } from "fs";
import { log } from "console";

const PADDING_BLOCK_SIZE = 16;
const ALGORITHM = "aes-256-cbc";
const PLAINTEXT_DATA_KEY = "poSENHhkGVG/4fEHvhRO6j9W3goETWZAg+ZgTWxhw34=";
const ENCRYPTED_DATA_KEY =
  "AQIDAHg4CVTLXnBsl/PpZjf+n1uhc03fcdvJT7Ytc8qauQHURgE9yj3RsilBWvaeXRwGWajRAAAAfjB8BgkqhkiG9w0BBwagbzBtAgEAMGgGCSqGSIb3DQEHATAeBglghkgBZQMEAS4wEQQMik8zaXe7Vw+JvKPrAgEQgDsyaXx8zP7xpmOEfMUuqD0VWhp0eAZZHXFWd6KymDhpEp1Y6+Mo8wdYBycT0m96VWw/56pWtXJ+Pngw9g==";
const IV = "X1bIRjgIoDn/BDFhHIbg7g==";

const CHUNK_SIZE = 16 * 1024; 

class Cipher {
  private pkcs7Pad(buffer: Buffer, blockSize: number = PADDING_BLOCK_SIZE): Buffer {
    const padding = blockSize - (buffer.length % blockSize);
    const padBuffer = Buffer.alloc(padding, padding);
    return Buffer.concat([buffer, padBuffer]);
  }

  private pkcs7Unpad(buffer) {
    const padding = buffer[buffer.length - 1];
    return buffer.subarray(0, buffer.length - padding);
  }

  async encrypt(source: string, dest: string, calcMd5: boolean = true, debugMode: boolean = false) {
    let flag = true;
    const iv = base64ToBuffer(IV);
    let hash: any;
    let hashEnc: any;
    if (calcMd5) {
      hash = createHash(MD5).setEncoding(HEX);
    }
    if (debugMode && flag) hashEnc = createHash(MD5).setEncoding(HEX);

    const cipher = createCipheriv(ALGORITHM, base64ToUint8Array(PLAINTEXT_DATA_KEY), iv);
    cipher.setAutoPadding(false);

    await ensurePaths(dest);

    const readStream = createReadStream(source, { highWaterMark: CHUNK_SIZE });
    const writeStream = createWriteStream(dest, { highWaterMark: CHUNK_SIZE });

    let encryptedBase64WriteStream;
    if (debugMode) {
      await ensurePaths("outfiles/encrypt/encrypted-buffer.log");
      encryptedBase64WriteStream = createWriteStream("outfiles/encrypt/encrypted-buffer.log", { highWaterMark: CHUNK_SIZE });
    }

    const encryptedDataKeyBuffer = Buffer.from(ENCRYPTED_DATA_KEY, BASE64_ENCODING);
    const initialBuffer = Buffer.concat([iv, encryptedDataKeyBuffer]);
    writeStream.write(initialBuffer);

    let totalSize = 0;
    let tempChunkStorage = Buffer.alloc(0);
    readStream.on(DATA_EVENT, (chunk) => {
      totalSize += chunk.length;

      if (typeof chunk === "string") {
        chunk = Buffer.from(chunk);
      }

      tempChunkStorage = Buffer.concat([tempChunkStorage, chunk]);

      while (tempChunkStorage.length >= CHUNK_SIZE) {
        const block = tempChunkStorage.subarray(0, CHUNK_SIZE);
        log(`Read plaintext Len: ${block.length}`);
        writeStream.write(block);

        if (calcMd5) hash.update(block);

        const encryptedBuffer = cipher.update(block);
        console.log(`cipher text len: ${encryptedBuffer.length}`);
        if (debugMode) encryptedBase64WriteStream.write(bufferToBase64(encryptedBuffer) + "n");
        if (debugMode && flag) hashEnc.update(encryptedBuffer);

        console.log(`base64 encode cipherText len: ${bufferToBase64(encryptedBuffer).length}`);

        if (flag) {
          const finalHashEnc = hashEnc.digest("hex");
          log(`MD5 hash of encrypted data: ${finalHashEnc}`);
          flag = false;
        }
        // process.exit();

        tempChunkStorage = tempChunkStorage.subarray(CHUNK_SIZE);
        log(`Leftover data size: ${tempChunkStorage.length}`);
      }
    });
    readStream.on("end", () => {
      if (tempChunkStorage.length > 0) {
        log(`Writing remaining data of size: ${tempChunkStorage.length}`);
        writeStream.write(tempChunkStorage);
        if (calcMd5) hash.update(tempChunkStorage);

        const encryptedBuffer = cipher.update(this.pkcs7Pad(tempChunkStorage));
        console.log(`cipher text len: ${encryptedBuffer.length}`);

        if (debugMode) encryptedBase64WriteStream.write(bufferToBase64(encryptedBuffer) + "n");

        if (debugMode && flag) hashEnc.update(encryptedBuffer);

        cipher.final();
      }

      writeStream.end();
      if (calcMd5) {
        const finalHash = hash.digest("hex");
        log(`MD5 hash of downloaded data: ${finalHash}`);
      }
      if (debugMode && flag) {
        const finalHashEnc = hashEnc.digest("hex");
        log(`MD5 hash of encrypted data: ${finalHashEnc}`);
      }

      log("Finished downloading.");
      log("Total size: " + totalSize);
    });
    readStream.on("error", (err) => {
      log("Error downloading: " + err);
      writeStream.close();
    });
  }
}

const test = async () => {
  const cipher = new Cipher();
  await cipher.encrypt("outfiles/rawinputfiles/a.txt", "outfiles/encrypt/a.txt.enc", true, true);
};
test();

First few bytes of encrypted data (base64)

PHP: 2SV3m3J4+vp+ZAX/LBrKjfKVzayHCPjaVbP2ws6hYvVtFW6xdxm8FwjhTA5JsNMYyjXdRggFXpCIjKJb2FX19h5NQnEm2hbVTJEhK1iakzeGpQTR2JpQFXX3gw8++/2XFdBP3MdKGd4mRqzDN4F3XFvkO7DXici5H6ND0R223MxhCht1ICnsQFdZudbZhWd2oyHB04gPrkmts+MdoMTZ2biSdrui3nROSuXW41oraQijwwLMMSMAEAT8ZszxIplL3yYzQdP1X8c11tgzRsB3pBuBbFqhuPYi6qD6ws5un0RC1Uo2ZW03GJ8t3uhgBnewd02wnnpNifknviI5m6enZj1kHN9Eq/GZqt9tvdTL0s0+vg2Z5dUOpqi/

NodeJS:

tFV7iqTTCwsFoQAGmOpWPZOqE0yqDRTJHzBKs6eKafHfsxeZBqIBubz9ZiLd06trzmsYBnWT46B+Jxx+kio17dKYqyYQ61eGOKWs3hHStobi5854PSbx97acyCzCQmuQ8Nif3bps5GRAkJl77O6OzswxlD8YEvAByAqAI40fq+lMPASu41dvWOsFP0TOak36EKDoUGRhNeDmItcgAXKydrDjbyz5LEi

Text File:
text