I need to create embeddings in NextJS 15, so I started looking into @xenova/transformers and @huggingface/transformers. It was too simple to implement in a Node script (without NextJS), but I am struggling to make this work.
After the project creation, I created an API route as usual (/app/api/route.ts
) and proceeded with the given example:
import { pipeline } from '@huggingface/transformers'
export async function GET() {
const extractor = await pipeline(
'feature-extraction',
'Xenova/all-MiniLM-L6-v2'
)
const sentences = ['This is an example sentence', 'Each sentence is converted'];
const output = await extractor(sentences, { pooling: 'mean', normalize: true });
console.log(output)
}
But when I visit the API URL I get this error:
./node_modules/@huggingface/transformers/node_modules/onnxruntime-node/bin/napi-v3/darwin/arm64/onnxruntime_binding.node
Module parse failed: Unexpected character ‘�’ (1:0)
You may need an appropriate loader to handle this file type, currently no loaders are configured to process this file. See https://webpack.js.org/concepts#loaders
(Source code omitted for this binary file)
I have tried also to update the Next configuration file next.config.ts
with:
import type { NextConfig } from "next";
const nextConfig: NextConfig = {
output: 'standalone',
serverExternalPackages: ['sharp', 'onnxruntime-node']
};
export default nextConfig;
I don’t really know how to proceed and implement this in the server side (API routes). If I need a loader, I don’t know how to implement it.