HuggingFace Transformer.js

Finally, I am able to talk about the huggingface transformer.js project. This project uses ONNX to make it possible to run HuggingFace models in the browser.

Just calling the API is boring, there is a react translation tutorial which is the best way to get started.

Download the model: in the model, they used a worker.js to download the model. the progress_callback is used to show the progress of the download.

this.instance ??= pipeline(this.task, this.model, { progress_callback });

The pipeline function is used to create a pipeline for the model. You can add parameter dtype: "fp32" for higher precision, but the file will be bigger.

device: "webgpu" is used to run the model on the GPU.

the progress_callback will return the progress event. The structure is like this.

{
    status: "initiate" | "progress" | "done" | "ready",
    data: {
        file: string, // file name
        progress: number, // progress percentage
        ...
    }
}

The following code shows how to stream the result. If you are using typescript, you may get errors, maybe the type definition is wrong.

// Listen for messages from the main thread
self.addEventListener('message', async (event) => {
  // Retrieve the translation pipeline. When called for the first time,
  // this will load the pipeline and save it for future use.
  const translator = await MyTranslationPipeline.getInstance(x => {
      // We also add a progress callback to the pipeline so that we can
      // track model loading.
      self.postMessage(x);
  });

  // Capture partial output as it streams from the pipeline
  const streamer = new TextStreamer(translator.tokenizer, {
      skip_prompt: true,
      skip_special_tokens: true,
      callback_function: function (text) {
          self.postMessage({
              status: 'update',
              output: text
          });
      }
  });

  // Actually perform the translation
  const output = await translator(event.data.text, {
      tgt_lang: event.data.tgt_lang,
      src_lang: event.data.src_lang,

      // Allows for partial output to be captured
      streamer,
  });

  // Send the output back to the main thread
  self.postMessage({
      status: 'complete',
      output,
  });
});

I can download the model and load locally.

import { env } from '@huggingface/transformers';
env.allowRemoteModels = false;
env.localModelPath = '/path/to/local/models/';

Most important is the pipelines. It is a list of pipelines that defines what it can do and what it cannot do.

• ImageToImagePipeline： enlarger an image
• DepthEstimationPipeline: make depth graph