We are excited to announce LiteRT.js, a JavaScript binding of LiteRT for running AI directly in the web browser. By bringing the trusted on-device inference library LiteRT to the web, web developers can now run ML and AI models completely locally at maximum performance. This means improved user privacy, no server costs, and ultra-low latency for real-time experiences. For developers with existing .tflite models, LiteRT.js makes deployment to mobile and desktop web browsers smoother than ever and serves as a powerful evolution of TensorFlow.js for running .tflite models.
While previous web AI solutions like TensorFlow.js relied on less powerful JavaScript-based kernels, we now make our native, cross-platform runtime, with all its optimizations, directly available to web developers via WebAssembly. LiteRT.js enables impressive performance by running your .tflite models directly in the browser, leveraging LiteRT’s state-of-the-art hardware acceleration, including XNNPACK for CPU, ML Drift for GPU, and the upcoming WebNN for NPUs.
Our first release provides all the tools you need to get started, including the new LiteRT.js npm package and a collection of demos showing real-world implementation.
Unfortunately, your browser does not support playback of this video
Vector search directly in the browser, supported by LiteRT.js and EmbeddingGemma. Try it here.
Table of Contents
What benefits LiteRT.js offers for web developers
With LiteRT.js, web developers can integrate models written in JavaScript or TypeScript into their apps to handle complex tasks such as text generation, object recognition, and audio processing entirely on the client side. Because LiteRT.js shares a unified cross-platform stack with LiteRT, your web applications automatically benefit from the latest performance enhancements, quantization improvements, and hardware optimizations designed for Android, iOS, and desktop.
By leveraging LiteRT’s lower data flow and runtime, you get easy conversion of models from a variety of Python ML frameworks and native hardware acceleration across all major accelerators (CPU/GPU/NPU). To help you easily unlock these AI capabilities, here are the key highlights of LiteRT.js:
1.PyTorch conversion and tailored quantization
With LiteRT Torch, PyTorch models can be converted in a single step, making them immediately ready to use advanced browser-based hardware acceleration. Get started today by following the LiteRT Torch guide.
For further optimization, AI Edge Quantizer allows you to configure tailored quantization schemes across different model levels. This achieves significant size reductions and performance increases while maintaining the overall quality of the model. Explore the Quantize Colab to see this in action.
2.Native hardware acceleration for CPU, GPU and NPU
LiteRT.js enables powerful AI inference for a variety of hardware backends.
- CPU: uses XNNPACKGoogle’s highly optimized library for on-device CPU acceleration, offering robust multi-threaded support and a relaxed SIMD build for improved performance.
- GPU: powered by ML driftGoogle’s leading solution for on-device GPU acceleration. LiteRT.js uses WebGPU to enable state-of-the-art GPU acceleration on the web.
- NPU: uses emergence WebNN API (currently experimental in Chrome and Edge) to address dedicated NPUs for energy-efficient, ultra-low latency inference.
Are you ready to accelerate your web applications? First, dive into the LiteRT.js documentation.
Overview of LiteRT.js architecture
Performance and real-world impact
To demonstrate the real-world impact of the unified runtime and hardware-accelerated backends, we evaluated LiteRT.js against existing web solutions. Compared to classic computer vision and audio processing models, LiteRT.js delivers significant speed improvements, outperforming other web runtimes by up to 3x in both CPU and GPU inference.
Note: Performance benchmarks were conducted on a 2024 Apple MacBook Pro with M4 Apple Silicon in a controlled browser environment. Individual user performance may vary depending on local GPU capabilities, thermal throttling, and browser driver optimization.
To support these claims in real-world efficiency, we compared popular AI models using LiteRT.js across three different web execution backends: CPU (via XNNPACK), WebGPUAnd WebNN (via Apple CoreML). For demanding real-time applications such as object tracking, audio transcription, or image editing, using the GPU or NPU over WebGPU or WebNN results in a 5x to 60x speed increase compared to standard CPU execution, providing lower latency without sacrificing performance.
Note: Performance benchmarks were conducted on a 2024 Apple MacBook Pro with M4 Apple Silicon in a controlled browser environment. Individual user performance may vary depending on local GPU capabilities, thermal throttling, and browser driver optimization.
See it in action
To see LiteRT.js in action, explore our live implementations. LiteRT.js demo source code is available in the LiteRT GitHub repository and through Ultralytics.
LiteRT Ultralytics YOLO integration
Ultralytics is an artificial intelligence company specializing in the development of computer vision tools and models. It is best known as the creator of the YOLO (You Only Look Once) framework, a family of real-time object detection and image segmentation models.
We are pleased to introduce official LiteRT export support, built directly into the Ultralytics Python package. Easily deploy Ultralytics YOLO models to mobile, edge, and browsers – and go from compile to runtime in just a few lines of code.
Unfortunately, your browser does not support playback of this video
Demo: YOLO26, family of real-time vision models
Depth estimation
Depth Anything – monocular depth estimation shows how to convert a standard webcam feed into an interactive 3D point cloud in real time. Powered by LiteRT.js over WebGPU, it leverages the Depth-Anything V2 model to instantly calculate depth data and map video pixels into a responsive 3D space.
Unfortunately, your browser does not support playback of this video
Demo: Monocular depth estimation with DepthAnything and WebGPU.
Image upscaling
Upscale images 4x in the browser using the Real-ESRGAN model with LiteRT.js. This involves upscaling 128 x 128 pixel patches to 512 x 512, which are then reassembled to form the final image.
Unfortunately, your browser does not support playback of this video
Demo: An image of a dog is placed on an image upscaler website where it is upscaled to four times its size. Photo credit
Start with LiteRT.js
Integrating LiteRT.js into your development workflow is straightforward, whether you’re launching a new implementation or migrating an existing application to our high-performance runtime. LiteRT.js abstracts the complexity of optimization at the hardware level, allowing you to deliver responsive, privacy-focused experiences without the hassle of manual platform optimization.
The following snippet highlights the optimized process for initializing, compiling, and running a .tflite GPU acceleration model. Clean, modern JavaScript lets you load your model, feed input tensors, and capture high-speed inference results in real-time. For more detailed instructions, demos and guides, see our documentation here.
import { loadLiteRt, loadAndCompile, Tensor } from '@litertjs/core';
await loadLiteRt('path/to/wasm/directory/');
const model = await loadAndCompile('path/to/your/model.tflite',{ accelerator: webgpu });
const inputTypedArray = new Float32Array(1 * 3 * 244 * 244);
const inputTensor = new Tensor(inputTypedArray, [1, 3, 244, 244]);
const results = await model.run(inputTensor);
// results is a Tensor stored on GPU. To move it to CPU & convert to a typedArray we use
const resultArray = (await results[0].moveTo('wasm')).toTypedArray();JavaScript
What’s next?
We are committed to continually expanding LiteRT.js’ performance, model coverage, and developer tools. Looking forward, our development roadmap focuses on advancing WebNN integration for native NPU performance and providing highly optimized support for on-device generative AI.
Acknowledgments
Ultralytics for providing YOLO26 media and performance data. Jason Mayes for LiteRT.js demos.
https://developers.googleblog.com/litertjs-googles-high-performance-web-ai-inference/
