Google ML Drift Accelerates On-Device AI Inference Across Diverse GPUs
Google open-sourced ML Drift under Apache 2.0, delivering unified GPU inference across diverse edge hardware with a 12% memory reduction on Gemma models.
By Dillip Chowdary • Oct 25, 2026 • Source: InfoQ

At a glance
- Product
- ML Drift, open-source cross-platform GPU compute engine
- License
- Apache 2.0
- Memory overhead
- 12% lower than llama.cpp and MLX LM (Gemma)
- Shorts latency
- Up to 40% reduction in average frame latency
- Adobe speedup
- Up to 30% in Photoshop and Lightroom
- Deprecated
- Legacy TensorFlow Lite GPU delegate
In this post
Google released ML Drift, an open-source cross-platform GPU compute engine distributed under the Apache 2.0 license to accelerate on-device artificial intelligence and machine learning inference across diverse edge hardware, according to InfoQ's report. The software abstracts low-level graphics programming interfaces including OpenGL ES, OpenCL, Metal, and WebGPU, operating as the core GPU acceleration engine that powers LiteRT. Google open-sourced the underlying technology to let engineers construct custom graphics and inference runtimes for traditional computer vision, audio, depth processing, and generative AI workloads without writing device-specific shader code for every target.
This technical breakdown examines the architecture of ML Drift, the migration path away from legacy TensorFlow Lite components, and benchmark results across mobile and desktop production apps. The guide is written for mobile software engineers, AI system architects, and graphics developers who run spatial, vision, and transformer models locally on client hardware. It details memory allocation improvements, operating system backends across Android, iOS, Windows, Linux, and macOS, and performance metrics reported across Google, Adobe, and Snapchat deployments.
01What shipped
Google packaged ML Drift with multi-platform backend support designed to bridge mobile and desktop environments under a shared abstraction layer. The runtime targets mobile graphics APIs such as OpenGL ES and OpenCL on Android, while using a native Metal backend on macOS.
For desktop deployments on Windows and Linux, the engine routes GPU operations through Dawn, Chromium's WebGPU implementation, avoiding direct calls to platform-specific graphics APIs like Vulkan or DirectX. This unified abstraction lets developers target diverse mobile and workstation chipsets from Arm, Intel, and Qualcomm Technologies using consistent shader pipelines.
The release ships as an Apache 2.0 open-source project containing both a high-level runtime bridge and a low-level C++ programming interface. Application developers can access the system through the LiteRT ML Drift GPU accelerator, while systems and graphics engineers can use the low-level C++ API directly to construct custom GpuModel compute graphs and write specialized compute shaders. Google confirmed that the engine is already deployed at scale, running inside production consumer applications including Google Chrome, YouTube Shorts, Google Photos, and Google Meet across millions of active devices.
02What changed for builders
Architecturally, the software introduces unified shaders through tensor virtualization, decoupling logical model tensors from physical GPU memory allocations to prevent runtime fragmentation. It adds native support for five-dimensional tensors without requiring custom memory layout hacks, enabling edge GPUs to execute transformer architectures and multidimensional spatial models directly. The engine also incorporates extensible custom operations alongside dedicated micro-architectural optimizations tuned for Qualcomm Technologies, Intel, and Arm hardware pipelines to maximize compute throughput.
| Metric or Feature | Legacy TFLite GPU Delegate | ML Drift Engine |
|---|---|---|
| Tensor Virtualization | No | Yes (Unified Shaders) |
| Native 5D Tensor Support | Layout hacks required | Native without hacks |
| Gemma Memory Overhead | Baseline | 12% lower overhead |
| YouTube Shorts Latency | Baseline | Up to 40% reduction |
| Snapchat Model Latency | Baseline | 30% reduction |
| Adobe Feature Gains | Baseline | Up to 30% speedup |
| Google Photos Speedup | Baseline | Up to 2 seconds faster |
Benchmarking conducted on developer workstation hardware with Gemma models showed a 12% lower memory overhead compared to local execution engines llama.cpp and MLX LM. Production applications show measured reductions in model execution time, such as YouTube Shorts achieving up to a 40% decrease in average frame latency for segmentation effects on iOS and Android. Adobe reported performance gains reaching 30% in Photoshop and Lightroom for Select Subject, Select Sky, and Adaptive Portrait features, while Snapchat reduced face and style-generation model latency by 30% on Android, allowing the deployment of larger diffusion models.
03How to install or upgrade
Advertisement
Tech Pulse Daily
Get tomorrow's pulse first
Join engineers who read Tech Pulse before stand-up. Free, weekday mornings.
Teams adopting the engine must migrate from the deprecated TensorFlow Lite GPU delegate to LiteRT GPU acceleration components. Mobile and edge developers integrating through standard application pipelines update their project dependencies to consume the LiteRT ML Drift GPU accelerator package. For command-line model evaluation, automated workflows, and environment orchestration, developers update their workspace tooling and configure model execution runtimes through the standard environment toolchain:
npm install -g @google/litert-toolslitert-compile --model gemma-2b --accelerator ml-driftlitert-runner --model gemma-2b.litert --accelerator ml-drift --gpu_backend autoexport LITERT_DEFAULT_ACCELERATOR=ml_driftGPU engineers developing bespoke compute pipelines can bypass the standard model runner and compile directly against the ML Drift C++ library. This low-level surface provides programmatic control over custom GpuModel graph nodes, direct shader binding, and hardware buffer passing across OpenGL ES, OpenCL, Metal, and Dawn WebGPU contexts. Google advises builders with existing TFLite models to swap out the legacy delegate runtime initialization calls for the LiteRT ML Drift delegate builder to preserve model compatibility while unlocking the updated memory scheduler.
04Gotchas and compatibility
A critical transition requirement is Google's deprecation of the legacy TensorFlow Lite GPU delegate, which will receive no further feature updates following the release of ML Drift. Developers maintaining legacy production codebases must plan transitions to the LiteRT ML Drift accelerator to receive driver compatibility updates and performance patches. While Google highlighted lower memory footprints against llama.cpp and MLX LM on desktop hardware, the company did not provide direct latency or memory benchmarks comparing ML Drift against Apple's native Core ML framework on recent iPhone hardware.
Compatibility on Windows and Linux systems depends on Dawn and its underlying WebGPU translation layer, which abstracts Vulkan and DirectX rather than exposing raw hardware queues directly. Graphics developers working with non-standard tensor ranks or custom operations must verify their operator graphs against the newly supported 5D tensor specification. Decoupling logical tensors from physical memory eliminates allocations during live inference, but teams targeting older mobile GPUs must test whether their target driver sets reliably handle the unified shader translations generated across OpenGL ES and OpenCL backends.
05What to watch next
The release signals Google's consolidation of edge machine learning acceleration around the LiteRT branding, creating a clear break from older TensorFlow Lite delegate paths. Because Google Photos gained up to two seconds in computational photography and segmentation pipelines, and Snapchat unlocked higher parameter diffusion models through its 30% latency reduction, third-party developers will likely push heavier generative AI architectures directly onto mobile client silicon. Observers are tracking whether Google will expand benchmarking data to cover Apple silicon head-to-head against Core ML across iOS devices.
Engineering teams should monitor upstream source contributions within the Apache 2.0 repository to track emerging operator kernels and vendor-specific tuning patches. As Arm, Intel, and Qualcomm Technologies release new neural and graphics processing architectures, ML Drift's unified shader generator will serve as the primary testbed for on-device inference efficiency. Developers should watch future LiteRT SDK point releases for automated graph converters that simplify transitioning historical flatbuffer models into optimized ML Drift runtime graphs.
Frequently asked questions
What is Google ML Drift?
ML Drift is an open-source, cross-platform GPU compute engine licensed under Apache 2.0 that accelerates on-device AI and machine learning inference across mobile and desktop GPUs.
Which platforms and graphics APIs does ML Drift support?
It supports OpenGL ES and OpenCL on Android, native Metal on macOS, and desktop Windows and Linux through Dawn, Chromium's WebGPU implementation.
How does ML Drift differ from the legacy TensorFlow Lite GPU delegate?
ML Drift introduces tensor virtualization for unified shaders, native 5D tensor support without layout workarounds, and replaces the deprecated TFLite delegate with lower memory overhead.
What performance improvements does ML Drift deliver in production applications?
It reduces YouTube Shorts effect latency by up to 40%, cuts Snapchat model latency by 30%, speeds up Adobe photo tools by up to 30%, and lowers Gemma model memory overhead by 12% versus llama.cpp and MLX LM.
Sources
Author
Dillip Chowdary
Writes Tech Bytes coverage of AI, engineering, and the tools that actually ship. Editor of Tech Pulse Daily.
Related on Tech Bytes
Advertisement