facebookresearch/mobile-vision

What it solves

It addresses the challenge of designing efficient convolutional neural networks (ConvNets) for mobile devices, where traditional metrics like FLOPs do not always correlate with actual on-device latency.

How it works

The project implements hardware-aware and platform-aware model design strategies. It uses Differentiable Neural Architecture Search (NAS) to create models (FBNet) and platform-aware model adaptation (ChamNet) to optimize network design for specific hardware. To facilitate this, it provides a CNN latency look-up table (LUT) based on the Caffe2 int8 inference engine (QNNPACK), allowing developers to estimate actual on-device latency without needing to run every model on a physical phone.

Who it’s for

It is designed for machine learning researchers and engineers who need to deploy high-performance computer vision models on mobile hardware with strict latency and power constraints.

Highlights

• Pre-trained models for FBNet and ChamNet available in both PyTorch and Caffe2 formats.
• Support for int8 quantized models to maximize mobile performance.
• A comprehensive latency look-up table for operator-level latency estimation on mobile phones.
• Models trained and evaluated on the ImageNet 1k dataset.