@ -10,8 +10,11 @@ This guide covers exporting and deploying Ultralytics YOLOv8 models to Raspberry
Deploying computer vision models on devices with limited computational power, such as [Raspberry Pi AI Camera](https://www.raspberrypi.com/products/ai-camera/), can be tricky. Using a model format optimized for faster performance makes a huge difference.
The IMX500 model format is designed to use minimal power while delivering fast performance for neural networks.
It allows you to optimize your [Ultralytics YOLOv8](https://github.com/ultralytics/ultralytics) models for high-speed and low-power inferencing. In this guide, we'll walk you through exporting and deploying your models to the IMX500 format, making it easier for your models to perform well on various edge devices.
The IMX500 model format is designed to use minimal power while delivering fast performance for neural networks. It allows you to optimize your [Ultralytics YOLOv8](https://github.com/ultralytics/ultralytics) models for high-speed and low-power inferencing. In this guide, we'll walk you through exporting and deploying your models to the IMX500 format while making it easier for your models to perform well on the [Raspberry Pi AI Camera](https://www.raspberrypi.com/products/ai-camera/).
<palign="center">
<imgwidth="100%"src=""alt="Placeholder for Raspberry Pi AI Camera image">
</p>
## Why Should You Export to IMX500
@ -32,54 +35,53 @@ The IMX500 works with quantized models. Quantization makes models smaller and fa
**Before You Begin:** For best results, ensure your YOLOv8 model is well-prepared for export by following our [Model Training Guide](https://docs.ultralytics.com/modes/train/), [Data Preparation Guide](https://docs.ultralytics.com/datasets/), and [Hyperparameter Tuning Guide](https://docs.ultralytics.com/guides/hyperparameter-tuning/).
### Introduction to Sony Model Compression Toolkit (MCT)
[Sony's Model Compression Toolkit (MCT)](https://github.com/sony/model_optimization) is a powerful tool for optimizing deep learning models through quantization and pruning. It supports various quantization methods and provides advanced algorithms to reduce model size and computational complexity without significantly sacrificing accuracy. MCT is particularly useful for deploying models on resource-constrained devices, ensuring efficient inference and reduced latency.
### IMX500 Converter Tool (compiler)
The IMX500 Converter Tool is integral to the IMX500 toolset, allowing the compilation of models for deployment on Sony's IMX500 sensor (for instance, Raspberry Pi AI Cameras). This tool facilitates the transition of Ultralytics YOLOv8 models processed through Ultralytics software, ensuring they are compatible and perform efficiently on the specified hardware. The export procedure following model quantization involves the generation of binary files that encapsulate essential data and device-specific configurations, streamlining the deployment process on the Raspberry Pi AI Camera.
## Usage Examples
### Usage Examples
Export an Ultralytics YOLOv8 model to IMX500 format and run inference with the exported model.
Export an Ultralytics YOLOv8 model to IMX500 format.
!!! note
Here we perform inference just to make sure the model works as expected. However, for deployment and inference on the Raspberry Pi AI Camera, please jump to [Using IMX500 Export in Deployment](#using-imx500-export-in-deployment) section.
!!! example
=== "Python"
```python
from ultralytics import YOLO
```python
from ultralytics import YOLO
# Load a YOLOv8n PyTorch model
model = YOLO("yolov8n.pt")
model = YOLO("yolov8n.pt") # Load Ultralytics YOLOv8n model
model.export(format="imx500") # exports with PTQ quantization by default
# model.export(format="imx500", gptq=True) export with Gradient-based Post-Training Quantization (GPTQ)
imx500_model = YOLO("yolov8n_imx500_model.onnx")
imx500_model.val(data=”coco128.yaml”) # Validate the exported imx500 ONNX model
```
# Export the model
model.export(format="imx500") # exports with PTQ quantization by default
# Alternatively, export with Gradient-based Post-Training Quantization (GPTQ)
It might take longer to export with GPTQ quantization based on the hardware specifications.
The export process will create an ONNX model for quantization validation, along with a directory named imx500_model. This directory will include the packerOut.zip file, which is essential for packaging the model for deployment on the IMX500 hardware. Additionally, the imx500_model folder will contain a text file listing all the labels associated with the model and other relevant metadata.
The export process will create an ONNX model for quantization validation, along with a directory named `imx500_model`. This directory will include the `packerOut.zip` file, which is essential for packaging the model for deployment on the IMX500 hardware. Additionally, the `imx500_model` folder will contain a text file (`labels.txt`) listing all the labels associated with the model.
```bash
├── yolov8n_imx500_model
@ -93,16 +95,16 @@ The export process will create an ONNX model for quantization validation, along
## Arguments
When exporting a model to IMX500 format, you can specify various arguments:
When exporting a model to IMX500 format, you can specify various arguments:
| `format` | `'imx500'` | Format to export to (imx500) |
| `format` | `imx500` | Format to export to (imx500) |
| `gptq` | `False` | Use Gradient-based Post-Training Quantization (GPTQ) instead of PTQ |
## Using IMX500 Export in Deployment
After exporting YOLOv8 models to Sony IMX500 format, they can be deployed to Raspberry Pi AI Camera for inference. Raspberry Pi AI Camera uses the Sony IMX500 imaging sensor to provide low-latency, high-performance AI capabilities to fit different applications.
After exporting Ultralytics YOLOv8n model to IMX500 format, it can be deployed to Raspberry Pi AI Camera for inference.
### Hardware Prerequisites
@ -131,7 +133,7 @@ Step 2: Install IMX500 firmware which is required to operate the IMX500 sensor a
sudo apt install imx500-all imx500-tools
```
Step 3: Install prerequisites to run \`picamera2\` application. We will use this applicaiton later for the deployment process.
Step 3: Install prerequisites to run `picamera2` application. We will use this applicaiton later for the deployment process.
```bash
sudo apt install python3-opencv python3-munkres
@ -158,7 +160,7 @@ The above will generate a `network.rpk` file inside the specified output folder.
Step 2: Clone `picamera2` repository and navigate to the imx500 examples
[Sony's Model Compression Toolkit (MCT)](https://github.com/sony/model_optimization) is a powerful tool for optimizing deep learning models through quantization and pruning. It supports various quantization methods and provides advanced algorithms to reduce model size and computational complexity without significantly sacrificing accuracy. MCT is particularly useful for deploying models on resource-constrained devices, ensuring efficient inference and reduced latency.
### Supported Features of MCT
Sony's MCT offers a range of features designed to optimize neural network models:
### Quantization
1. **Graph Optimizations**: Transforms models into more efficient versions by folding layers like batch normalization into preceding layers.
2. **Quantization Parameter Search**: Minimizes quantization noise using metrics like Mean-Square-Error, No-Clipping, and Mean-Average-Error.
3. **Advanced Quantization Algorithms**:
**Shift Negative Correction**: Addresses performance issues from symmetric activation quantization.
**Outliers Filtering**: Uses z-score to detect and remove outliers.
**Clustering**: Utilizes non-uniform quantization grids for better distribution matching.
**Mixed-Precision Search**: Assigns different quantization bit-widths per layer based on sensitivity.
4. **Visualization**: Use TensorBoard to observe model performance insights, quantization phases, and bit-width configurations.
#### Quantization
MCT supports several quantization methods to reduce model size and improve inference speed:
@ -202,36 +223,17 @@ MCT also supports various quantization schemes for weights and activations:
2. Symmetric
3. Uniform
### Main Features
1. **Graph Optimizations**: Transforms models into more efficient versions by folding layers like batch normalization into preceding layers.
2. **Quantization Parameter Search**: Minimizes quantization noise using metrics like Mean-Square-Error, No-Clipping, and Mean-Average-Error.
3. **Advanced Quantization Algorithms**:
**Shift Negative Correction**: Addresses performance issues from symmetric activation quantization.
**Outliers Filtering**: Uses z-score to detect and remove outliers.
**Clustering**: Utilizes non-uniform quantization grids for better distribution matching.
**Mixed-Precision Search**: Assigns different quantization bit-widths per layer based on sensitivity.
4. **Visualization**: Use TensorBoard to observe model performance insights, quantization phases, and bit-width configurations.
#### Enhanced Post-Training Quantization (EPTQ)
As part of GPTQ, MCT includes the Enhanced Post-Training Quantization (EPTQ) algorithm for advanced optimization. EPTQ aims to further reduce quantization error without requiring labeled data. For more details, refer to the paper: [EPTQ: Enhanced Post-Training Quantization via Label-Free Hessian](https://github.com/sony/model_optimization).
### Structured Pruning
#### Structured Pruning
MCT introduces structured, hardware-aware model pruning designed for specific hardware architectures. This technique leverages the target platform's Single Instruction, Multiple Data (SIMD) capabilities by pruning SIMD groups. This reduces model size and complexity while optimizing channel utilization, aligned with the SIMD architecture for targeted resource utilization of weights memory footprint. Available via Keras and PyTorch APIs.
## Benefits of Using MCT
### IMX500 Converter Tool (Compiler)
1. **Model Size Reduction**: Significantly reduces the model size through quantization and pruning.
2. **Inference Speedup**: Improves inference speed by optimizing computations.
3. **Hardware Compatibility**: Generates models optimized for specific hardware architectures.
4. **Advanced Algorithms**: Utilizes state-of-the-art quantization and pruning algorithms.
5. **Ease of Integration**: Seamlessly integrates with PyTorch models.
The IMX500 Converter Tool is integral to the IMX500 toolset, allowing the compilation of models for deployment on Sony's IMX500 sensor (for instance, Raspberry Pi AI Cameras). This tool facilitates the transition of Ultralytics YOLOv8 models processed through Ultralytics software, ensuring they are compatible and perform efficiently on the specified hardware. The export procedure following model quantization involves the generation of binary files that encapsulate essential data and device-specific configurations, streamlining the deployment process on the Raspberry Pi AI Camera.
###**Real-World Use Cases**
## Real-World Use Cases
**Export to** IMX500 format has wide applicability across industries. Here are some examples:
Export to IMX500 format has wide applicability across industries. Here are some examples:
- **Edge AI and IoT**: Enable object detection on drones or security cameras, where real-time processing on low-power devices is essential.
- **Wearable Devices**: Deploy models optimized for small-scale AI processing on health-monitoring wearables.
Lakshantha
3 weeks ago