ultralytics/docs/en/modes/benchmark.md

comments	description	keywords
true	Learn how to evaluate your YOLOv8 model's performance in real-world scenarios using benchmark mode. Optimize speed, accuracy, and resource allocation across export formats.	model benchmarking, YOLOv8, Ultralytics, performance evaluation, export formats, ONNX, TensorRT, OpenVINO, CoreML, TensorFlow, optimization, mAP50-95, inference time

Model Benchmarking with Ultralytics YOLO

Introduction

Once your model is trained and validated, the next logical step is to evaluate its performance in various real-world scenarios. Benchmark mode in Ultralytics YOLOv8 serves this purpose by providing a robust framework for assessing the speed and accuracy of your model across a range of export formats.

Watch: Ultralytics Modes Tutorial: Benchmark

Why Is Benchmarking Crucial?

• Informed Decisions: Gain insights into the trade-offs between speed and accuracy.
• Resource Allocation: Understand how different export formats perform on different hardware.
• Optimization: Learn which export format offers the best performance for your specific use case.
• Cost Efficiency: Make more efficient use of hardware resources based on benchmark results.

Key Metrics in Benchmark Mode

• mAP50-95: For object detection, segmentation, and pose estimation.
• accuracy_top5: For image classification.
• Inference Time: Time taken for each image in milliseconds.

Supported Export Formats

• ONNX: For optimal CPU performance
• TensorRT: For maximal GPU efficiency
• OpenVINO: For Intel hardware optimization
• CoreML, TensorFlow SavedModel, and More: For diverse deployment needs.

!!! Tip "Tip"

* Export to ONNX or OpenVINO for up to 3x CPU speedup.
* Export to TensorRT for up to 5x GPU speedup.

Usage Examples

Run YOLOv8n benchmarks on all supported export formats including ONNX, TensorRT etc. See Arguments section below for a full list of export arguments.

!!! Example

=== "Python"

    ```python
    from ultralytics.utils.benchmarks import benchmark

    # Benchmark on GPU
    benchmark(model="yolov8n.pt", data="coco8.yaml", imgsz=640, half=False, device=0)
    ```

=== "CLI"

    ```bash
    yolo benchmark model=yolov8n.pt data='coco8.yaml' imgsz=640 half=False device=0
    ```

Arguments

Arguments such as model, data, imgsz, half, device, and verbose provide users with the flexibility to fine-tune the benchmarks to their specific needs and compare the performance of different export formats with ease.

Key	Default Value	Description
model	None	Specifies the path to the model file. Accepts both .pt and .yaml formats, e.g., "yolov8n.pt" for pre-trained models or configuration files.
data	None	Path to a YAML file defining the dataset for benchmarking, typically including paths and settings for validation data. Example: "coco8.yaml".
imgsz	640	The input image size for the model. Can be a single integer for square images or a tuple (width, height) for non-square, e.g., (640, 480).
half	False	Enables FP16 (half-precision) inference, reducing memory usage and possibly increasing speed on compatible hardware. Use half=True to enable.
int8	False	Activates INT8 quantization for further optimized performance on supported devices, especially useful for edge devices. Set int8=True to use.
device	None	Defines the computation device(s) for benchmarking, such as "cpu", "cuda:0", or a list of devices like "cuda:0,1" for multi-GPU setups.
verbose	False	Controls the level of detail in logging output. A boolean value; set verbose=True for detailed logs or a float for thresholding errors.

Export Formats

Benchmarks will attempt to run automatically on all possible export formats below.

Format	format Argument	Model	Metadata	Arguments
PyTorch	-	yolov8n.pt	✅	-
TorchScript	torchscript	yolov8n.torchscript	✅	imgsz, optimize, batch
ONNX	onnx	yolov8n.onnx	✅	imgsz, half, dynamic, simplify, opset, batch
OpenVINO	openvino	yolov8n_openvino_model/	✅	imgsz, half, int8, batch
TensorRT	engine	yolov8n.engine	✅	imgsz, half, dynamic, simplify, workspace, int8, batch
CoreML	coreml	yolov8n.mlpackage	✅	imgsz, half, int8, nms, batch
TF SavedModel	saved_model	yolov8n_saved_model/	✅	imgsz, keras, int8, batch
TF GraphDef	pb	yolov8n.pb	❌	imgsz, batch
TF Lite	tflite	yolov8n.tflite	✅	imgsz, half, int8, batch
TF Edge TPU	edgetpu	yolov8n_edgetpu.tflite	✅	imgsz
TF.js	tfjs	yolov8n_web_model/	✅	imgsz, half, int8, batch
PaddlePaddle	paddle	yolov8n_paddle_model/	✅	imgsz, batch
NCNN	ncnn	yolov8n_ncnn_model/	✅	imgsz, half, batch

See full export details in the Export page.