You can not select more than 25 topics
Topics must start with a letter or number, can include dashes ('-') and can be up to 35 characters long.
165 lines
11 KiB
165 lines
11 KiB
--- |
|
comments: true |
|
description: Learn how to optimize YOLOv5 with hyperparameter evolution using Genetic Algorithm. This guide provides steps to initialize, define, evolve and visualize hyperparameters for top performance. |
|
keywords: Ultralytics, YOLOv5, Hyperparameter Optimization, Genetic Algorithm, Machine Learning, Deep Learning, AI, Object Detection, Image Classification, Python |
|
--- |
|
|
|
📚 This guide explains **hyperparameter evolution** for YOLOv5 🚀. Hyperparameter evolution is a method of [Hyperparameter Optimization](https://en.wikipedia.org/wiki/Hyperparameter_optimization) using a [Genetic Algorithm](https://en.wikipedia.org/wiki/Genetic_algorithm) (GA) for optimization. UPDATED 25 September 2022. |
|
|
|
Hyperparameters in ML control various aspects of training, and finding optimal values for them can be a challenge. Traditional methods like grid searches can quickly become intractable due to 1) the high dimensional search space 2) unknown correlations among the dimensions, and 3) expensive nature of evaluating the fitness at each point, making GA a suitable candidate for hyperparameter searches. |
|
|
|
## Before You Start |
|
|
|
Clone repo and install [requirements.txt](https://github.com/ultralytics/yolov5/blob/master/requirements.txt) in a [**Python>=3.8.0**](https://www.python.org/) environment, including [**PyTorch>=1.8**](https://pytorch.org/get-started/locally/). [Models](https://github.com/ultralytics/yolov5/tree/master/models) and [datasets](https://github.com/ultralytics/yolov5/tree/master/data) download automatically from the latest YOLOv5 [release](https://github.com/ultralytics/yolov5/releases). |
|
|
|
```bash |
|
git clone https://github.com/ultralytics/yolov5 # clone |
|
cd yolov5 |
|
pip install -r requirements.txt # install |
|
``` |
|
|
|
## 1. Initialize Hyperparameters |
|
|
|
YOLOv5 has about 30 hyperparameters used for various training settings. These are defined in `*.yaml` files in the `/data/hyps` directory. Better initial guesses will produce better final results, so it is important to initialize these values properly before evolving. If in doubt, simply use the default values, which are optimized for YOLOv5 COCO training from scratch. |
|
|
|
```yaml |
|
# YOLOv5 🚀 by Ultralytics, AGPL-3.0 license |
|
# Hyperparameters for low-augmentation COCO training from scratch |
|
# python train.py --batch 64 --cfg yolov5n6.yaml --weights '' --data coco.yaml --img 640 --epochs 300 --linear |
|
# See tutorials for hyperparameter evolution https://github.com/ultralytics/yolov5#tutorials |
|
|
|
lr0: 0.01 # initial learning rate (SGD=1E-2, Adam=1E-3) |
|
lrf: 0.01 # final OneCycleLR learning rate (lr0 * lrf) |
|
momentum: 0.937 # SGD momentum/Adam beta1 |
|
weight_decay: 0.0005 # optimizer weight decay 5e-4 |
|
warmup_epochs: 3.0 # warmup epochs (fractions ok) |
|
warmup_momentum: 0.8 # warmup initial momentum |
|
warmup_bias_lr: 0.1 # warmup initial bias lr |
|
box: 0.05 # box loss gain |
|
cls: 0.5 # cls loss gain |
|
cls_pw: 1.0 # cls BCELoss positive_weight |
|
obj: 1.0 # obj loss gain (scale with pixels) |
|
obj_pw: 1.0 # obj BCELoss positive_weight |
|
iou_t: 0.20 # IoU training threshold |
|
anchor_t: 4.0 # anchor-multiple threshold |
|
# anchors: 3 # anchors per output layer (0 to ignore) |
|
fl_gamma: 0.0 # focal loss gamma (efficientDet default gamma=1.5) |
|
hsv_h: 0.015 # image HSV-Hue augmentation (fraction) |
|
hsv_s: 0.7 # image HSV-Saturation augmentation (fraction) |
|
hsv_v: 0.4 # image HSV-Value augmentation (fraction) |
|
degrees: 0.0 # image rotation (+/- deg) |
|
translate: 0.1 # image translation (+/- fraction) |
|
scale: 0.5 # image scale (+/- gain) |
|
shear: 0.0 # image shear (+/- deg) |
|
perspective: 0.0 # image perspective (+/- fraction), range 0-0.001 |
|
flipud: 0.0 # image flip up-down (probability) |
|
fliplr: 0.5 # image flip left-right (probability) |
|
mosaic: 1.0 # image mosaic (probability) |
|
mixup: 0.0 # image mixup (probability) |
|
copy_paste: 0.0 # segment copy-paste (probability) |
|
``` |
|
|
|
## 2. Define Fitness |
|
|
|
Fitness is the value we seek to maximize. In YOLOv5 we define a default fitness function as a weighted combination of metrics: `mAP@0.5` contributes 10% of the weight and `mAP@0.5:0.95` contributes the remaining 90%, with [Precision `P` and Recall `R`](https://en.wikipedia.org/wiki/Precision_and_recall) absent. You may adjust these as you see fit or use the default fitness definition in utils/metrics.py (recommended). |
|
|
|
```python |
|
def fitness(x): |
|
# Model fitness as a weighted combination of metrics |
|
w = [0.0, 0.0, 0.1, 0.9] # weights for [P, R, mAP@0.5, mAP@0.5:0.95] |
|
return (x[:, :4] * w).sum(1) |
|
``` |
|
|
|
## 3. Evolve |
|
|
|
Evolution is performed about a base scenario which we seek to improve upon. The base scenario in this example is finetuning COCO128 for 10 epochs using pretrained YOLOv5s. The base scenario training command is: |
|
|
|
```bash |
|
python train.py --epochs 10 --data coco128.yaml --weights yolov5s.pt --cache |
|
``` |
|
|
|
To evolve hyperparameters **specific to this scenario**, starting from our initial values defined in **Section 1.**, and maximizing the fitness defined in **Section 2.**, append `--evolve`: |
|
|
|
```bash |
|
# Single-GPU |
|
python train.py --epochs 10 --data coco128.yaml --weights yolov5s.pt --cache --evolve |
|
|
|
# Multi-GPU |
|
for i in 0 1 2 3 4 5 6 7; do |
|
sleep $(expr 30 \* $i) && # 30-second delay (optional) |
|
echo 'Starting GPU '$i'...' && |
|
nohup python train.py --epochs 10 --data coco128.yaml --weights yolov5s.pt --cache --device $i --evolve > evolve_gpu_$i.log & |
|
done |
|
|
|
# Multi-GPU bash-while (not recommended) |
|
for i in 0 1 2 3 4 5 6 7; do |
|
sleep $(expr 30 \* $i) && # 30-second delay (optional) |
|
echo 'Starting GPU '$i'...' && |
|
"$(while true; do nohup python train.py... --device $i --evolve 1 > evolve_gpu_$i.log; done)" & |
|
done |
|
``` |
|
|
|
The default evolution settings will run the base scenario 300 times, i.e. for 300 generations. You can modify generations via the `--evolve` argument, i.e. `python train.py --evolve 1000`. |
|
|
|
The main genetic operators are **crossover** and **mutation**. In this work mutation is used, with an 80% probability and a 0.04 variance to create new offspring based on a combination of the best parents from all previous generations. Results are logged to `runs/evolve/exp/evolve.csv`, and the highest fitness offspring is saved every generation as `runs/evolve/hyp_evolved.yaml`: |
|
|
|
```yaml |
|
# YOLOv5 Hyperparameter Evolution Results |
|
# Best generation: 287 |
|
# Last generation: 300 |
|
# metrics/precision, metrics/recall, metrics/mAP_0.5, metrics/mAP_0.5:0.95, val/box_loss, val/obj_loss, val/cls_loss |
|
# 0.54634, 0.55625, 0.58201, 0.33665, 0.056451, 0.042892, 0.013441 |
|
|
|
lr0: 0.01 # initial learning rate (SGD=1E-2, Adam=1E-3) |
|
lrf: 0.2 # final OneCycleLR learning rate (lr0 * lrf) |
|
momentum: 0.937 # SGD momentum/Adam beta1 |
|
weight_decay: 0.0005 # optimizer weight decay 5e-4 |
|
warmup_epochs: 3.0 # warmup epochs (fractions ok) |
|
warmup_momentum: 0.8 # warmup initial momentum |
|
warmup_bias_lr: 0.1 # warmup initial bias lr |
|
box: 0.05 # box loss gain |
|
cls: 0.5 # cls loss gain |
|
cls_pw: 1.0 # cls BCELoss positive_weight |
|
obj: 1.0 # obj loss gain (scale with pixels) |
|
obj_pw: 1.0 # obj BCELoss positive_weight |
|
iou_t: 0.20 # IoU training threshold |
|
anchor_t: 4.0 # anchor-multiple threshold |
|
# anchors: 3 # anchors per output layer (0 to ignore) |
|
fl_gamma: 0.0 # focal loss gamma (efficientDet default gamma=1.5) |
|
hsv_h: 0.015 # image HSV-Hue augmentation (fraction) |
|
hsv_s: 0.7 # image HSV-Saturation augmentation (fraction) |
|
hsv_v: 0.4 # image HSV-Value augmentation (fraction) |
|
degrees: 0.0 # image rotation (+/- deg) |
|
translate: 0.1 # image translation (+/- fraction) |
|
scale: 0.5 # image scale (+/- gain) |
|
shear: 0.0 # image shear (+/- deg) |
|
perspective: 0.0 # image perspective (+/- fraction), range 0-0.001 |
|
flipud: 0.0 # image flip up-down (probability) |
|
fliplr: 0.5 # image flip left-right (probability) |
|
mosaic: 1.0 # image mosaic (probability) |
|
mixup: 0.0 # image mixup (probability) |
|
copy_paste: 0.0 # segment copy-paste (probability) |
|
``` |
|
|
|
We recommend a minimum of 300 generations of evolution for best results. Note that **evolution is generally expensive and time-consuming**, as the base scenario is trained hundreds of times, possibly requiring hundreds or thousands of GPU hours. |
|
|
|
## 4. Visualize |
|
|
|
`evolve.csv` is plotted as `evolve.png` by `utils.plots.plot_evolve()` after evolution finishes with one subplot per hyperparameter showing fitness (y-axis) vs hyperparameter values (x-axis). Yellow indicates higher concentrations. Vertical distributions indicate that a parameter has been disabled and does not mutate. This is user selectable in the `meta` dictionary in train.py, and is useful for fixing parameters and preventing them from evolving. |
|
|
|
 |
|
|
|
## Environments |
|
|
|
YOLOv5 is designed to be run in the following up-to-date verified environments (with all dependencies including [CUDA](https://developer.nvidia.com/cuda)/[CUDNN](https://developer.nvidia.com/cudnn), [Python](https://www.python.org/) and [PyTorch](https://pytorch.org/) preinstalled): |
|
|
|
- **Notebooks** with free GPU: <a href="https://bit.ly/yolov5-paperspace-notebook"><img src="https://assets.paperspace.io/img/gradient-badge.svg" alt="Run on Gradient"></a> <a href="https://colab.research.google.com/github/ultralytics/yolov5/blob/master/tutorial.ipynb"><img src="https://colab.research.google.com/assets/colab-badge.svg" alt="Open In Colab"></a> <a href="https://www.kaggle.com/ultralytics/yolov5"><img src="https://kaggle.com/static/images/open-in-kaggle.svg" alt="Open In Kaggle"></a> |
|
- **Google Cloud** Deep Learning VM. See [GCP Quickstart Guide](https://docs.ultralytics.com/yolov5/environments/google_cloud_quickstart_tutorial/) |
|
- **Amazon** Deep Learning AMI. See [AWS Quickstart Guide](https://docs.ultralytics.com/yolov5/environments/aws_quickstart_tutorial/) |
|
- **Docker Image**. See [Docker Quickstart Guide](https://docs.ultralytics.com/yolov5/environments/docker_image_quickstart_tutorial/) <a href="https://hub.docker.com/r/ultralytics/yolov5"><img src="https://img.shields.io/docker/pulls/ultralytics/yolov5?logo=docker" alt="Docker Pulls"></a> |
|
|
|
## Status |
|
|
|
<a href="https://github.com/ultralytics/yolov5/actions/workflows/ci-testing.yml"><img src="https://github.com/ultralytics/yolov5/actions/workflows/ci-testing.yml/badge.svg" alt="YOLOv5 CI"></a> |
|
|
|
If this badge is green, all [YOLOv5 GitHub Actions](https://github.com/ultralytics/yolov5/actions) Continuous Integration (CI) tests are currently passing. CI tests verify correct operation of YOLOv5 [training](https://github.com/ultralytics/yolov5/blob/master/train.py), [validation](https://github.com/ultralytics/yolov5/blob/master/val.py), [inference](https://github.com/ultralytics/yolov5/blob/master/detect.py), [export](https://github.com/ultralytics/yolov5/blob/master/export.py) and [benchmarks](https://github.com/ultralytics/yolov5/blob/master/benchmarks.py) on macOS, Windows, and Ubuntu every 24 hours and on every commit.
|
|
|