|
|
## About code isolation |
|
|
|
|
|
This `downstream_d2` is isolated from pre-training codes. One can treat this `downstream_d2` as an independent codebase 🛠️. |
|
|
|
|
|
|
|
|
## Fine-tuned ResNet-50 weights, log files, and performance |
|
|
|
|
|
<div align="center"> |
|
|
|
|
|
[[`weights (pre-trained by SparK)`](https://drive.google.com/file/d/1H8605HbxGvrsu4x4rIoNr-Wkd7JkxFPQ/view?usp=share_link)] |
|
|
[[`weights (fine-tuned on COCO)`](https://drive.google.com/file/d/1Ue7SiQ1E_AwgtYo56Fm-iUlQPZ8vIwYj/view?usp=share_link)] |
|
|
[[`metrics.json`](https://drive.google.com/file/d/1wfbUWh4svV8sPWya_0PAhsLHVayDQRCi/view?usp=share_link)] |
|
|
[[`log.txt`](https://drive.google.com/file/d/11zVo_87pe9DMAmfNQK9FUfyjQWHTRKxV/view?usp=share_link)] |
|
|
[[`tensorboard file`](https://drive.google.com/file/d/1aM1qj8c3-Uka1dZuYmKhgp1lNJpeMDMl/view?usp=share_link)] |
|
|
</div> |
|
|
|
|
|
<p align="center"> |
|
|
<img src="https://user-images.githubusercontent.com/39692511/211497479-0563e891-f2ad-4cf1-b682-a21c2be1442d.png" width=80%> |
|
|
<p> |
|
|
|
|
|
|
|
|
## Installation [Detectron2 v0.6](https://github.com/facebookresearch/detectron2/releases/tag/v0.6) before fine-tuning ResNet on COCO |
|
|
|
|
|
|
|
|
1. Let you in some python environment, e.g.: |
|
|
```shell script |
|
|
$ conda create -n spark python=3.8 -y |
|
|
$ conda activate spark |
|
|
``` |
|
|
|
|
|
2. Install `detectron2==0.6` (e.g., with `torch==1.10.0` and `cuda11.3`): |
|
|
```shell script |
|
|
$ pip install detectron2==0.6 -f https://dl.fbaipublicfiles.com/detectron2/wheels/cu113/torch1.10/index.html |
|
|
``` |
|
|
|
|
|
You can also find instructions for different pytorch/cuda versions on [this page](https://github.com/facebookresearch/detectron2/releases/tag/v0.6). |
|
|
|
|
|
|
|
|
3. Put the COCO dataset folder at `downstream_d2/datasets/coco`. |
|
|
The folder should follow the [directory structure](https://github.com/facebookresearch/detectron2/tree/master/datasets) requried by `Detectron2`, which should look like this: |
|
|
``` |
|
|
downstream_d2/datasets/coco: |
|
|
annotations/: |
|
|
captions_train2017.json captions_val2017.json |
|
|
instances_train2017.json instances_val2017.json |
|
|
person_keypoints_train2017.json person_keypoints_val2017.json |
|
|
train2017/: |
|
|
a_lot_images.jpg |
|
|
val2017/: |
|
|
a_lot_images.jpg |
|
|
``` |
|
|
|
|
|
|
|
|
## Training from pre-trained checkpoint |
|
|
|
|
|
The script file for COCO fine-tuning (object detection and instance segmentation) is [downstream_d2/train_net.py](https://github.com/keyu-tian/SparK/blob/main/downstream_d2/train_net.py), |
|
|
which is a modification of [Detectron2's tools/train_net.py](https://github.com/facebookresearch/detectron2/blob/v0.6/tools/train_net.py). |
|
|
|
|
|
|
|
|
Before fine-tuning a ResNet50 pre-trained by SparK, you should first convert our checkpoint file to Detectron2-style `.pkl` file: |
|
|
|
|
|
```shell script |
|
|
$ cd /path/to/SparK/downstream_d2 |
|
|
$ python3 convert-timm-to-d2.py /some/path/to/resnet50_1kpretrained_timm_style.pth d2-style.pkl |
|
|
``` |
|
|
|
|
|
For a ResNet50, you should see a log reporting `len(state)==318`: |
|
|
```text |
|
|
[convert] .pkl is generated! (from `/some/path/to/resnet50_1kpretrained_timm_style.pth`, to `d2-style.pkl`, len(state)==318) |
|
|
``` |
|
|
|
|
|
Then run fine-tuning on single machine with 8 gpus: |
|
|
|
|
|
```shell script |
|
|
$ cd /path/to/SparK/downstream_d2 |
|
|
$ python3 ./train_net.py --resume --num-gpus 8 --config-file ./configs/coco_R_50_FPN_CONV_1x_moco_adam.yaml \ |
|
|
MODEL.WEIGHTS d2-style.pkl \ |
|
|
OUTPUT_DIR <your_output_dir> |
|
|
``` |
|
|
|
|
|
For multiple machines, plus these args: |
|
|
```shell script |
|
|
--num-machines <total_num> --machine-rank <this_rank> --dist-url <url:port> |
|
|
``` |
|
|
|
|
|
In `<your_output_dir>` you'll see the log files generated by `Detectron2`. |
|
|
|
|
|
|
|
|
## Details: how we modify the official Detectron2's [tools/train_net.py](https://github.com/facebookresearch/detectron2/blob/v0.6/tools/train_net.py) to get our [downstream_d2/train_net.py](https://github.com/keyu-tian/SparK/blob/main/downstream_d2/train_net.py) |
|
|
|
|
|
1. We add two new hyperparameters: |
|
|
- str `SOLVER.OPTIMIZER`: use 'ADAM' (the same as 'ADAMW') or 'SGD' optimizer |
|
|
- float `SOLVER.LR_DECAY`: the decay ratio (from 0. to 1.) of layer-wise learning rate decay trick |
|
|
|
|
|
2. We implement layer-wise lr decay in [downstream_d2/lr_decay.py](https://github.com/keyu-tian/SparK/blob/main/downstream_d2/lr_decay.py). |
|
|
|
|
|
3. We write a script to convert our timm-style pre-trained ResNet weights to Detectron2-style in [downstream_d2/convert-timm-to-d2.py](https://github.com/keyu-tian/SparK/blob/main/downstream_d2/convert-timm-to-d2.py). |
|
|
|
|
|
4. We also add a hook for logging results to `cfg.OUTPUT_DIR/d2_coco_log.txt`. |
|
|
|
|
|
All of our modifications to the original are commented with `# [modification] ...` in [downstream_d2/train_net.py](https://github.com/keyu-tian/SparK/blob/main/downstream_d2/train_net.py) or other files.
|
|
|
|
