You can not select more than 25 topics Topics must start with a letter or number, can include dashes ('-') and can be up to 35 characters long.
captainfffsama 9d964c5e9d 添加了一些测试和学习性代码 2 years ago
common 添加了一些测试和学习性代码 2 years ago
dataset 添加了一些测试和学习性代码 2 years ago
experiment_dispatcher update 3 years ago
experiments eval part update 3 years ago
loss 添加了一些测试和学习性代码 2 years ago
model 添加了一些测试和学习性代码 2 years ago
.gitignore 添加了一些测试和学习性代码 2 years ago
README.md Update README.md 2 years ago
debug_tools.py 添加了一些测试和学习性代码 2 years ago
evaluate.py 添加了一些测试和学习性代码 2 years ago
infer.py 添加了一些测试和学习性代码 2 years ago
search_hyperparams.py init 3 years ago
train.py init 3 years ago

README.md

[ICCV 2021 Oral] Motion Basis Learning for Unsupervised Deep Homography Estimation with Subspace Projection paper

Nianjin Ye1, Chuan Wang1, Haoqiang Fan1, Shuaicheng Liu2,1

1. Megvii Research, 2. University of Electronic Science and Technology of China

Presentation Video

[Youbube], [Bilibili]

Abstract

In this paper, we introduce a new framework for unsupervised deep homography estimation. Our contributions are 3 folds. First, unlike previous methods that regress 4 offsets for a homography, we propose a homography flow representation, which can be estimated by a weighted sum of 8 pre-defined homography flow bases. Second, considering a homography contains 8 Degree-of-Freedoms (DOFs) that is much less than the rank of the network features, we propose a Low Rank Representation (LRR) block that reduces the feature rank, so that features corresponding to the dominant motions are retained while others are rejected. Last, we propose a Feature Identity Loss (FIL) to enforce the learned image feature warp-equivariant, meaning that the result should be identical if the order of warp operation and feature extraction is swapped. With this constraint, the unsupervised optimization is achieved more effectively and more stable features are learned. Extensive experiments are conducted to demonstrate the effectiveness of all the newly proposed components, and results show that our approach outperforms the state-of-the-art on the homography benchmark datasets both qualitatively and quantitatively.

Motivation

motivation

Basis Homo Representation

basisHomo

Pipeline

pipeline

Requirements

  • Python 3.5
  • Pytorch 1.1.0
  • Numpy 1.15.4

Data pre-processing

Please refer to Content-Aware Unsupervised Deep Homography Estimation.

  1. Download raw data
# GoogleDriver
https://drive.google.com/file/d/19d2ylBUPcMQBb_MNBBGl9rCAS7SU-oGm/view?usp=sharing
# BaiduYun
https://pan.baidu.com/s/1Dkmz4MEzMtBx-T7nG0ORqA (key: gvor)
  1. Unzip the data to directory "./dataset"

  2. Run "video2img.py"

Pretrained model

# GoogleDriver
https://1drv.ms/u/s!AglwI3TlqfaZhlU4BXo7iYiyLdI5?e=tCygKh
# BaiduYun
https://pan.baidu.com/s/1v4tF_RPEKgc6YF_mfRl7jQ (key: b3yq)

Test

python evaluate.py

Citation

All code is provided for research purposes only and without any warranty. Any commercial use requires our consent. If you use this code or ideas from the paper for your research, please cite our paper:

@InProceedings{Ye_2021_ICCV,
    author    = {Ye, Nianjin and Wang, Chuan and Fan, Haoqiang and Liu, Shuaicheng},
    title     = {Motion Basis Learning for Unsupervised Deep Homography Estimation With Subspace Projection},
    booktitle = {Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV)},
    month     = {October},
    year      = {2021},
    pages     = {13117-13125}
}