自监督多视角立体视觉深度估计MVSNet系列论文整理

文摘   科技   2021-11-29 21:37  
 有监督的MVSNet系列已经有比较多工作,自监督/无监督的MVS目前工作主要有以下几篇:

Unsup_MVS(CVPR 2019 Workshop)、MVS2(3DV 2019)、M3VSNet(ICIP 2021)、Self-supervised-CVP-MVSNet(CVPR 2021)、U-MVS(ICCV 2021)JDACS-MS(AAAI 2021)。

1 Learning Unsupervised Multi-View Stereopsis via Robust Photometric Consistency

链接:https://arxiv.org/abs/1905.02706

github:https://tejaskhot.github.io/unsup_mvs/

发表:CVPR 2019 Workshop

作者:CMU, Facebook

评价:第一篇 自监督MVS论文

To overcome this, we propose a robust loss formulation that: a) enforces first order consistency and b) for each point, selectively enforces consistency with some views, thus implicitly handling occlusions.

2 MVS2: Deep Unsupervised Multi-View Stereo with Multi-View Symmetry

链接:https://ieeexplore.ieee.org/abstract/document/8885975

github:无

发表:3DV 2019

评价:提出自监督MVS的一致性损失函数

 Our network is symmetric in predicting depth maps for all views simultaneously, where we enforce cross-view consistency of multi-view depth maps during both training and testing stages.

3 M3VSNet: Unsupervised Multi-metric Multi-view Stereo Network

链接:https://ieeexplore.ieee.org/abstract/document/9506469

github:https://github.com/whubaichuan/M3VSNet

发表:ICIP 2021

评价:

we improve the robustness and completeness of point cloud reconstruction, we propose a novel multi-metric loss function that combines pixel-wise and feature-wise loss function to learn the inherent constraints from different perspectives of matching correspondences. Besides, we also incorporate the normal-depth consistency in the 3D point cloud format to improve the accuracy and continuity of the estimated depth maps.

4 Self-supervised Learning of Depth Inference for Multi-view Stereo

链接:https://openaccess.thecvf.com/content/CVPR2021/papers/Yang_Self-Supervised_Learning_of_Depth_Inference_for_Multi-View_Stereo_CVPR_2021_paper.pdf

github:https://github.com/JiayuYANG/Self-supervised-CVP-MVSNet

发表:CVPR 2021

评价:自监督MVS领域第一篇发表到视觉顶会上的工作,CVP-MVSNet同一作者

We start by learning to estimate depth maps as initial pseudo labels under an unsupervised learning framework relying on image reconstruction loss as supervision. We then refine the initial pseudo labels using a carefully designed pipeline leveraging depth information inferred from a higher resolution image and neighboring views. We use these high-quality pseudo labels as the supervision signal to train the network and improve, iteratively, its performance by self-training.

5 Digging into Uncertainty in Self-supervised Multi-view Stereo

链接:https://openaccess.thecvf.com/content/ICCV2021/papers/Xu_Digging_Into_Uncertainty_in_Self-Supervised_Multi-View_Stereo_ICCV_2021_paper.pdf

github:https://github.com/ToughStoneX/U-MVS暂无

发表:ICCV 2021

评价:自监督MVS领域第二篇发表到视觉顶会上的工作

To alleviate ambiguous supervision in foreground, we involve extra correspondence prior with a flow-depth consistency loss. The dense 2D correspondence of optical flows is used to regularize the 3D stereo correspondence in MVS. To handle the invalid supervision in background, we use Monte-Carlo Dropout to acquire the uncertainty map and further filter the unreliable supervision signals on invalid regions.

6 Self-supervised Multi-view Stereo via Effective Co-Segmentation and Data-Augmentation

链接:https://www.aaai.org/AAAI21Papers/AAAI-2549.XuH.pdf

github:https://github.com/ToughStoneX/Self-Supervised-MVS

发表:AAAI 2021

作者:和上一篇一个人

评价:效果已经和有监督的差不多了

To address the issue, we propose a framework integrated with more reliable supervision guided by semantic co-segmentation and dataaugmentation. Specially, we excavate mutual semantic from multi-view images to guide the semantic consistency. And we devise effective data-augmentation mechanism which ensures the transformation robustness by treating the prediction of regular samples as pseudo ground truth to regularize the prediction of augmented samples.


目前有监督的多视角立体视觉三维重建算法MVSNet系列论文比较多,自监督的论文开始被顶会接收!


建了一个多视角立体视觉深度估计MVS群,欢迎交流讨论:

机器人EmbodiedAI
与世界交互
 最新文章