单帧非自然图像深度估计与动态合成

Depth Estimation and Dynamic Synthesis of Single Frame Unnatural Images

摘要: 深度学习在单目深度估计任务上表现优异，通过学习单帧图像与深度图像之间存在的映射关系来估计图像的深度。但是，目前单目深度估计的研究仅关注于自然图像，当把它应用于非自然图像，如绘画图像时，相对于自然图像，它们有着低纹理、切边锐利、平滑过渡相对少的特点，会出现深度估计中前后不同物体的层次感不明显，以及同一物体上出现深度不一致的问题。本文根据这类图像设计了一个由单目深度估计模块和RGB图像指导的精细化模块构成的精细单目深度估计网络RefineDepth来改善以上问题。同时，由于绘画图像缺乏对应深度信息，本文通过三维场景卡通风格渲染图像来模拟绘画类非自然图像的方式，制作了两个绘画图像数据集SSMO和SU3D，并建立了一个真实的山水画测试集。实验结果表明，模型在测试的数据集上都取得了出色的结果。最后，将绘画图像进行基于深度图像渲染，动态合成立体效果。

Abstract: Deep learning performs well in monocular depth estimation tasks, estimating the depth of an image by learning the mapping relationship between a single image and a depth image. However, the current research on monocular depth estimation only focuses on natural images. When it is applied to unnatural images, such as painting images, they have low texture, sharp cutting edges, and relatively few smooth transitions. In depth estimation, the layering of different objects before and after is not obvious, and the depth of the same object is inconsistent. Based on such images, this paper designs a refined monocular depth estimation network RefineDepth, consisting of a monocular depth estimation module and a RGB image-guided refinement module to improve the above problems. Meanwhile, due to the lack of corresponding depth information in the painting image, we render images in a cartoon style of 3D scenes to simulate the way of painting unnatural images to make two painting image datasets SSMO and SU3D, and build a real landscape painting test set. The experimental results show that the model has achieved excellent results on the tested datasets. Fi-nally, the painting image is rendered based on the depth image, and the three-dimensional effect is dynamically synthesized.