Depth images usually contain pixels with invalid measurements. This paper presents a deep learning approach that receives as input a partially-known volumetric model of the environment and a camera pose, and it predicts the probability that a pixel would contain a valid depth measurement if a camera was placed at the given pose. The proposed network architecture consists of a 3D Convolutional Neural Network (CNN) module and a 2D CNN module, connected by a deep learning attention-based projection module. The method was integrated into a CNN-based probabilistic Next Best View plan-ner, resulting in a more realistic prediction of the information gain for each possible viewpoint with respect to state of the art approaches. Experiments were carried out in tabletop scenarios using a robot manipulator with an eye-in-hand depth camera.

Prediction of Depth Camera Missing Measurements Using Deep Learning for Next Best View Planning / Monica, Riccardo; Aleotti, Jacopo. - (2022), pp. 8711-8717. ((Intervento presentato al convegno 39th IEEE International Conference on Robotics and Automation, ICRA 2022 tenutosi a Philadelphia, USA nel 2022 [10.1109/ICRA46639.2022.9812358].

Prediction of Depth Camera Missing Measurements Using Deep Learning for Next Best View Planning

Riccardo Monica
;
Jacopo Aleotti
2022

Abstract

Depth images usually contain pixels with invalid measurements. This paper presents a deep learning approach that receives as input a partially-known volumetric model of the environment and a camera pose, and it predicts the probability that a pixel would contain a valid depth measurement if a camera was placed at the given pose. The proposed network architecture consists of a 3D Convolutional Neural Network (CNN) module and a 2D CNN module, connected by a deep learning attention-based projection module. The method was integrated into a CNN-based probabilistic Next Best View plan-ner, resulting in a more realistic prediction of the information gain for each possible viewpoint with respect to state of the art approaches. Experiments were carried out in tabletop scenarios using a robot manipulator with an eye-in-hand depth camera.
Prediction of Depth Camera Missing Measurements Using Deep Learning for Next Best View Planning / Monica, Riccardo; Aleotti, Jacopo. - (2022), pp. 8711-8717. ((Intervento presentato al convegno 39th IEEE International Conference on Robotics and Automation, ICRA 2022 tenutosi a Philadelphia, USA nel 2022 [10.1109/ICRA46639.2022.9812358].
File in questo prodotto:
Non ci sono file associati a questo prodotto.

I documenti in IRIS sono protetti da copyright e tutti i diritti sono riservati, salvo diversa indicazione.

Utilizza questo identificativo per citare o creare un link a questo documento: https://hdl.handle.net/11381/2928632
Citazioni
  • ???jsp.display-item.citation.pmc??? ND
  • Scopus 0
  • ???jsp.display-item.citation.isi??? ND
social impact