Annotation of Tomato Plants RGB-D Images via Point-to-Pixel Label Transfer

Data driven perception is finding increasing applications through the extensive use of artificial intelligence and neural networks. Although dataset collection and annotation is a time-consuming process, it is essential for generating ground-truth data for supervised machine learning. Data collection and annotation in unstructured and irregular environments like agricultural fields is challenging and cannot be easily automated. Label transfer between different sensor domains, e.g. 3D shapes and images, is a viable solution, but it requires well-crafted robust solution to produce accurate annotation. We propose an algorithm to annotate RGB-D images of tomato plants starting from an annotated 3D reconstruction of the plant. Unlike other approaches in the literature, our method does not rely solely on reprojection of the annotated point cloud. Instead, it combines reprojection with a neighbor search guided by pixel depth. To demonstrate the effectiveness of the proposed method, we perform both qualitative and quantitative evaluations. In particular, we show that the annotated data generated by our approach can be successfully employed in AI tasks, such as training a deep-learning semantic segmentation network.