Ground truth evaluation of computer vision based 3D reconstruction of synthesized and real plant images

There is an increasing interest in using 3D computer vision in precision agriculture. This calls for better quantitative evaluation and understanding of computer vision methods. This paper proposes a test framework using ray traced crop scenes that allows in-depth analysis of algorithm performance and finds the optimal hardware and light source setup before investing in expensive equipment and field experiments. It was expected to be a valuable tool to structure the otherwise incomprehensibly large information space and to see relationships between parameter configurations and crop features. Images of real plants with similar structural categories were annotated manually for comparison in order to validate the performance results on the synthesised images. The results showed substantial correlation between synthesized and real plants, but only when all error sources were accounted for in the simulation. However, there were exceptions where there were structural differences between the virtual plant and the real plant that were unaccounted for by its category. The test framework was evaluated to be a valuable tool to uncover information from complex data structures.