Optimal State Estimation for DNN Visual Servoing Systems with Detection Loss

The introduction of deep learning techniques, such
as object detection in visual servoing systems, has produced
more sophisticated robotic systems capable of working in
unknown environments. However, the interaction between the
object detection network and the controller, when detection loss
occurs, has received little attention. In this paper, we investigate
a way of mitigating the effect of detection loss in VSNN systems.
In our approach detection losses are modeled as a Bernoulli
random variable and integrated into the state space model of
the dynamic system. To mitigate the effect of detection loss,
we propose a variation of a Kalman filter, that artificially
inflates the measurement noise covariance when detection loss
occurs. The Kalman filter was implemented on a 6DOF robotic
manipulator with an eye-in-hand configuration with YOLOv5
as the object detection network. The results show, that the
proposed Kalman filter decreases the effects of detection losses
and significantly improves performance compared to having a
standard Kalman filter, and not having a state estimator at
all. The benefit of our approach is especially noticeable when
detection loss occurs frequently and for relatively long periods
of time