Temporal Cues from Socially Unacceptable Trajectories for Anomaly Detection

State-of-the-Art (SoTA) deep learning-based ap-proaches to detect anomalies utilize limited temporal in-formation, including basic information from motion, e.g.,optical flow computed between consecutive frames. In thispaper, we compliment the SoTA methods by including long-range dependencies from trajectories for anomaly detec-tion. To achieve that, we first created trajectories by run-ning a tracker on two SoTA datasets, namely Avenue andShanghai-Tech. We propose a prediction-based anomalydetection method using trajectories based on Social GANs,also called in this paper as temporal-based anomaly de-tection. Then, we hypothesize that late fusion of the re-sult of this temporal-based anomaly detection system withspatial-based anomaly detection systems produces SoTAresults. We verify this hypothesis on two spatial-basedanomaly detection systems. We show that both cases pro-duce results better than baseline spatial-based systems, in-dicating the usefulness of the temporal information com-ing from the trajectories for anomaly detection. We observethat the proposed approach depicts the maximum improve-ment in micro-level Area-Under-the-Curve (AUC) by 4.1%on CUHK Avenue and 3.4% on Shanghai-Tech over one ofthe baseline method. We also show a high performance oncross-data evaluation, where we learn the weights to com-bine spatial and temporal information on Shanghai-Techand perform evaluation on CUHK Avenue and vice-versa.