On the ground truth problem of malicious DNS traffic analysis

DNS is often abused by Internet criminals in order to provide flexible and resilient hosting of malicious content and reliable communication within their network architecture. The majority of detection methods targeting alicious DNS traffic are data-driven, most commonly having machine learning algorithms at their core. These methods require accurate ground truth of both malicious and benign DNS traffic for model training as well as for the performance evaluation. This paper elaborates on the problem of obtaining such a ground truth and evaluates practices employed by contemporary detection methods. Building upon the evaluation results, we propose a novel semi-manual labeling practice targeting agile DNS mappings, i.e. DNS queries that are used to reach a potentially malicious server characterized by fast changing domain names or/and IP addresses. The proposed approach is developed with the purpose of obtaining ground truth by incorporating the operator's insight in efficient and effective manner. We evaluate the proposed approach on a case study based on DNS traffic from an ISP network by comparing it with the popular labeling practices that rely on domain name and IP blacklists and whitelisting of popular domains. The evaluation indicates challenges and limitations of relying on existing labeling practices and shows a clear advantage of using the proposed approach in discovering a more complete set of potentially malicious domains and IP addresses. Furthermore, the novel approach attains time-efficient labeling with limited operator's involvement, thus is promising in view of the adoption in operational ISP networks.