Is our Ground-Truth for Traffic Classification Reliable?

Valentín Carela-Español, Tomasz Bujlow, Pere Barlet-Ros

Research output: Contribution to book/anthology/report/conference proceedingArticle in proceedingResearchpeer-review

24 Citations (Scopus)

Abstract

The validation of the different proposals in the traffic classification literature is a controversial issue. Usually, these works base their results on a ground-truth built from private datasets and labeled by techniques of unknown reliability. This makes the validation and comparison with other solutions an extremely difficult task.

This paper aims to be a first step towards addressing the validation and trustworthiness problem of network traffic classifiers. We perform a comparison between 6 well-known DPI-based techniques, which are frequently used in the literature for ground-truth generation. In order to evaluate these tools we have carefully built a labeled dataset of more than 500 000 flows, which contains traffic from popular applications. Our results present PACE, a commercial tool, as the most reliable solution for ground-truth generation. However, among the open-source tools available, NDPI and especially Libprotoident, also achieve very high precision, while other, more frequently used tools (e.g., L7-filter) are not reliable enough and should not be used for ground-truth generation in their current form.
Original languageEnglish
Title of host publicationPassive and Active Measurement : Passive and Active Measurement, 15th International Conference, PAM 2014, Los Angeles, USA, March 10-11, 2014, Proceedings Series:
Number of pages11
Volume8362
PublisherSpringer Science+Business Media
Publication date11 Mar 2014
Pages98-108
DOIs
Publication statusPublished - 11 Mar 2014
SeriesLecture Notes in Computer Science
ISSN0302-9743

Fingerprint

Classifiers

Cite this

Carela-Español, V., Bujlow, T., & Barlet-Ros, P. (2014). Is our Ground-Truth for Traffic Classification Reliable? In Passive and Active Measurement: Passive and Active Measurement, 15th International Conference, PAM 2014, Los Angeles, USA, March 10-11, 2014, Proceedings Series: (Vol. 8362, pp. 98-108). Springer Science+Business Media. Lecture Notes in Computer Science https://doi.org/10.1007/978-3-319-04918-2_10
Carela-Español, Valentín ; Bujlow, Tomasz ; Barlet-Ros, Pere. / Is our Ground-Truth for Traffic Classification Reliable?. Passive and Active Measurement: Passive and Active Measurement, 15th International Conference, PAM 2014, Los Angeles, USA, March 10-11, 2014, Proceedings Series:. Vol. 8362 Springer Science+Business Media, 2014. pp. 98-108 (Lecture Notes in Computer Science).
@inproceedings{8fc51768657240e9a8b2f625f3af3805,
title = "Is our Ground-Truth for Traffic Classification Reliable?",
abstract = "The validation of the different proposals in the traffic classification literature is a controversial issue. Usually, these works base their results on a ground-truth built from private datasets and labeled by techniques of unknown reliability. This makes the validation and comparison with other solutions an extremely difficult task.This paper aims to be a first step towards addressing the validation and trustworthiness problem of network traffic classifiers. We perform a comparison between 6 well-known DPI-based techniques, which are frequently used in the literature for ground-truth generation. In order to evaluate these tools we have carefully built a labeled dataset of more than 500 000 flows, which contains traffic from popular applications. Our results present PACE, a commercial tool, as the most reliable solution for ground-truth generation. However, among the open-source tools available, NDPI and especially Libprotoident, also achieve very high precision, while other, more frequently used tools (e.g., L7-filter) are not reliable enough and should not be used for ground-truth generation in their current form.",
author = "Valent{\'i}n Carela-Espa{\~n}ol and Tomasz Bujlow and Pere Barlet-Ros",
year = "2014",
month = "3",
day = "11",
doi = "10.1007/978-3-319-04918-2_10",
language = "English",
volume = "8362",
pages = "98--108",
booktitle = "Passive and Active Measurement",
publisher = "Springer Science+Business Media",
address = "United States",

}

Carela-Español, V, Bujlow, T & Barlet-Ros, P 2014, Is our Ground-Truth for Traffic Classification Reliable? in Passive and Active Measurement: Passive and Active Measurement, 15th International Conference, PAM 2014, Los Angeles, USA, March 10-11, 2014, Proceedings Series:. vol. 8362, Springer Science+Business Media, Lecture Notes in Computer Science, pp. 98-108. https://doi.org/10.1007/978-3-319-04918-2_10

Is our Ground-Truth for Traffic Classification Reliable? / Carela-Español, Valentín; Bujlow, Tomasz; Barlet-Ros, Pere.

Passive and Active Measurement: Passive and Active Measurement, 15th International Conference, PAM 2014, Los Angeles, USA, March 10-11, 2014, Proceedings Series:. Vol. 8362 Springer Science+Business Media, 2014. p. 98-108.

Research output: Contribution to book/anthology/report/conference proceedingArticle in proceedingResearchpeer-review

TY - GEN

T1 - Is our Ground-Truth for Traffic Classification Reliable?

AU - Carela-Español, Valentín

AU - Bujlow, Tomasz

AU - Barlet-Ros, Pere

PY - 2014/3/11

Y1 - 2014/3/11

N2 - The validation of the different proposals in the traffic classification literature is a controversial issue. Usually, these works base their results on a ground-truth built from private datasets and labeled by techniques of unknown reliability. This makes the validation and comparison with other solutions an extremely difficult task.This paper aims to be a first step towards addressing the validation and trustworthiness problem of network traffic classifiers. We perform a comparison between 6 well-known DPI-based techniques, which are frequently used in the literature for ground-truth generation. In order to evaluate these tools we have carefully built a labeled dataset of more than 500 000 flows, which contains traffic from popular applications. Our results present PACE, a commercial tool, as the most reliable solution for ground-truth generation. However, among the open-source tools available, NDPI and especially Libprotoident, also achieve very high precision, while other, more frequently used tools (e.g., L7-filter) are not reliable enough and should not be used for ground-truth generation in their current form.

AB - The validation of the different proposals in the traffic classification literature is a controversial issue. Usually, these works base their results on a ground-truth built from private datasets and labeled by techniques of unknown reliability. This makes the validation and comparison with other solutions an extremely difficult task.This paper aims to be a first step towards addressing the validation and trustworthiness problem of network traffic classifiers. We perform a comparison between 6 well-known DPI-based techniques, which are frequently used in the literature for ground-truth generation. In order to evaluate these tools we have carefully built a labeled dataset of more than 500 000 flows, which contains traffic from popular applications. Our results present PACE, a commercial tool, as the most reliable solution for ground-truth generation. However, among the open-source tools available, NDPI and especially Libprotoident, also achieve very high precision, while other, more frequently used tools (e.g., L7-filter) are not reliable enough and should not be used for ground-truth generation in their current form.

U2 - 10.1007/978-3-319-04918-2_10

DO - 10.1007/978-3-319-04918-2_10

M3 - Article in proceeding

VL - 8362

SP - 98

EP - 108

BT - Passive and Active Measurement

PB - Springer Science+Business Media

ER -

Carela-Español V, Bujlow T, Barlet-Ros P. Is our Ground-Truth for Traffic Classification Reliable? In Passive and Active Measurement: Passive and Active Measurement, 15th International Conference, PAM 2014, Los Angeles, USA, March 10-11, 2014, Proceedings Series:. Vol. 8362. Springer Science+Business Media. 2014. p. 98-108. (Lecture Notes in Computer Science). https://doi.org/10.1007/978-3-319-04918-2_10