3 Citationer (Scopus)
268 Downloads (Pure)

Resumé

Varying weather conditions, including rainfall and snowfall, are generally regarded as a challenge for computer vision algorithms. One proposed solution to the challenges induced by rain and snowfall is to artificially remove the rain from images or video using rain removal algorithms. It is the promise of these algorithms that the rain-removed image frames will improve the performance of subsequent segmentation and tracking algorithms. However, rain removal algorithms are typically evaluated on their ability to remove synthetic rain on a small subset of images. Currently, their behavior is unknown on real-world videos when integrated with a typical computer vision pipeline. In this paper, we review the existing rain removal algorithms and propose a new dataset that consists of 22 traffic surveillance sequences under a broad variety of weather conditions that all include either rain or snowfall. We propose a new evaluation protocol that evaluates the rain removal algorithms on their ability to improve the performance of subsequent segmentation, instance segmentation, and feature tracking algorithms under rain and snow. If successful, the de-rained frames of a rain removal algorithm should improve segmentation performance and increase the number of accurately tracked features. The results show that a recent single-frame-based rain removal algorithm increases the segmentation performance by 19.7% on our proposed dataset, but it eventually decreases the feature tracking performance and showed mixed results with recent instance segmentation methods. However, the best video-based rain removal algorithm improves the feature tracking accuracy by 7.72%.
OriginalsprogEngelsk
Artikelnummer8510919
TidsskriftI E E E Transactions on Intelligent Transportation Systems
Vol/bind20
Udgave nummer8
Sider (fra-til)2802-2819
Antal sider18
ISSN1524-9050
DOI
StatusUdgivet - aug. 2019

Fingerprint

Rain
Snow
Computer vision
Pipelines

Citer dette

@article{72b182c41a4745c2bee7530fd77b5db9,
title = "Rain Removal in Traffic Surveillance: Does it Matter?",
abstract = "Varying weather conditions, including rainfall and snowfall, are generally regarded as a challenge for computer vision algorithms. One proposed solution to the challenges induced by rain and snowfall is to artificially remove the rain from images or video using rain removal algorithms. It is the promise of these algorithms that the rain-removed image frames will improve the performance of subsequent segmentation and tracking algorithms. However, rain removal algorithms are typically evaluated on their ability to remove synthetic rain on a small subset of images. Currently, their behavior is unknown on real-world videos when integrated with a typical computer vision pipeline. In this paper, we review the existing rain removal algorithms and propose a new dataset that consists of 22 traffic surveillance sequences under a broad variety of weather conditions that all include either rain or snowfall. We propose a new evaluation protocol that evaluates the rain removal algorithms on their ability to improve the performance of subsequent segmentation, instance segmentation, and feature tracking algorithms under rain and snow. If successful, the de-rained frames of a rain removal algorithm should improve segmentation performance and increase the number of accurately tracked features. The results show that a recent single-frame-based rain removal algorithm increases the segmentation performance by 19.7{\%} on our proposed dataset, but it eventually decreases the feature tracking performance and showed mixed results with recent instance segmentation methods. However, the best video-based rain removal algorithm improves the feature tracking accuracy by 7.72{\%}.",
keywords = "Cameras, Image segmentation, Mathematical model, Rain, Rain removal, Snow, Surveillance, image restoration, object segmentation, road transportation., snow removal, traffic surveillance",
author = "Bahnsen, {Chris Holmberg} and Moeslund, {Thomas B.}",
year = "2019",
month = "8",
doi = "10.1109/TITS.2018.2872502",
language = "English",
volume = "20",
pages = "2802--2819",
journal = "I E E E Transactions on Intelligent Transportation Systems",
issn = "1524-9050",
publisher = "IEEE",
number = "8",

}

Rain Removal in Traffic Surveillance : Does it Matter? / Bahnsen, Chris Holmberg; Moeslund, Thomas B.

I: I E E E Transactions on Intelligent Transportation Systems, Bind 20, Nr. 8, 8510919, 08.2019, s. 2802-2819.

Publikation: Bidrag til tidsskriftTidsskriftartikelForskningpeer review

TY - JOUR

T1 - Rain Removal in Traffic Surveillance

T2 - Does it Matter?

AU - Bahnsen, Chris Holmberg

AU - Moeslund, Thomas B.

PY - 2019/8

Y1 - 2019/8

N2 - Varying weather conditions, including rainfall and snowfall, are generally regarded as a challenge for computer vision algorithms. One proposed solution to the challenges induced by rain and snowfall is to artificially remove the rain from images or video using rain removal algorithms. It is the promise of these algorithms that the rain-removed image frames will improve the performance of subsequent segmentation and tracking algorithms. However, rain removal algorithms are typically evaluated on their ability to remove synthetic rain on a small subset of images. Currently, their behavior is unknown on real-world videos when integrated with a typical computer vision pipeline. In this paper, we review the existing rain removal algorithms and propose a new dataset that consists of 22 traffic surveillance sequences under a broad variety of weather conditions that all include either rain or snowfall. We propose a new evaluation protocol that evaluates the rain removal algorithms on their ability to improve the performance of subsequent segmentation, instance segmentation, and feature tracking algorithms under rain and snow. If successful, the de-rained frames of a rain removal algorithm should improve segmentation performance and increase the number of accurately tracked features. The results show that a recent single-frame-based rain removal algorithm increases the segmentation performance by 19.7% on our proposed dataset, but it eventually decreases the feature tracking performance and showed mixed results with recent instance segmentation methods. However, the best video-based rain removal algorithm improves the feature tracking accuracy by 7.72%.

AB - Varying weather conditions, including rainfall and snowfall, are generally regarded as a challenge for computer vision algorithms. One proposed solution to the challenges induced by rain and snowfall is to artificially remove the rain from images or video using rain removal algorithms. It is the promise of these algorithms that the rain-removed image frames will improve the performance of subsequent segmentation and tracking algorithms. However, rain removal algorithms are typically evaluated on their ability to remove synthetic rain on a small subset of images. Currently, their behavior is unknown on real-world videos when integrated with a typical computer vision pipeline. In this paper, we review the existing rain removal algorithms and propose a new dataset that consists of 22 traffic surveillance sequences under a broad variety of weather conditions that all include either rain or snowfall. We propose a new evaluation protocol that evaluates the rain removal algorithms on their ability to improve the performance of subsequent segmentation, instance segmentation, and feature tracking algorithms under rain and snow. If successful, the de-rained frames of a rain removal algorithm should improve segmentation performance and increase the number of accurately tracked features. The results show that a recent single-frame-based rain removal algorithm increases the segmentation performance by 19.7% on our proposed dataset, but it eventually decreases the feature tracking performance and showed mixed results with recent instance segmentation methods. However, the best video-based rain removal algorithm improves the feature tracking accuracy by 7.72%.

KW - Cameras

KW - Image segmentation

KW - Mathematical model

KW - Rain

KW - Rain removal

KW - Snow

KW - Surveillance

KW - image restoration

KW - object segmentation

KW - road transportation.

KW - snow removal

KW - traffic surveillance

UR - http://www.scopus.com/inward/record.url?scp=85055724971&partnerID=8YFLogxK

U2 - 10.1109/TITS.2018.2872502

DO - 10.1109/TITS.2018.2872502

M3 - Journal article

VL - 20

SP - 2802

EP - 2819

JO - I E E E Transactions on Intelligent Transportation Systems

JF - I E E E Transactions on Intelligent Transportation Systems

SN - 1524-9050

IS - 8

M1 - 8510919

ER -