Multimodal Heartbeat Rate Estimation from the Fusion of Facial RGB and Thermal Videos

Publikation: Bidrag til bog/antologi/rapport/konference proceedingKonferenceartikel i proceedingForskningpeer review

125 Downloads (Pure)

Resumé

Measuring Heartbeat Rate (HR) is an important tool for monitoring the health of a person. When the heart beats the influx of blood to the head causes slight involuntary movement and subtle skin color changes, which cannot be seen by the naked eye but can be tracked from facial videos using computer vision techniques and can be analyzed to estimate the HR. However, the current state of the art solutions encounter an increasing amount of complications when the subject has voluntary motion on the face or when the lighting conditions change in the video. Thus the accuracy of the HR estimation using computer vision is still inferior to that of a physical Electrocardiography (ECG) based system. The aim of this work is to improve the current non-invasive HR measurement by fusing the motion-based and color-based HR estimation methods and using them on multiple input modalities, e.g., RGB and thermal imaging. Our experiments indicate that late-fusion of the results of these methods (motion and color-based) applied to these different modalities, produces more accurate results compared to the existing solutions.
OriginalsprogEngelsk
Titel11th International Conference on Machine Vision (ICMV)
Antal sider8
Vol/bind1104101
ForlagSPIE - International Society for Optical Engineering
Publikationsdato2019
Sider110410R
StatusUdgivet - 2019
BegivenhedThe 11th International Conference on Machine Vision - Munich, Tyskland
Varighed: 1 nov. 20183 nov. 2018

Konference

KonferenceThe 11th International Conference on Machine Vision
LandTyskland
ByMunich
Periode01/11/201803/11/2018

Fingerprint

Fusion reactions
Color
Computer vision
Infrared imaging
Electrocardiography
Skin
Blood
Lighting
Health
Monitoring
Hot Temperature
Experiments

Emneord

    Citer dette

    Johansen, A. S., Henriksen, J. W., Haque, M. A., Jahromi, M. N. S., Nasrollahi, K., & Moeslund, T. B. (2019). Multimodal Heartbeat Rate Estimation from the Fusion of Facial RGB and Thermal Videos. I 11th International Conference on Machine Vision (ICMV) (Bind 1104101, s. 110410R ). SPIE - International Society for Optical Engineering.
    Johansen, Anders Skaarup ; Henriksen, Jesper Wædeled ; Haque, Mohammad Ahsanul ; Jahromi, Mohammad Naser Sabet ; Nasrollahi, Kamal ; Moeslund, Thomas B. / Multimodal Heartbeat Rate Estimation from the Fusion of Facial RGB and Thermal Videos. 11th International Conference on Machine Vision (ICMV). Bind 1104101 SPIE - International Society for Optical Engineering, 2019. s. 110410R
    @inproceedings{b64f33f5ab9d4d0d8470f21e0c2e634d,
    title = "Multimodal Heartbeat Rate Estimation from the Fusion of Facial RGB and Thermal Videos",
    abstract = "Measuring Heartbeat Rate (HR) is an important tool for monitoring the health of a person. When the heart beats the influx of blood to the head causes slight involuntary movement and subtle skin color changes, which cannot be seen by the naked eye but can be tracked from facial videos using computer vision techniques and can be analyzed to estimate the HR. However, the current state of the art solutions encounter an increasing amount of complications when the subject has voluntary motion on the face or when the lighting conditions change in the video. Thus the accuracy of the HR estimation using computer vision is still inferior to that of a physical Electrocardiography (ECG) based system. The aim of this work is to improve the current non-invasive HR measurement by fusing the motion-based and color-based HR estimation methods and using them on multiple input modalities, e.g., RGB and thermal imaging. Our experiments indicate that late-fusion of the results of these methods (motion and color-based) applied to these different modalities, produces more accurate results compared to the existing solutions.",
    keywords = "Heartbeat, Heartbeat rate, Facial video, RGB, Video, Thermal, Multimodal, PPG, Camera",
    author = "Johansen, {Anders Skaarup} and Henriksen, {Jesper W{\ae}deled} and Haque, {Mohammad Ahsanul} and Jahromi, {Mohammad Naser Sabet} and Kamal Nasrollahi and Moeslund, {Thomas B.}",
    year = "2019",
    language = "English",
    volume = "1104101",
    pages = "110410R",
    booktitle = "11th International Conference on Machine Vision (ICMV)",
    publisher = "SPIE - International Society for Optical Engineering",
    address = "United States",

    }

    Johansen, AS, Henriksen, JW, Haque, MA, Jahromi, MNS, Nasrollahi, K & Moeslund, TB 2019, Multimodal Heartbeat Rate Estimation from the Fusion of Facial RGB and Thermal Videos. i 11th International Conference on Machine Vision (ICMV). bind 1104101, SPIE - International Society for Optical Engineering, s. 110410R , Munich, Tyskland, 01/11/2018.

    Multimodal Heartbeat Rate Estimation from the Fusion of Facial RGB and Thermal Videos. / Johansen, Anders Skaarup ; Henriksen, Jesper Wædeled; Haque, Mohammad Ahsanul; Jahromi, Mohammad Naser Sabet; Nasrollahi, Kamal; Moeslund, Thomas B.

    11th International Conference on Machine Vision (ICMV). Bind 1104101 SPIE - International Society for Optical Engineering, 2019. s. 110410R .

    Publikation: Bidrag til bog/antologi/rapport/konference proceedingKonferenceartikel i proceedingForskningpeer review

    TY - GEN

    T1 - Multimodal Heartbeat Rate Estimation from the Fusion of Facial RGB and Thermal Videos

    AU - Johansen, Anders Skaarup

    AU - Henriksen, Jesper Wædeled

    AU - Haque, Mohammad Ahsanul

    AU - Jahromi, Mohammad Naser Sabet

    AU - Nasrollahi, Kamal

    AU - Moeslund, Thomas B.

    PY - 2019

    Y1 - 2019

    N2 - Measuring Heartbeat Rate (HR) is an important tool for monitoring the health of a person. When the heart beats the influx of blood to the head causes slight involuntary movement and subtle skin color changes, which cannot be seen by the naked eye but can be tracked from facial videos using computer vision techniques and can be analyzed to estimate the HR. However, the current state of the art solutions encounter an increasing amount of complications when the subject has voluntary motion on the face or when the lighting conditions change in the video. Thus the accuracy of the HR estimation using computer vision is still inferior to that of a physical Electrocardiography (ECG) based system. The aim of this work is to improve the current non-invasive HR measurement by fusing the motion-based and color-based HR estimation methods and using them on multiple input modalities, e.g., RGB and thermal imaging. Our experiments indicate that late-fusion of the results of these methods (motion and color-based) applied to these different modalities, produces more accurate results compared to the existing solutions.

    AB - Measuring Heartbeat Rate (HR) is an important tool for monitoring the health of a person. When the heart beats the influx of blood to the head causes slight involuntary movement and subtle skin color changes, which cannot be seen by the naked eye but can be tracked from facial videos using computer vision techniques and can be analyzed to estimate the HR. However, the current state of the art solutions encounter an increasing amount of complications when the subject has voluntary motion on the face or when the lighting conditions change in the video. Thus the accuracy of the HR estimation using computer vision is still inferior to that of a physical Electrocardiography (ECG) based system. The aim of this work is to improve the current non-invasive HR measurement by fusing the motion-based and color-based HR estimation methods and using them on multiple input modalities, e.g., RGB and thermal imaging. Our experiments indicate that late-fusion of the results of these methods (motion and color-based) applied to these different modalities, produces more accurate results compared to the existing solutions.

    KW - Heartbeat

    KW - Heartbeat rate

    KW - Facial video

    KW - RGB

    KW - Video

    KW - Thermal

    KW - Multimodal

    KW - PPG

    KW - Camera

    M3 - Article in proceeding

    VL - 1104101

    SP - 110410R

    BT - 11th International Conference on Machine Vision (ICMV)

    PB - SPIE - International Society for Optical Engineering

    ER -

    Johansen AS, Henriksen JW, Haque MA, Jahromi MNS, Nasrollahi K, Moeslund TB. Multimodal Heartbeat Rate Estimation from the Fusion of Facial RGB and Thermal Videos. I 11th International Conference on Machine Vision (ICMV). Bind 1104101. SPIE - International Society for Optical Engineering. 2019. s. 110410R