Enabling the CUDA Unified Memory model in Edge, Cloud and HPC offloaded GPU kernels

Raffaele Montella; Diana Di Luccio; Ciro Giuseppe De Vita; Gennaro Mellone; Marco Lapegna; Giuliano Laccetti; Sokol Kosta; Giulio Giunta

doi:10.1109/CCGrid54584.2022.00099

Enabling the CUDA Unified Memory model in Edge, Cloud and HPC offloaded GPU kernels

Raffaele Montella, Diana Di Luccio, Ciro Giuseppe De Vita, Gennaro Mellone, Marco Lapegna, Giuliano Laccetti, Sokol Kosta, Giulio Giunta

Research output: Contribution to book/anthology/report/conference proceeding › Article in proceeding › Research › peer-review

1 Citation (Scopus)

Abstract

The use of hardware accelerators, based on code and data offloading devoted to overcoming the CPU limitations in cores, is one of the main distinctive trends in high-end computing and related applications in the last decade. However, while code offloading is convenient for performance improvement, becoming a commonly used paradigm, memory access and management are a source of bottlenecks due to the need to interact with different address spaces. In this regard, NVidia introduced the CUDA Unified Memory model to avoid explicit memory copies between the machine hosting the accelerator device and the device itself and vice-versa. This paper shows a novel design and implementation of the support to the CUDA Unified Memory in open-source GPGPU virtualization services. The performance evaluation demonstrates that the overhead due to the virtualization and remoting is acceptable considering the possibility of sharing CUDA-enabled GPUs between various and heterogeneous machines hosted at the edge, in cloud infrastructures, or as accelerator nodes in an HPC scenario. A prototype implementation of the proposed solution is available as open-source.

Original language	English
Title of host publication	Proceedings - 22nd IEEE/ACM International Symposium on Cluster, Cloud and Internet Computing, CCGrid 2022
Editors	Maria Fazio, Dhabaleswar K. Panda, Radu Prodan, Valeria Cardellini, Burak Kantarci, Omer Rana, Massimo Villari
Number of pages	8
Publisher	IEEE Signal Processing Society
Publication date	2022
Pages	834-841
ISBN (Print)	978-1-6654-9957-6
ISBN (Electronic)	978-1-6654-9956-9
DOIs	https://doi.org/10.1109/CCGrid54584.2022.00099
Publication status	Published - 2022
Event	22nd IEEE/ACM International Symposium on Cluster, Cloud and Internet Computing, CCGrid 2022 - Taormina, Italy Duration: 16 May 2022 → 19 May 2022

Conference

Conference	22nd IEEE/ACM International Symposium on Cluster, Cloud and Internet Computing, CCGrid 2022
Country/Territory	Italy
City	Taormina
Period	16/05/2022 → 19/05/2022

Series	Proceedings - 22nd IEEE/ACM International Symposium on Cluster, Cloud and Internet Computing, CCGrid 2022

Bibliographical note

Funding Information:
This work is supported in part by the grant ADMIRE (Adaptive Multi-tier intelligent data manager for Exascale - H2020-JTI-EuroHPC-2019-1), in part by the grant MytilAI (Modelling mytilus farming with Artificial Intelligence technologies - CUP I65F21000040002), and it is conducted in the framework of the research agreement “High-Performance Computing at the Edge” between the Department of Mathematics and Applications of the University of Naples Federico II and the Department of Sciences and Technologies of the University of Naples Parthenope.

Publisher Copyright:
© 2022 IEEE.

Keywords

CUDA
GPU
offloading
remoting
unified memory access
virtualization

Access to Document

10.1109/CCGrid54584.2022.00099

AUB Link

Search for the material in Aalborg University Library's search engine

Cite this

Montella, R., Di Luccio, D., De Vita, C. G., Mellone, G., Lapegna, M., Laccetti, G., Kosta, S., & Giunta, G. (2022). Enabling the CUDA Unified Memory model in Edge, Cloud and HPC offloaded GPU kernels. In M. Fazio, D. K. Panda, R. Prodan, V. Cardellini, B. Kantarci, O. Rana, & M. Villari (Eds.), Proceedings - 22nd IEEE/ACM International Symposium on Cluster, Cloud and Internet Computing, CCGrid 2022 (pp. 834-841). IEEE Signal Processing Society. https://doi.org/10.1109/CCGrid54584.2022.00099

Montella, Raffaele ; Di Luccio, Diana ; De Vita, Ciro Giuseppe et al. / Enabling the CUDA Unified Memory model in Edge, Cloud and HPC offloaded GPU kernels. Proceedings - 22nd IEEE/ACM International Symposium on Cluster, Cloud and Internet Computing, CCGrid 2022. editor / Maria Fazio ; Dhabaleswar K. Panda ; Radu Prodan ; Valeria Cardellini ; Burak Kantarci ; Omer Rana ; Massimo Villari. IEEE Signal Processing Society, 2022. pp. 834-841 (Proceedings - 22nd IEEE/ACM International Symposium on Cluster, Cloud and Internet Computing, CCGrid 2022).

@inproceedings{db55a3a7441c498ea2ec33c8ce4ccd5a,

title = "Enabling the CUDA Unified Memory model in Edge, Cloud and HPC offloaded GPU kernels",

abstract = "The use of hardware accelerators, based on code and data offloading devoted to overcoming the CPU limitations in cores, is one of the main distinctive trends in high-end computing and related applications in the last decade. However, while code offloading is convenient for performance improvement, becoming a commonly used paradigm, memory access and management are a source of bottlenecks due to the need to interact with different address spaces. In this regard, NVidia introduced the CUDA Unified Memory model to avoid explicit memory copies between the machine hosting the accelerator device and the device itself and vice-versa. This paper shows a novel design and implementation of the support to the CUDA Unified Memory in open-source GPGPU virtualization services. The performance evaluation demonstrates that the overhead due to the virtualization and remoting is acceptable considering the possibility of sharing CUDA-enabled GPUs between various and heterogeneous machines hosted at the edge, in cloud infrastructures, or as accelerator nodes in an HPC scenario. A prototype implementation of the proposed solution is available as open-source.",

keywords = "CUDA, GPU, offloading, remoting, unified memory access, virtualization",

author = "Raffaele Montella and {Di Luccio}, Diana and {De Vita}, {Ciro Giuseppe} and Gennaro Mellone and Marco Lapegna and Giuliano Laccetti and Sokol Kosta and Giulio Giunta",

note = "Funding Information: This work is supported in part by the grant ADMIRE (Adaptive Multi-tier intelligent data manager for Exascale - H2020-JTI-EuroHPC-2019-1), in part by the grant MytilAI (Modelling mytilus farming with Artificial Intelligence technologies - CUP I65F21000040002), and it is conducted in the framework of the research agreement “High-Performance Computing at the Edge” between the Department of Mathematics and Applications of the University of Naples Federico II and the Department of Sciences and Technologies of the University of Naples Parthenope. Publisher Copyright: {\textcopyright} 2022 IEEE.; 22nd IEEE/ACM International Symposium on Cluster, Cloud and Internet Computing, CCGrid 2022 ; Conference date: 16-05-2022 Through 19-05-2022",

year = "2022",

doi = "10.1109/CCGrid54584.2022.00099",

language = "English",

isbn = "978-1-6654-9957-6",

series = "Proceedings - 22nd IEEE/ACM International Symposium on Cluster, Cloud and Internet Computing, CCGrid 2022",

pages = "834--841",

editor = "Maria Fazio and Panda, {Dhabaleswar K.} and Radu Prodan and Valeria Cardellini and Burak Kantarci and Omer Rana and Massimo Villari",

booktitle = "Proceedings - 22nd IEEE/ACM International Symposium on Cluster, Cloud and Internet Computing, CCGrid 2022",

publisher = "IEEE Signal Processing Society",

address = "United States",

}

Montella, R, Di Luccio, D, De Vita, CG, Mellone, G, Lapegna, M, Laccetti, G, Kosta, S & Giunta, G 2022, Enabling the CUDA Unified Memory model in Edge, Cloud and HPC offloaded GPU kernels. in M Fazio, DK Panda, R Prodan, V Cardellini, B Kantarci, O Rana & M Villari (eds), Proceedings - 22nd IEEE/ACM International Symposium on Cluster, Cloud and Internet Computing, CCGrid 2022. IEEE Signal Processing Society, Proceedings - 22nd IEEE/ACM International Symposium on Cluster, Cloud and Internet Computing, CCGrid 2022, pp. 834-841, 22nd IEEE/ACM International Symposium on Cluster, Cloud and Internet Computing, CCGrid 2022, Taormina, Italy, 16/05/2022. https://doi.org/10.1109/CCGrid54584.2022.00099

Enabling the CUDA Unified Memory model in Edge, Cloud and HPC offloaded GPU kernels. / Montella, Raffaele; Di Luccio, Diana; De Vita, Ciro Giuseppe et al.
Proceedings - 22nd IEEE/ACM International Symposium on Cluster, Cloud and Internet Computing, CCGrid 2022. ed. / Maria Fazio; Dhabaleswar K. Panda; Radu Prodan; Valeria Cardellini; Burak Kantarci; Omer Rana; Massimo Villari. IEEE Signal Processing Society, 2022. p. 834-841 (Proceedings - 22nd IEEE/ACM International Symposium on Cluster, Cloud and Internet Computing, CCGrid 2022).

Research output: Contribution to book/anthology/report/conference proceeding › Article in proceeding › Research › peer-review

TY - GEN

T1 - Enabling the CUDA Unified Memory model in Edge, Cloud and HPC offloaded GPU kernels

AU - Montella, Raffaele

AU - Di Luccio, Diana

AU - De Vita, Ciro Giuseppe

AU - Mellone, Gennaro

AU - Lapegna, Marco

AU - Laccetti, Giuliano

AU - Kosta, Sokol

AU - Giunta, Giulio

N1 - Funding Information: This work is supported in part by the grant ADMIRE (Adaptive Multi-tier intelligent data manager for Exascale - H2020-JTI-EuroHPC-2019-1), in part by the grant MytilAI (Modelling mytilus farming with Artificial Intelligence technologies - CUP I65F21000040002), and it is conducted in the framework of the research agreement “High-Performance Computing at the Edge” between the Department of Mathematics and Applications of the University of Naples Federico II and the Department of Sciences and Technologies of the University of Naples Parthenope. Publisher Copyright: © 2022 IEEE.

PY - 2022

Y1 - 2022

N2 - The use of hardware accelerators, based on code and data offloading devoted to overcoming the CPU limitations in cores, is one of the main distinctive trends in high-end computing and related applications in the last decade. However, while code offloading is convenient for performance improvement, becoming a commonly used paradigm, memory access and management are a source of bottlenecks due to the need to interact with different address spaces. In this regard, NVidia introduced the CUDA Unified Memory model to avoid explicit memory copies between the machine hosting the accelerator device and the device itself and vice-versa. This paper shows a novel design and implementation of the support to the CUDA Unified Memory in open-source GPGPU virtualization services. The performance evaluation demonstrates that the overhead due to the virtualization and remoting is acceptable considering the possibility of sharing CUDA-enabled GPUs between various and heterogeneous machines hosted at the edge, in cloud infrastructures, or as accelerator nodes in an HPC scenario. A prototype implementation of the proposed solution is available as open-source.

AB - The use of hardware accelerators, based on code and data offloading devoted to overcoming the CPU limitations in cores, is one of the main distinctive trends in high-end computing and related applications in the last decade. However, while code offloading is convenient for performance improvement, becoming a commonly used paradigm, memory access and management are a source of bottlenecks due to the need to interact with different address spaces. In this regard, NVidia introduced the CUDA Unified Memory model to avoid explicit memory copies between the machine hosting the accelerator device and the device itself and vice-versa. This paper shows a novel design and implementation of the support to the CUDA Unified Memory in open-source GPGPU virtualization services. The performance evaluation demonstrates that the overhead due to the virtualization and remoting is acceptable considering the possibility of sharing CUDA-enabled GPUs between various and heterogeneous machines hosted at the edge, in cloud infrastructures, or as accelerator nodes in an HPC scenario. A prototype implementation of the proposed solution is available as open-source.

KW - CUDA

KW - GPU

KW - offloading

KW - remoting

KW - unified memory access

KW - virtualization

UR - http://www.scopus.com/inward/record.url?scp=85135763356&partnerID=8YFLogxK

U2 - 10.1109/CCGrid54584.2022.00099

DO - 10.1109/CCGrid54584.2022.00099

M3 - Article in proceeding

AN - SCOPUS:85135763356

SN - 978-1-6654-9957-6

T3 - Proceedings - 22nd IEEE/ACM International Symposium on Cluster, Cloud and Internet Computing, CCGrid 2022

SP - 834

EP - 841

BT - Proceedings - 22nd IEEE/ACM International Symposium on Cluster, Cloud and Internet Computing, CCGrid 2022

A2 - Fazio, Maria

A2 - Panda, Dhabaleswar K.

A2 - Prodan, Radu

A2 - Cardellini, Valeria

A2 - Kantarci, Burak

A2 - Rana, Omer

A2 - Villari, Massimo

PB - IEEE Signal Processing Society

T2 - 22nd IEEE/ACM International Symposium on Cluster, Cloud and Internet Computing, CCGrid 2022

Y2 - 16 May 2022 through 19 May 2022

ER -

Montella R, Di Luccio D, De Vita CG, Mellone G, Lapegna M, Laccetti G et al. Enabling the CUDA Unified Memory model in Edge, Cloud and HPC offloaded GPU kernels. In Fazio M, Panda DK, Prodan R, Cardellini V, Kantarci B, Rana O, Villari M, editors, Proceedings - 22nd IEEE/ACM International Symposium on Cluster, Cloud and Internet Computing, CCGrid 2022. IEEE Signal Processing Society. 2022. p. 834-841. (Proceedings - 22nd IEEE/ACM International Symposium on Cluster, Cloud and Internet Computing, CCGrid 2022). doi: 10.1109/CCGrid54584.2022.00099

Enabling the CUDA Unified Memory model in Edge, Cloud and HPC offloaded GPU kernels

Abstract

Conference

Bibliographical note

Keywords

Access to Document

AUB Link

Other files and links

Fingerprint

CUDA virtualization and remoting for GPGPU based acceleration offloading at the edge

A virtualized software based on the NVIDIA cuFFT library for image denoising: performance analysis

Cite this