Probabilistic Models with Deep Neural Networks

Andres Masegosa, Rafael Cabañas, Helge Langseth, Thomas Dyhre Nielsen, Antonio Salmerón

Research output: Contribution to journalReview articlepeer-review

4 Citations (Scopus)
58 Downloads (Pure)

Abstract

Recent advances in statistical inference have significantly expanded the toolbox of probabilistic modeling. Historically, probabilistic modeling has been constrained to very restricted model classes, where exact or approximate probabilistic inference is feasible. However, developments in variational inference, a general form of approximate probabilistic inference that originated in statistical physics, have enabled probabilistic modeling to overcome these limitations: (i) Approximate probabilistic inference is now possible over a broad class of probabilistic models containing a large number of parameters, and (ii) scalable inference methods based on stochastic gradient descent and distributed computing engines allow probabilistic modeling to be applied to massive data sets. One important practical consequence of these advances is the possibility to include deep neural networks within probabilistic models, thereby capturing complex non-linear stochastic relationships between the random variables. These advances, in conjunction with the release of novel probabilistic modeling toolboxes, have greatly expanded the scope of applications of probabilistic models, and allowed the models to take advantage of the recent strides made by the deep learning community. In this paper, we provide an overview of the main concepts, methods, and tools needed to use deep neural networks within a probabilistic modeling framework.
Original languageEnglish
Article number117
JournalEntropy
Volume23
Issue number1
Pages (from-to)1-27
Number of pages27
ISSN1099-4300
DOIs
Publication statusPublished - 2021

Keywords

  • Bayesian learning
  • Deep probabilistic modeling
  • Latent variable models
  • Neural networks
  • Variational inference

Fingerprint

Dive into the research topics of 'Probabilistic Models with Deep Neural Networks'. Together they form a unique fingerprint.

Cite this