Parameter Estimation and Model Selection for Mixtures of Truncated Exponentials

Helge Langseth, Thomas Dyhre Nielsen, Rafael Rumí, Antonio Salmerón

Research output: Contribution to journalJournal articleResearchpeer-review

26 Citations (Scopus)
309 Downloads (Pure)

Abstract

Bayesian networks with mixtures of truncated exponentials (MTEs) support efficient inference algorithms and provide a
flexible way of modeling hybrid domains (domains containing both discrete and continuous variables). On the other hand, estimating an MTE from data has turned out to be a difficult task, and most prevalent learning methods treat parameter estimation as a regression problem. The drawback of this approach is that by not directly attempting to find the parameter estimates that maximize the likelihood, there is no principled way of performing subsequent model selection using those parameter estimates. In this paper we describe an estimation method that directly aims at learning the parameters of an MTE potential following a maximum likelihood approach. Empirical results demonstrate that the proposed method yields significantly better likelihood results than existing regression-based methods. We also show how model selection, which in the case of univariate MTEs amounts to partitioning the domain and selecting the number of exponential terms, can be performed using the BIC-score.
Original languageEnglish
JournalInternational Journal of Approximate Reasoning
Volume51
Issue number5
Pages (from-to)485-498
ISSN0888-613X
DOIs
Publication statusPublished - Jun 2010

Fingerprint

Model Selection
Parameter estimation
Parameter Estimation
Likelihood
Regression
Bayesian networks
Hybrid Modeling
Discrete Variables
Maximum likelihood
Continuous Variables
Bayesian Networks
Estimate
Univariate
Maximum Likelihood
Partitioning
Maximise
Term
Demonstrate
Learning

Cite this

@article{5fef43304b7b11dead10000ea68e967b,
title = "Parameter Estimation and Model Selection for Mixtures of Truncated Exponentials",
abstract = "Bayesian networks with mixtures of truncated exponentials (MTEs) support efficient inference algorithms and provide aflexible way of modeling hybrid domains (domains containing both discrete and continuous variables). On the other hand, estimating an MTE from data has turned out to be a difficult task, and most prevalent learning methods treat parameter estimation as a regression problem. The drawback of this approach is that by not directly attempting to find the parameter estimates that maximize the likelihood, there is no principled way of performing subsequent model selection using those parameter estimates. In this paper we describe an estimation method that directly aims at learning the parameters of an MTE potential following a maximum likelihood approach. Empirical results demonstrate that the proposed method yields significantly better likelihood results than existing regression-based methods. We also show how model selection, which in the case of univariate MTEs amounts to partitioning the domain and selecting the number of exponential terms, can be performed using the BIC-score.",
author = "Helge Langseth and Nielsen, {Thomas Dyhre} and Rafael Rum{\'i} and Antonio Salmer{\'o}n",
year = "2010",
month = "6",
doi = "10.1016/j.ijar.2010.01.008",
language = "English",
volume = "51",
pages = "485--498",
journal = "International Journal of Approximate Reasoning",
issn = "0888-613X",
publisher = "Elsevier",
number = "5",

}

Parameter Estimation and Model Selection for Mixtures of Truncated Exponentials. / Langseth, Helge; Nielsen, Thomas Dyhre; Rumí, Rafael; Salmerón, Antonio.

In: International Journal of Approximate Reasoning, Vol. 51, No. 5, 06.2010, p. 485-498.

Research output: Contribution to journalJournal articleResearchpeer-review

TY - JOUR

T1 - Parameter Estimation and Model Selection for Mixtures of Truncated Exponentials

AU - Langseth, Helge

AU - Nielsen, Thomas Dyhre

AU - Rumí, Rafael

AU - Salmerón, Antonio

PY - 2010/6

Y1 - 2010/6

N2 - Bayesian networks with mixtures of truncated exponentials (MTEs) support efficient inference algorithms and provide aflexible way of modeling hybrid domains (domains containing both discrete and continuous variables). On the other hand, estimating an MTE from data has turned out to be a difficult task, and most prevalent learning methods treat parameter estimation as a regression problem. The drawback of this approach is that by not directly attempting to find the parameter estimates that maximize the likelihood, there is no principled way of performing subsequent model selection using those parameter estimates. In this paper we describe an estimation method that directly aims at learning the parameters of an MTE potential following a maximum likelihood approach. Empirical results demonstrate that the proposed method yields significantly better likelihood results than existing regression-based methods. We also show how model selection, which in the case of univariate MTEs amounts to partitioning the domain and selecting the number of exponential terms, can be performed using the BIC-score.

AB - Bayesian networks with mixtures of truncated exponentials (MTEs) support efficient inference algorithms and provide aflexible way of modeling hybrid domains (domains containing both discrete and continuous variables). On the other hand, estimating an MTE from data has turned out to be a difficult task, and most prevalent learning methods treat parameter estimation as a regression problem. The drawback of this approach is that by not directly attempting to find the parameter estimates that maximize the likelihood, there is no principled way of performing subsequent model selection using those parameter estimates. In this paper we describe an estimation method that directly aims at learning the parameters of an MTE potential following a maximum likelihood approach. Empirical results demonstrate that the proposed method yields significantly better likelihood results than existing regression-based methods. We also show how model selection, which in the case of univariate MTEs amounts to partitioning the domain and selecting the number of exponential terms, can be performed using the BIC-score.

U2 - 10.1016/j.ijar.2010.01.008

DO - 10.1016/j.ijar.2010.01.008

M3 - Journal article

VL - 51

SP - 485

EP - 498

JO - International Journal of Approximate Reasoning

JF - International Journal of Approximate Reasoning

SN - 0888-613X

IS - 5

ER -