This repository contains all codes, datasets and results for the paper Conditionally Tractable Density Estimation using Neural Networks published on AISTATS 2022.
In this work, we propose a novel probabilistic model that factorizes the full joint distribution as the product of a local, complex distribution over a small subset of variables and a fully tractable conditional distribution whose parameters are controlled using a neural network.
This model admits exact inference when all variables in the local distribution are observed, and although the model is not fully tractable in general, we show that “cutset” sampling can be employed to efficiently generate accurate predictions in practice.
Some of the experiments and features may only works if you are using Linux system (MAC OS may work as well, but not Windows).
- Clone this repository to your local disk.
- Download the zipped dataset from Here and extract all content into dataset folder.
- Install the required packages show in
reqs.txt - Source
core/set_env.profilebefore executing any command.
We use main.py as the entry scripts for running our experiment. The usage of this script can be found by running the script without providing any command line arguments (i.e. run python main.py).
Similarly, we use run.py to run a set of experiments sequentially and automatically save all the results into corresponding directories.
All command is assumed to execute at directory experiment unless otherwise noted.
Predictive performance against other models
The relation of Loglikelihood and Prediction accuracy (RMSE) is only correlated for models in the same family.
We provided the trained instance of all models (SPN, RNADE, OURS) in the experiement/output directory. From which you can fully recover our results shown in the paper. Retrain the whole model do incur slightly difference, but the overall results are same.
Please cite our work if you find it is helpful for your research!
@InProceedings{pmlr-v151-dong22a,
title = { Conditionally Tractable Density Estimation using Neural Networks },
author = {Dong, Hailiang and Roy, Chiradeep and Rahman, Tahrima and Gogate, Vibhav and Ruozzi, Nicholas},
booktitle = {Proceedings of The 25th International Conference on Artificial Intelligence and Statistics},
pages = {6933--6946},
year = {2022},
editor = {Camps-Valls, Gustau and Ruiz, Francisco J. R. and Valera, Isabel},
volume = {151},
series = {Proceedings of Machine Learning Research},
month = {28--30 Mar},
publisher = {PMLR},
pdf = {https://proceedings.mlr.press/v151/dong22a/dong22a.pdf},
url = {https://proceedings.mlr.press/v151/dong22a.html},
abstract = { Tractable models such as cutset networks and sum-product networks (SPNs) have become increasingly popular because they have superior predictive performance. Among them, cutset networks, which model the mechanics of Pearl’s cutset conditioning algorithm, demonstrate great scalability and prediction accuracy. Existing research on cutset networks has mainly focused on discrete domains, and the best mechanism to extend cutset networks to continuous domains is unclear. We propose one possible alternative to cutset networks that models the full joint distribution as the product of a local, complex distribution over a small subset of variables and a fully tractable conditional distribution whose parameters are controlled using a neural network. This model admits exact inference when all variables in the local distribution are observed, and although the model is not fully tractable in general, we show that “cutset” sampling can be employed to efficiently generate accurate predictions in practice. We show that our model performs comparably or better than existing competitors through a variety of prediction tasks on real datasets. }
}
If you have any questions or need help regarding our work, you can email us and we are happy to discuss the work (the email addresses of each author are included in the paper).
In case my school email being deactivated, you can email me using my personal email address HailiangDong@hotmail.com.