Uncertainty quantification for variational Bayesian dropout based deep bidirectional LSTM networks

Abstract

Time series classification is a critical task in various domains, requiring robust models to handle inherent uncertainties in temporal data. These uncertainties, categorized as aleatoric and epistemic, pose significant challenges in achieving accurate predictions. In real-world applications, models often encounter unseen data that were not present during training process. Bayesian inference has been widely utilized for uncertainty quantification in statistics and machine learning. In this study, we proposed a Bayesian Deep Bi-LSTM model incorporating Variational Bayesian dropout with a Gaussian prior and Variational Autoencoder (VAE). The proposed technique efficiently handles uncertainty in both the model and data while VAE reducing the dimensionality of model parameters. We apply this framework to univariate time series datasets from the UCR repository and compare its performance with four traditional machine learning methods and four sequential deep learning models. Experimental results demonstrate that the Bayesian deep Bi-LSTM model effectively improves overall classification performance. In particular, the model benefits significantly from data augmentation using SMOTE when handling imbalanced dataset. The Variational Bayesian dropout model exhibits lower total uncertainty across both datasets, indicating more stable and reliable predictions compared to the VAE-based model. Future research should explore additional datasets from the UCR repository and investigate advanced uncertainty modeling techniques to further enhance performance and scalability.