Bayesian likelihoods for intractable data

Abstract

We live in the Information Age, specific historical moment in which making well-informed decisions is often decisive for the survival of projects and investments. In the past decade, consistent efforts have been dedicated to studies concerning the use of AI for handling the boundless quantity of data that is created each day. Nevertheless, studying mathematical models and techniques suitable for calibrating them using consciously designed data synthesis techniques is preferable whenever possible, as the interpretation of the outcome is often much better founded. The present work explores the estimation techniques for challenging situations involving the chaotic and Stochastic Differential Equation (SDE) systems used in finance and meteorology, where the standards present in the literature struggle. In general, SDEs are a specific model type designed to describe scenarios in which the underlying dynamics of the studied events are known only partially, and the uncertain part is given solely as a distribution of possibilities. The multiple flavours of SDEs differ mainly by how the stochasticity, or randomness, is added to the model. When the system consists of a deterministic drift and a linearly added diffusion component with a fixed distribution at every time point of the system, we speak of linear SDEs. There are solutions in the literature involving filtering approaches et al. to estimate the model parameters of such models from data. It is different in those cases where the diffusion part includes ‘jumps’ from multimodal distributions. A third option is that the stochasticity is added either in front of each component of the system equation or to the parameters of the drift part, addressing the uncertainties both in the physics and in the model. Moreover, differently from purely deterministic systems, SDEs also have the additional issue of potentially fitting ‘by chance’ the values given in a reference dataset, thereby leading to a potentially ‘good looking’ solution that could mislead a not-careful-enough user to rely on a wrong model when trying to predict future outcomes of a system.

In this work, we present an emerging type of Bayesian inference based on Gaussian likelihoods that utilise squeezed data representations to address these challenges. We include an extensive series of test cases ranging from the most basic to more complex types of SDE systems to test the reliability of the approach as well as to give hints on how to tune the rather few method parameters.