LG176: Discovering and Characterizing Hidden Variables in Streaming Multivariate Time Series

Friday, January 17, 2014 - 12:30
Soumi Ray, PhD
Time series data naturally arises in many domains, such as robotics, industrial process control, finance, medicine, climatology, and many others. In many cases variables known to be causally relevant cannot be measured directly, or the existence of such variables is unknown. We present a neural network architecture, called the LO-net, for inferring both the existence and values of hidden variables in streaming multivariate time series, leading to deeper understanding of the domain and more accurate prediction. The core idea is to first make predictions with one network (the observable or O-net) based on a time delay embedding, following this with a gradual reduction in the temporal scope of the embedding, thus forcing a second network (the latent or L-net) to learn to approximate the value of a single hidden variable. This estimate is then input to the O-net based on the original time delay embedding. Experiments show the utility of this proposed approach using discrete time dynamical systems in which some of the state variables are hidden, and sensor data obtained from the camera of a mobile robot in which the sizes and locations of objects in the visual field are observed but their sizes and locations (distances) in the three-dimensional world are not. We observe in some experiments that the L-net tries to approximate the output from the O-net instead of learning the hidden variable. To avoid this, the LO-net is regularized and experiments are run using different regularization terms. We see in some situations that the regularized LO-net provides better performance improvement than the vanilla LO-net. In summary, we present a novel method for discovering and characterizing hidden variables in multivariate time series data using different dynamical systems and also real world data.