Early in the 20th century, experts tried to forecast the weather by noting current conditions, patterns of winds, temperatures and air pressures, and looking into historical records to find previous moments when similar conditions prevailed. Looking a few days forward in the records, they could then make predictions by assuming the atmosphere would evolve as it had in the past. This was a fully data-centric approach to weather prediction, exploiting no actual knowledge of the mathematical laws of atmospheric physics.
The field later shifted to prediction based on computational simulation, aiming to explicitly approximate the true atmospheric physics, and today this is the dominant technique. Yet in other areas of science a tension endures between different approaches to prediction, based either on knowledge of the underlying system dynamics, or instead on raw data gathering and analysis. The latter technique has become particularly popular given advances in machine learning.
A natural question is whether there may be scope for improving prediction methods in general by combining the two methods – bringing knowledge and data together in some coherent way. In a recent paper, LML Fellow Michelle Girvan and colleagues have explored this idea, examining how a hybrid prediction scheme that combines knowledge-based and machine learning approaches performs in predicting the future state of a low dimensional chaotic system: the Lorenz system of equations. Surprisingly, they find that even when the two methods used on their own give poor results, the hybrid technique works much better. Even for a chaotic system, it makes useful predictions over relatively long times.
The figure below (figure 4 from the paper) shows the typical error of the hybrid prediction scheme vs the time length of the attempted prediction. Here the error is defined as the magnitude of the difference between the predicted and true values, normalized by the root mean value of this quantity through time. For the hybrid predictor, the error remains below a defined threshold for roughly 10-12 Lyapunov times. In contrast, the prediction errors of both the knowledge-based and machine learning methods used on their own grow much faster. The machine learning method predicts well for about 4 Lyapunov times, and the model alone for only one.
The authors also examined how the hybrid scheme worked when applied to a system exhibiting spatio-temporal chaos, the Kuramoto-Sivashinsky equations. In particular, they examined how the hybrid method fared in the case in which the model used was a very poor approximation of the true underlying equations. Remarkably, even when the model was so poor, the hybrid made accurate predictions over relatively long times, suggesting that highly inaccurate models can be improved by including a component of machine learning.
The authors suggest that the proposed hybrid scheme could find wide applicability in many areas of science and technology. Examples of potential areas of application include network anomaly detection, assessment of credit ratings or modelling of complex chemical processes. Importantly, the hybrid methods should offer improvements even if the nature of the inaccuracies in an underlying model are completely unknown.
Of course, in predicting the evolution of a system, one might expect the best results when making use of all available information. The hybrid systems introduced here work by making an optimal weighting of the information coming from the independent knowledge-based and data-centric components. But the magnitude of improvement was surprising even to the authors, who expected the prediction time for the hybrid method to be near the sum of the prediction times for the two methods. It was instead twice that large.
The paper is available at https://aip.scitation.org/doi/10.1063/1.5028373 or as a preprint: https://arxiv.org/abs/1803.04779
Leave a Reply