Data reformation - A novel data processing technique enhancing machine learning applicability for predicting streamflow extremes

Abstract

Hydrologists have been actively exploring the utility of machine learning (ML) models for predicting streamflow. While ML methods have proven to be as accurate as conventional modeling techniques for streamflows well represented in the training set, they continue to lack satisfactory skills for extreme events. In this study, a novel ‘data reformation’ technique is proposed based on the Relative Strength Index (RSI) – a measure of speed and direction of changes in the time series. RSI homogenizes all observations to a constrained 0–100 range, and all ‘out-of-sample’ data in the testing set fall within the space of the training set. Long Short-Term Memory network with an attention mechanism is used to train three ML models using 55,055 events from the CAMELS dataset (670 basins, 1980–2014). Predictions are made for 12,424 events, of which 3,810 are significantly higher than streamflows in the training set. The ML model based on RSI-reformed data exhibits superior performance, as compared to other advanced ML models without data reformation. Peaks up to 15 times larger than those in the training events are accurately predicted, leading to an outperforming model skill for 433 out of 670 catchments. These findings indicate that incorporating a new data reformation technique into the data pre-processing step in ML modeling can enhance the utility of ML models for extreme events. This research encourages further exploration to identify better data reformation methods to enable confident ML predictions.