Evaluation-Free Time-Series Forecasting Model Selection via Meta-Learning
MUSTAFA ABDALLAH, Indiana University-Purdue University Indianapolis, USA
RYAN A. ROSSI, Adobe Research, USA
KANAK MAHADIK, Adobe Research, USA
SUNGCHUL KIM, Adobe Research, USA
HANDONG ZHAO, Adobe Research, USA
SAURABH BAGCHI, Purdue University, USA

Time-series forecasting models are invariably used in a variety of domains for crucial decision-making.
Traditionally these models are constructed by experts with considerable manual effort. Unfortunately, this
approach has poor scalability while generating accurate forecasts for new datasets belonging to diverse
applications. Without access to skilled domain-knowledge, one approach is to train all the models on the new
time-series data and then select the best one. However, this approach is nonviable in practice. In this work, we
develop techniques for fast automatic selection of the best forecasting model for a new unseen time-series
dataset, without having to first train (or evaluate) all the models on the new time-series data to select the best
one. In particular, we develop a forecasting meta-learning approach called AutoForecast that allows for the
quick inference of the best time-series forecasting model for an unseen dataset. Our approach learns both
forecasting models performances over time horizon of the same dataset and task similarity across different
datasets. The experiments demonstrate the effectiveness of the approach over state-of-the-art (SOTA) single
and ensemble methods and several SOTA meta-learners (adapted to our problem) in terms of selecting better
forecasting models (i.e., 2X gain) for unseen tasks for univariate and multivariate testbeds. We release our
meta-learning database corpus (348 datasets), performances of the 322 forecasting models on the database
corpus, meta-features, and source codes for the community to access them for forecasting model selection and
to build on them with new datasets and models which can help advance automating time-series forecasting
problem. In our released database corpus, we are unveiling new traces of Adobe computing cluster usage for
production workloads.