EWMA Based Two-Stage Dataset Shift-Detection in Non-stationary Environments - Artificial Intelligence Applications and Innovations Access content directly
Conference Papers Year : 2013

EWMA Based Two-Stage Dataset Shift-Detection in Non-stationary Environments

Haider Raza
  • Function : Author
  • PersonId : 1000668
Girijesh Prasad
  • Function : Author
  • PersonId : 1000669
Yuhua Li
  • Function : Author
  • PersonId : 1000670

Abstract

Dataset shift is a major challenge in the non-stationary environments wherein the input data distribution may change over time. In a time-series data, detecting the dataset shift point, where the distribution changes its properties is of utmost interest. Dataset shift exists in a broad range of real-world systems. In such systems, there is a need for continuous monitoring of the process behavior and tracking the state of the shift so as to decide about initiating adaptive corrections in a timely manner. This paper presents a novel method to detect the shift-point based on a two-stage structure involving Exponentially Weighted Moving Average (EWMA) chart and Kolmogorov-Smirnov test, which substantially reduces type-I error rate. The algorithm is suitable to be run in real-time. Its performance is evaluated through experiments using synthetic and real-world datasets. Results show effectiveness of the proposed approach in terms of decreased type-I error and tolerable increase in detection time delay.
Fichier principal
Vignette du fichier
978-3-642-41142-7_63_Chapter.pdf (502.78 Ko) Télécharger le fichier
Origin : Files produced by the author(s)
Loading...

Dates and versions

hal-01459655 , version 1 (07-02-2017)

Licence

Attribution

Identifiers

Cite

Haider Raza, Girijesh Prasad, Yuhua Li. EWMA Based Two-Stage Dataset Shift-Detection in Non-stationary Environments. 9th Artificial Intelligence Applications and Innovations (AIAI), Sep 2013, Paphos, Greece. pp.625-635, ⟨10.1007/978-3-642-41142-7_63⟩. ⟨hal-01459655⟩
114 View
604 Download

Altmetric

Share

Gmail Facebook X LinkedIn More