A New Approach for Concept Drift Detection in Visual Data

Abstract

The problem of concept drift, when the underlying data distribution changes over time, poses major challenges to the implementation of machine learning models across many domains. Recent research has investigated the detection of drifting samples by computing a non-conformity measure based on a distance function combined with dimensionality reduction to aid in the handling of high-dimensional data. However, while these new methods are effective for structured datasets, similar approaches for unstructured datasets remain unexplored. In this research, we address the problem of concept drift detection for image data in a multi-class classification setting, caused by the introduction of new classes. We propose a new pipeline method to monitor incoming data streams and detect drifting samples that deviate from the original training distribution. Our approach combines a pre-trained transformer model with a dimension reduction algorithm to extract meaningful latent representations for images. From these representations, we employ a distance-based algorithm that detects outliers and takes into account different levels of tightness among different classes. Through various experiments and evaluations on image datasets, we prove the effectiveness of our method in detecting drifting samples and addressing concept drift in classification tasks, as well as providing semantical insights into data distribution.