English

应用科学学报 ›› 2006, Vol. 24 ›› Issue (3): 256-261.

• 论文 • 上一篇    下一篇

数据流上异常数据的在线检测与修正

王永利1,2, 徐宏炳1, 董逸生1, 钱江波1, 刘学军1   

  1. 1. 东南大学计算机科学与工程系, 江苏南京 210018;
    2. 佳木斯大学计算机公共教研部, 黑龙江佳木斯 154007
  • 收稿日期:2005-01-20 修回日期:2005-05-24 出版日期:2006-05-31 发布日期:2006-05-31
  • 作者简介:王永利,博士生,研究方向:数据流处理、现代数据管理技术,E-mail:wyl_seu@126.com;董逸生,教授,博导,研究方向:现代数据管理、信息系统建模、体系结构、开发方向,E-mail:ysdong@seu.edu.cn
  • 基金资助:
    江苏省高技术(BG2004034);江苏省2004年度研究生创新计划(xm04-36)资助项目

Detection and Repairing Method for Outliers over Data Streams

WANG Yong-li1,2, XU Hong-bing1, DONG Yi-sheng1, QIAN Jiang-bo1, LIU Xue-jun1   

  1. 1. Department of Computer Science & Engineering, Southeast University, Nanjing 210018, China;
    2. Department of Computer for Public Teaching, Jiamusi University, Jiamusi 154007, China
  • Received:2005-01-20 Revised:2005-05-24 Online:2006-05-31 Published:2006-05-31

PDF

103

被引次数

13

摘要： 给出了带有遗忘因子改进的Kalman滤波预测算法,能够检测未来时刻的异常数据;提出了一种新颖的数据流上的异常数据修正方法,应用插值小波根据连续异常数据数量的不同,实现了可变插值尺度的异常数据修补,能够自适应修正精度.在实际电力负荷数据上的仿真实验证明这种方法可以在线准确地检测到异常数据,并能提供精确的异常数据修正.

关键词: Kalman滤波, 异常数据检测与修正, 数据流, 自适应, 插值小波

Abstract: Outlier detection and repairing in a data stream environment are discussed.An online detection method for outliers over data streams, called amnesia Kalman filtering (AKF), is proposed.In order to identify outliers, an improved Kalman filter is applied with the amnesia factor to forecast data-value in the future times tamp. A novel online adaptive repairing method for outliers over data streams, called adaptive interpolating wavelet (AdaptiveIW), is then proposed.The AdaptiveIW applies a method of variable-resolution interpolation, named the interpolation wavelet with adaptive resolution, to repair outliers, which determines interpolation resolution based on the number of continuous outliers.It adapts to different requested precision for out lier repairing over evolving data streams very well.Experimental results on actual power load data show that this method can provide precise instantaneous detection and accurate repairing for outliers over data streams.

Key words: adaptive, data streams, outlier detection and repairing, Kalman filtering, interpolating wavelet

中图分类号: