In order to process the massive agricultural environmental data stored in distributed system and improve the production efciency, the clustering algorithm based on Gaussian Mixture Model (GMM) is modifed in this paper. Based on this, an environmental anomaly detection method during crop growth is proposed. Under the Spark distributed computing framework, frstly, a pre-clustering algorithm is adopted to initialize the models. Secondly, Spark is utilized to update the models iterationally until it gets stable. In each iteration, Map phase distributes sample points to the models, Reduce phase renews the numbers of models and parameters. Finally, the detection of environmental anomaly is completed by taking advantages of the clustering result. The experimental results show that this approach is practically feasible and effective.
YU Yue, DENG Li, PANG Hong-lin, FEI Min-rui
. Environmental Anomaly Detection Method during Crop Growth Based on Distributed Clustering[J]. Journal of Applied Sciences, 2018
, 36(6)
: 1010
-1021
.
DOI: 10.3969/j.issn.0255-8297.2018.06.013
[1] Ruß G, Kruse R, Scheider M. A clustering approach for management zone delineation in precision agriculture[J]. Journal of Applied Physics, 2010, 26(9):1165-1172.
[2] Russ G, Kruse R. Machine learning methods for spatial clustering on precision agriculture data[C]//Scandinavian Conference on Artifcial Intelligence, 2011, 227:40-49.
[3] Cao L, Zhang X, San X, Zhao Y, Chen G. Application of fuzzy clustering algorithm in precision agriculture[C]//2012 IEEE World Automation Congress (WAC), 2012:1-4.
[4] Ananthara M G, Arunkumar T, Hemavathy R. CRY-An improved crop yield prediction model using bee hive clustering approach for agricultural data sets[C]//IEEE International Conference on Pattern Recognition, Informatics and Mobile Engineering, 2013:473-478.
[5] Wu M, Wang Y, Liao Z. A new clustering algorithm for sensor data streams in an agricultural IoT[C]//IEEE International Conference on High Performance Computing and Communications, Zhangjiajie. 2013:2373-2378.
[6] 时雷. 基于物联网的小麦生长环境数据采集与数据挖掘技术研究[D]. 郑州:河南农业大学,2013.
[7] 曹振丽. 面向养殖环境监测的数据流处理方法研究[D]. 北京:中国农业大学,2015.
[8] Oh S H, Lee W S. An anomaly intrusion detection method by clustering normal user behavior[J]. Computers & Security, 2003, 22(7):596-612.
[9] Li H, Achim A, Bull D. Unsupervised video anomaly detection using feature clustering[J]. IET Signal Processing, 2012, 6(5):521-533.
[10] Ahmed M, Mahmood A N, Maher M J. Heart disease diagnosis using co-clustering[M]. Germany:Springer International Publishing, 2015:61-70.
[11] Bilmes J A. A gentle tutorial of the EM algorithm and its application to parameter estimation for Gaussian mixture and hidden Markov models[R]. International Computer Science Institute Technical Report, UC Berkeley. 1998.
[12] Figueiredo M A T, Jain A K. Unsupervised learning of fnite mixture models[J]. IEEE Transactions on Pattern Analysis & Machine Intelligence, 2002, 24(3):381-396.
[13] Yang M S, Lai C Y, Lin C Y. A robust EM clustering algorithm for Gaussian mixture models[J]. Pattern Recognition, 2012, 45(11):3950-3961.
[14] Zaharia M, Chowdhury M, Franklin M J, Shenker S, Stoica I. Spark:cluster computing with working sets[C]//Usenix Conference on Hot Topics in Cloud Computing, USENIX Association, 2010:1765-1773.
[15] 侯加林. 温室番茄生长发育模拟模型的研究[D]. 北京:中国农业大学,2005.
[16] Mccallum A. Efcient clustering of high-dimensional data sets with application to reference matching[J]. Knowledge Discovery & Data Mining, 2010:169-178.
