{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2025,10,27]],"date-time":"2025-10-27T16:15:57Z","timestamp":1761581757524,"version":"build-2065373602"},"reference-count":50,"publisher":"MDPI AG","issue":"11","license":[{"start":{"date-parts":[[2018,10,25]],"date-time":"2018-10-25T00:00:00Z","timestamp":1540425600000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"DOI":"10.13039\/501100001809","name":"the National Natural Science Foundation of China","doi-asserted-by":"publisher","award":["71501096, 61502234, 61502250, 61471199, 61772274, 61872185 and 61802185"],"award-info":[{"award-number":["71501096, 61502234, 61502250, 61471199, 61772274, 61872185 and 61802185"]}],"id":[{"id":"10.13039\/501100001809","id-type":"DOI","asserted-by":"publisher"}]},{"name":"Natural Science Foundation of Jiangsu Province of China","award":["BK20150785 and BK20180470"],"award-info":[{"award-number":["BK20150785 and BK20180470"]}]},{"DOI":"10.13039\/501100012226","name":"Fundamental Research Funds for the Central Universities","doi-asserted-by":"publisher","award":["30916011325 and 30917015104"],"award-info":[{"award-number":["30916011325 and 30917015104"]}],"id":[{"id":"10.13039\/501100012226","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Sensors"],"abstract":"<jats:p>Anomaly detection aims to separate anomalous pixels from the background, and has become an important application of remotely sensed hyperspectral image processing. Anomaly detection methods based on low-rank and sparse representation (LRASR) can accurately detect anomalous pixels. However, with the significant volume increase of hyperspectral image repositories, such techniques consume a significant amount of time (mainly due to the massive amount of matrix computations involved). In this paper, we propose a novel distributed parallel algorithm (DPA) by redesigning key operators of LRASR in terms of MapReduce model to accelerate LRASR on cloud computing architectures. Independent computation operators are explored and executed in parallel on Spark. Specifically, we reconstitute the hyperspectral images in an appropriate format for efficient DPA processing, design the optimized storage strategy, and develop a pre-merge mechanism to reduce data transmission. Besides, a repartitioning policy is also proposed to improve DPA\u2019s efficiency. Our experimental results demonstrate that the newly developed DPA achieves very high speedups when accelerating LRASR, in addition to maintaining similar accuracies. Moreover, our proposed DPA is shown to be scalable with the number of computing nodes and capable of processing big hyperspectral images involving massive amounts of data.<\/jats:p>","DOI":"10.3390\/s18113627","type":"journal-article","created":{"date-parts":[[2018,10,26]],"date-time":"2018-10-26T03:16:16Z","timestamp":1540523776000},"page":"3627","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":7,"title":["A Distributed Parallel Algorithm Based on Low-Rank and Sparse Representation for Anomaly Detection in Hyperspectral Images"],"prefix":"10.3390","volume":"18","author":[{"given":"Yi","family":"Zhang","sequence":"first","affiliation":[{"name":"School of Computer Science and Engineering, Nanjing University of Science and Technology, Nanjing 210094, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-7162-0202","authenticated-orcid":false,"given":"Zebin","family":"Wu","sequence":"additional","affiliation":[{"name":"School of Computer Science and Engineering, Nanjing University of Science and Technology, Nanjing 210094, China"},{"name":"Lianyungang E-Port Information Development Co. Ltd., Lianyungang 222042, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Jin","family":"Sun","sequence":"additional","affiliation":[{"name":"School of Computer Science and Engineering, Nanjing University of Science and Technology, Nanjing 210094, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Yan","family":"Zhang","sequence":"additional","affiliation":[{"name":"Lianyungang E-Port Information Development Co. Ltd., Lianyungang 222042, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Yaoqin","family":"Zhu","sequence":"additional","affiliation":[{"name":"School of Computer Science and Engineering, Nanjing University of Science and Technology, Nanjing 210094, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Jun","family":"Liu","sequence":"additional","affiliation":[{"name":"School of Computer Science and Engineering, Nanjing University of Science and Technology, Nanjing 210094, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Qitao","family":"Zang","sequence":"additional","affiliation":[{"name":"School of Computer Science and Engineering, Nanjing University of Science and Technology, Nanjing 210094, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Antonio","family":"Plaza","sequence":"additional","affiliation":[{"name":"Hyperspectral Computing Laboratory, Department of Technology of Computers and Communications, University of Extremadura, 10003 Caceres, Spain"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"1968","published-online":{"date-parts":[[2018,10,25]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","first-page":"583","DOI":"10.1109\/JSTARS.2015.2498664","article-title":"Fusion of spectral and spatial information for classification of hyperspectral remote-sensed imagery by local graph","volume":"9","author":"Liao","year":"2016","journal-title":"IEEE J. Sel. Top. Appl. Earth Obs. Remote Sens."},{"key":"ref_2","doi-asserted-by":"crossref","first-page":"381","DOI":"10.1016\/j.patcog.2011.03.035","article-title":"A spatial-spectral kernel-based approach for the classification of remote-sensing images","volume":"45","author":"Fauvel","year":"2012","journal-title":"Pattern Recognit."},{"key":"ref_3","doi-asserted-by":"crossref","first-page":"4955","DOI":"10.1109\/TGRS.2013.2286195","article-title":"Hyperspectral remote sensing image subpixel target detection based on supervised metric learning","volume":"52","author":"Zhang","year":"2014","journal-title":"IEEE Trans. Geosci. Remote Sens."},{"key":"ref_4","doi-asserted-by":"crossref","first-page":"2224","DOI":"10.1109\/JSTARS.2013.2279693","article-title":"(Semi-) supervised probabilistic principal component analysis for hyperspectral remote sensing image classification","volume":"7","author":"Xia","year":"2014","journal-title":"IEEE J. Sel. Top. Appl. Earth Obs. Remote Sens."},{"key":"ref_5","doi-asserted-by":"crossref","first-page":"2852","DOI":"10.1109\/TGRS.2011.2176341","article-title":"Representative Multiple Kernel Learning for Classification in Hyperspectral Imagery","volume":"50","author":"Gu","year":"2012","journal-title":"IEEE Trans. Geosci. Remote Sens."},{"key":"ref_6","doi-asserted-by":"crossref","first-page":"2811","DOI":"10.1109\/TGRS.2017.2783902","article-title":"When Deep Learning Meets Metric Learning: Remote Sensing Image Scene Classification via Learning Discriminative CNNs","volume":"56","author":"Cheng","year":"2018","journal-title":"IEEE Trans. Geosci. Remote Sens."},{"key":"ref_7","doi-asserted-by":"crossref","first-page":"1701","DOI":"10.1109\/TNET.2018.2852754","article-title":"Noise Tolerant Localization for Sensor Networks","volume":"26","author":"Xiao","year":"2018","journal-title":"IEEE\/ACM Trans. Netw."},{"key":"ref_8","first-page":"1841","article-title":"Foreword to the Special Issue on Hyperspectral Image and Signal Processing","volume":"7","author":"Zare","year":"2010","journal-title":"IEEE Trans. Geosci. Remote Sens."},{"key":"ref_9","doi-asserted-by":"crossref","first-page":"1235","DOI":"10.1109\/LGRS.2013.2290531","article-title":"Structured Priors for Sparse-Representation-Based Hyperspectral Image Classification","volume":"11","author":"Sun","year":"2014","journal-title":"IEEE Geosci. Remote Sens. Lett."},{"key":"ref_10","doi-asserted-by":"crossref","first-page":"2912","DOI":"10.1109\/JSTARS.2015.2413831","article-title":"Parallel Implementation of Sparse Representation Classifiers for Hyperspectral Imagery on GPUs","volume":"8","author":"Wu","year":"2015","journal-title":"IEEE J. Sel. Top. Appl. Earth Obs. Remote Sens."},{"key":"ref_11","doi-asserted-by":"crossref","first-page":"1131","DOI":"10.1109\/JSTARS.2017.2755639","article-title":"GPU Parallel Implementation of Spatially Adaptive Hyperspectral Image Classification","volume":"11","author":"Wu","year":"2017","journal-title":"IEEE J. Sel. Top. Appl. Earth Obs. Remote Sens."},{"key":"ref_12","doi-asserted-by":"crossref","first-page":"2926","DOI":"10.1109\/JSTARS.2015.2413931","article-title":"Parallel Spatial-Spectral Hyperspectral Image Classification With Sparse Representation and Markov Random Fields on GPUs","volume":"8","author":"Wu","year":"2015","journal-title":"IEEE J. Sel. Top. Appl. Earth Obs. Remote Sens."},{"key":"ref_13","doi-asserted-by":"crossref","first-page":"341","DOI":"10.1109\/TGRS.2017.2747862","article-title":"Feature-Driven Active Learning for Hyperspectral Image Classification","volume":"56","author":"Liu","year":"2017","journal-title":"IEEE Trans. Geosci. Remote Sens."},{"key":"ref_14","doi-asserted-by":"crossref","unstructured":"Chang, C.I. (2007). Hyperspectral Data Exploitation: Theory and Applications, Wiley.","DOI":"10.1002\/0470124628"},{"key":"ref_15","doi-asserted-by":"crossref","first-page":"2665","DOI":"10.1016\/j.rse.2007.12.011","article-title":"Ash decline assessment in emerald ash borer-infested regions: A test of tree-level, hyperspectral technologies","volume":"112","author":"Pontius","year":"2008","journal-title":"Remote Sens. Environ."},{"key":"ref_16","doi-asserted-by":"crossref","first-page":"652","DOI":"10.1109\/JPROC.2012.2197589","article-title":"Advances in Spectral-Spatial Classification of Hyperspectral Images","volume":"101","author":"Fauvel","year":"2013","journal-title":"Proc. IEEE"},{"key":"ref_17","doi-asserted-by":"crossref","unstructured":"Feng, L., Zhu, S., Lin, F., Su, Z., Yuan, K., Zhao, Y., He, Y., and Zhang, C. (2018). Detection of Oil Chestnuts Infected by Blue Mold Using Near-Infrared Hyperspectral Imaging Combined with Artificial Neural Networks. Sensors, 18.","DOI":"10.3390\/s18061944"},{"key":"ref_18","doi-asserted-by":"crossref","unstructured":"Chu, B., Yu, K., Zhao, Y., and He, Y. (2018). Development of Noninvasive Classification Methods for Different Roasting Degrees of Coffee Beans Using Hyperspectral Imaging. Sensors, 18.","DOI":"10.3390\/s18041259"},{"key":"ref_19","doi-asserted-by":"crossref","first-page":"1376","DOI":"10.1109\/TGRS.2015.2479299","article-title":"A Low-Rank and Sparse Matrix Decomposition-Based Mahalanobis Distance Method for Hyperspectral Anomaly Detection","volume":"54","author":"Zhang","year":"2016","journal-title":"IEEE Trans. Geosci. Remote Sens."},{"key":"ref_20","doi-asserted-by":"crossref","first-page":"1578","DOI":"10.1109\/TGRS.2010.2081677","article-title":"Random-Selection-Based Anomaly Detector for Hyperspectral Imagery","volume":"49","author":"Du","year":"2011","journal-title":"IEEE Trans. Geosci. Remote Sens."},{"key":"ref_21","doi-asserted-by":"crossref","first-page":"1760","DOI":"10.1109\/29.60107","article-title":"Adaptive multiple-band CFAR detection of an optical pattern with unknown spectral distribution","volume":"38","author":"Reed","year":"1990","journal-title":"IEEE Trans. Acoust. Speech Signal Process."},{"key":"ref_22","first-page":"696604","article-title":"Regularization for spectral matched filter and RX anomaly detector","volume":"6966","author":"Nasrabadi","year":"2008","journal-title":"Proc. Spie Int. Soc. Opt. Eng."},{"key":"ref_23","doi-asserted-by":"crossref","first-page":"258","DOI":"10.1117\/1.3386069","article-title":"Improved estimation of local background covariance matrix for anomaly detection in hyperspectral images","volume":"49","author":"Matteoli","year":"2010","journal-title":"Opt. Eng."},{"key":"ref_24","doi-asserted-by":"crossref","first-page":"2351","DOI":"10.1109\/JSTARS.2014.2302446","article-title":"Weighted-RXD and Linear Filter-Based RXD: Improving Background Statistics Estimation for Anomaly Detection in Hyperspectral Imagery","volume":"7","author":"Guo","year":"2014","journal-title":"IEEE J. Sel. Top. Appl. Earth Obs. Remote Sens."},{"key":"ref_25","doi-asserted-by":"crossref","first-page":"384","DOI":"10.1109\/LGRS.2010.2078795","article-title":"An SVDD-Based Algorithm for Target Detection in Hyperspectral Imagery","volume":"8","author":"Sakla","year":"2011","journal-title":"IEEE Geosci. Remote Sens. Lett."},{"key":"ref_26","doi-asserted-by":"crossref","first-page":"6844","DOI":"10.1109\/TGRS.2014.2303895","article-title":"A Discriminative Metric Learning Based Anomaly Detection Method","volume":"52","author":"Du","year":"2014","journal-title":"IEEE Trans. Geosci. Remote Sens."},{"key":"ref_27","doi-asserted-by":"crossref","first-page":"676","DOI":"10.1109\/LGRS.2010.2099640","article-title":"Simultaneous Joint Sparsity Model for Target Detection in Hyperspectral Imagery","volume":"8","author":"Chen","year":"2011","journal-title":"IEEE Geosci. Remote Sens. Lett."},{"key":"ref_28","doi-asserted-by":"crossref","first-page":"217","DOI":"10.1109\/TGRS.2012.2201730","article-title":"Hyperspectral Image Classification via Kernel Sparse Representation","volume":"51","author":"Chen","year":"2012","journal-title":"IEEE Trans. Geosci. Remote Sens."},{"key":"ref_29","doi-asserted-by":"crossref","first-page":"1463","DOI":"10.1109\/TGRS.2014.2343955","article-title":"Collaborative Representation for Hyperspectral Anomaly Detection","volume":"53","author":"Li","year":"2015","journal-title":"IEEE Trans. Geosci. Remote Sens."},{"key":"ref_30","doi-asserted-by":"crossref","first-page":"1990","DOI":"10.1109\/TGRS.2015.2493201","article-title":"Anomaly Detection in Hyperspectral Images Based on Low-Rank and Sparse Representation","volume":"54","author":"Xu","year":"2016","journal-title":"IEEE Trans. Geosci. Remote Sens."},{"key":"ref_31","doi-asserted-by":"crossref","first-page":"267","DOI":"10.1007\/s12532-013-0056-5","article-title":"GPU accelerated greedy algorithms for compressed sensing","volume":"5","author":"Blanchard","year":"2013","journal-title":"Math. Program. Comput."},{"key":"ref_32","doi-asserted-by":"crossref","first-page":"2043","DOI":"10.1080\/01431161.2017.1356489","article-title":"GPU-accelerated algorithms for compressed signals recovery with application to astronomical imagery deblurring","volume":"39","author":"Fiandrotti","year":"2017","journal-title":"Int. J. Remote Sens."},{"key":"ref_33","doi-asserted-by":"crossref","first-page":"78100R","DOI":"10.1117\/12.860213","article-title":"GPU implementation of target and anomaly detection algorithms for remotely sensed hyperspectral image analysis","volume":"7810","author":"Paz","year":"2010","journal-title":"Proc. SPIE"},{"key":"ref_34","doi-asserted-by":"crossref","first-page":"681","DOI":"10.1109\/JSTARS.2016.2636362","article-title":"A set of methods to support object-based distributed analysis of large volumes of earth observation data","volume":"10","author":"Ferreira","year":"2017","journal-title":"IEEE J. Sel. Top. Appl. Earth Obs. Remote Sens."},{"key":"ref_35","doi-asserted-by":"crossref","first-page":"2270","DOI":"10.1109\/JSTARS.2016.2542193","article-title":"Parallel and dstributed dimensionality reduction of hyperspectral data on cloud computing architectures","volume":"9","author":"Wu","year":"2016","journal-title":"IEEE J. Sel. Top. Appl. Earth Obs. Remote Sens."},{"key":"ref_36","doi-asserted-by":"crossref","first-page":"409","DOI":"10.1109\/JSTARS.2016.2603120","article-title":"A new cloud computing architecture for the classification of remote sensing data","volume":"10","author":"Quirita","year":"2017","journal-title":"IEEE J. Sel. Top. Appl. Earth Obs. Remote Sens."},{"key":"ref_37","first-page":"1","article-title":"Exploiting different types of parallelism in distributed analysis of remote sensing data","volume":"PP","author":"Costa","year":"2017","journal-title":"IEEE Geosci. Remote Sens. Lett."},{"key":"ref_38","doi-asserted-by":"crossref","first-page":"74550X","DOI":"10.1117\/12.825458","article-title":"Comparative analysis of different implementations of a parallel algorithm for automatic target detection and classification of hyperspectral images","volume":"7455","author":"Paz","year":"2009","journal-title":"Proc. SPIE"},{"key":"ref_39","unstructured":"Paz, A., Molero, J.M., Garzon, E.M., Martinez, J.A., and Plaza, A. (2010, January 27\u201330). A New Parallel Implementation of the RX Algorithm for Anomaly Detection in Hyperspectral Images. Proceedings of the International Conference on Computational and Mathematical Methods in Science and Engineering, Almeria, Spain."},{"key":"ref_40","doi-asserted-by":"crossref","first-page":"915639","DOI":"10.1155\/2010\/915639","article-title":"Clusters versus GPUs for parallel target and anomaly detection in hyperspectral images","volume":"2010","author":"Paz","year":"2010","journal-title":"EURASIP J. Adv. Signal Process."},{"key":"ref_41","doi-asserted-by":"crossref","first-page":"107","DOI":"10.1145\/1327452.1327492","article-title":"MapReduce: Simplified data processing on large clusters","volume":"51","author":"Dean","year":"2008","journal-title":"Commun. ACM"},{"key":"ref_42","doi-asserted-by":"crossref","unstructured":"Yang, M., Song, W., and Mei, H. (2017). Efficient Retrieval of Massive Ocean Remote Sensing Images via a Cloud-Based Mean-Shift Algorithm. Sensors, 17.","DOI":"10.3390\/s17071693"},{"key":"ref_43","doi-asserted-by":"crossref","unstructured":"Li, Z., Su, D., Zhu, H., Li, W., Zhang, F., and Li, R. (2017). A Fast Synthetic Aperture Radar Raw Data Simulation Using Cloud Computing. Sensors, 17.","DOI":"10.3390\/s17010113"},{"key":"ref_44","doi-asserted-by":"crossref","unstructured":"Liu, Q., Cai, W., Jin, D., Shen, J., Fu, Z., Liu, X., and Linge, N. (2016). Estimation Accuracy on Execution Time of Run-Time Tasks in a Heterogeneous Distributed Environment. Sensors, 16.","DOI":"10.3390\/s16091386"},{"key":"ref_45","doi-asserted-by":"crossref","unstructured":"Hussain, S., Bang, J.H., Han, M., Ahmed, M.I., Amin, M.B., Lee, S., Nugent, C., Mcclean, S., Scotney, B., and Parr, G. (2014). Behavior life style analysis for mobile sensory data in cloud computing through MapReduce. Sensors, 14.","DOI":"10.3390\/s141122001"},{"key":"ref_46","unstructured":"Ryza, S., Laserson, U., Owen, S., and Wills, J. (2015). Advanced Analytics with Spark: Patterns for Learning from Data at Scale, O\u2019Reilly Media."},{"key":"ref_47","doi-asserted-by":"crossref","first-page":"121","DOI":"10.1016\/j.procs.2015.07.286","article-title":"Big Data Analytics in the Cloud: Spark on Hadoop vs MPI\/OpenMP on Beowulf","volume":"53","author":"Oneto","year":"2015","journal-title":"Procedia Comput. Sci."},{"key":"ref_48","unstructured":"Zaharia, M., Chowdhury, M., Franklin, M.J., Shenker, S., and Stoica, I. (2010, January 22\u201325). Spark: Cluster computing with working sets. Proceedings of the Usenix Conference on Hot Topics in Cloud Computing, Boston, MA, USA."},{"key":"ref_49","doi-asserted-by":"crossref","first-page":"1304","DOI":"10.1109\/TNNLS.2012.2199516","article-title":"Study on the impact of partition-induced dataset shift on k-fold cross-validation","volume":"23","author":"Saez","year":"2012","journal-title":"IEEE Trans. Neural Netw. Learn. Syst."},{"key":"ref_50","doi-asserted-by":"crossref","first-page":"1759","DOI":"10.1007\/s12040-014-0513-1","article-title":"GPU-based normalized cuts for road extraction using satellite imagery","volume":"123","author":"Senthilnath","year":"2014","journal-title":"J. Earth Syst. Sci."}],"container-title":["Sensors"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/1424-8220\/18\/11\/3627\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,11]],"date-time":"2025-10-11T15:26:14Z","timestamp":1760196374000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/1424-8220\/18\/11\/3627"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2018,10,25]]},"references-count":50,"journal-issue":{"issue":"11","published-online":{"date-parts":[[2018,11]]}},"alternative-id":["s18113627"],"URL":"https:\/\/doi.org\/10.3390\/s18113627","relation":{},"ISSN":["1424-8220"],"issn-type":[{"type":"electronic","value":"1424-8220"}],"subject":[],"published":{"date-parts":[[2018,10,25]]}}}