{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,8,8]],"date-time":"2026-08-08T19:50:53Z","timestamp":1786218653121,"version":"build-2736575974"},"reference-count":32,"publisher":"MDPI AG","issue":"21","license":[{"start":{"date-parts":[[2022,11,6]],"date-time":"2022-11-06T00:00:00Z","timestamp":1667692800000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"name":"Technology Project of State Grid Tianjin Electric Power Company","award":["KJ22-1-47"],"award-info":[{"award-number":["KJ22-1-47"]}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Sensors"],"abstract":"<jats:p>The rapidly growing power data in smart grids have created difficulties in security management. The processing of large-scale power data with the use of artificial intelligence methods has become a hotspot research topic. Considering the early warning detection problem of smart meters, this paper proposes an abnormal data detection network based on Deep Reinforcement Learning, which includes a main network and a target network composed of deep learning networks. This work uses the greedy policy algorithm to find the action of the maximum value of Q based on the Q-learning method to obtain the optimal calculation policy. It also uses the reward value and discount factor to optimize the target value. In particular, this study uses the fuzzy c-means method to predict the future state information value, which improves the computational accuracy of the Deep Reinforcement Learning model. The experimental results show that compared with the traditional smart meter data anomaly detection method, the proposed model improves the accuracy of meter data anomaly detection.<\/jats:p>","DOI":"10.3390\/s22218543","type":"journal-article","created":{"date-parts":[[2022,11,7]],"date-time":"2022-11-07T03:02:22Z","timestamp":1667790142000},"page":"8543","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":12,"title":["Deep Reinforcement Learning for the Detection of Abnormal Data in Smart Meters"],"prefix":"10.3390","volume":"22","author":[{"given":"Shuxian","family":"Sun","sequence":"first","affiliation":[{"name":"Marketing Service Center, State Grid Tianjin Electric Power Company, Tianjin 300120, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Chunyu","family":"Liu","sequence":"additional","affiliation":[{"name":"Marketing Service Center, State Grid Tianjin Electric Power Company, Tianjin 300120, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Yiqun","family":"Zhu","sequence":"additional","affiliation":[{"name":"Marketing Service Center, State Grid Tianjin Electric Power Company, Tianjin 300120, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Haihang","family":"He","sequence":"additional","affiliation":[{"name":"Marketing Service Center, State Grid Tianjin Electric Power Company, Tianjin 300120, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-4058-8120","authenticated-orcid":false,"given":"Shuai","family":"Xiao","sequence":"additional","affiliation":[{"name":"School of Electrical and Information Engineering, Tianjin University, Tianjin 300072, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Jiabao","family":"Wen","sequence":"additional","affiliation":[{"name":"School of Electrical and Information Engineering, Tianjin University, Tianjin 300072, China"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"1968","published-online":{"date-parts":[[2022,11,6]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","unstructured":"Suthaharan, S. (2016). Machine Learning Models and Algorithms for Big Data Classification, Springer.","DOI":"10.1007\/978-1-4899-7641-3"},{"key":"ref_2","first-page":"729","article-title":"Reinforcement learning","volume":"12","author":"Wiering","year":"2012","journal-title":"Adapt. Learn. Optim."},{"key":"ref_3","doi-asserted-by":"crossref","unstructured":"Mousavi, S.S., Schukat, M., and Howley, E. (2016, January 21\u201322). Deep reinforcement learning: An overview. Proceedings of the SAI Intelligent Systems Conference, London, UK.","DOI":"10.1007\/978-3-319-56991-8_32"},{"key":"ref_4","doi-asserted-by":"crossref","first-page":"26","DOI":"10.1109\/MSP.2017.2743240","article-title":"Deep reinforcement learning: A brief survey","volume":"34","author":"Arulkumaran","year":"2017","journal-title":"IEEE Signal Process. Mag."},{"key":"ref_5","doi-asserted-by":"crossref","first-page":"219","DOI":"10.1561\/2200000071","article-title":"An introduction to deep reinforcement learning","volume":"11","author":"Henderson","year":"2018","journal-title":"Found. Trends\u00ae Mach. Learn."},{"key":"ref_6","doi-asserted-by":"crossref","unstructured":"Chen, L. (2021). Deep Learning and Practice with MindSpore, Springer Nature.","DOI":"10.1007\/978-981-16-2233-5"},{"key":"ref_7","doi-asserted-by":"crossref","first-page":"397","DOI":"10.1109\/COMST.2016.2616442","article-title":"Survey of Security Advances in Smart Grid: A Data Driven Approach","volume":"19","author":"Tan","year":"2017","journal-title":"IEEE Commun. Surv. Tutor."},{"key":"ref_8","doi-asserted-by":"crossref","unstructured":"Li, F., Bo, L., and Peng, L. (2010, January 4\u20136). Secure Information Aggregation for Smart Grids Using Homomorphic Encryption. Proceedings of the 2010 First IEEE International Conference on Smart Grid Communications (SmartGridComm), Gaithersburg, MD, USA.","DOI":"10.1109\/SMARTGRID.2010.5622064"},{"key":"ref_9","doi-asserted-by":"crossref","first-page":"796","DOI":"10.1109\/TSG.2011.2159818","article-title":"Distributed Intrusion Detection System in a Multi-Layer Network Architecture of Smart Grids","volume":"2","author":"Zhang","year":"2012","journal-title":"IEEE Trans. Smart Grid"},{"key":"ref_10","doi-asserted-by":"crossref","first-page":"216","DOI":"10.1109\/TSG.2015.2425222","article-title":"Electricity theft detection in AMI using customers\u2019 consumption patterns","volume":"7","author":"Jokar","year":"2015","journal-title":"IEEE Trans. Smart Grid"},{"key":"ref_11","first-page":"17","article-title":"Fraud detection in electric power distribution networks using an ann-based knowledge-discovery process","volume":"4","author":"Costa","year":"2013","journal-title":"Int. J. Artif. Intell. Appl."},{"key":"ref_12","doi-asserted-by":"crossref","first-page":"267","DOI":"10.1016\/j.enpol.2012.11.034","article-title":"On the electricity shortage, price and electricity theft nexus","volume":"54","author":"Jamil","year":"2013","journal-title":"Energy Policy"},{"key":"ref_13","doi-asserted-by":"crossref","first-page":"1798","DOI":"10.1109\/TPWRS.2011.2121350","article-title":"Variability and trend-based generalized rule induction model to NTL detection in power companies","volume":"26","author":"Biscarri","year":"2011","journal-title":"IEEE Trans. Power Syst."},{"key":"ref_14","doi-asserted-by":"crossref","unstructured":"Fontugne, R., Tremblay, N., Borgnat, P., Flandrin, P., and Esaki, H. (2013, January 26\u201331). Mining anomalous electricity consumption using ensemble empirical mode decomposition. Proceedings of the 2013 IEEE International Conference on Acoustics, Speech and Signal Processing, Vancouver, BC, Canada.","DOI":"10.1109\/ICASSP.2013.6638662"},{"key":"ref_15","doi-asserted-by":"crossref","first-page":"1284","DOI":"10.1109\/TPWRD.2010.2055670","article-title":"Improving SVM-based nontechnical loss detection in power utility using the fuzzy inference system","volume":"26","author":"Nagi","year":"2011","journal-title":"IEEE Trans. Power Deliv."},{"key":"ref_16","first-page":"69","article-title":"Study on the anti-electricity stealing based on outlier algorithm and the electricity information acquisition system","volume":"43","author":"Cheng","year":"2015","journal-title":"Power Syst. Prot. Control"},{"key":"ref_17","unstructured":"Mnih, V., Kavukcuoglu, K., Silver, D., Graves, A., Antonoglou, I., Wierstra, D., and Riedmiller, M. (2013). Playing atari with deep reinforcement learning. arXiv."},{"key":"ref_18","doi-asserted-by":"crossref","first-page":"529","DOI":"10.1038\/nature14236","article-title":"Human-level control through deep reinforcement learning","volume":"518","author":"Mnih","year":"2015","journal-title":"Nature"},{"key":"ref_19","first-page":"1","article-title":"A survey on deep reinforcement learning","volume":"41","author":"Liu","year":"2018","journal-title":"Chin. J. Comput."},{"key":"ref_20","unstructured":"Silver, D., Lever, G., Heess, N., Degris, T., Wierstra, D., and Riedmiller, M. (2014, January 21\u201326). Deterministic policy gradient algorithms. Proceedings of the International Conference on Machine Learning, Beijing, China."},{"key":"ref_21","unstructured":"Lillicrap, T.P., Hunt, J.J., Pritzel, A., Heess, N., Erez, T., Tassa, Y., Silver, D., and Wierstra, D. (2015). Continuous control with deep reinforcement learning. arXiv."},{"key":"ref_22","doi-asserted-by":"crossref","first-page":"484","DOI":"10.1038\/nature16961","article-title":"Mastering the game of Go with deep neural networks and tree search","volume":"529","author":"Silver","year":"2016","journal-title":"Nature"},{"key":"ref_23","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1109\/TCIAIG.2012.2186810","article-title":"A survey of monte carlo tree search methods","volume":"4","author":"Browne","year":"2012","journal-title":"IEEE Trans. Comput. Intell. AI Games"},{"key":"ref_24","doi-asserted-by":"crossref","first-page":"279","DOI":"10.1146\/annurev-statistics-031219-041220","article-title":"Q-learning: Theory and applications","volume":"7","author":"Clifton","year":"2020","journal-title":"Ann. Rev. Stat. Appl."},{"key":"ref_25","doi-asserted-by":"crossref","first-page":"48231","DOI":"10.1109\/ACCESS.2018.2863036","article-title":"Enhanced network anomaly detection based on deep neural networks","volume":"6","author":"Naseer","year":"2018","journal-title":"IEEE Access"},{"key":"ref_26","doi-asserted-by":"crossref","first-page":"657","DOI":"10.1007\/s10207-019-00482-7","article-title":"A context-aware robust intrusion detection system: A reinforcement learning-based approach","volume":"19","author":"Sethi","year":"2020","journal-title":"Int. J. Inf. Secur."},{"key":"ref_27","doi-asserted-by":"crossref","unstructured":"Mahfouz, A.M., Venugopal, D., and Shiva, S.G. (2020). Comparative analysis of ML classifiers for network intrusion detection. Fourth International Congress on Information and Communication Technology, Springer.","DOI":"10.1007\/978-981-32-9343-4_16"},{"key":"ref_28","doi-asserted-by":"crossref","unstructured":"Jaiswal, R., Chakravorty, A., and Rong, C. (2020, January 3\u20136). Distributed fog computing architecture for real-time anomaly detection in smart meter data. Proceedings of the 2020 IEEE Sixth International Conference on Big Data Computing Service and Applications (BigDataService), Oxford, UK.","DOI":"10.1109\/BigDataService49289.2020.00009"},{"key":"ref_29","doi-asserted-by":"crossref","first-page":"106162","DOI":"10.1016\/j.ijepes.2020.106162","article-title":"Energy theft detection in an edge data center using threshold-based abnormality detector","volume":"121","author":"Zhang","year":"2020","journal-title":"Int. J. Electr. Power Energy Syst."},{"key":"ref_30","doi-asserted-by":"crossref","first-page":"354","DOI":"10.1016\/j.patcog.2017.10.013","article-title":"Recent advances in convolutional neural networks","volume":"77","author":"Gu","year":"2018","journal-title":"Pattern Recognit."},{"key":"ref_31","doi-asserted-by":"crossref","unstructured":"Graves, A. (2012). Long short-term memory. Supervised Sequence Labelling with Recurrent Neural Networks, Springer.","DOI":"10.1007\/978-3-642-24797-2"},{"key":"ref_32","unstructured":"Lea, C., Flynn, M.D., Vidal, R., Reiter, A., and Hager, G.D. (July, January 21\u2013). Temporal convolutional networks for action segmentation and detection. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Honolulu, HI, USA."}],"container-title":["Sensors"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/1424-8220\/22\/21\/8543\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,11]],"date-time":"2025-10-11T01:11:28Z","timestamp":1760145088000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/1424-8220\/22\/21\/8543"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2022,11,6]]},"references-count":32,"journal-issue":{"issue":"21","published-online":{"date-parts":[[2022,11]]}},"alternative-id":["s22218543"],"URL":"https:\/\/doi.org\/10.3390\/s22218543","relation":{},"ISSN":["1424-8220"],"issn-type":[{"value":"1424-8220","type":"electronic"}],"subject":[],"published":{"date-parts":[[2022,11,6]]}}}