There are 2 missing blocks. The following files may be corrupted

最新推荐文章于 2023-12-27 09:20:59 发布
原创 最新推荐文章于 2023-12-27 09:20:59 发布 · 2.1k 阅读 · 4 ·
本内容遵循CC 4.0 BY-SA版权协议
版权声明：本文为博主原创文章，遵循 CC 4.0 BY-SA 版权协议，转载请附上原文出处链接和本声明。
标签
#hadoop
本文介绍了HDFS（Hadoop Distributed File System）中出现数据块丢失的问题，以及如何通过`hdfs fsck /-list-corruptfileblocks`命令查找丢失的文件块，并使用`hdfs fsck路径-delete`命令删除损坏数据块信息的解决方案。针对HDFS文件可能的损坏情况，提供了关键的故障排查和恢复策略。

项目场景：

集群断电重启块丢失,50070页面报错:There are 2 missing blocks. The following files may be corrupted

问题描述：

There are 2 missing blocks. The following files may be corrupted

解决方案：

1.哪一个文件的数据块丢失了:hdfs fsck / -list-corruptfileblocks
2.删除数据块信息: hdfs fsck 路径 -delete

There are 2 missing blocks The following files may be corrupted
一杯派蒙的博客
05-05 9520
There are 2 missing blocks. The following files may be corrupted: 步骤1，检查文件缺失情况 可以看到， /user/hive/warehouse/mid_table/part-00000-2434bd33-8a21-4249-9c0c-17e0f4ba397b-c000.snappy.parquet /user/hi...
There are 2 missing blocks. The following files may be corrupted: blk_1073755429 /tmp/logs/atguigu
dxy
03-18 621
There are 2 missing blocks. The following files may be corrupted: blk_1073755429 /tmp/logs/atguigu 步骤1，检查文件缺失情况 可以看到， [atguigu@hadoop102 ~]$ hadoop fsck /user/hive/warehouse | grep parquet WARNING: Use of this script to execute fsck is deprecated. WARNI
There are 4 missing blocks. The following files may be corrupted
weixin_46039745的博客
12-27 585
因为我的环境为测试环境，可删除缺失文件，生产环境还需要斟酌判断。确实各自都丢失了1个Block， 文件已经损坏，不可恢复。
hdfs missing blocks. The following files may be corrupted
lixgjob的博客
06-13 2028
https://www.cnblogs.com/itboys/p/9596441.html There are 2 missing blocks. The following files may be corrupted： 步骤1，检查文件缺失情况 可以看到， blk_1074785806 /var/log/yarn_hislog/yarn/apps/root/logs/app...
hdfs问题2:There are 1 missing blocks. The following files may be corrupted:
weixin_43391069的博客
07-12 1308
hdfs问题2 There are 1 missing blocks. The following files may be corrupted: ![在这里插入图片描述](https://img-blog.csdnimg.cn/2021071219495295.png?x-oss-process=image/watermark,type_ZmFuZ3poZW5naGVpdGk,shadow_10,text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3dlaXhpbl80MzM5MTA2O
There are 1 missing blocks. The following files may be corrupted:
kkk109A的博客
04-13 1376
1、hdfs fsck / -list-corruptfileblocks，查看哪些数据块损坏。2、删除数据块信息: hdfs fsck 路径 -delete。
hadoop报错：There are * missing blocks. The following files may be corrupted
Kang Hao‘s Blog
08-02 7821
错误如下： /user/hive/warehouse/student01.txt: MISSING 1 blocks of total size 57 B.. /user/hive/warehouse/student2/student02.txt: CORRUPT blockpool BP-1635359950-127.0.0.1-1538059327626 block blk_1073741...
There are 7 missing blocks. The following files may be corrupted
qq_15138049的博客
05-21 1098
断电、hdfsweb提示blocks missing [root@nufront-master-01 ~]# hdfs fsck / Connecting to namenode via http://nufront-master-02:50070/fsck?ugi=root&path=%2F FSCK started by root (auth:SIMPLE) from /172.16.34.121 for path / at Fri May 21 16:36:27 CST 2021 ...
启动Hive时报错，Name node is in safe mode.
qq_41508919的博客
04-02 1363
启动Hive时报错，Name node is in safe mode. 查看解决方式，关闭safe mode 解决。 bin/hadoop dfsadmin -safemode leave //在bin下执行 //若配置环境变量，使用以下命令 hadoop dfsadmin -safemode leave 但是关闭hive 和 hadoop之后重启还是同样的错误。 打开hdsf查看 master:50070 发现报错： There are 259 missing blocks. The follow
There are 86 missing blocks. The following files may be corrupted:
cjwfinal的博客
09-26 826
1.错误背景 集群断电重启块丢失，master:50070页面报错:There are 2 missing blocks. The following files may be corrupted： 2.报错现象 There are 86 missing blocks. The following files may be corrupted: 3.解决方案 删除缺失文件 hadoop fsck -delete 缺失文件路径 检查集群健康状态 ...
集群的一些问题
dianhe8193的博客
05-25 355
　Web页面出现一些错误： namenode 出现 missing blocks的解决方案 There are 2 missing blocks. The following files may be corrupted: blk_1078468526 /user/spark/sparkstreamingcheckpoint/checkpoint-1517487380000blk_...
There are 4 missing blocks. The following files may be corrupted:
yujinlong2002的博客
01-31 1179
缺少 4 个块。以下文件可能已损坏： **** 请检查日志或运行 fsck 以识别缺失的块。请参阅 Hadoop FAQ 了解常见原因和潜在解决方案。
hbase执行list命令报错：ERROR: org.apache.hadoop.hbase.PleaseHoldException: Master is initializing
记录日常开发过程中遇到的一些问题
03-25 1773
今天，可能误删了什么文件，导致在Hbase的hbase shell中执行list命令报如下错误： HDFS前端报错：There are 1 missing blocks The following files may be corrupted 解决办法： 1、检查文件缺失情况： hadoop fsck /user/hive/warehouse | grep parquet 2、删...
关于Hadoop数据块Miss在页面提示的问题
热门推荐
dajuezhao的专栏
04-13 1万+
这个文档转至台湾的已了论坛，地址是：http://hadoop.nchc.org.tw/phpbb/viewtopic.php?f=4&t=1938請教一個hadoop運行維護中的問題：背景：調整hadoop配置：1. hadoop的dfs.replication之前設置為1，現在改為3；2. 去掉一個節點，增加6個節點。現狀50070頁面出現提示：WARNING : There are about 900 missing blocks. Please check the log or run fsck.運
There are 17 missing blocks. The following files may be corrupted:
Knight
01-30 1045
打开hadoop网页出现 There are 17 missing blocks. The following files may be corrupted: 发现data怎么也起不来！ 解决方案 1.查看缺失文件 hdfs fsck / 2.删除缺失的文件 hadoop fsck -delete
WARNING : There are about 34 missing blocks. Please check the log or run fsck.
风行天下
09-06 7010
'jifeng01:9000' Started: Sat Sep 06 22:41:47 CST 2014 Version: 1.2.1, 1503152 Compiled: Mon Jul 22 15:23:09 PDT 2013 by mattf Browse the filesystem  jifeng01:9000 Log
hadoop hdfs missing blocks & 安全模式
数据之美的博客
11-19 5778
近期在做磁盘有关的一些测试时，最先是发现habse出现了RIT而且还是meta表的RIT，查看hadoop，出现了hdfs  missing blocks的问题，hadoop master上一上来就能看见报该问题。 There are 88 missing blocks. The following files may be corrupted: blk_1076662996 /**/d...
评论
添加红包

请填写红包祝福语或标题

个

红包个数最小为10个

元

红包金额最低5元

当前余额3.43元 前往充值 >
需支付：10.00元
成就一亿技术人!
领取后你会自动成为博主和红包主的粉丝 规则
hope_wisdom
发出的红包

打赏作者

花沐人

你的鼓励将是我创作的最大动力

¥1 ¥2 ¥4 ¥6 ¥10 ¥20
扫码支付：¥1
获取中
扫码支付

您的余额不足，请更换扫码支付或充值

打赏作者

实付元
使用余额支付
点击重新获取
扫码支付
钱包余额 0

抵扣说明：

1.余额是钱包充值的虚拟货币，按照1:1的比例进行支付金额的抵扣。
2.余额无法直接购买下载，可以购买VIP、付费专栏及课程。

余额充值